Melting 'frozen memories,' AI helps Japanese recall war days

Tokyo — When Tokuso Hamai saw the colorized version of an old black-and-white photo of a picnic held under cherry tree blossoms sometime before World War II, forgotten memories of family members, most of whom died in the atomic bombing of Hiroshima in 1945, came pouring out.

“In colorized photos, people come to life,” said Hamai, now 86. “I often played near (the picnic site), and sometimes I would do some naughty things and get scolded by my father.”

The power of a colorized photo to reignite lost memories was eye-opening for Anju Niwata, a student who gave Hamai the colorized photo as a present three years ago.

The 75th anniversary of the end of World War II is Saturday, and Niwata, now 18, said she hopes it will bring attention to her project with a Tokyo University professor to painstakingly colorize photos using artificial intelligence and their own research to spark lost memories for the rapidly aging generation who experienced the war.

“Seeing Niwata share the colorized pictures with Hamai, and then watching him recall his old memories one after another, made it feel like the ice around his frozen memories was melting away,” said Hidenori Watanave, the professor who taught Niwata how to colorize monochrome pictures using AI.

Niwata and Watanave call their photo colorization project “Rebooting Memories,” and they published a book last month of the colorized versions of about 350 monochrome pictures taken before, during and after the war.

Watanave and Niwata use three different types of AI photo coloring software. The AI is useful in identifying the accurate colors of natural things, such as the sea, the sky and human skin, but it cannot accurately colorize human-made objects like roofs and clothes, Watanave said.

So Niwata and Watanave painstakingly finish the AI-colorized photos by hand to get more accurate colors based on the photo owners’ memories and advice from experts. They also look through historical documents and archives that show what the colors should look like.

Some photos take a few months to finish.

For Watanave, Twitter has become a powerful platform to pursue the colorization project.

When he posted a picture of the Hiroshima atomic bomb mushroom cloud that the Al software had colorized as white, a film director suggested that it should be more orange.

Watanave checked the testimonies of those who saw the mushroom cloud and also researched the components of the atomic bomb to see if it could actually make an orangish color.

After he confirmed that it could, Watanave added orange to the picture.

While the accuracy of the color is important, Niwata and Watanave said the most vital thing is that the colorized photos match the memories of the photo owners.

Time, however, is running out; the average age of the atomic bomb survivors is about 83.

There are often moments of wonder when the elderly see the revitalized photos.

When Niwata showed the colorized version of a family photograph to a war survivor who had dementia, for example, he remembered the type of flowers in the photograph; just a few weeks later, he was unable to speak.

Niwata said that publishing the colorization book during the coronavirus outbreak has made her think about the pandemic's link to the war.

“Our everyday lives have been stolen away by the coronavirus in a flash, which I think resembles what happened in the war. That’s why I feel like now is an opportunity for people to imagine (wartime life) as their own experience,” she said.

Watanave hopes that using new technology will help younger Japanese feel more of an attachment to those who lived through the war.

“People are forgetting wartime memories. We need to revitalize those old memories by using the latest method of expression and delivering it to the hearts of many people,” he said. “By the time we mark the 80th or 85th anniversary, we need to come up with a new way of expressing (wartime memories).”