How did the socialites of Shanghai dress a century ago? What did the Great Wall look like before a century’s worth of human and natural damage? What would one of the terracotta soldiers of Xi’an look like in real life?
New York-based artist Hu Wengu, better known by the online handle of DGSpitzer, has pondered long and hard on these questions. In his search for answers, he’s using AI and machine-learning software to restore archival footage of China, offering audiences a high-definition glimpse into the country’s past.
一百年前的上海名媛穿什么颜色的衣服？没有被拆掉的老北京城墙是什么样子？兵马俑的真人长相如何？
旅居纽约的独立艺术家胡文谷（DGSpitzer）用 AI 技术，通过对百年前的视频资料进行人脸修复，让我们看到了属于中国的、跨越时代的高清过往。
Hu works full-time as a game developer, and his original impetus for learning AI was to learn how the technology can be incorporated into his game-design workflow. He quickly realized that it could be applied in nearly every area of game development, whether it be training in-game NPCs, generating vast maps, or even composing original music. “It actually has interesting applications in a wide range of fields, even social and human sciences,” Hu says. “So this year I made a series of videos introducing people to how AI can compose music, paint, generate faces, and more.”
As Hu’s understanding of AI deepened, he began exploring how the technology could carry over into other domains. This interest eventually led him to AI-enhanced video. Since then, he’s leveraged the power of AI to create a stunning collection of color-restored videos showcasing China’s past.
作为游戏开发者的 DGSpitzer，最开始主动学习人工智能的动机很现实，是为了尝试一下这些技术“能否为自己的游戏开发节省时间和成本”。但随着对 AI 技术了解的逐渐深入，他发现，人工智能不仅能在训练游戏 NPC、制作游戏贴图、音乐作曲素材上有着各种便捷的使用方式，“在更广阔的领域，甚至社会人文科普上都有着很有意思的应用前景。” DGSpitzer 说，“于是最近这一年我也开始制作了一系列视频，来介绍 AI 作曲、绘画、面部生成的各种小工具的使用方法。”
于是乎，尘封多年的黑白影像就得以有机会还原了本初的鲜艳面貌。而目前，在 DGSpitzer 的个人频道上，他正在制作人工智能相关的科普介绍视频，100 年黑白影片 AI 上色修复，也正是他的其中一个视频作品。
无法观看？前往哔哩哔哩
Three of Hu’s AI-enhanced works include footage sourced from the University of South Carolina Moving Image Research Collections, which showcase a Beijing streetside barber, a Shanghai fashion show, and the construction of the Broadway Mansion Hotel. Hu discovered their footage archives by chance, and as he dug through the video library, he became more and more fascinated. “MIRC has recordings of China from the 1930s, which is extraordinary,” he says. “In the old Beijing barber video, there was a bald, charismatic guy who left a profound impression on me. So much so that, after I watched it, I got in touch with the university and asked if I could use their black-and-white videos in my AI project. They responded quickly and were happy to oblige. They even helped look into the exact dates of when the films were shot.”
在 GDSpitzer 修复的老影像中，其中有三期是来自南卡罗莱纳大学影像研究馆藏（University of South Carolina Moving Image Research Collections，以下简称 MIRC），分别是老北京剃头、上海时装秀、百老汇大厦建造纪录。
着手制作的契机来得很偶然，GDSpitzer 是不经意在网络上浏览到相关年代有声的影像，本着好奇的探索欲，他找到了他们的影像资料库网站，“MIRC 所藏的中国有声影像最早可以追溯到1930年代，非常的珍贵，尤其老北京剃头那期视频中光头精神小伙的逗乐劲儿，颇为打动我。”GDSpitzer 说道，“所以在官网找到他们的联系邮箱后，我果断就发去了电邮。邮件里介绍了我之前做过的项目，以及希望他们可以授权我来上色这几部黑白影像，并允许我发布在社交网络上。之后不久收到了回复，那边的工作人员欣然应允，还帮忙扫描了当时的影像卡片确认影片的拍摄日期等信息。”
Hu’s video restorations happen in three stages. The first involves adding in extra frames, then the footage is colorized, and finally, the resolution is upscaled. His arsenal of AI software, which includes DAIN, DeOldify, DeepRemaster, and PaddleGAN, all tackle these tasks independently. “For example, the AI that adds missing frames, it analyzes the first frame and last frame to logically work out what should fit between them to smooth out the framerate,” he explains. “As for the colorizing AI, it almost like a human toddler. It examines large amounts of black-and-white photos, eventually learning that when a plant-shaped object appears, it should be green; when the sky and clouds appear, they should be blue and white; and when a human face appears, it should be flesh colored. The resolution upscaling AI works in a similar way, and by combining the output of these AI software, the footage is restored.”
人工智能上色主要分为三个阶段：补帧、上色、和分辨率提升。在这几个步骤里，每个 AI 都是单独分工的，DGSpitzer 动用了多个海外开源的 AI 项目来实现修复影片，比如 DAIN、DeOldify、DeepRemaster和 PaddleGAN 等。“例如补帧AI，就是 AI 通过联系前后帧画面形状和景深的逻辑关系，绘制出一个不存在的中间帧，实现提升画面流畅度的效果。” DGSpitzer 分享道，“而在上色 AI 当中，我个人觉得这个人工智能就像和人类从幼儿开始成长观察世界一样，通过慢慢用双眼收集数据，这个 AI 就是通过学习大量黑白彩色的对比图片，了解到有植物形状特征出现时，画面区域应该是绿色的；天空云朵出现时，应该是蓝天白云；有人脸特征出现时则是肉色……由此实现了黑白影片的上色。提升分辨率的人工智能也是同理。这几个人工智能各司其职，一起完成了影片的修复工作。”
无法观看？前往哔哩哔哩
Despite the amount of work and time Hu has invested in the project, he confesses that he’s not exactly a history buff. Instead, he considers himself a sci-fi nerd. “When I watched black-and-white videos in the past, they felt like history lessons,” he shrugs. “I would fast forward it to get through it quicker. They just didn’t catch my interest.”
However, to restore a piece of footage, sped-up viewing was no longer an option. Watching at regular speeds, he was able to discover a ton of details that he overlooked on the initial watch, such as a young man staring at the cameraman from behind a tree, a group of carefree kids playing in the street, or a curious worker studying the camera as he passes by.
These little moments of connection fascinate Hu. “It reminds me of the fact that even though we’re living centuries apart, we’re still under the same sky and walking the same earth,” he says. “It’s like watching a century-old vlog.”
出乎意料的是，DGSpitzer 说自己并不是历史纪录片的死忠粉，日常的观影爱好更偏向于科幻系列。“以前看黑白影片的时候，更多仅仅是把它当作一种历史资料，尤其很多影片还是快放版本，一下就看完了，也没有留下特别深刻的印象。”DGSpitzer 说。但唯独在进行人工智能上色修复时，他必须把影片的时间流速减慢到正常速度，从前不曾注意到的细节一下就浮现出来了。比如，“悄悄躲在树干后面偷望摄像机的小哥”、“天真无邪的小孩”、“充满好奇心手头上依然忙活没有闲着的大人”……而这样的细枝末节，却恰恰是最能抓住人心的时刻，DGSpitzer 直言他的代入感大幅提升了，“让我感受到，这些人原来曾和我一同生活在同一片天空下，同一片土地上，有点像百年前的 Vlog。”
Hu has also begun looking into restoring historical footage from other countries. Recently, he’s working with a German artist to restore a documentary shot in Dresden in 1936, even taking it a step further with the implementation of AR technology that will allow viewers to overlay the AI-enhanced scenes atop locations in real-life to see how the area has changed over the years.
Aside from video-related endeavors, he’s also begun using AI to colorize and upscale old portrait photography from China. Through his use of neural networks, Hu hopes for viewers to experience the simplicity of the past and immerse themselves in the lives of these people long gone—if only for a few short minutes. “It can be a moment of respite when you allow people to look into the past,” he says. “Stepping into the shoes of these people on camera and looking at history through the lens of the present can perhaps help us gain a greater understanding of culture and what it means to be human.”
History can often seem dull, appearing as unrelatable lines of text tucked inside of course books, but Hu believes it doesn’t have to be. Through his AI-enhanced videos, Hu hopes that people who may have previously dismissed how interesting history can be are able to find renewed excitement for the subject, and in turn, develop a healthy appetite for knowledge in all forms and shapes.
现在，DGSpitzer 还在进一步寻找合适的影片来修复，比如最近会和德国的一位艺术家朋友一起合作，用 AI 上色复原一部德国德累斯顿 1936 年的老纪录片，使用 AR 技术把纪录片中的景点镜头和现实中的景点坐标对接。另一方面，最近他也在进行一些古代历史人物画像的写实修复。让 AI “脑补”出古代人像接近真实的相貌，比如秦朝兵马俑、明太祖朱元璋、宋徽宗赵佶等，下一步，他还会继续进行人物画像还原相关的内容，比如四大美女、竹林七贤等等。
以强大的数据支持为依托，DGSpitzer 说，他想制作出这种“五分钟的彼岸”的意象传达，即让大家能有那么五分钟，通过这些 AI 处理过的复原影像，“可以从纷繁复杂的现代生活中脱离出来，感受一下前人的生活是怎样的，这些古人们的所思所想又是如何，已故鉴今，也许会有不一样的体会，可以说是一种人文关怀的感觉吧。”
在大多数人对历史的理解还停留在课本上白纸黑字的知识时，DGSpitzer 表示，他更希望通过这样有趣且贴近生活的创新形式，去激起网络上的诸多观众对历史文化的兴趣，从而更多地去了解影片背后的故事，从而抛砖引玉、寓教于乐。
Like our stories? Follow us on Facebook and Instagram.
Youtube: ~/DGSpitzer
Weibo: ~/大谷Spitzer
Bilibili: ~/大谷的游戏创作小屋
Contributor: Chen Yuan
