digital de-ageing
Buying time for Hollywood’s ageing movie stars

A Hollywood star’s run at the top of Tinseltown can be fleeting. But artificial intelligence is powering de-ageing technology and even generating lifelike digital humans that could change movie-making forever.

By Peter Griffin

My all-time favourite movie scene is the one in the diner in the middle of Heat, Michael Mann’s sprawling heist movie that hit cinema screens the week before Christmas of 1995.
 
It was the first time we’d seen Al Pacino and Robert De Niro, arguably the greatest Hollywood stars of their generation, on screen together. They were actually both in another movie, one of my all-time favourites bar none: The Godfather - Part II.
 
But De Niro only appeared in flashbacks in that cinema classic, never in the same shot as Pacino. Here they were then, 20 years later, sitting opposite each other in a dimly lit Los Angeles coffee shop, the bank robber and the cop, striking an uneasy truce in order to figure out what made the other tick.
 
Surrounded by oblivious diners, the two men take a moment to describe the dreams that haunt them. It’s cinematic gold. Two method actors already in their fifties but at the top of their game, showing what masters of character they really are.
 
Any A-list actor is lucky enough to get one, maybe two iconic scenes like that in a career, before the march of time pushes them aside in favour of fresher, more box office-worthy faces.
 
But what if Pacino and De Niro could keep on appearing together in new movies well into their 70s, 80s or even long beyond their time on Earth? Well, that’s an increasingly realistic prospect, thanks to the march of technology.

 

Youthful ambition

We saw a hint of that future last year in The Irishman, Martin Scorsese’s US$140 million Netflix release about the late American labour union leader and mafia associate, Jimmy Hoffa.
 
Scorsese’s lavish production explores the relationship between Hoffa, played by Pacino, and his friend Frank Sheeran, the mafia hitman who was Hoffa’s friend but, the movie suggests, also his assassin. De Niro’s performance as Sheeran is an understated show-stealer.
 
But the movie posed a dilemma for the septuagenarian filmmaker Scorsese. The real-life events it portrays span decades from the 1940s to the 1970s. His favoured stars were themselves in their mid-seventies when production on The Irishman began. No amount of make-up or digital touch-ups was going to transform them into the 1950s-era Hoffa and Sheeran.
 
Instead, Scorsese turned to cutting-edge de-ageing technology. Such techniques have been attempted before, notably in The Curious Case of Benjamin Button released in 2008 and starring Brad Pitt ageing in reverse from a wrinkly old man to a fresh-faced boy.
 
The results weren’t entirely convincing. More effective were the techniques used to portray Will Smith as a 23-year-old in Ang Lee’s sci-fi thriller Gemini Man, which beat The Irishman to release by a few months last year.
Will Smith stars opposite his 23-year-old self in the film "Gemini Man" | Photo credit: Jerry Bruckheimer Films / Album / Alamy Stock Photo

Joining the dots

The movie sees Henry, an ageing assassin, forced to confront a clone of his younger self. The film studio that came up with the de-ageing technology applied to Smith, is Weta Digital. I live a few kilometres from the studios in Wellington, New Zealand, where visual effects wizards toil away in secrecy on big-budget releases.
 
The Weta team used a motion capture stage to shoot Smith’s performance, with tracking dots on his face to map all of the 49-year-old actor’s expressions and movements. Then Weta employed technology that wasn’t available over 20 years ago, when the Gemini Man script was written.
 
It took clips from Smith’s early performances, like the TV show The Fresh Prince of Bel Air, creating a database of reference images. Digital replicas derived from those images were then merged with the movements and expressions captured during the Gemini Man shoot and manipulated with software to create a fully-digital Henry, who appears on screen for around 40 minutes of the movie.
 
This was a leap forward. But Gemini Man has that sci-fi sheen that comes with effects-heavy movies. Would the effect work as well in the hands of an old-school director like Scorsese who favours a naturalistic aesthetic? He immediately ruled out decamping to a motion capture stage where his actors would have to wear “golf balls on their faces” to capture the required facial expressions.

The de-ageing process

Instead, visual effects designer Pablo Helman, from US special effects studio Industrial Light and Magic (ILM), suggested a solution. The Irishman could be shot with multi-camera rigs - the main camera to capture the scene and two infrared cameras to get all the structural data on which to map de-aged versions of Pacino, De Niro and Joe Pesci, another veteran of Scorsese classics including Goodfellas and Casino.
 
It involved using bulky equipment and added considerably to the cost of the film and the length of time required in post-production. Software called Flux was created to meld the infra-red information with the imagery created of the younger characters. Artificial intelligence also was used to scan catalogues of reference images from the actors’ past movie performances to find suitable images that could be used to inform the de-ageing process. The AI sped up the painstaking work of presenting convincing de-aged actors through a movie that spans over three and a half hours.
 
Scorsese achieved his wish of working with his actors without the distraction of green screens and facial dots. The result is the most convincing example yet of actors being given the appearance of their much younger selves.
 
The techniques employed by Weta and ILM are set to fuel a new wave of filmmaking, one increasingly informed by AI algorithms and software. They not only effectively allow ageing actors to stay fresh-faced forever, but could see iconic dead actors re-animated, their digitized performances from the past used to assemble reference databases to piece together their likeness.
 
It was all foreshadowed in 2002’s Simone, which also starred Al Pacino and is written and directed by New Zealand filmmaker Andrew Niccol. Al Pacino and Rachel Roberts in "Simone" | Photo credit: AF archive / Alamy Stock Photo

Virtual movie stars

Pacino plays movie director Viktor Taransky who faces a potentially career-ending crisis when his star, Nicola Anders, played by Winona Ryder, walks off the set and refuses to come back.
 
Unable to use any of the footage featuring Anders due to contractual agreements and without the budget to re-shoot the movie with a new actress, Taransky is thrown a lifeline when he examines a computer program he inherited from an acquaintance. The sophisticated animation software, Simulation One, allows him to create a full computer-generated woman - Simone.
 
Her debut in the movie creates a stir - everyone wants to know who this young, beautiful actress is and Taransky has to concoct an elaborate ruse to explain the ongoing absence of his lead actress.
 
It’s not a particularly successful movie and the irony is that Simone is played by the actress Rachel Roberts - there was no state of the art visual effects employed there. But Niccol is the writer who gave us Gattaca and The Truman Show, two great movies that were before their time in presenting a vision of how technology will shape society.
 
Now that reality is within reach. Advances in AI will increasingly allow Simone-like actors to be generated from big data catalogues of human attributes - noses and mouths, eyes and tattooed arms. Similar AI applied to audio will pair with them voices sampled from decades of recorded dialogue.

Faking it

There is one potential big downside to these advances - the rise of the deep fake. We’ve already seen an uncannily lifelike Barack Obama conjured up using similar techniques to those being employed by the visual effects studios.
In a world awash with misinformation, we face the prospect of being bombarded, on social media anyway, with authentic-looking versions of our leaders. Will we even know who to trust?
 
Elsewhere, AI predictive analytics, drawing on demographic and social media data are already being used to determine what audiences want to see on screen. The Netflix recommendation algorithm is just one manifestation of that.
 
The movie stars of the future may well be fully-synthetic, their body shape, ethnicity and age determined by machine learning algorithms tuned to please the audience. But what movies are even greenlighted will be informed by algorithms, taking the mystery and luck out of the Hollywood process.
 
Even the scripts may be computer-generated, with OpenAI’s Generative Pre-trained Transformer 3 (GPT-3) language generating deep learning models, which have recently started producing convincing short stories and essays with just a few prompts as a starting point.
 
If it all sounds a bit manipulative and unreal, well it is. This technology is not without its risks. But if it means we get to see the likes of Pacino and De Niro conquer the ravages of time to keep on entertaining us, maybe it is worth it.
 
As the hitman Frank Sheeran says in The Irishman, “Oh boy, you don't know how fast time goes by until you get there.”

This article is part of Kulturtechniken 4.0, a web project from Goethe-Institut in Australia which looks at the interplay between artificial intelligence and traditional cultural skills.