Written by The Location Guide on Apr 24, 2024. Posted in General Interest

Is AI Double Trouble For Filmmakers?

The use of AI to double, de-age or even replace actors is a huge talking point. But is there potential as well as problems for Hollywood with this new tech and how can image rights issues be resolved? We speak to experts and insiders to find out.

AI was first used in film nearly 100 years ago in the dystopian drama Metropolis but it has come to prominence in recent years with the de-aging of actors. Robert De Niro in Martin Scorsese’s The Irishman, Harrison Ford in Indiana Jones and the Dial of Destiny and Tom Hanks in Robert Zemeckis’ Here, are some examples.

A GAME CHANGER

“AI has definitely changed the game,” says Kevin Baillie, production VFX supervisor on Here. “With a standard CGI of a face, you’re starting from quite a sparse data set where you have your 3D scans with say 100 poses, but that’s a tiny fraction of the maximum activations of each muscle. While the motion capture approach can be quite onerous with multiple cameras and actors wearing helmets and covered in thousands of dots, which are still missing a lot of information. And then there’s blood flow to factor in on top of that.

“But with AI it can be trained to scan thousands of images of the actor and it builds an understanding of the permutations of what the face could go through and builds a map. It’s like a digital genetic code. There’s no need for lots of cameras and the crew teams can be smaller, so it’s cost effective.”

If you think that’s a groundbreaking proposal, I suggest you think again.

Done well, the results can be very impressive. The ultimate goal being that “nobody will think about the technology when they’re watching the film because it works,” adds Baillie.

“Broadly speaking, they’re just useful tools for creating a different performance or look, and not really a threat to actors,” insists Alex Connock, senior fellow in management practice at Said Business School, University of Oxford.

Where the potential threat and concern comes in is with the use of AI to take an actor’s image and voice to recreate an artificial likeness that can be used in other films and TV projects.

When Hollywood actors confirmed through the Union SAG-AFTRA that they were going on strike, their rights with the use of AI was a chief sticking point. The Alliance of Motion Picture and Television Producers (AMPTP) responded in a statement that it “presented a groundbreaking AI proposal that protects actors’ digital likeness for SAG-AFTRA members.”

But Duncan Crabtree-Ireland, the chief negotiator for SAG-AFTRA, responded to the statement by saying: “They proposed that our background performers should be able to be scanned, get one day’s pay, and their companies should own that scan, their image, their likeness and should be able to use it for the rest of eternity on any project they want, with no consent and no compensation. If you think that’s a groundbreaking proposal, I suggest you think again.”

Secure for the actors’ original authorial rights on content propositions and ideally watermark imagery of them.

The AMPTP denied this, but the potential science fiction-turned reality scenario does raise interesting moral and legal questions.

Could film studios and production companies potentially keep using an actor’s image repeatedly because they signed a contract once on a project? Connock thinks this unlikely. “If you’re a Disney and you’re making a USD200 million Marvel movie, you wouldn’t just rely on someone’s word that they used an actor for a commercial two years before and got them to sign a buyout for their full image rights, especially without checking with the actor first. No studio is going to compromise their IP at that level.”

But he does stress that actors’ unions (and agents) “need to be careful to stop actors being signed up for contracts that take ownership of the synthetic rendition of their face. Plus, secure for the actors’ original authorial rights on content propositions and ideally watermark imagery of them.”

SUPPORTING RIGHTS

There are organisations, including those working in AI, who are actually supporting actors and other creatives in how to protect their rights and trademark their likeness. Metaphysic is one, and the UK’s Equity is another. The latter launched a toolkit in June to help protect performers from what it describes as a “surge in unregulated technology”.

The toolkit, produced in partnership with intellectual property expert Mathilde, was set up after Equity received a considerable increase in members contacting the union for advice, support and legal representation in the wake of the growing use of AI, and the government introducing a voluntary code of practice on copyright and AI, as opposed to firm regulation.

It’s also important to note that actors bring something special to the screen. It’s not just about their pretty face, it’s the underlying performance.

It includes resources like how AI can be applied ethically by the industry; a template AI contract to protect artists engaging with performance cloning work and model AI clauses to protect artists from having their performances cloned without their consent.

ACTORS V ROBOTS

“It’s also important to note that actors bring something special to the screen. It’s not just about their pretty face, it’s the underlying performance,” stresses Baillie. “I have seen the really talented ones work hard with directors, cinematographers, costume designers etc to make sure their character is represented right on screen. You can’t just replace that with Joe Blow standing in front of the camera pretending to be Christian Bale, and then put Bale’s face over the top. The audience would be like ‘what the hell am I watching?’.”

I don’t think synthetic actors are going to meaningfully replace real actors any time soon.

Equally, just having robots performing wouldn’t necessarily work either. “As Microsoft’s CTO Kevin Scott recently pointed out, a Queen’s Gambit series could’ve been made about two machines playing each other and nobody would’ve watched it. What makes the series interesting is it’s a female chess player and it’s her human journey and connection,” says Connock. “I don’t think synthetic actors are going to meaningfully replace real actors any time soon.”

However, it’s also worth pointing out that not all actors are fully opposed to the idea of AI creating doubles of them. The prospect of their careers being prolonged longer (even after they die) and their family/estate benefiting financially is not necessarily a bad thing.

Last year Bruce Willis became the first Hollywood star to sell his image rights to technology company, Deepcake, which specialises in creating movies through AI. This was after he retired from acting having been diagnosed with aphasia.

What is a bona fide possibility right now is, if I wanted to, I could get together and pitch a series of seven movies that would star me in them in which I would be 32 years old from now until Kingdom come.

Even Tom Hanks said recently that the technology may well be used to recreate his image, ensuring he continued to appear in movies long after he dies. He told the Adam Buxton podcast that “there are discussions going on in all of the guilds, all of the agencies, and all of the legal firms in order to come up with the legal ramifications of my face and my voice and everybody else's being our intellectual property," he said.

"What is a bona fide possibility right now is, if I wanted to, I could get together and pitch a series of seven movies that would star me in them in which I would be 32 years old from now until Kingdom come.”

Some actors, though, are concerned about the creation of entirely AI-generated actors or metahumans stealing their roles. There’s even talk of studios potentially creating synthetic performers from an amalgamation of actors’ images.

“Actors (and everybody) have their images ingested all the time on places like facebook, Instagram or Flickr. There are billions of image datasets available, from which new synthetic faces could be created and used,” explains Connock. “Background artists may no longer be required in the future.”

CONTROLLING AI

There’s also the problem of controlling AI. “It can be a little unwieldy, especially when it produces results that you don’t like, such as interpreting an actor’s expression slightly wrong,” admits Baillie. “How do you trick it or guide it into doing the right thing? Does the training data need to be changed or removed. What if you don’t have the footage that shows the expression you need? Do you need to create it? Then there’s the potential issue of misappropriating data. That is why you really need the right team of people, like at Metaphysic where they have guys with PhDs and lab coats, alongside talented artists from places like ILM who understand the creative side. They can train the AI and coerce the output in the right way.”

However, he admits there are a limited number of people who can perform these tasks. That will change as capability expands, but it’s still a point of concern. As is the lack of footage of actors when they were younger limiting the capabilities of de-aging using AI, and the hardware limitations on things like resolution, insists Baillie.

Ultimately, though, AI is here to stay. More filmmakers are experimenting with it. There are even AI tools, like Wonder Studio [set up by Nikola Todorovic and actor Tye Sheridan], which allow live actors to be replaced by CGI characters at the simple touch of a button. The key thing is grasping both the potential and limits of the tech.