We ask colleagues in VFX, Art Direction and Illustrations to share their thoughts on using AI in their work.
OLIVER ZELLER - Art Director and VFX Supervisor
Oliver Zeller is the co-chair of United Scenic Artists' Tech and Innovation Committee, and a member of the Producers Guild of America. He has used his multidisciplinary background in set design and visual effects as a VFX supervisor and art director on projects such as "Birds of Paradise", "The Drop", "The Midnight Swim"," Buster's Mal Heart" and the upcoming series "Penelope". Other art department credits include "Prince of Persia", "The Woman in the Window", "After Yang" and "Samarkand". In-house designer for Transforma Theatre, Oliver has also designed events, commercials and motion graphics for clients including Mercedes, Lincoln and Nike.
What is your background and how did you get interested in digital design?
I developed an interest in 3D early, leaning toward architecture and narrative space. After graduation and following a stint as a writer, I worked at studio KDLAB on architectural visualization and digital sets. Despite the digital acumen, I had a preference for built sets, and my desire to see some of these digital concepts realized in the physical realm in part pushed me toward the art department.
Have you been creating images in AI systems, or used AI in other ways in your design process?
I gained a familiarity with neural networks and deep AI in high school in the nineties, and later did design and visualization work for a facial recognition AI tech startup called IMRSV. Several years on, I joined Blackbox AI as partner and Creative Director, and that's when I really started to consider AI’s role in the design process, and began image upscaling AI for moodboards, design and director treatments.
I'm still at the stage of assessing, testing and understanding the nuances… I haven’t used the latest generative AI in production yet, but did leverage an early precursor, Google’s DeepDream for a psychedelic sequence in Amazon Studios' film "Birds of Paradise".
What are some potential benefits of using AI in production design?
A lot of what we do is incredibly mundane and trivial, so why waste our time, our lives, on that? I want to be focused on the creative side of the design.
For example, if you're doing graphic design or motion graphics in the art department, of if you're a VFX artist, there's lots of rotoscoping involved. On the hero shot I did on "The Drop" I used AI rotoscoping, and got two days of work in an hour.
After working on "The Woman in the Window," I created a searchable archive of my assets in the cloud, allowing me to more rapidly populate and develop 3D concept art at a speed closer to rivaling paintovers. Creating the archive, adding keywords to images, was very laborious and I’ve been wanting to apply AI to automatically tag incoming content. With tech like LG’s Exaone that’s coming to fruition.
One should recognize where AI excels in alleviating tedious and involved tasks. A good example for where a designer could use AI assistance are Art Nouveau elements, because the principles are derived from nature. AI actually ends up handling it very well, and I would go to AI immediately.
AI can help better virtual scouting; automate drafting for existing locations or 3D modeled builds; make efficient translation of photos (without laborious photogrammetry), art or text descriptions into 3D models; perform variant generation, particularly for graphic and motion design work, as well as intricate ornament.
As designers, we're actually very, very particular. Anybody that's been in an art department knows how much conversation goes into one kind of specific detail, that the audience might just see for a few seconds. So when it comes to communicating an idea to AI, it can be difficult, and require a lot of cleanup. The experience (50,000+ prompts, 800-1000 hours) needed to even use generative AI effectively can’t be overlooked.
I prefer to look at AI as a team member. And not one that takes the slot of somebody else, especially not at this point. But rather somebody that helps. An entity that helps us alleviate some of the more mundane or just gives us a another perspective.
What about current legal issues with copyright and derivative art?
Designers have a long history of borrowing. Look how often Ken Adam’s war room design from "Dr. Strangelove" has been imitated. How many interrogation room sets are littered with Emeco’s Navy Chair? I’ve even seen distinct designs like Aalvar Alto’s Finnish Pavilion transformed into something uniquely magical. Ridley Scott was advised against using The Bradbury Building in "Blade Runner" because it was so familiar, but he went ahead - it was dressed and shot in a way like nobody had seen before and became something uniquely distinct and evocative.
A lot of the concept designs coming out of Hollywood are feeling very similar these days, because the artists are all sort of working from the same playbook. Since they needed to produce imagery so fast, they create tool sets and processes to accelerate the work, but that also lends itself to that sameness. A lot of artists come up with a specific style when they're young, and then continue creating more work in that style for decades. I've always felt that that's actually losing a bit of the artistry. An artist needs to always explore and find new avenues. I wonder if AI will push artists not to not become complacent? It could be a very good motivator.
What effects do you think AI will have on the future of art and entertainment in general?
Many art departments are doing complex sophisticated design work on a scale that just a few decades ago would not have been feasible. Compare 1990s Star Trek production design to today’s Star Trek, for example. Thanks to more efficient tools, workflows, the internet and smartphones we’re able to consume information and create at a far greater pace than just two decades ago. Technology has allowed us to do this and AI is the next step.
Some are dismissive of AI, but design is in part based on rules and systems, which AI can learn, from color theory to spatial balance and standards. It’s here, and it’s here to stay. Not adopting AI in some capacity will put one at a competitive disadvantage.
However, people are the drivers and catalysts behind creating visual stories, and we need to assert ourselves in that regard. AI should be viewed as an aid, a supplemental team member where suited, and one driven by people. Even with generative AI, the experience of a talented designer behind it is usually evident and really consistently shines when built upon.
LUKE DiTOMMASO - VFX Supervisor
Luke DiTommaso is a Brooklyn native and a graduate of the Computer Arts program at the School of Visual Arts in NYC. He co-founded visual effects company Molecule VFX in 2005, whose credits include "Billions", "Gossip Girl", "Da Five Bloods", "The Americans" and "The Plot Against America". After an acquisition by Crafty Apes in 2022, Luke is now the senior VFX supervisor at the NYC office. He has taught VFX and animation at SVA, and is a member of the Directors Guild of America.
What is your background and how did you get interested in digital design?
As a kid I wanted to be a cartoonist and animator. As I got older I discovered films and by the time I got to college, computer art made sense because I felt that if I knew computers, I wouldn't be a starving artist. So, an interest in animation, film, and technology led me to visual effects and I’ve been doing that since I graduated from SVA NYC in 2001.
Have you used AI in your design process?
We have used AI imagery in pre-production to produce quick concepts. For example, there was a project where we were working on a fabric that was supposed to be laying on top of an actor, but didn't feel right on the shoot. We wanted it to feel more attenuated to the skin. We pulled references of hellenic statues for the drapery and the look of wet robes, and then basically fed our references and conversation surrounding the shot into DALL-E to produce some images. Was it perfect? Was it exactly what we needed? No, but it got us to a solid point quickly. In terms of creating concept art really quickly, and giving us something to point to, and was really useful.
There’s a part of the company that is very focused on AI and developing techniques to use it. There are applications where we use AI to automate a significant part of the VFX process where it may previously have taken a little bit more brute force labor in the past. However, I think of the tasks that AI completes for us like a wave or a current - you have to surf it and manage it. It doesn’t solve all your problems without finessing.
What are some potential benefits of using AI in production design?
With regards to production design from a practical standpoint, I think it could be really helpful. AI can aid the designer in quickly producing images during the early collaborative creative decision-making process. With apps like Midjourney or DALL-E, designers can show directors a space in a certain style, with certain lighting, which can serve as a good jumping-off point. In the beginning, when everything is potential and there are a lot of creative exchanges between collaborators, AI images can be used to visually communicate different directions and move pre-production conversations along quicker.
How many times has a design been limited by time and budget? Probably every single time. AI allows us to do more working within the constraints we already have, by lowering the thresholds to solve what used to be impossible problems. Part of the reason I got into VFX was to solve these sorts of problems and help the production designer realize their vision. AI could greatly help with communication between the production designer, art department and visual effects team and contribute to creating a unified vision across a large department.
What are some of the dangers of using AI in production design?
Rather than dangers, I try to think about some significant and appropriate questions. For example, how fast do we roll out AI technologies without really having an understanding of them? There’s a potential for AI to accelerate beyond what we might be comfortable with, so I do have some trepidation and am maybe not gung-ho about all aspects of it. Generally speaking though, I am embracing it. You see it in its infancy now and it doesn’t take much to imagine its future potential.
VFX has always spearheaded a lot of new technology and AI is another aspect of that. There's definitely things to be cautious of, but generally I think my message is not to fear it, but to learn how to harness it. People have jobs now that didn’t exist before, so it’s better to prepare ourselves to be constant learners and critical thinkers than to pigeon hole our processes and roles. I think those who are able to work with AI and embrace the technology as it grows will get further faster. The ability to learn, adapt and be flexible will be important because change is the only constant.
AI can help to evolve the design process but can’t replace the design work itself. There are few parts of the filmmaking process that are repetitive types of assembly line work, but you still need an artist behind the wheel. For example, motion capture has evolved the way an animator works, but has not become a substitute for their work. Animators are taking motion capture data and using it as a foundation to build and expand upon to reach their end goal more quickly. There's also still plenty of traditional animation like stop motion and 2D animation. There’s a space and need for everything.
I think AI is going to become a bigger and bigger part of our lives whether we like it or not. Our fundamental philosophy is that it’s scary and it’s new and there are a lot of unknowns, but it’s also the future. There’s AI that can learn chemistry or write a college paper in seconds. As professionals and people moving into the future, the question becomes, how do we harness this technology and make use of it? We can influence the use of AI within the industry with how we use it responsibly in a way that makes sense.
What effects do you think AI will have on the future of art and entertainment in general?
AI allows us as storytellers to be less inhibited by physical limitations. I think an individual artist will be able to do much more than they can now. There's an interesting opportunity for a new class of designers: whereas in the past a career path was more linear - from assistant to art director to designer - the new generation may not follow in the same footsteps as their predecessors, because they'll have these new tools.
KRISTY KAY - Visual Development Artist and Illustrator
Kristy Kay is a Los Angeles based Visual Development Artist and Illustrator with a focus on color and light, currently working at Nickelodeon Animation. She has previously worked with studios including Netflix Animation, Paramount Animation, Warner Bros., Cartoon Network, Sony Pictures Animation, Sony Playstation VR, Laika/ House. Illustration clients include Adidas Outdoor, Touchstone Climbing, Women’s Weekly Magazine UK, and LaLopez Studio in Spain. Film and Series credits include the upcoming "Thelma the Unicorn" and "Under the Boardwalk", "Back to the Outback", "The Spongebob Movie: Sponge on the Run", "Hotel Transylvania 2" and more.
As an artist, how do you feel about working with AI technology?
As a visual development artist for animation, I was nervous when I first saw some of the artwork created by AI technology based on a given prompt. At first I felt a sense of impending doom about how our society defines "art", and that the touch of a human hand will lose value, making many creative jobs obsolete... including my own. Once I started playing around with it, I felt slightly more empowered, and I could see how I might incorporate it into my own work someday - mostly for reference sourcing in a similar way to google images.
What are some potential benefits and dangers of using AI in production design?
I think incorporating AI technology into some of the more technical aspects of a concept artist's work could be useful and save time. However, I think there is danger in relying on AI for idea generation. The entire model of AI is based on what already exists, and my fear is that everything will start to feel derivative and watered down.
What effects do you think AI will have on the future of art and entertainment in general?
Despite my early fears of the human touch losing its value... I think that art created by humans will become even more valuable, and precious. We all want to see and hear real human stories, so that we feel seen and connected. The best art comes from a place of honesty, and imperfection. I think the novelty of AI generated stories and pictures will lose its shine.