Get a glimpse at the future of VFX in our interview with Asa Bailey — one of the industry’s leading authorities on virtual production and real-time visual effects!
We had the opportunity to sit down with Asa Bailey, Director of Virtual Production and Founder of the UK-based virtual production company On-Set Facilities to discuss the evolving trends of VFX, and how virtual production and real-time VFX are rapidly changing the industry.
Asa is an accomplished author and creative director, and has worked with major studios throughout his career, and even directed a short film with Jackie Chan. He is currently serving as DVP for several upcoming movies and TV shows, as well. He lives on the forefront of virtual production technology.
His company was founded for the very purpose of propelling the industry forward into this new realm of visual effects production by providing unique hardware and software solutions, as well as thorough R&D to discover and establish best practices. They also provide a tremendous level of both on and off-site support to help studios reach their goals in the world of virtual production.Asa Bailey, Director of Virtual ProductionMy questions and statements are in bold, and each of Bailey's answers are below it! Let's go!
First of all, can you give us an overview of what virtual production is?
Virtual production, in its early sense, started around 2008, 2009, obviously with James Cameron’s developments with Avatar. In that regard, from a movie perspective and from a content perspective, they were utilizing game engine and virtual engine technology to pre-visualize and to create this human input of actors' movement and the camera movement.
They were taking those human and mechanical things, but they were interpreting them into 3D data. Then that 3D data was passed along the pipeline, so it added that human element. It came down to motion capture, as well as facial capture.
It then went into an off-line pipeline. They were using virtual production to capture those human performances, but then traditional post via the pipelines. Quite frankly, a lot of the VFX, the virtual production work that's done today by the big movies, is still that kind of work.
The technology had been trickling down from the top, but in the meantime, there have been guys like us coming from the advertising background and games industry background, where we've been coding from the bottom up.Realtime Pipeline™ Demo Live Compositing | VimeoIt has now collided - only what we've been doing is fighting for real-time visual effects. We looked at the game engines and we thought to ourselves, "Well, you never wait for the explosion in Call of Duty." They've gotten better and better. It’s like, "Right, okay, can we start to utilize those game engines for Final or Final Pixel," as we call it?
Really, that's where we are today. Now we're doing a lot of work in the engine to create more VFX and scenery on the virtual set, so everything is on-set. That brings the control box to the people that are doing the performances on-set, from the director through to the DVP, through to the cast and the rest of the crew. Everybody can see it.
Right, and I think that's a natural fit for where we're at today, because there's a higher demand for content now than ever before. There are these new tools and new processes now being forged to enable this wider creation of content at a higher volume.
That's an interesting point that you make about the quantity of content that's required. Virtual production is not very economical in any smaller set. You might as well get a small camera out there and film something.
When you start to look at scale, when you start to look at the quantity of production content that's required nowadays, that's where virtual production really comes into its space, because you can centralize the production into a single location.
You can cut down all of those production costs that are involved in flying large crews to many parts of different countries. Why go to the Taj Mahal in India when you can bring the Taj Mahal to Texas?Virtual Production Stage in Madrid
What’s a favorite project you’ve worked on so far?
I've done a lot of show content. I have done a lot of commercials, music videos, and shot movies traditionally, as well.
Last year, I was asked to create ten films for a Japanese company called NTT Data. They employ 140,000 people. They're a technology company, and they operate in every market from finance, to medical, to automotive. They wanted to create a movie for each of their divisions of the business to brief their people all around the world as to what the future was within these sectors; everything from artificial intelligence, to robotics and genomics.
We made ten ten-minute long films. I think each film took about four or five days to shoot, and that's the kind of work you'll see on our website. It was one of the first projects that we have done where it was a full virtual production. It was all done in a green screen, with nothing. [Simply] actors, props, and the chair they sit on.
Wow, that's incredible. I did get a chance to see one of the NTT Data videos, and I was just mesmerized when I first started watching it, because there's an intangible quality to it. We talk about the uncanny valley and photorealism - but I think there's this sense of hyper-realism that comes into play with the look you achieved.
I couldn't discern the difference between a lot of those things when I was watching that even with a critical eye, and that's stunning.
Yeah. One of my specialties is putting real actors into virtual environments and virtual sets, and having them interact with virtual characters and virtual VFX. It could even be an explosion effect. I'm a great believer in humans, and I always wanted to be able to place real humans or at least have their face in virtual environments.
A lot of companies are looking at how they can digitize humans. I'm not taking anything away from that, because I do believe there's a great future in volumetric capture, combining that with face animation and 3D rigs - but there's something very special about humans. That's my firm focus; photographing humans and placing them into these spaces, so we don't get that uncanny valley. Director Asa Bailey uses Mo-Sys Director’s ViewfinderI think you can fall into the area of belief when you see a human. When you see the eyes and when you hear them talk, your disbelief is sustained. That's why I place real people, real actors - and shoot them in virtual space.
I love your explanation of that. That really shines a light, I think, on the value this type of production can bring. Now that we’ve talked about the history of virtual production, can you tell us about your company, On-Set Facilities?
On-Set Facilities grew from my work. I've kind of invented a new space as a Director of Virtual Production, which is like a cinematography creative role in the virtual production space, as opposed to a very technical role like the Virtual Production Supervisor. I found that I was wanting to develop tools and equipment. [We have] various Frankenstein pieces of equipment that are in the process of becoming development products for the marketplace.
Basically, the company started like that: developing. There weren’t computers. There were no real-time machines. There were game computers, but we needed more than a 1080 Ti GPU. I've got some in the range of two Titans, but it's running on a server architecture, so it's rugged and reliable for our sets.
You could buy game computers that were all clocked out. You put it on set, and then it breaks. You can't have that. We needed to start building production quality virtual production kits.
That's where the company started out, and now we are a virtual production company servicing movies and TV shows, not just here in the UK, but also in America. We're also building studios in Canada, South Africa and across the states. The company has gone from one guy, a crazy dude, to now having it all over the place.Director Asa Bailey uses Mo-Sys Director’s Viewfinder
That’s so impressive! That must be so exciting for you - all these achievements in such a relatively short amount of time. It seems like you have been at the right place at the right time as everything has exponentially grown in the last several years.
I have been a geek for a long while. I started out as a developer way back in 1996. I was a Web Designer and a Graphic Artist. I then moved into the area of human computer interaction research, and I started to get into interfacing to try and understand why people use computers, how they use computers, and then eventually I brought people into the computers. I deployed them.
I have been doing this forever, it feels, and now the technologies have reached a point where it becomes a viable product for the market.
When did you first become interested in virtual production?
I think it really started from an interactive design side of things, because we started designing in virtual reality. It must have been five or six years ago, as we started designing. I remember way back, almost even before the turn of the century, I was involved with one of the first virtual reality companies. They were a company in Leicester, in the UK.
We were their marketing company, and we started getting involved in creating virtual spaces way back to the first-ever headsets. You could see that this was going to be the future even then.
I think what’s interesting is virtual reality is almost still in the future. Virtual production goes to the future, steals all of the good stuff, and brings it back to today - and puts it to use now for 2D screens, for all of these new ways that we interact. My first interest was in virtual reality, and then it became virtual production when it blended with my filmmaking.
I could start combining the two things together, which really started to happen about three or four years ago, when it began. We started stealing things from broadcast. We started taking things from gaming. We took things from the movie industry, and we started just picking things that we could to start building the systems.
That's awesome. I love that definition that you gave of it goes to the future and brings back what we can use today. That is a really cool way to look at that.
Let's talk for just a minute about that Unreal virtual production demo video that dropped recently - the one with the Indian motorcycle. I understand your team was invited to be there for the demo?
We were one of the select few to be there at the inception of these things. When we were given that opportunity, it was another lightbulb moment. You start using, again, other technologies a little bit from the future today, so you've got the virtual world displaying from the LED screens, which is giving a real background or the sense of a real background with then light, as well, falling on the actors, generated from more LED screens.
The light, as well, is connected to the scene. People would say, "What do you think about the LED technology and shooting everything in that way?" I'm going to say that it's like another tool in the box. There are shots where you want real-time reflections on your cast and on your props.
Now, we're working with a number of technology partners to develop ways to read the video or read the scene and immediately generate those lights as if real from the scene. So the technology’s looking at the virtual scene, reading that light information, and relaying it to actual lights on the set, and then projecting that light, in my case, onto people that are still in a green or blue screen. I see the demo not as a golden bullet that is the all-solution, but as a useful part of the toolbox.
[Virtual production] has opened up a whole new era, a whole new path of research and development. I'm flying to the Netherlands next month to take some cameras and to shoot tests on different LED walls with content on them at various distances. This is very much our company's area of expertise: breaking it, fixing it, discovering it.Director Asa Bailey On Set
That's so cool! ActionVFX is excited to be the world’s largest stock VFX library, and we’re always listening to the community and industry to ensure we’re producing the very best VFX assets to meet the needs of our customers.
Is there a role ActionVFX could potentially play in enhancing the virtual production pipeline by bringing stock VFX assets to virtual production studios?
Yes, totally. With the game engines at the moment now, it's possible to bring in a plane and attach a VFX to that plane. That could be a fire, so you may have a building that's in your level, and why not, instead of going for procedural fire - which would be more like a Houdini effect - why not bring in an actual fire, a real photograph. Photography of real visual effects. There's no reason why you couldn't use those in there.
Again, I think in the realm of virtual production, we're quite prescribed in our shots. We work in shots, whereas if you're thinking about a VR experience with freedom of movement, then you've got to consider 2D planes and where they are, and where that's going to be broken by the movement of the user.
Whereas virtual production is slightly different. If I want a fire, and I want the camera to be here and the fire over there, it’s like, "Well, that's what I want, so get me the fire." I'll put it on a plane and [it will] let me burn the house down.
Right, and with things like with a fire asset, explosions, or muzzle flashes...you can think about the practical lighting on-set being able to perfectly respond with that as well!
That's the sort of thing I'm talking about. We can have the fire burning and now, if you'd like the lights - it's analyzing that fire, and the replicating those lights so it is hitting the axis. I think asset libraries have got a massive part to play. Again, it all comes down to this demand for content.
Instead of starting from scratch, we start with assets available in marketplaces. We will give them a good kick into shape and we will change them around, but the point is, where do you start? I think that's interesting. I think that's very interesting for the asset. It's complex.
We work a lot with Quixel. With Quixel, we can't quite live without our asset libraries, and you're one of those. You're an asset library when you start being into 2D and 3D and other things. That's your choice. At the moment, when I look at your blood effects and things like that - well, why wouldn't I use them? Director Asa Bailey On Set
Absolutely. I think it would be a fair statement to say, maybe based on what you're talking about, that stock asset libraries can help provide a baseline for virtual production when you're getting ready, getting things set up. It gives you a good starting point of knowing where we go from here.
Yes. It's the artists that need them. I will give my artists a brief, which could draw imagery from assets and everything else. I say to them, "Look, this is what we see, and what can you do?" That comes down into time, budget and those kinds of things.
Obviously, the larger those things get, the closer you get to the original vision of what you're trying to do. That's where we start. It starts with our vision sent to the artist, and then the artist will then hit the libraries to try to find and create these [scenes] to match the direction.
What do you envision in the next 1-2 years in the realm of virtual production?
Just more shows, more films - more people will be using it. I think at the very top end of the content industry, the entertainment industry, it has been used in pre-vis and those kinds of ways. I think what you're going to see more of is final pixels ending up in shots.
What investments does a studio need to make to enter the world of virtual production in terms of hardware, software, and specialists?
A quarter of a million dollars to even get started.
That's great, that’s practical.
Right, because you're going to need computers. You're going to need hardware from a data management side of things, too. If you want to put real humans into virtual spaces, then you're going to need to invest in chroma hardware.
It's a blend between equipment, but then the real difficulty is the human skills, because you can buy a heap of kit and if you've got no experience in the engines, or if you've not got experience within 3D modeling or in compositing or in these other areas, you're going to find yourself in a world of hurt, because it's really hard anyway.
It's becoming easier, though. Things are becoming more stable, but there's probably about a two-year learning curve for any studio that wants to just get into this space. I mean that with the greatest respect for everybody's brains and abilities and everything, but if you're a user, you can't just buy the box. That's the thing.VPD Asa Bailey’s AC Sets Up 180 Track and Panther Dolly VP Rig
Yeah, that makes a lot of sense. That leads to my next question - what advice can you give to filmmakers and VFX artists who are interested in pursuing careers in virtual production in the future?
Learn the Unreal Engine. It's the first thing you need to do. It's a big place. It's a wide open thing. You need to learn the engine, but you also need to keep focus on your specialization, as well. For me, it's about the capture. It's about the points of capture. The camera, the light, working with the actors, so I have to try and keep myself focused. I'm a director. That's where I came from. I now direct virtual productions, but I'm still a director.
I think that the important part is to bring your traditional VFX discipline and see how you can start to bring that into the engine. Don't just come into the engine and try and start learning blueprints and all these things because you've heard about them. Try and explore what you do, how you can do it in the engine, like modeling, for instance.
I mean, we've been planning this all day. We've imported .FBX, and all of a sudden the whole thing goes black when we rebuild the lighting. It's like, 'What's going on? I'm not a modeler, so I need my modeler.'
You still need all your disciplines, but the only thing is I want them now on the set. I want an orchestra of real-time VFX artists, real-time compositors, real-time modelers and real-time animators. All of these jobs learn starting in the engine.
We want the quality of offline revved up in the engine. People sometimes start from a game background and we get all these game graphics. Quite frankly, we don't want game graphics. We want to make that as an option if we want it, but we don't want to have to have it.
It's really important that we start to bring in the skills and quality of traditional VFX pipelines into the engine.Asa shoots real actors in virtual worlds using high-end cinema lenses like the 40mm 1.8 Cooke FF+ Anamorphic.”
What can On-Set Facilities bring to studios interested in virtual production?
We provide a full turnkey solution to a studio that wants to get into this space. They give us a budget as to what they're looking to invest, and then we work to assemble the products that they want to make, as well.
If they're predominantly wanting to broadcast, we have a pipeline for full broadcast. If they want to do movie production, we have a movie production pipeline. The components are very, very different in each and every one.
It really starts with the level of investment that a studio wants to make, what they see as their business opportunity going forward as to what they're going to be making, and then comes the solution. It's the only way to do it.
Absolutely. That makes total sense. Our last question - how can people learn more about you and your company?
Thanks so much for your time, Asa - this has been so insightful. Our whole team at ActionVFX wishes you and your company the best, and we look forward to what the future holds for On-Set Facilities and the world of virtual production!http://www.onsetfacilities.com
First time here? ActionVFX creates action stock footage for visual effects and filmmaking. (We also have some great free stuff!)
Want to see more blogs? Scroll up to the top and select a category or simply the "All" button for the most recent entries.
Remember to connect with us on our social networks to stay updated on our latest news, giveaways, announcements, and more!
