Capture motion data for facial and full-body motions in Blender.
From impressive real-time environment rendering to procedural terrain building tools powerful enough to build an entire forest, we certainly think there hasn’t been a better time than now to invest in learning Blender.CG Geek | www.youtube.com/channel/UCG8AxMVa6eutIGxrdnDxWpQNot only can Blender take care of environment shaping and character modeling, but you can even use an iPhone to capture motion data for facial animations for rapid VFX production. Even if you’re only using it for animatics or pre-vis work, knowing how to bring motion capture data into Blender 2.8 can bring immense value to your project.
In today’s article, we’re going to look at two different techniques to generate, capture, and animate a face inside of Blender 2.8, as well as a unique option for capturing full-body motion.
Let’s get started!
Record Standalone Video with Facial Tracking Markers
If you’re on the journey of learning Blender and you haven’t subscribed to CGMatter, do yourself a favor and subscribe now! You’ll find a lot of great resources to optimize your workflow and build some amazing stuff along the way.
This technique is more of a comprehensive approach. It covers everything from recording and tracking markers to mesh creation, projecting the tracking data onto geometry, connecting the tracking points to bones, and adding in scripting to automate the process.Some takeaways on tracking marker placement are:
• Place markers where the most deformation occurs - typically the mouth, eyes, and eyebrows.
• The more markers you use, the more detail you’ll be able to capture.
• Place markers symmetrically so you don’t have an uneven amount of capture detail.
• Keep the camera as still as possible with all markers in view throughout the duration of the capture.
iPhone/iPad Capture with FaceCap
A much more fluid process for capturing facial motion data is to use an iOS app like FaceCap. Using the iOS device’s TrueDepth sensor, you can record a performance and export as an FBX file with mesh, blendshapes, and animation data for use in a 3D application like Blender. You can also export a simple .TXT file with the animation data. Check it out in action below!Going a step further, you can even stream the performance capture straight into Blender via a wireless OSC feed from your iPhone/iPad to a Mac thanks to the AddOSC plugin. The video above demonstrates how easy the wireless motion capture can be (provided you don’t have any wireless issues), but the segment on setting up the wireless connection begins at the 3:54 mark. You can record and playback within Blender, and even change material and shaders on the fly.
Use AI for Motion Capture in Blender
Without a doubt, the easiest method for bringing full-body motion capture into Blender is by using an app called MOTiON. MOTiON uses machine learning to recognize distinctly human motion from 2D video capture and interpolate that information into 3D motion data.
Similar to bringing in the .FBX data from FaceCap, just import MOTiON’s .FBX and link it to your rigged 3D model in Blender. You can see the entire workflow below.Having all these different options for importing motion capture data in Blender can make creating your next animation or VFX sequence a lot easier. As Blender continues to add new features all the time, it’s easy to imagine the lines will soon be blurred between performance capture and user interaction.
Do you have any tips or tricks for working with motion capture data in Blender? Let us know in the comments below!
First time here? ActionVFX creates the best pre-keyed stock footage for VFX and filmmaking. (We also have some great free stuff!)
