Q&As of Being an Animator: Question #2

Question #2: When is it better to use mo-cap over animation?

Before we answer when it is better to use mo-cap, we must first explain what mo-cap is. Mo-cap, shortened for motion capture, is the process of utilizing technology to record the movement of people or objects. The data from the recording is transferred to a computer program to create photorealism in a virtual environment. Mo-cap is used to add natural movements from human actors into computer-generated 3D characters, while excluding the actors’ real visual appearance. The animation data is then mapped to a 3D model that will perform the actors’ captured movement(s). Mo-cap is most often applied to films and TVs for previsualization during pre-production. This helps directors filter out certain motions in a scene and prepare for visual effects (VFX). Mo-cap also helps game developers establish a large range of motions for their characters. Beyond the entertainment industry, mo-cap is also used to diagnose injuries and for rehabilitation, along with enhancing military training if coupled with virtual reality (VR).

There are four different methods to record an actor for mo-cap. The most flexible and common method is by optical passive: infrared cameras are used to track retroreflective passive marker motion capture. Optical active uses special cameras that track LED motion capture markers as they emit light. In comparison to both optical passive and optical active, video/marker-less relies solely on software instead of markers. The last method is inertial: the actor wears inertial sensors, or IMUs, that transmit data wirelessly to a computer or smart device and use cameras only as a localization tool.

Now that “what” is answered, let’s answer the main question at hand: “when?” The first advantage of mo-cap is that because the data comes from almost real-time recordings, the costs of keyframe-based animation would be reduced. However, one disadvantage is the very costs to begin mo-cap sessions: the costs of equipment and software that will be used for the capture method of mo-cap being used, and the costs needed to hire personnel. This initial cost may prevent small productions from being able to use mo-cap, as just the equipment and software alone can range depending on the method: $25k to $500k for optical-based mo-cap, or $2k to $25k for inertial-based mo-cap. However, there is still the video/markerless mo-cap method that would likely cost less than the other methods since this method does not require a specific camera, markers. Therefore, in terms of costs, mo-cap has a financial barrier to begin, but if production has enough funds to hire personnel suited for mo-cap and secure the correct equipment and software, it would reduce the costs of creating keyframe-based animations.

The second advantage is production effort. Once all the necessary prerequisites for mo-cap are ready, mo-cap does not depend on the complexity or performance length as traditional animation techniques. This allows for the animator to test different styles and deliveries, only restricted by the actor’s talent and the restricted movements an actor can physically perform. However, this is under the assumption that the mo-cap was recorded flawlessly, as if there was a problem with the recording, there are only a few systems that allow for real-time viewing of the data to determine is a retake is needed. Because of the limited availability for real-time viewing, it would be easier to reshoot the whole scene than try to change the data.

Furthermore, the camera’s field-of-view must match the mo-cap system’s specific requirement, while also taking account for potential magnetic distortion. Another prospect to consider is that if the computer model is proportionally different from the capture subject, artifact distortion may occur. Artifact distortion is when the character’s parts mistakenly intersect as the actor is being recorded, causing the character to become unintentionally deformed. Thus, the actor’s movement must be precise to avoid artifact distortions. Therefore, in terms of production effort: production must flawlessly record the actor without needing to the actor to perform actions that defy the laws of physics; production must follow the mo-cap system’s requirements and the actor’s movements must be suited for the character they are representing; once all conditions are met, production can more easily experiment on how they want to animate and deliver the scene.

The third advantage comes from the result of successful mo-cap recordings. Mo-cap produces large amounts of animation data in comparison to traditional animation, which paired along the low latency recordings, reduces the costs of animating and helps meet production deadlines. Because mo-cap is low latency, the recreation of an actor’s performance and other realistic physical can be tangibly accurate. Although mo-cap produces large amounts of realistic data, traditional animation techniques, such as manipulating the shape of the character, must be added later to refine the data for the animation.

Overall, it is best to use mo-cap if the production’s budget can buy the necessary crew, software, and equipment for the method of mo-cap being used. Mo-cap would also be best used for animations or other applications that are still grounded in the laws of physics for characters that like the subject’s physique. Once all the prerequisites are met, the results of mo-cap would yield more data that animators can work with while also reducing the costs of creating the animation.

Sources:

https://www.adobe.com/uk/creativecloud/animation/discover/motion-capture.html

https://en.wikipedia.org/wiki/Motion_capture

https://www.rokoko.com/insights/the-complete-guide-to-professional-motion-capture#:~:text=While%20reflective%20markers%20are%20often,%2425K%20and%20%24500k.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9185486/