Animating a 3D Character from Ordinary Video
Motion capture used to require marker suits, a studio and special equipment. Animatic lets you transfer a person's movements from ordinary video onto your 3D character. A phone or any camera is enough.
This article explains how to shoot the footage, how the frames are prepared and how to get an animation you can use in a game or a clip.
What You Will Need
- A video of a single person performing the motion you need.
- A finished 3D character with a skeleton and weights (see rigging and skinning).
- A computer with an NVIDIA graphics card. The more powerful the card, the faster the processing.
How to Shoot Video that Transfers Well
The quality of the source video determines the quality of the animation. A few shooting rules:
- A stationary camera. A tripod or a level surface beats handheld shooting. If the camera shakes, the character's motion will jitter.
- The whole person in frame. Legs, arms and head should be visible throughout the entire motion.
- Even light. Harsh shadows and contrast against the background make it harder to determine body position accurately.
- One person. If there are several people in frame, you will have to pick the right one manually.
- A simple background. A plain wall or an even floor makes it easier to separate the person from the surroundings.
- Moderate speed. Very fast motions, such as strikes or jumps, are harder to transfer without losing quality.
How Motion Transfer Works
The process consists of several stages performed one after another:
- Splitting into frames. The video is split into individual images, which are saved to the inputs/[name]/imgs/ folder.
- Tracking points. The program finds the key body points and follows them from frame to frame. This gives the motion trajectories.
- Fitting the skeleton. The character's skeleton is adjusted to the trajectories so that its movements match the person's movements in the video.
- Checking and smoothing. The result is cleaned of small jitters that can appear due to inaccurate frames.
- Export. The finished animation is saved to FBX or GLB.
How Long It Takes
Classic frame-by-frame motion fitting gives an accurate result but runs for a long time. A long high-quality video may take anywhere from several minutes to several hours, depending on the computer's power and the clip's length. For short clips the time is noticeably less.
If you need a quick result — for example, a preview — use the fast mode. It cuts the wait significantly, but the frame-by-frame fitting accuracy may be lower.
How to Check the Animation
After processing, open the result in the viewer and play it through several times. Pay attention to:
- how well the key poses match the original motion;
- the feet: do they sink into the floor or "slide" across it;
- sharp jerks in the joints, which may indicate a tracking error.
Export to a Game Engine
The animation can be saved in the FBX format for Unity and Unreal Engine, or GLB for Godot. More about the formats — in the article "Export to FBX and GLB".
Frequently Asked Questions
Can I shoot on a phone?
Yes, the key things are a steady camera and good light. Landscape filming usually gives a better result.
Will a video of an animal work?
The program is built around the human body and its skeleton. For other creatures the result may be inaccurate.