Matchmoving is one of the terms we often encounter when reading or watching explanations about visual effects, and for good reason. Every time a computer generated element needs to be put on a live action shot, matchmoving is needed. Matchmoving is probably in most, if not all of our modern science fiction or action films we see today, and in most of the audiovisual media we consume regularly. However, it mostly goes unnoticed–when done right.
And since proper, well done matchmoving is almost invisible, this begs the question: how does it work? Matchmoving workflows could vary, depending on the scale of the project, but its basic premise is using computer programs to construct a 3D environment from the 2D live footage. This 3D environment will be the site for the different visual effects and animations needed for a particular scene.
Read on and I will tell you how a basic matchmoving workflow goes, how it fits in with the other visual effects departments, why it is important, and some examples of its applications in visual effects.
Matchmoving in Visual Effects
Matchmoving is an essential part of visual effects, even if it is not as flashy as animation jobs or modeling computer generated images. Without it, the level of realism we get today with computer generated visual effects would simply not be possible.
Matchmovers usually work alongside rotoscopers and 3D modellers earlier on in the visual effects production process, to create 3D environments that would seamlessly fit in the world of the recorded footage. Having the matchmoving done right in the earliest stages of film or visual effects production will contribute to the rest of the steps going more smoothly, saving the rest of the team a lot of time and effort if it is done well.
Almost every major action or science fiction film uses matchmoving these days. If you want to insert a nonexistent alien monster onto a real life set, you will need matchmoving. If there is a spaceship floating above a major city, you will need matchmoving. If you need to stage a plane crash onto the Statue of Liberty, you will need matchmoving.
A matchmover will need to be extremely detail oriented and thorough, since the smallest aberrations could make or break the realism of a scene, losing the audience’s trust in the realism and authenticity that a film is trying to build. When watching movies with special effects, we tend to be immersed in the world of a film with immaculate matchmoving and visual effects, and this is the reaction that filmmakers and visual effects artists are constantly trying to achieve during the creative process.
Check out these several other posts i have published that will give you more information about career opportunities as well as some more personal experiences from my career that started in Matchmoving as well:
- What is Matchmoving and what does a Matchmove Artist do?
- Creating the Otherworldly: Visual Effects Artists and What They Do
Basic Matchmoving Workflow
Making or ‘solving’ a matchmove is like solving a puzzle. You will need to replicate the 3D environment that was captured on 2D through the camera. Depending on the scale or purposes of a project, there may be some variations to the workflow. However, this is a basic workflow that will take you through the different necessary steps in order to make a good matchmove.
Evaluating Source Footage
Before doing anything to the footage, the matchmover will need to watch it first. They will have to take note of the various details that will be important when solving the matchmove.
Look at the camera. How far do you think it is from the action? Is it moving? In what direction? How fast is the movement? Where and when will the computer generated elements be added? Did they use any markers? If not, what points in the footage could you possibly use as markers?
These are just a few of the questions you could ask and answer, in order to give you a general idea of what needs to be done and how long the process is going to take, which incidentally are going to be some of the first questions a client will ask you. Asking and answering these questions could help you plan out more carefully your workflow as well.
Applying Information
When shooting for a scene that requires visual effects, a person from the visual effects team would usually be on set in order to gather the information that would help with the matchmoving process. Matchmoving needs to be exact in order to work seamlessly, and having this information available will save the matchmover a lot of time, since they won’t have to guess what is going on in the scene.
The necessary information can be categorized into three: camera information, set measurements, and survey data. Having this information on hand is ideal, but not all of this will be available all the time, thus affecting the possible difficulty of the job and how long it would take to solve.
Camera Information
This will include the camera’s focal length, the lens used, and the aperture. The matchmover will try to replicate the scene in accordance with what the camera captured, and how it captured that particular image. Different lenses, focal lengths, as well as camera settings like aperture can have a lot of subtle (or obvious) variations that affect what the footage looks like.
Set Measurements
The set will also need to be measured. This could include the camera height, measurements of various props, distances between actors and objects in the scenes, etc. This is needed so that the matchmover could accurately plot out the 3D environment they are trying to recreate.
Survey Data
For larger sets or locations, the matchmover may also need survey data. This is a detailed measurement of the set–its different points, distances, and contours. Very much like surveying land, this is done by a professional surveyor.
Create Tracking Points
In matchmoving, tracking points are points in the footage which could be referenced in order to create a 3D map of the environment. The more points, the better. The best places to put tracking points are points with high color contrast, and they also need to be visible at all times, so their position could be tracked throughout the scene.
Some software has an option to automatically create tracking points, but some of these may be inaccurate or clunky, which is why following it up manually is best for accurate results. The software then scans through the footage, following how these points move, and plotting the environment from the information it gets from these tracking points.
When shooting footage for visual effects, some put visual markers such as dots throughout the set in order to help out the matchmover. Examples of these are those dots all over blue screens and green screens that we see in behind the scenes footage of some major films.
Set Fitting
After creating the tracking points and analyzing them, the software then generates 3D markers which it assumes will fit the scene. However, the matchmover will still need to adjust it, referencing the different planes of the footage in order to fit the scene. This is the “base” of sorts, the 3D environment where the computer generated images will be put in and animated.
In doing this, the matchmover will need to have some math skills, as you will need to work with the x, y, and z axes to make sure that your 3D construction perfectly matches with the world being portrayed in the source footage. This can consist of a lot of trial and error as well. Having a lot of set information available is especially helpful in this step, since the less information you have at hand, the more guesswork you will have to do.
Testing
After setting all of that up, the matchmover is now ready to test it! Since matchmoving is usually done earlier on in the visual effects pipeline, it could be possible that there are no 3D models yet, so the matchmover could just use simple shapes, or any 3D object they have on hand to test the matchmove.
To make it easier to spot any mistakes, the matchmover could put on a checkerboard or grid overlay, to see if there are any mistakes or aberrations to the matchmove throughout the footage. A successful matchmove should be seamless and almost invisible. It should look and move realistically, or else it could break the perceived reality of the project.
If there are any mistakes, the matchmover should troubleshoot and adjust accordingly. If there are none, it is ready to be submitted and used by the other visual effects artists.
Conclusion
Matchmoving is one of those jobs that when it is done right, almost nobody notices it. However, just because it is invisible does not mean that it is less important. In fact, it is one of the most important parts of modern visual effects. Matchmovers provide the 3D foundation of the different worlds we see in movies and television. The job may not be as glamorous as other artists or animators, but they sure are important.