Motion capture suits are probably one of the best inventions created for movies and video games. It brings to life the characters we can only imagine in a fantasy world as it moves and strides like the real thing. Such technology helps in making a connection to the audience and conveys the story in a more effective way.
How do mocap suits work? Motion capture suit or mocap suit records the real-life movements of an actor or an object and sends it to a computer program where it will be applied to a 3D character. The 3D character will then move exactly how the movements were captured from the suit.
Let’s learn more about the magic of the mocap suit, a little bit of its history, its mechanism, and how it works. We’ll also check out the characters in movies and TV shows that the mocap suit has brought to life.
Dress To Impress – Digitally
To start, a mocap suit is a device that an actor wears in order to help record each move they make. It is fitted to the actor which helps in properly placing the markers like dots or LEDs. The more updated versions are now using sensors rather than just markers to track the movements of the actor.
These markers are tracked by specialized cameras where the movements are being sent to a computer using motion capture software. The more cameras used for motion tracking, the more precise the outcome will be. A “skeleton” is generated in the computer which moves in real-time and the actions recorded will be used on a 3D character.
Once the movements are recorded and transmitted into the computer, VFX artists will then “clean up” the shots for any anomalies, bumps, or jitters due to occlusion, reflections, or glitches.
Aside from the mocap suit, a head-mounted gear is also essential for a full motion capture experience. This technique is called “facial capture” where it records the actor’s exact expressions and reactions by using markers or dots on their faces. Though there are some that don’t require markers anymore.
Image source: PhaseSpace MoCap Spandex Suits
Mocap suits are made to be skin-tight yet breathable and comfortable and are usually plain colored. There are suits that have traditional markers or sensors to capture the full body movements while others rely on an inertial measurement unit (IMU). IMU has sensors with accelerometers, gyroscopes, and magnetometers for tracking. Most of the mocap suits available in the market are IMU-based. Here are some of the well-known IMU-based mocap suits:
Perception Neuron 2.0
One of the affordable mocap suits in the market, Perception Neuron 2.0 is a full bodysuit with full hand and finger tracking. It has 32 IMUs they call “neurons” and is compatible with software such as Autodesk Maya, MotionBuilder, and Moveshelf.
Rokoko Smartsuit Pro
Probably the most popular mocap suit is the Rokoko because it is considered to be one of the best mocap suits out there. It has 19 IMUs, or 9DOF as they refer to, and comes with Rokoko Studio Pro software. It is also compatible with popular plugins such as Unity, Maya, Houdini, Blender, and MotionBuilder.
Xsens MVN Link And MVN Awinda
MVN Link and MVN Awinda both operate with 17 sensors. The only difference is that MVN Link is a Lycra suit with wired trackers while MVN Awinda is a strap-based suit with wireless trackers. Both suits have magnetic immunity and are best used in fight scenes and those that involve fast movements.
Head-Mounted Cameras
Head-mounted cameras have become an important part of facial capture. There are also cameras that capture the same without using any markers. Known products are Faceware and Dynamixyz. Faceware has ProHD cam, fiberglass helmet with an onboard mini lightbox while the latter offers markerless capture, has video recording, and custom software.
Motion Capture And The Science Behind It
Motion Capture is a CGI technique that records the movements of an actor or an object which will then be transferred to a 3D character on a computer screen. This type of technique is used in video games, TV, movies, and even in the medical field.
Before it was called motion capture, it started out as an invention of Max Fleischer called the rotoscope in 1915. It is a device that projects a film on a glass panel where the artist places a paper and painstakingly draws the projected image frame by frame. This technique is called rotoscoping.
To read more about rotoscoping and how it is still used today, check out my extensive article on it here.
But the first motion-tracking system was developed in the early 80’s. It is called the “Graphical Marionette.” For this technique, two cameras are set up to track and record the location of LED attached to the actor’s joints. By using the information obtained, the system computer will then generate a stick figure that moves exactly how the actor does.
It took years to develop motion capture as we know it today. Not only does motion capture replicate the movements of the actor (performance capture), facial capture has brought even more realism in the character by capturing their expressions exactly as it is portrayed.
Types of Motion Capture
Optical Motion Capture System
This type of motion capture uses two or more special cameras within the scene to capture the actor or the object’s movement from different angles. Markers are placed on to a particular location in the actor’s body. Once the movements are captured, it will then be reconstructed and applied to a 3D computer-generated model. There are two types of optical motion capture technologies – passive and active.
Passive
Optical passive uses inert objects such as small white balls covered with retroreflective markers. These markers are tracked by infrared cameras to record all activity done when worn. This type of technique is widely used as it gives a more accurate result.
Active
Optical active uses LED as its marker and each one of these markers is assigned to specific identifiers. Special cameras track the lights it emits to capture movements. Since it uses LED, this type of optical motion capture system can be used in a location outdoors even in bright light.
All of the outdoor scenes in War for the Planet of the Apes for example were shot with an active motion capture system. Weta worked hard to improve their system to work in the wet and unpleasant environment and as always, managed to pull that off as well.
Video / Markerless
For this technique, markers are pretty much non-existent. It uses algorithms from the software to track the actor’s movements. The downside though is that room for errors is quite frequent compared to techniques using markers.
Inertial
Inertial motion capture uses a mocap suit with tiny sensors referred to as inertial measurement units or IMU. This type of sensor has accelerometers, gyroscopes, and magnetometers. An accelerometer measures the force and speed, gyroscopes for angular force, and magnetometers for magnetic fields whether from the natural or artificially.
Magic Of Mocap Suits In Movies And TV
Breathing life into characters is definitely the best thing about motion capture. Because of this cutting-edge technology, we were able to see different characters we thought can only exist in a fantasy world. Here are some of the movies and TV Shows where fictional characters came to life with the help of a mocap suit.
Lord of the Rings And The Hobbit Trilogy
Image source: Warner Bros
There’s only one character that immediately pops into our minds when we talk about these movies – Gollum, the one that started it all. And one does not simply forget the actor that brought this character to life – Andy Serkis. In his character performance, Andy Serkis wore a mocap suit with markers that are adjacent to his body keypoints.
The first movie is more of just capturing the movements of the characters rather than the emotion. With lots of rotoscoping used. From the second movie onwards, facial capture was used where Andy Serkis wore a head-mounted camera with markers on his face. His mocap suit has infrared LEDs that are bright enough to be spotted even in daylight.
Also, Benedict Cumberbatch, who played the dragon in The Hobbit: The Desolation of Smaug, acted out his role in a mocap suit. To get into the character and be engaging, Benedict Cumberbatch rolled around the floor in a mocap suit with a head-mounted camera to complete the look. In the end however, all of his motion was too obscure and not really transferrable onto the Smaug puppet.
The animators at Weta Digital animated all of Smaug by hand. It was however paraded through the media as if it was a new breakthrough in Mocap technology.
Avatar
Avatar, which held on the biggest box office for almost a decade, is a movie that heavily relied on CG and motion capture. A massive sound stage was created which was surrounded by a grid of almost a hundred cameras—the specific number depending on the needs of the scene— and whose walls, ceiling, and floor were replaced with digitally-rendered structures and environments. Reflective dots were marked on actors’ suits (that they kept on throughout filming) and props as they were tracked by the array of cameras.
A computer then recorded the movement of these dots, triangulated their locations, then assembled them into “wireframe skeletons” which became the foundation of the Na’vi bodies. Even facial expressions were perfectly captured using a unique camera that hung 6 inches away from the actors’ faces.
The actors were filmed in 3D so they closely resembled their digital alien counterparts. For this purpose, the Fusion 3D Camera System was utilized to capture scenes as if they were being viewed by human eyes. This camera was equipped with a pair of lenses that sit close together and had an adjustable line of sight for better focus. The performances were inserted into the digital environment using a virtual camera.
Optical and radio detectors tracked this camera’s location which was used to render Pandora’s environments. Thanks to this camera, any kind of shot was made possible in the virtual world, even those that needed cranes or helicopters. As this is directed by James Cameron, we can expect more jaw-dropping scenes and animation in the upcoming Avatar 2.
Planet of the Apes Trilogy
Yes, Andy Serkis is back again, this time as the ape leader Caesar.
I am quite proud that I had the chance to work on the third part, ‘War for the Planet of the Apes’ myself.
I even got the chance to get onto Weta’s big mocap stage myself and perform some Ape mocap. You might see my motion somewhere in the prison camp on one of the background apes.
Turning the actors into hordes of primates involved a comprehensive filming process. First, basic facial expressions and shapes, sounds, and a few bits of dialogue were captured. The data gathered in this step was crucial in extracting the motion details needed for Weta Digital’s facial solver, and for the reconstruction of an animated mesh that was used as a reference for the creation of shapes for the character rigs.
During the shoot, actors wore mocap suits that had active markers dotted on select areas. These suits emitted LED lights that enhanced the amount, volume, and size that can be captured.
The first movie, Rise of the Planet of the Apes, had 60 Motion Analysis cameras that captured four suit-wearing actors. The second film changed to 80 Standard Deviation cameras which were ideal for shooting outdoor scenes. Wireless motion capture cameras were included in the arsenal to ensure that scenes in forests were filmed without hassle.
In the third movie, 140 cameras were now used, and actors wore capture cameras that were mounted on helmets. These cameras—which were aimed at their faces—boasted faster frame rates and higher resolutions. Accompanied by tracking markers that were painted on specific areas of the face, the expressions of the apes were very much human-like.
Alita Battle Angel
Another one i had the pleasure of getting to work on.
Rosa Salazar stars as the titular Alita in full CGI glory alongside live actors. Using a complex process, the actress’ eyes were scanned so her digital counterpart will have eyes that can perfectly portray emotion. It was so detailed that a single eye of Alita contained more definition and geometry than the entirety of Gollum. Also, a pair of HD cameras tracked every little nuance of Salazar’s face and body. This ensured that every bit of Salazar’s expressions and movements were captured.
The specialized mocap suit that she wore tracked her breathing which was crucial in making the CGI character act more natural. The suit also had embedded marker strands and detachable components which made it easier for Salazar to suit up, though this meant that the marker placements can’t be changed. The mocap suit did a great job in combining real life with CGI because this movie is full of action packed scenes that required details to make it look seamless, awesome, and realistic all at the same time.
Marvel Cinematic Universe
Mocap suit and its technology has given life to the superheroes and villains of Marvel that we’ve only read in comics. The Hulk, Thanos, Groot, Rocket, Spiderman, and Ironman are just some of the characters that donned mocap suits. But the most notable character would be Mark Ruffalo’s Smart Hulk. Wearing a mocap suit complete with facial markers, the VFX artists were able to give this green giant a more human form.
To be able to get into the character while wearing a mocap suit, Mark Ruffalo was mentored by none other than Andy Serkis. Aside from mentoring, he also had a part in two MCU movies: The Age of Ultron and Black Panther. He played the same character for both movies as Klaw.
And because Andy Serkis has been in a lot of movies involving mocap suits (Baloo in Mowgli: Legend of the Jungle, King Kong, Captain Haddock in The Adventures of Tintin), he has become a household name. He also runs and co-founded the performance capture studio – The Imaginarium. Only proves how versatile of an actor Andy Serkis is, with or without a mocap suit.
Westworld
The third season of Westworld featured robots that were created with the help of an Xsens MVN mocap suit. To emulate mechanical action that’s fitting for robots, a keyframe approach was used for animation, and the actual robots in Boston Dynamics became references. Textures were hand-painted using MARI, and other details and edges were created using Substance Designer.
The Umbrella Academy
The Netflix adaptation of the graphic novel features Pogo, an intellectual chimpanzee, who was created with the voice and facial performance of actor Adam Godley, and the movements and on-set performance of mocap suit-wearing actor Ken Hall.
Pogo is a monkey butler that is a father figure to the main characters of the series. The character required more than just the ability to move but also the ability to show emotions and have human interactions with the other. This is where the mocap suits were needed.
Weta Digital was tapped in creating the CG of Pogo (which was a great choice given their track record of incredible CG) and the results were astounding as always. The comic book writer even had complete confidence with the company when it was discussed that they contacted Weta to do the character (he still got surprised when he first saw it though).
Conclusion
There is so much more to look forward to in motion capture. And the suits are getting better as the years go by. I am fairly certain that animation will get better in the coming years and I am excited to see and experience what other upgrades a mocap suit will have in the future. And with all the improvements and continuous advancements in motion capture, we can very much expect more jaw-dropping films, with a seamless blend of CG characters acting with live actors, and fascinating creatures to love (and hate) in the near future.