Jo Plaete

Winning the screen test with a small startup team of 20 people, competing against top-tier VFX studios, validated the team’s technology and secured the trust of director Robert Zemeckis and Visual Effects Supervisor Kevin Baillie. Their partnership was instrumental in pushing the boundaries of AI in visual effects, enabling the team to achieve performances that were previously unthinkable. To meet the demands of the project, Jo was instrumental in scaling the studio from 20 to 140 people, recruiting creative and engineering talent to form high-performing teams capable of innovation and problem solving for world-class clients.

On set, the team implemented real-time AI-powered face-swapping to provide the filmmakers with instant feedback at minimal latency. This allowed Zemeckis to direct younger versions of the characters with precision and gave actors the opportunity to align their performances with live previews of their de-aged selves. The team also introduced the `Youth Mirror` system, enabling actors to rehearse in real-time with AI-generated previews, helping them adapt body language and expression to achieve seamless authenticity.

In post-production, Jo supervised the iterative refinement of AI outputs through neural performance editing and expert compositing, ensuring that every shot met the film's emotional and technical standards. He collaborated closely with the hair and makeup departments to align practical and digital elements, while custom compositing workflows ensured the flawless blending of AI-generated visuals into the film.

The groundbreaking work on HERE sets a new standard in filmmaking, demonstrating how AI technology and human artistry can unlock transformative creative possibilities.

In the "Houdini" music video, Eminem's iconic alter ego, Slim Shady, makes a dramatic return, stepping out of a portal to save Marshall Mathers' career. The video explores the duality of Slim Shady and Marshall Mathers, culminating in a rooftop standoff where the two characters merge into a unified Eminem. The team’s role in this project was to faithfully recreate Slim Shady using AI, capturing his youthful appearance with the signature white hair, which posed significant technical challenges.

To achieve the high fidelity required, the team trained AI models on data from Slim Shady's prime. Marshall Mathers played both himself and Slim Shady, with the AI synthesizing the latter's iconic look. The biggest challenge was recreating Slim Shady's hair, a task made difficult by the variability in hair data and the need for temporal stability in the final synthesis. The team’s solution involved creating a separate AI model specifically for the hair, ensuring it was consistent with the face generation.

The team pushed its neural performance toolset further by applying latent space constraints, guiding the model to synthesize the hair in the correct place with high-resolution detail. This approach allowed the team to achieve a believable and coherent recreation of Slim Shady that could stand up to close scrutiny. The process also involved extensive plate preparation and compositing, particularly in cleaning up the on-set wig to ensure the AI could accurately synthesize the hair.

The success of this project was not just in the technology but also in the collaboration between the AI synthesis and Marshall Mathers' on-set performance. His ability to portray both characters convincingly allowed the team’s AI models to capture and enhance the nuances of Slim Shady's look and behavior. The final product was a seamless blend of performance and technology, bringing Slim Shady back to life in a way that resonated strongly with fans.

Completed in just under three weeks, the "Houdini" music video was a technical achievement that showcased the potential of AI in visual effects. The video went viral, receiving millions of views and widespread acclaim for its innovative use of technology to recreate one of the most recognizable personas in hip-hop history.

During the live performance, once the system identified Josh's face, the AI model performed the neural swap in real time. Driven by Josh's facial movements and positioning on stage, the AI resynthesized Slim Shady's face with remarkable precision. Every step of the pipeline was optimized to operate within a strict millisecond budget, keeping the entire processing time under 200 milliseconds. This ensured that the AI-generated content remained perfectly synchronized with the live performance, free from perceptible delays.

Collaboration with the venue and broadcast team was essential to maintain seamless integration. The processed signal was pushed back to the video truck in full resolution, where it was incorporated into the multicamera broadcast without causing any delays with other cameras. Redundancy and fail-safes were also in place, with a secondary system running concurrently to take over instantly if needed, ensuring uninterrupted broadcast quality.

Jo was responsible for the full pipeline deployed for creating the digital ABBATARs from asset builds to hair and cloth simulations, to a streamlined facial performance pipeline and last but not least pioneering neural tools within ILM to enhance the lifelikeness of the digital characters.

Working closely with director Fede Álvarez and VFX Supervisor Eric Barba, the team injected the stand-in actor's performance into a neural network trained on archival footage of Holm. The team’s system provided the filmmakers with precise control over the final output, enabling them to tweak expressions, eye direction, and performance nuances. This innovative workflow gave Álvarez the flexibility to align each shot with his creative vision, ensuring Rook's look and performance felt authentic and emotionally resonant within the Alien universe.

Creating the Genie presented unique challenges, particularly balancing realism with fantasy. In addition to integrating performance capture with hand-animated enhancements, Jo and his teams developed the Genie's signature wispy smoke tail, a highly complex FX element. This required intricate simulations to make the smoke appear fluid and dynamic while maintaining artistic control over its shape and movement to complement the character's animated performance. The smoke tail had to feel ethereal yet responsive, seamlessly blending with the Genie's physical form to reflect his mystical, ever-changing nature.

Jo also oversaw the creation of The Cave of Wonders, ensuring the environment felt both mysterious and immersive, with careful attention to lighting and atmospheric effects. Additionally, he supervised the execution of the expansive desert sequences, which involved intricate sand simulations, integrated seamlessly with live-action photography and complex shape-shifting character effects.

The team’s work on Aladdin demonstrated the power of combining advanced performance capture technology with visual artistry, setting a new benchmark for digital characters. The Genie's creation exemplified how technical precision and creative collaboration can deliver a character that is both visually spectacular and emotionally engaging, enhancing the magic of the film's storytelling.

The sequence posed unique challenges, as it required the integration of characters from various franchises—each with distinct behaviors, movements, and visual styles. He led both the technical and creative teams to ensure the characters retained their unique identities while moving believably within the crowd system. Collaborating closely with animators and crowd specialists, the team used a hybrid approach that combined advanced simulations with hand-animated hero moments to ensure standout performances amidst the chaotic action.

The battle took place in highly complex digital environments, featuring shifting terrains, destructible elements, and large-scale combat interactions. He supervised the development and optimization of these environments, ensuring they could accommodate a vast number of dynamic characters, vehicles, and environmental elements without compromising visual fidelity or rendering performance. The process required close collaboration across departments, from animation and FX to lighting, environments, and compositing, to ensure every aspect aligned with the director's vision and enhanced the immersive, high-energy aesthetic of the film.

The team’s work on Ready Player One pushed the boundaries of CG filmmaking, setting a new benchmark for large-scale VFX sequences. The integration of ILM's crowd system with intricate hero interactions and massive digital environments demonstrated how innovative technology, artistry, and seamless collaboration can deliver a visually spectacular and narratively impactful experience.

The space battle system Jo developed allowed for real-time interactions between fleets, such as ships colliding, forming formations, and breaking through defensive lines. It featured flexible behavior frameworks, ensuring that each faction, from the Ravagers to the Nova Corps, moved with unique combat strategies. He led the integration of procedural simulations with hand-animated hero shots, creating seamless transitions between automated crowd-like behaviors and meticulously choreographed sequences, such as the Nova Corps' energy shield formations.

One of the key achievements was optimizing the system to support rapid iterations, giving the creative teams freedom to explore different choreography options while maintaining efficiency and performance. The battle included complex environmental interactions with debris fields, laser fire, and explosions, heightening the spectacle and tension of the sequence. Jo’s work ensured that every element of the battle, from swarming fighters to large cruisers, felt authentic to the comic-book aesthetic while delivering the high-impact visual storytelling Marvel is known for.

This large-scale battle system exemplified the blend of technical innovation and creative collaboration, requiring close coordination with animation, FX, lighting, and compositing departments. The team’s work on Guardians of the Galaxy set a new standard for cinematic space battles, combining technical precision with artistry to create a visually thrilling and narratively compelling finale.

This work required the development of custom tools and workflows to handle the scale and complexity of the simulations. He worked closely with the VFX supervisors and animation teams to fine-tune the behavior of the crowds, ensuring the movement of the zombies felt both chaotic and purposeful. The team had to strike the perfect balance between random, mindless swarming and coordinated group behavior to make the action feel believable. The team developed intricate logic systems to control thousands of individual agents, enabling them to interact realistically with both their environment and each other.

The pyramid-building sequence in Jerusalem was a particular highlight, requiring highly advanced simulations where the zombies not only swarmed but used each other as leverage points to build unstable towers. This scene, along with the flowing crowd dynamics throughout the film, represented a significant leap forward in the field of crowd simulation. The team’s work set new standards for large-scale digital crowds in visual effects, influencing how complex simulations would be approached in the years to come.

Working on World War Z was an incredible technical challenge that demanded both innovation and collaboration across departments. From optimizing the simulations for efficient rendering to ensuring the animation and physics interacted smoothly, every step required careful planning and creative problem-solving. This experience allowed Jo to blend technical expertise with artistic direction, ensuring the zombie crowds were not just visually spectacular but also served the film's narrative with impact and intensity.

To achieve the desired realism, the team extended MPC's proprietary crowd system, ALICE, to integrate complex ragdoll physics simulations with motion-captured and hand-animated clips. This allowed the team to maintain artistic control over individual characters within the larger crowd, blending automated movement with precise direction for key moments. The ragdoll system ensured that the Comanche warriors and horses responded dynamically to impacts, stumbles, and gunfire, creating lifelike interactions between the crowd agents and the environment.

A major technical challenge was developing interactions between the riders and their horses, ensuring the ragdoll simulations accurately captured the weight and momentum of both. The team built a system where the movement of each horse could seamlessly influence the rider's behavior, from coordinated galloping to violent tumbles caused by gunfire. This required extensive testing and refinement, balancing physical accuracy with artistic direction to meet the filmmakers' creative vision.

The result was a highly detailed and visually compelling battle sequence where hundreds of Comanche warriors and their horses stampede across the battlefield in chaotic yet controlled motion. Each crowd agent had to react believably to the carnage, with bodies toppling, horses stumbling, and riders colliding mid-charge. This project pushed the limits of MPC's crowd technology and set new standards for how physically interactive crowds could be simulated, combining ragdoll physics with precise artistic control at an unprecedented scale.