What Insomniac's Spider-Man Looks Like Without Special Effects
Since the 2018 release of "Marvel's Spider-Man," Playstation fans have been blown away by the web-slinging action and high quality production value of this friendly neighborhood franchise. From the bustling playground of New York to the dramatic voice acting and visceral visual effects, a ton of work went in to creating an experience that has been revered by gamers and Spidey fans alike.
Insomniac has already graced fans with two hit titles in the form of "Marvel's Spider-Man" and spin-off "Spider-Man: Miles Morales." With "Marvel's Spider Man 2" due for release on October 20, there's no better time to take a look under the hood and see what exactly goes into producing a game of this scope. Thankfully, Insomniac Games has been very forthcoming in sharing its production process for these games over the years, and it's very easy to understand why. The games represent a fine body of work from a dedicated team that understands and appreciates these characters. While it's very clear each and every person involved worked really hard, it's also obvious they had a ton of fun producing each part of it. With that said, join us as we answer the question: "What does Insomniac's 'Spider-Man' series look like without the special effects?"
Pre-viz looks weird, but it's where things come together
Visualization refers to the final product of a game with all special effects added in. This is the result of an accumulative effort between various studio departments, involving weeks and months of work, editing, and further tweaks before it's a product worthy of a "chef's kiss."
Insomniac has shared pre-visualization footage many times through development talks and presentations over the years. As it turns out, these games still look pretty darn cool without the effects. This pre-viz stage is also called a "block out," and it serves as a reference point. Models are basic shapes and silhouettes. They're also barely animated, offering rudimentary directional movement only. Without further materials and lighting, the scene is rendered as a smooth grey limbo completely void of atmosphere — but hey, you have to start somewhere.
It's fascinating to see, because it proves that it takes the unified work of many departments to not only realize the final vision, but also to seamlessly transform that monotone liminal framework into a cinematic and engaging player experience.
Constructing New York
New York: If you can make it there, you'll make it anywhere! When Insomniac took on development of "Marvel's Spider-Man," the team was faced the daunting task of creating a New York that not only represented the density and character of this urban environment, but was also fun play in. While the game's island of Manhattan and surrounding boroughs are not a 1:1 facsimile New York, Insomniac aimed to capture its essence. The team had to translate the vibe of this bustling metropolis while also keeping the game's memory budget in mine. Insomniac also had to tailor the architecture to web-slinging shenanigans.
So that this process would not take years, Insomniac sought to utilize procedural generation. This allowed the team to focus its limited time on fine-tuning the art and atmosphere elements of the city. To pull this off, Insomniac used a third-party program called Houdini to generate sets of versatile assets for all roads, alleyways, terrain, buildings, cars, pedestrians and basic props. These elements could then be tweaked and combined with hand-crafted scenery and materials to make up the version of New York seen in "Marvel's Spider-Man." This also condensed development time so that Insomniac and Sony could allow multiple passes to ensure quality.
Very early on in development, Insomniac crafted a prototype of Manhattan with placeholder buildings, allowing the devs to test the correct feel of Spidey's momentum-based web-slinging. This version consisted of a dense collection of untextured gray buildings, but still provided the perfect backdrop for developers to master Spidey's preferred traversal methods.
Bringing characters to life with voice acting
Insomniac hired a hugely talented cast of actors for the "Marvel's Spider-Man" series. Spidey and his rogues' gallery aren't the only ones to benefit from this, as but pedestrians and NPCs are also fully voice acted. To do this, actors can spend hours in the sound booth reciting from a laundry list of lines and acting out multiple variations of the same scenes. These are often recorded with one actor at a time, necessitating a great deal of focus from the performer.
In some cases, however, voice actors also offer their likeness and physicality through vigorous mo-cap sessions. These scenes often play out much in the same way a studio would capture a TV show or movie, with all actors present. Unlike a movie or TV show, however, the set dressing, costumes and props are all designed to be realized digitally. That means green screens and full mo-cap suits (more on that in a moment).
Regardless of the method involved, voice actors take it very seriously. Yuri Lowenthal, a voice acting veteran and voice of Peter Parker, is a great example. In an interview with Game Informer, Miles Morales actor Nadji Jeter revealed that Lowenthal even advised him on the physical acting of "swinging and doing the lines" in order to capture "what a swing sounds like" in their voice. After all, Spider-Man needs to sound like he's actually doing the work, right? Jeter also explained that he took the time to learn the intricacies of Puerto Rican Spanish so that he could nail the authenticity of the character and respectfully represent his character's multicultural heritage.
Mo-cap, mo problems
When it comes to realistically portraying human movement on a character asset, you can't get more genuine than the real thing. In that sense, motion capture saves animators thousands of hours of work. Insomniac uses mo-cap technology for cinematic scenes and to assist with animations for all of its NPC's. In a capture space known as a "Volume," voice artists and stunt actors will perform scenes or sets of actions. Using high-tech mo-cap suits that resemble something out of "Tron," Insomniac captures movement data all the way down to small facial moments.
This data is then uploaded into a library where it will be then be used as movement data for the various character models. For a cinematic, the raw mo-cap data might mostly be presented just as it captured. However, for in-game character animations, mo-cap footage is cut down and used in conjunction with hand-crafted animation work and clever blending techniques to create lifelike and dynamic movements, which play nicely with the rest of the game engine. These techniques are utilized by the pedestrians interactions with Spiderman that occur in the open world and during cutscenes.
You have to hand it to the actors, because physically acting in a set that looks like a floor in the Backrooms is difficult. Acting is a visual process; without the contextual cues of set dressing and many props, actors have to go the extra mile.
Rigging the game
You see, in order for a 3D asset to move, it first needs to be rigged. The more complex the character and more moving parts the asset has, the more intricate the rig. It is essentially a skeletal frame within a 3D model that can be fed animation and mo-cap data and allows for physical movement and manipulation of the asset. It also is provides readable data for the game engine itself so that the animated asset can function with all the other game elements.
A rigger's job is a technical one that is not always apparent in the final product, yet it's a very essential component in the development process. Without it, characters would simply T-pose and float around — and that would be weird.
It probably comes as no surprise to anyone that the most crucial rig of all would be that belonging to our two Spider-Men (and their many, many variant outfits). After all, Spidey is the player character and is used in almost every cutscene and cinematic. All that animation work and mo-cap data requires a rig worthy of Spider-Man's varied and complex movements, especially when you consider that a lot of Spider-Man's personality shown in his physicality. You know, on account of the mask.
Redesign: Putting the 'Friendly' in Friendly Neighborhood Spider-Man
There are many obstacles in translating the iconic appearance of a comic book character and their many foes into another medium. Thankfully for fans, this is one obstacle that Insomniac tackled with finesse and grace. While keeping true to established design rules, Insomniac's art team made a combination of creative and logical tweaks to the design of Spider-Man's iconic suit, all the way down to the musculature of Peter Parker's body beneath it.
A creative use of color — in particular, the "White Spider" design — makes Spider-Man's red and blue suit to pop off the screen. Other stylistic choices were made, including the use of paneling and even a specific style of mesh within the webbing on the suit. Even Spider-Man's boots were given modern and logical redesigns, all while still following that golden "rule of cool." In Insomniac's games, Peter's boots go up to his thighs, while the actual shoe component cleverly resembles a kind of high-end athletic shoe (which is perfect for Peter Parkouring around the city).
Spider-Man wasn't the only one who received tweaks to their appearance, as several of his iconic villains were also redesigned for the game. Perhaps the most drastic change was made to Electro, who notoriously wears a green and gold jumpsuit and sports a giant yellow lightning bolt mask in the comics. In "Marvel's Spider-Man," he receives a more grounded aesthetic that him to Insomniac's world, yet still maintains notable features that pay tribute to the original design.
Populating a city
New York City is one of the most densely populated urban environments in the world. In order to create a city that felt like it was worth saving, Insomniac wanted to populate it with an accurate number of pedestrians who weren't just set-dressing. These people needed to be dynamic and brimming with personality, especially in the presence of Spider-Man himself. That's not an easy feat by any measure. For character placement and numbers, Insomniac once again utilized those procedural systems. Houdini enabled Insomniac to generate hundreds of pedestrians on the fly.
In order for these characters to be brimming with personality dynamically interacting with Spider-Man and the world at large, pedestrians also needed to be moc-capped, animated, and voiced. This work would then need to be edited and tweaked to generate a randomized library of dynamic interactions and passive activities. Then, these animations needed to be carefully blended into the open world, taking into account space and all conceivable camera angles. You don't want Spider-Man to go in for a selfie and have him clip right through some poor civilian. That'd be kinda weird.
The animation situation
Animators are the puppet masters of the special effects world. Once character models, rigging and motion capture footage are all in place, it's up the animators to create fluid action that will play nice with the game engine and its many elements. NPC animations are one small corner that play a huge part. In the open world, animations take the form of vignettes, passive animations that seamlessly loop and have to look great from every camera angle. Examples of these include groups of people doing Tai Chi in the park or playing a basketball game. These animations provide flavor to the open world and make it feel more lived in.
For interactions, both with pedestrians and side-mission NPC's, Insomniac used fixed camera angles and specially edited animation clips that trigger when a voice-line is played. These animations were either partial or full-bodied, depending on the length and type of interaction.
Animators are also the ones who will clean up motion capture data used in cinematics to create a smooth final result. In a cinematic, the camera angle is fixed and the timing abides more strictly to sound and timing. When it comes to cutscenes, the animation work becomes more deliberate and intentional, resulting in cinematic feel.
Cinematics are a huge undertaking
Cinematics take control away from the player, but reward them with dramatic scenarios that push the narrative forward. In Insomniac's "Marvel's Spider-Man" cinematics are rendered in-engine in real time, which is why you can see whatever variant costume you've got Spidey wearing at all times.
The process of crafting a cinematic involves several important steps and multiple departments working in a collaborative effort to produce the final scene. The first step is to lock the story in place. Once the script and the narrative beats of the cinematic have been decided, every department comes together to discuss what they want the scene to look like. Now, you'll see every piece we've discussed so far coming into play.
The next stage is previsualization, when the team blocks out camera angles, animation times, and sound effect placement. After that, the motion capture unit will run a performance in the Volume and data is collected. During this process, actors and directors can often see the recorded data on monitors around the studio. After the mo-cap data has been compiled, the animation is cleaned up and is then reviewed and critiqued over a period of weeks before being distributed to sound, lighting, VFX, and rigging teams, all of whom work towards creating the final product that you see in the game.
Ever wonder why certain cutscenes in "Marvel's Spider-Man" are unskippable? It's because they're basically animated load screens. Assets, lighting and other effects are added in off-camera. For the player, this a creates seamless transitions between set pieces. It's totally not magic, but it may as well be.
Real time VFX
How do you visualize speed without going faster? How do you signify to the player a specific area is dangerous? This is the job of a Visual FX artist. For a game based on a comic book, the VFX here need to be loud, vibrant and communicative.
Visual FX for "Marvel's Spider-Man" are divided into two main categories, each of which contain two sub categories. The first is the Narrative category, which contains Cinematic and Environmental VFX. The second is Gameplay, which contains Combat and Puzzle/Mission VFX. While each of these categories contain their own challenges and require different approaches, the underlying goal for these visual FX is communication. They all need to work in harmony. These effects telegraph a certain mood, atmosphere or visceral response to the player. They can also prompt certain actions from the player, letting them know they're gaining health or that they can interact with something.
Visual FX are responsible for the blue impact marker that tells you how powerful your attack is during combat. They're also responsible for the explosions, liquid, and particle effects during cutscenes. Visual FX artists also add the final touches to the atmosphere in the Insomniac's New York. Everything from the wakes of boats to sirens and the steam rising from the city streets is the work of a Visual FX artist. Without skilled Visual FX artists, "Marvel's Spider-Man" would not be feel as cinematic or atmospheric, and certain game elements wouldn't read nearly as well.
Let there be light
For the open world of NYC, Insomniac made a conscious decision to forgo dynamic day/night and weather cycles, opting instead to lock in a specific atmosphere to reflect story beats. These pre-sets for lighting were set into four categories: day, sunset, overcast and night.
Day lighting provides a vibrant blue sky box with a warm key light (the sun) cast directly above the city to represent a perfect clear midday. This locked value acted as a basis for the other light models to work from. The Day lighting in "Marvel's Spider-Man" delivers a hyper-realistic vibe that really shows off the city. In other words, it's a great introductory atmosphere for the player.
To contrast, Sunset delivers something more dramatic by basking the city in orangeish light and low-hanging fog. This light casts dramatic shadows over open spaces, while amber rays peek through alleyways and between buildings. The Sunset lighting offers ambiance for days, making it the perfect time to do some recreational web-slinging. By contrast, Overcast provides a more moody backdrop, but blanketing a dense concrete jungle with gray clouds is no easy task. How do you sell "bleak" while still making it fun to play in? Insomniac was able to overcome this concern by tweaking color and contrast while keeping the key light on above clouds coverage.
Portraying New York at night required focusing twilight on the rooftops, then illuminating it on the street level through predetermined lights from buildings and street lights. Another requirement for the night setting was ensuring important landmarks were illuminated, preserving the New York of it all.
The cutting room floor
In a talk at GDC, Insomniac Games creative director Brian Horton shared the motion capture footage of a cutscene from "Marvel's Spider-Man: Miles Morales." The scene depicts Miles (Nadji Jeter), Phin (Jasmin Savoy Brown) and Rio (Jacqueline Pinol) sat around a dinner table, having a difficult conversation about the dangers of living in Manhattan. In the mo-cap footage itself, fans get an excellent behind-the-scenes look at how much work goes in to preparing a Volume for a shoot, as well as the dedication of the actors involved. It also allows viewers to really appreciate the complexity of each of the actor's mo-cap suits and how props are utlized in the scene.
However, this exemplary body of work was left on the cutting room floor because it created issues that conflicted with the narrative flow of the game. Snipping this was not an easy decision, but Brian Horton justifies these cuts quite eloquently in the talk, explaining that it felt repetitive with previous character introductions, made the tone inconsistent, and provided too early of a motivation for Phin being revealed as the Tinkerer later on.
Still, the footage stands as a great example of hard work, even without all of the fancy effects layered over. It also makes you does make you wonder what else was cut from the game, and how difficult it must be to bid farewell to hours of work.