Ben Snow, visual effects supervisor for Pirates of the Caribbean: On Stranger Tides, answered some questions about the making-of the film, available on 3D Blu-ray, DVD and Blu-ray on October 18.
Q: When producing the mermaids-scenes, what where your inspirations for their movement and the general “look” of the mermaids?
Ron Snow: We wanted to avoid the classic ‘woman in a mono-fin’ that has been seen in past mermaid films so we studied a variety of marine mammals and fish for the motion of the mermaids. We also had synchronized swimmers and professional athletes that we filmed on set to help guide the animation. As for the look, we were trying to create something different than what you’ve seen before but keep it rooted in reality. We used reference of sea life, human skin, etc. and also bought some fish at the fish market and photographed the heck out of them. In the end, the mermaids had a scaly body with a membrane of sheer, almost jelly-fish like, tendrils.
Q: How much influence did filming in 3D have on your workflow, and which problems / possibilities did you have with that?
Ron Snow: We were excited by the prospect of 3D. ILM’s 3D pipeline was fairly solid after our work on Disney’s Star Tours 3D and Avatar, and we did some initial work to make sure the 3D set-up for visual effects was rock solid. Creatively we played with the concept of adding more depth for the underwater scenes. We also made sure we were viewing our work in the largest theater at ILM because with 3D the size of the image is important when reviewing work – we always want to see it as an audience would.
Q: When you are working on an animated character, what is most important to you?
Ron Snow: We want the character to be something unique and fantastic but also rooted enough in reality that it is believable. For the mermaids having the footage of marine animals and the swimmers and actors for reference in the filmed material helped make this possible.
Q: If you compare your work on Pirates of the Caribbean to Iron Man, and before that to The Mummy (for example), what are the most notable differences in your workflow and in the tools you use?
Ron Snow: Pirates had more organic creatures rather than the metallic hard surfaces of Iron Man, so it had some of the challenges of The Mummy. However today’s tools have advanced a long way since I worked on that film. We were still experimenting with facial motion capture on that film and since then it has become a useful and reliable technique. Our abilities to capture the motions of performers on real sets or locations is far more advanced now than what we had for The Mummy. We were able to leverage improvements we had made to those techniques for Iron Man and other films and push them even further to get a seamless blend between the actresses’ bodies and the mermaid tails.
Q: If you compare Pirates of the Caribbean: On Stranger Tides to the other parts of the franchise, where do you think are the main visual differences?
Ron Snow: We were conscious of fitting into the world of the Pirates films and to live up to the quality of the effects that had been established in previous entries. However, Rob Marshall brought his own visual sense to the series. He wanted to be sure that the most fantastical elements also were rooted in reality, and brought a grittiness to the film. He also emphasized the story and characters in Pirates of the Caribbean: On Stranger Tides and the visuals, though still high quality, are in more of a supporting role this time ’round – it’s more about the adventure and the story.
Q: What’s the most fun part of your job?
Ron Snow: The best part of my job is coming up with what something is going to look and move like, and then how on earth we’re going to execute it. And the mermaids represented a tremendous opportunity in that way. I’ve been a fan of mermaids for years and particularly liked the fact these were predatory rather than the friendly mermaids. The biggest challenge was maintaining their beauty while making them scary. Working with Rob Marshall, Charlie Gibson and the team to come up with the look was a lot of fun, and the enthusiasm of Astrid Berges-Frisby, who came to ILM and let us photograph and scan her and have her do motion tests to help us make a better mermaid, was infectious. Working out how to best capture the performances on set and apply them to our CG mermaid was very cool because it involved using all our latest motion tracking tools and some new ones we invented as well.
Q: What are your favorite films that you have worked on in your career?
Ron Snow: I loved working on Pirates of the Caribbean: On Stranger Tides. Even with the challenges of finding the right look for the mermaids. I also had a lot of fun on Galaxy Quest, which is a film I still enjoy watching. The Iron Man films were also a blast because we got to collaborate with Jon Favreau and Marvel in coming up with the filmic persona of a new hero and helped launch a franchise. And my work on Star Wars Episode 2 was both a great opportunity in terms of career and also gave me a lot of freedom to come up with new ways of doing effects because of the sheer amount of the work and the collaborative relationship George Lucas fostered.
Q: Where did you get the ideas for the look of the mermaids from?
Ron Snow: The look of the mermaids followed a lengthy discussion process with Rob Marshall and a lot of concept art exploration. In talking to Rob he showed us some previous design work that had been done as well as some classic paintings of mermaids, underwater fashion photography and even a couple of mermaid pinups. He told Aaron McBride, our VFX art director, and myself what he liked and didn’t like about this material. One image he liked had a woman in a sheer fabric tail that you could still see the outline of her legs through. He also talked a lot about the beauty of Astrid Berges-Frisby and how some of the previous concepts had been too creature-like. At the same time we had been looking at different types of marine life, and trying to tie the mermaid into the marine life feel of previous Pirates characters we wanted to integrate, and also discussed how she might actually transform. So we came up with a slightly scaly body covered with a sheer membrane. The idea was that when she was wet the membrane was transparent and you could see flashes of scales, but as she dried it became more like human skin. It was a fun process to invent the science behind these characters.
Q: During the planning of the scenes, what were your major concerns for the final product?
Ron Snow: Our major concerns were to deliver the best mermaids we could – to try and create definitive filmic mermaids, and to live up to the quality of work that we’ve seen in previous Pirates of the Caribbean films.
Q: Please describe how you used 3D in the effects process. Was it a mixture of conversion and shot-for-3D, and how did you resolve any 3D challenges?
Ron Snow: We made the decision to shoot in 3D early on, so the only conversion we really had was for underwater shots (which all heavily involved digital effects anyway). Something that helped simplify our lives a little bit in dealing with 3D was the decision to shoot with parallel cameras which made it possible to play with the 3D more during the editing process than has ever been possible on previous films. We had several 3D experts on our crew (ILM has been doing 3D projects dating back to Captain EO in the ’80s, and we had one of our senior crew members who’d been at ILM when they were doing that project) and there was a lot of expertise on the Disney/client side.
Q: What kind of education did you get to get where you are today?
Ron Snow: I studied computing in college with a major in film (which was somewhat unusual back then and I had to get special permission). But after a couple of years of working in computing and doing film as a hobby, I actually started in the industry as a runner – delivering parcels and making cups of tea. So it was kind of the school of hard knocks. Luckily for me this was 1988 and The Abyss and Terminator 2 came out within a couple of years of each other and the industry exploded with a need for computer skilled filmmakers.
Q: Normally, ILM works on single characters or effects, but with ”Rango” they did their first complete movie. Would you be tempted to work on such a movie?
Ron Snow: A lot of my friends at ILM had wanted to do an animated film for years, and I’m very happy they got a chance to do so and I am very impressed by the result. I’m more interested in the magic trick aspect of visual effects, in trying to get the audience caught up in the fantasy and wondering later on how we did it. While I prefer effects for live action, I do love animated films and I hope ILM gets to work on another one.
Q: Do you involve the actors in your work and if so, how?
Ron Snow: Yes, we depend upon the collaboration of the actors in our work. One of the sequences we worked on had Jack Sparrow playing with a droplet of water and I had to draw dots on Johnny Depp’s fingers with a sharpie. He was very helpful and tolerant. Johnny and several of the other actors went through an elaborate photographic and scanning process so we could make digital versions of them if needed. For the mermaids, it was even more elaborate but they were great about it. We’d put them in these psychedelic bathing suits we had designed to make tracking them easier and blending the tails with their bodies, and painted dots all over their faces. Astrid Berges-Frisbey (the key mermaid) was a little concerned when we had to paint dots on her face for a key emotional scene (she was originally going to transform at the end of it) but I promised her they’d be painted out and no one would know. Thank goodness for our talented roto and paint crew who were able to paint out every single dot, and in 3D no less.
Q: When you look at all the films you have made, has post production and Visual effects changed its role in film making in general? Has the perception of “those FX-guys” changed to a more creative part in the production, or does it slowly develop into something that is “sourced out and forgotten”?
Ron Snow: We’re still a creative part of production and in some ways have more respect afforded to us in recent years since the visual effects are such a big part of the process now. Of course our credits are still way down in the end of the film and I’m hoping that will change. It’s certainly never outsourced and forgotten because on a project like Pirates the visual effects are such a key part of the film.
Q: In your opinion: when James Cameron and Peter Jackson can convince the industry to go up to 60 frames per second: will this benefit the experience of visual effects, or just make it that much harder for the post-production?
Ron Snow: It will certainly make it harder for post-production. The faster frame rate should make for something that feels more real to the audience but that could make it harder to fool them with our visual effects magic. I’m excited to see the results.
Q: What initially got you interested in visual effects?
Ron Snow: My dad was a film fanatic and we went to the drive-in a lot when I was growing up. But I pin my initial interest in visual effects to finding a book called “Monsters from the Movies” in my high school library and then later in high school reading an American cinematographer article by Dennis Muren about his visual effects photography for The Empire Strikes Back. So I’d say it was a combination of that and an interest in stage magicians of the golden age.
Q: If you have to decide on one single shot that represents the film’s visual effects work, which one would it be and why?
Ron Snow: I’m cheating with two shots. I love the shot where the Queen Anne’s Revenge comes chasing the mermaids belching fire. CG ship, CG fire, CG water and CG mermaids and then panning onto real pirates on a set with a digital extension. It was an exciting shot that used a lot of different techniques. My other favorite is panning down from Syrena to her tail just after Phillip immobilizes her with his sword. Astrid looks great in the shot. The lighting on the tail is lovely and the blend is seamless, so I think it’s a beautiful shot that showcases the mermaid tails nicely.
Q: You have been nominated a few times for an Oscar. Do you feel any pressure to uphold or improve that level of “perfection” in future projects?
Ron Snow: Absolutely. We always try and top ourselves with each project.
Q: Besides the mermaids, you also were involved in the shrinking ships in bottles. What could you explain to us about this?
Ron Snow: This was a fun sequence that was done by our Singapore ILM studio under my supervision. We shot some ships in bottle props on set and then replaced the ships in the closer shots with a full environment with the Black Pearl, stormy CG water, lightning and so on. In one shot we used a real monkey and in others it was a digital monkey.
Q: Did Disney give you any guidelines as to how scary or dark you could go with the mermaids?
Ron Snow: Yes, they actually encouraged us to really explore the scary side. In the end, Rob Marshall decided to reign back a little on the more creaturesque aspects and go for something which preserved more of the beauty of the original actresses. And I think this was the right decision – they still seem plenty scary enough.
Q: When putting the different scenes together – at what point did you switch between the actresses and the swimmer athletes, and how did you comp it together?
Ron Snow: The swimmers were used as a reference guide for our animators adding CG mermaids to shots. We used the actresses when we were closer to the mermaids and added scales, tendrils and tails.
Q: You said that in 3D size is important – concerning that, what are your opinions towards mobile TV / Movies on handheld devices?
Ron Snow: I had a lot of fun playing with the small screens of 3D portable players but I’m not sure how a 3D film would play on them. I think it’s fun but I’d prefer Pirates of the Caribbean: On Stranger Tides be seen in a theater or on a large 3D television screen and in Blu-ray 3D. That said, I did recently see the film on a plane in 2D and equally enjoyed the story and characters…so it works in any format.
Q: Did a fluid medium (something the normal watcher of a movie doesn’t experience on a daily basis, compared to moving on land) give you a bigger range of possible actions, or do you think that when switching mediums you have to be more constricted to physical possibilities?
Ron Snow: We tried to keep things as physically real as possible but definitely exploited the underwater medium to play with depth and perspective with the mermaids – so it was quite freeing in that way.
Q: Who are your mentors and do you confer with them frequently?
Ron Snow: For Pirates it was great to have John Knoll and Hal Hickel who worked on the previous films at ILM close by. John has been a mentor over the years – he was a supervisor on Star Trek: Generations, which was the first film I did at ILM. Dennis Muren has long been an inspiration and a mentor – he was a big collaborator on Star Wars, Episode II and is someone I can always bounce ideas off of or ask for an opinion.
Q: Its one thing to use ILM’s tracking suits on dry land / on the deck of a ship in “Pirates of the Caribbean: Dead Man’s Chest,” but quite another to use this kind of technology in a water-filled environment. Can you talk about how you and your team worked around / dealt with these production challenges on “On Stranger Tides”?
Ron Snow: We definitely had to up the ante. While we utilized our patented Imocap suits and technology, we also developed some new tracking approaches for getting a match to the surface (since we had to seamlessly blend skin to skin) and tools for trying to cancel out the refraction of the water.
Q: How much input do directors have in your work? Are some more “hands on” while others give you more freedom to work?
Ron Snow: Some directors are more hands on than others but all have lots of input and the final say. I’ve been lucky that most directors I’ve worked with are very collaborative, inviting us to contribute ideas, using some and rejecting others. It’s all part of the creative process.
Q: What makes a visual effect stunning and memorable?
Ron Snow: For me it’s when the effect is wonderful and fantastical and something that you haven’t seen before. When you either don’t realize it’s an effect or are scratching your head because you know it can’t be real but can’t work out how it was done.
Q: Did you have any conversations with Astrid in regards to the CG component of her character? What she could expect to see in the finished film, etc?
Ron Snow: Astrid came to Industrial Light & Magic for scanning and photography her first day on the film and right from the get go, discussed her character and how we could make the mermaid work well. She was conscious of helping us do our job and even let us re-photograph her later on to help with one of the trickier shots once the filming was done and she happened to be visiting San Francisco and came to say hello. She was a terrific collaborator and making her character Syrena work was something she was passionate about.
Q: What were the visual effects created by Industrial Light & Magic besides the amazing mermaids?
Ron Snow: We did the ships in bottles, the fountain of youth scenes, Blackbeard’s death, a bunch of digital matte paintings for the island and the mermaid cove. We also supervised work done at Scanline VFX in Germany on the sequences when Jack Sparrow had drops of water dancing on his fingers and the animated water as they enter the fountain.
Q: What proved to be the biggest challenge in the making of the mermaid scenes?
Ron Snow: The biggest challenge was the creative one of how much to play them as creatures and how much as human. Ultimately, Rob Marshall decided to keep them as human as possible, and that was itself a huge challenge because of a phenomenon that they call “The Uncanny Valley” in robotics. The more human something artificial starts looking, the creepier it looks. So it was a lot of work to make the CG mermaids more and more human looking, changing their seaweed hair to human hair and their faces to matching the actresses more exactly.
Q: What do you do when you want to create a certain effect in a film and you don’t have the technology to obtain that effect? Was there ever the case?
Ron Snow: A big part of the fun of my job is coming up with the technology to achieve a certain effect. Luckily I work with an amazing group of people here at ILM and I don’t think we would ever turn down a challenge.
Q: Do you have any anecdotes that you can share about the Mermaid Lagoon shoot on the back lot of Universal Studios Hollywood?
Ron Snow: We had a great time working on the mermaid lagoon, surrounded by the lovely mermaid actresses and swimmers. My colleagues who got to apply our motion tracking bands to the mermaid costumes told me I’d given them the greatest job of their careers. The mermaids all knew their names and would call out friendly greetings as they were working around the set, arousing the envy of the rest of the crew. It was also a blast working at Falls Lake in the middle of Universal Studios and being part of the studio tour as we were setting up for the evenings work.
Q: What advice would you give to someone who dreams about wanting to get a job in the visual effects industry?
Ron Snow: Don’t give up! I was a dreamer for many years. I started in the industry at the bottom as a runner and then worked my way up. Try and get the skills you think would help but most importantly, grab a video camera and make movies with your friends, at school, etc. You really have to love this job because of the hours and the stress but it is immensely rewarding.
Q: After working in the ‘Iron Man’ franchise, with cold technological armors, you had to deal with mermaids in ‘Pirates’. How it was the experience? Where did you find the main challenges in a first approach?
Ron Snow: It was great to work on an organic creature again after all the metal of Iron Man and Terminator. But like Iron Man, we had to use the best of our existing on-set tracking tools and invent a few new ones to match the bodies to the performers but in this case it was the surface of their skin. We also had to create skin, hair and everything else that goes into an organic creature, which is a different set of challenges to a metal suit.
Q: You used a hybrid approach for the mermaids, with CG and real action performance. Was it particularly complicated to find the right balance? How did you create these mermaids?
Ron Snow: It was complicated, and a big creative question we struggled with. Ultimately Rob Marshall decided to go with more human mermaids and we used the actresses when we had them, got a couple of the mermaids back to film them for a couple of shots that had been more creaturesque, and put all our skills with creating digital doubles to use. We used every trick in the book – match-animation, facial motion capture, digital skin and a lot of compositing techniques with splashes and fog to make it all hang together.
Q: Ben, any final thoughts that you’d like to share on Pirates of the Caribbean: On Stranger Tides?
Ron Snow: I’ve really enjoyed the questions in this roundtable session, so thank you all. I loved the chance to create our version of the mermaids for Pirates of the Caribbean: On Stranger Tides. There were definitely some big challenges but I’m happy with the results and am very glad that audiences around the world have embraced the film, and the mermaids. Although I’d warn people to be careful about embracing mermaids as the movie shows.
Pirates of the Caribbean: On Stranger Tides is now available on 3D Blu-ray, Blu-ray and DVD.
For more information on Pirates of the Caribbean: On Stranger Tides , visit the official Pirates of the Caribbean web site.