PIXAR Deep Dive on SSS: SIGGRAPH Preview

Pixar’s RenderMan is one of the most respected pieces of software in the industry. We look at the new release and it’s two new Sub Surface Scattering Brute Force approaches.

Christophe Hery joined Pixar in June 2010, where he holds the position of Global Tech and Research TD. Christophe has just finished helping to write new skin lighting models and rendering methods for RenderMan to be used in upcoming (unnamed) Pixar projects. But the new approaches are also included in the next release of RenderMan 21.5 – the details of which will be announced at SIGGRAPH 2017 in LA.

An alumnus of ILM, Christophe previously served as a research and development lead, supporting the facility’s shaders and providing rendering guidance.

---

Must Watch: Meet Mike

---

Brief History

As white light falls on a face, in broad terms, the light either reflects immediately as a specular highlight, or it is diffused. The highlights are white as the specular (spec) is effectively a mirror-like bounce. The spec shows a lot of detail in the pores of the skin since the light is sharply reflected. Diffuse is, well,…everything else. Spec + Diffuse = all light energy.

Most methods have assumed some Spec, and the rest is made up of diffuse and sub-surface sampling or SSS. Even with low Spec, most previous models of SSS assumed some degree of diffusion to look right.

Sub surface skin based diffusion is happening due to light entering the skin, bouncing around and coming out. As the skin has hemoglobin (blood) and various flesh properties, the light comes out pinkish and our non-specular skin looks red-ish and natural. The problem is so complex to simulate in the computer as the light does not scatter at a single point: it enters the surface at one point, and exits (often multiply scattered) at various other points. Various techniques have been proposed to emulate this, normally as just an approximation. Traditionally this approximation is based on prebuilt a point cloud, distributed on the surface of the object. These points are used as the exit points for the light.

SSS sounds simple, but simulating this in a computer is both very complex, and it is something that we humans can sense with enormous sensitivity. For this reason, the quest for believable humans has been a quest for believable skin shaders. The largest breakthrough in believable skin shaders came with the journey to map, simulate and emulate this sub-surface scattering.

The earliest work in SSS came from Screen Space blurring, one of the latest version of which is Jorge Jimenez et al’s Separable Screen Space Subsurface Scattering. This approach is an effective approximation that simulates subsurface light transport by approximating the radially symmetric non-separable diffusion kernel with a sum of separable Gaussians, which requires multiple (up to twelve) 1D convolutions. This is sometimes known as the ‘Activision solution’ and it was shown very effectively at SIGGRAPH 2013 in a collaboration between Activision and USC ICT at the Real-Time Live Event for Digital Ira.

Slightly away from the Screen Surface SSS approach has been the ray tracing solutions. A key start of this process was A Practical Model for Subsurface Light Transport, published at SIGGRAPH 2001. 16 years later, at SIGGRAPH 2017, Pixar will launch RenderMan 21.5 which has their latest solution to this pivotal computer graphics problem. Unlike Screen Surface SSS, the new RenderMan process is not aimed at Real time but it is also able to aspire to being more accurate and involve less approximations.

While the initial approaches have been approximation approaches, built from this initial Dipole approach, the new solution is a brute force path tracing approach. Most of this computer graphics work parallels developments in the Nuclear Physics Research community dating back to the 1940s. From the Nuclear Industry the approaches moved to medical imaging and today computers are allowing companies like Pixar to attempt some of the most complete and direct solution yet attempted. But more relevantly, the newest Pixar SSS directly embraces new Nuclear physics theories from Berkeley’s Nuclear Engineering Department.

The dipole method that first offered reasonable SSS approximations was based on a Bidirectional Scattering Surface Reflectance Distribution Function (BSSRDF), that used a dipole (meaning two or pair) diffusion approximation allowing for efficient simulation of highly scattering materials such as skin, milk, or wax.

A BSSRDF captures the behavior of light entering a material such as human skin and scattering multiple times before exiting. The light is approximated by a single accurate scatter combined with a dipole (twin light) method. This is has been the most common SSS approach. (Note the twin lights are NOT equally above and below the surface). The dipole method approximates a BSSRDF by combining an exact solution for a single scattering with a dipole point source diffusion approximation, that simulates the real world multiple scattering. For some time, from the world of medical physics, it was known that this ‘dipole’ (dual light) method could represent incident source distribution on the surface of say human skin. A single dipole method was thought to be, initially to be accurate enough.

As the name implies the dipole method consists of placing two point sources near the surface – one light source beneath the surface and the other located above the surface. Any incoming ray is transformed into a dipole source for the diffusion approximation, then combined for the final approximate scattering, although the mathematical details rarely figure to any lighting artist in production. What matters is that in a move to gain rendering speed, with the dipole approach, – accuracy was scarified.

Monte Carlo path tracing techniques can be employed with high accuracy to simulate the scattering of light inside a translucent object, but at the cost of long render times. “Ryusuke Villemin and I started “playing” with this full PT method during Finding Dory, not sure of the date, maybe around 2 years ago”, comments Hery. Who has worked on this problem for many years. “Jacob Kuenzel tested it on various assets, but we all realized that we did not get a correct albedo inversion for it, so we parked the research for a while”, he adds.

Then at last year, at SIGGRAPH 2016, Matt Jen-Yuan Chiang from Walt Disney Animation Studios had a short talk on Path Tracing SSS at which he showed some great tests that the Disney team had done for the film Moana. These were only test and they themselves abandoned it for use in that film, but it re-ignited Pixar (also part of Disney, “So we were re-assured that this could be done, and started re-working on it” commented Hery. “I came up with a working first version of the albedo inversion, and then Magnus Wrenninge joined our group and started making it better, in particular he built a table/fit that respected the phase function (what we call directionality).” “Ryusuke added Dwivedi sampling in it (for performance), and Magnus, who had been in touch with some nuclear scientists at Berkeley, pushed for implementing a non exponential version.”

In the end, it just came down to Subsurface scattering rendered using brute-force volumetric path tracing looks more natural and is also more robust compared to popular diffusion-based approximations.

New Versions

Pixar Surface in the current version of RenderMan 21 has many modes of SSS. There are three main types of Multiple Scatter, Jenson, D’Eon and Burley, along with a user requested Multiple DMFP mode, (which is the least physical of the set). In addition, there is single scatter as a separate lobe(s). The new version of RenderMan has two new modes of Sub Surface one exponential and one not (more on that below).

These new methods move the SSS to a more accurate simulation of what photons do inside a transmissive medium like skin. Pixar is now doing a full random walk as opposed to using a diffusion simulation model, and this has real advantages, but isn’t actually computational more expensive, “and it can actually be faster in some cases”, explains Hery.

For some time, RenderMan has been moving to more accuracy, away from approximations to a more directly simulated, physically correct, ray tracing approach. This is the latest step in that process. “It is a more complete solution, in the past you had to think about diffusion models and which profile to use, such as the original Jensen Dipole or Burley Normalised PBS, but you’d have to also add single scatter to that” comments Hery. As noted above, all the existing methods also required a single scatter to be added.

This new solutions are more complex but they do everything in one solution. Prior to this a TD may have had use two lobes and spend more time going across the two lobes, both artistically and mathematically.  Now, there is available two new approaches:

1. Brute force path tracing (albedo inversion)
2. Non-exponential brute force path tracing.

“So we are going from 3 + 1 approaches, (the +1 being the GFP), to effectively 6 options or modes”, explains Hery. But the big difference with the last two models is that they would not need an additional single scatter.  “Both of the two new Pixar solutions incorporate single scatter and multiple scatter and even ’zero scatter’” adds Hery.

Zero scatter SSS?

In the case of thin skin such as with ears, it is quite possible to have a backlight that would allow some of the light to transmit through the skin without scattering. “It is almost like refraction, the skin is thin enough that it goes from the back to the front without bouncing”, adds Hery. The original algorithms assume locally flat semi-infinite mediums, .. “so in effect, they only looked at it from the event happening on one side of the interface(skin).” In other words, the points the light exits is the same side from which the light entered. Look at any diagram explaining SSS and that is how it is normally drawn. “But obviously in practice it is never the case  in production, so we always reach some point where this assumption breaks” says Hery.  Zero scatter is not the most significant of the three types of scattering, but is a serious enough problem, to be a real issue inside Pixar when doing production work, and requires compensation to look correct. The two new SSS approaches in RenderMan solve this as part of a more comprehensive and complete brute force solution.

The Two New Solutions:

There are two new solutions:

• Exponential and
• Non-Exponential

As noted above, SSS in CGI is based on some core science such as Nuclear physics and Chemistry. To understand the new SSS one needs to look briefly at some key laws in scattering and absorption in materials.

Lambert’s law states that absorbance of a material is directly proportional to its thickness (path length). Beer’s law states that absorbance is proportional to the concentrations of the attenuating ‘species’ in the material. Together they define what is happening in a material.

In the past it has been argued that this is combined Beer Lambert law is exponential. “Recently, there has been a new view.” comments Hery. “People such as Magnus Wrenninge at Pixar, (author of Production Volume Rendering: Design and Implementation), were wondering if there was a newer view on this from research into Nuclear Physics. Eugene d’Eon (now at 8i) while he was still at Weta Digital, had gone down this same path a while ago,… anyway, this research was in partnership with University of California, Berkeley. Richard Vasques at Berkelely pointed out that  in Photon and Neutron transports in Nuclear Physics, they had discovered that the exponential attenuation was not what they had observed previously”, explained Hery. In simple terms, if the distribution of particles is closer to that of skin, then the fall off does not need to be exponential. Using the new maths from Berkeley, not only is the results more accurate, but it produced a new variable that RenderMan can offer to an artist: bleed.

Bleed is a result of the new approach mathematically and as a variable it makes the skin have more ‘bleed’ between edges.

---

In the image above, focusing on the two images on the left: the first is not sampling the phase function and it assumes exponential fall off and isotropic. It is the first new ‘exponential SSS’. The next face image (second from the left) shows more added definition and the sharpness in the pores, that comes from moving from isotropic (phase of 0) to forward scattering (phase of 0.7 or 0.8).

All these faces – other than the far left one,  are sampling the phase function and forward scattering.  These are all using the second new SSS option, with forward scattering and with various levels of Bleed (from 0  to 10).

How does this compare to previous methods?

If you look closely at the nose shadow region in the older methods in the image below, you can see how the Dipole method seems to have light under the nose, coming from nowhere. This is actually the sub skin dipole light showing through incorrectly. This section of skin looks backlit for no reason. In the new methods, (three faces on the left) the model is more accurate and as such it is solved with brute force and the Dipole artifact is gone.

The other aspect to note in the new SSS image above (on the left) is how the forward scattering with 07 and 08 provide sharper pore detail. “In a sense the 07 or 08 give you more pore detail, it is almost a sharpening kernel… and then from there you can dial in the amount of bleed” explains Hery.

What does the 00 or 07 numbers indicate? “If you think about the light inside the skin inside a sphere, at 00 the light can go in any direction. At 0.7 it is more likely to go forward, 70% forward, 30% backwards,” explains Hery. Clearly, this number becomes very relevant on backlight skin like ears. Changing this number will have a real and visible effect.

Picking up on the points above regarding zero scatter, this also reflects that Pixar no longer assuming the skin is semi infinite. All the previous algorithms assume the skin is so deep as to only worry about the light coming out the same side it goes in.  The new models acknowledge how often backlight means the light is exiting from on a different side of the skin, to where it entered.

Additional Diffuse (for rigorous accuracy)

The new world order in rendering today is energy preserving, physical shaders. To that end, there is a tiny amount of light not accounted for with these Pixar SSS models.

In theory, while the new SSS from Pixar provide a complete solution, it is possible that there is some diffuse not accounted for by the SSS model (in theory, but rarely in practice). In production, a TD would use two Beckmann distribution specular lobes for specular highlights of the skin. The second of these will normally hide any tiny diffuse component not in the SSS but RenderMan even offers this as an option, although it is normally so trivially small, that this option is set to off by default.

Speed

How is it that these new methods are not vastly slower?

Firstly, the new approaches are optimised, and so they are efficient. But the main speed advantage is that the older solutions required both a single scatter and a multiple scatter. Now those two are being replaced with one solution, “It is hard to give estimates, but I would say the new version may be within 10%, but if the skin is darker in colour, the new method could be much faster,” says Hery. Due to Monte Carlo rendering and how things happen in the sampling.

Is there any reason to not use these?

Everything comes with complications. One of the reason that these new methods are better is that they do not assume a semi infinite one sided piece of skin. But this means that the interior (or inside) modelling of the face now matters more. Teeth and bone modeling become important, and also by the same logic, surfaces need to be closed. “If the geometry is single sided and not a closed volume, then you can’t use path tracing subsurface, as it will keep marching and marching and never exit”, he explains.

---

This new SSS represents another major step to realism. “It is subtle, but you can see a difference”, explains Hery. If SSS was not something that is featured in faces, then perhaps the audience would not need Pixar and others to push so hard to improve their algorithms. But solving skin and human faces still remains one of the greatest challenges of computer graphics and visual effects. With these new SSS models, Pixar moves significantly closer to ground truth.

For more information: Click Here