Concept

In general, Cel Shading refers to a technique used to render Anime-styled contend in CG. Normally, for something to be considered "cel shaded", it must be rendered in a way that draws hard lines for shadows, uses only few different shades (mostly only 2-3), and the contures of the object should be rendered as black lines.
The classical approach to this in rasterisation is to simply shade the object with a shadow texture of 1x32 greyscale-pixels. When a point is shaded, it's color is simply multiplied by one of the pixels. The pixel is taken accordingly by the dot product of the surface normal and the light direction (normalized to [0,1])
The outlines are drawn by rendering a second object with the line-width increased, it's surfaces flipped, and surface culling activated. This leaves only the outline of the Object, which is then colored solid black
Unfortunately, neither of these approaches map well to raytracing, thus we had to implement it differently.

Examples of Cel-shading in production

Implementation (cel.h, cel.cpp)

The code we're referring to here can be found in the cel.h and cel.cpp files in the material folder
Our first attempt was to simply create a lambertian-model, with it's diffuse component as the color and it's specular component as the shadow texture. We would then simply make a lookup of the shadow texture by the dot product of light direction and surface normal for each light.
This had the problem of not providing a sharp separation of the shades as the number of lights increased. Area-lights we're especially bad. Our current implementation uses a specific vector to calculate most of the lightning (as described above), and uses that for emission. Reflections are then rendered at ca. 1/10th of their strength. This has the drawback of really discombobulating with physics. like, discombobulated-on-the-gasstation-toilet bad. But who cares if it looks good.
The outlines we're adding in post-production. We use the edge-detection algorithm described in the following pages to detect edges and render those black instead, as all other approaches would have required to make the rays aware of each other, which would discombobulate up multithreading, which was to much of a hassle to implement.
This is what we use to shade Kallen and Sakura in our scene.