Look development in Cel-look CG

(Polygon Pictures Inc. / Studio Phones)

The moderators of the "Craftsmanship and Artist Techniques" seminar series have prepared this document on the occasion of the development of that series.
Please see the history page linked below for details on this seminar series.
https://a-film-production-technique-seminar.com/crafts/crafts_history.html
translated by PPI Translation Team

■Overview


Look development for cel-look CG is a phase of production which often involves controlling the areas to which different colors are applied to create a cel-look, setting up on/off controls for the rendering of contour lines depending on the dynamics and shape of the contours, and applying texture to assets, particularly characters. In domestic productions, we see many studios that have constructed workflows which combine this phase of production with modeling work; in particular, shaders and textures are applied to model data, and the parameters of each are set, in this phase.
Because the data created in this look development phase will be used directly for rendering purposes, it is necessary, especially in the case of cel-look CG that makes use of 3DCG, to set parameters meticulously so that, when positioned in a 3D space and animated, these textures and contours can be stably rendered from any camera angle with minimum possible breakdown.
Even as they inherit the cel-look of traditional hand-drawn animation, in recent years studios are experimenting constantly with various solutions and individualized techniques in order to depict these things using 3DCG. In this document, we shall explain how, as simply a single example of cel-look CG, Polygon Pictures grapples with these issues, and examine important points related to cel-look CG look development techniques, as well as issues pertaining to intra-studio coordination.
Also, in the “Conclusion” section at the end of the document, we shall, by setting in order the challenges facing this look development phase, present ideas on technical approaches that take account of coordination with other departments which can contribute usefully to developments hereafter.


■Look development and pre-production


Look development, mainly for things like characters, is taken into consideration from the character design stage during pre-production, where the fundamentals of the look are established.
A flow that has become widespread in cel-look CG production is for such things as textures and contour lines to be included in the 2D design image in pre-production, and then for 3DCG data to be constructed on the basis of this information during the production phase.
This phase of production is important in that it synthesizes various settings from things like color design (described below), but with regard to settings also needed in actual animation production phases like rigging, it is positioned as a phase in which, via various trials and prototypes, quality is steadily raised with a view toward production, making it an important phase truly responsible for developing, in a broad sense of the term, things related to the look.
This document focuses mainly on explanations of aspects related to the look, but please also see the documents below for material including rigging.

Character modeling in Cel-look CG with a procedural perspective
https://a-film-production-technique-seminar.com/fppat/materials/ppi_phones_cel_look_character_modeling/index.html

Important points to keep in mind based on rig design
https://a-film-production-technique-seminar.com/fppat/materials/ppi_phones_rig_design/index.html


Character Look Concept Image
©2018 TOHO CO.,LTD.




■Representation of 2nd shadow


The method whereby the character is given a three-dimensional look is by using a darkened version of the character's usual color (base color) as a shadow color. The color of the shadow areas is called the shadow color.
When a darker version of the standard shadow color is used to portray even deeper shadows, this is known by the industry term 2° shadow. The use of 2° shadows originated in traditional cel-look animation, and this technique is widely used in cel-look CG as well, where it is generally implemented as a function within the shader.
The image below is an example of shader parameters. The region indicated with a red arrow is the 2° shadow.

Shader 2° Shadow Settings



We shall use the following illustrations to briefly explain how 2° shadows are generated in cel-look CG.
When a directional light source illuminates a given object, the side opposite the light source will be in shadow.



Although a threshold can be used to easily generate an area within the side of the object being illuminated, this differs from the 2° shadow effect we are seeking.
The effect we wish to create with a 2° shadow is to depict a shadow within a shadow.



However, because no light is striking the shadowed surface, there is no information with which to subdivide the shadow and 2° shadow areas, making it infeasible to divide these areas using a threshold. For this reason, we added a system which generates an area as an effect on the opposite side of the the light source of a normal light.
(This is the same concept as the method used in compositing to generate rim lights.)





2° Shadow Control
©2018 TOHO CO.,LTD.




■Highlight expression of hand-drawing hair


One distinguishing trait of the depiction of characters in cel-look CG is the adoption of methods to portray hair highlights with a hand-drawn shape and texture. While methods for depicting highlights, which include creating the highlights as texture images beforehand and mapping them, or generating them dynamically within shaders, vary by studio and by the style of the film or show, here we should like to describe a method of dynamically generating hair highlights within a shader.

The image below is an example of a hair highlight, motivated by a desire to portray, in cel-look CG, the sort of highlight shape seen in cel animation.

Depicting Hair Highlights
©2018 TOHO CO.,LTD.


In order to achieve this sort of depiction, in this example up to five semi-transparent ovals overlap. The aim is to produce a painterly anime-style highlight similar to repeated vertical watercolor brushstrokes.

1. A belt-shaped area is generated and projected onto a sphere at a fixed height. In this image, it is slightly tilted.
2. The shape is altered using a sine wave relative to the band in 1.
3. Vertically oriented distortion is added to each of the ovals.
4. Randomness is added by adjusting the size and transparency of one of the ovals, and a vertical fade is added. (This has only one layer displayed.)
5. The other four ovals are also adjusted for randomness, using varying parameters. (This has all ovals displayed.)
6. Finally, intervals are randomly deleted and spaces are created between the ovals.

Process to Generate Hair Highlights
©2018 TOHO CO.,LTD.


After achieving the look above and satisfying other design requirements, the final character look development result is generated.
The colors differ from the design, but since there is a phase of production in cel-look CG separate from design, called color design, in which the cel color scheme is designed, there are cases in which the colors after color design is completed differ somewhat from the initial design.

An Asset Adjusted in the Look Development Phase
©2018 TOHO CO.,LTD.





■Color design and changing color


In color design, work is done to create the basic colors for a character. Color is added to a line drawing of the character drawn by the character designer, gradually constructing the character's image in consultation with the director, character designer, art director, producer and others. Below is a color specification image for a character from an actual project. On the basis of this image, the colors are transferred to the CG character in the look development phase, where they are defined within the asset as the base colors.

Color Specification Image
©2018 TOHO CO.,LTD.


During color design (also called color specification), instructions are also created for changes to the color to harmonize with the background of each scene. The image below shows what color switches used in an actual project look like. Color correction is applied to characters on the basis of the background image.

Example of Color Switches by Scene
©2018 TOHO CO.,LTD.


For color switches, Polygon Pictures initially used grid-shaped image data called color palettes. In this method, an image was created in which single colors were arranged in a grid, and a shader would obtain color information from this and apply it to a CG texture. The image below was provided by the color design (color specification) phase of an actual project.

Color Palettes for Each Character
© Gamon Sakurai, KODANSHA/AJIN Production Committee. All Rights Reserved.


When we made use of the method above, there were two significant disadvantages.
• When color changes occurred, CG re-rendering would become necessary, and render time and cost would accrue.
• Since color specification is undertaken after 2DBG is created, it is not possible to render characters in the absence of 2DBG.

The above two issues reinforced the linear workflow and were major causes preventing optimization of the flow, resulting in delays to the schedule and increased workload for subsequent phases of production. To avoid these problems, we considered making adjustments to character color changes not in the look development phase, but in the later compositing phase.

Workflow



First of all, we studied the workflow used by color design staff to see what sort of color changes they were implementing via 2D image processing.
At that time we learned that, because it would be unwieldy to change each color within the palette individually, to a certain extent they were applying color correction in batches to each type of color.
Each row in the palette has a separate use. The top row is for highlights, followed by base color and shadow color, with color trace in the bottom row.
Color correction is applied to these using such effects functionalities of image processing software as hue/saturation, color balance or color plane layer composition.
(※These are simply examples of color change methods and may vary by color design artist.)

Color Correction in Photoshop



Since color changes are to be applied in compositing, we created a color change node in order to be able to do this work within compositing software (Nuke, in this example). The red node near the center of the node graph is the color change node. Within this single node we have embedded the same sort of functionality as the color correction undertaken in image processing software, so that color changes can be carried out along similar lines.

Color Specification in Nuke
©2018 TOHO CO.,LTD.


The interior of the color correction node is as in the image below. Once the image is input, the necessary image is extracted from the AOV stored within the image file. From left to right are contour lines, base colors, and masks. These are used to implement color correction.
©2018 TOHO CO.,LTD.


The image below is of the base color. When compared to the completed image, the color scheme looks a little lighter, but this is due to the fact that only the elements (areas) to which color change are to be applied have been displayed.
Examples of elements to which color changes are not applied include textures for things like injuries (things composited using multiplication or addition) or rim light information (composited by adjusting the influence of the surrounding environment on the character during compositing); such elements are not included in the image layers below.

Base Color Image
©2018 TOHO CO.,LTD.


The next image is of contour lines only. These would actually have some color applied, but since it is an extremely dark color, in the image below the alpha channel has been displayed.

Contour Line Image
©2018 TOHO CO.,LTD.


Next is the mask image. The image on the left is output from the shader as part of the contents of the AOV. This is an image in which the cel-look CG gradation area has been optionally assigned to RBGA. Color correction is not applied to each individual gradation. Two types of area mask are generated from this image for the brighter areas (highlights, base color areas) and the darker areas (1° shadows, 2° shadows).

Mask Image
©2018 TOHO CO.,LTD.


We color-correct using these elements. After overall color correction is done, it is applied first to bright areas, then to dark areas.
In order to optimize the work, the same color correction that is applied to dark areas is also applied to contour lines. The reason is that we've determined that, since the standard color of contour lines is the darkest, it is fine to apply the same color correction as the darker areas.
In this example, we've made adjustments by applying hue/saturation overall once, color balance to the bright areas once, and finally hue/saturation twice and color balance once to the dark areas.
Once these color corrections have been applied, an image such as the following is generated with the color change used in the scene.

Completed Image
©2018 TOHO CO.,LTD.


During compositing, various forms of editing are done to the rendered image, and a number of types of component image are divided up and stored within an EXR file in AOV form so that the rendered images can be reconstructed.
The output settings for these AOVs are also determined in the look development phase. In a single rendering, multiple component images are output, and we enable control of which of the output AOVs are checked, limiting them to those needed in the shot work to be done.

・Regarding EXR files (OpenEXR)
http://www.openexr.com

AOV Output for Use in Composite Work
©2018 TOHO CO.,LTD.



In this document, we have focused on an explication of points to explore during pre-production pertaining to expressive methods centering mainly on colors and textures in terms of look, but in film production, the primary objective is not the character design or look development of a single asset, but rather an ongoing refinement that takes into account balance within look development of all characters appearing in a film or show.
In this context, although we progressively construct a pipeline and workflow in terms of concept work, we also concurrently explore phases of production, such as 2DBG, which at the final stage are composited together, or the sorts of artistic touches that define limited animation. For this reason, it can be said that there is a tendency, within each phase of production that takes coordination into account, for things like test automation to take on importance as support for design processes qua teamwork. Against a backdrop of fine-tuning from various angles (working to achieve balance) within character design and proceduralization of overall work processes, explorations of levels of freedom within artwork and efficient information sharing on the production side seem likely to be future challenges within film production.

Difficulty in designing pipeline of concept work
https://a-film-production-technique-seminar.com/fppat/materials/ppi_phones_concept_work_pipeline/index.html

A review of Cel-look style basis until 2017
https://a-film-production-technique-seminar.com/fppat/materials/ppi_phones_cel_look_2017/index.html

Cel-look CG pipeline and issues
https://a-film-production-technique-seminar.com/fppat/materials/ppi_phones_cel_look_pipeline_issue/index.html


■Summary


This document has presented, as examples, characteristic areas of look development for cel-look CG, but lighting in cel-look CG is not conducted as in normal photorealistic CG; rather, the only control exercised over lights when creating imagery is over their direction. For this reason, the various settings done in the look development phase for cel-look work has an extraordinarily large impact on the final image quality.
However, methods mainly involving AOV output and subsequent compositing are built into the present pipeline, resulting in a workflow in which a single iteration of trial and error—such things as the calculation time needed for rendering output or the time taken to make myriad adjustments in compositing—and the confirmation of its result can require a considerable amount of time.
For this reason, circumstances dictate that the number of ideas that can be trialled during the production period are limited, and a host of challenges await hereafter, such as whether to create systems which will make a large number of trials possible while continuing to output AOVs, or to explore a workflow in which final images can be checked via rendering alone, eschewing AOV output.
In addition, whether it is the expansion of the level of freedom within shaders themselves, the establishment of a look development system in which it is possible to speedily incorporate color changes within color design and check final images, the question of how to implement developmental capabilities for look development ideas within a pipeline, or depictions unique to cel-look CG, a plethora of challenges are certain to face us hereafter.
One example of these things within the explanation in this document is the challenges within the creation of images within scenes by controlling areas and applying color correction.
With the methods we've presented, the fact that color correction cannot be used to change a single area to a precisely specified color, as well as the fact that the bright outlines known as color tracing cannot be controlled, present limitations to expressiveness, and this poses problems both for the expansion of the expressive power of shaders themselves, and for flexibly addressing art direction instructions. Also, due to characteristics of the software used, there are problems preventing the easy use of information from look development within the FX phase of production, so that a future challenge will be to develop seamless integration among all phases of production, allowing information to move freely, not only within the confines of the look development phase, but throughout the entire image production pipeline.
However, we believe that the look development phase will play a substantial role in meeting the difficulties and challenges of developing cel-look CG, and that it will grow increasingly important to create, through the look development phase, an image production pipeline system allowing broad visualization of all information.