Cel-look CG pipeline and issues

(Polygon Pictures Inc. / Studio Phones)
This material is written based on content discussed in previously held sessions. We plan on brushing up the content of this material and making revisions as we hold future sessions.
translated by PPI Translation Team


Cel-look CG is a style of animation that has received international acclaim among anime in Japan, and whose production is alluring in that it requires simultaneously managing elaborate craftsmanship and creativity. For this and many other reasons, the cel-look CG style of animation is currently receiving a large amount of attention in Japan.
This style originated as an attempt to recreate the look of hand-drawn cel-era animation using CG. It passed through a period in which it developed through imitation, and is expected to continue to progress due to the introduction in recent years of photorealistic CG techniques into non-photorealistic CG, of which cel-look is one particular style. Furthermore, the large question of the cel-look CG pipeline's construction in context of large-scale production and international expansion brought about many discussions during the sessions.
This document will take into consideration and make revisions to the related document A Review of Cel-Look Anime Visuals Until 2017, and then continue to reconsider future challenges particular to the construction of the cel-look pipeline, and the infrastructure requirements necessary to accomplish this.

A review of Cel-look style basis until 2017

■Cel-look from the viewpoint of pipeline construction

CG animation styles can be broadly divided into the categories photorealistic and non-photorealistic. While the photorealistic style has gone through many developments, one can say that the production of movies that widely use traditional photorealistic CG gave photorealistic techniques a stage on which they could play an important role. Compared with non-photorealistic techniques, one may be lead to believe that the extent of the latter's tricks are limited. However, non-photorealistic CG’s tricks and workarounds, which were maintained and fostered within studios, can be considered more effective in their respective environment.
Many CG techniques developed as non-photorealistic expressions were created when handling a diverse range of styles, like manga, illustration, sketches, watercolor paintings, oil paintings, ink wash paintings, and pixel art. These styles each brought distinct techniques and know-how to the studio. Looked at from another perspective, reconciling this know-how with modern techniques by constructing a practical system, including infrastructure and pipeline construction, is something we recognize as one of the major future challenges in pipeline creation and infrastructure design within animation studios.

Furthermore, many developments continue to be made in recent years in the production of CG in the anime genre, a field of media that represents Japan. These include the timing that frames are dropped in limited animation and other techniques that make CG animation look closer to hand-drawn animation. Other developments include the deformation of shapes as a result of free perspectives, and the recreation in CG of outlines as seen in hand-drawn 2D cel animation. However, various aspects of these techniques still rely on the skill of individual artists, which makes it difficult to automate and optimize in many areas. Also, when rendering, many buffer images are rendered along with the original image, which are used during compositing to create pictures using 2D-like treatments. These buffer images are all included as AOVs within the render image, leading to the use of image formats that can hold information in many layers, like exr. Of course, exr-formatted files which hold large amounts of information end up increasing in size. Disregarding demands for storage, there is still much left to be investigated in order to make production efficient by improving how progress is managed and how each phase is planned.
In the process of handling these challenges, we believe that we should not adhere to workflows or customs left as tradition from the analog era, and that ideas that boldly change the workflow are crucial.

■About calculation clusters in Cel-look

In rendering cel-look CG, the main process is generally judging the area of cels, so the amount of time spent computing shading is relatively short compared to photorealistic CG. However, many image treatments are output as buffers, meaning that large amounts of memory may be spent on these processes.
Also, when utilizing renderers that use ray tracing to draw outlines, the number of ray trace samples needs to be raised to a high value to rectify outlines flickering between frames, or to draw clean outlines, each of which extends time spent on rendering. Further, there are cases in which the renderer’s and shader's algorithms do not provide adequate support for multi thread computing, leading to problems when rendering with multi-core machines. The cel look and currently mainstream rendering techniques like path tracing and ray tracing may be considered incompatible in this sense.

■Challenges in Cel-look-specific workflow

Firstly, one process whose methods are unique to the workflow of cel-look CG is modeling. In cel-style animation, creating shapes simply by modeling based on a design image will not lead to the desired rendering results. Because the cel look is composed of cel shading and outlines, it is necessary to construct shapes during modeling so that outlines appear in their desired locations.
For example, noses are sometimes modeled to be extremely thin in order to draw outlines on the bridge of the nose. Also, it is very inefficient to render and check outlines multiple times. Therefore, it would be much more efficient if there was a system to check images in real time that are close to the final rendered image in the viewport, where work is done.

A model with a thin nose bridge.


A viewport in which shading and silhouettes can be checked in real time.


However, there are modeling techniques specific to the cel look that rely on modelers' experience, and the workflow is currently reliant upon individual artists' skill. Furthermore, minor differences exist based on each project's style, making it challenging to create tools for these types of work.

Backgrounds in cel-look CG make wide use of 2D background paintings, but cameras in 3D CG are occasionally animated dynamically, in which case background paintings need to be created to suit the camera work.

A layout diagram and background painting


Another area to be developed in the cel-look workflow is the production of background paintings. In productions like TV series, a single episodes has around three hundred shots, and even if all of these do not require a unique background, it still generally requires several hundred layout diagrams (sometimes referred to as genzu) and background paintings. Background paintings are generally hand-drawn meaning some of them may require extra time, and their creation can easily affect an entire project's schedule. It is desirable to start work on background paintings as early as possible, but it is not possible to create genzu, which are the base of background paintings, if their corresponding shot's camerawork is not finalized. For this reason, the production of background paintings always runs on a tight schedule.
Because background paintings are often created by hand, it is difficult to create tools to aid their production. However, if parts of this process were automated or optimized, it would have a positive effect on the workflow in general.

Another phase in the workflow specific to cel-look CG is color design, which has continued as a tradition from 2D cel anime. In this phase, the color scheme of each cel is specified in a form called a pallette, and these color schemes vary minutely between scenes and locations.

A color palette for a standard color scheme.


A color pallette adjusted to fit a specific scene.


The color schemes in each of these palettes are adjusted for each scene, as well as for bright and dark places within individual scenes. Hundreds of these color palettes may be created for works like TV series, so managing their data and mapping shaders tends to be a complex task.
Generally speaking, adjustments are made to an entire scene's coloring in a process called color correction. Here, changes are at times made to single colors in a palette to reflect the staff's sensibilities or for artistic reasons. Because of this, automation of the process is challenging, and the process requires time.
Endeavoring to use color palettes to decide the colors of cels in even 3D CG may itself be a method exclusive to Japan. The method, from its workflow to the development of shaders, was created through repeated trial and error.
While continuing to create cel-look CG productions in Polygon Pictures, we have constantly grappled with parts of workflows that need further development. During our sessions, we discussed how we were not currently using color palettes, rather improving our workflow and reducing costs by incorporating a system with the same function during color correction in compositing.
After the sessions were held, participants stated that, when viewed from the perspective of color balancing in a scene, it may be possible to make use of machine learning for color balancing by fitting to a specified directive color balance. Further progress is expected to be made in the future in automating artisanal workflows like these, but we believe that developing under a paradigm that is capable of handling creative direction would be most useful for the industry. We believe that every area handled in future productions will continue to follow both the pursuit of creative direction and incorporation of automation. We thought these subjects would make for interesting topics in future sessions.

■Challenges in Cel-look-pipeline infrastructure

One aspect that requires further development in cel-look CG's pipeline infrastructure is related to compositing. A large amount of image information is output during rendering, which is then put together as a picture during compositing. The amount of information added at this point varies among studios, but in many cases the necessary information is output as images for adding post effects.

Output images.


Using Polygon Picture's flow as an example, we see that these types of image information, called AOVs, output twenty to thirty different types of images in a single render. We use a file format called OpenEXR that stores AOVs in layers, but in many cases the file size of a single frame ranges from several MBs to several tens of MBs, which puts a large burden on storage.
Also, when using these images in-house, each compositing artist loads multiple layered images simultaneously from the file server in order to work on them, which puts a large burden on the storage system's I/O performance.
The image quality of future content is expected to increase continuously, meaning these types of burdens will also increase with every year. If the storage system's performance is lacking, the processing speed of artists' machines will decrease, leading to additional stress on artists and a decrease in their productivity.
While these elements currently provide us with sufficient information to create cel-look images, during the sessions, participants expressed that improvements could be made to the current workflow where these functions are performed in compositing. It may be necessary to avoid caching and storing this information in files as much as possible, and to incorporate these processes into rendering instead.
Rendering in photorealistic productions is shifting away from creating pictures using tricks in compositing, and towards spending slightly more resources when rendering to greatly reduce the amount of treatments done in compositing. Rendering in cel-look productions, a type of non-photorealistic production, currently resists this trend. Because picture creation in non-photorealistic productions relies heavily on treatments in post-production, this challenge can be considered specific to non-photorealistic productions. It is a challenge that needs to be addressed in the future, including when constructing infrastructure.

■Future challenges in Cel-look CG pipeline and issues

There are many difficult challenges that need to be addressed, like adjusting our infrastructure to support artists' intuitive use of our tools without inhibiting their work. Separately, many ideas are created during production, not just pre-production. This brings into question how to integrate pre-production servers and production servers, and how to plan their fundamental functions within the server.
There are many areas with much room left for development, like artisanal techniques that have not been completely recreated in CG, as well as ones whose applications are too costly to support. Topics like UI/UX, scheduling for the render cluster, and the high-end integration of frequent access storage each require further development. Technological developments within our infrastructure are expected to increase the amount of ideas artists are able to develop, so we expect technology to develop fiercely in the future.

The production of anime in countries outside of Japan, including in North America and Asia has increased dramatically in recent years. As a result, the number of new styles particular to individual countries has started to increase. Even shader languages like OSL which were developed to work with ray tracing tend to take rasterization-era techniques into consideration. This trend is now becoming gradually apparent, and the moderator of this session believes that in order to unite the cel look style and hand-drawn anime techniques, a reform at the level of infrastructure must occur simultaneously with developments in renderers themselves.

During our sessions, Studio Phones stated that they adopted rasterization-era techniques with ray tracers as a part of expanding on OSL. However, using traditional ray tracers required many calculations using unfamiliar settings, and they had to make many adjustments to the infrastructure in order to be able to flexibly send tasks to the render cluster. Even if the scope of our sessions is limited to our two companies, by comparing each of our company's trends, we were able to visualize the development of a diverse number of techniques. Discussions were held about the infrastructure for rendering and post-production, which included far-reaching talks about the technical aspects of infrastructure, for example IO demands when referencing on the asset server.