2.5/3D background workflow in large-scale animation production

(Polygon Pictures Inc.)
This material is described in a form supplementing the content of the lecture by Polygon Pictures from the meeting while summarizing the contents.
translated by PPI Translation Team

■Overview


This document summarizes the content of the in-session seminar "2.5/3D background workflow in large-scale animation production (Polygon Pictures Inc.)". It will explain how Polygon Pictures deals with the creation of matte paintings and background paintings in large-scale productions.
We hope this information is of benefit to related parties as no more than a single example of CG animation production.


■Implementing Matte Paintings


The scale of productions in Polygon Pictures has increased and our in-company workflow and pipeline system changed dramatically since we started production of large-scale international TV series around 2010. When producing large-scale series, it is especially important to satisfy clients' expectations for high quality while simultaneously improving productivity in order to meet strict deadlines and produce large quantities of animation consistently. In order to achieve these results, we developed our production techniques and systems through trial and error.
One of the problems we grappled with when creating TV series was producing large quantities of backgrounds. We introduced matte paintings as backgrounds in order to strike a balance between quality and efficiency.
A single episode in a TV series is made up of around 300-350 shots, so producing a standard 26 episode series equates to the length of 6 movies. Creating all of the backgrounds used in a series as 3D CG assets would require large amounts of time and man-days.
We introduced matte paintings for the mid and backgrounds while continuing to create 3D assets for the foreground. By doing so, we established a workflow that allows us to produce backgrounds more efficiently than by using 3D assets. We aimed to increase productivity by introducing this workflow to our pipeline. In order to ultimately composite 2D background paintings using projection mapping (camera mapping), a degree of restraint must be placed on the camera. Next, we would like to use a scenario in which animation from a 3D nodal pan is added to the camera as an example to introduce the production process.
First, using a camera in a 3D space that has a nodal pan animation added to it, we use Lat Long Lens Shader, set the area necessary for the matte painting, and begin rendering.

a) Camera and lens shader in a 3D space


A matte painting layout diagram is created using the rendered image which contains direction for the matte painting artists.

b) Rendered image and layout diagram

The matte painting is created using the layout diagram as a reference.

c) A finished matte painting


Compositing is done in compositing software like Nuke, etc., using projection mapping on the matte painting.

d) Compositing with projection mapping

The entire workflow is supported by the production pipeline. Sending and receiving data is handled smoothly, and multiple support tools are being developed to promote automation.
By making use of matte paintings, it became possible to reduce the cost of asset production. On the other hand, doing so required a separate phase called layout diagramming (referred to colloquially in-house by its Japanese name, “genzu”). Adding this function created drawbacks including requiring additional resources for the new phase, and further complicating the workflow.


■Producing 2.5 Backgrounds and Turning Matte Paintings into Assets


When producing a series, it is inefficient to create backgrounds on a shot-by-shot basis for the same location. For this reason, in the past we created frequently used backgrounds beforehand using Nuke so they can be used in any shot in a given sequence. These assets were managed as general-purpose assets.
Middle and far distance matte paintings are particularly likely candidates for re-use. These would often be turned into assets in their projection-mapped states for re-use in multiple shots. Also, turning matte paintings into assets helped alleviate schedule issues for supplying background data to the lighting and compositing phases, which improved the production schedule in general.


■Utilizing Background Paintings in Cel-look CG


Since 2013, we began utilizing matte paintings in our background creation workflow when producing domestic animation as well. In order to create a look more similar to hand-drawn anime, most backgrounds are created using 2D background paintings. 2D background paintings are also used in Cel-look CG anime works. We were able to use background paintings for domestic Cel-look CG productions by applying the production methods which we created through trial and error and refined over time.
Most of our overseas projects use 3D CG to create foregrounds. However, in works that recreate the style of hand-drawn animation, we often create all background elements as 2D background paintings, including the foreground, in order to create a more anime-like look.

d) Background paintings in Cel-look CG
©︎TSUTOMU NIHEI・KODANSHA／KOS PRODUCTION COMMITTEE

In the production of TV series - the most common format for domestic Cel-look CG productions - comparatively tight schedule management is required, and in many cases must be carried out in cooperation with branch and outsource studios. Because of this, many in-house discussions about efficiency and revisions to pipeline design in Cel-look CG productions are held among the upcoming project’s staff members before projects begin.
It can take up to ten times as many man-days to create complicated layout diagrams compared to simple ones. Because of this, we have animators and layout artists examine camera work, evaluate the parallax and necessary work for creating matte paintings, and control the methods employed in production in order to avoid incurring excessive amounts of work.
Attention must be payed to creating layout diagrams and matte paintings during layout and other preceding steps in order to work efficiently. In previous cases that required excessive amounts of work, production methods were changed, for example changing 3D camera work to 2D camera work.
We took measures to maintain productivity, like setting parameters for the ratio of 2D and 3D camera work in each episode, and simultaneously tracking whether those parameters were being followed on average.


■Developing Tools to Increase Productivity (Maya)


This tool takes the animated camera in Maya, automatically calculates the necessary resolution for a specified area of background painting(s), and sets the resolution for the rendered images.
It can efficiently fit images to a specified resolution and reduce unnecessary rendering for unused areas of images.

a) Automatically calculated resolution

This tool automatically calculates the rendering area and necessary resolution when using a camera technique called a 3D nodal pan.
When using 3D camera techniques, the area of the film gate changes dynamically based on the area of the animation. This tool automatically calculates the necessary rendering resolution based on each of these areas.

b) Drawing guide for film gate range and automatically calculated resolution

This tool is used when re-using background paintings that were turned into assets in multiple shots. It calculates the area of the background from the camera in all necessary shots, automatically calculates the range that must be prepared for background paintings, and also determines their resolution.

c) Setup for reuse across multiple shots

■Developing Tools to Increase Productivity (Photoshop)


This tool automatically generates a PSD file from data published in the pipeline and arranges the file into a specified layer structure.
Data split into “book” layers can also be constructed automatically, which allows for much more efficient organization of layout diagrams. Before this tool was created, organizing layout diagrams took several days to complete, and now can be finished in several hours.

a) Tool to automatically create layout diagram PSDs.


By using our in-house review system to handle layout diagrams, background painting checks, and publishing in the pipeline, we were able to improve the often cumbersome management of layout diagram and background painting data, and make their check flow much more efficient.

b) Integration to our in-house review system
© Gamon Sakurai, KODANSHA/AJIN Production Committee. All Rights Reserved



■Developing Tools to Increase Productivity (Nuke)


This tool supports projection mapping in Nuke. While the layout diagram is created, this tool automatically generates the Nuke file for projection mapping that is used during compositing.
The tool recreates a projection that works with a 3D camera simply by loading a finished background painting. It makes it possible to immediately start compositing in Nuke.

a) Automatically generated nk file for projection

■Challenges Caused by Creating Backgrounds Using 3D Assets


In order to maintain the balance between productivity and quality for Cel-look CG production we often use matte paintings for 2D backgrounds. We plan to continue to utilize these in the future, but in recent years, we have also made progress in the creation of 3D CG assets for backgrounds. By utilizing 3D assets, we expect to be able to give more freedom to camera work and simplify our workflow. Interest in using 3D assets smartly in production has been on the rise recently.
However, creating 3D assets requires large amounts of time and work and requires the simultaneous development of new technologies like procedural modeling, drawing with 3D paint, and the use of real time CG techniques. The creation of 3D assets is a future endeavor for background production which we plan on adapting soon.
We also plan on developing a more stable in-house system that is capable of handling large-scale international projects. We hope to do so by automatically generating system reports through our internal infrastructure that detail how many human resources and computational resources each phase uses, and integrating check tasks from each phase within the infrastructure (which has become important in recent years).

Film production pipeline and computing resources
https://a-film-production-technique-seminar.com/fppat/materials/ppi_phones_compute_resource/index.html

Challenges of automatic test environment in film production infrastructure
https://a-film-production-technique-seminar.com/fppat/materials/ppi_phones_automatic_test/index.html

Expansion of check scope of infrastructure management
https://a-film-production-technique-seminar.com/fppat/materials/ppi_phones_infra_check_scope/index.html


■Anime Style and the Importance of Pipeline System Structure


In the Cel-look CG workflow, most production work is done digitally, but it is also necessary to incorporate into the workflow parts of traditional animation production, like background paintings. Some of these workflows, like those used for hand-drawn animation techniques, are difficult to incorporate into the CG workflow that we developed by accumulating knowledge and by optimizing over time. There are also many cases in which work is sent to outsource studios instead of being done in-house. In these cases, work is done across multiple studios, making it difficult to include in the pipeline system.
However, there are cases in which in which a predetermined schedule and productivity rate have to be followed closely, as represented by large-scale international productions. In cases where partner studios must cooperate in tasks like tracking production, understanding progress, and creating reports quickly, every phase must be included in the pipeline system.
There are challenges that occur when incorporating traditional anime production methods and workflows into a CG-centric workflow. We place high value on our artists' expressive abilities using the anime production methods which we have cultivated over the years. However, the importance of maintaining infrastructure and making additions to the pipeline system is on the rise, and Polygon Pictures continues to make adjustments and look for solutions on a daily basis.
The construction of a pipeline system and the furthering thereof requires the detailed calculation of development-related costs and price-performance comparisons, mid- and long-term plans, and in-company approval and decision-making. Because of this, it may naturally become a precondition that all members who work on infrastructure construction and pipeline development to be employed as in-house staff. Constructing in-house systems through repetitions of daily trial-and-error should foster a studio's distinctive traits as well.

Diversity of video production pipeline and infrastructure technology
https://a-film-production-technique-seminar.com/fppat/materials/ppi_phones_diif_design/index.html

Film production pipeline and system design
https://a-film-production-technique-seminar.com/fppat/materials/ppi_phones_pipeline_system_design/index.html

Continue to build in-house system
https://a-film-production-technique-seminar.com/fppat/materials/ppi_phones_build_inhouse_system/index.html