Character modeling in Cel-look CG with a procedural perspective

(Polygon Pictures Inc. / Studio Phones)
We plan to continue refining and adding to this document as we observe the situation in future seminars.
translated by PPI Translation Team

■Overview


In this document, as we look back on character design, which is often thought of as being difficult to handle with procedurals, and the modeling process for it, we should like to explain what comes into view by looking back mainly on the preproduction phase and the various strategies used there, including various techniques which support optimization and a structure which can be thought of as proceduralization, among the large number of processes which support character design on large-scale visual entertainment productions.

■Background


Since the word “character” can refer to anything from a photorealistic character to a cartoon or cel-look character, the fact is that, depending on differences in style, the approach to modeling them will vary. Our discussion here will presuppose a focus on modeling cel-look characters, but there are many situations in which a modeling approach conforming to certain fixed rules, as in procedural modeling, receives little attention in character production, in which what is expected is a one-of-a-kind item. The character’s quality is seen as being a product of the modeler’s incorporation of their interpretations of such nebulous elements as the character’s or creator’s essential traits or other affectations, and such things are prioritized when modeling, and in a sense, this arises from the belief, presupposed as being beyond doubt, that the only possible method for controlling quality is to do the work by hand. However, we also think it possible to say that every studio is also continually devising new techniques for optimization.
For films and television series, a great number of characters appear, including assets other than the hero assets, and during the stylization done in the design process, a kind of consistency is sought; in addition, if the design process is not systematic, it may begin to imbalance the world of the film or series, and it may be difficult to impart upon the work a framework that allows the viewers to achieve an overall concept of the appearance of the characters. Also, there are tendencies in character design that that studio takes as its specialty, then by granting structure to and optimizing the design process, it can be said to be possible to begin to expect improvements in the studio’s intrinsic design capabilities by pursuing optimization as a studio and by driving forward the studio’s predominance.

Also, various tactics related to design efficiency can also be said to be points which studios continue to devote day-by-day internal effort to improve or devise strategies for, in the sense of creating systems to provide support for trial-and-error efforts in all areas of design for character designers, directors and supervisors, as a result of various tactics for optimization in the pre-production phase. In other words, especially in the design process, in which it is said that no progress can be made without great amounts of trial and error, we are thinking that a flow exists in which, by a fixed procedure, this design process can be deployed instantly, and a great number of indices can be provided pertaining to design in the pre-production phase.

For example, even as an early-stage tactic, it is routine to design by preparing things such as a library-like approach or a system based on the same topology, and then granting a fundamental structure to trials within the design. This experimentation within design is conducted in the form of “fixed procedures” so as to be able to produce quantity while maintaining quality, and in the course of the development of film production pipelines in recent years, the optimization of each kind of experimentation within design and the procedural perspective for supporting this have gradually taken on importance.

In this document, we shall give an explanation based only on practical examples at Polygon Pictures, as well as provide an explanation of emerging possibilities for proceduralization within these examples, but we should also like this document to serve as a resource for a large number of studios as they reevaluate their own pre-production phases and explore the possibilities of proceduralization within their own character modeling.


■Character Modeling Approaches


uWhen designing photorealistic characters, because the goal is to achieve final output close to an actually existing person or object, it is possible to use 3D-scanned data as a base and then edit it, and it can also be effective to match the modeling of textures or musculature to that data as well. For example, at this point, modeling might be done while making reference to a large number of human bodies, by handling the modeling of the shapes or textures of musculature or textures through a fixed procedure, in the sense of converting these things to a model, so that it may become necessary in future to begin to consider some form of procedural system for handling these things.

Also, when designing characters like Disney’s or Pixar’s which employ effective stylization, it is important at the modeling stage to proceed with the character design study process while thinking structurally about caricaturization, and also, by proceeding structurally with look development of things like a turntable which allows the overall body shape to be reviewed, it may be possible to investigate effective forms of character design, in the sense of bringing to life traits definitive of the creator. In recent years, the number and variety of characters required in a single work have increased, and, in the sense of achieving effective communication with the client or director as well, it seems likely that it will become possible to execute these design processes as fixed procedures. In light of the fact that it will become possible to investigate, through such things as machine learning or deep learning, the systematic acquisition of feature and traits of these forms of data, the proceduraliation of study processes during this pre-production phase may possibly take on greater importance hereafter.

When we take into view not only the modeling stage but also texturing, what will become necessary to the modeling process even for cel-look depictions is some form of structuralization of character stylization or caricaturization. With stylized or caricatured characters, even more than in photo-realistic character design, a clear understanding of the stylization used in the design is crucial. This is because to grant the design a distinctiveness that is intuitively and easily grasped by the viewer serves the animation’s storytelling purpose, and the more that the work overall has been designed with a distinct design strategy, the easier it becomes for the structure of the work to be grasped by the viewer. It can naturally be assumed that, by anticipating this ease of understanding, more difficult to grasp or subtly nuanced depictions will also become easier to realize, but in the case of cel-look animation, there is a historical backdrop in which a distinctive design featuring the clear separation of lines and colors has come to be implicitly understood as a pattern by the viewer, and a tendency has developed for higher levels of completeness at the pre-production stage, taking into account interaction with things like the maquette structure stage or the the color script, and to seek balance not just for a single character, but among all characters.

The trend emerging in recent years is for these complex pre-production phases to become more prominent on large-scale overseas projects, but the more complex a phase of production becomes, the more things like the tracking of the design process or changeability and retakes in design process retakes, similarly to retakes on work within the pipeline, will become linked to the specific characteristics of the studio.
Due to these reasons, it seems that a reexamination of the design process from a procedural viewpoint – that is, managing of a fixed process – is in order, and during the seminar, we should like to hold a thorough discussion on this with the participants.


■Design Approaches


The style or feel required in a project varies greatly by project. However, the style or feel required in the pre-production process, where a repeated process of reworking ideas takes place, also sometimes tends to be only broadly defined. In addition, on a large number of projects, once a large number of designers have already been involved, information often needs to be passed on to different designers. Even in cases when an original intellectual property already exists, there are times when stylization is explored within the domain of possibility of contemporary CG technology, and, as a way of determining a design within limits that seem capable of being implemented in production, taking into account such things as the computational cost of such things as textures, there are occasions when the design adopted deviates to a degree from the original property.

However, at studios which, like Polygon Pictures, run production on a for-hire basis, when expected to reevaluate stylization or design within the contemporary expressive possibilities of CG technology, we proceed with the design process while making reference to such things as the prior works of a great number of the character designers with which we have previously been involved, or the areas in which we have concentrated our energies in prior work, with attention toward those designers’ distinguishing characteristics, habits and preferred modes of expression and, on that basis, with an awareness of a variety of elements such as the origins, visualizations, personalities and moods of the characters as specified in the character descriptions. In other words, we often repeatedly conduct studies that amount to “prototypes” which, with a view of things like that character’s behavior and patterns or forms of action, while also securing to ourselves sufficient scope to incorporate the intentions of the original creator or the director of an adaptation, allow us a degree of expressive freedom with that character within the film or series, etc.
At the stage at which a particular artistic course has begun to be settled upon, we progressively finalize things like the specifications for all characters associated with the work, bearing in mind things such as the direction the overall work will go, the details of the individual characters or the sense of density within the characters overall. Also, based on the specifications that have been brought into shape at that point, and also bearing in mind things like the production budget and schedule, we determine the minimum standard of quality, and begin to prepare reference documents regarding points that each artist associated with the production should bear in mind.

On large-scale projects, this sort of quality balance will be especially sought by the client as well, and, in the same way as retakes or tracking done at the production stage, the design process also requires the capability within the pipeline to rapidly refer to retakes, tracking and records of things considered midway through, and on that basis, when things like retakes or other modifications are performed, the same sort of design process will once again rapidly take place. We sense possibilities in proceduralization and the rapid repeatability of this design process and, at the seminar, should like to hold a discussion on the prospects that we and all the participants now foresee.


■How Things Look to the Camera


When things like anime characters are drawn in 2D, whether it is at the stage of being drawn by the concept artist or by the original manga artist, to a degree, distinctive characteristics will appear. When handling these things in CG as part of film production, we often proceed with production in a way that will allow us to maintain a level of consistency throughout the work, basing it on camera settings or indices in the models on the basis of information that we have attempted to organize while bearing in mind these characteristics, or, putting it another way, at the point in time when we construct the production environment, we proceed with system development by imparting structure on these distinctive characteristics.
For example, when the character is drawn in a front-facing composition, in our experience, at present it will generally be considered to be drawn with the eyes as the basis, and we approach production on the assumption that information on things like angle or perspective will be reconstructed from this. When the character being photographed is depicted in profile view, the camera location is often set roughly near the center of the head of the character when viewed from the side, rather than being viewed absolutely in profile at a ninety-degree angle, so on this basis, we create the overall shape from roughly an eighty-degree angle view. The fact is that we often approach work on such characters using approaches like this.

Left: Low-angle look; center: eye-level; right: top-angle look
©︎Tsutomu Nihei, KODANSHA / BLAME! Production Committee

Front, 0°, diagonal, 33°, side view, 80°
©︎Tsutomu Nihei, KODANSHA / BLAME! Production Committee

These things may vary somewhat depending on the creator’s style or habits, but another common practice is for the ears to be positioned nearer the back of the head, and in our experience, the goal of this appears to be less an accurate portrayal, and more a characteristic arising from an awareness of overall appearance or a subjective sense of how things appear.
From the point of view of aiming for realistic depictions, there may be many who find these things unnatural at times. However, when taking actual photographs of existing persons, photographers often use things like board reflectors to achieve desired light compositions, and these approaches seem to us to be quite similar in spirit, used to adjust the angle of view as an appropriately matching element of composition while keeping in mind the way it looks as an image. We continue to cultivate various distinctive characteristics which can also be said to have established the generally popularized anime look. We think it easiest to understand this as a form of stylizing or exaggerating the proportions of characters, but if the three-dimensional model is simply faithful to the drawn version, the distance from the outer edge of the eye to the ear is bizarrely wide, leaving visible traces of unnatural manipulation.

In order for the three-dimensional object to appear as intended when placed in space, it is common for the angle of the camera and its distance from the character, the way perspective will appear through the eye of the camera, and the sort of angles with which it will be used in the final imagery to play a role in a design process which understands such characteristics peculiar to 2D design, and the present situation seems to be that, to a degree, rules are worked out and information which puts these rules into order is used as a basis as modeling work commences. While this can be thought of as signifying the maintenance of consistency when working models up in three dimensions, when this is incorporated into the system for purposes of production, it is also possible to proceed with work effectively.
In other words, if it is possible to grasp the stage of imparting exaggerated representation on a shape as a fixed process, then the time may come when design is done, both for well- and poorly-proportioned exaggerated representations, in a way similar to searching for combinations of musical scales, in order to clarify the state of a well-proportioned exaggerated representation.


■Cel-look CG Modeling


Just like manga, the traditional hand-drawn anime style makes deft use of things like contour lines to express the entire shape, and then painting color into the area so delineated to give form to the overall balance; and by doing so, can be said to impart on the character, not only human characteristics, but also form.
Meanwhile, for characters created in CG, perhaps it can be said that what is done is, by defining various forms of calculation on a given shape in a three-dimensional space, necessary lines such as contour lines are generated, and thereafter each area is painted, and rendering and post-processing are done via shaders, resulting in the imparting of form.

What are raised here are two important points: lines and the painting of colors.
Regarding lines, what are often done are things like altering the shape itself to adjust them or, along with this, driving adjustments by adding post-processing, but, although things like outlines of clearly shaped silhouettes can be easy to work with expressively, but for things like clothing, whose shapes move and additionally have complications like pockets, tactics are employed like portraying these things through a combination of multiple shape data, rather than constructing them from a single-mould model.

Fine detail to impart an effective appearance has also been incorporated into the shape
©︎Tsutomu Nihei, KODANSHA / BLAME! Production Committee

Rendered image
©︎Tsutomu Nihei, KODANSHA / BLAME! Production Committee

For example, in order to depict things like creases in clothing, while a photo-realistic depiction is not expected, an method is sought to give the appearance that a crease is there, and in tandem with the circumstance that some form of expressive technique that allows the artist to work intuitively is more fitting, it is possible to understand that the search itself for a relevant intuitive expressive method will become important.
For this reason, rather than approaching this via simulation of realistic creases, we proceed with the design process by creating a maquette, exploring shapes that will look like creases, and think about expressive methods that will result in the sought-for look, pushing forward a form of stylization appropriate for the work in question.
At that time, although the artist will explore the desired look by repeatedly adjusting things like the way the edges of the shape are packed together or the bumps and hollows in the shape which depict the crease, this also increases the complexity of the modeling, and design proceeds in a form similar to arriving at the desired look after repeatedly performing trials in stylization, asking such questions as, “What sort of shape will produce folds in clothing?”

Maquette and wireframe
©︎Tsutomu Nihei, KODANSHA / BLAME! Production Committee

The idea of this can be said to be that, instead of realistic modeling, the artist proceeds in a way similar to gradually shaping a figurine.
It seems possible, by organizing the patterns used when shaping models in the design process related to this, to perform a type of proceduralization on the principle of modeling tool sets and combinations of the same.
However, there are also lines which cannot be supplemented through shapes. A technique often used at present is to portray such lines using textures. At present, approaches often employed include modeling lines using geometry in places which move a lot, such as around the eyes, and placing this geometry on the skin, thus avoiding unwanted stretching or contracting the textures due to movement, and also establishing methods for finding an artistic balance in the thickness of lines depending on their distance from the camera.

Since with textures themselves it is not possible to optimize the thickness of lines depending on the distance from the camera, and perhaps because this cannot be solved at present with an effective approach, in order not to make the lines look pasted on and appear bizarre depending on the angle, at present we limit the places to which they are added to the greatest possible degree. Put another way, if it were possible to think of a procedure which could address complex cases like this, these issues might be able to be solved, and in the event that an environment can be constructed within the pipely allowing procedures to be simply applied to such complex use cases on all enormous assets, a new and different production paradigm could be achieved.

For hair, while this depends on the style of the film or series, at present the prevailing method is, to the greatest extent possible, to create fine strands of hair during the modeling phase in such a way that a large number of lines appear. Hair work alone, done in a way based on each strand being positioned one at a time, can represent fifty to seventy percent of character modeling work, making it extraordinarily heavy work.

Model with strands of hair put in place
©︎Tsutomu Nihei, KODANSHA / BLAME! Production Committee

As we adopt methods like those above, by making sure that models are created in such a way that as many lines as possible appear, not only for hair, but also for things like clothing, we prepare ourselves to be able to stylize the final look by controlling shader settings and normals in the look development phase, as well as by using texture masks. As it cannot always be said at present that procedures for these things are being skilfully constructed and effectively used, if we are to implement an environment in which consistent procedurals are to be assigned within a project, in conjunction with an asset management system, this also may have the potential to become a new production paradigm.

Texture masks are currently used to control the normals of a shape when shadows are cast to emphasize the musculature of a body or its joints, but even at the present stage, approaches are in use which allow them to be cast or erast depending on the camera angle, and, for shadows added to a body, the fact that there are many parts with complex shapes and with a view toward their area not becoming very large, in such places they can begin to look like a pattern printed on the surface depending on the angle, so we address this with such techniques. A great number of areas continue to exist in which manual work methods like these are still used to handle such issues in production, but in the event that an environment is constructed in which these things can be cleaned up and procedure can be designed in order to be applied on a case-by-case basis, this may form a large step forward for future production.


■Data for Animation


In cel-look or cartoon character animation, it is typical for characters to be designed such that they possess functionality for handling such that the edges of the shape themselves are animated. In other words, while the design is also done simply to depict the required shape, the need to design a shape to be animated also looms large, and, although it can lead to certain differences depending on design or direction, there is also the fact that a design is not valid if it lacks functionality when being animated.
In an film production pipeline, while designing the way a shape will look, an environment is simultaneously necessary for surveying its manipulability when it is moved, so it has become increasingly typical to adopt methods of combining modules into procedures as part of the modular rigging systems which have grown more common in recent years, and to move forward the construction of environments allowing design of functionality to proceed concurrently with the design of how a thing looks.

For further details, please refer to the more detailed explanation below.

Study of areas likely to be procedural
https://a-film-production-technique-seminar.com/fppat/materials/ppi_phones_to_be_procedural/index.html
While hair may depend to a degree on design or direction, there is a tendency to proceed with modeling by first understanding the position of the hair line in the front, and then imagining how the hair will look when it moves and understanding its structure. If the roots of the strands of hair are not distributed in this way, when it is flipped upward during animation, objects not meant to be seen may end up being visible, or the roots of the hair may be insufficiently long, making it impossible to animate them. There may be differences in the motion based on the style of the film or show, or the level of importance of the character, but the basic way of thinking about these things remains essentially the same; however, due to circumstances surrounding the sorts of functionalities that studios are capable of preparing internally at present, there are still many cases in which such tactics are employed. At present we believe that it is precisely points like these for which it would be of interest to construct check functionality, as a fixed procedure, for whether these setups or other appropriate setups have been performed.

The work required on the edges of characters’ faces is both important and delicate. In particular, perhaps because the area around the eyes tends to attract the attention of the viewer, at present it is easy for faults in the modeling to appear there if the work is not done with care. The role played by each of the edges within the face is fixed as well, so, in order to maintain its consistency as an asset, adjustments are made to the model in consideration of the reusing of weights at the time of setup or the optimization of creation of target shapes.
Since there is a concentration of finely detailed parts in and around the face, work is expected to be done in a rather prudent manner; further, when the facial expression is manipulated in a broad way, the edges of things like the eyebrows and eyelashes must be aligned to the edges of the face, or else misalignment can occur between the shapes. For this reason, for the design process as well, which requires such precise work, to effectively register and use fixed processes, proceduralization may become an important perspective hereafter.

For the area around the mouth, there is a tendency to align the edges of the upper and lower lips in order to make it easier to apply weights. As a result of the fact that, when the upper and lower lips are moved in tandem, the shape of the mouth will remain clean, it is also possible to prevent unforeseen gaps from appearing. On the presumption that the character will speak during the film or show, it becomes necessary to properly structure the inside of the mouth on the assumption that it will move. At present such work is done manually, character by character, but it may be effective to provide a tool, using a procedural approach, which allows such tools to be created specifically for each character.

The role of mesh
The role of mesh
(1) A line for the wrinkle between the brows (2) A line for the side of the nostril (3) A laugh line
(4) A cheekbone indentation line (5) A face silhouette line (6) An upper eyelid indentation line
(7) A lower eyelid indentation line (8) An eyelid crease line (9) A line for the wrinkle in the outer corner of the eye

The (full-body) model that forms the base for the character will always be modeled with left-right symmetry preserved, barring any special circumstances. Objects attached to the head, such as the eyebrows, eyeballs, teeth and tongue, are likewise modeled with left-right symmetry preserved. The reason is that, if work proceeds with the model in an asymmetrical form, the adjustment of weights during the rigging phase will become inadvertently cumbersome, and will also tend to inadvertently raise the difficulty level of set-up when creating targets for the facials. If things that must be covered at each step are registered and managed as a process in this way, this not only contributes to the optimization of the time spent on modeling, but may also make design correlations between the characters throughout the film or series easier to grasp and more interesting.

For objects, like belts around the characters’ waists, which move together with the body, it is necessary to align the edges with the clothing underlying the belt, and tactics are required for things like matching the edges of parts which can be presumed to move together.
In other words, it is important to take prior consideration of things to be done later during set-up, structuring data so as to avoid causing difficulties in later phases of production. This means that the plan for this design process also requires exactly the same ideas as the plan for an film production pipeline, and we think the coordination of the procedures in each design-related process will become a challenge to address hereafter.

It can be easy to neglect things like sleeves or cuff shapes, so attention is required here, but this work involves care not only toward visible portions, but also toward the structure and data of parts not visible on the surface. Even in cases where the look is superficially good, data which causes problems when it is actually animated and moved is not suitable for practical use. For this reason, future innovation of the pre-production phase may require proceduralization of the design process for things like characters which takes into account the fact that they are to be animated.
At the seminar, we should like to conduct discussions with the participants from the point of view expressed above, but we believe there may be even more unanticipated possibilities in the future than we have touched on above for optimization via proceduralization as the design process is reevaluated, so we look forward to hearing and discussing everyone’s ideas at the seminar.