Monitor graphics in science fiction works

(Polygon Pictures Inc. / Studio Phones)

This document is prepared by the moderators of the lecture series “Craftsmanship and Artist Techniques” in concurrence with the start of the series.
Please check the details of the lecture series in the history page, which can be accessed from the following link.
https://a-film-production-technique-seminar.com/crafts/crafts_history.html
translated by PPI Translation Team

■Overview


We often handle elements like console displays for futuristic computer consoles and vehicles, or display-type user interfaces, largely in SF works. These displays are often created using motion graphics, and within animation production they are treated as and continue to develop in a phase of production called "monitor graphics".
Monitor graphics are generally created with the objective of naturally drawing viewers into the work, and of carefully drawing out the stories behind its characters and backgrounds and building up the work's worldview. Additionally, monitor graphics add information to the film that the viewer recognizes as significant to the work's historical backdrop, or as an SF element that does not exist in the present. For this reason, monitor graphics as elements play an important role as symbols that express a work's worldview.
They are also used as a dramatic expression to support transitions in the story. They are used to explain the situation in a scene, and to simply explain complicated information, like a strategy. For these reasons, monitor graphics are important for conveying a work's story to its viewers.
The demand for and importance of monitor graphics in animation production, including from a directorial point of view, has been on the rise. In recent years they have been recognized as an indispensable element in SF film works, and their importance continues to increase every year.

This document will discuss the process of producing monitor graphics within SF animation production while bringing up real examples from production. Examples will be taken from BLAME!, which was produced in Polygon Pictures, and will explain their work flow starting with the design phase in pre-production, and moving on to their actual production.
While this process does vary greatly depending on an individual work's style or background, I hope for this document to serve as a reference for the background behind and design process of a work's worldview, using the production of science fiction monitor graphics as a single example.

The moderators believe that it is necessary to reassess former artwork and to learn from the production phases that supported previous works, and consider this a large factor in the “changes in the production environment and in matters of importance.” Please consider reading the following document as well.

Changes in the production environment and changes in matters of importance
https://a-film-production-technique-seminar.com/crafts/materials/ppi_phones_production_environment_importance/index.html


■Monitor graphics workflow


In some studios, the monitor graphic workflow in animation production is known as a phase called "display", but especially in science-fiction works it is often included as part of the design phase. Plans for designs are created at a relatively early period in the overall workflow. The flow often consists of brushing up designs over multiple iterations during this phase.




Currently, display work is spread across multiple phases of production, but we think that in the future, the coordination of each of these processes will be broadly re-evaluated, along with their integration into each phase, and the graphic design phase's environment itself.


■Design review in pre-production


The base designs of all monitor graphics throughout the work, called primary designs, are created during pre-production. These graphics are created with the work's worldview in mind, and are treated as key designs that guide production. They are used as points of reference in all subsequent phases of production. Especially in science fiction works, monitor graphics that appear in UIs in a story’s mechs or displays are an element that express the work's worldview, and are often treated as important design elements.
Also, monitor graphics are most often animated, so we must consider cases in which they are animated as motion graphics. In other words, creating graphics using rigging and other set-ups that are manipulable but also balance design quality requires future work. It is important to balance the search for freedom in design while simultaneously conducting development to ensure manipulability in graphics.
Examples of Primary Designs.
©︎Tsutomu Nihei, KODANSHA/BLAME! Production Committee


These designs are important in deciding a work's background or worldview, so we believe it is important to create designs while communicating closely with the director. One role of monitor graphics is supporting drama within the story. The assortment of monitor graphic designs needs to be filled in while breaking down the story's developments, in order to decide in which parts of the story to use them in what way.
At this current stage, there is a loose framework that these design processes follow. However, this framework has not been developed enough to create a process for developing and making ideas tangible during the idea stage of design work. Certain types of systematized processes, including the in-depth search for flexibility when developing ideas, and sharing the processes used when exploring designs within a team, can be considered an area that merits further development.
Sharing Concepts with the Director and Design Ideas (sketch)
©︎Tsutomu Nihei, KODANSHA/BLAME! Production Committee


However, processes do already exist for part of the process of expanding upon a work's concepts that are in an abstract form, like developing motifs.
For example, in early stages of design, we expand upon the concept of the work while studying its background and original works, then create several motifs that match these concepts. Especially when creating works based on manga, we closely study its minutely drawn buildings and people, the shapes of objects, gimmicks like weapons, and reference materials. By digging deep for inspiration in a wide variety of sources, we are able to shape straight lines and curved lines into monitor graphics that follow the original work closely while expanding upon it.
Creating Motifs
©︎Tsutomu Nihei, KODANSHA/BLAME! Production Committee


After a certain amount of progress is made searching for motifs and establishing the ideas we generate, we combine these motifs to create elements. These elements are ultimately displayed using actual graphics and are created for specific objects. We create many variations of these elements, mostly out of motifs used frequently in the work like windows, gauges, meters, reticles, maps, vital information, etc. Doing so contributes to design work which helps dig deeper into the work's worldview.
All of these elements are generally composed by placing, scaling, and repeating motifs. However, large amounts of effort is currently spent on configuring and creating these elements. We think that in the future, a procedural approach will become more important in creating monitor graphics, even when expanding upon the ideas on which they are based.
Creating Elements



©︎Tsutomu Nihei, KODANSHA/BLAME! Production Committee


However, design processes that benefit from reusability have become more efficient by taking advantage of reusing elements.
One example of this in monitor graphics is screen-based consoles. Because these consoles appear frequently, the designs of letters and symbols that are displayed on them are often reused. Artists reuse previous designs, which are created and registered as fonts.
Sharing assets among a variety of font-related tools makes it possible to use assets in a way that pays more attention to integration in production. This means creating font assets that can be applied to the text field in software that is used to create monitor graphics, quickly use letters or designs that are assigned to each key on a keyboard, and simply edit the resulting designs. The design process demonstrates artists' sensibilities via software. We believe that in the future, it will be important to design a pipeline that pays close attention to design processes by archiving font assets. Similarly, we will need to to promote a production environment that excels in developing ideas for many genres of film, including but not limited to science fiction.

A Font we Created
©︎Tsutomu Nihei, KODANSHA/BLAME! Production Committee


Monitor graphics are largely created in production, but headway is also made during pre-production, during which their base designs are created and they are proceduralized. We set a baseline in advance, for example for patterns added for directorial reasons that help develop a work's worldview, or for methods of depicting machines specific to science fiction. By doing this, we think it will be possible in the future to simultaneously consider balancing the overarching direction of each artist's work during production, and increasing the precision of their expressions. Doing so allows us to anticipate each development in the plot, which we expect will let us adequately address more complicated plot developments in the future.

Even after pre-production, artists often look back on these ideas and reference them during production. We constantly keep in mind integrating these designs into the movie production pipeline in an effective and easy-to-reference way, and create a variety of services to facilitate this. Considering that one challenge related to the production framework in recent years has been cooperating with partner and outsource companies, one can assume that the handling of the display phase will also require further thought in the future. Challenges like these that are close to the problems faced in hand-drawn animation hint at ways to reassess how we make use of artists' skills.
Challenges in producing hand-drawn anime have been collected in the following documents, so please take a look if you are interested.

Difficulty in designing pipeline of concept work
https://a-film-production-technique-seminar.com/fppat/materials/ppi_phones_concept_work_pipeline/index.html

The possibility of cel-look pipeline and challenges to take advantage of the hand-drawn animation
https://a-film-production-technique-seminar.com/fppat/materials/ppi_phones_possibility_celook_pipeline_challenges/index.html

Recent production process of anime and difficulty of constructing production pipeline
https://a-film-production-technique-seminar.com/fppat/materials/ppi_phones_graphinica_animation_process/index.html


■Primary design for each main character


The way primary designs are created during pre-production differs depending on the work, but this next section will introduce primary designs created for BLAME!

Examples of Primary Designs for Each Character
Electrofishers
Killy
Sana-Kan
Cibo
©︎Tsutomu Nihei, KODANSHA/BLAME! Production Committee


In this movie, characters have monitors built into their vision, and each character's base designs are created to express each of their personalities.
Also, one of the key design concepts used in making this movie's monitor graphics is a "grid.” Each element is placed during the design phase to create a layout that pays attention to grid space.

Designs That Pay Attention to Grid Space
©︎Tsutomu Nihei, KODANSHA/BLAME! Production Committee


The motif image that is used is common to all characters, but in order to depict a character’s traits and emotion, we amalgamate ideas and consider various combinations. Comparatively analyzing a lot of variations and making repeated adjustments currently tends to cost artists a lot of work and time. It is believed that implementing procedural methods or, for example, constructing a support system for automatic generation from design keywords will allow more variations to be generated in a short time and also make it possible to improve the number of iterations for design planning.

The initial design during pre-production of this film, BLAME! was initiated based on the ideas of "technology development” and "a huge amount of data being processed intricately”; however, the design made it hard to see what we want to see due to reasons like overfilling the picture with an increased amount of visual information and being too concerned with the overall picture, which resulted in a lack of uniqueness in the finer details.
Having learned our lessons, in the final design, we deployed designs which kept in mind the gaps within a space instead of filling the picture with various elements.

Part (1) of the Pre-production Initial Design
©︎Tsutomu Nihei, KODANSHA/BLAME! Production Committee


As we worked on the designs, there were times that they began to resemble a game worldview.
Although the design is close to the final one, the details became complex and gave a somewhat kitschy impression, and since it is close to the UI of a game screen, there were times that the design failed to depict the fineness, precision, and delicacy that fit the worldview.
After that, we went back to the starting point, recreated the motif and elements, and then reached the final design.

Part (2) of the Pre-production Initial Design
©︎Tsutomu Nihei, KODANSHA/BLAME! Production Committee


In summary, there are a lot of cases where a design is explored by repeating the review process and sometimes even changing an idea itself, in order to make an idea concrete when planning designs in pre-production.
These design trials are done in a way that ideas are expanded from various perspectives step by step, but at present, we are considering gradually combining these design trials with automation in order to make production efficient, and we are also planning to raise the importance of "prompt idea expansion and sharing” even more in future productions.
From the perspective of a system that gives support to artwork in a production, we are also considering implementing those things as server-side services. Please refer to the documents below.

Design of server-side services to support the variance of artwork
https://a-film-production-technique-seminar.com/fppat/materials/ppi_phones_support_variance_artwork/index.html


■Working in Production


Motion graphics production work can mainly be divided into two cases: supplying materials for compositing, and adding graphics to materials for which composite work is already done or completing the image by executing video processing, etc.
Regarding the work phase timing, the former work is before compositing, and the latter is work after compositing, but there are times during the latter when the workflow is to fine tune a final picture which has been replaced after compositing. This is done by having a temporary image created in an earlier phase before compositing such as layout or animation, confirming the direction to some degree and making adjustments to it in advance.

Passing Data to Composite Phase
©︎Tsutomu Nihei, KODANSHA/BLAME! Production Committee

Combining Motion Graphics with Composited Images
©︎Tsutomu Nihei, KODANSHA/BLAME! Production Committee


In order to depict a three-dimensional display that uses 3DCG, we output locators and camera information that indicate the character’s eye position using 3DCG software, and place them to create depth by using the 3D layer functions and such in motion graphics software to create finely detailed graphics.
We import the locator (positional information) and camera information, place the created graphic at the locator, add parent-child connection and create a picture that moves together with the camera. As creating graphics with many layers using 3D objects takes time, we use a method that increases efficiency, in which animation and placements of the graphics are done in the compositing software.

Working with 3DCG software.
©︎Tsutomu Nihei, KODANSHA/BLAME! Production Committee


Noise effect animation is often seen in sci-fi shows, and depicting noise is one indispensable technique in monitor graphics.
At a glance, it is just an image that looks glitchy, but the rhythm of animation is very important for noise and we believe that the rhythm that results from the size, shape and fluctuation timing of noise gives a significant dramatic effect. The scene in the following picture shows a shot where the character appears as a hologram with noise. Our creative intention for this was to create an eerie feeling when the character stands like a ghost in front of us.
If we fluctuate it with a constant rhythm using scripting and such, the audience will eventually become aware of that systematic rhythm, and there’s a possibility that they will get bored of it, so in order to depict such creative intention, we worked on this by watching the video many times to find the best timing (discomforting, eerie, or surprising timing for this shot) and manually applied noise.

Combining Motion Graphics with Composited Images
©︎Tsutomu Nihei, KODANSHA/BLAME! Production Committee



■Overview


There are many cases in works of science fiction in which it is not enough simply to try to make monitor graphics look cool. Producing designs that correspond to the look and feel of the fictional world has the effect of helping to make the work feel convincing. Since this phase has such importance within a production, it seems safe to say that it is a phase which each studio, in its own way, is constructing via exploration of the sorts of methods and approaches this document has presented.
As we live in a time in which, as IT devices proliferate in daily life, devices presented as user interfaces have become an everyday sight, a cycle has been born in which, in each new film or show, designs which feel like fresh and new progressions from designs seen every day are expected to excite viewers with a taste of futurism. What matters here is that the designer deeply understands the work and develop designs by envisioning the designs the world implied by the work is likely to have cultivated.
Also, because these designs play an important role as storytelling or theatrical elements, just as when considering the rhythm of noise, it is necessary to present elements which do not stand out more than necessary, and which make the information the work seeks to convey in a way that is easy for viewers to grasp. By interweaving unobtrusive visuals during scenes intended to raise the intensity of the story with flamboyant graphics which stand out in their own right, the role that motion graphics play within works of science fiction of highlighting what makes that work alluring is seen as of great importance.
This is why it has become crucial to make improvements to pipelines to allow for the straightforward flow of data between each DCC, to enable frictionless testing and development of a greater variety of ideas while sustaining a sufficient number of trials. As this document has put forth, even when the single production phase of monitor graphics is examined on its own, at present a number of production-related challenges present themselves, but the need for various similar trials is not confined to monitor graphics alone, but rather exists in many phases of production, where, in order to conduct trials to exploit more flexible ideas, it seems as though a radical reevaluation of workflows and pipelines has come to be necessary.

This document has been written using production examples from BLAME! and may be somewhat biased toward the worldview or particular characteristics of that film, but we hope it might be of use as a reference during production of monitor graphics.