Download Facial Skin Shading Parameterization Methodology for Rendering

4ª Conferencia de Videojogos, Dezembro de 2011
Facial Skin Shading
Parameterization
Methodology for
Rendering Emotions
Metodologia de
parametrização de pintura da pele facial
para renderizar emoções
Teresa Vieira
Verónica Costa Orvalho
Xenxo Alvarez
Instituto de
Instituto de
Instituto de
Telecomunicações/
Telecomunicações/
Telecomunicações
Faculdade de Engenharia da
Faculdade de Ciências da
Universidade do Porto
Universidade do Porto
[email protected]
[email protected]
Resumo
[email protected]
Abstract
A animação de expressões faciais de personagens 2D
ou 3D requer que todas as texturas para retratar as
suas emoções sejam pintadas manualmente, e de
modo empírico por artistas, porque até à data não
existem directrizes. O que dá às expressoes uma
aparência mais artificial, uma vez que não reproduzem
a perfusão sanguínea da pele, que nos faz corar ou
empalidecer como quando expressamos raiva ou
medo.
Animating 2D or 3D character’s facial expressions
requires that all textures for painting emotions are hand
painted empirically by artists, as there are currently no
guidelines. This makes expressions look more artificial,
since they do not follow the dynamic changes of blood
under the skin, which makes us blush or turn pale, like
anger or fear.
We propose the creation of a template methodology
Propomos a criação de uma metodologia com regras
sobre como misturar cores para pintar emoções. A
nossa metodología analisa e compara empíricamente
dados científicos e artísticos, tais como mapas de
hemoglobina e melanina e retratos pintados, que
reflectem a percepção visual.
with rules on how to mix colors for painting emotions.
Our methodology empirically compares and analyses
scientific and artistic data, such as hemoglobin maps,
melanin maps and painted portraits, which reflect
gaze’s perception.
Como resultado a industria de cinema e videojogos As a result, the film and videogame industry will gain
beneficiará de personagens mais realistas, uma vez
increased lifelike characters, because their skin is
que a sua pele é representada como um órgão vivo,
represented as a living organ, reflecting blood perfusion
que reflecte os padrões de perfusão sanguínea.
patterns.
Palavras- chave: Texturização de Expressões Faciais
Keywords: Facial Texturing, Emotions
«From the earliest days, it has been the portrayal of emotions that has given the Disney
Characters the illusion of life» – Thomas F. and Johnston. O. (1981).
1
4ª Conferencia de Videojogos, 2011
Introduction
Facial skin color changes according to blood circulation: «When feeling shame or
embarrassment, cheeks, ears, nose and forehead are blushing, and when being sick or
feeling disgust or fear, the face gets pale», (Jung and Knöpfle, 2006). Although these
skin color changes are not usually animated for facial expressions. Skin color portrayal is
often done as a perfectly even surface, being the major emphasis placed on the muscular
animation, giving the skin a less lifelike appearance (Giard, F.; Guitton, M., 2010).
Scientific parameters, like hemoglobin or melanin maps, were never considered for
representing skin blood perfusion when animating emotions (Image 1.c and 1.b
respectively and Image 2). We know, by empirical experience as observers, that
believable characters require natural looking skin and emotions. This is particularly true
for human characters that must display very subtle, human-like expressions.
Image 1.a) Normal color; 1.b) Melanin; 1.c) Hemoglobin. © Matts, P. (2008)
State of the art
Portraying human-like expressions requires high definition textures which slow down
real-time rendering for interactive applications. Skin painting is a hand-made task, being
done empirically, without any scientific basis. Furthermore is very time consuming: as an
example, an experienced digital artist can take an estimated time of 30 hours to create
texture maps for 3D animation of each one of the six basic facial emotions, as defined by
Ekman (Jimenez et al, 2010). Despite the remarkable progress made in recent years
(Devebec et al, 2000; Jensen et al, 2001; Borshukov and Lewis, 2003; Donner and
Jensen, 2005 and 2006; Weyrich et al, 2005 and 2006; D’eon and Luebke, 2007), the
traditional techniques for facial skin color representation of 3D characters, based on
texture mapping, are not enough to dynamically synthesize the skin color variation when
expressing emotions, unless when animated frame by frame. Some computational models
were created with the purpose of trying to mimic human skin coloration. Kalra and
2
4ª Conferencia de Videojogos, 2011
Thalmann (1998) presented a model for rendering of emotion, which includes color
variation during the execution of vascular emotion. However their model was not adopted
for the current commercial pipelines. G. Borshukov, J. Montgomery and J. Hable (2007)
achieved lifelike facial expressions animation through details capture from human
characters and storing that information as animated diffuse maps. This technique requires
so much data collection and processing that becomes highly demanding for real time
applications. We do not want to individually capture skin color variation for each
character’s animation, nor is it our purpose to reproduce mathematically skin’s complex
behavior, because both are computationally expensive. Instead we want to provide a
generic methodology that allows any character (2D or 3D) to be individually animated.
Some research in the field was done by Jung et al, (2009), which proposed a comparative
table (Table 1) of each expression and its skin color change, based on physiological
knowledge and on Plutchik psycho-evolutionary theory. Jung et al (2009) table offered
our methodology guidance on how emotions change skin color that can be seen on the
preliminary results (Image 8).
Table 1
Emotion
Facial color changes
Neutral
Joy
Enthusiasm/Ecstasy
Surprise
Disgust
Down
Sadness
Grief
Apprehension
Fear
Panic
Annoyance
Anger
Rage
Neutral face color, no changes
Rosy cheeks
Rosy cheeks, tears of joy
Rosy cheeks
Pale cheeks
Low lacrimation
Blushing cheeks, raised lacrimation
Blushing cheeks, red blotches, intensive lacrimation
Pale cheeks
Pale in the whole face (sweat)
Pale face, low lacrimation, sweat on the forehead
Blushing cheeks
Blushing cheeks, red blotches in the face
Blushing cheeks, red blotches in the face, red face
Skin is a multilayered and non-homogeneous structure, whose color derives from light
interaction (mainly absorption and scattering) with chromophores concentration of
melanin and hemoglobin perfusion, as stated by Igarashi et al. (2007). The Image 2
describes the pathways of light through the skin: part of the incident light is reflected at
the surface of the skin but the remaining light penetrates into the skin layers.
Image 2: Human Skin Layers
CG Maps example
3
4ª Conferencia de Videojogos, 2011
Image 3: Epidermal map: semitranslucent,
where
light
is
absorbed by melanin
Image 4: Subdermal map: the
light is scattered multiple times by
collagen fibers and absorbed by
hemoglobin.
Light pathways through human skin layers. © Igarashi, T. et al, 2007
CG maps. © Wade, D. (ed. by)
2007.
Subsurface Scattering (hence SSS) is the best computer graphics technique for human
skin simulation, since it modules the behavior of light in interaction with the skin layers.
The primary light absorbers of skin are the chromophores melanin (present in the
epidermis) and hemoglobin (present in the dermis). The Images 3 and 4 are some of the
most important computer graphics maps for animation of skin color: the Epidermal map
with gives the melanin color and the Subdermal map for representation of blood
perfusion. Any change on blood perfusion should be animated through the Subdermal
map for 3D skin simulation.
Methodology
Our study will empirically analyze and compare several data, in four different phases,
namely: 1) hemoglobin maps for definition of areas and intensity of major blood
perfusion (Image 5); 2) melanin maps for definition of skin basic epidermal color (Image
1.b); 3) photos of actors depicting emotions for color and expression comparison (Image
6); and 4) artists painted emotions portraits for color comparison, since they reflect the
gaze’s perception, (Image 7). Melanin and hemoglobin maps were captured from in vivo
subjects, using non-contact SIAscopy technology. They are important for understanding
accurately skin color and blood perfusion. The next phase of our methodology is to
4
4ª Conferencia de Videojogos, 2011
collect high definition photos of human subjects depicting the six basic universal
expressions as defined by Ekman – joy, disgust, anger, fear, surprise and sadness – to
support our study.
Table 2: maps and expressions comparison
Image 5: Hemoglobin maps portraying the six basic emotions: happy, surprise, anger, disgust, fear and
sadness (in order of appearance). (© Jimenez et al., 2010)
Image 6: Female portraying the six basic emotions. ( © www.rafd.nl)
Image 7: Painted Portraits of the six basic emotions.
Image 8: Preliminary results rendered in a 3D character textured for the six basic emotions.
By empirical observation we can state that the lips, because of its thin epidermis and the
large content of blood on the dermis, exhibit the reddest appearance on the face.
Followed by the lips, the highest blood perfusion is found on the cheeks and then the
nose, the ears, the jaw and forehead. By visual comparison of the Images 5, 6 and 7 a 3D
character was rendered (Image 8) having his textures hand painted following the
aforementioned observations. For the six basic emotions the reddest one is the anger and
the palest one is the fear (Jung et al, 2009).
5
4ª Conferencia de Videojogos, 2011
Conclusion
The two main contributions of our work are: first to define skin areas most affected by
color variation and the second is to design a standard methodology that will be used as a
tool for helping animators to paint and animate their character’s expressions. Through the
visual comparison of hemoglobin and melanin maps, photos of subjects and painted
portraits depicting the six universal emotions, we propose the accurate definition of
standard guidelines for painting skin’s emotions. Our template methodology can open new
implementation opportunities of skin shading configuration for interactive applications.
The repercussions of our work extend beyond lifelike skin, which behaves as a live organ
– reflecting blood perfusion patterns – resulting in more natural emotions and engaging
characters for the entertainment industry, since they reflect “the illusion of life”.
Acknowledgements
This research is partially supported by the European Union FP7 Inte-grated Project VERE
(No. 257695), IT - Instituto de Telecomunicações and FCT - Fundação para a Ciência e
Tecnologia.
References
Thomas, F. and Johnston, O. (1981). Illusion of Life. Disney Editions.
Jung, Y. and Knöpfle, C. 2006. Dynamic aspects of real-time face-rendering. In
Proceedings of the ACM symposium on Virtual reality software and technology
(VRST’2006). ACM, New York, NY, 193-196.
Giard, F.; Guitton, M., (2010). Beauty or realism: The dimensions of skin from cognitive
sciences to computer graphics. Computers in Human Behavior, 26:1748-1752.
Jimenez, J et al. (2010). A practical appearance model for dynamic facial color. In ACM
SIGGRAPH Asia 2010 papers (SIGGRAPH ASIA '10). ACM, New York, NY, USA,
n.141.
Kalra, Prem; Thalmann, Nadia, (1998) – Modeling of vascular expressions in facial
animation. MIRAlab, University of Geneva.
Borshukov, G; Hable, J; Montgomery, J. Playable Universal Capture, (2007). GPU Gems
3, Addison Wesley, volume 3.
Jung, Y; Weber, Christine; Keil, Jens; Franke, Tobias (2009). Real-time rendering of skin
changes caused by emotions. In Ruttkay, Zsófia (ed.) et al.: Intelligent Virtual Agents: 9th
International Conference, IVA 2009. Berlin; Heidelberg; New York: Springer.
Igarashi, Takanori ; Nishino, K.; and Nayar, S. K. (2007). The Appearance of Human
Skin: A Survey. Found. Trends. in Comput. Graph. Vis. 3, 1 (Jan. 2007), 1-95.
6