Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
4ª Conferencia de Videojogos, Dezembro de 2011 Facial Skin Shading Parameterization Methodology for Rendering Emotions Metodologia de parametrização de pintura da pele facial para renderizar emoções Teresa Vieira Verónica Costa Orvalho Xenxo Alvarez Instituto de Instituto de Instituto de Telecomunicações/ Telecomunicações/ Telecomunicações Faculdade de Engenharia da Faculdade de Ciências da Universidade do Porto Universidade do Porto [email protected] [email protected] Resumo [email protected] Abstract A animação de expressões faciais de personagens 2D ou 3D requer que todas as texturas para retratar as suas emoções sejam pintadas manualmente, e de modo empírico por artistas, porque até à data não existem directrizes. O que dá às expressoes uma aparência mais artificial, uma vez que não reproduzem a perfusão sanguínea da pele, que nos faz corar ou empalidecer como quando expressamos raiva ou medo. Animating 2D or 3D character’s facial expressions requires that all textures for painting emotions are hand painted empirically by artists, as there are currently no guidelines. This makes expressions look more artificial, since they do not follow the dynamic changes of blood under the skin, which makes us blush or turn pale, like anger or fear. We propose the creation of a template methodology Propomos a criação de uma metodologia com regras sobre como misturar cores para pintar emoções. A nossa metodología analisa e compara empíricamente dados científicos e artísticos, tais como mapas de hemoglobina e melanina e retratos pintados, que reflectem a percepção visual. with rules on how to mix colors for painting emotions. Our methodology empirically compares and analyses scientific and artistic data, such as hemoglobin maps, melanin maps and painted portraits, which reflect gaze’s perception. Como resultado a industria de cinema e videojogos As a result, the film and videogame industry will gain beneficiará de personagens mais realistas, uma vez increased lifelike characters, because their skin is que a sua pele é representada como um órgão vivo, represented as a living organ, reflecting blood perfusion que reflecte os padrões de perfusão sanguínea. patterns. Palavras- chave: Texturização de Expressões Faciais Keywords: Facial Texturing, Emotions «From the earliest days, it has been the portrayal of emotions that has given the Disney Characters the illusion of life» – Thomas F. and Johnston. O. (1981). 1 4ª Conferencia de Videojogos, 2011 Introduction Facial skin color changes according to blood circulation: «When feeling shame or embarrassment, cheeks, ears, nose and forehead are blushing, and when being sick or feeling disgust or fear, the face gets pale», (Jung and Knöpfle, 2006). Although these skin color changes are not usually animated for facial expressions. Skin color portrayal is often done as a perfectly even surface, being the major emphasis placed on the muscular animation, giving the skin a less lifelike appearance (Giard, F.; Guitton, M., 2010). Scientific parameters, like hemoglobin or melanin maps, were never considered for representing skin blood perfusion when animating emotions (Image 1.c and 1.b respectively and Image 2). We know, by empirical experience as observers, that believable characters require natural looking skin and emotions. This is particularly true for human characters that must display very subtle, human-like expressions. Image 1.a) Normal color; 1.b) Melanin; 1.c) Hemoglobin. © Matts, P. (2008) State of the art Portraying human-like expressions requires high definition textures which slow down real-time rendering for interactive applications. Skin painting is a hand-made task, being done empirically, without any scientific basis. Furthermore is very time consuming: as an example, an experienced digital artist can take an estimated time of 30 hours to create texture maps for 3D animation of each one of the six basic facial emotions, as defined by Ekman (Jimenez et al, 2010). Despite the remarkable progress made in recent years (Devebec et al, 2000; Jensen et al, 2001; Borshukov and Lewis, 2003; Donner and Jensen, 2005 and 2006; Weyrich et al, 2005 and 2006; D’eon and Luebke, 2007), the traditional techniques for facial skin color representation of 3D characters, based on texture mapping, are not enough to dynamically synthesize the skin color variation when expressing emotions, unless when animated frame by frame. Some computational models were created with the purpose of trying to mimic human skin coloration. Kalra and 2 4ª Conferencia de Videojogos, 2011 Thalmann (1998) presented a model for rendering of emotion, which includes color variation during the execution of vascular emotion. However their model was not adopted for the current commercial pipelines. G. Borshukov, J. Montgomery and J. Hable (2007) achieved lifelike facial expressions animation through details capture from human characters and storing that information as animated diffuse maps. This technique requires so much data collection and processing that becomes highly demanding for real time applications. We do not want to individually capture skin color variation for each character’s animation, nor is it our purpose to reproduce mathematically skin’s complex behavior, because both are computationally expensive. Instead we want to provide a generic methodology that allows any character (2D or 3D) to be individually animated. Some research in the field was done by Jung et al, (2009), which proposed a comparative table (Table 1) of each expression and its skin color change, based on physiological knowledge and on Plutchik psycho-evolutionary theory. Jung et al (2009) table offered our methodology guidance on how emotions change skin color that can be seen on the preliminary results (Image 8). Table 1 Emotion Facial color changes Neutral Joy Enthusiasm/Ecstasy Surprise Disgust Down Sadness Grief Apprehension Fear Panic Annoyance Anger Rage Neutral face color, no changes Rosy cheeks Rosy cheeks, tears of joy Rosy cheeks Pale cheeks Low lacrimation Blushing cheeks, raised lacrimation Blushing cheeks, red blotches, intensive lacrimation Pale cheeks Pale in the whole face (sweat) Pale face, low lacrimation, sweat on the forehead Blushing cheeks Blushing cheeks, red blotches in the face Blushing cheeks, red blotches in the face, red face Skin is a multilayered and non-homogeneous structure, whose color derives from light interaction (mainly absorption and scattering) with chromophores concentration of melanin and hemoglobin perfusion, as stated by Igarashi et al. (2007). The Image 2 describes the pathways of light through the skin: part of the incident light is reflected at the surface of the skin but the remaining light penetrates into the skin layers. Image 2: Human Skin Layers CG Maps example 3 4ª Conferencia de Videojogos, 2011 Image 3: Epidermal map: semitranslucent, where light is absorbed by melanin Image 4: Subdermal map: the light is scattered multiple times by collagen fibers and absorbed by hemoglobin. Light pathways through human skin layers. © Igarashi, T. et al, 2007 CG maps. © Wade, D. (ed. by) 2007. Subsurface Scattering (hence SSS) is the best computer graphics technique for human skin simulation, since it modules the behavior of light in interaction with the skin layers. The primary light absorbers of skin are the chromophores melanin (present in the epidermis) and hemoglobin (present in the dermis). The Images 3 and 4 are some of the most important computer graphics maps for animation of skin color: the Epidermal map with gives the melanin color and the Subdermal map for representation of blood perfusion. Any change on blood perfusion should be animated through the Subdermal map for 3D skin simulation. Methodology Our study will empirically analyze and compare several data, in four different phases, namely: 1) hemoglobin maps for definition of areas and intensity of major blood perfusion (Image 5); 2) melanin maps for definition of skin basic epidermal color (Image 1.b); 3) photos of actors depicting emotions for color and expression comparison (Image 6); and 4) artists painted emotions portraits for color comparison, since they reflect the gaze’s perception, (Image 7). Melanin and hemoglobin maps were captured from in vivo subjects, using non-contact SIAscopy technology. They are important for understanding accurately skin color and blood perfusion. The next phase of our methodology is to 4 4ª Conferencia de Videojogos, 2011 collect high definition photos of human subjects depicting the six basic universal expressions as defined by Ekman – joy, disgust, anger, fear, surprise and sadness – to support our study. Table 2: maps and expressions comparison Image 5: Hemoglobin maps portraying the six basic emotions: happy, surprise, anger, disgust, fear and sadness (in order of appearance). (© Jimenez et al., 2010) Image 6: Female portraying the six basic emotions. ( © www.rafd.nl) Image 7: Painted Portraits of the six basic emotions. Image 8: Preliminary results rendered in a 3D character textured for the six basic emotions. By empirical observation we can state that the lips, because of its thin epidermis and the large content of blood on the dermis, exhibit the reddest appearance on the face. Followed by the lips, the highest blood perfusion is found on the cheeks and then the nose, the ears, the jaw and forehead. By visual comparison of the Images 5, 6 and 7 a 3D character was rendered (Image 8) having his textures hand painted following the aforementioned observations. For the six basic emotions the reddest one is the anger and the palest one is the fear (Jung et al, 2009). 5 4ª Conferencia de Videojogos, 2011 Conclusion The two main contributions of our work are: first to define skin areas most affected by color variation and the second is to design a standard methodology that will be used as a tool for helping animators to paint and animate their character’s expressions. Through the visual comparison of hemoglobin and melanin maps, photos of subjects and painted portraits depicting the six universal emotions, we propose the accurate definition of standard guidelines for painting skin’s emotions. Our template methodology can open new implementation opportunities of skin shading configuration for interactive applications. The repercussions of our work extend beyond lifelike skin, which behaves as a live organ – reflecting blood perfusion patterns – resulting in more natural emotions and engaging characters for the entertainment industry, since they reflect “the illusion of life”. Acknowledgements This research is partially supported by the European Union FP7 Inte-grated Project VERE (No. 257695), IT - Instituto de Telecomunicações and FCT - Fundação para a Ciência e Tecnologia. References Thomas, F. and Johnston, O. (1981). Illusion of Life. Disney Editions. Jung, Y. and Knöpfle, C. 2006. Dynamic aspects of real-time face-rendering. In Proceedings of the ACM symposium on Virtual reality software and technology (VRST’2006). ACM, New York, NY, 193-196. Giard, F.; Guitton, M., (2010). Beauty or realism: The dimensions of skin from cognitive sciences to computer graphics. Computers in Human Behavior, 26:1748-1752. Jimenez, J et al. (2010). A practical appearance model for dynamic facial color. In ACM SIGGRAPH Asia 2010 papers (SIGGRAPH ASIA '10). ACM, New York, NY, USA, n.141. Kalra, Prem; Thalmann, Nadia, (1998) – Modeling of vascular expressions in facial animation. MIRAlab, University of Geneva. Borshukov, G; Hable, J; Montgomery, J. Playable Universal Capture, (2007). GPU Gems 3, Addison Wesley, volume 3. Jung, Y; Weber, Christine; Keil, Jens; Franke, Tobias (2009). Real-time rendering of skin changes caused by emotions. In Ruttkay, Zsófia (ed.) et al.: Intelligent Virtual Agents: 9th International Conference, IVA 2009. Berlin; Heidelberg; New York: Springer. Igarashi, Takanori ; Nishino, K.; and Nayar, S. K. (2007). The Appearance of Human Skin: A Survey. Found. Trends. in Comput. Graph. Vis. 3, 1 (Jan. 2007), 1-95. 6