Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
“report” — 2005/7/31 — 20:15 — page — #1 Implementing a visualization tool for myocardial strain tensors Master’s thesis by Anders Rönnbrant LiTH-IMT/BMS20-EX–05/403–SE 31st July 2005 “report” — 2005/7/31 — 20:15 — page — #2 “report” — 2005/7/31 — 20:15 — page i — #3 Implementing a visualization tool for myocardial strain tensors Master’s thesis performed at Biomedical modelling and simulation, Department of Biomedical engineering at Linköpings universitet by Anders Rönnbrant LiTH-IMT/BMS20-EX–05/403–SE Supervisor: Katarina Kindberg Dept. of Biomedical Engineering, Linköpings universitet Examiner: Prof. Matts Karlsson Dept. of Biomedical Engineering, Linköpings universitet Linköping, 31st July 2005 “report” — 2005/7/31 — 20:15 — page ii — #4 “report” — 2005/7/31 — 20:15 — page iii — #5 Avdelning, Institution Division, Department Datum Date Linköpings universitet Dept. of Biomedical Engineering University Hospital 581 85 Linköping Språk 31st July 2005 Rapporttyp Report category ISBN Language Svenska/Swedish Licentiatavhandling ISRN × Engelska/English × Examensarbete C-uppsats D-uppsats — LITH-IMT-EX-BMS20-EX–05/403–SE-2005 Serietitel och serienummer Title of series, numbering ISSN — Övrig rapport URL för elektronisk version http://www.imt.liu.se http://www.ep.liu.se/exjobb/imt/2005/403/ Titel Implementering av ett visualiseringsverktyg för töjningstensorer i myokardium Title Implementing a visualization tool for myocardial strain tensors Författare Author Anders Rönnbrant Sammanfattning Abstract The heart is a complex three-dimensional structure with mechanical properties that are inhomogeneous, non-linear, time-variant and anisotropic. These properties affect major physiological factors within the heart, such as the pumping performance of the ventricles, the oxygen demand in the tissue and the distribution of coronary blood flow. During the cardiac cycle the heart muscle tissue is deformed as a consequence of the active contraction of the muscle fibers and their relaxation respectively. A mapping of this deformation would give increased understanding of the mechanical properties of the heart. The deformation induces strain and stress in the tissue which are both mechanical properties and can be described with a mathematical tensor object. The aim of this master’s thesis is to develop a visualization tool for the strain tensor objects that can aid a user to see and/or understand various differences between different hearts and spatial and temporal differences within the same heart. Preferably should the tool be general enough for use with different types of data. Nyckelord Keywords cardiac strain; deformation; glyph; interactive; tensor; visualization; Coin3d; Open Inventor “report” — 2005/7/31 — 20:15 — page iv — #6 “report” — 2005/7/31 — 20:15 — page v — #7 Abstract The heart is a complex three-dimensional structure with mechanical properties that are inhomogeneous, non-linear, time-variant and anisotropic. These properties affect major physiological factors within the heart, such as the pumping performance of the ventricles, the oxygen demand in the tissue and the distribution of coronary blood flow. During the cardiac cycle the heart muscle tissue is deformed as a consequence of the active contraction of the muscle fibers and their relaxation respectively. A mapping of this deformation would give increased understanding of the mechanical properties of the heart. The deformation induces strain and stress in the tissue which are both mechanical properties and can be described with a mathematical tensor object. The aim of this master’s thesis is to develop a visualization tool for the strain tensor objects that can aid a user to see and/or understand various differences between different hearts and spatial and temporal differences within the same heart. Preferably should the tool be general enough for use with different types of data. Keywords: cardiac strain; deformation; glyph; interactive; tensor; visualization; Coin3d; Open Inventor v “report” — 2005/7/31 — 20:15 — page vi — #8 Preface This thesis was written between late autumn of 2004 and spring 2005 and concludes my studies at the M.Sc. programme in Applied Physics and Electrical Engineering at Linköpings universitet. Most of the work was done in the north west corner of house G at campus Valla. Most strain and effort were spent in my living quarters the weeks prior to finishing. Acknowledgment First and foremost I would like to thank my supervisor Katarina Kindberg for the excellent guiding through the thesis work and for her patience and acceptance with my somewhat optimistic time-table concerning certain parts of the thesis work. Likewise, I would like to thank my examinator Matts Karlsson for his shearful1 and enthusiastic manner regarding many things and also for his comments and idea’s regarding the thesis work. I also want to thank the personnel at the National Supercomputer Center (NSC) at Linköpings universitet, located in house G at campus Valla, for housing me during my thesis work and for creating a tender and pleasant environment not always suitable for thesis writing. A special thanks could be sent to Lennarts Game Cube for its flawless ability to produce entertainment, spoiling hundreds of valuable work hours. Last but not least my gratitude goes to all those people around me, too many to be mentioned by name but too few to be left out, for their unconscious support and encouragement which I know affected me on a subconscious level. Anders Rönnbrant, Linköping 15th June 2005 1 The misspelling is noted but intentional vi “report” — 2005/7/31 — 20:15 — page vii — #9 Symbols x, X d CLR c clr c fsn δ Boldface letters are used for vectors and matrices. Lower-case letters are used for vectors and capital letters for matrices. Heart coordinate system (left ventricle). Bead coordinate system. Fiber coordinate system. 1, i = j The Kronecker delta. δi j = 0, i 6= j Glossary Affine transformation A special class of transformation that preserves lines and parallelism. Points on a line are still points on a straight line after deformation. Check valve A mechanical device that forces gas or liquid in a system to flow in only one direction. Collagen Connective tissue. Diastole The relaxation phase of the heart’s pumping cycle. Geons Simple forms used for constructing more complex ones. Index gymnastics Technique for extracting the content from geometric (tensor) equations by working in component notation and rearranging indices [12]. Systole The contraction phase of the heart’s pumping cycle. vii “report” — 2005/7/31 — 20:15 — page viii — #10 viii “report” — 2005/7/31 — 20:15 — page ix — #11 Contents Abstract v Preface and Acknowledgment vi Symbols and Glossary vii 1 Introduction 1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Problem description . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1 1 2 2 Anatomy of the heart 2.1 Anatomy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.1 The myocardium . . . . . . . . . . . . . . . . . . . . . . 2.2 Fiber sheets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 3 5 5 3 Strain and tensors 3.1 Strain and deformation . . . . . . . . . . . . . . 3.2 The Tensor . . . . . . . . . . . . . . . . . . . . 3.2.1 Eigenvector-eigenvalue decomposition . . 3.2.2 Symmetric-antisymmetric decomposition 3.3 Strain tensors . . . . . . . . . . . . . . . . . . . 4 Data and data acqusition 4.1 Data acquisition . . . . . . . . . . 4.1.1 MRI . . . . . . . . . . . . 4.1.2 Ultrasonic . . . . . . . . . 4.1.3 Marker tracking . . . . . . 4.2 Strain calculation . . . . . . . . . 4.2.1 Coordinate transformation 4.2.2 Calculation method . . . . 4.2.3 Synthetic data . . . . . . . ix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 9 10 10 10 11 . . . . . . . . 15 15 15 16 16 17 17 19 20 “report” — 2005/7/31 — 20:15 — page x — #12 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 25 28 30 31 32 32 33 34 35 Tensview 6.1 Tensview . . . . . . . . . . . . . . . 6.1.1 Work flow . . . . . . . . . . . 6.2 Implementation details . . . . . . . . 6.2.1 Data files . . . . . . . . . . . 6.2.2 Time series . . . . . . . . . . 6.2.3 The Inventor file . . . . . . . 6.2.4 Glyph’s . . . . . . . . . . . . 6.2.5 Deformation and coloring . . 6.3 Usage . . . . . . . . . . . . . . . . . 6.3.1 Arguments and options . . . . 6.3.2 Mouse and keyboard usage . . 6.3.3 Camera interaction mode . . . 6.3.4 Scene graph interaction mode 6.3.5 The input file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 37 37 38 40 40 41 42 42 42 42 43 43 44 44 Future work 7.1 Software improvements . . . . . . . . . . . . . . . . . . . . . . . 7.2 Other applications . . . . . . . . . . . . . . . . . . . . . . . . . . 47 47 48 A Tensview A.1 Details about key commands . . . . . . . . . . . . . . . . . . . . A.2 Source code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 51 53 B IV-file viewer and example B.1 IV-file viewer source code . . . . . . . . . . . . . . . . . . . . . B.2 IV-file advanced example . . . . . . . . . . . . . . . . . . . . . . 55 55 56 C Compact disc 59 6 7 Visualization 5.1 Homogeneous coordinates . . 5.2 Techniques . . . . . . . . . . 5.2.1 Other ideas . . . . . . 5.3 Available tools . . . . . . . . 5.3.1 Ensight . . . . . . . . 5.3.2 Matlab . . . . . . . . 5.3.3 OpenDX . . . . . . . 5.3.4 Open Inventor/Coin 3D 5.3.5 Why Coin? . . . . . . x . . . . . . . . . . . . . . . . . . . . . . . . . . . “report” — 2005/7/31 — 20:15 — page xi — #13 List of Tables 6.1 6.2 6.3 Command line options. . . . . . . . . . . . . . . . . . . . . . . . Camera interaction controls. . . . . . . . . . . . . . . . . . . . . Scene graph interaction keyboard controls. . . . . . . . . . . . . . xi 43 44 45 “report” — 2005/7/31 — 20:15 — page xii — #14 xii “report” — 2005/7/31 — 20:15 — page xiii — #15 List of Figures 2.1 2.2 Illustration showing the main parts of the mammalian heart. . . . . An illustration of the laminar fiber sheets within the myocardium. 4 7 3.1 Transformation of one neighborhood into another. . . . . . . . . . 11 4.1 4.2 4.3 4.4 4.5 Position of the beads and markers . . . . . Illustration of the bead array . . . . . . . . Synthetic bead positions before deformation Synthetic bead positions after deformation . Result from synthetic calculations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 18 22 22 23 5.1 5.2 5.3 5.4 5.5 5.6 5.7 5.8 5.9 5.10 Antisymmetric and symmetric shearing on a rectangle . Example of shearing in 3D. . . . . . . . . . . . . . . . Three different 3D glyph’s . . . . . . . . . . . . . . . Example showing common region and connectedness. . Morphing cone . . . . . . . . . . . . . . . . . . . . . The Müller-Lyon and relative size illusions . . . . . . A simple OpenDX network . . . . . . . . . . . . . . . Screen-shot from OpenDX result . . . . . . . . . . . . Inventor file example, three shapes . . . . . . . . . . . Inventor file example, two cylinders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 29 30 31 31 32 33 34 36 36 6.1 6.2 6.3 Data flow diagram . . . . . . . . . . . . . . . . . . . . . . . . . . Example of an input file. . . . . . . . . . . . . . . . . . . . . . . Tensview undeformed/deformed screen-shots . . . . . . . . . . . 39 45 46 xiii . . . . . . . . . . . . . . . . . . . . . . . . . “report” — 2005/7/31 — 20:15 — page xiv — #16 xiv “report” — 2005/7/31 — 20:15 — page 1 — #17 Chapter 1 Introduction 1.1 Background The heart is an intrinsic part of the mammalian creature. As a result of the biological evolution it has become a complex three-dimensional structure with mechanical properties that are inhomogeneous, non-linear, time-variant and anisotropic. These properties affect major physiological factors within the heart, such as the pumping performance of the ventricles, the oxygen demand in the tissue and the distribution of coronary blood flow. Further, malfunction in these physiological factors are the source of various heart diseases, for example ischemia and infarct. During the cardiac cycle the heart muscle tissue (myocardium) is deformed as a consequence of the active contraction of the muscle fibers and their relaxation respectively. A mapping of this deformation would give increased understanding of the mechanical properties of the heart and the heart’s pumping function. The strain induced in the myocardium during the heart cycle is complex and closely related to the muscle fiber orientation, the irregular shape of the left ventricle and local differences in ventricular radii of curvature and wall thickness. 1.2 Problem description When calculating strain in three dimensions the use of tensors is extensive. Since tensors are collections of numbers it is hard do integrate them mentally by only examining the numbers or making 2D plots. To get the right “feeling” for what is happening it is necessary to examine the tensors in another way. The aim of this master’s thesis is to develop a visualization tool for the strain tensor objects that can aid a user to see and/or understand various differences between different hearts and spatial and temporal differences within the same heart. Preferably should the tool be general enough for use with different types of data. Rönnbrant, 2005. 1 “report” — 2005/7/31 — 20:15 — page 2 — #18 2 1.3 Chapter 1. Introduction Overview Chapter 2 of this thesis gives a brief description of the anatomy of the heart. In chapter 3 a definition of strain will be presented together with the tensor object. A deduction of the formulas for strain calculations will be performed at the end. The different types of data acquisition are presented in chapter 4 together with the data used in this thesis and the method used for extracting the strain from it. Also, a synthetic data set is presented used for verification of the method used. Chapter 5 contains techniques for visualization and the connection they have with the tensor, presentation of the currently available tools and the argumentation about their pros and cons. In chapter 6 the presentation of the software Tensview is made followed by some of the internal details. Future work is discussed in the seventh and last chapter. There is also a CD enclosed with the thesis which contains source code, example data, screen-shots and movies plus related material. A full listing of the content is presented in appendix C. “report” — 2005/7/31 — 20:15 — page 3 — #19 Chapter 2 Anatomy of the heart In this chapter a brief description of the heart’s anatomy and functions are made with focus on describing the special structure of the myocardium which is the bulk material in the heart’s musculature and the tissue that performs the contractions and relaxations. 2.1 Anatomy The heart is a muscular organ with a size little larger than a fist. In contrary to skeletal muscles it is not consciously controlled, it contracts and relaxes as a consequence of auto-rhythmic cells. Primarily the heart is built up by four pumping chambers and connecting artery and veins leading blood to and from the body and a smaller circulatory system which runs back and forth through the lungs. The four chambers are the left and right atria and ventricles. The left and right atria receives blood that returns to the heart from the body and the lungs respectively. The ventricles pumps blood away from the heart. When oxygen depleted blood returns from the body it first enters the right atrium via the superior and inferior vena cava. The superior vena cava returns blood coming from the upper parts of the body and inferior vena cava returns the blood returning from the lower parts, the legs and from lower parts of the torso. From the right atrium the blood flows through the tricuspid valve and into the right ventricle. There are two valves separating the atria from the ventricles, the tricuspid valve and the mitral valve, where the mitral valve separates the left atrium and ventricle. The valves have the function of preventing blood from re-entering the atria during the contraction phase of the ventricles. The contraction of the right ventricle during the systolic phase of the heart’s pumping cycle pushes the blood through the pulmonary valve and into the pulmonary artery which leads to the lungs where the blood will be re-oxygenated. The pulmonary valve has the same function as the tricuspid and mitral valves Rönnbrant, 2005. 3 “report” — 2005/7/31 — 20:15 — page 4 — #20 4 Chapter 2. Anatomy of the heart which is to prevent blood from flowing backward in the system, like check valves in a mechanical system. Returning from the lungs through the pulmonary vein the oxygenated blood ends up in the left atrium which empties into the left ventricle through the mitral valve. During the contraction phase when the left ventricle contracts the blood is pushed through the aortic valve, which is the fourth valve of the heart, into the aorta and from the aorta the blood continues to the rest of the body via the circulatory system. The left ventricular wall is the thickest wall in the heart since it is the left ventricle which has to push the blood into the rest of the body. Figure 2.1: Illustration showing the main parts of the mammalian heart. Between the two ventricles lies the septum, a fairly thick wall separating the two stronger pumping chambers and which also has an important role in propagating the electrical signal which makes the heart contract and relax. There are three important points in the heart for the signal propagation mechanism, the sinoatrial node (SA node), atrioventricular node (AV node) and the apex area located at the bottom of the left ventricle, in the meeting point of septum and the left ventricular wall. The SA node is located on the right atrium wall inferior to the superior vena cava and the AV node to the upper right of septum. From the apex area the signal is propagated through the purkinje fibers to and along the ventricular walls. “report” — 2005/7/31 — 20:15 — page 5 — #21 2.2. Fiber sheets 5 The heart walls are divided into three layers, the epicardium which covers the outside, the myocardium in the middle which is the pumping tissue of the wall and the endocardium that lines the inside. The epicardium and endocardium consist primarily of collagen and are fairly thin compared to the myocardium. Since the heart contracts and relaxes continuously and mainly consists of the myocardium muscle tissue it also needs a constant supply of oxygen and nutrients. The epicardium that covers the inside of the chambers does not let any oxygenated blood into the myocardium why the heart does not get its supply of blood that way. Instead the heart has coronary arteries which supplies the heart’s muscle tissue with blood. There are two coronary arteries, left and right, which emerges from the aorta at the top of the heart. They stretch along the sides of each respective ventricle and branches into smaller and smaller vessels that penetrates the tissue having become small enough. Failure in these vessels of some kind that prevents the normal and/or demanded blood flow causes damage to the myocardium and is referred to as an infarct or ischemia. [18, 21]. 2.1.1 The myocardium The heart’s muscle tissue differs from the type found in the skeletal muscles and is referred to as myocardium. The myocardium is built up by small cells called myocytes. Skeletal muscle fibers can be up to a meter in length while the length of a single myocyte only reaches some hundred µ-meters with a typical length of 30 − 50µm. What also differs the myocytes from the skeletal muscle cells are that they are mono-nuclear cells with a highly irregular shape. Skeletal muscle fibers have more the appearance of real fibers or strings since they can be up to a meter in length, they also have multiple nuclei. The myocytes connect to each other through small tentacles called intercalated discs in a somewhat irregular way, forming a anastomosing network with spongy appearance, instead of the more deterministic connectivity found in skeletal muscle tissue. The capillaries transporting blood to the tissue runs through this spongy network of cells. This spongy appearance is not completely random though, there are distinct directional features at any point in the myocardium especially tangential to the cardiac wall [13]. 2.2 Fiber sheets The directional features in the spongy network are of great interest in todays research on how the myocardium works since a fiber structure within the heart muscle would give some answers to how the contracting and relaxing forces are generated. Fibers are only affective in one direction and the complex movements of the heart muscle cannot be explained with the same simple contracting/relaxing solution used by for example the skeletal biceps muscle, which is attached to bone at both ends. “report” — 2005/7/31 — 20:15 — page 6 — #22 6 Chapter 2. Anatomy of the heart It has been found that the left ventricle (LV) muscle tissue not only has a single varying fiber direction, but also is organized into laminar sheets, approximately 4 ± 2 cells thick. These sheets play a crucial role in the LV wall thickening and blood ejecting process. As mentioned above fibers can only contract in one direction, and the amount of contraction possible is approximately 15% along the fiber axis. Ejected volume fractions and systolic wall thickening of 60% and 40% respectively have been observed and only the volume fractions can be explained solely by the fiber shortening. [5, 8, 11]. The direction of the fiber structure varies depending on the location in the heart muscle but is considered to have a smooth transition from epicardium to endocardium. The typical value is about −60◦ at the epicardium to ∼ 70◦ at the endocardium from the circumferential axis ( Ĉ in figure 2.2) around the heart [13]. This angle is referred to as the α-angle. Further, the orientation of the sheets are described by adding a second angle, β, which specifies the rotation around the fiber axis in the fiber sheet coordinate system defined by the laminar sheets. Simply put, the β-angle folds the sheets like an accordion along the radial axis, R̂ in figure 2.2. This angle does not vary smoothly from epicardium to endocardium, instead it has a more complex pattern and a complete map of this angle does not exist today. Also, two families of βangles can be detected in the LV myocardium. A more in depth description about the laminar fiber sheets can be found in [5, 8]. Figure 2.2 illustrates how the fibers are located in the heart muscle. The coord is related to the heart, where Ĉ is the circumferential direction dinate system CLR around the heart, L̂ the axis going from the apex area up through the left ventricle, c coordinate sysand R̂ the direction pointing out from the heart wall. The fsn tem is the fiber sheet system, where f̂ is the fiber direction, ŝ the sheet direction orthogonal to f̂, and n̂ the normal to the plane defined by f̂ and ŝ. “report” — 2005/7/31 — 20:15 — page 7 — #23 2.2. Fiber sheets 7 L̂ f̂ Ĉ R̂ ŝ n̂ Figure 2.2: An illustration of the laminar fiber sheets within the myocardium. The c where f̂ is pointing in the coordinate system associated with the fiber sheet is fsn, direction of the fibers, ŝ in the sheet plane orthogonal to f̂, and n̂ is the normal to the fiber sheet. The α-angle is measured between f̂ and Ĉ, the β-angle between ŝ d is related to the heart, where Ĉ is the circumferential direction and R̂. The CLR around the heart, L̂ the axis going from the apex area up through the left ventricle, and R̂ the direction pointing out from the heart wall. “report” — 2005/7/31 — 20:15 — page 8 — #24 8 Chapter 2. Anatomy of the heart “report” — 2005/7/31 — 20:15 — page 9 — #25 Chapter 3 Strain and tensors In this chapter a description of mechanical strain and strain tensors are presented. A deduction of the formulas for strain calculations is performed at the end of the chapter and how these are formed into a tensor. 3.1 Strain and deformation When studying the mechanics of materials the concept of strain and stress are used widely, and the two are closely related. For example, a soft rubber ball laying on a table is exposed to stress due to the gravitational force and the opposite force from the table. The rubber ball is in a state of equilibrium. If one would squeeze the ball it is still exposed to stress. But also, during the deformation from the original to a deformed shape the ball is exposed to strain due to the change in distance between points in the material. Stress is the distribution of internal forces and tensions that balance a given set of external tractions and forces, while strain is the measure for the amount of deformation. The definition of deformation is the change in distance between any two points within a body from one time to another. If the distance between any pair of points has not changed the body has not been deformed, though it may have been rotated and translated. Thus, rotations and translations does not contribute to the deformation. Strain is the dimensionless ratio between the length in the original state and the length in the deformed state. It is not meaningful to talk about strain unless there is a point of reference, also there is no absolute reference point when calculating strain since the two states can be chosen arbitrarily. Today only strain is possible to study and measure when it comes to the heart’s muscle tissue. The reason for this is that stress depends on the material properties of the tissue which are unknown today and quite complex. As mentioned in section 1.1 they are non-linear, inhomogeneous, time variant and anisotropic. Rönnbrant, 2005. 9 “report” — 2005/7/31 — 20:15 — page 10 — #26 10 3.2 Chapter 3. Strain and tensors The Tensor A tensor is a mathematical object that represents physical data independent from any coordinate systems in which the data is measured and/or calculated. The tensor components are associated with a specific coordinate system or frame and undergoes a linear transformation when the frame is changed. Tensors are generalizations of scalars, vectors and matrices to an object that has an arbitrary number of indices. Scalars have zero indices or are tensors of order zero, a vector has order one and a matrix order two [12]. Strain in three dimensions (Euclidean space) is represented by second order tensors which have nine independent components, also referred to as three dimensional Cartesian tensors. To better understand the physical meaning of a tensor it is sometimes useful to decompose the tensor into smaller pieces. Second order tensors are matrices and can thus be decomposed in the same way as those using ordinary matrix algebra. 3.2.1 Eigenvector-eigenvalue decomposition One useful and very common decomposition is the eigenvector-eigenvalue decomposition which satisfies the equations Te = eλ TP = PD where T is the tensor, e the eigenvector and λ the corresponding eigenvalue. Using only matrices P contains the eigenvectors (column wise) and the diagonal elements of D holds the eigenvalues. If the tensor is symmetric the eigenvectors become orthogonal [12], which is a preferred situation when visualizing using for example glyph’s, which is further explained in section 5.2. 3.2.2 Symmetric-antisymmetric decomposition Another way of decomposing a tensor is into one symmetric and one antisymmetric part. This holds for all tensors of any rank. So, a Cartesian tensor A can be written in the form A = AS + AA (3.1) where AS = AA = 1 A + AT 2 1 A − AT 2 (3.2) (3.3) The symmetric part has six independent components and the antisymmetric part has three. These can be interpreted as different transformations when deforming objects using homogeneous coordinates which is described in section 5.1 on page 27. “report” — 2005/7/31 — 20:15 — page 11 — #27 3.3. Strain tensors 11 3.3 Strain tensors Figure 3.1: Transformation of one neighborhood into another. The triangle PP′ P′′ is transformed into QQ′ Q′′ . To fully describe a deformation of a body the coordinates of any particle in the deformed state must be functions of the coordinates of that same particle in the original configuration. Letting (X1 , X2 , X3 ) be the original coordinates and (x1 , x2 , x3 ) the deformed coordinates equation (3.4) describes the relation between the original and deformed shapes. Assuming the deformation is continuous, a neighborhood in the original configuration is transformed into a neighborhood in the deformed configuration, with a one-to-one mapping it follows that it is also the other way around. The original coordinates must be functions of the deformed coordinates, as in equation (3.5), since it always possible to choose the original frame arbitrary. xi = xi (X1 , X2 , X3 ) (3.4) Xi = Xi (x1 , x2 , x3 ) (3.5) From this point the description of strain in three dimensions can be made as follows. An infinitesimal line element PP′ , see figure 3.1, connecting two points in a neighborhood, P(X1 , X2 , X3 ) and P′ (X1 + dX1 , X2 + dX2, X3 + dX3) will have the squared length ds0 2 = dX12 + dX22 + dX32 (3.6) A deformation of the neighborhood transforms the line element PP′ into a new line element QQ′ , which has the squared length ds2 = dx21 + dx22 + dx23 (3.7) “report” — 2005/7/31 — 20:15 — page 12 — #28 12 Chapter 3. Strain and tensors From equations (3.4) and (3.5) it follows that the infinitesimal line components can also be written as dxi = ∑ j ∂xi dX j ∂X j dXi = ∑ j ∂Xi dx j ∂x j i, j ∈ {1, 2, 3} or by using the Einstein summation convention as dxi = ∂xi dX j ∂X j dXi = ∂Xi dx j ∂x j By using the Kronecker delta operator and performing some index gymnastics the squared length elements become ds20 = δi j dXi dX j = ds2 = δi j dxi dx j = ∂Xi ∂X j dxm dxn ∂xm ∂xn ∂xi ∂x j δi j dXm dXn ∂Xm ∂Xn δi j (3.8) (3.9) From this the squared length difference can be formulated as either one of the two equations (3.10) and (3.11), depending on which frame of reference is preferred. ds2 − ds20 = ds2 − ds20 = ∂xa ∂xb − δi j dXi dX j δab ∂Xi ∂X j ∂Xa ∂Xb δi j − δab dxi dx j ∂xi ∂x j (3.10) (3.11) There is one strain tensor associated with each one of these notations, Ei j with equation (3.10) and ei j with (3.11). Ei j is referred to as the Lagrangian strain tensor and ei j as the Eulerian. The tensor used in this thesis is the Lagrangian version. The tensors are defined as ∂xa ∂xb 1 − δi j δab (3.12) Ei j = 2 ∂Xi ∂X j ∂Xa ∂Xb 1 δi j − δab ei j = (3.13) 2 ∂xi ∂x j or, in a simpler notation ds2 − ds20 2 ds − ds20 = 2Ei j dXi dX j (3.14) = 2ei j dxi dx j (3.15) “report” — 2005/7/31 — 20:15 — page 13 — #29 3.3. Strain tensors 13 Using the less obscure matrix notation, Ei j can be written as E= 1 (C − I) 2 (3.16) where C is the Cauchy-Green deformation tensor and I is the identity matrix. The Cauchy-Green deformation tensor is defined as the inner product of the deformation gradient tensor F by itself, and thus becomes a symmetric tensor [16]. C = FT F C being symmetric and I diagonal it follows that E is a symmetric tensor as well. This can also be deducted directly from the definition in equation (3.12), though a little less obvious. Without any further ado E can be expressed as E= 1 T F F−I 2 (3.17) where gradient tensor F is F= ∂xi = xi, j ∂X j F= ∂x1 ∂X1 ∂x2 ∂X1 ∂x3 ∂X1 i, j ∈ {1, 2, 3} ∂x1 ∂X2 ∂x2 ∂X2 ∂x3 ∂X2 ∂x1 ∂X3 ∂x2 ∂X3 ∂x3 ∂X3 (3.18) “report” — 2005/7/31 — 20:15 — page 14 — #30 14 Chapter 3. Strain and tensors “report” — 2005/7/31 — 20:15 — page 15 — #31 Chapter 4 Data and data acqusition In this chapter three data acquisition methods used for collecting raw data for strain calculations are presented followed by the calculations made on the data that where at hand for this thesis. The last section presents a syntheticall data set which is used for validation purposes. 4.1 Data acquisition There are different ways of collecting the data needed for calculating strains within the myocardium. Two non-invasive methods are magnetic resonance imaging (MRI) and the use of ultrasonic Doppler effect. The data available for this thesis is collected with the use of marker tracking which is an invasive method. 4.1.1 MRI The MRI techniques are based on the fact that all nuclei that have a net charge also have a magnetic moment. In theory all nuclei with a net charge could be used for MRI but in practice the hydrogen atom is used (1 H) since it is common in living tissue. MRI can be used for creating both 2D and 3D images, which can also be captured as a sequence of images creating 3D and 4D data respectively. The data obtained are scalar values in the three spatial directions, usually with some certainty values attached. Protons in the nucleus are constantly spinning around the magnetic moment and when a nucleus is put into a magnetic field the magnetic moment axis will start to precess around the field axis with a certain frequency called the Larmor frequency. This precession can, for the hydrogen atom, be in either one of two states, a higher and a lower energy state. Applying a radio frequency magnetic field which equals the Larmor frequency protons in the lower energy state can be excited into the higher state. When the radio frequency is turned off the protons Rönnbrant, 2005. 15 “report” — 2005/7/31 — 20:15 — page 16 — #32 16 Chapter 4. Data and data acqusition return to their previous state sending out the energy they absorbed. By measuring the strength and rate of decrease in the magnitude of the signal emitted from the deexcitation it is possible to determine the kind of tissue being observed. Two MRI methods for capturing data are Phase contrast MRI (PC-MRI) and MR tagging. By adding gradients to the magnetic pulse sequence PC-MRI creates phase information in the captured data which is proportional to the velocity. MR tagging works by pre-magnetizing areas within the tissue creating a grid which deforms with the tissue and that shows up as dark bands in the image data. Having this grid makes it possible to track certain points within the image [15]. 4.1.2 Ultrasonic The use of ultrasonic Doppler effects or myocardial velocity imaging is the most widely used method for cardiac strain rate. It is does not demand the kind of equipment needed for MRI and can be performed on humans without problems. Though, it does not nearly give the amount of data MRI techniques do, can only be used for 1D or perhaps 2D data in certain setups, and only gives the velocity of larger structures of the measured tissue [6]. 4.1.3 Marker tracking The use of markers for measuring the position of points in the myocardium is an invasive method. This way of studying the deformation of the heart is done by surgically implanting metallic markers into the cardiac muscle and then track those with radio-graphic tracking. Tracking the markers during the cardiac cycle gives a high spatial and temporal resolution, much better than can be achieved using MRI or ultrasound [9]. Although this method is very effective for getting the high spatial and temporal resolution data needed for strain calculations within the myocardium it cannot be implemented clinically since it needs a surgical operation. Furthermore, the surgery may cause scarring on the tissue resulting in artifacts in the data [9]. The data used for the strain calculations in this thesis where obtained from experiments performed at Falk Cardiovascular Research Center, Stanford University School of Medicine, Stanford, CA, USA. There were several data sets available but only one where used since they all represents the same kind of data. The data set used where obtained from sheep and results from the data are presented, together with a more detailed description of the marker method, in articles by Harrington et al. [8] and Cheng et al. [4]. Positions of the implanted markers are shown in figure 4.1. The small cluster of points to the right is the area where the strain was calculated, and the markers in this area are smaller, 0.7 mm in diameter compared to ∼ 1.5 × 3.0 mm, and referred to as beads. “report” — 2005/7/31 — 20:15 — page 17 — #33 4.2. Strain calculation 17 Figure 4.1: The position of the beads and markers in the data set obtained from sheep. The small cluster of points to the right are the beads and also the location where the strain was calculated. 4.2 Strain calculation The calculation method for obtaining the strain tensors is presented in the currently unpublished paper [10] by Kindberg, Karlsson, Ingels, and Criscione. It presents a method using linear-quadratic or linear-cubic polynomial adaption and gives identical results, or for a sparse array smaller errors, than for example a method which uses finite elements. A very brief work flow diagram of the method is illustrated in the Matlab part of figure 6.1 on page 39. 4.2.1 Coordinate transformation The beads and markers are measured in a laboratory reference coordinate system. Even though strain is defined as depending on the relative distance change between any two points in any coordinate system a transformation to a new system should be done before any visualization is performed. The use of homogeneous coordinates presented in section 5.1 for visualizing purposes contains translation components and it is much more convenient if the data represents structures that do not move to much relative the global reference system. In a animated scene this “report” — 2005/7/31 — 20:15 — page 18 — #34 18 Chapter 4. Data and data acqusition would otherwise cause the interesting objects to fly around. Also, to be able to compare different hearts it is necessary to have another reference system which is related to the heart itself and not the surroundings in which the data was measured. l̂ r̂ ĉ Figure 4.2: Illustration showing the (ideal) looks of the bead array and how the beads are numbered. The origin is located at the crossing point of the coordinate axes. To describe the transformation two coordinate systems are used. The interd also presented as the heart’s coordinate system mediate cylindrical system CLR, c a local Cartesian system around the bead array, see figure in section 2.2, and clr 4.2. The L̂ axis is defined as the vector from point 1 and the centroid of points {4, 7, 10, 13} from figure 4.1. Ĉ is the circumferential axis around L̂ and R̂ c The the cross product Ĉ × L̂, though they are not really needed for finding clr. c clr system is then defined using the L̂ and the normal to the plane formed by the points {15, 19, 23} as r̂. The normal being the cross product r̂ = v15→23 × v15→19 where v15→23 and v15→19 are the vectors from point 15 to points 19 and 23 respectively. Having L̂ and r̂ the ĉ direction becomes ĉ = L̂ × r̂ c become a At this point it is not certain that L̂ and l̂ are aligned and to have clr orthogonal system l̂ is defined as the cross product l̂ = r̂ × ĉ From this a 4-by-4 transformation matrix M is constructed using homogeneous coordinates. The inverse of M is put together as in (4.1) and then used for finding M. “report” — 2005/7/31 — 20:15 — page 19 — #35 4.2. Strain calculation 19 The translation component moves the origin to the centroid of points {15, 19, 23}. The transformation matrix is then calculated and applied to all points in each time frame of the data set, having the face of the bead array that is pointing out from the heart wall becoming the new origin. Homogeneous coordinates are described in section 5.1. c1 l1 r1 t1 c2 l2 r2 t1 M−1 = (4.1) c3 l3 r3 t1 0 0 0 1 4.2.2 Calculation method An estimation of the coordinates in the deformed configuration is approximated with a linear-quadratic polynomial which is a function of the coordinates in a chosen reference frame. The polynomial is then used for constructing the gradient tensor F in (3.18). The reference coordinates (X1 , X2 , X3 ) corresponds to ĉ, l̂ and r̂ respectively. Each deformed coordinate component can be expressed as in (4.2), but the arrangement of the beads possess restrictions on the function dividing it into a quadratic and linear part. x̂i = f (X1 , X2 , X3 ) , i ∈ {1, 2, 3} (4.2) Having four beads in the radial direction enables the use of a second order polynomial (4.3). f1 = a1 X32 + a2 X3 + a3 (4.3) The two other directions lying in the plane normal to r̂ can only be fitted to a first order polynomial each. Combining these gives a linear polynomial of two variables where the bilinear term is removed, thus resulting in f2 = (b1 X1 + b2)(b3 X2 + b4 ) − b1b2 X1 X2 = b1 b4 X1 + b2 b3 X2 + b2b4 (4.4) Combining (4.3) and (4.4) gives the linear-quadratic polynomial p(X1 , X2 , X3 ) = a1 X32 + a2 X3 + a3 (b1 b4 X1 + b2 b3 X2 + b2 b4 ) (4.5) which can be used for estimating a single coordinate component, since the polynomial is a scalar valued function. By using a different set of coefficient’s in the polynomial for each direction each of the components x̂i in (4.2) can be estimated. Written in matrix notation this becomes b X = PT C (4.6) “report” — 2005/7/31 — 20:15 — page 20 — #36 20 Chapter 4. Data and data acqusition where the size of C is a 9-by-3 matrix and P is 12-by-9. Twelve points are being b estimated and the polynomial got nine components. The resulting the matrix X holds the estimates for each point in a row order fashion. x̂c 1 x̂l 2 x̂r 3 .. .. .. . . . x̂c 12 x̂l 12 x̂r 12 The coefficient’s in C are found by minimizing the difference between the measured and estimated coordinates. The minimization can be done by solving equation (4.6) for C and inserting the coordinates for the chosen reference frame b This system is over determined but is solved by using the Mooreinto P and X. Penrose pseudo inverse, thus C = + PT0 X0 −1 T where ·+ is the Moore-Penrose inverse defined as B+ = BT B B [12]. Following this the gradient tensor F in equation (3.18) can be expressed as F= ∂PT C = PT,i C ∂Xi resulting in the the strain tensor E= 1 T C P,i PT,i C − I 2 All calculations have been implemented in Matlab scripts which are available on the enclosed CD. The script performing the calculations on the sheep data is called Run sham.m. It uses some other scripts which are found in the same directory for parts of the calculations. 4.2.3 Synthetic data For validation of the polynomial method a synthetic data set is used. It is created by deforming a cylinder from undeformed coordinates (R, Θ, Z) to deformed coordinates (r, θ, z) related by equations θ = R2 − R21 + r12 d aR + Θ + bZ z = cR + dZ r2 = where R1 = 2.00, r1 = 1.65, a = 0.10, b = 0.20, c = 0.30 and d = 0.80. Here R1 and r1 defines the inner undeformed and deformed radius, the other constants “report” — 2005/7/31 — 20:15 — page 21 — #37 4.2. Strain calculation 21 can be chosen arbitrarily for specifying the amount of twist and height shortening of the cylinder in the deformed state. For example a and b adjusts the angular component in the deformed state depending on the radial and longitudinal location in the undeformed state. Having (a, b, c, d) = (0, 0, 0, 1) would give no twisting and shortening of the cylinder. The exact solution in cylindrical coordinates is Frθz = ∂r R∂Θ ∂θ R∂Θ ∂z R∂Θ ∂r ∂R ∂θ ∂R ∂z ∂R ∂r ∂Z ∂θ ∂Z ∂z ∂Z which converts to the Cartesian system used in the bead array as Fzxy T = R′ Frθz R′0 (4.7) where R′ and R′0 are rotation matrices aligning the deformed and undeformed systems, R′0 being the matrix associated with the undeformed state, and converts it into Cartesian coordinates. The mapping from Cartesian to cylindrical coordinates is defined by left multiplication with the rotation matrix (4.8) and the opposite by its transpose, thus explaining the mapping in (4.7) [16]. The primed rotation matrices in (4.7) are permuted versions of the matrices in (4.8) and (4.9). This is because the definition of a rotation matrix in (4.8) maps the cylindrical coordinate system to the Cartesian as (r, θ, z) → (xc , yc , zc ). But the coordinate system used in the bead array must be mapped to the cylindrical as (r, θ, z) → (zc , xc , yc ), (the c coordinate system). Right multiplying the matrices (4.8) and (4.9) with the clr permutation matrix A in (4.10) creates the correct primed versions R′ and R′0 . cos θ sin θ 0 R = − sin θ cos θ 0 (4.8) 0 0 1 cos Θ sin Θ 0 R0 = − sin Θ cos Θ 0 (4.9) 0 0 1 0 0 1 A = 1 0 0 (4.10) 0 1 0 By placing beads in a suitable configuration in the undeformed state their corresponding position in the deformed state can be retrieved and put into the polynomial method, figure 4.3 and 4.4 shows the positions of the beads and the sample positions where the strain is calculated, before and after deformation. It is not just the bead positions that needs to be transformed but also the sample positions. The difference between polynomial and exact solution is plotted in figure 4.5. The solid lines (plus rings) shows the polynomial solution and the dashed lines “report” — 2005/7/31 — 20:15 — page 22 — #38 22 Chapter 4. Data and data acqusition 3 −0.4 x z −0.2 2.5 0 0.2 2 −0.5 0.4 0 0 0.2 0.4 y 0.6 0.5 0.8 0.5 0 Figure 4.3: Synthetic bead positions before deformation. The filled dots in the center of the structure are the sample positions. 0.2 0.4 2.5 x z 0.6 2 0.8 1 1.2 0.5 1 0.6 0.8 1 1.2 1.4 1 1.5 y Figure 4.4: Synthetic bead positions after deformation. The filled dots in the center of the structure are the sample positions. the exact solution. The bead array generated in this case had four beads in each column, the triangle base length was 1.04 cm at the outer triangle and slightly smaller at the most inner one. The two other sides where 0.96 cm. To validate the strain calculations only two frames are needed, the undeformed and the fully deformed. But by choosing start and end values for the constants a − d and increasing those a small amount for each frame a data set containing an arbitrary number of frames can be obtained. This method has been implemented in the Matlab scripts for generating data that can be displayed as an animation in the visualization tool Tensview described in chapter 6. “report” — 2005/7/31 — 20:15 — page 23 — #39 4.2. Strain calculation 23 0 0.3 −0.05 0.25 E E12 0.35 11 0.05 −0.1 0.2 −0.15 0.15 2 2.2 2.4 2.6 2.8 0.1 3 0.2 −0.02 0.18 −0.04 0.16 −0.06 0.14 E13 0 E 22 −0.2 −0.08 0.12 −0.1 0.1 −0.12 0.08 −0.14 2 2.2 2.4 2.6 2.8 0.06 3 0.7 0.2 0.6 0.18 2.2 2.4 2.6 2.8 3 2 2.2 2.4 2.6 2.8 3 E E23 0.22 33 0.8 2 0.5 0.16 0.4 0.14 0.3 2 2.2 2.4 2.6 2.8 3 Radius of undeformed cylinder [cm] 0.12 2 2.2 2.4 2.6 2.8 3 Radius of undeformed cylinder [cm] Figure 4.5: Difference between the exact solution and the polynomial method for the synthetic case. Solid line (plus rings) shows the polynomial solution and the dashed line the exact solution. “report” — 2005/7/31 — 20:15 — page 24 — #40 24 Chapter 4. Data and data acqusition “report” — 2005/7/31 — 20:15 — page 25 — #41 Chapter 5 Visualization In this chapter the aspects of visualizing strain tensors and which tools that might be used for doing it are presented. The first section describes the use of homogeneous coordinates followed by a brief description of the visualization techniques relating to tensors. Primarily second order 3D tensors are discussed. In the last section the tools are presented with a brief background and their pros and cons for the problem addressed in this thesis. 5.1 Homogeneous coordinates Homogeneous coordinates are used extensively in computer graphics for coordinate calculations. It integrates the sense of vectors and points into the same object by extending the representation with one value. For vectors and points related to 3-dimensional space it becomes a 4-valued vector instead of just a 3-valued. The transformation operations on these points and vectors are then performed with 4by-4 matrices. It could be generalized into N-dimensions using (N + 1) vectors and (N + 1)-by-(N + 1) matrices. There is no way of telling the difference between the vector a and point P in 3dimensional space using only 3 values. For example the point P located at (x, y, z) would in a frame defined by point P0 and the basis (e1 , e2 , e3 ) be defined by x p = y ⇒ P = P0 + [ x z y e1 z ] e2 e3 A vector with components (u, v, w) would have the same form u e1 a = v ⇒ a = [ u v w ] e2 w e3 Rönnbrant, 2005. 25 “report” — 2005/7/31 — 20:15 — page 26 — #42 26 Chapter 5. Visualization Thus, to avoid confusion, an extension with a fourth value is made and the point is instead represented with e1 x y ⇒ P = x y z 1 e2 p= e3 z P0 1 and the vector u v a= w 0 ⇒ a= u v e1 e2 w 0 e3 P0 From this point it is easy to carry out operations on both points and vectors using the homogeneous notation and simple matrix algebra. For example, a change from frame (e1 , e2 , e3 , P0 ) to ( f1 , f2 , f3 , Q0 ) would be formulated as follows f1 = m11 e1 + m12 e2 + m13 e3 f2 = m21 e1 + m22 e2 + m23 e3 f3 Q0 = = m31 e1 + m32 e2 + m33 e3 m41 e1 + m42 e2 + m43 e3 + P0 where m· are the components of the new basis expressed in terms of the old one. Using matrix notation this becomes f1 e1 m11 m12 m13 0 f2 e2 m21 m22 m23 0 M= f 3 = M e3 , m31 m32 m33 0 Q0 P0 m41 m42 m43 1 where M is referred to as the transformation matrix from f̂ to ê since the components of f̂ are known in ê. Transforming any point or vector between the two frames then becomes ae = MT af af = (M−1 )T ae For doing various types of affine transformations, like translation, scaling, rotation and shearing the use of the M matrix is straight forward. Shearing can be constructed by combining the three basic types, translation, scaling and rotation, but is usually regarded as a basic type as well because of its importance. The different type of transformations can be combined using ordinary matrix algebra forming one single matrix at the end which is then applied to all points and vectors needing transformation. The matrices for each of the transformations are “report” — 2005/7/31 — 20:15 — page 27 — #43 5.1. Homogeneous coordinates 27 Translation T = Scaling S = Rotation Shearing Rz (θ) = Hxy (θ) = 1 0 0 t1 0 1 0 t2 0 0 1 t3 0 0 0 1 s1 0 0 0 0 s2 0 0 0 0 s3 0 0 0 0 1 cos θ − sin θ 0 sin θ cos θ 0 0 0 1 0 0 0 1 cot θ 0 0 0 1 0 0 0 0 1 0 0 0 0 1 (5.1) (5.2) 0 0 0 1 (5.3) (5.4) Rotation has three degrees of freedom and the other matrices can be constructed by simple row and colon operations changing the coordinate system. The shearing can also be made in three different directions [1]. The interpretation of the different components are interesting when visualizing symmetric tensors. The two most interesting transformations are scaling and shearing. As seen in (5.2) and (5.4) the scaling only uses the diagonal elements while shearing only uses the off-diagonal elements. Thus, the symmetric second order 3D tensor expanded into a 4-by-4 matrix can be interpreted as a scaling and shearing transformation. As mentioned in 3.2.2 the antisymmetric part of a tensor after decomposition will have 3 independent components, these components will correspond to the rigid body rotation in each direction [1, 12, 23]. The following example in 2D displays the difference between antisymmetric and symmetric shearing on a rectangle. The two transformation matrices 1 0.3 0 1 0.3 0 MA = 0 1 0 MS = 0.3 1 0 0 0 1 0 0 1 antisymmetric and symmetric respectively are operating on a rectangle ABCD described by the four points (0,0), (0,1), (1,0) and (1,1) giving the transformed rectangles A′ B′C′ D′ pictured in figure 5.1. Each point is transformed as xi p′i = M• pi , pi = yi i ∈ {A, B,C, D} 1 “report” — 2005/7/31 — 20:15 — page 28 — #44 28 Chapter 5. Visualization Figure 5.1: Antisymmetric and symmetric shearing on a rectangle In figure 5.2 six different shearings of a 3D object are displayed. The shearing value is 0.5 and placed in the transformation matrix positions m12 , m13 , m23 , m21 , m31 , m32 for the sub-figures (a) through (f). A simple way of keeping track of how each component is affecting the object is to consider the matrix in equation (5.5) constructed as the outer product x − xy xz y x y z = yx − yz (5.5) z zx zy − where x, y and z are the axes in the coordinate system. xy indicates change in x with increased y, while yx would be the other way around. The same goes for the other components. 5.2 Techniques The concept of tensors are used in many scientific and engineering applications, some more complex than others. Though the challenge of visualizing the tensors can be made in a rather intuitive way. There are a few methods with which to visualize tensors such as pseudo-coloring, tensor glyph’s, deformation and hyperstreamlines which is a little more complex than the three others [23]. Pseudo-coloring is very simple, it codes the tensor components into colors visualizing them in a grid collage. This is not very effective for the user who must mentally integrate nine separate color fields in the case of a second order 3D tensor [23]. For animation purposes or where the amount of data is large the color “report” — 2005/7/31 — 20:15 — page 29 — #45 5.2. Techniques 29 Figure 5.2: The result of six different antisymmetric shearings in 3D. Only one shearing component with the value 0.5 is used for each sub-figure. (a) Matrix component m12 . (b) m13 . (c) m23 . (d) m21 . (e) m31 . (f) m32 . coding is even worse. In fact for small amounts of data it might be easier looking at the values directly and in case of a larger data sets a separate 2D plot of each component would be much easier to relate to since it catches the whole set of data in one image forming a line shape which humans have far more easier to percept and remember than color collages [17]. The use of tensor glyph’s involves the mapping of the tensor components to various features of the glyph, which might be altering its size, color and other more distinct features of the glyph as well. In figure 5.3 three examples of glyph’s are showed. The simplest tensor glyph is perhaps the ellipsoid, it is a sphere scaled by the eigenvalues in the direction of the corresponding eigenvectors. This only works for symmetric tensors [23]. The hyper-streamline method uses the three eigenvectors to form lines in space by integrating along the direction of for example the largest eigenvector and then letting the other to eigenvector/eigenvalues manipulate glyph’s along this line. This could be implemented using ones favourite glyph (for example the ellipsoid) and then having a user interface which lets the user drag the glyph along the streamline. The deformation method is an intuitive and also a very effective and simple “report” — 2005/7/31 — 20:15 — page 30 — #46 30 Chapter 5. Visualization method to implement. It is used extensively in mechanics textbooks to illustrate compressive forces [3]. This method also uses glyph’s, which are the objects being compressed and stretched, and might as a well be called a tensor glyph method. This is the main method used in Tensview. By interpreting the nine components in the tensor as part of a homogeneous transformation matrix (see 5.1) each of them can be visualized when using the matrix to deform an object, for example a cube which is a common object used for deformation. Figure 5.3: Example of three glyph’s that could be used showing different kinds of tensor data. (a); A simple arrow that can be pointed into the direction associated with the largest eigenvalue. (b); The tensor ellipsoid, it is scaled according to the eigenvalues in the direction of the associated eigenvectors. (c); A combination of a deformation cube and colored bars. The bars scale as well but also emphasizes the diagonal components by changing their colors. 5.2.1 Other ideas Since visualization of tensors is somewhat intuitive this section presents some judicious ideas that could be used. They where not implemented in any way into the final version of the program, but are presented here to give a perspective of what could be done and also what tricks could be used to boost the perceptive cues embedded into the visualization. As well as turning numbers into geons, shapes and colors the problem of tensor visualization should also be considered a problem on how the perception of these are interpreted. • “Twisting bars”. By turning the three cylinders showed in figure 5.3 (c) into rectangular bars and twisting them along their axes according to the current shear gives an extra cue about the shear value. Exaggerating the twisting would probably be necessary, for example one revolution. This “report” — 2005/7/31 — 20:15 — page 31 — #47 5.3. Available tools 31 would probably give a better view of all nine components of the second order 3D tensor. The diagonal values are still representing the scaling in each major direction. • Adding illusionary effects like the famous Müller-Lyer illusion or using the relative size illusion, see figure 5.6 (b). This could be achieved by using for example a cone as an arrow at the end of the bars/cylinders that either changes its shape between two static shapes, inward and outward, depending on the sign of the tensor component, or smoothly morphs its shape from one to the other, see figure 5.5. • Example of other simple perceptual cues that could be implemented with fairly small means are symmetry, common region and connectedness [2, 17]. The effect of symmetry can be exemplified with these two 8 character strings: [ ( ) ] h { } i and [ i ) { ] h } (. Examples of common region and connectedness are equally simple to illustrate, figure 5.4 below. u u u u u Common region u ur r r u ur r r u ur r r u Connectedness Figure 5.4: Example showing common region and connectedness. Figure 5.5: The cone smoothly morphs its shape from left to right and then back again resulting in a flat disc at the zero/reference value in the middle. 5.3 Available tools In this section the available tools considered for use in the visualization task are discussed and the reason for choosing the Coin library is motivated. “report” — 2005/7/31 — 20:15 — page 32 — #48 32 Chapter 5. Visualization Figure 5.6: (a); A relative size illusion. The circle in the middle appears larger to the right, even though the size is the same as the left one. (b); The classical Müller-Lyon illusion, the vertical line is perceived to be longer to the right, though it has the same length as the left one. 5.3.1 Ensight Ensight is a interactive post-processing visualization tool for various types of data, such as computational fluid dynamics, electromagnetics and thermodynamics. It is especially well suited for post-processing of large finite element and CFD simulations. It can also create animations from the data. Although it is not solely developed for visualizing strain tensors it would not be a problem using it for that purpose. Though, Ensight is quite advanced, and too advanced for the task at hand. Also, the licensing is very expensive. 5.3.2 Matlab Matlab is a very well known software by most engineers and scientists and which use could be considered very common, both by universities and companies. There is also a free software program, Octave [7], which can substitute Matlab in most cases. Matlab (and Octave) is on the other hand primarily intended for numerical computations and not visualization, even though there are extra toolboxes for Matlab which addresses the issue. A quick overview of the abilities to visualize 3-dimensional data in Matlab shows that there are some limitations in performance and a somewhat complicated structure to quickly develop a tool that can handle rapidly altering data (25+ times per second), which would be the case when showing the data described in chapter 4. Further Matlab’s Visualization toolbox is not a part of the “basic” set of toolboxes which is a major limitation since this would cause the application to run only on a limited set of Matlab installations, where they exist. “report” — 2005/7/31 — 20:15 — page 33 — #49 5.3. Available tools 33 5.3.3 OpenDX OpenDX is a tool introduced in 1991 developed by IBM which has been released as open source software since May 1999. It is a general purpose data exploring tool aimed at visualizing both simple and complex data using a graphical user interface (GUI) for programming the data manipulation and visualization features. Besides supporting scientific data file formats it also has data importing features which makes it fairly easy for any user to import his or her data into the program [14]. A simple example of a network is pictured in figure 5.7 and a screen-shot of the result is displayed in figure 5.8. The data used is the sheep data described in chapter 4 and what is seen within the bright ring i figure 5.8 are the 12 beads inserted into the left ventricular wall. Although it is fairly simple, with a bit of previous practice, to rapidly visualize ones data in OpenDX the 3D performance is not good enough for animated data and the GUI is not easy to work with yet, it needs some improvements to integrate better with current Linux distributions. Also, the amount of memory consumed by OpenDX is on average ∼600 megabytes, and thus not really suitable for the average computer dated 2004 or 2005. Figure 5.7: A simple OpenDX network. The network starts at A with file selection, then continues to a data importer which extracts the data from the file. The format of the data is specified in a text file used by the import-box. After that a portion of the data is selected, the integer box specifies which frame should be used, then a glyph is assigned to the data, in this case a sphere, and finally the image is rendered at B. “report” — 2005/7/31 — 20:15 — page 34 — #50 34 Chapter 5. Visualization Figure 5.8: A screen-shot from OpenDX showing the 12 bead points from the sheep data in the brightened circle. This is the result from the network in figure 5.7 and it is possible to zoom, pan and rotate the scene. 5.3.4 Open Inventor/Coin 3D Open Inventor (OI) is a C++ library software for 3D graphics rendering used for creating 3D-programs and tools for visualization of scientific data but also for creating any type of 3D software. It was developed by Silicon Graphics (SGI) in the early 1990s as an extension or clue to the well known OpenGL 3D rendering standard. The use of OpenGL was and is very low level and to ease the programmers task OI was developed to be a simpler applications programmers interface [22]. Rather than building shapes from simple polygons as in OpenGL and which is very tedious, OI uses simpler shapes as boxes, cones and so forth which are put together into more advanced shapes when needed. The rendering world is put into a so called scene graph which is controlled by OI making all occlusion culling on objects in the graph automatic. There is also a number of controller objects available and a system for applying these to the scene graph. This makes the development of common interaction tasks much easier. OI also implements a file format for storing “worlds” and the code for saving and loading the worlds from these files. One downside with OI is that it is always slower than hand written OpenGL code [22]. “report” — 2005/7/31 — 20:15 — page 35 — #51 5.3. Available tools 35 In August 2000 OI was licensed as open source after being available only as proprietary code from the company TGS which had 3rd-party licensed OI from SGI. At that time the clone library Coin was released by the Norwegian company Systems In Motion (SIM http://www.sim.no). Coin has been written entirely from scratch in a clean room design fashion, sharing only the original API from OI. Coin is available under the GNU GPL license for development of free software, but also under a proprietary license for the right to develop proprietary commercial software. The scene graph The scene graph in OI central part where all objects that shall be rendered are stored. It is build in a tree fashion way starting with a top node, or the root node. The root node then holds a subtree of nodes which contains more subtrees. When the scene graph is rendered by the OI frame work the tree is traversed from top to bottom and from left to right. The type of nodes available are shape, transform, property, grouping and separator nodes. Shape nodes defines various kinds of shapes that shall be rendered. Transform nodes defines a transformation, for example scaling or translation, which is applied to all nodes following the transform node, within the current separator node. A property alters the rendering state for the following nodes, making it possible to change for example the color and the draw style of shapes. Grouping and separator nodes are used primarily for storing other nodes. The main difference between them is that separator nodes protects its children from previous applied transform and property nodes [19]. The world file format The Inventor file format is a text based format which makes it easy to use and simple to construct the scene graph using a text processor. A user can with little or no experience of 3D programming and brief knowledge of the scene graph concept together with a list of available OI nodes put together a world file which describes a fairly advanced 3D scene. The file can be read and rendered by OI with a very simple program which is presented with full source code in appendix B. Below are two examples of simple scene graphs. The first will display a cone, a sphere and a cube lined up in a row and the second two rotating cylinders. In appendix B.2 a more advanced example is presented. 5.3.5 Why Coin? Coin being open source under the GNU GPL license, following the OI API and thus supporting the Inventor file format for scene graph description, having all the convenient classes implemented in OI for data handling, and also being under constant development by SIM made it the best choice for implementing the software “report” — 2005/7/31 — 20:15 — page 36 — #52 36 Chapter 5. Visualization for visualizing the tensors. Also, the implementation in object oriented C++ integrates nicely with the concept of using nodes in a scene graph for representation of the graphical environment, since the nodes and other objects often have properties which can be seen as a derivative of a already implemented node or object. #Inventor V2.1 ascii Separator { Cone {} Transform { translation 2 0 0 } Sphere {} Transform { translation 2 0 0 } Cube {} } Figure 5.9: Simple example illustrating the inventor file format. Displays three simple shapes aligned in a row, a cone, a sphere, and a cube. #Inventor V2.1 ascii Separator { Separator { RotationXYZ { axis X angle 0 = ElapsedTime {} . timeOut } Material { diffuseColor 1.0 0.5 0.2 } Cylinder { radius 2 height 10 } } Transform { translation 0.5 0 0 } Separator { RotationXYZ { axis Z angle 0 = ElapsedTime {} . timeOut } Material { diffuseColor 0.2 0.5 1.0 } Cylinder { radius 2 height 12 } } } Figure 5.10: Simple example illustrating the inventor file format. Will display two cylinders where one is rotating around the X and the other around the Z axis. “report” — 2005/7/31 — 20:15 — page 37 — #53 Chapter 6 Tensview This chapter describes the main parts and the use of Tensview, the software developed for the visualization task using the Coin software library. When references are made to some C++ classes/objects more about those can be found in the HTML documentation on the enclosed CD. 6.1 Tensview Tensview is a simple program for visualizing matrix and scalar data in animated form. Or, put in another way, it handles time series containing lists of matrices and scalar values that can be connected to various 3D nodes that alters their shape, color, transparency etc based on the data. It is written using C++ and complies with the most common compilers on both Linux and Windows. As of now only one data set of matrices can be loaded and which is visualized by two graphical objects, which of one is a coordinate transformed version of the other. Also one set of scalar data can be loaded, which is visualized as a 2D chart plot in the rendering window. It is based on the 3D graphics rendering library Coin described in 5.3.4. Coin’s application programmers interface is fully compatible with SGI’s Open Inventor library, which is the de facto standard 3D graphics API for complex visualization applications. For a more extensive overview of the Open Inventor library there are two very good books covering the subject, The Inventor Mentor [19] and The Inventor Toolmaker [20], both of which where found very useful during the development of Tensview. 6.1.1 Work flow In figure 6.1 an overview of the work flow from Matlab to Tensview is shown. The Tensview box is perhaps the important one since the data source can be any Rönnbrant, 2005. 37 “report” — 2005/7/31 — 20:15 — page 38 — #54 38 Chapter 6. Tensview program. Furthermore, many details have been left out since they are more to consider being specific solutions of the implementation itself. Though, the boxes shown explains some of the more interesting events. Load data Tensview begins by examining the input file, which is the main way of specifying the input arguments. The input file tells which files to load the data from. Loading data into Tensview can at the moment only be done from two files, but the extending the loading part to more files is fairly easy. The loaded data are stored into objects which can handle time series data and which also utilizes some Coin engines for interpolating purposes so that the data can be interpolated between two time frames. These objects derived from the class myTimeSeries. c coordinate system presented in section 2.2 is an interCreate FSN data The fsn esting coordinate system in which to analyze the strain tensors. For doing c system, which they are so the tensors needs to be transformed from the clr c system. assumed to be in when read from the files, to the fsn This part is quite essential in the program but makes it less general. Having this part become a general preprocessing would be more suitable. Create scene graph When all data is loaded and “preprocessed” the scene graph is generated. In this process two essential parts are present, the first contains a in-code defined scene which sets up cameras and lights, register callback functions into the scene for user interaction and connects the loaded data to the scene. The shape of the scene nodes visualizing the data are not specified in the code. Instead this is done in the Inventor ASCII format (see section 5.3.4) which is loaded from a file. Start data scheduler The scheduler is an object of class myTimeSeriesTicker which controls all data containers of class myTimeSeries holding loaded data. The scheduler keeps track of the time and controls which time frame to view by updating all time series registered to it. The user controls the scheduler via callbacks telling it how fast to run and when to pause etc. Run main loop In the main loop Tensview is fully up and running and the user can interact with the scene with the mouse and control Tensview via keyboard commands. 6.2 Implementation details In this section the most important implementation details are covered. The visualization where presented in section 5.2 and here only a short note is made on how the deformation and coloring techniques where implemented. “report” — 2005/7/31 — 20:15 — page 39 — #55 6.2. Implementation details 39 Figure 6.1: Data flow diagram showing the main operations performed in Matlab before entering Tensview and what then happens in Tensview. “report” — 2005/7/31 — 20:15 — page 40 — #56 40 Chapter 6. Tensview 6.2.1 Data files The data files read by Tensview have no special format, they only need to contain floats separated by a space character, which can be newline, tab or one o more spaces. Any ordering of the data is specified by the time series object it is read into. For example the myTimeSeriesMFMatrix class holds lists 4-by-4 matrices, one list corresponding to each frame. The 4-by-4 matrix needs 16 float values at least, this is enough for filling one matrix in one frame. By specifying the number of matrices contained in each frame (argument matrixlength) together with the data file, Tensview will read and fill matrices until the end of the data file is reach. When the number of specified matrices have been filled in the current frame the next frame is filled and so forth. This means that the number of frames is specified implicitly by the number of floats in the file and the matrixlength argument. For example, the typical data files for the sheep data have six or eleven matrices (tensors) in each frame and the number of frames are 135. The number of floats in these data files are therefore 16 × 11 × 135 = 23760 floats. The values fills the matrix column-wise, so the sequence {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16} of floats gives the matrix 1 5 9 13 2 6 10 14 3 7 11 15 4 8 12 16 If other time series classes are implemented they need a similar way of specifying their internal structure of the data so that the user can arrange his or her data accordingly when generating the data files. The series myTimeSeriesMFFloat is trivial since a single float does not have an internal structure. 6.2.2 Time series The time series objects are data containers, as mentioned above, and where implemented to make it easier to handle data related to animation. There are already a number of classes in Coin for handling data of various types, for example matrices and different kinds of vectors. These are referred to as fields and are divided into single and multiple fields. Single fields store one basic type, and the multiple fields store many. Thus, one multiple fields might had been useful if only one basic type where needed for each frame, but that is not the case. Instead there are several tensors which are stored as 4-by-4 matrices related to each time frame, so there was a need for a container that could store lists of multiple fields, and that is what the time series classes do. Also, they provides some convenient functions for interpolating data between two consecutive time frames and can be controlled by an object of class myTimeSeriesTicker, mostly called the scheduler. “report” — 2005/7/31 — 20:15 — page 41 — #57 6.2. Implementation details 41 The scheduler is used as a single point of interaction for controlling any number of time series data objects. It can be considered the main controlling instance of Tensview. Many user interacting callback functions are connected to the scheduler which provides features such as pause/play, forward and backward stepping through the time series, and also for the reloading of data from disk and for time controlling before doing screen-shots. Currently only one scheduler is available. Having several would enable the feature of displaying scenes with data from different time frames. But, this feature could as easily be accomplished by starting two instances of Tensview with different data sets as input. 6.2.3 The Inventor file As mention in the work flow section 6.1.1 the inventor file used by Tensview specifies the shape of the objects used for visualization. It accomplishes this is by using a feature of the file format where nodes specified in the file can be named, and thus be fetched very easy from within the program. By replacing the named node in the file any type of scene graph subtree can be sent to Tensview, which could define a rather complex glyph. This is an important feature making it very simple to replace the shapes, colors etc for the objects used for visualization. The subtrees which must be specified are c system. TENSOROBJECT CLR The glyph displaying the tensors in the clr c but TENSOROBJECT CLR STATIC The glyph displaying the tensors in clr, which does not animate since it is connected to the static data in the reference time frame. c system. TENSOROBJECT FSN Same as above, but for the fsn c system. TENSOROBJECT FSN STATIC Same as above, but for the fsn EXTRA OBJECTS SEPARATOR This subtree can contain any type of scene graph subtree which a user wants to put into the scene. Nothing is connected to any altering data fields but by using engines animation can be accomplished. Mostly this would be useful for putting objects into the scene which does not change but relates to the data some how. Currently a small object picturing a coordinate system with red-green-blue arrows relating to ĉ, l̂ and r̂ is defined in this subtree. The file used in Tensview is quite large and is not included in the report but is available on the enclosed CD in the demodata-directory. “report” — 2005/7/31 — 20:15 — page 42 — #58 42 Chapter 6. Tensview 6.2.4 Glyph’s Since the glyph’s are specified in the Inventor ASCII file it is important to understand that they themselves are not “magic” in any way. A user could invent and define a more suitable glyph with some brief knowledge on how to write scene graphs using the Inventor ASCII file. 6.2.5 Deformation and coloring Having the glyph’s as simple objects the task of deforming and coloring them are easy. The aspects of deformation were presented in sections 5.1 and 5.2, 5.2 also mentions coloring. What happens in Tensview is that the tensors, actually before they are loaded, are expanded to 4-by-4 matrices and than just interpreted as homogeneous transformation matrices without any further change. For each matrix in every frame a transformation using the matrix of a glyph is performed making just as many copies of the glyph as there are matrices. The coloring of nodes/shapes can briefly be described as follows. When the copying and transformation of the glyph is done the values in the transformation matrix is present to all subparts of the glyph as well, this is an effect of the scene graph tree structure, and by having special nodes which can alter the current color setting used for rendering parts of the glyph can be colored according to the values in the transformation matrix. The class myModelMatrixColorConverter implementes this special coloring node, which is a part of Tensview rather then Coin, but can still be specified in the Inventor file format. Inserting this type of node before the nodes (glyph parts) selected for coloring and also specifying which part of the transformation matrix be used for altering the color the extra cue of coloring is put into the scene. Examples of how the deformation and coloring looks like can either be seen by running Tensview or viewing one the movies in the /media-directory on the enclosed CD. In figure 6.3 are two screen-shots displayed, showing the reference frame and a second frame for a certain set of data in which the deformation and coloring should be observable. The row of objects in the background are the tensors c coordinate system and the parallel gray bars are symbolise transformed to the fsn the laminar fiber sheets, also pictured in figure 2.2. 6.3 Usage In this section the usage part of Tensview is described. Which arguments it takes, keyboard usage and the format of the input file. 6.3.1 Arguments and options Tensview was developed for use with a console (text based user interface). There are no menus that lets the user load or save data after the software has been started. “report” — 2005/7/31 — 20:15 — page 43 — #59 6.3. Usage 43 Instead the files containing the data to be visualized must be supplied as arguments at startup. In Linux the software uses a couple of options that the user must specify to let the program know which files to read and how the data in the tensor/matrix data file is aligned, see table 6.1. It is also possible to use an input file which specifies the options. The input file is a plain text file which specifies the options and their corresponding argument separated by whitespace characters on separate lines, see section 6.3.5 for an example. Table 6.1: Command line options. -h, --help -l, --matrixlength -r, --referenceframe --matrixfile --signalfile --objectfile --inputfile Print usage information. Matrices per time frame. Reference frame. Matrix data file. Simple scalar data file. Inventor scene objects file. Input file which specifies the options. 6.3.2 Mouse and keyboard usage Since the program should compile on both Unix and Microsoft Windows platforms the use of menus have been removed since the usage of such involves platform specific programming. Instead all interaction with the program is done with the mouse and keyboard. The SoQt/SoWin-library comes with an extensive set of useful windows widget classes which can be used by the developer to quickly get a graphical window running for any type of application. The class chosen for this program is one that already defines two basic user interaction modes, camera interaction and scene graph interaction. The camera interaction mode is completely untouched from its default behavior. Instead all interaction that is specific for this program is done in the scene graph interaction mode. Changing from one mode the other is done by pressing the escape key (’ESC’), which is also the default behavior implemented by the library. 6.3.3 Camera interaction mode In camera interaction mode only the camera is affected. The only available controls are those built into the library and since they are not modified in any way they might differ between different version of the library. Though it is not likely that would be the case. The controls for camera interaction as they are defined in the SoQt/SoWin 1.2.0 manual are presented in table 6.2. “report” — 2005/7/31 — 20:15 — page 44 — #60 44 Chapter 6. Tensview Table 6.2: Camera interaction controls. Left + move Middle + move Scroll wheel Left + middle Right Holding ALT Home s q ESC Rotate camera around focal point. Pan scene. Zoom/dolly, only works in XFree/Xorg. Zoom/dolly. Use Ctrl + Shift + left on a 2-button mouse. Opens the pop-up menu. Temporary switches to scene graph mode. Resets camera orientation and focal point. Seek mode, clicking on a object moves the focal point to it. Quit application. Toggle to scene graph mode. 6.3.4 Scene graph interaction mode The available keys are presented in table 6.3. A more detailed description of what some of the commands do is presented in appendix A.1. The “CTRL” and “SHIFT” words are referring to the function keys on the keyboard, it does not matter which control or shift key is pressed. The “NUMPAD” means that it is the corresponding key on the numeric pad on the keyboard that should be used. 6.3.5 The input file The input file should, if used, be the only argument to the program. In Linux the input file is specified by the option --inputfile or if a single argument is sent to application that is interpreted as the name of an input file. In Windows the program only takes one argument which must be the name of the input file. A convenient way to supply the filename as argument in Windows is to drag-and-drop the input file on the program executable, that is the .exe-file used for starting the program. It is also be possible to drag-and-drop the input file on a shortcut-icon connected to the executable. The format of the input file is rather simple, it consists of key-value pairs where the name of the key means something specific to the program. An example of an input file is shown in figure 6.3.5. The keys recognised by the program are the same as the names of the option flags presented in table 6.1. “report” — 2005/7/31 — 20:15 — page 45 — #61 6.3. Usage 45 Table 6.3: Scene graph interaction keyboard controls. NUMPAD+ NUMPADCTRL + NUMPAD+ CTRL + NUMPADCTRL + l CTRL + s CTRL + d F1 F2 F3 F4 Space Home Backspace Left arrow Right arrow Down arrow Up arrow z x SHIFT + z SHIFT + x SHIFT + c f e v SHIFT + v ESC # Blank lines and lines # Options are specified # one option per line. objectfile matrixfile signalfile matrixlength referenceframe Increase time speed. Decrease time speed. Increase tick delay. Decrease tick delay. Reload all data from source. Screen-shot of the current view. Continuous screen-shots. Increase view FPS. Decrease view FPS. Increase data FPS. Decrease data FPS. Toggle pause. Jump to first frame in sequence. Change direction. Step backward, (pause mode). Step forward, (pause mode). Step backward faster, (pause mode). Step forward faster, (pause mode). Increase (zoom) chart for signal data. Decrease chart plot for signal data. Decrease chart plot width. Decrease chart plot height. Next tensor scene object. Previous tensor scene object. Disable CLR glyph’s. Disable FSN glyph’s. Disable extra shapes read from objects.vi. Disable chart plot. Enable chart plot. Toggle to camera interaction mode. beginning with ’#’ are ignored. like "key[:space:]value" pairs, ../objects.iv data_tensor.txt data_volume.txt 11 0 Figure 6.2: Example of an input file. “report” — 2005/7/31 — 20:15 — page 46 — #62 46 Chapter 6. Tensview Figure 6.3: Screen-shots taken from the data set r15p06 tensor.dat. The bead c is visualized as the red-green-blue arrows in the scene. coordinate system clr (Top) This shows the neutral state, or reference frame. All bars are gray and the deformation cube and the reference cube are aligned. (Bottom) Here the ĉ and l̂ axes have been compress, which is indicated by the bluish color and the shortening compared to the reference state. The r̂ axis has been stretch, indicated by the reddish color and lengthening compared to the reference state. “report” — 2005/7/31 — 20:15 — page 47 — #63 Chapter 7 Future work This chapter contains the list of improvements that could be made to the software and also presents some other applications in which it could be used. 7.1 Software improvements There are several improvements that can be made to the software to have it become more general and perhaps more user friendly. Also, it lacks some of the visualizations ideas presented earlier in 5.2.1. These could be implemented in the current frame work of the software code without too much effort. In the follow list all the features that came up as good ideas during the development, but where left out of the first working version, are presented. This is the current “todo-list” presented in the HTML documentation on the enclosed CD. Of course if any further work is made in the software there might be even more changes. • Make it possible to have an arbitrary number of input data files for both types of data. That is, the arguments --matrixfile and --signalfile should take a comma separated list of filenames and be possible to specify more than one time using the command-line. • It should be possible to move the scene graphs objects (subgraphs) relative to each other. Doing so with separate data sets might be necessary for the user to see anything if the glyph’s from different data sets occupies the same spatial point. Though, a more dynamic solution to this would be to implement a type of bounding box generating function that finds the bounding box for each data sets and separates them accordingly. Rönnbrant, 2005. 47 “report” — 2005/7/31 — 20:15 — page 48 — #64 48 Chapter 7. Future work • Make it possible to toggle between the tensor glyph’s when using matrices and just showing the pure spatial position represented with a simple sphere, (or perhaps some other glyph). • Fix scaling in class myTransformScaler for example by using symmetric antisymmetric properties, or even remove the possibility completely. Idea: Assume the matrices are non-rotated and only scale the diagonal elements in the most obvious and simple way. All other form of matrix scaling should be performed before the data is loaded into Tensview. • Modify class myTimeSeriesTicker so that it also handles static series, or series that are not affected at normal running, but still are being controlled by the ticker for other purposes. The “ticker” is a better place for handling all series since it handles everything in a list sense of way instead as named objects. • Must have a way of sending the α and β values to the application so that those can be set by a user, rather then being statically set in the source code. Idea: It would work with a key-value-pair in the input file where the key is alpha or beta followed by two floats, and for each occurrence a new value is added to the alpha and beta configuration inputs. • Fixing so that some enabling/disabling of viewable objects can be made by declaring the objects in the Inventor object file or not. For example, this would be a simple solution to the fact that one might not want the FSN-object visible. In the glyph-objects assignment phase there will be a null pointer which clearly states that there will not be any object of this kind. • Make the parsing more dynamic. For example reading a list of keys of what should be parsed together with the number of arguments. The keys and number of arguments should be defined in the MyOptions struct. This todo-item is solely for making expansion of the option handling more easy and does not affect the workings of the program. 7.2 Other applications The source code for Tensview changed a couple of times during the development becoming more generalized. Also, the idea of making the source code more general was appealing since it then could be used even if the data that where the primary target for this thesis would change. Even though the use of three dimensional second order strain tensors are general in itself and not solely targeted for use in myocardial strain calculations, some implementations used from the start where “report” — 2005/7/31 — 20:15 — page 49 — #65 7.2. Other applications 49 aimed at only dealing with the amount of tensors generated by the myocardial data at hand, which is in the range 3-6. But at the end of writing this thesis Tensview seemed to be able to be used in another area, namely the visualization of blood velocity vectors with associated certainty scalars. This has not been tested but the current implementation should be able to visualize this data if it can be arranged in a way that has some meaning in a 4-by-4 homogeneous transformation matrix. A solution would be to use the vectors for creating transformation matrices which aligns the glyph’s, then using the certainty values for scaling the diagonal values and using the coloring mechanism to put extra cues into the scene. “report” — 2005/7/31 — 20:15 — page 50 — #66 50 Chapter 7. Future work “report” — 2005/7/31 — 20:15 — page 51 — #67 Appendix A Tensview A.1 Details about key commands This sections gives a more detailed description of each of the keyboard commands to the extent that a normal user should not need any other documentation running the program. NUMPAD+ Increases time speed, which will make the animation run faster relative real time. This affects the step length taken within all time series. For example data might have been sampled at 100fps, running in real time at a view fps rate of 25fps would only use each fourth sample in that data. NUMPAD- Decreases time speed, 100fps data at 25fps view speed but with the time speed reduced to 0.1 real time would need 250 frames for each second and thus interpolates the data to accomplish this. CTRL + NUMPAD+ Increases tick speed. The tick speed is used for slowing down the update speed of the screen without affecting the value of view fps, which is also controllable. The reason for this is that one might want to warp the time and view each screen update for a couple of seconds. Just decreasing view fps does not give this effect since it would increase the step size at the same time so that the application runs at the selected time speed. CTRL + NUMPAD- Decreases tick speed, which is what one should start with when trying the warp the time. Increasing tick speed from the default value 1.0 only makes the application loop run faster without any beneficial effects on the screen. CTRL + l Reloads data from the files specified as input files. This might be useful since there is no way of open “new” files when the application has started. For example, by using a FIFO-file or just overwrite the data files with new Rönnbrant, 2005. 51 “report” — 2005/7/31 — 20:15 — page 52 — #68 52 Appendix A. Tensview data from an external source it is possible to view new data without restarting the program. CTRL + s Takes a single screen-shot. It is not possible to name the files shot to, instead the application keeps track of how many screen-shots have been taken since the application started and names the files accordingly, screenshotXXX.tiff where XXX is a number starting from 001 and increases for each screen-shot taken. CTRL + d Starts a continuous dumping of each new frame to a file, beginning from the current frame. Dumping stops when the beginning of the sequence is reached, that is frame 0.0f. The application will run much slower during dumping but it is possible to interact and to rotate and zoom the scene to make a more exciting movie, (because there is no point in dumping each frame unless one is going to make a move of all the image files). F1 Increase view fps. The default view speed or screen update speed is 25 fps which is enough for most people to consider movements to be smooth. On a very slow computer it might be that 25 fps is to fast for it to handle and a lower fps must be set. Increasing it to more then 30 fps should not give any benefits unless there are special needs for creating a movie. F2 Decreases the view fps. F3 Increases data fps. All data have some kind of sample speed since it should contain at least one spatial component for it to be useful in this application. (There might be other ways of interpreting data but usually time is a crucial component). By setting the data fps a correct animation speed will be achieved which is probably what is preferred. F4 Decreases the data fps. Space Toggles pause on or off. Home Jumps to the beginning of the animation sequence, frame 0.0f. Backspace Switches direction of the animation. Left arrow Steps backward in the sequence. When doing screen-shots or just view the data stepping back and forth is a necessity. Only available in paused mode. Right arrow Steps forward in the sequence. Only available in paused mode. Down arrow Steps backward a little faster. Only available in paused mode. Up arrow Steps forward a little faster. Only available in paused mode. “report” — 2005/7/31 — 20:15 — page 53 — #69 A.2. Source code 53 z Zooms in on the chart plot if any. The chart has a default size that is rather small and if there are detailed parts in it those might be easier to see when enlarged. x Zooms out on the chart plot if any. SHIFT + z Decreases the height of the chart plot. SHIFT + x Decreases the width of the chart plot. - Steps forward in a list of different scene graph objects used for visualizing the matrix data. These objects are defined in the objects.iv file and can changed quite easily by the user. Different types of scene graph objects are suitable for different kinds of data. SHIFT + - Steps backward in the scene graph node-list. c Disables the glyph’s used for displaying the CLR-system. This could be used if there are so many glyph’s being displayed that the whole scene is sluggish and only the FSN glyph’s are interesting. f Disables the glyph’s for the FSN-system, if the CLR glyph’s are the interesting ones. e Disable shapes read from objects.vi. The extra nodes read from the objects.ivfile could be any type of nodes and it could be useful to turn these off. v Disables the chart plot. This does not affect performance but is useful when doing for example screen-shots or when the data plotted is of no interest. SHIFT + v Turns the chart plot back on. ESC Switch to camera interaction mode. A.2 Source code The source code for Tensview have been left out of the report and is only included on the enclosed CD. The full source code would have added approximately 7080 extra pages of dull reading. If the CD is unavailable the source should also be available at http://www.lysator.liu.se/˜andro/exjobb, or it could be obtained by contacting IMT (address is found on the third page) or the author on e-mail address [email protected]. “report” — 2005/7/31 — 20:15 — page 54 — #70 54 Appendix A. Tensview ‘‘report’’ --- 2005/7/31 --- 20:15 --- page 55 --- #71 Appendix B IV-file viewer and example B.1 IV-file viewer source code This is the complete source for a simple Inventor file viewer. It assumes the arguments are Inventor files and reads them one by one constructing the scene graph and finally displays them using the convenient ExaminerViewer. 1 2 3 4 5 #include #include #include #include #include <iostream> <Inventor/Qt/SoQt.h> <Inventor/Qt/viewers/SoQtExaminerViewer.h> <Inventor/SoInput.h> <Inventor/nodes/SoNodes.h> 6 7 using namespace std; 8 9 10 11 12 13 int main( int argc, char** argv ) { SoDB::init(); QWidget * Mainwindow = SoQt::init(argc, argv, argv[0]); 14 15 16 17 18 19 if ( argc < 2 ) { cout << "Need input files!" << endl; cout << "Usage: " << argv[0] << " <iv-file(s)>" << endl; exit(1); } 20 21 22 23 24 25 SoInput in; for ( int i=1; i<argc; ++i ) { if ( in.pushFile(argv[i]) ) cout << argv[i] << "...is ok" << endl; else Rönnbrant, 2005. 55 ‘‘report’’ --- 2005/7/31 --- 20:15 --- page 56 --- #72 56 Appendix B. IV-file viewer and example cout << argv[i] << "...read failed, skipping!" << endl; 26 } 27 28 SoSeparator * root = SoDB::readAll(&in); 29 30 if ( root ) root->ref(); else { cout << "Root separator is NULL! Something is wrong." << endl; exit(1); } 31 32 33 34 35 36 37 SoQtExaminerViewer * eviewer = new SoQtExaminerViewer(Mainwindow); eviewer->setSeekTime(0.6f); eviewer->setAntialiasing(true, 1); eviewer->setBackgroundColor(SbColor(1,1,1)); eviewer->setSceneGraph(root); eviewer->show(); 38 39 40 41 42 43 44 SoQt::show(Mainwindow); SoQt::mainLoop(); 45 46 47 if ( eviewer ) delete eviewer; if ( root ) root->unref(); 48 49 50 return 0; 51 52 } B.2 IV-file advanced example This is a more advanced example then the ones presented on page 36. The transform nodes in this scene are connected in a way that make them animate the shape nodes, which looks like a wobbling water molecule (or Micke Mouse head). This file is also available on the CD in the ivviewqt directory. 1 2 3 4 5 6 7 8 9 10 11 #Inventor V2.1 ascii Separator { Complexity { value 1.0 } Material { ambientColor .8 .5 .4 specularColor .90 .90 .90 shininess 0.4 diffuseColor = Calculator { a = ElapsedTime{}.timeOut expression "oA=vec3f(.8,.3,0.4+0.4*sin(a))" }.oA } Separator { Transform { “report” — 2005/7/31 — 20:15 — page 57 — #73 B.2. IV-file advanced example scaleFactor = DEF MAINCALC Calculator { a = ElapsedTime{}.timeOut expression "oA=vec3f( 1.0+0.1*exp(sin(a/.50))*sin(1.0*a), 1.0+0.1*exp(sin(a/2.0))*cos(2.0*a), 1.0+0.1*exp(cos(a/.75))*sin(2.0*a))" }.oA 12 13 14 15 16 17 } RotationXYZ { axis Y angle 0 = ElapsedTime {} . timeOut } Sphere {} 18 19 20 } Separator { Material { ambientColor 1.0 1.0 1.0 } Transform { translation .8 .5 0 scaleFactor = Calculator { a = ElapsedTime{}.timeOut expression "oA=vec3f( 0.5+0.04*exp(sin(a/.320))*sin(4.00*a), 0.5+0.04*exp(sin(a/.420))*cos(3.30*a), 0.5+0.04*exp(cos(a/.120))*sin(2.50*a))" }.oA } RotationXYZ { axis Z angle 0 = ElapsedTime {} . timeOut } Sphere {} } Separator { Material { ambientColor 1.0 1.0 1.0 } Transform { translation -.8 .5 0 scaleFactor = Calculator { a = ElapsedTime{}.timeOut expression "oA=vec3f( 0.5+0.04*exp(sin(a/.120))*sin(2.00*a), 0.5+0.04*exp(sin(a/.420))*cos(2.20*a), 0.5+0.04*exp(cos(a/.420))*sin(2.40*a))" }.oA } Sphere {} } 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 } 57 “report” — 2005/7/31 — 20:15 — page 58 — #74 58 Appendix B. IV-file viewer and example “report” — 2005/7/31 — 20:15 — page 59 — #75 Appendix C Compact disc Below is the complete directory listing of the contents on the enclosed CD. After that follows a description of the content in the most vital ones. In the base directory lies a HTML-file that links to the HTML-documentation, the report-files and the media files for easy access with a browser. /extras-linux /extras-win32 /ivviewqt /media /media/opendx_movies /media/tensview_movies /media/tensview_screenshots /report /tensview /tensview/build /tensview/build/autoconf_build /tensview/build/build_linux /tensview/build/build_win32 /tensview/build/doc /tensview/build/doc/html /tensview/build/doc/man /tensview/build/doc/man/man3 /tensview/build/src /tensview/demodata /tensview/demodata/sh15br02 /tensview/demodata/syntetic-cylinder /tensview/tensview-fc3-1.0.1 /tensview/tensview-sid-amd64-1.0.1 /tensview/tensview-win32-1.0.1 Rönnbrant, 2005. 59 “report” — 2005/7/31 — 20:15 — page 60 — #76 60 Appendix C. Compact disc /extras-linux Prerequisites for Linux, source for Coin and SoQt libraries. /extras-windows Prerequisites for Windows, installtion files for Coin and the Tiff-image library. /media Various media files, movies and images showing the looks and workings of Tensview. There are also three movies created with OpenDX. /ivviewqt Contains the source for the ivviewqt application and some inventor example files to test it with. /tensview/build Contains source and documentation for Tensview and all other files needed for building Tensview in both Linux and Windows. /report The full report in various formats. A PDF version for viewing and some Postscripts ready for easy printing in both simplex and duplex page mode. /tensview/demodata Data for testing Tensview. The real data from sheep and the synthetic data are available. /tensview/tensview-... Pre-compiled Tenview binaries for Fedora Core 3, Debian Sid pure-64 and Windows. All binaries are dynamically linked and needs some required libraries to be installed. The best and most easy way is to compile Tensview before using it, at least on Linux. “report” — 2005/7/31 — 20:15 — page 61 — #77 Bibliography [1] Edward Angel. Interactive Computer Graphics: A Top-Down Approach using OpenGL (3th Edition). Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 2002. ISBN 0-2017-7343-0. [2] Douglas A. Bernstein, Louis A. Penner, Alison Clarke-Stewart, and Edward J. Roy. Psychology. Houghton-Mifflin, Boston, New York, 6 edition, 2003. [3] Ed Boring and Alex Pang. Interactive deformations from tensor fields. In VIS ’98: Proceedings of the conference on Visualization ’98, pages 297– 304, Los Alamitos, CA, USA, 1998. IEEE Computer Society Press. ISBN 1-5811-3106-2. [4] Allen Cheng, Frank Langer, Filiberto Rodriguez, John C. Criscione, George T. Daughters, D. Craig Miller, and Neil B. Jr. Ingels. Transmural cardiac strains in the lateral wall of the ovine left ventricle. AJP - Heart and Circulatory Physiology, 288:1546–1556, 2005. [5] Kevin D. Costa, Yasuo Takayama, Andrew D. McCulloch, and James W. Covell. Laminar fiber architecture and three-dimensional systolic mechanics in canine ventricular myocardium. AJP - Heart and Circulatory Physiology, 276(2):595–607, February 1999. [6] Jan D’hooge, Fadi Jamal, Bart Bijnens, Jan Thoen, Frans Van de Werf, George R. Sutherland, and Suetens Paul. Two-dimensional ultrasonic strain rate measurement of the human heart in vivo. In Functional Imaging and Modeling of the Heart: First International Workshop - FIMH, pages 47–52, Helsinki, Finland, November 15-16 2001. [7] GNU Octave - a high-level language for numerical computations. Internet, May 2005. URL http://www.octave.org/. 2005-05-27. [8] Katherine B. Harrington, Filiberto Rodriguez, Allen Cheng, Frank Langer, Hiroshi Ashikaga, George T. Daughters, John C. Criscione, Neil B. Jr. Ingels, Rönnbrant, 2005. 61 “report” — 2005/7/31 — 20:15 — page 62 — #78 62 Bibliography and Craig Miller. Direct measurement of transmural laminar architecture on the anterolateral wall of the ovine left ventricle: new implications for wall thickening mechanics. AJP - Heart and Circulatory Physiology, 288:1324– 1330, 2004. [9] Katarina Kindberg. Regional kinematics if the heart: Investigation with marker tracking and with phase contrast magnetic resonance imaging. Master’s thesis, Linköpings universitet, December 2003. LiTH-IMT/BMS20EX–03/354–SE. [10] Katarina Kindberg, Matts Karlsson, N.B. Jr. Ingels, and J.C. Criscione. Nonhomogeneous strain from sparse marker arrays for analysis of transmural myocardial mechanics. Submitted, 2005. [11] I. J. LeGrice, B. H. Smaill, L. Z. Chai, S. G. Edgar, J. B. Gavin, and P. J. Hunter. Laminar structure of the heart: ventricular myocyte arrangement and connective tissue architecture in the dog. AJP - Heart and Circulatory Physiology, 269(2):571–582, 1995. [12] Mathworld. Internet, June 2005. URL http://mathworld.wolfram.com. 2005-06-05. [13] A. D. McCulloch. Cardiac biomechanics. In J. D. Bronzino, editor, The Biomedical Engineering Handbook, pages 418–439. CRC Press, Boca Raton, 1995. ISBN 0-8493-8346-3. [14] Open Visualization Data Explorer. Internet, May 2005. URL http://www.opendx.org/,http://www.research.ibm.com/dx/. 2005-05-27. [15] Pernilla Selskog. Kinematics of the Heart: Strain and Strain-rate using Timeresolved Three-dimensional Phase Contrast MRI. PhD thesis, Linköpings universitet, 2004. [16] A. J. M. Spencer. Continuum mechanics. Longman Inc., New York, 1980. ISBN 0-5824-4282-6. [17] Robert J. Sternberg. Cognitive Psychology. Harcourt Collage Publishers, Orlando, FL, 2 edition, 1999. [18] Gerald J. Tortora and Sandra Reynolds Grabowski. Principles of Anatomy and Physiology. John Wiley and Sons, Inc., New York, 9th edition, 2000. [19] Joise Wernecke. The Inventor Mentor: Programming Object-Oriented 3d Graphics with Open Inventor, Release 2. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 1993. ISBN 0-2016-2495-8. “report” — 2005/7/31 — 20:15 — page 63 — #79 Bibliography 63 [20] Josie Wernecke. The Inventor Toolmaker: Extending Open Inventor, Release 2. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 1994. ISBN 0-2016-2493-1. [21] Wikipedia - Heart. Internet, Apr 2005. URL http://en.wikipedia.org/wiki/Heart. 2005-04-20. [22] Wikipedia - Open Inventor. Internet, June 2005. URL http://en.wikipedia.org/wiki/Open_Inventor. 2005-06-05. [23] Xiaoqiang Zheng and Alex Pang. Volume deformation for tensor visualization. In VIS ’02: Proceedings of the conference on Visualization ’02, Washington, DC, USA, 2002. IEEE Computer Society. ISBN 0-7803-7498-3. “report” — 2005/7/31 — 20:15 — page 64 — #80 64 Bibliography “report” — 2005/7/31 — 20:15 — page 65 — #81 Copyright Svenska Detta dokument hålls tillgängligt på Internet - eller dess framtida ersättare - under en längre tid från publiceringsdatum under förutsättning att inga extra-ordinära omständigheter uppstår. Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner, skriva ut enstaka kopior för enskilt bruk och att använda det oförändrat för ickekommersiell forskning och för undervisning. Överföring av upphovsrätten vid en senare tidpunkt kan inte upphäva detta tillstånd. All annan användning av dokumentet kräver upphovsmannens medgivande. För att garantera äktheten, säkerheten och tillgängligheten finns det lösningar av teknisk och administrativ art. Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman i den omfattning som god sed kräver vid användning av dokumentet på ovan beskrivna sätt samt skydd mot att dokumentet ändras eller presenteras i sådan form eller i sådant sammanhang som är kränkande för upphovsmannens litterära eller konstnärliga anseende eller egenart. För ytterligare information om Linköping University Electronic Press se förlagets hemsida: http://www.ep.liu.se/ English The publishers will keep this document online on the Internet - or its possible replacement - for a considerable time from the date of publication barring exceptional circumstances. The online availability of the document implies a permanent permission for anyone to read, to download, to print out single copies for your own use and to use it unchanged for any non-commercial research and educational purpose. Subsequent transfers of copyright cannot revoke this permission. All other uses of the document are conditional on the consent of the copyright owner. The publisher has taken technical and administrative measures to assure authenticity, security and accessibility. According to intellectual property law the author has the right to be mentioned when his/her work is accessed as described above and to be protected against infringement. For additional information about the Linköping University Electronic Press and its procedures for publication and for assurance of document integrity, please refer to its WWW home page: http://www.ep.liu.se/ c Anders Rönnbrant Linköping, 31st July 2005