Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Hierarchical Volume Visualization for Financial Data Liew Soon Bin and Edmond C. Prakash School of Computer Engineering Nanyang Technological University Nanyang Avenue Singapore-639798 [email protected] [email protected] Abstract Financial engineers often use visualization of processed financial data as decision support information. However, three major weaknesses in this process were identified from the research carried out. It was found that, 1. very little 3D visualization had been used in the area of financial engineering. 2. visualization of financial data over the Internet was generally not possible. 3. The massive data presented often results in cluttered displays; disjoint displays; and limited access and lack of expandability. With the weak areas identified, the following work was carried out to improve the situation. 1. Proposed improvements to a visualization model for the visualization of financial data. 2. Implemented the proposed visualization model with the use of a hierarchical structure. This paper describes the work done to contribute to the financial engineering community by providing a visualization tool which is not only able to provide volume visualization of data but also segment the data set into smaller clusters so that users are able to focus on particular areas of interest. It also has the potential to be implemented as a distributed application system that supports remote visualization of financial data through the Internet. 1. Introduction 1.1 Introduction to Data Visualization and Financial Engineering. The project aims to bring together three very different areas in the fields of data visualization, financial engineering and the Internet for the benefit of financial engineers. What is Data Visualization? It is the art and science of turning complicated sets of data into visual insight and enables people to easily make sense out of what otherwise would just be a set of meaningless numbers. It is also able to use the power of the human eye and brain to discern relationships by presenting complex data as multidimensional colour images and animations [Heng97]. In data visualization, the focus is not in trying to squeeze as much information as possible into the available space, nor is it in trying to represent data in as complex a form as possible. Instead, the focus should be on finding the most effective way to present useful information in a meaningful fashion. Since certain visualization patterns will reveal more than others, depending on factors such as the type of analysis and the amount of data, the need to identify such suitable techniques is very important. Ultimately, a visualization system should be able to spot patterns, trends or anomalies that were previously hidden from view. What is Financial Engineering? It is a very broad term that is used to describe the analysis of data from the financial market in a scientific manner. Such analysis usually takes the form of mathematical algorithms or financial models, and they are often used in the different areas of the financial market with little modification. Currency trading, option pricing and futures are just some examples of these areas. The use of financial engineering tools and techniques allows financial engineers to understand the financial market better, and consequently gives them a leverage over other financial traders. This is very important because the financial market is a fast moving one, and any trading decisions must be made quickly and accurately. The need to apply data visualization in financial engineering arises because of the immense size of data that is present in the financial market. In order for traders to gain an advantage over others, all these data must be processed, analysed and the results must then be presented in a meaningful fashion that would assist traders in making their trading decisions. Traditional visualization tools uses 2D charts, with time on one axis and exchange rates or technical indicators on the other. Unfortunately, even in deft hands, 2D charts do not reveal all there is to see [Wright95]. New visualization tools are therefore required to harness additional information from existing data so as to predict trends and patterns better, to detect anomalies and to gain a better understanding of historical data of the financial market. 1.2 Contributions The following work has been done as part of this research: i) Implementation of a hierarchical structure for volume visualization ii) Implementation of the ability to work with a smaller data set to speed up data manipulation. An additional requirement is portability, as this system is designed for distribution through the Internet. 2. The Financial Visualization Model 2.1Overview of Data Visualization Recently, the fast growth of information and the Internet have led to the availability of large volumes of data. Industries ranging from financial services to telecommunications now use their computers to collect huge amounts of information every second. As a result they have databases and data warehouses full of incredibly valuable data. Unfortunately, in many cases, organisations fail to turn this data into insight that can lead to new discoveries that can improve their competitive position [Wright95]. This is where data visualization comes into the picture. Data visualization solves the problem of understanding multivariate data. When data is captured, independent variables such as time, location, and temperature are typically stored together with each record. By looking at simple graphs of data values versus an independent variable, complex interactions and trends may be overlooked. What appears as random data along any one axis, can have discernible trends when viewed in a multidimensional format. Data visualization therefore uses techniques such as 3D imaging, colorization, animation, and spatial annotation to extract better understanding from multivariate data. 3D visualization in particular, enables the use of our most accessible tools for observation and processing - our eyes and our brains- to pull understanding and insight from an impossibly large morass of data. By showing data as a 3D landscape, volume graphics or representing data in 3D topological ordering, we will have a natural ability to understand and comprehend the visualization. Figure 1 Comparison of 2D Charts and 3D Surfaces of Financial Data A survey on existing work done on visualization of financial data was carried out to gain a better understanding of current trends and techniques used. This is necessary in order to be able to identify weak areas that can then be improved on. The research activities carried out led to several discoveries and they are summarised as follows. It was found that, 1. 2. 3. very little 3D visualization had been used in the area of financial engineering, especially the use of volume graphics. visualization of financial data over the Internet was generally not possible. the massive data presented often results in cluttered displays; disjoint displays; and limited access and lack of expandability. There is no doubt that advances are being made in visualization techniques and available visualization toolkits. There are also countless new financial engineering tools and models being developed. However, it was found that very few financial engineering tools and models were actually making use of the advancement in visualization technology. Most tools were still using 2D graphs and charts, with a few displaying data using 3D surfaces to present the results. By and large however, other forms of 3D visualization such as volume rendering, has not been fully exploited. Such forms of visualization may offer further insight on existing data than was previously not possible. Therefore, the application of 3D visualization techniques such as volume rendering should be explored and demonstrated. Although the Internet has grown tremendously in recent years, its use in the field of financial engineering is almost unheard of. Although it is true that the Internet is not useful in the analysis of financial data, it can nevertheless contribute significantly in the presentation of the results. The Internet is ideal for remote visualization as well as collaborative data analysis. By supporting remote visualization, financial engineers will no longer be physically constrained to the location of the central processing server, and will be able to view the analysis at remote locations using home PCs, mobile computers and even the new generation of mobile phones. With collaborative data analysis, financial engineers at different locations will then be able to view the same data, manipulate them and carry out discussions without being physically close together. It should be clear now that the three fields discussed so far, financial engineering, data visualization and the Internet, are not new areas that lacked basic understanding. Each field is quite established on its own and much research had already been done. New mathematical algorithms and financial models are being developed for financial engineering. Volume visualization and other advanced visualization techniques had already been available for quite some time. The Internet is growing exponentially all these while. However, there lacks a visualization tool which is able to join the 3 fields together. Currently available financial engineering visualization systems are often localized and isolated pieces of software that applies financial tools to raw data in order to produce simple 2D graphs and charts. Remote visualization and collaboration is not possible using such systems. New financial engineering tools are also being developed without the intention of employing advanced visualization capabilities. Existing tools are also neglected as visualization of their results using new techniques are not being explored. With the weak areas clearly defined, it has become much easier to identify a possible solution. A new model for the visualization of financial data needs to be developed. Such a system would then allow financial engineering tools and models to employ the use of both advanced visualization techniques and the Internet. The potential benefits of using advanced visualization techniques and the Internet for financial engineering had already been discussed, but it is important to note that such benefits must be made available to new financial engineering tools and techniques being developed, as well as existing tools and techniques currently available. 2.2 General Model for Data Visualization Visualization comes in many forms and serves many different purposes. However, they all have a common general model, which typically involves raw data that is first processed, before being presented to the user. Data Processing Raw Data Visualization Fig 2 General Model for Data Visualization Although such a model is also currently used for financial visualization, a more detailed breakdown of each component is required for a model designed for financial visualization. In particular, the general model was extended and components specific to financial visualization added. The result is the Extended Model for Financial Visualization described in the next section. This model was proposed by Mr Ang Toon Wu, an Honours student from Nanyang Technological University, in May 2000. 2.3 Extended Model for Financial Visualization Reuters Live Data Reuters Histrorical Data Synthetic Data 2D graphs Financial Engineering etc Volume Graphics Exponential Moving Average 2D graphs, bars Scale of Market Shocks Iso-surf ace Lookup tables Fractal Scaling Volume graphics RGBA Colour Maps etc... etc... Technical Indicators/ Financial Models Data Source Visualization Techniques Data Processing Visual Models Visualization Processing Rendering Fig 3 Extended Model for Financial Visualization 2.4 Rendering The final step in the visualization process is the actual presentation of the data in the form processed earlier. This is the focus of the present project. The visualization of the data was done using Java with the VisAD class library. This provided for possibility of visualization over the Internet and also platform independence is achieved. The VisAD system's general data model and thorough use of Java RMI provide a way to build a shared, active network of scientific data, display and computations. This network could: 1. Change dynamically. 2. Have many simultaneous users with their own sets of display and user interface objects. 3. Have an indefinite life span, with users connecting and disconnecting but the basic network remaining. 4. Support numerous interacting execution threads. 5. Provide entrance points via web pages. For certain visualization techniques, especially volume rendering, additional factors need to be considered, as they will also affect how the final output will look like. The problem with volume graphics is that certain parts of the volume will be blocked from view and the area may be the area of interest to the user. This is the problem which we hope to solve in the current project. The visualization model that was originally proposed by Ang Toon Wu should be able to visualize existing technical indicators and financial models, and also allows extension of these tools with relative ease. However, there were several drawbacks with the visualization model proposed. They are: 1. Functionality for focusing on smaller parts of data set not efficient. 2. As a volume is generated from the data sets, certain parts of the volume would be blocked from view. 3. Speed for manipulation of data was much slower as we always had to work with a full data set. For this project, to solve the above problems, the decision was made to implement the following: 1. Hierarchical structure for the 3D visualization. 2. Ability to work with a new Data set which is smaller. 3. Implementation of Hierachical Volume Visualisation for Financial Data Recently, the fast growth of information and the Internet have led to the availability of large volumes of data. This is especially true for data used in the financial sector. As it is the wish of many financial analysts to detect trends and patterns through the financial data available in order to make predictions, it is definitely necessary to have a large set of data to work on. However, current information visualization systems are designed to handle moderate amounts of structured data. New information visualization systems will be built around the navigation of, and interaction with, massive volumes of unstructured information. The challenge in the design of information visualization systems is actually to find methods for presenting valuable information from large volumes of data so as to enable a user to quickly identify exceptions and to distinguish interesting patterns visually. There are some issues in today's visual mining of massive volumes of data: (1)cluttered display; (2)disjoint displays; and (3)limited access and lack of expandability. Information visualization based on a single complex view often causes display clutter and visual confusion. Besides, single view visualization does not allow users to visualize the inter-relationships among different sets of high-dimensional data. A common solution to provide multiple views is to use many displays. But users have to click through display after display to find the information. For example, in a telecom switch mining application, suppose a user wants to selectively monitor overloaded telephone links in a certain country. Starting with a map of the country, the user would need to click through each display of progressively greater detail (at the state level, at the city level, etc) until the user finds the overloaded links. With multiple views, the user can see presentations at different levels of detail simultaneously to identify the problem real time. [Ming99] At HP Laboratories, they have devised some visualization solutions to solve the above difficulties. The first idea is to hide visual relationships and structure to reduce display cluttering and visual confusion. This method hides all non-primary relationships; it only shows objects when the user focuses on them. All other structures and relationships are hidden in the property of each object. The second idea is to directly interact with the user and mining engines to slice and dice large complex knowledge into multiple simultaneous presentations. This method allows a user to easily discover knowledge relationships and exceptions. The third idea is to define new visual interfaces to plug into existing graphic toolkits, such as TGS' 3DMSJava and Inxight's Hyperbolic Tree Toolkits, thus expanding the use of their visualization infrastructure to a wide variety of visual applications. These solutions are driven by information content. The similar problem of dealing with increasingly fast stores of information was also dealt with in another research paper "Discovery Visualization Using Fast Clustering"[William99]. The paper discusses about burrowing down into a data collection, extracting features, then unfolding those features to reveal inner details. The authors discuss the method of clustering. Spatial clustering partitions a set of n data points into m subsets so as to minimize the sum of distances (or a similar metric) between each data point and the center of its clusters. Clustering achieves simplification by replacing all the points in a cluster with a single, average point at the center. Clustering can be very computationally expensive. There exists kn/k! ways to assign n points to k clusters. Choosing the optimal clustering method represents a problem. A straight clustering may not bring out certain features very well. Certain collections of clusters may not represent certain shapes well without a rather large number of clusters. However, if users had a highly interactive means to change the number of clusters and thus the amount of detail in the clustering, the lack of optimality in describing certain shapes would become less of an issue. Users could easily adjust the detail level for any feature of interest (assuming they had an effective mode of interaction and display). A complete visual mining approach needs a structure that supports a highly interactive exploration and discovery process for data of any scale. The structure must also support fast queries and collection of data, where necessary. An appropriate hierarchical structure can fulfill these needs. However the hierarchical structure must be designed in a way appropriate to visual data mining, which means supporting rapid display and providing the data in the appropriate context. (For example, a query does not just return a piece of data but rather returns that data so that it can be displayed in relation to other data.) It was decided that the idea to directly interact with the user and mining engines to slice and dice large complex knowledge into multiple simultaneous presentations to be used for this project. The concept of a hierarchical structure will also be adopted. By segmenting the data into several sets, users would also be able to see within the volume generated which would otherwise be blocked originally. 4. Results of Implementation Several sets of data were used to test the program developed. The data sets were generated using the Black-Scholes Option Pricing model. The 3 variables compared are Price, Days and Value. Each data set keeps track of a portfolio of stocks. Price represents the price of the portfolio, Days represent the day where the value is taken while the Value represents the value of the portfolio in relation to the price of another item at the particular time. The 3D graphics and its 3 subsets generated for 4 sets of the test data are shown in the following pages. Each graph has been sliced into 3 subsets and each subset represents 1/3 of the overall data set. Fig 4 Stock Potfolio 1 Data Set generated Subset 1 of Fig 4 Subset 2 of Fig 4 Subset 3 of Fig 4 Fig 4 Stock Portfolio 2 Data Set generated Subset 1 of Fig 4 Subset 2 of Fig 4 Subset 3 of Fig 4 The user is now able to work on smaller clusters of the data set and focus on his area of interest. This also improves the performance of the rendering of the volume as a smaller set of data is generated. 4. Conclusion Based on the findings from the research activities that were carried out, three major drawbacks in the system of visualization of financial data were identified. With a set of goals and directions determined, work was carried out to improve the situation by proposing a new model for the visualization of financial data. Such a system was also implemented and demonstrated. To demonstrate how advanced visualization techniques such as volume rendering could benefit financial engineering tools and techniques, a program was developed to do 3D visualization of financial data generated using the Black-Scholes Option Pricing model.The added functionality to segment the large data set to smaller clusters was also implemented in the model. This thus allows the users to focus on their area of interest and avoid blocking of certain internal data due to the generated volume. Rendering would also be speeded up as a smaller data set is dealt with. Thus hierachical volume visualization of financial data is achieved. References [Bishop92] Bishop P, and Dixon D, Foreign Exchange Handbook, McGraw-Hill, 1992[5]. [Black73] Black F, Scholes M, The Pricing of Options and Corporate Liabilities, Journal of Political Economy Vol.81 No.3, 1973. [Galely96] Gately E, Neural Networks for Financial Forecasting, John Wiley & Sons Inc, 1996 [3]. [Gencay98] Gencay R, Ballocchi G, Dacorogna M, Olsen R, Pictet O, Real-Time Trading Models and the Statistical Properties of Foreign Exchange Rates, Olsen & Associates, December 1998. [Heng97] Heng P H, Ng T T, and Ng T E, Visualization of Financial Instruments, Applied Research Project for Nanyang Business School, 1997[4]. [Hibbard99] Hibbard B, Volume Rendering for VisAD, http://www.unidata.ucar.edu/glimpse/visad-list/527, 1999. [Laszlo97] Laszlo S, Garbor M, Tibor F, Tamas H, Development of Process Visualization Systems: An Object-Oriented Approach, Journal of Systems Architecture 46(2000)275-296, 1999. [Laurent96] Laurent L J, Management and Control of Foreign Exchange Risk, Kluwer Academic Publishers, 1996 [6]. [Ming99] Ming C. Hao, Umesh Dayal, Meichun Hsu, Jim Baker, Robert D'Eletto, A Javabased Visual Mining Infrastructure and Applications, IEEE 0-7695-0431-0/99, 1999. [Piche95] Piche S.W., Trend Visualization, IEEE Comp Intelligence 0-7803-2145-6/95, 1995. [Walmsley96] Walmsley J, International Money and Foreign Exchange Markets, John Wiley & Sons Ltd, 1996[1]. [Wdorf98] Wasendorf R, Foreign Currency Trading, McGraw-Hill, 1998. [William99] William Ribarsky, Jochen Katz, Frank Jiang, and Aubrey Holland, Discovery Visualization Using Fast Clustering IEEE Computer Graphics Application 0272-1716/99, 1999. [Wright95] Wright W., Information Animation Applications in the Capital Markets, IEEE Info Visualization 0-8186-7201-3/95, 1995. [Zukowski98] John Zukowski, Mastering Java 1.2, Sybex, 1998. [Zumbach98] Zumbach G.O., Dacorogna M.M., Olsen J.L. and Olsen R.B., Introducing a Scale of Market shocks, Olsen & Associates GOZ 1998-10-1, 1998.