Download Hierarchical Volume Visualization

Hierarchical Volume Visualization
for Financial Data
Liew Soon Bin and Edmond C. Prakash
School of Computer Engineering
Nanyang Technological University
Nanyang Avenue
Singapore-639798
[email protected]
[email protected]
Abstract
Financial engineers often use visualization of processed financial data as decision support information.
However, three major weaknesses in this process were identified from the research carried out. It was
found that,
1. very little 3D visualization had been used in the area of financial engineering.
2. visualization of financial data over the Internet was generally not possible.
3. The massive data presented often results in cluttered displays; disjoint displays; and limited access
and lack of expandability.
With the weak areas identified, the following work was carried out to improve the situation.
1. Proposed improvements to a visualization model for the visualization of financial data.
2. Implemented the proposed visualization model with the use of a hierarchical structure.
This paper describes the work done to contribute to the financial engineering community by providing
a visualization tool which is not only able to provide volume visualization of data but also segment the
data set into smaller clusters so that users are able to focus on particular areas of interest. It also has the
potential to be implemented as a distributed application system that supports remote visualization of
financial data through the Internet.
1. Introduction
1.1 Introduction to Data Visualization and Financial Engineering.
The project aims to bring together three very different areas in the fields of data visualization, financial
engineering and the Internet for the benefit of financial engineers.
What is Data Visualization? It is the art and science of turning complicated sets of data into visual
insight and enables people to easily make sense out of what otherwise would just be a set of
meaningless numbers. It is also able to use the power of the human eye and brain to discern
relationships by presenting complex data as multidimensional colour images and animations [Heng97].
In data visualization, the focus is not in trying to squeeze as much information as possible into the
available space, nor is it in trying to represent data in as complex a form as possible. Instead, the focus
should be on finding the most effective way to present useful information in a meaningful fashion.
Since certain visualization patterns will reveal more than others, depending on factors such as the type
of analysis and the amount of data, the need to identify such suitable techniques is very important.
Ultimately, a visualization system should be able to spot patterns, trends or anomalies that were
previously hidden from view.
What is Financial Engineering? It is a very broad term that is used to describe the analysis of data from
the financial market in a scientific manner. Such analysis usually takes the form of mathematical
algorithms or financial models, and they are often used in the different areas of the financial market
with little modification. Currency trading, option pricing and futures are just some examples of these
areas. The use of financial engineering tools and techniques allows financial engineers to understand
the financial market better, and consequently gives them a leverage over other financial traders. This is
very important because the financial market is a fast moving one, and any trading decisions must be
made quickly and accurately.
The need to apply data visualization in financial engineering arises because of the immense size of data
that is present in the financial market. In order for traders to gain an advantage over others, all these
data must be processed, analysed and the results must then be presented in a meaningful fashion that
would assist traders in making their trading decisions. Traditional visualization tools uses 2D charts,
with time on one axis and exchange rates or technical indicators on the other. Unfortunately, even in
deft hands, 2D charts do not reveal all there is to see [Wright95]. New visualization tools are therefore
required to harness additional information from existing data so as to predict trends and patterns better,
to detect anomalies and to gain a better understanding of historical data of the financial market.
1.2 Contributions
The following work has been done as part of this research:
i)
Implementation of a hierarchical structure for volume visualization
ii)
Implementation of the ability to work with a smaller data set to speed up data
manipulation. An additional requirement is portability, as this system is designed for
distribution through the Internet.
2. The Financial Visualization Model
2.1Overview of Data Visualization
Recently, the fast growth of information and the Internet have led to the availability of large volumes
of data. Industries ranging from financial services to telecommunications now use their computers to
collect huge amounts of information every second. As a result they have databases and data
warehouses full of incredibly valuable data. Unfortunately, in many cases, organisations fail to turn this
data into insight that can lead to new discoveries that can improve their competitive position
[Wright95]. This is where data visualization comes into the picture.
Data visualization solves the problem of understanding multivariate data. When data is captured,
independent variables such as time, location, and temperature are typically stored together with each
record. By looking at simple graphs of data values versus an independent variable, complex
interactions and trends may be overlooked. What appears as random data along any one axis, can have
discernible trends when viewed in a multidimensional format. Data visualization therefore uses
techniques such as 3D imaging, colorization, animation, and spatial annotation to extract better
understanding from multivariate data.
3D visualization in particular, enables the use of our most accessible tools for observation and
processing - our eyes and our brains- to pull understanding and insight from an impossibly large
morass of data. By showing data as a 3D landscape, volume graphics or representing data in 3D
topological ordering, we will have a natural ability to understand and comprehend the visualization.
Figure 1 Comparison of 2D Charts and 3D Surfaces of Financial Data
A survey on existing work done on visualization of financial data was carried out to gain a better
understanding of current trends and techniques used. This is necessary in order to be able to identify
weak areas that can then be improved on.
The research activities carried out led to several discoveries and they are summarised as follows. It was
found that,
1.
2.
3.
very little 3D visualization had been used in the area of financial engineering, especially the use of
volume graphics.
visualization of financial data over the Internet was generally not possible.
the massive data presented often results in cluttered displays; disjoint displays; and limited access
and lack of expandability.
There is no doubt that advances are being made in visualization techniques and available visualization
toolkits. There are also countless new financial engineering tools and models being developed.
However, it was found that very few financial engineering tools and models were actually making use
of the advancement in visualization technology. Most tools were still using 2D graphs and charts, with
a few displaying data using 3D surfaces to present the results. By and large however, other forms of 3D
visualization such as volume rendering, has not been fully exploited. Such forms of visualization may
offer further insight on existing data than was previously not possible. Therefore, the application of 3D
visualization techniques such as volume rendering should be explored and demonstrated.
Although the Internet has grown tremendously in recent years, its use in the field of financial
engineering is almost unheard of. Although it is true that the Internet is not useful in the analysis of
financial data, it can nevertheless contribute significantly in the presentation of the results. The Internet
is ideal for remote visualization as well as collaborative data analysis. By supporting remote
visualization, financial engineers will no longer be physically constrained to the location of the central
processing server, and will be able to view the analysis at remote locations using home PCs, mobile
computers and even the new generation of mobile phones. With collaborative data analysis, financial
engineers at different locations will then be able to view the same data, manipulate them and carry out
discussions without being physically close together.
It should be clear now that the three fields discussed so far, financial engineering, data visualization
and the Internet, are not new areas that lacked basic understanding. Each field is quite established on its
own and much research had already been done. New mathematical algorithms and financial models are
being developed for financial engineering. Volume visualization and other advanced visualization
techniques had already been available for quite some time. The Internet is growing exponentially all
these while. However, there lacks a visualization tool which is able to join the 3 fields together.
Currently available financial engineering visualization systems are often localized and isolated pieces
of software that applies financial tools to raw data in order to produce simple 2D graphs and charts.
Remote visualization and collaboration is not possible using such systems. New financial engineering
tools are also being developed without the intention of employing advanced visualization capabilities.
Existing tools are also neglected as visualization of their results using new techniques are not being
explored.
With the weak areas clearly defined, it has become much easier to identify a possible solution. A new
model for the visualization of financial data needs to be developed.
Such a system would then allow financial engineering tools and models to employ the use of both
advanced visualization techniques and the Internet. The potential benefits of using advanced
visualization techniques and the Internet for financial engineering had already been discussed, but it is
important to note that such benefits must be made available to new financial engineering tools and
techniques being developed, as well as existing tools and techniques currently available.
2.2 General Model for Data Visualization
Visualization comes in many forms and serves many different purposes. However, they all have a
common general model, which typically involves raw data that is first processed, before being
presented to the user.
Data
Processing
Raw Data
Visualization
Fig 2 General Model for Data Visualization
Although such a model is also currently used for financial visualization, a more detailed breakdown of
each component is required for a model designed for financial visualization. In particular, the general
model was extended and components specific to financial visualization added. The result is the
Extended Model for Financial Visualization described in the next section. This model was proposed by
Mr Ang Toon Wu, an Honours student from Nanyang Technological University, in May 2000.
2.3 Extended Model for Financial Visualization
Reuters
Live Data
Reuters
Histrorical
Data
Synthetic
Data
2D
graphs
Financial
Engineering
etc
Volume
Graphics
Exponential Moving
Average
2D graphs, bars
Scale of Market Shocks
Iso-surf ace
Lookup tables
Fractal Scaling
Volume graphics
RGBA Colour
Maps
etc...
etc...
Technical Indicators/
Financial Models
Data Source
Visualization
Techniques
Data Processing
Visual Models
Visualization Processing
Rendering
Fig 3 Extended Model for Financial Visualization
2.4 Rendering
The final step in the visualization process is the actual presentation of the data in the form processed
earlier. This is the focus of the present project. The visualization of the data was done using Java with
the VisAD class library. This provided for possibility of visualization over the Internet and also
platform independence is achieved. The VisAD system's general data model and thorough use of Java
RMI provide a way to build a shared, active network of scientific data, display and computations. This
network could:
1. Change dynamically.
2. Have many simultaneous users with their own sets of display and user interface objects.
3. Have an indefinite life span, with users connecting and disconnecting but the basic network
remaining.
4. Support numerous interacting execution threads.
5. Provide entrance points via web pages.
For certain visualization techniques, especially volume rendering, additional factors need to be
considered, as they will also affect how the final output will look like. The problem with volume
graphics is that certain parts of the volume will be blocked from view and the area may be the area of
interest to the user. This is the problem which we hope to solve in the current project.
The visualization model that was originally proposed by Ang Toon Wu should be able to visualize
existing technical indicators and financial models, and also allows extension of these tools with relative
ease. However, there were several drawbacks with the visualization model proposed.
They are:
1. Functionality for focusing on smaller parts of data set not efficient.
2. As a volume is generated from the data sets, certain parts of the volume would be blocked from
view.
3. Speed for manipulation of data was much slower as we always had to work with a full data set.
For this project, to solve the above problems, the decision was made to implement the following:
1. Hierarchical structure for the 3D visualization.
2. Ability to work with a new Data set which is smaller.
3. Implementation of Hierachical Volume Visualisation for Financial Data
Recently, the fast growth of information and the Internet have led to the availability of large volumes
of data. This is especially true for data used in the financial sector. As it is the wish of many financial
analysts to detect trends and patterns through the financial data available in order to make predictions,
it is definitely necessary to have a large set of data to work on.
However, current information visualization systems are designed to handle moderate amounts of
structured data. New information visualization systems will be built around the navigation of, and
interaction with, massive volumes of unstructured information. The challenge in the design of
information visualization systems is actually to find methods for presenting valuable information from
large volumes of data so as to enable a user to quickly identify exceptions and to distinguish interesting
patterns visually. There are some issues in today's visual mining of massive volumes of data:
(1)cluttered display; (2)disjoint displays; and (3)limited access and lack of expandability.
Information visualization based on a single complex view often causes display clutter and visual
confusion. Besides, single view visualization does not allow users to visualize the inter-relationships
among different sets of high-dimensional data. A common solution to provide multiple views is to use
many displays. But users have to click through display after display to find the information. For
example, in a telecom switch mining application, suppose a user wants to selectively monitor
overloaded telephone links in a certain country. Starting with a map of the country, the user would need
to click through each display of progressively greater detail (at the state level, at the city level, etc)
until the user finds the overloaded links. With multiple views, the user can see presentations at different
levels of detail simultaneously to identify the problem real time.
[Ming99] At HP Laboratories, they have devised some visualization solutions to solve the above
difficulties. The first idea is to hide visual relationships and structure to reduce display cluttering and
visual confusion. This method hides all non-primary relationships; it only shows objects when the user
focuses on them. All other structures and relationships are hidden in the property of each object. The
second idea is to directly interact with the user and mining engines to slice and dice large complex
knowledge into multiple simultaneous presentations. This method allows a user to easily discover
knowledge relationships and exceptions. The third idea is to define new visual interfaces to plug into
existing graphic toolkits, such as TGS' 3DMSJava and Inxight's Hyperbolic Tree Toolkits, thus
expanding the use of their visualization infrastructure to a wide variety of visual applications. These
solutions are driven by information content.
The similar problem of dealing with increasingly fast stores of information was also dealt with in
another research paper "Discovery Visualization Using Fast Clustering"[William99]. The paper
discusses about burrowing down into a data collection, extracting features, then unfolding those
features to reveal inner details. The authors discuss the method of clustering. Spatial clustering
partitions a set of n data points into m subsets so as to minimize the sum of distances (or a similar
metric) between each data point and the center of its clusters. Clustering achieves simplification by
replacing all the points in a cluster with a single, average point at the center. Clustering can be very
computationally expensive. There exists kn/k! ways to assign n points to k clusters. Choosing the
optimal clustering method represents a problem. A straight clustering may not bring out certain features
very well. Certain collections of clusters may not represent certain shapes well without a rather large
number of clusters. However, if users had a highly interactive means to change the number of clusters
and thus the amount of detail in the clustering, the lack of optimality in describing certain shapes would
become less of an issue. Users could easily adjust the detail level for any feature of interest (assuming
they had an effective mode of interaction and display). A complete visual mining approach needs a
structure that supports a highly interactive exploration and discovery process for data of any scale. The
structure must also support fast queries and collection of data, where necessary. An appropriate
hierarchical structure can fulfill these needs. However the hierarchical structure must be designed in a
way appropriate to visual data mining, which means supporting rapid display and providing the data in
the appropriate context. (For example, a query does not just return a piece of data but rather returns that
data so that it can be displayed in relation to other data.)
It was decided that the idea to directly interact with the user and mining engines to slice and dice large
complex knowledge into multiple simultaneous presentations to be used for this project. The concept of
a hierarchical structure will also be adopted. By segmenting the data into several sets, users would also
be able to see within the volume generated which would otherwise be blocked originally.
4. Results of Implementation
Several sets of data were used to test the program developed. The data sets were generated using the
Black-Scholes Option Pricing model. The 3 variables compared are Price, Days and Value. Each data
set keeps track of a portfolio of stocks. Price represents the price of the portfolio, Days represent the
day where the value is taken while the Value represents the value of the portfolio in relation to the price
of another item at the particular time. The 3D graphics and its 3 subsets generated for 4 sets of the test
data are shown in the following pages. Each graph has been sliced into 3 subsets and each subset
represents 1/3 of the overall data set.
Fig 4 Stock Potfolio 1 Data Set generated
Subset 1 of Fig 4
Subset 2 of Fig 4
Subset 3 of Fig 4
Fig 4 Stock Portfolio 2 Data Set generated
Subset 1 of Fig 4
Subset 2 of Fig 4
Subset 3 of Fig 4
The user is now able to work on smaller clusters of the data set and focus on his area of interest. This
also improves the performance of the rendering of the volume as a smaller set of data is generated.
4. Conclusion
Based on the findings from the research activities that were carried out, three major drawbacks in the
system of visualization of financial data were identified. With a set of goals and directions determined,
work was carried out to improve the situation by proposing a new model for the visualization of
financial data. Such a system was also implemented and demonstrated.
To demonstrate how advanced visualization techniques such as volume rendering could benefit
financial engineering tools and techniques, a program was developed to do 3D visualization of
financial data generated using the Black-Scholes Option Pricing model.The added functionality to
segment the large data set to smaller clusters was also implemented in the model. This thus allows the
users to focus on their area of interest and avoid blocking of certain internal data due to the generated
volume. Rendering would also be speeded up as a smaller data set is dealt with. Thus hierachical
volume visualization of financial data is achieved.
References
[Bishop92]
Bishop P, and Dixon D, Foreign Exchange Handbook,
McGraw-Hill, 1992[5].
[Black73]
Black F, Scholes M, The Pricing of Options and Corporate Liabilities,
Journal of Political Economy Vol.81 No.3, 1973.
[Galely96]
Gately E, Neural Networks for Financial Forecasting,
John Wiley & Sons Inc, 1996
[3].
[Gencay98]
Gencay R, Ballocchi G, Dacorogna M, Olsen R, Pictet O, Real-Time Trading Models
and the Statistical Properties of Foreign Exchange Rates, Olsen & Associates,
December 1998.
[Heng97]
Heng P H, Ng T T, and Ng T E, Visualization of Financial Instruments,
Applied Research Project for Nanyang Business School, 1997[4].
[Hibbard99]
Hibbard B, Volume Rendering for VisAD,
http://www.unidata.ucar.edu/glimpse/visad-list/527, 1999.
[Laszlo97]
Laszlo S, Garbor M, Tibor F, Tamas H, Development of Process
Visualization Systems: An Object-Oriented Approach,
Journal of Systems Architecture 46(2000)275-296, 1999.
[Laurent96]
Laurent L J, Management and Control of Foreign Exchange Risk,
Kluwer Academic Publishers, 1996 [6].
[Ming99]
Ming C. Hao, Umesh Dayal, Meichun Hsu, Jim Baker, Robert D'Eletto, A Javabased Visual Mining Infrastructure and Applications,
IEEE 0-7695-0431-0/99, 1999.
[Piche95]
Piche S.W., Trend Visualization,
IEEE Comp Intelligence 0-7803-2145-6/95, 1995.
[Walmsley96]
Walmsley J, International Money and Foreign Exchange Markets,
John Wiley & Sons Ltd, 1996[1].
[Wdorf98]
Wasendorf R, Foreign Currency Trading,
McGraw-Hill, 1998.
[William99]
William Ribarsky, Jochen Katz, Frank Jiang, and Aubrey Holland, Discovery
Visualization Using Fast Clustering
IEEE Computer Graphics Application 0272-1716/99, 1999.
[Wright95]
Wright W., Information Animation Applications in the Capital Markets, IEEE Info
Visualization 0-8186-7201-3/95, 1995.
[Zukowski98]
John Zukowski, Mastering Java 1.2,
Sybex, 1998.
[Zumbach98]
Zumbach G.O., Dacorogna M.M., Olsen J.L. and Olsen R.B.,
Introducing a Scale of Market shocks,
Olsen & Associates GOZ 1998-10-1, 1998.