Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Bond University ePublications@bond Information Technology papers Bond Business School December 2003 A Survey of the Application of Soft Computing to Investment and Financial Trading Bruce Vanstone Bond University, [email protected] Clarence Tan Bond University, [email protected] Follow this and additional works at: http://epublications.bond.edu.au/infotech_pubs Recommended Citation Bruce Vanstone and Clarence Tan. "A Survey of the Application of Soft Computing to Investment and Financial Trading" Dec. 2003. http://epublications.bond.edu.au/infotech_pubs/13 This Conference Proceeding is brought to you by the Bond Business School at ePublications@bond. It has been accepted for inclusion in Information Technology papers by an authorized administrator of ePublications@bond. For more information, please contact Bond University's Repository Coordinator. A Survey of the Application of Soft Computing to Investment and Financial Trading Bruce Vanstone School of IT, Bond University Gold Coast, Queensland, Australia [email protected] Clarence Tan School of IT, Bond University Gold Coast, Queensland, Australia [email protected] Abstract istic Reasoning (which includes Genetic Algorithms, Chaos Theory, etc). The arena of investment trading is one such field where there is an abundance of noisy data. It is in this area that traditional computing typically gives way to soft computing, as the rigid conditions applied by traditional computing cannot be met. This is particularly evident where the same sets of input conditions may appear to invoke different outcomes, or there is an abundance of missing or poor quality data. This paper surveys recent literature in the domain of applying Soft Computing to Investment and Financial Trading. It analyses the literature according to the style of soft computing used, the investment discipline used, the successes demonstrated, and the applicability of the research to real world trading. This papers contribution is to expose the key areas where research is being undertaken, and to attempt to quantify the degree of successes associated with the different research approaches. 1 There are a number of approaches within the literatures which deal with applying soft computing techniques to investment and trading. Although there appears to be no formal segmentation of these different approaches, this paper classifies the literature into the topics proposed by Tan [1], and augments these classifications with one more category, namely, Hybrid. These categories of soft computing, then, are: Introduction This paper surveys the literature on the utilization and successes of Soft Computing techniques in the Investment arena. Where applicable, it suggests improvements to the methodologies used to increase their practicality. Investment trading is normally divided into two major disciplines, Fundamental Analysis, and Technical Analysis. Papers concerned with applying Soft Computing to these two disciplines are reviewed. x Time Series – forecasting future data points using historical data sets. Research reviewed in this area generally attempts to predict the future values of some time series. Possible time series include: Base time series data (e.g. Closing Prices), or time series derived from base data, (e.g. Indicators - frequently used in Technical Analysis). x Pattern Recognition and Classification – attempts to classify observations into categories, generally by learning patterns in the data. Research reviewed in this area involved the detection of patterns, and segregation of base data into ‘winner’ and ‘loser’ categories. x Optimization – involves solving problems where patterns in the data are not known, often non-polynomial (NP)-complete problems. Research reviewed in this area covered the optimal selection of parameters, and determining the optimal point at which to enter transactions. x Hybrid – this category was used to distinguish research which attempted to exploit the synergy effect by combining more than one of the above styles. There is a wide acceptance of the benefit of the synergy effect, whereby the whole is seen as being greater than the sum of the individual parts. This can be easily seen by inspecting Table 1, which clearly shows that Hybrid techniques account for more than one-third of the research reviewed. The papers reviewed are assessed and categorized according to the style of soft computing used, their investment discipline, the degree to which they demonstrate their success, as well as their practicality for use as real world trading models. 2 2.1 Analysis of Methodologies Used Analysis by Soft Computing Technique Soft computing represents that area of computing adapted from the physical sciences. Artificial Intelligence techniques within this realm attempt to solve problems by applying physical laws and processes. This style of computing is particularly tolerant of imprecision and uncertainty, making the approach attractive to those researching within ‘noisy’ realms, where the signal-to-noise ratio is quite low. Soft computing is normally accepted to include the three key areas of Fuzzy Logic, Neural Networks, and Probabil- 211 Table 2. Reviewed papers classified by Investment Discipline Further, the bias in this style of research towards technical analysis techniques (see below) is also evident from the table, with one-third of the research pursuing the area of Pattern Recognition and Classification. Technical analysis particularly lends itself to this style of research, as a large focus of technical analysis concerns the detection of patterns in data, and the examination of the behavior of market participants when these patterns are manifest. Table 1. Reviewed papers classified by style of soft computing used Research Number Paper Classification of papers References Time Series Prediction 6 [2], [3], [4], [5], [6], [7] Pattern Recognition & Classification 11 [8], [9], [10], [11], [12], [13], [14], [15], [16], [17], [18] Optimization 2 [19], [20] Hybrid 11 [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31] 2.2 Research Number Paper Classification of papers References Fundamental Analysis 3 [29], [7], [13] Technical Analysis 23 [20], [19], [28], [27], [26], [25], [24], [23], [22], [21], [31], [30], [5], [4], [2], [18], [16], [15], [14], [12], [10], [9], [8] Combined 4 [6], [3], [11], [17] Although many practitioners feel a strong divide between Fundamental techniques, and Technical techniques, there appears to be a growing acceptance amongst the investment community that the techniques may be viewed as complementary. Indeed, as earlier as 1992, Taylor and Allen conducted a survey on behalf of the Bank of England, and their findings indicated that a high percentage of chief foreign exchange dealers already considered fundamental and technical techniques as complementary strategies, preferring fundamental analysis for long-term horizons, and technical analysis for shorter term horizons [32]. Analysis by Investment Discipline There are predominantly two schools of thought concerning investment trading, namely Fundamental Analysis and Technical Analysis. Fundamental Analysis provides a framework for examining the underlying forces that affect the price of an investment. Fundamental analysis techniques give rise to investment approaches like Value Investment. This style of investment espouses the view that investing is the process of determining the fair value of an individual investment, and purchasing that investment at or below that fair value. Research using soft computing techniques is particularly data intensive, as soft computing techniques generally require a large set of data on which to learn. This may go some way towards explaining the apparent large bias of the research of this nature to focus on technical analysis. Typically, the technical analysis data used in the majority of research is daily data, which is easily obtained and available in large quantities. Conversely, the data normally used for fundamental analysis is yearly data, often sourced from company balance sheets, or other accounting publications. The other main school of thought concerning investment trading is Technical Analysis. This style of trading is concerned primarily with the study of past trading data, believing this is reflecting the behavior of market participants. Such trading data includes prices, volume and open interest. 3 Both disciplines are concerned with attempting to predict likely future prices movements, and seek to capitalize on these predictions with trading models. Analysis of Investment Performance There are a number of measures of success in common use throughout the research reviewed. Soft computing achieves its benefits from a learning process, which generally allows the specific soft computing techniques the ability to repeatedly assess their own learning using some form of feedback mechanism. This lends itself to the creation of an error function, and most forms of soft computing attempt to minimize this error function. As the value of the error function is decreased, the soft computing technique is This paper classifies the reviewed literature in one of three categories, namely Fundamental, Technical, or Combined, with Combined representing some mix between the two techniques. Table 2 presents the results of this classification for the papers reviewed. 212 said to be learning the relationship between the various inputs and outputs provided. Although this is an excellent way of providing a soft computing technique with the ability to assess its own degree of learning, it does not necessarily deliver results consistent with the expectations of the trainer. In the area of investment trading, a trainer typically wants to maximize returns, or minimize time in the market, or some other specific attribute of the domain being studied. This leads to something akin to an impedance mismatch, which may result in a trained process delivering a poor outcome in its specific domain. This problem may also result from the trainer attempting to predict something too specific, such as the price of an individual investment, when the trainer may well have achieved a far better result by attempting to predict only the direction of the price movement, as suggested by Tan [1]. In the case of Mizuno, Kosaka et al. [9] which did not outperform the buy-and-hold strategy overall, the results presented demonstrate that the system developed did exceed the buy-and-hold strategy for buy situations. However, performance achieved during sell situations was less than optimal. Overall, the combined results achieved a slightly lower rate of profit for the soft computing approach (1.20) compared with a rate of profit (1.21) for the buy-and-hold strategy. The authors propose that this could be due to the increasing upward trend in price change throughout the simulation period. Table 4 below categorizes those papers which do not lend themselves to the primary classification of increasing investment returns. These papers occasionally rate their own performance by comparison to statistical techniques, or by comparison to the results which would be obtained without the use of soft computing, for example, by applying technical analysis or fundamental analysis alone. More often, however, there is a tendency to conclude with statements of a broadly positive nature. It should be mentioned that much of the work in the area of applying soft computing to investment trading has been of an evolutionary approach. That is, much of the earlier work was concerned only with establishing whether a particular soft computing approach could outperform some corresponding regression technique. It was some time before soft computing firmly established itself, and the research involving forecasting was mature. Until this point had been reached, it was not common to continue on and implement the particular technique as a trading mechanism. Table 4. Benchmarking performance comparator other than Buy-and-Hold Research Classification Result1 For the purposes of this review, the research is classified according to what is assumed to be the most likely desired criteria of success, namely, the ability to increase investment returns above and beyond that achieved by using a buy-and-hold naïve approach. Table 3 shows whether the papers reviewed outperformed the naïve strategy, or whether their performance was classified in another way. Comparison to Statistical Technique Comparison to same technique without soft computing Table 3. Benchmark performance against the buy-andhold approach Other Research Number Paper Classification of papers References Outperforms Buyand-Hold 4 [27], [6], [12], [18] Does not outperform Buy-andHold 1 [9] See Table 4 25 4 [19], [20], [29], [30], [21], [31], [22], [23], [24], [25], [26], [28], [4], [7], [2], [3], [5], [8], [10], [13], [14], [15], [16], [17], [11] using Number Paper of papers References 1 1 [7] 8 8 [19], [20], [29], [22], [25], [26], [2], [3] N/A 16 [30], [21], [31], [23], [24], [28], [4], [5], [8], [10], [13], [14], [15], [16], [17], [11] a Analysis of Applicability to Non-Academic Community The final categorization of the papers reviewed concerns the applicability of the research results to the non-academic investment community. This section briefly outlines the requirements of the non-academic community in the form 1 213 Number of Papers that Outperformed relative to the basic comparison technique of two major constraints, namely Money Management, and Trading Costs. Money Management techniques are directly related to the methods by which traders control their capital. Every trader has restrictions not only on the amount of capital that can be employed, but also how that capital is to be expended in trading activities. Typically, this relates to various position sizes that a trader can control, and also the restrictions on holding those positions in the event of price moving against the trader. These restrictions placed on a trader would be rigidly enforced, in terms of position size, equity invested in specific positions, and the amount of drawdown which could be tolerated before trading was halted. None of the papers reviewed implemented money management techniques. For a detailed explanation of money management techniques, the reader may wish to refer to Elder [33]. - other Not applicable2 5 x transaction costs (brokerage), x gaps (potentially large price changes, outside of trading hours), x bid-ask spreads For a detailed explanation of the costs involved in the trading process, the reader may wish to refer to Babcock [34]. Perhaps the greatest difficulty in surveying this style of literature is to be found in classifying successes. Very few papers measure success in terms of outperforming a buyand-hold approach. This measure is extremely important in this domain, as the majority of implementations can only be compared to each other in this way. There are a number of papers which demonstrate success within a very narrow time frame, or small data window, or indeed, propose several alternative models, and demonstrate one model to be superior to the other alternatives. No research was found to be directly transferable to the non-academic arena, due to the lack of trading constraints implemented. Further, it was clear that a number of papers demonstrated results that could not be captured in practice. This was found, for example, in papers that demonstrated a superior return, but only in the absence of transaction costs. Earning a small return per transaction without taking transaction costs into account demonstrates a successful result. The effect of transaction costs upon such a system would be catastrophic. It is felt that this situation could be improved by implementing simple constraints such as ac- Table 5. Reviewed papers classified by implementation of real-world trading constraints Paper Classification Money Management 0 None 5 [23], [25], [30], [2], [4] References Trading Costs - transaction costs 0 Conclusions Within the area of soft computing styles, the survey finds that the majority of research is within the area of both Hybrid systems, and Pattern Recognition and Classification. It is suggested the reason for this is that the technical analysis approach lends itself towards the pattern recognition and classification areas. Also, many hybrid systems include pattern recognition and classification as one of their constituents. Accounting for these costs can easily turn a system with a positive return into one with a negative return. For example, a system which has a high number of trades, each generating very little profit would appear as a successful system in terms of rate of return. However, taking transaction costs into account could see the profit earned quickly eroded in paying transaction costs. Only five of the papers reviewed accounted for trading costs, and these papers specifically only accounted for transaction (brokerage style) costs. Typically, these papers allowed 1% of the cost of the trade as expended on brokerage. Table 5 demonstrates the degree to which the reviewed papers implemented realworld trading constraints. Number of papers [21], [31], [7], [14], [15], [16] This paper has surveyed recent and key literature in the domain of applying Soft Computing to Investment and Financial Trading. It categorized the papers reviewed according to their investment discipline, the style of soft computing they used, the degree to which they demonstrated their success, and finally, their applicability as real world trading models. Within the context of investment discipline, the survey shows that the majority of this type of research is being conducted in the field of Technical Analysis. As discussed in the survey, soft computing is particularly data intensive, and it is suggested that this observation goes someway to explaining this obvious bias in research. Trading costs refers to the amount of cost expended to earn a given return. Traders are subject to a variety of costs, such as: Research 6 2 214 These papers did not implement trading systems, thus they had no need for money management techniques, or accounting for cost. Proceeding of Artificial Intelligence in the Pacific Rim. 1991. counting for transaction costs, and simple money management techniques to control position sizing. REFERENCES 1. 2. Tan, C.N.W., Artificial Neural Networks: Applications in Financial Distress Prediction and Foreign Exchange Trading. 2001, Gold Coast, QLD: Wilberto. Chan, K.C.C. and K.T. Foo. Enhancing Technical Analysis in the Forex market using Neural Networks. in IEEE International Conference on Neural Networks. 1995. 3. Quah, T.-S. and B. Srinivasan. Utilizing Neural Networks in Stock Pickings. in Proceedings of the International Conference on Artificial Intelligence. 2000. 4. Yao, J. and H.-L. Poh. Forecasting the KLSE index using Neural Networks. in IEEE International Conference on Neural Networks. 1995. 5. Hobbs, A. and N.G. Bourbakis. A Neurofuzzy Arbitrage Simulator for Stock Investing. in Computational Intelligence for Financial Engineering (CIFER). 1995. New York. 6. Austin, M., C. Looney, and J. Zhuo, Security Market timing using Neural Network Models. New Review of Applied Expert Systems, 1997. 3: p. 3-14. 7. Falas, T., A. Charitou, and C. Charalambous. The Application of Artificial Neural Networks in the Prediction of Earnings. 1994. Orlando: IEEE. 16. Kamijo, K. and T. Tanigawa. Stock Price Pattern Recognition: A Recurrent Neural Network Approach. in Proceedings of the International Joint Conference on Neural Networks. 1990. 17. Baba, N. and H. Handa. Utilization of Neural Network for Constructing a User Friendly Decision Support System to Deal Stocks. in IEEE International Conference on Neural Networks. 1995. 18. Baek, J. and S. Cho. "Left Shoulder" Detection in Korea Composite Stock Price Index using an Auto-Associative Neural Network. in Intelligent Data Engineering and Automated Learning - IDEAL 2000: Data Mining, Financial Engineering and Intelligent Agents. 2000. 19. Chen, A., C. Chianglin, and H. Chung, Establishing an Index Arbitrage model by applying Neural Networks method - A case study of Nikkei 225 Index. International Journal of Neural Systems, 2001. 11(5): p. 489-496. 20. Tan, P.-Y. Using Genetic Algorithm to optimize an oscillator-based market timing system. in Proceedings of the Second Singapore International Conference on Intelligent Systems SPICIS'94. 1994. Singapore. 21. Lee, R.S.T. and J.N.K. Liu, NORN Predictor - Stock Prediction using a neural oscillatory-based recurrant network. International Journal of Computational Intelligence and Applications, 2001. 1,4: p. 439-451. 22. Lam, K.P., K.C. Chiu, and W.G. Chan. An embedded fuzzy knowledge base for technical analysis of stocks. in Neural Networks in Financial Engineering. Proceedings of the Third International Conference on Neural Networks in the Capital Markets. 1996. 8. Yamaguchi, T., A Technical Analysis Expert System in the Stock Market. Future Generation Computer Systems, 1989. 5(1): p. 21-27. 9. Mizuno, H., et al., Application of Neural Network To Technical Analysis of Stock Market Prediction. Studies in Informatics and Control, 1998. 7(2): p. 111-120. 23. Liu, N.K. and K.K. Lee, An Intelligent Business Advisor System for Stock Investment. Expert Systems, 1997. 14(3): p. 129-139. 10. Leus, M., et al. GAs and Financial Analysis. in 14th International Conference on Industrial Engineering Applications of Artificial Intelligence and Expert Systems. 2001. Berlin. 24. 11. Baba, N., et al. Knowledge-Based Decision Support Systems for Dealing Nikkei-225 by Soft Computing Techniques. in Knowledge-Based Intelligent Information Engineering Systems and Allied Technologies KES 2001. 2001: IOS Press, Netherlands. Leigh, W., R. Purvis, and J.M. Ragusa, Forecasting the NYSE composite index with technical analysis, pattern recognizer, neural network, and genetic algorithm: a case study in romantic decision support. Decision Support Systems, 2002. 32(4): p. 361-377. 25. Dourra, H. and P. Siy, Investment using Technical Analysis and Fuzzy Logic. Fuzzy Sets and Systems, 2002. 127: p. 221-240. 26. Chou, S.-C.T., et al., A Stock Selection DSS combining AI and Technical Analysis. Annals of Operations Research, 1997. 75: p. 335-353. 27. Baba, N., N. Inoue, and Y. Yanjun. Utilization of Soft Computing Techniques for Contructing Reliable Decision Support Systems for Dealing Stocks. in IJCNN'02: Proceedings of the 2002 International Joint Conference on Neural Networks. 2002. Honolulu, Hawaii. 28. Jang, G., et al. An Intelligent Trend Prediction and Reversal Recognition System using Dual-Module Neural Networks. in First International Conference on Artificial Intelligence Applications on Wall Street. 1991. 29. Wong, F. and D. Lee. Hybrid Neural Network for Stock Selection. in The Second Annual International Confer- 12. Skabar, A. and I. Cloete. Discovery of Financial Trading Rules. in Proceedings of the IASTED International Conference on Artificial Intelligence and Applications. 2001. 13. Suh, Y.-H. and J.E. LaBarre, An Application of Artificial Neural Network Models to Portfolio Selection. Journal of Computer Information Systems, 1995. 36(1): p. 65-73. 14. Fu, T.-C., et al. Evolutionary Segmentation of Financial Time Series into Subsequences. in Proceedings of the 2001 Congress on Evolutionary Computation. 2001. 15. Yasunobu, C., et al. A Knowledge-Based Technical Analysis System for Financial Decision-Making. in 215 2000: Proceedings: Theme: Intelligent Systems and Technologies for the new millennium. 2000. ence on Artificial Intelligence Applications on Wall Street. 1993. New York. 30. 31. Chenoweth, T., Z. Obradovic, and S. Lee. Technical Trading rules as a Prior Knowledge to a Neural Networks Prediction System for the S&P 500 Index. in IEEE Technical Applications Conference and Workshops. 1995. Portland, Oregon. 32. Taylor, M.P. and H. Allen, The Use of Technical Analysis in the Foreign Exchange Market. Journal of International Money and Finance, 1992. 11: p. 304-314. 33. Elder, A., Trading for a Living. 1993: John Wiley & Sons. Abdullah, M.H.L.b. and V. Ganapathy. Neural Network Ensemble for Financial Trend Prediction. in Tencon 34. Babcock Jr., B., The Dow Jones-Irwin Guide to Trading Systems. 1989, Homewood, Illinois: Dow Jones-Irwin. 216