Download Robustness Test Results

Appendix C - Test Results
Robustness Test Results
The two robustness tests were carried out using 5 generations and 5 chromosomes. Firstly
I decided to use a very small amount of generations and chromosomes because otherwise
the program will take a very long time to run. More than one generation is required in
order to test the genetic operations because genetic operations are only run at the end of
every generation. Also keeping the tests a very small length, I was able to observe the
results as they happened during the run.
1. CrossoverProb = 1.5. Having observed the run of this program there was no specific
error in the run of the program. This is because “jgprog” does not have a validation
check on the control parameters inputted into the program. However, absolutely no
crossover was performed. The same chromosomes occurred in every generation.
2. ReproductionProb = 1.5. The results were very similar to the results in the previous
test. Even though this increased the reproduction probability, the crossover +
reproduction probability was greater than 1. This meant that no crossover was
performed, and each generation contained the same chromosomes from the first
generation. Therefore, the Reproduction probability did increase but this was by
default because crossover was not working.
Test Schedule
1. Test Investment Simulator: The output of a run of the system was paused to check
that all the functionality of the investment simulator was correct. Only showing the
output of the investment simulator allowed me to do this (ignoring the GP System
output). The number of generations and chromosomes is insignificant for this test. I
am only testing the Return on Investment function that allows the system to calculate
the Sharpe Ratio. For a particular chromosome results were returned:
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
The
Portfolio
return on
day 0
Portfolio
return on
day 1
Portfolio
return on
day 2
Portfolio
return on
day 3
Portfolio
return on
day 4
Portfolio
return on
day 5
Portfolio
return on
day 6
Portfolio
return on
day 7
Portfolio
return on
day 8
Portfolio
return on
day 9
Portfolio
return on
day 10
Portfolio
return on
day 11
Portfolio
return on
day 12
Portfolio
return on
day 13
Portfolio
return on
day 14
Portfolio
return on
day 15
Portfolio
return on
day 16
Portfolio
return on
day 17
Portfolio
return on
day 18
Value at the end of the day 100000.0
investment 0.0
Value at the end of the day 99978.64
investment -21.359375
Value at the end of the day 100260.45
investment 281.8125
Value at the end of the day 100607.61
investment 347.15625
Value at the end of the day 99681.25
investment -926.3594
Value at the end of the day 99965.45
investment 284.20312
Value at the end of the day 100244.305
investment 278.85156
Value at the end of the day 100496.74
investment 252.4375
Value at the end of the day 99806.734
investment -690.0078
Value at the end of the day 99317.305
investment -489.4297
Value at the end of the day 98304.44
investment -1012.8672
Value at the end of the day 97026.99
investment -1277.4453
Value at the end of the day 97316.914
investment 289.92188
Value at the end of the day 97227.9
investment -89.015625
Value at the end of the day 97888.51
investment 660.6094
Value at the end of the day 98087.94
investment 199.42969
Value at the end of the day 98659.016
investment 571.0781
Value at the end of the day 98441.086
investment -217.92969
Value at the end of the day 98037.09
investment -403.9922
The Portfolio Value at the end of the day 98991.516
The return on investment 954.4219
The day 19
The Portfolio Value at the end of the day 98936.63
The return on investment -54.882812
The day 20
The mean -53.1683601
330835.970727826
The Standard Deviation 575.1778489
The sharpe ratio -0.092438312
Refer to Section 3.5.4 in the project report to view the Sharpe Ratio calculation.
Having received these results I checked the Return on Investment function first. The sum
of all Return on Investment, not including day 0, equals -1063.367202. This gives a mean
value of -53.1683601. Looking at my results above, the figures match. Now I am going
to check that the standard deviation works correctly. All the values were input into an
excel spreadsheet and a variance was returned. The variance = 330835.9707. The
standard deviation is the square root of this value. The square root of this value =
575.1778489. Checking my results from above the values match. Using the mean and
standard deviation a Sharpe Ratio was calculated and the value returned equalled to 0.09238312. Therefore this test has been successful.
2. Test Chromosome: I used a pilot simulator, which tested that certain functions
worked, before I ported them over to my main program. In this case I tested only a
small amount of data, rather than testing that 21 days of data for 80 stocks could be
stored in an array. Instead I stored day 0 of trading for 80 stocks in an array for the
terminal “Price”. The array was returned, which was very encouraging. I checked
each stock CSV file to ensure that the values matched the values in the array. This
test was successful.
3. Test getPrice() Method: To test this method worked I decided to run the program
once with my normal data and then once with irregular data. The irregular data I used
was the string “abc”. An error message was returned, when running the program:
.java.lang.NumberFormatException: For input string: "abc"
at java.lang.NumberFormatException.forInputString(Unknown Source)
at java.lang.FloatingDecimal.readJavaFormatString(Unknown Source)
at java.lang.Float.valueOf(Unknown Source)
at harjInvestmentSimulator.Price.createPrice(Price.java:32)
at
harjInvestmentSimulator.GPWorld.getChromosomeValues(GPWorld.java:145)
at
harjInvestmentSimulator.GPWorld.computeRawFitness(GPWorld.java:207)
at
harjInvestmentSimulator.GPWorld.computeFitness(GPWorld.java:101)
at com.groovyj.jgprog.World.computeSome(World.java:337)
at com.groovyj.jgprog.World.computeAll(World.java:309)
at com.groovyj.jgprog.World.create(World.java:228)
at harjInvestmentSimulator.GPWorld.create(GPWorld.java:93)
at harjInvestmentSimulator.GPWorld.main(GPWorld.java:389)
java.lang.NumberFormatException: For input string: "abc"
The program worked consistently with the original data.
4. Check the fitness: The system was run for 5 generations and 5 chromosomes. I
monitored the best chromosome from each generation and noted down its Sharpe
Ratio. At the end of the program the best chromosome from each generation was
returned. The values I noted matched the values returned by the program. This test
was successful. I repeated this test, in which I was successful again.
5. Check GP Chromosome: The system was run for 5 generations with 10
chromosomes per generation. I noted down every chromosome in generation 0, to
check that a depth greater than 6 was not achieved. Then I noted down the depth of
each chromosome in the preceding generations, to check that the depth is always less
than 17. In both cases the test was successful.