Download final report - CSE

Artificial Intelligence ( CS 365 )
IMPLEMENTATION OF AI SCRIPT GENERATOR
USING DYNAMIC SCRIPTING FOR AOE2 GAME
Author: Saurabh Chatterjee Guided by: Dr. Amitabha Mukherjee Abstract: I have implemented an AI script generator that automatically determines strategies to be used in an AI script to be used in playing an AoE2 game ( a real time strategy game ). After the game, a function updates the rulebase based on the results of the game, so as to improve the strategy. Thus the AI script generated is a learning AI. I have used some concepts from CS365 course particularly utility functions for learning. Introduction In the computer gaming world, AI has been a very important part especially in single player games. AI in gaming is that element which comprises everything but the visual and auditory aspects of gameplay. It encompasses many areas such as interaction, pathfinding, machine learning, flocking, formations, difficulty scaling and decision making. Real Time Strategy (RTS) is a genre of computer games which can be viewed as simplified military simulations. Players gather resources and create troops to fight enemies. They are distinct from turn‐based games because the actions have to be performed at real time because all the players are issuing commands at the same time and no one is waiting. My project is to build an AI script generator for AOE 2 game. Although dozens of AI scripts have already been written for AOE 2 game, they are all hand‐coded and depend on strategies that human players derive after hours of playing and experimenting. Also there is no learning mechanism involved. My project realizes the learning technique by manually feeding the results of a game and changing the weights that affect the rulebase to be generated. Implementation of AI My project deals entirely with high‐level ‘strategic’ AI ( as opposed to low level AI such as pathfinding, formations , etc. ) including decision making skills. As such an individual unit cannot be controlled, but the policies made will be processed and executed by the game program which controls the units. For example you may give a rule saying that the percentage of woodcutters should be 30%. This means the game engine will ensure that 30% of the villagers you have are woodcutters, but you cannot decide the actions of a particular villager ( this is taken care of by ‘low level’ AI ). Nevertheless strategic AI can make a huge difference to the game especially as it can be scripted in a variety of ways and provides an abstraction that people can easily deal with. The AI generated by my program is nicknamed ‘Devastator AI’ and is referred to as that in the remainder of this document. Methodology of the AI agent There are many fundamental AI research problems posed by AI for RTS games. The following is a list of some listed by Michael Buro[2] , and how I have implemented it. •
Resource management: Players start off by gathering local resources to build up defenses and
attack forces, to upgrade weaponry, and to climb up the technology tree.
Devastator AI changes the percentages of the people gathering resources and builds resource
gathering buildings at appropriate locations.
•
Decision making under uncertainty: Initially, players are not aware of the enemies' base
locations and intentions. They have to gather intelligence by sending out scouts.
Devastator AI doesn’t do much scouting, it is more of a defensive AI. Only 3 units scout at any
time, this is fixed.
•
Spatial and temporal reasoning: Static and dynamic terrain analysis as well as understanding
temporal relations of actions is of utmost importance in RTS games - and yet, current game AIs
largely ignore these [Forbus et al., 2002].
Devastator AI evaluates a map and finds out the distribution of land and water in it. If there is
any area of water greater than 2000 square tiles, it trains a fixed navy ( galleons ).
•
Collaboration: In RTS games groups of players can join forces and intelligence.
Devastator AI can establish trade routes with allies, but nothing else.
•
Learning: One of the biggest shortcomings of most (RTS) game AI systems is their inability to
learn from experience. Human players only need a couple of games to spot opponents'
weaknesses and to exploit them in upcoming games. Devastator AI uses dynamic scripting technique for learning. •
•
Evaluation Functions: Devastator AI uses an evaluation function called firepower to find the
best combination of units to use. Adversarial real-time planning: In fine-grained simulations, agents cannot afford to think in
terms of micro actions. Instead, abstractions have to be found which allow a machine to conduct
forward searches in a manageable abstract space and to translate found solutions back. This is what is implemented automatically in commercial games like AoE2. Rules and Rulebases Rule based AI is the most commonly used method for the implementation of AI strategies. A ‘rule’ in general is an implementation of a certain set of effects when a certain set of conditions is satisfied. I am using the commercial game Age of Empires II – the Conquerors. It uses a fixed rule script in which a rule simply consists of some conditions and some effects. Here is an example of such a rule. (defrule (current‐age == dark‐age) (unit‐type‐count villager < 31) (can‐train villager) => (train villager)) A fixed rulebase is one in which the same script is used for every game and thus provides no opportunity for learning. Currently almost all games including AOEII use fixed rulebases because they are easier to design. However, I am modifiying the script after each game manually thus providing a sort of dynamic scripting. The biggest disadvantage of rule‐based AI is it’s predictability. After playing a few games, a human player can spot weaknesses in the AI and exploit them. Thus it is necessary for good AI’s to keep changing their strategies. Generation of initial rulebase and selection of strategies My program generates the AI based on the game data. Game data includes the parameters of the different units used. The parameters I have taken into account are attack, HP, range, armor, pierce armor, and costs of training. I have defined a ‘firepower’ function which generates a value which tells how good a unit is. The ‘goodness’ value is an extrinsic factor I will discuss in the next section. The remainder are parameters from game data. number = 50*80/(goldcost+foodcost+woodcost); numlimit = number; number *=goodness; if(range==0) attackvalue += number*(attack+10); else attackvalue += (number*number)*attack/20; hpvalue = (hp+armor+piercearmor)*numlimit; firepower = attackvalue+hpvalue; I then determine a combination of three units which is the optimal for that particular civilization. This firepower function accurately determines good combinations to be used as determined by practical experiences. It does commit some mistakes which are later corrected through learning. This is because there are a number of factors I have not taken into account. Eg. Teutonic knights seem to be very good units because of their high attack and armor. However they are actually very slow and useless. Learning permits the AI to recover from mistakes. The building and training rules are influenced by these combinations. Also there are a large number of strategic numbers which are updated according to their performance. An example of a strategic number is the percentage of villagers which are food gatherers and is represented by ‘sn‐food‐gatherer‐percentage’. Finally there is also a lot of ‘constant’ AI script which does not change every time the program is run. This is because the results obtained do not reflect every AI strategic rule in a direct manner and without any learning it is simpler to just use a direct hand coded AI script. I have hand coded the AI script for town size, trading and civilian training. Dynamic Scripting Dynamic Scripting[3] is a learning technique inspired by reinforcement learning ( Russell and Norvig ). The rulebase is adapted and generated every time the game is played, rather than being hand‐coded. Certain weights are attached to parameters in the game that will be changed every time the game is played. After a game is over, it’s resulting scores will then be used to update these weights. Thus different strategies can be selected as opposed to the static AI script discussed above. Image from [3]
Utility functions For units, I use a utility function which refer to as a unit’s ‘goodness’. This value is a ratio of how good a single unit of that unit type is with respect to a champion ( A champion is a type of unit). For strategic numbers, I store the numbers directly and update them. For the rest of the AI script there is no weightage stored and it is hand coded constant for all games. Updating functions The updating function for units determines the units used and increases the goodness value by (kills‐losses‐70)/(kills+losses) Thus the goodness decreases if kills is less than losses and increases if kills is greater than losses. The factor of 70 is there because kills‐losses should be at least 70. The economic strategic numbers are changed in the following way. Foodper = ( (food gathered by enemy )/ ( food gathered ) )*(original food percentage) Since the percentages must add up to 90%, they are then modified accordingly. Foodper = Foodper*90/(Foodper+Goldper+Woodper) (10% of the villagers are set aside for building.) This updating function can be considered to be an example of reinforcement learning. The AI has rewards and punishments. The AI thus can be considered to be an agent which changes it’s actions ( ie. the rulebase ) to maximize it’s reward, where the reward is the result parameters of the game, such as kills, losses, resources gathered etc. Results I ran 8 games for the following graphs. Since there is a lot of uncertainty involved in the game, we must try to deduce the overall result. Military graph – kills/losses ( kill ratio ), for a time of 10 min. The following AI’s were tested against Blitzkreig AI, a decent AI which is quite tough to beat. My AI is nicknamed ‘Devastator’. 3.5
3
2.5
2
Blitzkreig
Devastator
1.5
AllianceThunda
1
0.5
0
1
2
3
4
5
6
7
As we can see, Blitzkreig AI performing against its own runs up a kill ratio of near 1.0 , whereas the other two are a lot higher. Alliance thunda is a highly optimized AI, one of the best for AOE 2 game. Economic graph ( resource gathering) Total amount of resource gathered vs. game, for a time of 10 min. It seems from this graph that Blitzkreig AI is the best at resource gathering and AllianceThunda is the worst! This discrepancy can be explained because AllianceThunda is focused heavily on military and since Post Imperial DM starts with huge amounts of resources, resource gathering is not a priority in the initial minutes of the game. In the actual game, AllianceThunda would kill the other AI long before it’s poorer resource gathering started to come into picture. Learning curve To test if the learning function is effective, we test it with Teuton civilization. This picks the wrong combo at first, but rapidly finds the right combination of units. Thereafter there is only slow progress due to economic gain which finally levels off. Also, we can see the economic learning being done in the economic graph above. The graph is of kills‐losses vs. games Overall •
•
•
•
Performs very well against Blitzkreig AI killing units in a ratio 2:1 and beats standard AI ( which is basically pathetic ) very easily. Puts up a good defence vs the best AI’s such as AllianceThunda and beats AllianceThunda 25% of the time ( 2 in 8 games ). Can find the best combos very easily thanks to firepower function. Slightly weak economically. Learning function enables it to find the best combos if it made a mistake initially. The graphs given show statistical variations due to probability coming into play as the game is highly non deterministic even though the strategies are given by the AI, the actual implementation is up to the game engine. Conclusion ‘Devastator AI’ generated is a very good AI for post Imperial Deathmatch AoE2 game. Learning works at least partly and the evaluation function selects the effective combination in most cases. Thus it is possible to use some type of dynamic scripting in any game. References [1] Artificial Intelligence – A modern Approach – Stuart Russell and Peter Norvig
[2] Real-Time Strategy Games: A New AI Research Challenge Michael Buro, University of
Alberta, Edmonton, Canada
[3] Improving adaptive game AI with evolutionary learning (PDF). Thesis by Mark Ponson,
Delft University of Technology, Netherlands.