Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Comprehensive Overview of Game AI Competitions: History and Ecosystems Discussion Sehar Shahzad Farooq Group Meeting Feb 03, 14 Game AI Competitions • Game AI Test bed for AI and CI research • Competitions Tool to promote research and education in Challenging problems • Impact Provide Goals, rules, platforms and comparisons Why • Game AI Competitions provide a fair, transparent and reusable benchmarking algorithms • Allow an easy way to get in research field for new researchers • Problem solving What • Classical board games Chess, Checkers, Go, Othello • Strategy games Planet Wars (google competition), Defcon, ORTS, StarCraft, • Car Racing Tactics(overtaking), Strategy( damage, fault management) • Console Games Ms. Pac-Man, Mario AI Championship, Geometry Friends, Xpilot • First Person Shooter Games Unreal Tournament 04, 2K BotPrize, Deathmatch competition, • Hybrid gamesPhysical TSP (Racing, TSP) Where • CIG Conference on Computational Intelligence and Games • AIIDE Artificial Intelligence for Interactive Digital Entertainment • CEC Congress on Evolutionary Computation • WCCI World congress on Computational Intelligence • GECCO Genetic and Evolutionary Computation Conference • Evostar European events on evolutionary computation • Fuzz IEEE, AJCAI, IJCAI (Angry Bird), SSCAI (StarCraft) History/ When • • • • • • • • • • Car Racing (07-13) StarCraft(10-13) Angry Bird (12-13) Physical TSP(12-13) Fighting Game AI, Geometry Friends (13) Super Mario (09-12), Platformer Game AI (13) Ghosts Challenge (13) Ms. Pac-Man (07-11), Ghost Vs PaMan(11,12) Unreal Tournament (08-12) GO (5-12), Othello (6-8), X-Pilot (07) 2007 2008 2009 WCCI CEC O O O O O O O CIG O Fuzz-IEEE O CIG O O O GECCO O WCCI O O CEC O O CIG O O GECCO O O O O O O O AIIDE O WCCI O O O CIG O O O Evostar O GECCO O ICE-GIC O O O O O AI Platformer game Physical TSP AI Fighting Game Ghost AI Angry Bird Rubik's Cube O O ICE-GIC 2010 Friends O O Geometry CEC 2006 X-Pilot AI O 2005 Othello O CIG Go CEC Starcraft 2004 Tournament O Unreal Car Racing CEC Super Mario Conference 2003 Ms Pac-Man Year Game AI Competitions CEC 2011 O O O O AI Platformer game Physical TSP AI Fighting Game Ghost AI Angry Bird Rubik's Cube Friends Geometry X-Pilot AI Othello O O O SSCAI O O Fuzz-IEEE Evo Go O AIIDE CIG Starcraft Tournament O WCCI GECCO Unreal Super Mario Ms Pac-Man Car Racing Conference Year Game AI Competitions O O CEC WCCI O O O AIIDE SSCAI 2012 O O O O CIG O GECCO O O O O O AJCAI O O Fuzz-IEEE Evo O CEC 2013 AIIDE O SSCAI O CIG GECCO O O O Fuzz-IEEE IJCAI O O Game AI Competitions, Organizers, Participants and Winners (affiliations) Game Year 2013 2012 2011 2010 Car Racing Conference GECCO Evostar GECCO CIG Evostar GECCO CIG Evostar GECCO WCCI CIG GECCO 2009 2008 2007 Tournament Type Car Setup Optimization Car Setup Optimization CEC CIG WCCI CIG CEC CIG Fuzz-IEEE CIG Ms Pac-Man vs Gh osts League WCCI Ms Pac-Man vs Gh osts League 2012 63 55 80 Winner Mr. Racer Mr. Racer Mr. Racer Mr. Racer Mr. Racer Mr. Racer Mr. Racer J. Munoz AUTOPIA J. Munoz AUTOPIA E. Onieva et. al P. Versari et al. COBOSTAR COBOSTAR M. Simmerson L. Cardamone H. D. Thang et al. P. Burrow H. D. Thang et al. Maastricht Eiisolver Eiisolver Eiisolver Eiisolver Eiisolver Nozomu Screen Capture 5 CIG CIG CEC Pac-Man Ms Pac-Man vs Gh osts League Ghosts Pac-Man Ms Pac-Man vs Gh osts League Ghosts Screen Capture Screen Capture Screen Capture 15 18 11 8 6 4 5 2008 WCCI Screen Capture 11 2007 CEC Screen Capture 4 CIG Ms. PacMan Pacman Ghosts Combined Pacman Ghosts Combined # of Entries 8 1 3 4 7 6 7 5 6 7 7 11 3 6 13 5 5 10 6 4 2011 CEC 2010 2009 Spooks XSL11 James nqkien Pac-mAnt H. Matsumoto et al. H. Matsumoto et al. A. Fitzgerald et al . Unclaimed Affiliation TU Dortmund TU Dortmund TU Dortmund TU Dortmund TU Dortmund TU Dortmund TU Dortmund Carols III University CSIC, Madrid Carols III University CSIC, Madrid CSIC, Madrid Universita Degli Studi Di Parma University of Wuzburg University of Wuzburg Politecnico di Milano University of Nottingham University of Essex University of Nottingham Maastricth University, Netherlands Sweden Sweden Sweden Sweden Sweden University of Electro -Communications, Tok yo Private, United Kingdom Private, Canada University of Nottingham, Malaysia Ritsumeiken University, Japan Carolos III University Ritsumeikan University Ritsumeikan University University of Southern Maine Game AI Competitions, Organizers, Participants and Winners (affiliations) Game Year 2012 Conference Winner Affiliation Ya-Hung Chen et al. Vinay Sudha Ethiraj B. Weber S. Bojarski et al. S. Bojarski et al. Robin Baumgarten S.Polikarpov S. Bojarski et al. S. Bojarski et al. B. Weber R. Baumgarten S. Polikarpov R. Baumgarten R. Baumgarten Conscious-Robots SQLiteBot AMIS UAlbertaBot Skynet Tomas Vajda Skynet Skynet Matej Istenik Skynet Skynet Roman Danielis FreSCBot FreSCBot MimicBot Overmind Beau-Rivage Naïve Agent BLISS PurofMovio Slash Purofvio T National Taiwan Norman University Blekinge institute of technology UC Santa Cruz University Southern Maine University Southern Maine University of York, United Kingdom Australia University of York, United Kingdom TAIST-Tokyo Tech, Thailand CIG CIBot Sejong University, South Korea CIG Rinat Sadykov and Dexter Ju ng CIG CIG 2011 EvoStar WCCI Super Mario CIG 2010 2009 Unreal Tournament 2010 2009 2008 2013 2012 StarCraft 2011 Angry Bird Ghost AI Physical TSP 2010 AIIDE 2013 2012 2013 2013 IJCAI AJCAI GEISTER CIG WCCI CIG 2012 Fighting Game AI 2013 Geometry Friends 2013 Plaftormer AI Evostar WCCI ICE-CIG CIG CIG CIG CIG AIIDE CIG SSCAI AIIDE CIG SSCAI AIIDE CIG SSCAI 2013 Tournament Type Level Generation Turing test Level Generation Learning Gameplay Gameplay Gameplay Gamplay Learning Level Generation Gameplay Gameplay One VS One One VS One One VS One Micro Management Small-Scale Combat Tech-limited Game Complete Game Classic Track Ghost Challenge MO-PTSP Human Vs. Bot Turing Test # of Entries 5 6 4 7 7 4 6 6 6 9 15 11 10 6 8 8 38 10 10 52 30 10 50 7 2 8 17 19 8 8 8 23 31 10 211 IDSIA University Southern Maine University Southern Maine UC Santa Cruz Imperial College, London IDSIA Imperial College, London Imperial College, London Carlos III University Charles University in Prague University of Alberta, Canada freelancer Comenius University (Slovakia) freelancer freelancer University od Zilina (Slovakia) freelancer freelancer Comenius University, Bratislava RPI UC Berkeley EPFL (Switzerland) Competitions 20 18 16 # of Competitions 14 12 10 8 6 4 2 0 2003 2004 2005 2006 2007 2008 Year 2009 2010 2011 2012 2013 Paper Publications related to Games and Competitions 30 Platformer AI 25 Geometry Friends Fighting Game AI # of Papers 20 Ghost AI Angry Bird 15 Physical TSP StarCraft 10 Unreal Tournament Super Mario 5 Ms. Pac-Man Car Racing 0 2005 2006 2007 2008 2009 Year 2010 2011 2012 2013 Phylogeny of Car Racing • Simulated Car Racing (2007) – Julian Togelius and Simon Lucas – Java – 2D Racing…. Simple Physics – TORCS [3D Open Source] (2008) – Java, C++, Python Clients – Still ongoing – Warm-up, Qualifying and final stages Phylogeny of Ms. Pac-Man • Screen Capture (2007) – – – – – – – – Simon Lucas Java interface Goal: High Score Ms. Pac-Man vs. Ghosts (2010~2012) Philipp Rohlfshangen, David Robles and Simon Lucas Java Goal: High Schore (Pac-Man), Low Score (Ghosts) Online evaluation Phylogeny of Super Mario • Mario AI Championship (2010) – Game Play(Win level) – Learning(Evaluation) • Sergey Karakovskiy, Julian Togelius – Level Generation(Infinite Mario Bros) • Noor Shaker, Julian Togelius and Georgios Yannakakis – Turing Test (2012) • Noor Shaker, Julian Togelius and Georgios Yannakakis – Java – Platformer AI Competition (2013) • Level Generation and Turing Test Tracks – Goal: Play in a believable manner Eco System Centralized Management Platform (Suggestion) Game Ai Competition Portals Advertisement Organizers Game Ai Competitions Participants/ Affiliations Game AI Related Conferences Attraction/ Rewards Publications Education Free Video Games (Benchmark) Sharing Winners’ Source Code Special Publication Journal (Suggestion) Game AI Competition Survey https://docs.google.com/forms/d/1PCYJ5Zs1Lsb f7ttI7DoL5kpCL2wepF0k7zh09hakSvg/viewform