Game AI

Introduction

At the Institute for Information Processing (TNT), methods in the broad field of Machine Learning are developed and applied to various problems. One of these application areas is the development of artificial intelligence in games. Due to their diversity and accessibility, they offer an ideal test environment for machine learning algorithms and present them with numerous interesting challenges, which form the basis for numerous research projects at TNT.

Recent Research Topics

Efficient decision making:

Games such as Chess and Go have a wide range of possible actions per move. This leads to enormous complexity in decision making and often requires the use of heuristic techniques to approximate the best possible move. This problem is magnified in the area of digital games, where, especially in real-time strategy games, we often need to control multiple characters simultaneously and in real-time. At the TNT, we are investigating methods to abstract (simplify) this decision space to enable efficient decision making.

 

General Game-Playing and Transfer Learning:

Many methods of autonomous game-playing are related to a single game. In the context of general game-playing, agents are developed that not only play a single game successfully, but are able to perform well in multiple games. This can be done on the use of flexible search techniques or it can be enabled by transfer learning. In the latter, partial models of the developed solution are transferred between applications to build on previously learned knowledge and thus accelerate the re-learning process.

 

Causal Learning and Interpretable AI:

Especially in general game-playing, robust agent behavior should be learned. Numerous machine learning methods are based on the recognition of patterns in the form of correlations. However, these are often subject to a bias and can lead to false conclusions. Modeling such a bias and learning causalities instead of correlations is fundamental to developing more robust agent behavior. Causal learning methods can also significantly increase the interpretability of behavior by inferring relevant factors that impact the agent’s decision making.

 

Procedural Content Generation:

Game development is becoming increasingly complex. More and more developers are becoming involved in the process. Large-scale productions such as Grand Theft Auto and the Elder Scrolls series spend many hours creating huge immersive worlds that players can immerse themselves in. Procedural Content Generation describes a process to reduce development effort by automating the creation of game content. The TNT uses machine learning techniques to extract patterns from existing content and combine them to generate new and interesting content.

Recent Publications
  • Conference Contributions
    • Linjie Xu, Jorge Hurtado-Grueso, Dominic Jeurissen, Diego Perez Liebana, Alexander Dockhorn
      Elastic Monte Carlo Tree Search State Abstraction for Strategy Game Playing
      2022 IEEE Conference on Games (CoG), IEEE, 2022
    • Linjie Xu, Diego Perez-Liebana, Alexander Dockhorn
      Towards Applicable State Abstractions: a Preview in Strategy
      The Multi-disciplinary Conference on Reinforcement Learning and Decision Making (RLDM) - RL as a Model of Agency, pp. 1-7, 2022
    • Lars Wagner, Christopher Olson, Alexander Dockhorn
      Generalizations of Steering - A Modular Design
      2022 IEEE Conference on Games (CoG), IEEE, pp. 1-4, 2022
    • Alexander Dockhorn, Jorge Hurtado-Grueso, Dominik Jeurissen, Linjie Xu, Diego Perez-Liebana
      Portfolio Search and Optimization for General Strategy Game-Playing
      2021 IEEE Congress on Evolutionary Computation (CEC), IEEE, pp. 2085-2092, 2021
    • Alexander Dockhorn, Jorge Hurtado-Grueso, Dominik Jeurissen, Linjie Xu, Diego Perez-Liebana
      Game State and Action Abstracting Monte Carlo Tree Search for General Strategy Game-Playing
      Proceedings of the 2021 IEEE Conference on Games (CoG), IEEE, pp. 1--8, August 2021
    • Diego Perez-Liebana, Cristina Guerrero-Romero, Alexander Dockhorn, Linjie Xu, Jorge Hurtado, Dominik Jeurissen
      Generating Diverse and Competitive Play-Styles for Strategy Games
      2021 IEEE Conference on Games (CoG), IEEE, pp. 1-8, 2021
    • Alexander Dockhorn, Sanaz Mostaghim, Martin Kirst, Martin Zettwitz
      Multi-Objective Optimization and Decision-Making in Context Steering
      2021 IEEE Conference on Games (CoG), IEEE, pp. 1-8, 2021
    • Alexander Dockhorn, Simon Lucas
      Local Forward Model Learning for GVGAI Games
      IEEE Conference on Computational Intelligence and Games, CIG, pp. 716--723, August 2020
    • Alexander Dockhorn, Rudolf Kruse
      Forward Model Learning for Motion Control Tasks
      2020 IEEE 10th International Conference on Intelligent Systems (IS), pp. 1--5, Varna, Bulgaria, September 2020
    • Raluca D. Gaina, Martin Balla, Alexander Dockhorn, Raul Montoliu, Diego Perez-Liebana
      Design and Implementation of TAG: A Tabletop Games Framework.
      arXiv:2009.12065, 2020
    • Diego Perez-Liebana, Alexander Dockhorn, Jorge Hurtado Grueso, Dominik Jeurissen
      The Design Of “Stratega”: A General Strategy Games Framework
      arXiv:2009.05643, pp. 1--7, 2020
    • Raluca D Gaina, Martin Balla, Alexander Dockhorn, Raul Montoliu, Diego Perez-liebana
      TAG : A Tabletop Games Framework
      Joint Proceedings of the AIIDE 2020 Workshops co-located with 16th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE 2020); CEUR Workshop Proceedings (2020), CEUR Workshop Proceedings, pp. 1--7, Worcester, 2020, edited by J. C. Osborn
    • Alexander Dockhorn, Jorge Hurtado Grueso, Dominik Jeurissen, Diego Perez-Liebana
      “Stratega”: A General Strategy Games Framework
      Joint Proceedings of the AIIDE 2020 Workshops co-located with 16th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE 2020); Artificial Intelligence for Strategy Games, CEUR Workshop Proceedings, pp. 1--7, Worcester, 2020, edited by Osborn, Joesph C.
    • Alexander Dockhorn, Simon M Lucas, Vanessa Volz, Ivan Bravi, Raluca D Gaina, Diego Perez-Liebana
      Learning Local Forward Models on Unforgiving Games
      2019 IEEE Conference on Games (CoG), IEEE, August 2019
    • Simon M Lucas, Alexander and Volz Dockhorn, Raluca D Gaina, Ivan Bravi, Diego Perez-Liebana, Sanaz Mostaghim, Rudolf Kruse
      A Local Approach to Forward Model Learning: Results on the Game of Life Game
      2019 IEEE Conference on Games (CoG), IEEE, pp. 1--8, August 2019
    • Alexander Dockhorn, Sanaz Mostaghim
      Introducing the Hearthstone-AI Competition
      arXiv:1906.04238, pp. 1--4, May 2019
    • Alexander Dockhorn, Daan Apeldoorn
      Forward Model Approximation for General Video Game Learning
      Proceedings of the 2018 IEEE Conference on Computational Intelligence and Games (CIG’18), IEEE, p. 425–432, August 2018
    • Alexander Dockhorn, Tim Tippelt, Rudolf Kruse
      Model Decomposition for Forward Model Approximation
      2018 IEEE Symposium Series on Computational Intelligence (SSCI), IEEE, pp. 1751--1757, November 2018
    • Alexander Dockhorn, Max Frick, Ünal Akkaya, Rudolf Kruse
      Predicting Opponent Moves for Improving Hearthstone AI
      17th International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems, IPMU 2018, Springer International Publishing, pp. 621--632, 2018, edited by Medina, Jesus; Ojeda-Aciego, Manuel; Verdegay, José
  • Journals
    • Alexander Dockhorn, Martin Kirst, Sanaz Mostaghim, Martin Wieczorek, Heiner Zille
      Evolutionary Algorithm for Parameter Optimization of Context Steering Agents
      IEEE Transactions on Games, IEEE, pp. 1-12, 2022
    • Daan Apeldoorn, Alexander Dockhorn
      Exception-Tolerant Hierarchical Knowledge Bases for Forward Model Learning
      IEEE Transactions on Games, Vol. 13, No. 3, pp. 249-262, 2021
    • Alexander Dockhorn, Rudolf Kruse
      Modelheuristics for efficient forward model learning
      At-Automatisierungstechnik, De Gruyter, October 2021
    • Alexander Dockhorn, Rudolf Kruse
      Predicting Cards Using a Fuzzy Multiset Clustering of Decks
      International Journal of Computational Intelligence Systems (IJCIS), Atlantis Press, Vol. 13, No. 1, pp. 1207--1217, August 2020
  • Book Chapters
    • Alexander Dockhorn, Rudolf Kruse
      Balancing Exploration and Exploitation in Forward Model Learning
      Advances in Intelligent Systems Research and Innovation, Springer International Publishing, pp. 1--19, Cham, 2022, edited by Sgurev, Vassil; Jotsov, Vladimir; Kacprzyk, Janusz
    • Alexander Dockhorn, Chris Saxton, Rudolf Kruse
      Association Rule Mining for Unknown Video Games
      Fuzzy Approaches for Soft Computing and Approximate Reasoning: Theories and Applications, Springer Cham, pp. 257--270, October 2020, edited by Lesot, Marie-Jeanne; Marsala, Christophe