publications

2026

1. ChessFormer - Modeling Human Decision Making in Chess

   Jakub Zeman and Miroslav Čepek

   In Modeling Decisions for Artificial Intelligence, 2026

   Abs

   ChessFormer introduces a novel searchless chess engine leveraging transformer architecture to approximate human decision-making in chess. Trained on a vast dataset of 3 billion chess positions, our model learns its entire decision-making process directly from training data. Evaluations show an improvement in human move-matching accuracy over prior models in high-Elo ranges and the model’s ability to distinguish between human and algorithmic decision-making, offering potential applications in chess analysis or cheat detection.

2. Caïssa AI: A Neuro-Symbolic Chess Agent for Explainable Move Suggestion and Grounded Commentary

   Mazen Soliman and Nourhan Ehab

   In KI 2025: Advances in Artificial Intelligence, 2026

   Abs

   Despite the impressive generative capabilities of large language models (LLMs), their lack of grounded reasoning and susceptibility to hallucinations limit their reliability in structured domains such as chess. We present Caïssa AI, a neuro-symbolic chess agent that augments LLM-generated move commentary with symbolic reasoning, knowledge graph integration, and verification modules. Caïssa AI combines a fine-tuned chess-specific LLM with a Prolog-based rule engine encoding chess tactics and rules, along with a dynamically constructed Neo4j knowledge graph representing the current board state. This hybrid architecture enables the system to generate not only accurate move suggestions but also coherent, strategically grounded commentary. A LangGraph-based verification module cross-checks LLM outputs against symbolic logic to ensure consistency and correctness, effectively mitigating hallucinations. By aligning data-driven generation with formal domain knowledge, Caïssa AI enhances both trustworthiness and explainability. Our results demonstrate that this tight neuro-symbolic integration produces verifiable, high-quality commentary and serves as a generalizable blueprint for AI systems requiring real-time, interpretable decision support.

2025

1. Strength Estimation and Human-Like Strength Adjustment in Games

   Chun Jung Chen, Chung-Chin Shih, and Ti-Rong Wu

   In The Thirteenth International Conference on Learning Representations, 2025

   Abs Website

   Strength estimation and adjustment are crucial in designing human-AI interactions, particularly in games where AI surpasses human players. This paper introduces a novel strength system, including a strength estimator (SE) and an SE-based Monte Carlo tree search, denoted as SE-MCTS, which predicts strengths from games and offers different playing strengths with human styles. The strength estimator calculates strength scores and predicts ranks from games without direct human interaction. SE-MCTS utilizes the strength scores in a Monte Carlo tree search to adjust playing strength and style. We first conduct experiments in Go, a challenging board game with a wide range of ranks. Our strength estimator significantly achieves over 80% accuracy in predicting ranks by observing 15 games only, whereas the previous method reached 49% accuracy for 100 games. For strength adjustment, SE-MCTS successfully adjusts to designated ranks while achieving a 51.33% accuracy in aligning to human actions, outperforming a previous state-of-the-art, with only 42.56% accuracy. To demonstrate the generality of our strength system, we further apply SE and SE-MCTS to chess and obtain consistent results. These results show a promising approach to strength estimation and adjustment, enhancing human-AI interactions in games. Our code is available at https://rlg.iis.sinica.edu.tw/papers/strength-estimator.

2. Design of Electronic Chess Board Using Analogue Hall-effect Sensors for Piece Identification

   Justin Julius Chin Cheong, Praneel Bhatia, Matthias Krauledat, and 1 more author

   In 2025 7th International Congress on Human-Computer Interaction, Optimization and Robotic Applications (ICHORA), 2025

   DOI
3. Human Chess: A Novel Searchless RL-based Chess Agent Capable of Multi-ELO Human-Like Play

   Prerit Choudhary, Rikhil Vagadia, and Ankush Dhawan

   2025

   Final project for CS 224R Deep Reinforcement Learning, Spring 2025 https://cs224r.stanford.edu/projects/cs224r_final_projects.html
4. The Advantage of Moving First in Amateur Online Chess

   Tyler Cook

   ICGA Journal, 2025

   Abs DOI

   There is a long-held belief in the chess community that the player with the white pieces has an advantage in making the first move. This phenomenon has been observed repeatedly in over-the-board games between high-level players and professionals. However, less is known about the prevalence of white’s advantage in games played between amateurs in more casual settings. This article attempts to identify a first-move advantage in chess by examining a large database of amateur games played online on a dedicated chess website. Win rates are calculated for various rating levels, and the influence of opening move choice is also explored. These results can help determine whether there is an inherent first-move advantage in chess observable for all players in multiple settings, or if this effect is exclusively seen with players of high skill during games played in person.

5. Understanding the learned look-ahead behavior of chess neural networks

   Diogo Cruz

   CoRR, 2025

   keywords from rejected openreview submission: https://openreview.net/forum?id=OcBAd0JPxv&noteId=ZcMfmDpMpn

   Abs DOI

   We investigate the look-ahead capabilities of chess-playing neural networks, specifically focusing on the Leela Chess Zero policy network. We build on the work of Jenner et al. (2024) by analyzing the model’s ability to consider future moves and alternative sequences beyond the immediate next move. Our findings reveal that the network’s look-ahead behavior is highly context-dependent, varying significantly based on the specific chess position. We demonstrate that the model can process information about board states up to seven moves ahead, utilizing similar internal mechanisms across different future time steps. Additionally, we provide evidence that the network considers multiple possible move sequences rather than focusing on a single line of play. These results offer new insights into the emergence of sophisticated look-ahead capabilities in neural networks trained on strategic tasks, contributing to our understanding of AI reasoning in complex domains. Our work also showcases the effectiveness of interpretability techniques in uncovering cognitive-like processes in artificial intelligence systems.

6. Interpretation and Control of AI Model Behavior Through Direct Adjustment of Latent Representations

   Austin Davis

   2025

   Abs

   This dissertation investigates the structures and mechanisms underpinning the latent space representations that emerge within Generative Pretrained Transformer (GPT) models. Addressing the broader goal of enhancing AI trustworthiness through transparency, accountability, and controllability, we focus on techniques to understand, quantify, and manipulate these latent space representations. Through a series of analyses, we examine several chess-playing GPT models as controlled testbeds, leveraging their structured decision space to explore emergent representations and decision-making processes. Key contributions include a mechanistic analysis of the attention heads and latent representations, the development of novel metrics for evaluating intervention outcomes, and the application of linear probe classifiers to decode and edit the model’s internal world representations. Analysis of the probe weight vectors reveals that the chess-playing GPT developed an emergent world model of the game that includes pieces, positions, and movement rules, and provides empirical support for the linear representation hypothesis–the idea that abstract concepts are encoded as specific directions in the model’s hidden state space. Complementary analysis of the hidden state vectors demonstrates that the model’s internal representations honor the Markovian property of chess. Experimental results demonstrate that linear interventions can causally steer GPT outputs while preserving their semantic validity. Drawing on the dose-response analogy from medicine, we vary both the strength and position of interventions, showing that output quality is maximized when intervention strength follows an exponentially decaying schedule across token positions. Similar experiments using sparse autoencoders in place of linear probes yielded significantly poorer performance. These results highlight the effectiveness of simple linear probes as valuable tools for interpretability and control.

7. ChessMoveLLM: Large Language Models for Chess Next Move Prediction

   Kassim B. Diallo and Moulay A. Akhloufi

   In SoutheastCon 2025, 2025

   DOI
8. Data Serialization and Persistence

   Adarsh Divakaran

   2025

   DOI
9. Investigating Experiential Effects in Online Chess using a Hierarchical Bayesian Analysis

   Adam Gee, Sydney O. Seese, James P. Curley, and 1 more author

   2025
10. The Effect of Transcranial Direct Current Stimulation on Risky Decision-Making of Student Chess Players Based on their Introverted and Extroverted Personality Traits

    Shahrouz Ghayebzadeh, Mehrdad Moharramzadeh, and Maryam Zoghi

    Sport Psychology Studies, 2025

    Abs DOI

    The aim of this research was to investigate the effects of transcranial direct current stimulation (tDCS) on risky decision-making in student chess players, taking into account their personality traits. In this study, 28 high school students who were active in chess and participated in provincial and national chess leagues were selected. Based on the NEO Five-Factor Inventory of personality traits, they were divided into two groups: 14 extroverted students (17 \pm 0.88) and 14 introverted students (16.5 \pm 1.02). Each participant attended three separate sessions in the laboratory, with a minimum 72-hour rest period between sessions. In each session, participants performed the Iowa Gambling Task and the Lichess computer game before any stimulation. They were then subjected to one of three conditions: right anodal/left cathodal, right cathodal/left anodal, or sham stimulation, for 20 minutes at 2 mA intensity over the dorsolateral prefrontal cortex. After the stimulation, participants repeated the Iowa Gambling Task and the Lichess computer game. Data analysis using two-way mixed ANOVA revealed a significant difference in the Iowa Gambling Task between right anodal/left cathodal and right cathodal/left anodal stimulation based on personality traits (p = 0.001). The findings of this study indicated that transcranial direct current stimulation had a differential effect on decision-making in chess players based on their personality traits. Specifically, the study’s results showed that extroverted players exhibited more risk-taking behaviour, while introverted players acted more cautiously.

11. Investigating the capabilities of language models in puzzle reasoning: A survey and experimental analysis

    Panagiotis Giadikiaroglou

    Mar 2025

    Abs DOI

    Puzzle-solving has long served as a benchmark for evaluating artificial intelligence, testing a model’s ability to reason, infer, and strategize across complex problem spaces. Traditional AI and machine learning methods, such as symbolic reasoning and reinforcement learning, have made notable strides in structured domains like board games and logic puzzles. However, as neural networks and, more recently, large language models (LLMs) have evolved, new possibilities have emerged for tackling a broader range of puzzle types, including those requiring nuanced commonsense reasoning, abstract pattern recognition, and complex multi-step calculations. LLMs, with their vast data-driven language capabilities, hold unique potential to bridge structured logical tasks and less formal, knowledge-based puzzles. Despite these advances, the current landscape of puzzle-solving with LLMs reveals both achievements and limitations, particularly when models are tasked with problems that demand interpretative reasoning and precise calculation. This thesis explores the evolving role of LLMs in solving such complex reasoning tasks, specifically focusing on their puzzle-solving capabilities. Divided into two main sections, the thesis first provides a comprehensive survey of recent advancements in LLM methodologies, covering diverse prompting techniques, neuro-symbolic approaches, and fine-tuning strategies for puzzles. Using a newly proposed taxonomy, puzzles are categorized into rule-based and rule-less types, with each category examined for its unique cognitive demands on LLMs. The second section presents experimental evaluations conducted on four datasets–two math-based datasets (GSM8K, SVAMP) and two puzzle-focused datasets (Game of 24 and RiddleSense). Various reasoning techniques, including Input-Output (IO) prompting, Chain-of-Thought (CoT), Least-to-Most (LtM), and Faithful-CoT methods, are employed to assess LLM performance. Models of varying scales, particularly smaller LLMs like Llama-3.1 family and Mistral, are tested across settings such as zero-shot, few-shot, and self-consistency to evaluate their efficacy in solving complex and multi-step reasoning tasks. The thesis provides critical insights into the performance limitations of current LLMs in puzzle-solving, particularly noting that advanced reasoning methods like Faithful-CoT and puzzle translation techniques yield inconsistent improvements with smaller models. Finally, it outlines future research directions, advocating for expanded dataset creation, neuro-symbolic integration, and advancements in puzzle generation. This thesis aims to deepen our understanding of LLMs’ reasoning abilities and highlight pathways to enhance their performance in complex cognitive tasks.

12. Beyond Evaluation: Learning Contextual Chess Position Representations

    Ben Hull

    Mar 2025

    Technical report
13. Can Large Language Models Develop Strategic Reasoning? Post-training Insights from Learning Chess

    Dongyoon Hwang, Hojoon Lee, Jaegul Choo, and 2 more authors

    Mar 2025
14. Generalized Multi-Linear Models for Sufficient Dimension Reduction on Tensor Valued Predictors

    Daniel Kapla and Efstathia Bura

    Mar 2025
15. Binary Matching of Android and iOS Apps

    Alexander Keusch

    Mar 2025

    Abs DOI

    Android and iOS are the two dominant mobile operating systems in the rapidly expanding smartphone market, serving billions of users worldwide. Both platforms feature extensive app stores with millions of applications available for download. While security measures are in place to prevent the distribution of malicious or vulnerable apps, instances of malware have still been discovered in both stores. These incidents highlight a significant security risk that threatens user privacy and highlights the urgent need for advanced research in detecting malicious code and security vulnerabilities in mobile applications. As a step toward addressing this challenge, this thesis explores the feasibility of using binary similarity detection techniques to identify similarities between Android and iOS applications. To achieve this, we designed and implemented a novel analysis pipeline specifically tailored for comparing Android apps (compiled into binary OAT files) with iOS binaries. This pipeline enables the automatic identification of matches between corresponding applications across both platforms. The pipeline comprises several key stages, including preparing the apps and their third-party libraries for binary analysis, disassembling the binaries using both IDA Pro and Ghidra to account for potential variations introduced by different disassemblers, and conducting similarity analysis on the disassembly results. To assess the effectiveness of our pipeline, we conducted a comprehensive analysis on a dataset of 100 cross-platform apps. Our findings indicate that current binary similarity analysis methods have limitations in directly identifying cross-platform similarities between applications. However, we demonstrated that incorporating third-party libraries into the analysis significantly enhances similarity detection and can help to provide meaningful insights. This highlights the crucial role that third-party libraries play in cross-platform app analysis. Additionally, we found that the choice of disassembler has a significant impact on analysis results. Notably, no single disassembler proved to be clearly superior, as both exhibited their own strengths and limitations. Ultimately, this study offers valuable insights into the challenges and potential of cross-platform binary app analysis using traditional binary diffing techniques, laying a strong foundation for future research in this evolving field.

16. Collaborative versus Individual Chess Puzzle Solving

    Alex Knopps

    Feb 2025
17. Policies of Multiple Skill Levels for Better Strength Estimation in Games

    Kyota Kuboki, Tatsuyoshi Ogawa, Chu-Hsuan Hsueh, and 2 more authors

    Feb 2025
18. Optimizing AI-Driven Chess Bots: Strategies for Balancing Performance, Accuracy, and Computational Efficiency

    Girish Kumar D, K S Shiva Kumar, P Pani Rama Prasad, and 3 more authors

    In 2025 4th International Conference on Distributed Computing and Electrical Circuits and Electronics (ICDCECE), Feb 2025

    DOI
19. Exploring resource-rational planning under time pressure in online chess

    Ionatan Kuperwajs, Evan Russek, Lisa Schut, and 4 more authors

    Proceedings of the Annual Meeting of the Cognitive Science Society, Feb 2025

    Abs

    Human planning is incredibly efficient. Even in complex situations with many possible courses of action, people are able to make good decisions. Recent proposals suggest that a primary contributor to this efficiency is the intelligent use of cognitive resources, but how people allocate these resources under time constraints is not fully understood. In this work, we conduct a resource-rational analysis of planning in a large data set of online chess games. We first demonstrate that players spent more time thinking when they had more time to do so, and that this effect was especially prevalent when computation was more valuable. Then, we show that additional time spent planning resulted in better selected moves when one existed, and compare between signals of general and immediate time pressure. Finally, we highlight the role of expertise in this setting. Our results provide evidence that people make resource-rational choices when planning under time pressure.

20. Computer Vision for Chess Game Automation

    Samir Lemeš, Mirhad Kolić, and Edin Tabak

    In New Technologies, Development and Application VIII, Feb 2025

    Abs

    In the modern age of computing and technology, computer vision has become a key aspect of numerous innovations and solutions that make everyday life easier. Object recognition in images is one area where computer vision can contribute to significant improvements. Playing chess, one of the oldest and most challenging intellectual games requires concentration and tactical thinking. In computer vision, automating the recognition of chessboards and figures can contribute to developing advanced chess applications, help users analyse games, or even enable a game with a computer based on real-world chess setups. This paper describes how the system for recognising chessboards and figures using computer vision techniques was developed and implemented. Through analysing existing methods, designing a new algorithm, and experimental evaluation, the goal is to create an accurate and efficient system that can recognise chess positions and individual figures based on images.

21. Characterizing Chess Player Styles with Neural Network Embeddings from Stockfish

    Victor Lequenne

    Feb 2025

    Abs

    This thesis investigates how neural networks can be used to analyze and compare chess player styles. Using the last layer of Stockfish’s neural network, we process positions from historical World Chess Championships (1886–2024), as well as the 2024 World Blitz and Rapid Championships. We apply dimensionality reduction (PCA, t-SNE, MDS) and clustering (K-means) to build style-based player maps, measuring similarities through Jensen–Shannon divergence. Results show consistent stylistic signatures for some players–such as Firouzja and Dubov but may fail with others e.g. Nepomniachtchi. We also confirmed the evolution of chess player’s still accros time. Though the approach is promising, limitations remain, especially in data size and the influence of forced lines. Still, this work lays a foundation for future stylometry studies and applications in AI driven chess training

22. A Neural Network Approach to Chess Cheat Detection

    Maksim Iavich and Zura Kevanishvili

    In Information and Software Technologies, Feb 2025

    Abs

    With the development of chess engines, cheating online has never been easier, resulting in a need for more robust and accurate detection systems. This paper presents a novel approach to chess cheater detection that combines conventional chess engines and neural networks to help identify which games are authentically played by humans and which show signs of extraneous intervention. By utilizing Stockfish to measure centipawn loss and its mathematical derivatives, we can measure deviations from typical computer-generated moves much like in conventional anti-cheat systems. Additionally, the neural network Maia, designed specifically to mimic human play, transmutes centipawn loss data to highlight deviations from human style. This dual-measurement system addresses the limitations of the given traditional anti-cheat systems, which face the issue of distinguishing between strong human players and those using engines. The collected metadata is analyzed using a sequential neural network, which identifies patterns of fair play violation. Our approach offers a robust solution for maintaining the integrity of online chess by accurately detecting and preventing cheating.

23. Re-evaluating metamorphic testing of chess engines: A replication study

    Axel Martin, Djamel Eddine Khelladi, Théo Matricon, and 1 more author

    Information and Software Technology, Feb 2025

    Abs DOI

    Context: This study aims to confirm, replicate and extend the findings of a previous article entitled ”Metamorphic Testing of Chess Engines” that reported inconsistencies in the analyses provided by Stockfish, the most widely used chess engine, for transformed chess positions that are fundamentally identical. Initial findings, under conditions strictly identical to those of the original study, corroborate the reported inconsistencies. Objective: However, the original article considers a specific dataset (including randomly generated chess positions, end-games, or checkmate problems) and very low analysis depth (10 plies11A ply refers to a single turn taken by one player in a game. Two plies, one from each player, together constitute a complete move., corresponding to 5 moves). These decisions pose threats that limit generalizability of the results, but also their practical usefulness both for chess players and maintainers of Stockfish. Thus, we replicate the original study. Methods: We consider this time (1) positions derived from actual chess games, (2) analyses at appropriate and larger depths, and (3) different versions of Stockfish. We conduct novel experiments on thousands of positions, employing significantly deeper searches. Results: The replication results show that the Stockfish chess engines demonstrate significantly greater consistency in its evaluations. The metamorphic relations are not as effective as in the original article, especially on realistic chess positions. We also demonstrate that, for any given position, there exists a depth threshold beyond which further increases in depth do not result in any evaluation differences for the studied metamorphic relations. We perform an in-depth analysis to identify and clarify the implementation reasons behind Stockfish’s inconsistencies when dealing with transformed positions. Conclusion: A first concrete result is thus that metamorphic testing of chess engines is not yet an effective technique for finding faults of Stockfish. Another result is the lessons learned through this replication effort: metamorphic relations must be verified in the context of the domain’s specificities; without such contextual validation, they may lead to misleading or irrelevant conclusions; changes in parameters and input dataset can drastically alter the effectiveness of a testing method.

24. Observational Analysis of Mistakes in Chess Initiation, Using Decision Trees

    Jorge Miranda, Javier Arana, Daniel Lapresa, and 1 more author

    International Journal of Computer Science in Sport, Feb 2025

    DOI
25. Adaptive decision making in the wild: a case study of chess

    Supratik Mondal and Jakub Traczyk

    Thinking & Reasoning, Feb 2025

    Abs DOI

    Chess is a game of strategic thinking and time management, where a player can lose a game on time despite making all the best moves. Finding the best move is a deliberate and energy-intensive process in a game where players are often under time pressure. Therefore, players who can balance this trade-off will have a significant advantage. The current study explores such instances where winning is contingent on how well players balance their accuracy under time pressure. We found that winning players, compared to their opponents, followed a more adaptive decision strategy–they made more theoretical best moves (i.e., accurate moves) in highly critical positions. However, the accuracy difference between the opponents was very similar in less critical positions. We conclude that winning players have a better understanding of when and how to allocate their limited resources efficiently, even when controlling for differences in skill levels, compared to their opponents.

26. Learning when to rank: Estimation of partial rankings from sparse, noisy comparisons

    Sebastian Morel-Balbi and Alec Kirkley

    Feb 2025
27. Mining Security Documentation Practices in OpenAPI Descriptions

    Diana Carolina Muñoz Hurtado, Souhaila Serbout, and Cesare Pautasso

    Mar 2025

    Abs

    Security is an integral requirement of any trustworthy software architecture, particularly critical for application programming interfaces (APIs). In this paper, we survey security documentation practices, specifically API security schemes related to authentication and authorization, by mining a large collection of OpenAPI descriptions retrieved from open-source GitHub repositories. Our study focuses on detecting existing security schemes and evaluating their prevalence and positioning within API descriptions. We distinguish whether security schemes are introduced locally (at the path or operation level) or globally (for the entire API). Our analysis highlights scenarios where security schemes are featured in APIs in different proportions over time, thus tracking whether the API documentation tends to include more (or less) security details as the API evolves.

28. A Comparison of the Effects of Model Adaptation Techniques on Large Language Models for Non-Linguistic and Linguistic Tasks

    Khoa Nguyen, Sadia Jahan, and Rocky Slavin

    In Proceedings of the 40th ACM/SIGAPP Symposium on Applied Computing, Catania International Airport, Catania, Italy, Mar 2025

    Abs DOI

    Generative large language models (LLMs) have revolutionized natural language processing (NLP) by demonstrating exceptional performance in interpreting and generating human language. There has been some exploration of their application to non-linguistic tasks, which could lead to significant advancements in fields that rely heavily on structured data and specialized knowledge. However, there has been limited direct comparison of the effects of model adaptation techniques for non-linguistic compared to linguistic tasks with LLMs. To this end, the work in this paper investigates the effects of fine-tuning and few-shot learning on pre-trained LLMs for non-linguistic tasks using chess puzzles as a case study task. We compare the impact of fine-tuning and few-shot learning on models performing the same task represented in both chess notation (i.e., non-linguistic data) and natural language descriptions of the same chess notations (i.e., natural language data). Our experiments with Mixtral-8x7B-v0.1 and Meta-Llama-3-70B resulted in a 5% lower average increase in performance after fine-tuning for non-linguistic tasks compared to linguistic tasks. Similarly, few-shot learning on pre-trained models exhibited a 3% lower average increase in performance for on non-linguistic tasks compared to linguistic tasks. Furthermore, few-shot learning on fine-tuned models resulted in a significant accuracy drop, particularly for Mixtral, with a 24.82% decrease for non-linguistic tasks. These results suggest that fine-tuning and few-shot learning for generative LLMs have stronger effects on linguistic tasks and their data than for non-linguistic.

29. Soft Prompts for Evaluation: Measuring Conditional Distance of Capabilities

    Ross Nordby

    Mar 2025
30. Leveraging Systems and Control Theory for Social Robotics: A Model-Based Behavioral Control Approach to Human-Robot Interaction

    Maria Morão Patrı́cio and Anahita Jamshidnejad

    Mar 2025
31. Inferring Piece Value in Chess and Chess Variants

    Steven Pav

    Mar 2025

    Abs

    We use logistic regression to estimate the value of the pieces in standard chess and several chess variants, namely Chess 960, Atomic chess, Antichess, and Horde chess. We perform our regressions on several years of data from Lichess, the free and open-source internet chess server. We use the published player ratings to control for the confounding effect of differential player skill. We adjust for the attenuation bias in regressions due to the noise in observed ratings. We find that major piece values, relative to the value of a pawn, are fairly consistent with historical valuation systems. However we find slightly higher value to bishops than knights. We find that piece values are smaller, in absolute value, in Atomic and Antichess than standard chess. We also present approximate values of the pieces to equalize odds when players of varying skill face off.

32. Automatically Augmenting GitHub Issues with Informative User Reviews

    Arthur Pilone, Marco Raglianti, Michele Lanza, and 2 more authors

    In 2025 International Conference on Software Maintenance and Evolution (ICSME), Mar 2025

    Abs

    Development teams for mobile applications can receive thousands of user reviews daily. At the same time, these developers use different communication channels, such as the GitHub issue tracker. Although GitHub issues are accessible and manageable for developers, their content often differs starkly from what users write in app reviews. Issues may lack steps to reproduce bugs or insights that justify the priority of new feature requests. The sheer volume of user reviews for a popular app, combined with their heterogeneity and varying quality, makes manual integration into issue trackers unfeasible. We present an approach that automatically augments GitHub issues with informative user reviews to bridge the gap between user feedback and developer-managed issues. Using a state-of-the-art large language model (LLM), our approach automatically retrieves user reviews with high semantic textual similarity (STS) to the issue content and suggests reviews that augment developers’ understanding of the issue. In this paper, we present large-scale quantitative and qualitative analyses to assess the feasibility of enriching development workflows with user-written information. Using over 37,000 issues and 750,000 reviews from 19 popular Free/Libre/Open Source Software (FLOSS) mobile applications, our approach augments 3,017 (8%) issues with 7,287 (1%) potentially informative reviews. In addition to providing insights into user-reported bugs and feature requests, the information from these matches points toward a novel and promising way to leverage user reviews for concerted app evolution.

33. A metacognitive appraisal of quitting

    Hariharan Purohit and Nisheeth Srivastava

    Proceedings of the Annual Meeting of the Cognitive Science Society, Mar 2025

    Abs

    Stopping decisions are frequently modeled as decisions to switch to alternative activities once the current activity stops being adequately rewarding, such as in optimal foraging theory, as well as more recent metacognitive models. However, the sense of stopping and making decisions in such frameworks is highly platonic, with both decisions and stopping actions occurring instantaneously. In contrast, the phenomenology of quitting actions that one is undertaking appears to be temporally extended and metacognitively challenging. We study the metacognitive covariates of quitting decisions made by chess players using a large database of chess games sourced from an online chess portal. Our analysis reveals that players tend to persevere when they are playing against stronger opponents and after having played poor moves. We also find that a history of quitting games makes players more likely to quit in future games, but that having recently quit in a game offers some protective effect against quitting. Finally, we find that quitting a game makes it more likely that a player will play a game again soon. We place these results in the context of modeling quitting as a metacognitive choice affected by multiple competing goals.

34. ’Sounds like a skill issue’: what makes you quit at chess?

    Hariharan Purohit and Nisheeth Srivastava

    Mar 2025
35. EloMetrics: Advanced Outcome Prediction for Chess Matches with Elo Ratings and Logistic Regression

    Ma. Julianna Re-an DG. Reyes, Eirnan Dicreto, Emmanuel Gabriel D. Santos, and 2 more authors

    In 2025 International Conference on Electronics, Information, and Communication (ICEIC), Mar 2025

    DOI
36. Coordination in Multi-Agent LLM Systems: The Role of a Question-Asking Agent in Guiding Collaborative Consensus

    Keon Roohani

    Apr 2025
37. Beyond Workload: Paving the Road for the Next Generation of Implicit Prefrontal Cortex Based Brain-Computer Interfaces

    Matthew Russell

    Apr 2025

    Second two keywords are from the defense page: https://www.cs.tufts.edu/t/colloquia/current/?event=1651

    Abs

    The rapidly evolving field of Human-Computer Interaction (HCI) faces a fundamental constraint: the limited bandwidth of information exchange between users and computing systems. One promising approach to increasing this bandwidth is implicit interaction: a paradigm in which applications modify their state based on information gleaned from users, rather than direct input. Within the context of reading such information from human neural signals, this concept is formally recognized as implicit Brain-Computer Interfaces (implicit BCI). My work focuses on implicit BCIs which measure the prefrontal cortex (PFC); early prototypes have successfully leveraged the PFC to approximate mental workload, but much is left to be understood about the full potential of this region. Through three research projects spanning two brain measurement modalities, this dissertation makes targeted contributions to this area of research. With functional Near-Infrared Spectroscopy (fNIRS), I explore two facets of PFC activation demonstrated in functional Magnetic Resonance Imaging (fMRI)-based neuroscience research which are underexplored in applied contexts: episodic memory and brain-network based classification; in the first project, I study the measurable effects of episodic and working memory within the context of using Large Language Models (LLMs), and in the second project I develop a real-time implicit BCI designed to differentiate between different brain networks. The third project benchmarks low-cost EEG in three studies which distinguish brain states based on different factors: quality of moves made during chess playing, workload levels within standard cognitive psychology tasks, and cognitive states during the tasks. For all studies I use Linear Mixed Models (LMM) to observe macro patterns in the data, and machine learning to explore potential for implicit BCI. Results indicate that, in addition to the well-understood concept of measuring singular aspects of consciousness across a gradient (e.g. workload), promising potential exists for leveraging the PFC towards classification across tasks which engage different cognitive processes, both with fNIRS and low-cost EEG. Further, careful consideration of “noise” in implicit BCI introduces a new idea: Human-Sensor-Computer Interaction (HSCI). Taken together, this dissertation provides relevant context to inform the next generation of Human-Sensor-Computer systems, including PFC-based interfaces stretching past workload, and beyond.

38. Decoding Chess Puzzle Play and Standard Cognitive Tasks for BCI: A Low-Cost EEG Study

    Matthew Russell, Samuel Youkeles, William Xia, and 3 more authors

    CoRR, Apr 2025

    DOI
39. Domain Obedient Deep Learning

    Soumadeep Saha

    Apr 2025

    Check if http://hdl.handle.net/10263/7608 works and replace url

    Abs

    Deep learning, a family of data-driven artificial intelligence techniques, has shown immense promise in a plethora of applications, and it has even outpaced experts in several domains. However, unlike symbolic approaches to learning, these methods fall short when it comes to abiding by and learning from pre-existing established principles. This is a significant deficit for deployment in critical applications such as robotics, medicine, industrial automation, etc. For a decision system to be considered for adoption in such fields, it must demonstrate the ability to adhere to specified constraints, an ability missing in deep learning-based approaches. Exploring this problem serves as the core tenet of this dissertation. This dissertation starts with an exploration of the abilities of conventional deep learning-based systems vis-à-vis domain coherence. A large-scale rule-annotated dataset is introduced to mitigate the pronounced lack of suitable constraint adherence evaluation benchmarks, and with its aid, the rule adherence abilities of vision systems are analyzed. Additionally, this study probes language models to elicit their performance characteristics with regard to domain consistency. Examination of these language models with interventions illustrates their ineptitude at obeying domain principles, and a mitigation strategy is proposed. This is followed by an exploration of techniques for imbuing deep learning systems with domain constraint information. Also, a comprehensive study of standard evaluation metrics and their blind spots pertaining to domain-aware performance estimation is undertaken. Finally, a novel technique to enforce constraint compliance in models without training is introduced, which pairs a search strategy with large language models to achieve cutting-edge performance. A key highlight of this dissertation is the emphasis on addressing pertinent real-world problems with scalable and practicable solutions. We hope the results presented here pave the way for wider adoption of deep learning-based systems in pivotal situations with enhanced confidence in their trustworthiness.

40. The Memory Premium

    Yuval Salant, Jorg L Spenkuch, and David Almog

    Apr 2025

    Abs DOI

    We explore the role of memory for choice behavior in unfamiliar environments. Using a unique data set, we document that decision makers exhibit a "memory premium." They tend to choose in-memory alternatives over out-of-memory ones, even when the latter are objectively better. Consistent with well-established regularities regarding the inner workings of human memory, the memory premium is associative, subject to interference and repetition effects, and decays over time. Even as decision makers gain familiarity with the environment, the memory premium remains economically large. Our results imply that the ease with which past experiences come to mind plays an important role in shaping choice behavior.

41. Iterative Inference in a Chess-Playing Neural Network

    Elias Sandmann, Sebastian Lapuschkin, and Wojciech Samek

    Apr 2025

    Abs

    Do neural networks build their representations through smooth, gradual refinement, or via more complex computational processes? We investigate this by extending the logit lens to analyze the policy network of Leela Chess Zero, a superhuman chess engine. We find strong monotonic trends in playing strength and puzzle-solving ability across layers, yet policy distributions frequently follow non-smooth trajectories. Evidence for this includes correct puzzle solutions that are discovered early but subsequently discarded, move rankings that remain poorly correlated with final outputs, and high policy divergence until late in the network. These findings contrast with the smooth distributional convergence typically observed in language models.

42. The role of expertise, impulsivity, and preference for intuition on decision quality

    Robin Schrödter, Katrin Heyers, Jan Birkemeyer, and 1 more author

    Personality and Individual Differences, Apr 2025

    Abs DOI

    Decision-making researchers often face a trade-off when conducting controlled laboratory experiments, as these can limit the ability to identify stable relationships between decision-making quality and individual differences, such as expertise or personality traits. This study introduces an innovative paradigm that leverages the objective assessment capabilities of artificial intelligence in a naturalistic online chess setting. Ninety-four participants evaluated tactical chess positions and identified optimal moves under various time control conditions, followed by personality assessments. Expertise, measured by online Elo rating, emerged as a key predictor, accounting for 55 % of the variance in decision quality and 61 % in evaluation accuracy, underscoring the precision of the chosen approach. The study also highlights the significant impact of time control on decision-making. Additionally, the paradigm shows promise in examining the interplay between personality factors and decision-making processes, with a notable correlation between higher impulsivity scores and faster response times. However, high impulsivity was not associated with reduced decision quality, raising questions about the validity of this measurement. Overall, the results suggest that the chess paradigm, accessible to a broad sample due to the widespread appeal of online chess, provides a powerful tool that combines laboratory precision with real-world relevance.

43. Learning in online chess increases with more time spent thinking and diversity of experience

    Lisa Schut, Evan Russek, Ionatan Kuperwajs, and 3 more authors

    In , Apr 2025

    Abs

    What factors of our learning experiences enable us to best acquire complex skills? Recent ideas from artificial intelligence point to two such factors: (1) a balance of real experience with simulated experience acquired during planning itself, and (2) appropriate diversity in training examples. To test whether these factors influence the development of human expertise, we analyzed data from 1,873 chess players on the online platform Lichess, each of whom played hundreds to thousands of games over months to years. We found that both the time spent planning before moves and the diversity of opening positions encountered predict skill improvement over time. These findings suggest that principles shaping the development of expertise in artificial intelligence systems may also apply to human learning.

44. Human-Aligned Chess AI: A Multitask Transformer for Humanlike Decision-Making

    Easwar Gnana Hari Sekar and Roger Jin

    In 2025 IEEE Conference on Artificial Intelligence (CAI), Apr 2025

    DOI
45. Stress and Strategic Decision Making

    Benjamin G. Serpell, Blair T. Crewther, Phillip J. Fourie, and 2 more authors

    Adaptive Human Behavior and Physiology, Jun 2025

    Abs DOI

    Psychology and social science research offer some promising work in the field of decision-making science. However, given the qualitative nature of much of this research, understanding some physiological bases of decision-making may assist by providing more objectivity. The purpose of this study, therefore, was to explore hormonal and neurophysiological biomarkers of stress relative to strategic decision making, with and without an accompanying exercise stress.

46. Seqret: Mining Rule Sets from Event Sequences

    Aleena Siji, Joscha Cüppers, Osman Ali Mian, and 1 more author

    Jun 2025
47. A Behavior-Based Knowledge Representation Improves Prediction of Players’ Moves in Chess by 25%

    Benny Skidanov, Daniel Erbesfeld, Gera Weiss, and 1 more author

    Jun 2025
48. Maia-2: a unified model for human-AI alignment in chess

    Zhenwei Tang, Difan Jiao, Reid McIlroy-Young, and 3 more authors

    In Proceedings of the 38th International Conference on Neural Information Processing Systems, Vancouver, BC, Canada, Jun 2025

    Abs

    There are an increasing number of domains in which artificial intelligence (AI) systems both surpass human ability and accurately model human behavior. This introduces the possibility of algorithmically-informed teaching in these domains through more relatable AI partners and deeper insights into human decision-making. Critical to achieving this goal, however, is coherently modeling human behavior at various skill levels. Chess is an ideal model system for conducting research into this kind of human-AI alignment, with its rich history as a pivotal testbed for AI research, mature superhuman AI systems like AlphaZero, and precise measurements of skill via chess rating systems. Previous work in modeling human decision-making in chess uses completely independent models to capture human style at different skill levels, meaning they lack coherence in their ability to adapt to the full spectrum of human improvement and are ultimately limited in their effectiveness as AI partners and teaching tools. In this work, we propose a unified modeling approach for human-AI alignment in chess that coherently captures human style across different skill levels and directly captures how people improve. Recognizing the complex, non-linear nature of human learning, we introduce a skill-aware attention mechanism to dynamically integrate players’ strengths with encoded chess positions, enabling our model to be sensitive to evolving player skill. Our experimental results demonstrate that this unified framework significantly enhances the alignment between AI and human players across a diverse range of expertise levels, paving the way for deeper insights into human decision-making and AI-guided teaching tools. Our implementation is available https://github.com/CSSLab/maia2.

49. Is Elo Rating Reliable? A Study Under Model Misspecification

    Shange Tang, Yuanhao Wang, and Chi Jin

    Jun 2025
50. SEAM: Semantically Equivalent Across Modalities Benchmark for Vision-Language Models

    Zhenwei Tang, Difan Jiao, Blair Yang, and 1 more author

    In Second Conference on Language Modeling, Jun 2025

    Abs

    Evaluating whether vision-language models (VLMs) reason consistently across representations is challenging because modality comparisons are typically confounded by task differences and asymmetric information. We introduce SEAM, a benchmark that pairs semantically equivalent inputs across four domains that have existing standardized textual and visual notations. By employing distinct notation systems across modalities, in contrast to OCR-based image-text pairing, SEAM provides a rigorous comparative assessment of the textual-symbolic and visual-spatial reasoning capabilities of VLMs. Across 21 contemporary models, we observe systematic modality imbalance: vision frequently lags language in overall performance, despite the problems containing semantically equivalent information, and cross-modal agreement is relatively low. Our error analysis reveals two main drivers: textual perception failures from tokenization in domain notation and visual perception failures that induce hallucinations. We also show that our results are largely robust to visual transformations. SEAM establishes a controlled, semantically equivalent setting for measuring and improving modality-agnostic reasoning.

51. East Meets East: Inside The 2024 World Chess Championship In Singapore

    Olimpiu G Urcan

    Jun 2025

    Abs DOI

    This book presents a compelling account of the historic FIDE world chess championship match between Ding Liren of China and Gukesh Dommaraju of India, held in Singapore from November 25 to December 13, 2024. Sponsored by Google, the 14-game match marked several significant milestones: the first All-Asian world chess championship, the first to be hosted in Singapore, and the momentous crowning of the youngest undisputed world chess champion in history. Through detailed reports, in-depth analyses of each game, and a curated collection of over a hundred quality photographs, the book explores the players’ remarkable journeys, offering insights into their distinct playing styles, rigorous preparation, and the psychological challenges they faced both on and off the board. Set against the dynamic cultural backdrop of Singapore, it also examines the growing prominence of Asia in the world of chess. Combining strategic depth, expert commentary, and behind-the-scenes perspectives, this work provides an engaging narrative of resilience, rivalry, and excellence, making it an indispensable read for chess enthusiasts and sports aficionados.

52. Mixture of Experts Made Intrinsically Interpretable

    Xingyi Yang, Constantin Venhoff, Ashkan Khakzar, and 4 more authors

    Jun 2025
53. Implicit Search via Discrete Diffusion: A Study on Chess

    Jiacheng Ye, Zhenyu Wu, Jiahui Gao, and 4 more authors

    In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025, Jun 2025

    Abs

    In the post-AlphaGo era, there has been a renewed interest in search techniques such as Monte Carlo Tree Search (MCTS), particularly in their application to Large Language Models (LLMs). This renewed attention is driven by the recognition that current next-token prediction models often lack the ability for long-term planning. Is it possible to instill search-like abilities within the models to enhance their planning abilities without relying on explicit search? We propose DiffuSearch , a model that does \textitimplicit search by looking into the future world via discrete diffusion modeling. We instantiate DiffuSearch on a classical board game, Chess, where explicit search is known to be essential. Through extensive controlled experiments, we show DiffuSearch outperforms both the searchless and explicit search-enhanced policies. Specifically, DiffuSearch outperforms the one-step policy by 19.2% and the MCTS-enhanced policy by 14% on action accuracy. Furthermore, DiffuSearch demonstrates a notable 30% enhancement in puzzle-solving abilities compared to explicit search-based policies, along with a significant 540 Elo increase in game-playing strength assessment. These results indicate that implicit search via discrete diffusion is a viable alternative to explicit search over a one-step policy. All codes are publicly available at \hrefhttps://github.com/HKUNLP/DiffuSearchhttps://github.com/HKUNLP/DiffuSearch

54. Biological Processing Units: Leveraging an Insect Connectome to Pioneer Biofidelic Neural Architectures

    Siyu Yu, Zihan Qin, Tingshan Liu, and 4 more authors

    Jun 2025
55. General search techniques without common knowledge for imperfect-information games, and application to superhuman Fog of War chess

    Brian Hu Zhang and Tuomas Sandholm

    Jun 2025
56. Human-Aligned Chess With a Bit of Search

    Yiming Zhang, Athul Paul Jacob, Vivian Lai, and 2 more authors

    In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025, Jun 2025

    Abs

    Chess has long been a testbed for AI’s quest to match human intelligence, and in recent years, chess AI systems have surpassed the strongest humans at the game. However, these systems are not human-aligned; they are unable to match the skill levels of all human partners or model human-like behaviors beyond piece movement. In this paper, we introduce Allie, a chess-playing AI designed to bridge the gap between artificial and human intelligence in this classic game. Allie is trained on log sequences of real chess games to model the behaviors of human chess players across the skill spectrum, including non-move behaviors such as pondering times and resignations In offline evaluations, we find that Allie exhibits humanlike behavior: it outperforms the existing state-of-the-art in human chess move prediction and ponders at critical positions. The model learns to reliably assign reward at each game state, which can be used at inference as a reward function in a novel time-adaptive Monte-Carlo tree search (MCTS) procedure, where the amount of search depends on how long humans would think in the same positions. Adaptive search enables remarkable skill calibration; in a large-scale online evaluation against players with ratings from 1000 to 2600 Elo, our adaptive search method leads to a skill gap of only 49 Elo on average, substantially outperforming search-free and standard MCTS baselines. Against grandmaster-level (2500 Elo) opponents, Allie with adaptive search exhibits the strength of a fellow grandmaster, all while learning exclusively from humans.

2024

1. Unleashing Artificial Cognition: Integrating Multiple AI Systems

   Muntasir Adnan, Buddhi Gamage, Zhiwei Xu, and 2 more authors

   CoRR, Jun 2024

   DOI
2. Wasm-R3: Record-Reduce-Replay for Realistic and Standalone WebAssembly Benchmarks

   Doehyun Baek, Jakob Getz, Yusung Sim, and 4 more authors

   Proc. ACM Program. Lang., Oct 2024

   Abs DOI

   WebAssembly (Wasm for short) brings a new, powerful capability to the web as well as Edge, IoT, and embedded systems. Wasm is a portable, compact binary code format with high performance and robust sandboxing properties. As Wasm applications grow in size and importance, the complex performance characteristics of diverse Wasm engines demand robust, representative benchmarks for proper tuning. Stopgap benchmark suites, such as PolyBenchC and libsodium, continue to be used in the literature, though they are known to be unrepresentative. Porting of more complex suites remains difficult because Wasm lacks many system APIs and extracting real-world Wasm benchmarks from the web is difficult due to complex host interactions. To address this challenge, we introduce Wasm-R3, the first record and replay technique for Wasm. Wasm-R3 transparently injects instrumentation into Wasm modules to record an execution trace from inside the module, then reduces the execution trace via several optimizations, and finally produces a replay module that is executable standalone without any host environment-on any engine. The benchmarks created by our approach are (i) realistic, because the approach records real-world web applications, (ii) faithful to the original execution, because the replay benchmark includes the unmodified original code, only adding emulation of host interactions, and (iii) standalone, because the replay benchmarks run on any engine. Applying Wasm-R3 to web-based Wasm applications in the wild demonstrates the correctness of our approach as well as the effectiveness of our optimizations, which reduce the recorded traces by 99.53% and the size of the replay benchmark by 9.98%. We release the resulting benchmark suite of 27 applications, called Wasm-R3-Bench, to the community, to inspire a new generation of realistic and standalone Wasm benchmarks.

3. Skill vs. Chance Quantification for Popular Card & Board Games

   Tathagata Banerjee, Anushka De, Subhamoy Maitra, and 1 more author

   CoRR, Oct 2024

   DOI
4. Towards Robust and Cooperative Learning Algorithms

   Mark Beliaev

   Oct 2024

   https://escholarship.org/uc/item/6mc2d7q3
5. Solution Path Heuristics for Predicting Difficulty and Enjoyment Ratings of Roguelike Level Segments

   Colan Biemer and Seth Cooper

   In Proceedings of the 19th International Conference on the Foundations of Digital Games, Worcester, MA, USA, Oct 2024

   Abs DOI

   When generating levels, algorithmically evaluating the results is essential. In this paper, we looked at predicting a level’s difficulty and enjoyment. Past work has approached this problem for puzzle games like Sudoku by analyzing the characteristics of the initial level, the solved level, and the process that led to that solution. In this work, we examined a set of heuristics for Roguelike levels and their solutions, and their relationship to subjective player ratings of the levels. We gathered ratings of difficulty and enjoyment of levels in a study with 143 players. We ran an ablation study on the set of heuristics to find the best combination of heuristics for predicting difficulty and enjoyment with a linear regression model, and found solution path-based heursitics performed well. However, these models did not outperform a simple baseline for predicting enjoyment. Jaccard similarity on paths–a method we have not seen used in the field of game AI–was a useful predictor of difficulty. Testing proximity to enemies across a solution path is the only heuristic needed to predict how enjoyable a level will be.

6. Estimating the Puzzlingness of Chess Puzzles

   Sebastian Björkqvist

   In IEEE International Conference on Big Data, Big Data 2024, Washington DC, USA, December 15-18, 2024, Oct 2024
7. Decoding Chess Mastery: A Mechanistic Analysis of a Chess Language Transformer Model

   Austin L. Davis and Gita Sukthankar

   In Artificial General Intelligence: 17th International Conference, AGI 2024, Seattle, WA, USA, August 13–16, 2024, Proceedings, SEATTLE, WA, USA, Oct 2024

   Abs DOI

   Mechanistic interpretability (MI) studies aim to identify the specific neural pathways that underlie decision-making in neural networks. Here we analyze both the horizontal and vertical information flows of a chess-playing transformer. This paper introduces a new taxonomy of chessboard attention patterns that synchronize to guide move selection. Our findings show that the early layers of the chess transformer correctly identify moves that are highly ranked by the final layer. Experiments conducted on human chess players laid the foundation for much of our current understanding of human problem-solving, cognition, and visual memory. We believe that the study of chess language transformers may be an equally fruitful research area for AGI systems.

8. Performance Envelopes of Linear Probes for Latent Representation Edits in GPT Models

   Austin L Davis and Gita Sukthankar

   In 2024 International Conference on Machine Learning and Applications (ICMLA), Oct 2024
9. Generative Reinforcement Learning with Transformers

   Gregoire Deletang, Anian Ruoss, Li Kevin Wenliang, and 5 more authors

   Oct 2024
10. Easy2Hard-Bench: Standardized Difficulty Labels for Profiling LLM Performance and Generalization

    Mucong Ding, Chenghao Deng, Jocelyn Choo, and 8 more authors

    In NeurIPS 2024, Sep 2024

    Abs

    While generalization over tasks from easy to hard is crucial to profile language models (LLMs), the datasets with fine-grained difficulty annotations for each problem across a broad range of complexity are still blank. Aiming to address this limitation, we present Easy2Hard-Bench, a consistently formatted collection of 6 benchmark datasets spanning various domains, such as mathematics and programming problems, chess puzzles, and reasoning questions. Each problem within these datasets is annotated with numerical difficulty scores. To systematically estimate problem difficulties, we collect abundant performance data on attempts to each problem by humans in the real world or LLMs on the prominent leaderboard. Leveraging the rich performance data, we apply well-established difficulty ranking systems, such as Item Response Theory (IRT) and Glicko-2 models, to uniformly assign numerical difficulty scores to problems. Moreover, datasets in Easy2Hard-Bench distinguish themselves from previous collections by a higher proportion of challenging problems. Through extensive experiments with six state-of-the-art LLMs, we provide a comprehensive analysis of their performance and generalization capabilities across varying levels of difficulty, with the aim of inspiring future research in LLM generalization. The datasets are available at https://huggingface.co/datasets/furonghuang-lab/Easy2Hard-Bench.

11. Turing Tests in Chess: An Experiment Revealing the Role of Human Subjectivity

    Yke Bauke Eisma, Robin Koerts, and Joost de Winter

    Computers in Human Behavior Reports, Sep 2024

    Abs DOI

    With the growing capabilities of AI, technology is increasingly able to match or even surpass human performance. In the current study, focused on the game of chess, we investigated whether chess players could distinguish if they were playing against a human or a computer, and how they achieved this. A total of 24 chess players each played eight 5+0 Blitz games from different starting positions. They played against (1) a human, (2) Maia, a neural network-based chess engine trained to play in a human-like manner, (3) Stockfish 16, the best chess engine available, downgraded to play at a lower level, and (4) Stockfish 16 at its maximal level. The opponent’s move time was fixed at 10 seconds. During the game, participants verbalized their thoughts, and after each game, they indicated by means of a questionnaire whether they thought they had played against a human or a machine and if there were particular moves that revealed the nature of the opponent. The results showed that Stockfish at the highest level was usually correctly identified as an engine, while Maia was often incorrectly identified as a human. The moves of the downgraded Stockfish were relatively often labeled as ‘strange’ by the participants. In conclusion, the Turing test, as applied here in a domain where computers can perform superhumanly, is essentially a test of whether the chess computer can devise suboptimal moves that correspond to human moves, and not necessarily a test of computer intelligence.

12. Checkmating One, by Using Many: Combining Mixture of Experts with MCTS to Improve in Chess

    Felix Helfenstein, Jannis Blüml, Johannes Czech, and 1 more author

    CoRR, Sep 2024

    DOI
13. Detecting Fair Play Violations in Chess Using Neural Networks

    Maksim Iavich and Zura Kevanishvili

    In Proceedings of 29th International Conference Information Society and University Studies, Sep 2024
14. Evidence of Learned Look-Ahead in a Chess-Playing Neural Network

    Erik Jenner, Shreyas Kapur, Vasil Georgiev, and 3 more authors

    CoRR, Sep 2024

    DOI
15. Personalized recommendation of chess puzzles

    Aryan Karn, Chinmay Anil Biradar, Aryan Puranik, and 2 more authors

    In Computer Science Engineering: Proceedings of the 1st International Conference on Computing and Intelligent Information Systems (ICCIIS 2024), Bangalore, India, 19-20th April, 2024 Volume 1, Sep 2024
16. Measuring Progress in Dictionary Learning for Language Model Interpretability with Board Game Models

    Adam Karvonen, Benjamin Wright, Can Rager, and 6 more authors

    In ICML 2024 Workshop on Mechanistic Interpretability, Sep 2024
17. Emergent World Models and Latent Variable Estimation in Chess-Playing Language Models

    Adam Karvonen

    CoRR, Sep 2024

    DOI
18. Learning from rewards and social information in naturalistic strategic behavior

    Ionatan Kuperwajs, Bas Opheusden, Evan Russek, and 1 more author

    Aug 2024

    DOI
19. A Few-Shot Embedding Learning Approach for Predicting the Behavior of Individual Chess Players

    Lennart August

    Aug 2024

    Abs

    This study presents a few-shot embedding learning approach to predict the behavior of individual chess players, based on only 100 games. Traditional models have relied on extensive datasets, often requiring thousands of games to achieve accurate move predictions. In contrast, our method leverages a limited number of games to generate dense vector representations, or embeddings, that capture a player’s unique style. We trained a neural network to create these embeddings and used them to predict subsequent moves. Our results indicate that the embedding model performs well across various player sets and can accurately identify players even at scale among a great player population, picking out players with 84% accuracy from among 100k candidates. There are indications that including information on the clock situation during the game improves the embedding process, although our findings are inconclusive. Despite these limitations, our approach shows promise in making personalized chess training more accessible and highlights the potential for embedding learning in human-centered AI applications. Future work will aim to refine both the embedding and move prediction models and explore its application in other domains.

20. Predicting Chess Puzzle Difficulty with Transformers

    Szymon Miłosz and Paweł Kapusta

    In IEEE International Conference on Big Data, Big Data 2024, Washington DC, USA, December 15-18, 2024, Aug 2024
21. Measuring the Digital Welfare of Online Social Systems

    Lillio Mok

    Aug 2024

    http://hdl.handle.net/1807/140863
22. Predictive Analysis of Chess Player Performance: An Analysis of Factors Influencing Competitive Success Using Machine Learning Techniques

    Amar Mujagić, Adnan Mujagić, and Dželila Mehanović

    In Advanced Technologies, Systems, and Applications IX, Aug 2024

    Abs

    The world of competitive chess has long been a captivating arena for intellectual competition, where human intelligence, strategic thinking, and long-term planning converge. This study delves into the intricate web of factors that influence a chess player’s competitive success through the lens of predictive modeling and machine learning techniques.

23. Discovering High-Quality Chess Puzzles Through One Billion Plays with Offline Reinforcement Learning

    Allen Nie, Anirudhan Badrinath, Nicholas Tomlin, and 5 more authors

    Aug 2024
24. An alternative chess rating model based on latent variables

    Patrick O’Rourke

    Aug 2024

    http://dx.doi.org/10.13140/RG.2.2.18931.13604
25. Chess Rating Estimation from Moves and Clock Times Using a CNN-LSTM

    Michael Omori and Prasad Tadepalli

    Aug 2024
26. Sparse but Strategic: Quantitative Insights into Chess Middlegame Complexity

    Justin Ong, Merim Bilalić, and Nemanja Vaci

    Research Square, Aug 2024

    DOI
27. Empirical Evaluation of Concept Probing for Game-Playing Agents

    A\dhalsteinn Pálsson and Yngvi Björnsson

    In ECAI 2024 - 27th European Conference on Artificial Intelligence, 19-24 October 2024, Santiago de Compostela, Spain - Including 13th Conference on Prestigious Applications of Intelligent Systems (PAIS 2024), Aug 2024

    DOI
28. "Hunt Takes Hare": Theming Games Through Game-Word Vector Translation

    Younès Rabii and Michael Cook

    In Proceedings of the 19th International Conference on the Foundations of Digital Games, Worcester, MA, USA, Aug 2024

    Abs DOI

    A game’s theme is an important part of its design – it conveys narrative information, rhetorical messages, helps the player intuit strategies, aids in tutorialisation and more. Thematic elements of games are notoriously difficult for AI systems to understand and manipulate, however, and often rely on large amounts of hand-written interpretations and knowledge. In this paper we present a technique which connects game embeddings, a recent method for modelling game dynamics from log data, and word embeddings, which models semantic information about language. We explain two different approaches for using game embeddings in this way, and show evidence that game embeddings enhance the linguistic translations of game concepts from one theme to another, opening up exciting new possibilities for reasoning about the thematic elements of games in the future.

29. Pairwise Learning to Rank for Chess Puzzle Difficulty Prediction

    Andry Rafaralahy

    In IEEE International Conference on Big Data, Big Data 2024, Washington DC, USA, December 15-18, 2024, Aug 2024
30. The Temporal Differences in Chess Between ADHD and Neurotypical Individuals

    Benjamin Rosales

    Aug 2024
31. Amortized Planning with Large-Scale Transformers: A Case Study on Chess

    Anian Ruoss, Gregoire Deletang, Sourabh Medapati, and 7 more authors

    In The Thirty-eighth Annual Conference on Neural Information Processing Systems, Aug 2024
32. Moves Based Prediction of Chess Puzzle Difficulty with Convolutional Neural Networks

    Dymitr Ruta, Ming Liu, and Ling Cen

    In IEEE International Conference on Big Data, Big Data 2024, Washington DC, USA, December 15-18, 2024, Aug 2024
33. VALUED - Vision and Logical Understanding Evaluation Dataset

    Soumadeep Saha, Saptarshi Saha, and Utpal Garain

    Journal of Data-centric Machine Learning Research, Aug 2024
34. Optimizing language models for chess : the impact of custom notation and Elo-based fine-tuning

    Lars Schmid and Jerome Maag

    Aug 2024

    Abs DOI

    This research investigates the potential for large language models to learn to generate valid chess moves solely through pre-training on chess game data. The primary objective of this study is to investigate the impact of custom notation systems and tokenisation methods specifically designed for use with chess games. The study aims to improve the models’ understanding of game states and move sequences by developing and implementing a custom notation system, xLAN+. This notation system incorporates additional information about captures, checks and checkmates in order to increase the performance of the models. Furthermore, the strategic gameplay capabilities of the models are to be enhanced by fine-tuning them on datasets filtered by Elo rating. This approach postulates that filtering games by skill level can help models develop a deeper understanding of strategic gameplay, thereby improving their ability to generate high-quality moves. The research uses two different architectures, Transformer, based on OpenAI’s GPT-2 configuration, and Mamba, a State Space Model (SSM) optimised for long sequence processing. Initial findings indicate that the custom notation xLAN+ significantly improves the models’ ability to generate valid moves and maintain game state accuracy over extended sequences. The comparison between GPT-2 and Mamba reveals that while both architectures can learn chess rules and generate plausible moves, the SSM offers slight advantages in handling long-range dependencies and maintaining game context. This project demonstrates the potential of language models to learn complex tasks like chess through data-driven approaches, paving the way for their application in other strategic and decision-making domains.

35. Estimating Chess Puzzle Difficulty Without Past Game Records Using a Human Problem-Solving Inspired Neural Network Architecture

    Anan Schütt, Tobias Huber, and Elisabeth André

    In IEEE International Conference on Big Data, Big Data 2024, Washington DC, USA, December 15-18, 2024, Aug 2024
36. Mastering Board Games by External and Internal Planning with Language Models

    John Schultz, Jakub Adamek, Matej Jusup, and 13 more authors

    Aug 2024
37. Bad Crypto: Chessography and Weak Randomness of Chess Games

    Martin Stanek

    Aug 2024
38. Explore the Reasoning Capability of LLMs in the Chess Testbed

    Shu Wang, Lei Ji, Renxi Wang, and 4 more authors

    Aug 2024
39. Massively Parallel Inverse Block-sorting Transforms for bzip2 Decompression on GPUs

    André Weißenberger and Bertil Schmidt

    In Proceedings of the 53rd International Conference on Parallel Processing, Gotland, Sweden, Aug 2024

    Abs DOI

    Lossless data compression has evolved into an indispensable tool for reducing data transfer times in heterogeneous systems. However, performing decompression on host systems can create performance bottlenecks. Accelerator libraries, such as nvCOMP, address this problem by providing custom GPU-enabled versions of some general-purpose compression methods, including Snappy, ZStandard and gzip. However, popular bzip-like compression schemes, which rely on block-sorting transforms have not yet been integrated since their fine-grained parallelization is challenging. With a focus on decompression, we propose novel techniques for fine-grained parallelization of inverse Burrows-Wheeler transform (iBWT) and inverse Move-to-Front (iMTF) transform on GPUs, which enable efficient processing of bzip2-based archives on CUDA-enabled accelerators for the first time. Consequently, we present the first fully GPU-enabled bzip2 decompression pipeline as a use case for the proposed algorithms. Our experimental results reveal speedups of up to 6.1x over a multicore CPU implementation for iBWT, and throughput rates of up to 2400 MB/s for combined iBWT and iMTF on an A100 GPU. For decompression of bzip2 archives, a throughput of over 11.62 GB/s is achieved on a DGX H100 server. The source code of our parallel decoder implementation is available at https://github.com/weissenberger/bzip2gpu.

40. The bread emoji Team’s Submission to the IEEE BigData 2024 Cup: Predicting Chess Puzzle Difficulty Challenge

    Tyler Woodruff, Oleg Filatov, and Marco Cognetta

    In IEEE International Conference on Big Data, Big Data 2024, Washington DC, USA, December 15-18, 2024, Aug 2024
41. Predicting User Perception of Move Brilliance in Chess

    Kamron Zaidi and Michael Guerzhoy

    CoRR, Aug 2024

    DOI
42. IEEE Big Data Cup 2024 Report: Predicting Chess Puzzle Difficulty at KnowledgePit.ai

    Jan Zyśko, Maciej Świechowski, Sebastian Stawicki, and 3 more authors

    In IEEE International Conference on Big Data, Big Data 2024, Washington DC, USA, December 15-18, 2024, Aug 2024

2023

1. Making Superhuman AI More Human in Chess

   Daniel Barrish, Steve Kroon, and Brink Merwe

   In Advances in Computer Games - 18th International Conference, ACG 2023, Virtual Event, November 28-30, 2023, Revised Selected Papers, Aug 2023

   Abs DOI

   Computer chess research has traditionally focused on creating the strongest possible chess engine. Recently, however, attempts have been made to create engines that mimic the playing strength and style of human players. Our research proposes enhancements of models developed in this vein that more accurately imitate master-level players, as well as improve the prediction accuracy of existing models on weaker players. Our proposed enhancements are simple to apply by post-processing the output of existing chess engines. The performance of our enhancements was evaluated and compared using two metrics, prediction accuracy and average centipawn loss. We found that using an ensemble model over search depths maximised prediction accuracy, while an evaluation window filtering approach was preferable with respect to average centipawn loss.

2. Can artificial intelligence identify creativity?: An empirical study

   William Bart

   Journal of Creativity, Aug 2023

   Abs DOI

   This article reports an investigation of the extent to which a chess program with an artificial intelligence component (i.e., Stockfish with NNUE) can identify 10 chess moves that are recognized as outstanding chess moves. Stockfish with NNUE was able to identify seven of the ten moves. Although Stockfish with NNUE is a very powerful chess program, it has some limitations in identifying creative chess moves. There is a discussion of those limitations.

3. On the Limitations of the Elo, Real-World Games are Transitive, not Additive

   Quentin Bertrand, Wojciech Marian Czarnecki, and Gauthier Gidel

   In International Conference on Artificial Intelligence and Statistics, 25-27 April 2023, Palau de Congressos, Valencia, Spain, Aug 2023
4. Quantifying human performance in chess

   Sandeep Chowdhary, Iacopo Iacopini, and Federico Battiston

   Scientific Reports, Feb 2023

   Abs DOI

   From sports to science, the recent availability of large-scale data has allowed to gain insights on the drivers of human innovation and success in a variety of domains. Here we quantify human performance in the popular game of chess by leveraging a very large dataset comprising of over 120 million games between almost 1 million players. We find that individuals encounter hot streaks of repeated success, longer for beginners than for expert players, and even longer cold streaks of unsatisfying performance. Skilled players can be distinguished from the others based on their gaming behaviour. Differences appear from the very first moves of the game, with experts tending to specialize and repeat the same openings while beginners explore and diversify more. However, experts experience a broader response repertoire, and display a deeper understanding of different variations within the same line. Over time, the opening diversity of a player tends to decrease, hinting at the development of individual playing styles. Nevertheless, we find that players are often not able to recognize their most successful openings. Overall, our work contributes to quantifying human performance in competitive settings, providing a first large-scale quantitative analysis of individual careers in chess, helping unveil the determinants separating elite from beginner performance.

5. Quantifying the complexity and similarity of chess openings using online chess community data

   Giordano De Marzo and Vito D. P. Servedio

   Scientific Reports, Apr 2023

   Abs DOI

   Chess is a centuries-old game that continues to be widely played worldwide. Opening Theory is one of the pillars of chess and requires years of study to be mastered. In this paper, we use the games played in an online chess platform to exploit the “wisdom of the crowd” and answer questions traditionally tackled only by chess experts. We first define a relatedness network of chess openings that quantifies how similar two openings are to play. Using this network, we identify communities of nodes corresponding to the most common opening choices and their mutual relationships. Furthermore, we demonstrate how the relatedness network can be used to forecast future openings players will start to play, with back-tested predictions outperforming a random predictor. We then apply the Economic Fitness and Complexity algorithm to measure the difficulty of openings and players’ skill levels. Our study not only provides a new perspective on chess analysis but also opens the possibility of suggesting personalized opening recommendations using complex network theory.

6. ChessGPT: Bridging Policy Learning and Language Modeling

   Xidong Feng, Yicheng Luo, Ziyan Wang, and 6 more authors

   In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023, Apr 2023
7. Determining Chess Piece Values Using Machine Learning

   Aditya Gupta, Christopher Grattoni, and Arnav Gupta

   Journal of Student Research, Feb 2023

   Abs DOI

   This paper attempts to generate point values for chess pieces, as alternatives to the commonly accepted chess piece values. We use a database of over a million online chess games to heuristically determine the value of a chess piece, by using material imbalances to predict game results. We then explore how piece values change when we analyze material imbalances at various stages of a chess game. As further exploration, we determine what practical values chess pieces and imbalances have at various rating ranges. This creates practical data that players of varying rating can use to aid in chess calculation, as opposed to the rigid values that are typically accepted.</p>

8. Building a Natural Language Chess Engine with Pretraining and Instruction Fine-Tuning

   Bowen Jiang

   Feb 2023

   Stanford CS224N Custom Project, Winter 2023 (https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1234/project.html)

   Abs

   Although pretrained large language models (LLMs) can generate convincing natural language about games like chess, they lack positional and contextual knowledge and as such are poor game-playing agents. In this project, I utilize language pretaining; instruction fine-tuning, an additional training regimen with chess-specific tasks presented in natural language; and chain-of-thought prompting, a natural language description of problem reasoning prepended to the answer of a problem, to improve the performance of LLMs at chess move generation (validity/legality and quality of moves). I show that fine-tuned GPT-2-XL, a 1.5B parameter LLM, performs favorably well at move generation compared to ChatGPT with few-shot learning; I also validate the additional benefits of chain-of-thought prompting compared to plain prompts in ChatGPT while highlighting tradeoffs between the quality of natural language and the quality of chess when more verbose prompts are used in the smaller GPT-2-XL.

9. Merging Neural Networks with Traditional Evaluations in Crazyhouse

   Anei Makovec, Johanna Pirker, and Matej Guid

   In Advances in Computer Games: 18th International Conference, ACG 2023, Virtual Event, November 28–30, 2023, Revised Selected Papers, Siegen, Germany, Feb 2023

   Abs DOI

   In the intricate landscape of game-playing algorithms, Crazyhouse stands as a complex variant of chess where captured pieces are reintroduced, presenting unique evaluation challenges. This paper explores a hybrid approach that combines traditional evaluation functions with neural network-based evaluations, seeking an optimal balance in performance. Through rigorous experimentation, including self-play, matchups against a variant of the renowned program, Go-deep experiments, and score deviations, we present compelling evidence for the effectiveness of a weighted sum of both evaluations. Remarkably, in our experiments, the combination of 75% neural network and 25% traditional evaluation consistently emerged as the most effective choice. Furthermore, we introduce the use of Best-Change rates, which have previously been associated with evaluation quality, in the context of Monte Carlo tree search-based algorithms.

10. Improving the Strength of Human-Like Models in Chess

    Saumik Narayanan, Kassa Korley, Chien-Ju Ho, and 1 more author

    Feb 2023
11. Investigation of the Sicilian Defense: Winning rates and strategic discrimination

    Ziming Song

    Interdisciplinary Humanities and Communication Studies, Feb 2023
12. Can higher social status of competitors cause decision makers to commit more errors?

    Li Qian Tay

    In Proceedings of the Annual Meeting of the Cognitive Science Society, Feb 2023
13. Predicting Chess Player Rating Based on a Single Game

    Tim Tijhuis, Paris Mavromoustakos Blom, and Pieter Spronck

    In IEEE Conference on Games, CoG 2023, Boston, MA, USA, August 21-24, 2023, Feb 2023

    DOI
14. A Method for Estimating Online Chess Game Player Ratings with Decision Tree

    Habuki Yamada, Nobuko Kishi, Masato Oguchi, and 1 more author

    In IEEE International Conference on Big Data and Smart Computing, BigComp 2023, Jeju, Republic of Korea, February 13-16, 2023, Feb 2023

    DOI
15. Chess Piece Image Recognition Using Transfer Learning, Simple Neural Network, and Convolutional Neural Network

    Gabriel Yohanes, Mario Nursalim, Nicholas, and 1 more author

    In 2023 4th International Conference on Artificial Intelligence and Data Sciences (AiDAS), Feb 2023

    DOI

2022

1. A Practical Algorithm for Chess Unwinnability

   Miguel Ambrona

   In 11th International Conference on Fun with Algorithms, FUN 2022, May 30 to June 3, 2022, Island of Favignana, Sicily, Italy, Feb 2022

   DOI
2. Deep learning for computer chess (part 1)

   Manav Arora

   Feb 2022

   Final Year Project (FYP)

   Abs

   This report encompasses the implementation of two state-of-the-art machine learning algorithms for evaluating chess positions. The first algorithm makes use of artificial neural networks and manual feature representation thus closely following the implementation and architecture of Matthew Lai’s Giraffe. Giraffe learns to play chess largely by self-play and derives its own rules based on the data [1]. Giraffe was implemented as a 7 class classification problem on a dataset of over 10,000 grandmaster level games. Four different implementations of Giraffe were explored covering two different architectures and the effects of regularization on the model performance. The second algorithm implemented goes through an unsupervised learning phase to perform feature extraction followed by a supervised learning phase thus replicating David Eli’s DeepChess. DeepChess evaluates chess positions using a deep neural network without any a priori knowledge regarding the rules of chess. DeepChess is implemented as a siamese network of two disjoint deep belief networks connected to each other by fully connected layers [2]. This architecture was implemented as a binary classification problem on the same dataset as Giraffe and also on a larger dataset of LiChess games. Different implementations of DeepChess covering different training methodologies and parameter sets were executed.

3. Towards the automatic synthesis of interpretable chess tactics

   Abhijeet Krishnan and Chris Martens

   In Proceedings of the Explainable Agency in Artificial Intelligence Workshop, 36th AAAI Conference on Artificial Intelligence, Mar 2022
4. Synthesizing interpretable chess tactics from player games

   Abhijeet Krishnan and Chris Martens

   In Proceedings of the Workshop on Artificial Intelligence for Strategy Games (SG) and Esports Analytics (EA), 18th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Oct 2022
5. Towards Transparent Cheat Detection in Online Chess: An Application of Human and Computer Decision-Making Preferences

   Thijs Laarhoven and Aditya Ponukumati

   In Computers and Games - International Conference, CG 2022, Virtual Event, November 22-24, 2022, Revised Selected Papers, Oct 2022

   DOI
6. Gambits: Theory and evidence

   Shiva Maharaj, Nick Polson, and Christian Turk

   Applied Stochastic Models in Business and Industry, Oct 2022

   Abs DOI

   Abstract Gambits are central to human decision-making. Our goal is to provide a theory of Gambits. A Gambit is a combination of psychological and technical factors designed to disrupt predictable play. Chess provides an environment to study gambits and behavioral game theory. Our theory is based on the Bellman optimality path for sequential decision-making. This allows us to calculate the Q\ Q \\-values of a Gambit where material (usually a pawn) is sacrificed for dynamic play. On the empirical side, we study the effectiveness of a number of popular chess Gambits. This is a natural setting as chess Gambits require a sequential assessment of a set of moves (a.k.a. policy) after the Gambit has been accepted. Our analysis uses Stockfish 14.1 to calculate the optimal Bellman Q\ Q \\-values, which fundamentally measures if a position is winning or losing. To test whether Bellman’s equation holds in play, we estimate the transition probabilities to the next board state via a database of expert human play. This then allows us to test whether the Gambiteer is following the optimal path in his decision-making. Our methodology is applied to the popular Stafford and reverse Stafford (a.k.a. Boden–Kieretsky–Morphy) Gambit and other common ones including the Smith–Morra, Goring, Danish and Halloween Gambits. We build on research in human decision-making by proving an irrational skewness preference within agents in chess. We conclude with directions for future research.

7. Learning Models of Individual Behavior in Chess

   Reid McIlroy-Young, Russell Wang, Siddhartha Sen, and 2 more authors

   In KDD ’22: The 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Washington, DC, USA, August 14 - 18, 2022, Oct 2022

   DOI
8. Check Mate: A Sanity Check for Trustworthy AI

   Sascha Mücke and Lukas Pfahler

   In Proceedings of the LWDA 2022 Workshops: FGWM, FGKD, and FGDB, Hildesheim (Germany), Oktober 5-7th, 2022, Oct 2022
9. Time spent thinking in online chess reflects the value of computation

   Evan Russek, Daniel Acosta-Kane, Bas Opheusden, and 2 more authors

   PsyArXiv, Oct 2022

   Abs DOI

   Although artificial intelligence systems can now outperform humans in a variety of domains, they still lag behind in the ability to arrive at good solutions to problems using limited resources. Recent proposals have suggested that the key to this cognitive efficiency is intelligent selection of the situations in which computational resources are spent. We tested this hypothesis in the domain of complex planning by analyzing how humans managed time available for thinking in over 12 million online chess matches. We found that players spent more time thinking in board positions where planning was more beneficial. This effect was greater in stronger players, and additionally strengthened by considering only the information available to the player at the time of choice. Finally, we found that the qualitative features of this relationship were consistent with a policy that considers the empirically-measured cost of spending time in chess. This provides evidence that human efficiency is supported by intelligent selection of when to apply computation.

10. Complexity and Satisficing: Theory with Evidence from Chess

    Yuval Salant and Jorg L Spenkuch

    Apr 2022

    Abs DOI

    We develop a satisficing model of choice in which the available alternatives differ in their inherent complexity. We assume–and experimentally validate–that complexity leads to errors in the perception of alternatives’ values. The model yields sharp predictions about the effect of complexity on choice probabilities, some of which qualitatively contrast with those of maximization-based choice models. We confirm the predictions of the satisficing model–and thus reject maximization–in a novel data set with information on hundreds of millions of real-world chess moves by highly experienced players. These findings point to the importance of complexity and satisficing for decision making outside of the laboratory.

11. Measuring the Non-Transitivity in Chess

    Ricky Sanjaya, Jun Wang, and Yaodong Yang

    Algorithms, Apr 2022

    DOI
12. Dataset Related Experimental Investigation of Chess Position Evaluation Using a Deep Neural Network

    Dawid Wieczerzak and Pawel Czarnul

    In Parallel Processing and Applied Mathematics - 14th International Conference, PPAM 2022, Gdansk, Poland, September 11-14, 2022, Revised Selected Papers, Part I, Apr 2022

    DOI
13. Topological Data Analysis in chess

    Jakub Zelek

    Apr 2022

    https://ruj.uj.edu.pl/xmlui/handle/item/295689

2021

1. Automatic Recognition of Similar Chess Motifs

   Miha Bizjak and Matej Guid

   In Advances in Computer Games: 17th International Conference, ACG 2021, Virtual Event, November 23–25, 2021, Revised Selected Papers, Apr 2021

   Abs DOI

   We present a novel method to find chess positions similar to a given query position from a collection of chess games. We consider not only the static similarity resulting from the arrangement of chess pieces, but also the dynamic similarity involving the recognition of chess motifs and the tactical, dynamic aspects of position similarity. By encoding chess tactical problems as text documents, we use information retrieval techniques to enable efficient approximate searches. We have also developed a method for automatically generating tactical puzzles from a collection of chess games. We have experimentally shown the importance of including both static and dynamic features for successful recognition of similar chess motifs. The experiments have clearly shown that dynamic similarity plays a very important role in the evaluation of the similarity of chess motifs by both the program and chess experts.

2. Improving the Chess Elo System With Process Mining

   Niels Bos

   Jul 2021

   Abs

   Over the last decade, the amount of data generated by software applications e.g. information systems, websites, mobile applications etc. has increased tremendously. Process mining, a subdiscipline of data science, uses this data to analyse and improve processes. In this research, the possibilities of process mining on chess event logs are explored, to ultimately improve the chess Elo system. The chess Elo system is a widely used and well accepted rating system. The Elo system is, however, flawed in multiple ways. Two major flaws of the Elo system, are its incapability to review a player?s strength and the excessive time needed to gain the appropriate Elo rating. This research explores the potential of process mining to identify chess expertise. To be more specific, multiple process mining techniques are applied on chess event logs, and the generated process models are analysed to identify chess expertise. This research presents a method to analyse the differences between high and low rated players. This is achieved by comparing process models generated from high and low rated chess games. The results show that by comparing the process models differences between high and low rated players can be observed. Process mining is therefore a promising approach to improve the Elo system and might be applicable to other software too. However, only the first twelve moves of a game were used. To gain more insight into the differences between high and low rated players, the mid and end games should be included in the event logs as well. Future research should be conducted with more chess games added to the event logs to increase the validity.

3. Predicting Chess Opening Through Modelling Of Chess Opponents

   Debarpan Bose Chowdhury and Banashree Sen

   Webology (ISSN: 1735-188X), Jul 2021
4. A lightweight approach for predicting errors in chess matches

   Giovanni Comarela and Davi Silva

   In Anais do XVIII Encontro Nacional de Inteligência Artificial e Computacional, Evento Online, Jul 2021

   DOI
5. Improving AlphaZero Using Monte-Carlo Graph Search

   Johannes Czech, Patrick Korus, and Kristian Kersting

   In Proceedings of the Thirty-First International Conference on Automated Planning and Scheduling, ICAPS 2021, Guangzhou, China (virtual), August 2-13, 2021, Jul 2021
6. Who’s on First?: Probing the Learning and Representation Capabilities of Language Models on Deterministic Closed Domains

   David Demeter and Doug Downey

   In Proceedings of the 25th Conference on Computational Natural Language Learning, Nov 2021

   Abs DOI

   The capabilities of today’s natural language processing systems are typically evaluated using large datasets of curated questions and answers. While these are critical benchmarks of progress, they also suffer from weakness due to artificial distributions and incomplete knowledge. Artifacts arising from artificial distributions can overstate language model performance, while incomplete knowledge limits fine-grained analysis. In this work, we introduce a complementary benchmarking approach based on SimPlified Language Activity Traces (SPLAT). SPLATs are corpora of language encodings of activity in some closed domain (we study traces from chess and baseball games in this work). SPLAT datasets use naturally-arising distributions, allow the generation of question-answer pairs at scale, and afford complete knowledge in their closed domains. We show that language models of three different architectures can answer questions about world states using only verb-like encodings of activity. Our approach is extensible to new language models and additional question-answering tasks.

7. Risk-taking in adversarial games: What can 1 billion online chess games tell us?

   Cameron Holdaway and Ed Vul

   In Proceedings of the 43rd Annual Meeting of the Cognitive Science Society, CogSci 2021, virtual, July 26-29, 2021, Nov 2021
8. Classification of Chess Games: An Exploration of Classifiers for Anomaly Detection in Chess

   Masudul Hoque

   Nov 2021

   https://cornerstone.lib.mnsu.edu/etds/1119/
9. Detecting Individual Decision-Making Style: Exploring Behavioral Stylometry in Chess

   Reid McIlroy-Young, Yu Wang, Siddhartha Sen, and 2 more authors

   In Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, Nov 2021
10. The Complementary Nature of Perceived and Actual Time Spent Online in Measuring Digital Well-being

    Lillio Mok and Ashton Anderson

    Proc. ACM Hum.-Comput. Interact., Apr 2021

    Abs DOI

    As online platforms become ubiquitous, there is growing concern that their use can potentially lead to negative outcomes in users’ personal lives, such as disrupted sleep and impacted social relationships. A central question in the literature studying these problematic effects is whether they are associated with the amount of time users spend on online platforms. This is often addressed by either analyzing self-reported measures of time spent online, which are generally inaccurate, or using objective metrics derived from server logs or tracking software. Nonetheless, how the two types of time measures comparatively relate to problematic effects – whether they complement or are redundant with each other in predicting problematicity – remains unknown. Additionally, transparent research into this question is hindered by the literature’s focus on closed platforms with inaccessible data, as well as selective analytical decisions that may lead to reproducibility issues.In this work, we investigate how both self-reported and data-derived metrics of time spent relate to potentially problematic effects arising from the use of an open, non-profit online chess platform. These effects include disruptions to sleep, relationships, school and work performance, and self-control. To this end, we distributed a gamified survey to players and linked their responses with publicly-available game logs. We find problematic effects to be associated with both self-reported and data-derived usage measures to similar degrees. However, analytical models incorporating both self-reported and actual time explain problematic effects significantly more effectively than models with either type of measure alone. Furthermore, these results persist across thousands of possible analytical decisions when using a robust and transparent statistical framework. This suggests that the two methods of measuring time spent measure contain distinct, complementary information about problematic usage outcomes and should be used in conjunction with each other.

11. Cheat Detection on Online Chess Games using Convolutional and Dense Neural Network

    Reyhan Patria, Sean Favian, Anggoro Caturdewa, and 1 more author

    In 2021 4th International Seminar on Research of Information Technology and Intelligent Systems (ISRITI), Apr 2021

    DOI
12. Revealing Game Dynamics via Word Embeddings of Gameplay Data

    Younès Rabii and Michael Cook

    In Proceedings of the Seventeenth AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, AIIDE 2021, virtual, October 11-15, 2021, Apr 2021
13. Can You Learn an Algorithm? Generalizing from Easy to Hard Problems with Recurrent Networks

    Avi Schwarzschild, Eitan Borgnia, Arjun Gupta, and 4 more authors

    In Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, Apr 2021
14. Datasets for Studying Generalization from Easy to Hard Examples

    Avi Schwarzschild, Eitan Borgnia, Arjun Gupta, and 5 more authors

    CoRR, Apr 2021
15. Watching a Language Model Learning Chess

    Andreas Stöckl

    In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), Sep 2021

    Abs

    We analyse how a transformer-based language model learns the rules of chess from text data of recorded games. We show how it is possible to investigate how the model capacity and the available number of training data influence the learning success of a language model with the help of chess-specific metrics. With these metrics, we show that more games used for training in the studied range offers significantly better results for the same training time. However, model size does not show such a clear influence. It is also interesting to observe that the usual evaluation metrics for language models, predictive accuracy and perplexity, give no indication of this here. Further examination of trained models reveals how they store information about board state in the activations of neuron groups, and how the overall sequence of previous moves influences the newly-generated moves.

2020

1. Learning to Play the Chess Variant Crazyhouse Above World Champion Level With Deep Neural Networks and Human Data

   Johannes Czech, Moritz Willig, Alena Beyer, and 2 more authors

   Frontiers Artif. Intell., Sep 2020

   DOI
2. Leveraging rationales to improve human task performance

   Devleena Das and Sonia Chernova

   In Proceedings of the 25th International Conference on Intelligent User Interfaces, Cagliari, Italy, Sep 2020

   Abs DOI

   Machine learning (ML) systems across many application areas are increasingly demonstrating performance that is beyond that of humans. In response to the proliferation of such models, the field of Explainable AI (XAI) has sought to develop techniques that enhance the transparency and interpretability of machine learning methods. In this work, we consider a question not previously explored within the XAI and ML communities: Given a computational system whose performance exceeds that of its human user, can explainable AI capabilities be leveraged to improve the performance of the human? We study this question in the context of the game of Chess, for which computational game engines that surpass the performance of the average player are widely available. We introduce the Rationale-Generating Algorithm, an automated technique for generating rationales for utility-based computational methods, which we evaluate with a multi-day user study against two baselines. The results show that our approach produces rationales that lead to statistically significant improvement in human task performance, demonstrating that rationales automatically generated from an AI’s internal task model can be used not only to explain what the system is doing, but also to instruct the user and ultimately improve their task performance.

3. Aligning Superhuman AI with Human Behavior: Chess as a Model System

   Reid McIlroy-Young, Siddhartha Sen, Jon M. Kleinberg, and 1 more author

   In KDD ’20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, CA, USA, August 23-27, 2020, Sep 2020

   DOI
4. Learning Personalized Models of Human Behavior in Chess

   Reid McIlroy-Young, Russell Wang, Siddhartha Sen, and 2 more authors

   CoRR, Sep 2020
5. Explain Your Move: Understanding Agent Actions Using Specific and Relevant Feature Attribution

   Nikaash Puri, Sukriti Verma, Piyush Gupta, and 4 more authors

   In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020, Sep 2020
6. A Comparative Study of Chess Online Educational Products

   Qian Dong and Rong Miao

   In Blended Learning. Education in a Smart Learning Environment: 13th International Conference, ICBL 2020, Bangkok, Thailand, August 24–27, 2020, Proceedings, Bangkok, Thailand, Sep 2020

   Abs DOI

   With the development of technology, more and more online educational products emerge in chess, which makes it difficult for different users to choose from. It’s important to develop methodologies to assist different levels chess players to learn in varies environment. List method and rubric evaluation has been conducted, and advice has been put forward based on this approach. The results show that chess online educational products were rich in content and full featured, which could be divided into four categories: overall ecology, video tutorial, tactical training, live broadcast product. However, products still need to improve in product positioning and user experience to promote the development of chess online education.

2019

1. Chess Position Identification using Pieces Classification Based on Synthetic Images Generation and Deep Neural Network Fine-Tuning

   Afonso Sá Delgado Neto and Rafael Mendes Campello

   In 2019 21st Symposium on Virtual and Augmented Reality (SVR), Sep 2019

   DOI
2. Deep learning investigation for chess player attention prediction using eye-tracking and game data

   Justin Le Louedec, Thomas Guntz, James L. Crowley, and 1 more author

   In Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications, ETRA 2019, Denver , CO, USA, June 25-28, 2019, Sep 2019

   DOI
3. Playing Chess at a Human Desired Level and Style

   Hanan Rosemarin and Ariel Rosenfeld

   In Proceedings of the 7th International Conference on Human-Agent Interaction, HAI 2019, Kyoto, Japan, October 06-10, 2019, Sep 2019

   Abs DOI

   Human chess players prefer training with human opponents over chess agents as the latter are distinctively different in level and style than humans. Chess agents designed for human-agent play are capable of adjusting their level, however their style is not aligned with that of human players. In this paper, we propose a novel approach for designing such agents by integrating the theory of chess players’ decision-making with a state-of-the-art Monte Carlo Tree Search (MCTS) algorithm. We demonstrate the benefits of our approach using two sets of analyses. Quantitatively, we establish that the agents attain their desired Elo ratings. Qualitatively, through a Turing-inspired test with a human chess expert, we show that our agents are indistinguishable from human players.

4. Training a Convolutional Neural Network to Evaluate Chess Positions

   Joel Vikström

   Sep 2019

   Abs

   Convolutional neural networks are typically applied to image analysis problems. We investigate whether a simple convolutional neural network can be trained to evaluate chess positions by means of predicting Stockfish (an existing chess engine) evaluations. Publicly available data from lichess.org was used, and we obtained a final MSE of 863.48 and MAE of 12.18 on our test dataset (with labels ranging from -255 to +255). To accomplish better results, we conclude that a more capable model architecture must be used.

2018

1. The role of emotion in problem solving: first results from observing chess

   Thomas Guntz, James L. Crowley, Dominique Vaufreydaz, and 2 more authors

   In Proceedings of the Workshop on Modeling Cognitive Processes from Multimodal Data, MCPMD\@ICMI 2018, Boulder, CO, USA, October 16, 2018, Sep 2018
2. Learning to Evaluate Chess Positions with Deep Neural Networks and Limited Lookahead

   Matthia Sabatelli, Francesco Bidoia, Valeriu Codreanu, and 1 more author

   In 7th International Conference on Pattern Recognition Applications and Methods, Jan 2018

   7th International Conference on Pattern Recognition Applications and Methods ; Conference date: 16-01-2018 Through 18-01-2018

   Abs DOI

   In this paper we propose a novel supervised learning approach for training Artificial Neural Networks (ANNs) to evaluate chess positions. The method that we present aims to train different ANN architectures to understand chess positions similarly to how highly rated human players do. We investigate the capabilities that ANNs have when it comes to pattern recognition, an ability that distinguishes chess grandmasters from more amateurplayers. We collect around 3,000,000 different chess positions played by highly skilled chess players and label them with the evaluation function of Stockfish, one of the strongest existing chess engines. We create 4 different datasets from scratch that are used for different classification and regression experiments. The results show how relatively simple Multilayer Perceptrons (MLPs) outperform Convolutional Neural Networks (CNNs) in all the experiments that we have performed. We also investigate two different board representations, the first one representing if a piece is present on the board or not, and the second one in which we assign a numerical value to the piece according to its strength. Our results show how the latter input representation influences the performances of the ANNs negatively in almost all experiments.

3. Analysis of Chess Playing Skills on Mathematics Learning Outcomes Junior Athletes Raja Kombi Trenggalek Chess Club

   Andika Yogi Setiawan and Henri Gunawan Pratama

   PHEDHERAL, Jan 2018

   Abs DOI

   This study aims to determine the results of the analysis of chess playing skills on mathematics learning outcomes for junior athletes of the Raja Kombi Trenggalek chess club. The research method used is a qualitative descriptive method with a quantitative approach. Participants in this study were 8 junior athletes of Raja Kombi Trenggalek chess club. Data collection techniques using interviews, skills results and documentation. The data analysis in this study used the mean and percentage formula. After analyzing the data, the results of this study concluded that in this study, the average score of playing chess skills was 85.00, then the average score of mathematics learning outcomes was 86.25. This is of course the higher the level of achievement of skills or intellectual intelligence, the higher the level of problem solving such as in learning mathematics. This is also influenced by motor and psychological aspects as a support for intelligence skills that affect the thinking of athletes. Then from the data analysis it can be said that the higher the level of achievement of chess playing skills, the higher the level of problem solving as in learning mathematics