Research

2026

1. Verification of the Implicit World Model in a Generative Model via Adversarial Sequences

   András Balogh and Márk Jelasity ,

   PDF

   Generative sequence models are typically trained on sample sequences from natural or formal languages. It is a crucial question whether – or to what extent – sample-based training is able to captur... Read more

   Generative sequence models are typically trained on sample sequences from natural or formal languages. It is a crucial question whether – or to what extent – sample-based training is able to capture the true structure of these languages, often referred to as the "world model". Theoretical results indicate that we can hope for soundness at best, that is, generating valid sequences, but not necessarily all of them. However, it is still important to have practical tools that are able to verify whether a given sequence model is sound. In this study, we focus on chess, as it is a domain that provides enough complexity while having a simple rule-based world model. We propose adversarial sequence generation for verifying the soundness of the sequence model. Our adversaries generate valid sequences so as to force the sequence model to generate an invalid next move prediction. Apart from the falsification of soundness, this method is also suitable for a more fine-grained analysis of the failure modes and the effects of different choices during training. To demonstrate this, we propose a number of methods for adversarial sequence generation and evaluate the approach on a large set of chess models. We train models on random as well as high-quality chess games, using several training recipes. We find that none of the models are sound, but some training techniques and dataset choices are able to improve soundness remarkably. We also investigate the potential application of board state probes in both our training and attack methods. Our findings indicate that the extracted board states have no causal role in next token prediction in most of the models.

   Read less
2. Exploring the Limits of Probes for Latent Representation Edits in GPT Models

   Austin L. Davis, Robinson Vasquez Ferrer, and Gita Sukthankar In The Fourteenth International Conference on Learning Representations, 2026

   Probing classifiers are a technique for understanding and modifying the operation of neural networks in which a smaller classifier is trained to use the model’s internal representation to learn a p... Read more

   Probing classifiers are a technique for understanding and modifying the operation of neural networks in which a smaller classifier is trained to use the model’s internal representation to learn a probing task. Similar to a neural electrode array, probing classifiers help both discern and edit the internal representation of a neural network. This article evaluates the use of probing classifiers to modify the internal hidden state of a chess-playing transformer. We contrast the performance of standard linear probes against Sparse Autoencoders (SAEs), a latent space interpretability technique designed to decompose polysemantic concepts into atomic features via an overcomplete basis. Our experiments demonstrate that linear probes trained directly on the residual stream significantly outperform probes based on SAE latents. When quantifying the success of interventions via the probability of legal moves, linear probe edits achieved an 88% success rate, whereas SAE-based edits yielded only 41%. These findings suggest that while SAEs are valuable for specific interpretability tasks, they do not enhance the controllability of hidden states compared to raw vectors. Finally, we show that the residual stream respects the Markovian property of chess, validating the feasibility of applying consistent edits across different time steps for the same board state.

   Read less
3. From Entropy to Epiplexity: Rethinking Information for Computationally Bounded Intelligence

   Marc Finzi, Shikai Qiu, Yiding Jiang, Pavel Izmailov, J. Zico Kolter, and Andrew Gordon Wilson Preprints, 2026

   Can we learn more from data than existed in the generating process itself? Can new and useful information be constructed from merely applying deterministic transformations to existing data? Can the... Read more

   Can we learn more from data than existed in the generating process itself? Can new and useful information be constructed from merely applying deterministic transformations to existing data? Can the learnable content in data be evaluated without considering a downstream task? On these questions, Shannon information and Kolmogorov complexity come up nearly empty-handed, in part because they assume observers with unlimited computational capacity and fail to target the useful information content. In this work, we identify and exemplify three seeming paradoxes in information theory: (1) information cannot be increased by deterministic transformations; (2) information is independent of the order of data; (3) likelihood modeling is merely distribution matching. To shed light on the tension between these results and modern practice, and to quantify the value of data, we introduce epiplexity, a formalization of information capturing what computationally bounded observers can learn from data. Epiplexity captures the structural content in data while excluding time-bounded entropy, the random unpredictable content exemplified by pseudorandom number generators and chaotic dynamical systems. With these concepts, we demonstrate how information can be created with computation, how it depends on the ordering of the data, and how likelihood modeling can produce more complex programs than present in the data generating process itself. We also present practical procedures to estimate epiplexity which we show capture differences across data sources, track with downstream performance, and highlight dataset interventions that improve out-of-distribution generalization. In contrast to principles of model selection, epiplexity provides a theoretical foundation for data selection, guiding how to select, generate, or transform data for learning systems.

   Read less
4. Mixture of Masters: Sparse Chess Language Models with Player Routing

   Giacomo Frisoni, Lorenzo Molfetta, Davide Freddi, and Gianluca Moro , 2026

   Modern chess language models are dense transformers trained on millions of games played by thousands of high-rated individuals. However, these monolithic networks tend to collapse into mode-average... Read more

   Modern chess language models are dense transformers trained on millions of games played by thousands of high-rated individuals. However, these monolithic networks tend to collapse into mode-averaged behavior, where stylistic boundaries are blurred, and rare but effective strategies are suppressed. To counteract homogenization, we introduce Mixture-of-Masters (MoM), the first chess mixture-of-experts model with small-sized GPT experts emulating world-class grandmasters. Each expert is trained with a combination of self-supervised learning and reinforcement learning guided by chess-specific rewards. For each move, a post-hoc learnable gating network selects the most appropriate persona to channel depending on the game state, allowing MoM to switch its style dynamically–e.g., Tal’s offensive vocation or Petrosian’s defensive solidity. When evaluated against Stockfish on unseen standard games, MoM outperforms both dense individual expert networks and popular GPT baselines trained on aggregated data, while ensuring generation variety, control, and interpretability.

   Read less
5. Verifying Floating-Point Programs in Stainless

   Andrea Gilot, Axel Bergström, and Eva Darulova , 2026

   We extend the Stainless deductive verifier with floating-point support, providing the first automated verification support for floating-point numbers for a subset of Scala that includes polymorphis... Read more

   We extend the Stainless deductive verifier with floating-point support, providing the first automated verification support for floating-point numbers for a subset of Scala that includes polymorphism, recursion and higher-order functions. We follow the recent approach in the KeY verifier to axiomatise reasoning about mathematical functions, but go further by supporting all functions from Scala’s math API, and by verifying the correctness of the axioms against the actual implementation in Stainless itself. We validate Stainless’ floating-point support on a new set of benchmarks sampled from real-world code from GitHub, showing that it can verify specifications about, e.g., ranges of output or absence of special values for most supported functions, or produce counter-examples when the specifications do not hold.

   Read less
6. ChessFormer - Modeling Human Decision Making in Chess

   Jakub Zeman and Miroslav Čepek , 2026

   ChessFormer introduces a novel searchless chess engine leveraging transformer architecture to approximate human decision-making in chess. Trained on a vast dataset of 3 billion chess positions, our... Read more

   ChessFormer introduces a novel searchless chess engine leveraging transformer architecture to approximate human decision-making in chess. Trained on a vast dataset of 3 billion chess positions, our model learns its entire decision-making process directly from training data. Evaluations show an improvement in human move-matching accuracy over prior models in high-Elo ranges and the model’s ability to distinguish between human and algorithmic decision-making, offering potential applications in chess analysis or cheat detection.

   Read less
7. GenSTEG: A Light and Scalable Trinary-Based Encryption with Multimodal Generative Steganography

   K M Kaushal Karthik and R Ramesh In Modeling Decisions for Artificial Intelligence, 2026

   The increasing demand for robust and scalable encryption algorithms is driven by rapid advancements in computing technology, and emerging technologies like quantum cryptanalysis pose significant th... Read more

   The increasing demand for robust and scalable encryption algorithms is driven by rapid advancements in computing technology, and emerging technologies like quantum cryptanalysis pose significant threats to information security. This paper presents a novel trinary-based multistage encryption algorithm designed for scalability and enhanced security through the use of the less-explored trinary number system. By leveraging a dynamically adapting, pseudo-random, multi-dimensional key space and high diffusion transform, the algorithm introduces significant resistance to common attacks. Additionally, two innovative generative steganography schemes are proposed that algorithmically generate unique musical compositions or chess positions directly from the ciphertext. The overall algorithm operates at linear complexity, which ensures optimal performance. To contextualize practicality, the system is benchmarked against established lightweight cryptographic baselines, and a dataset of 1000 plaintext–ciphertext pairs has been publicly released to support independent analysis. Ultimately, this work presents a complete and flexible framework that uniquely combines a novel cipher with generative steganography, offering a robust and efficient system.

   Read less
8. Directions for Computational Theory of Mind: Data, Metrics, Models and Mathematical Formalization

   Prabhat Kumar, Erin Zaroukian, Douglas Summers-Stay, and Adrienne Raglin IEEE Access, 2026

   This study expands on previous surveys of computational theory of mind (ToM) focusing on four key areas. Data: We attempt to characterize data needed for this research and propose creating procedur... Read more

   This study expands on previous surveys of computational theory of mind (ToM) focusing on four key areas. Data: We attempt to characterize data needed for this research and propose creating procedurally generated, multi-modal synthetic data for training and testing ToM systems, addressing the lack of open-source data of agent behaviors in closed environments. Metrics: We explore ToM evaluation beyond the Sally-Anne Test, considering child development stages and natural language understanding as potential measures. Model: We investigate building on recent ToM models, exploring open-ended learning in reinforcement learning, and applying neuroscientific insights to model architecture. We also examine ToM applications in everyday technologies, leveraging state-of-the-art transformer technologies and multimodal datasets. Theoretical Formalization: We aim to bridge cognitive science and psychology concepts with mathematical approaches to facilitate algorithm development in ToM.

   Read less
9. Análisis observacional de los movimientos ilegales y erróneos en la iniciación al ajedrez

   Jorge Miranda Pérez In HCI International 2025 – Late Breaking Papers, 2026

   Within the observational methodology, and based on a detailed analysis of FIDE laws of Chess, an observation system has been developed ad hoc for analyzing the illegal moves that children commit in... Read more

   Within the observational methodology, and based on a detailed analysis of FIDE laws of Chess, an observation system has been developed ad hoc for analyzing the illegal moves that children commit in chess. The reliability of the resulting data was confirmed by analysis of interobserver agreement, using Cohen’s kappa statistic. The results of the generalizability study confirmed the generalizability of results and the validity of the observation instrument. A Lag Sequential Analysis was performed to identify significant associations between categorical variables in each of the seven types of illegal movements characterized (castling, pinned, king to threatened square, incorrect movement of piece, promotion, occupation, do not remove check). The results obtained in the analysis of illegal movements, reveal the difficulties that the child (under 12 years of age) finds in the understanding and practice of chess.

   Read less
10. Blunder prediction in chess

    Yarden Rokach and Bracha Shapira , Logroño, Spain, 2026

    The ability to predict blunders in chess plays a crucial role in improving players’ performance and enabling strategic decision-making. We introduce a novel, scalable, and personalized blunder pred... Read more

    The ability to predict blunders in chess plays a crucial role in improving players’ performance and enabling strategic decision-making. We introduce a novel, scalable, and personalized blunder prediction model for chess. Unlike prior work requiring a separate model per player, our unified architecture learns a collaborative user embedding space, allowing it to generalize weaknesses across players and new users. Our hybrid model, inspired by Deep Factorization Machines (DeepFM), fuses a frozen pre-trained CNN (for board embeddings) with dynamically learned user embeddings to model player-board interactions while still utilizing metadata about the state of the board and the user. We demonstrate that this latent ’blunder profile’ is a significantly more powerful predictor of error than a player’s explicit Elo rating. The system achieves state-of-the-art performance (0.801 AUC) on both immediate and non-immediate blunders, offering an efficient and data-sparse-friendly solution for personalized chess analysis. Ultimately, this approach demonstrates the practical viability of deep personalization in complex strategy games, facilitating highly efficient, user-centric learning environments.

    Read less
11. Machine Learning Approaches for Classifying Chess Game Outcomes: A Comparative Analysis of Player Ratings and Game Dynamics

    Kamil Samara, Aaron Antreassian, Matthew Klug, and Mohammad Sakib Hasan Applied Intelligence, 2026

    Online chess platforms generate vast amounts of game data, presenting opportunities to analyze match outcomes using machine learning approaches. This study develops and compares four machine learni... Read more

    Online chess platforms generate vast amounts of game data, presenting opportunities to analyze match outcomes using machine learning approaches. This study develops and compares four machine learning models to classify chess game results (White win, Black win, or Draw) by integrating player rating information with game dynamic metadata. We analyzed 11,510 complete games from the Lichess platform after preprocessing a dataset of 20,058 initial records. Seven key features were engineered to capture both pre-game skill parameters (player ratings, rating difference) and game complexity metrics (game duration, turn count). Four machine learning algorithms were implemented and optimized through grid search cross-validation: Multinomial Logistic Regression, Random Forest, K-Nearest Neighbors, and Histogram Gradient Boosting. The Gradient Boosting classifier achieved the highest performance with 83.19% accuracy on hold-out data and consistent 5-fold cross-validation scores (83.08% \pm 0.009%). Feature importance analysis revealed that game complexity (number of turns) was the strongest correlate of the outcome across all models, followed by the rating difference between opponents. Draws represented only 5.11% of outcomes, creating class imbalance challenges that affected classification performance for this outcome category. The results demonstrate that ensemble methods, particularly gradient boosting, can effectively capture non-linear interactions between player skill and game length to classify chess outcomes. These findings have practical applications for chess platforms in automated content curation, post-game quality assessment, and engagement enhancement strategies. The study establishes a foundation for robust outcome analysis systems in online chess environments.

    Read less
12. Caïssa AI: A Neuro-Symbolic Chess Agent for Explainable Move Suggestion and Grounded Commentary

    Mazen Soliman and Nourhan Ehab Electronics, 2026

    Despite the impressive generative capabilities of large language models (LLMs), their lack of grounded reasoning and susceptibility to hallucinations limit their reliability in structured domains s... Read more

    Despite the impressive generative capabilities of large language models (LLMs), their lack of grounded reasoning and susceptibility to hallucinations limit their reliability in structured domains such as chess. We present Caïssa AI, a neuro-symbolic chess agent that augments LLM-generated move commentary with symbolic reasoning, knowledge graph integration, and verification modules. Caïssa AI combines a fine-tuned chess-specific LLM with a Prolog-based rule engine encoding chess tactics and rules, along with a dynamically constructed Neo4j knowledge graph representing the current board state. This hybrid architecture enables the system to generate not only accurate move suggestions but also coherent, strategically grounded commentary. A LangGraph-based verification module cross-checks LLM outputs against symbolic logic to ensure consistency and correctness, effectively mitigating hallucinations. By aligning data-driven generation with formal domain knowledge, Caïssa AI enhances both trustworthiness and explainability. Our results demonstrate that this tight neuro-symbolic integration produces verifiable, high-quality commentary and serves as a generalizable blueprint for AI systems requiring real-time, interpretable decision support.

    Read less
13. Quantum King-Ring Domination in Chess: A QAOA Approach

    Gerhard Stenzel, Michael Kölle, Tobias Rohe, Julian Hager, Leo Sünkel, Maximilian Zorn, and Claudia Linnhoff-Popien In KI 2025: Advances in Artificial Intelligence, 2026

    The Quantum Approximate Optimization Algorithm (QAOA) is extensively benchmarked on synthetic random instances such as MaxCut, TSP, and SAT problems, but these lack semantic structure and human int... Read more

    The Quantum Approximate Optimization Algorithm (QAOA) is extensively benchmarked on synthetic random instances such as MaxCut, TSP, and SAT problems, but these lack semantic structure and human interpretability, offering limited insight into performance on real-world problems with meaningful constraints. We introduce Quantum King-Ring Domination (QKRD), a NISQ-scale benchmark derived from chess tactical positions that provides 5,000 structured instances with one-hot constraints, spatial locality, and 10–40 qubit scale. The benchmark pairs human-interpretable coverage metrics with intrinsic validation against classical heuristics, enabling algorithmic conclusions without external oracles. Using QKRD, we systematically evaluate QAOA design choices and find that constraint-preserving mixers (XY, domain-wall) converge approximately 13 steps faster than standard mixers (p<10^-7, d\approx0.5) while eliminating penalty tuning, warm-start strategies reduce convergence by 45 steps (p<10^-127, d=3.35) with energy improvements exceeding d=8, and Conditional Value-at-Risk (CVaR) optimization yields an informative negative result with worse energy (p<10^-40, d=1.21) and no coverage benefit. Intrinsic validation shows QAOA outperforms greedy heuristics by 12.6% and random selection by 80.1%. Our results demonstrate that structured benchmarks reveal advantages of problem-informed QAOA techniques obscured in random instances. We release all code, data, and experimental artifacts for reproducible NISQ algorithm research.

    Read less
14. VAM: Verbalized Action Masking for Controllable Exploration in RL Post-Training – A Chess Case Study

    Zhicheng Zhang, Ziyan Wang, Yali Du, and Fei Fang , 2026

    Exploration remains a key bottleneck for reinforcement learning (RL) post-training of large language models (LLMs), where sparse feedback and large action spaces can lead to premature collapse into... Read more

    Exploration remains a key bottleneck for reinforcement learning (RL) post-training of large language models (LLMs), where sparse feedback and large action spaces can lead to premature collapse into repetitive behaviors. We propose Verbalized Action Masking (VAM), which verbalizes an action mask in the prompt and enforces that the model outputs an action from the masked set. Building on this interface, we introduce iterative action-space pruning: if the target action is not sampled, we remove valid sampled actions from the mask and resample under the reduced candidate set, repeating until the target is sampled or a fixed budget is exhausted. We study VAM in chess and evaluate it under two training regimes: an engine-play regime that generates states via play against an engine opponent and a fixed-dataset regime that trains from a fixed dataset of positions with verifier scores. Across held-out chess puzzles and full-game play measured by average centipawn loss (ACPL), VAM improves learning efficiency and final performance over strong baselines, highlighting verbalized masking as a practical mechanism for controllable exploration in LLM RL post-training.

    Read less

2025

1. A self-trained engine for a chess variant

   Hannes Adolfsson, David Lewis, Anton Rahmn, Sigge Rajamäe, Edvin Rungardt, and Marco Tafani , 2026

   Chess has long been a benchmark for artificial intelligence (AI) research due to its complexity and well-defined rules. Recent advances, such as AlphaZero, introduced self-learning AI through reinf... Read more

   Chess has long been a benchmark for artificial intelligence (AI) research due to its complexity and well-defined rules. Recent advances, such as AlphaZero, introduced self-learning AI through reinforcement learning and self-play, achieving superhuman performance without prior strategic knowledge, relying solely on the rules of the game. AlphaZero defeated the world-champion chess engine Stockfish after only four hours of training, leveraging large-scale computational resources to rapidly learn and refine its strategies. This thesis presents the development of a chess engine for the chess variant Atomic Chess. The engine was developed in C++ and trained through self-play and reinforcement learning, taking inspiration from AlphaZero’s approach. This project explores the extent to which a chess engine with this approach is feasible for the average enthusiast. Cost-effective cloud-based virtual machine instances with powerful hardware were rented to manage training workloads. Given limited computational resources, we opted for a data-centric approach, focusing on refining the training pipeline to maximize the training data that could be produced, rather than hyperparameter tuning and experimenting with neural network architectures. The final engine was trained on approximately 450,000 self-play games in roughly 150 hours. The final engine was deployed on the chess platform Lichess and achieved an ELOrating of 1,729, which corresponded to the top 10th percentile of Atomic Chess players on Lichess. These results demonstrate that it is possible to achieve a competitive Atomic Chess engine within a budget of 3,000 SEK for cloud computation. This shows that strong self-play reinforcement learning agents for niche games can be developed without requiring large-scale computing infrastructure. These results highlight the viability of accessible, low-budget AI research for underexplored game variants.

   Read less
2. Statistical Analysis of Hungarian Board Game Sales

   Balázs Adorján and Éva Bednárik , 2025

   PDF

   The board game market has experienced significant growth worldwide over the past decade, and Hungary is no exception. The aim of this study is to analyse the state of the Hungarian board game marke... Read more

   The board game market has experienced significant growth worldwide over the past decade, and Hungary is no exception. The aim of this study is to analyse the state of the Hungarian board game market from both financial and macroeconomic perspectives, and to compare it with international trends. The analysis is based on international statistical data, as well as revenue data and website traffic statistics of the largest board game publishers and distributors in Hungary. Additionally, we have considered the price fluctuations of specific board games. The data have been analysed and processed using a quantitative approach and rigorous statistical methods. Primary and secondary data analysis methods were employed to conduct a correlation study. The results shed light on the structure of the domestic market, as well as the extent to which Hungarian consumer behaviours follow international trends, and which economic factors influence revenue growth. By comparing with international data, the study also examines the factors that either support or hinder Hungarian market players in the global competition. The research further addresses changes in consumer behaviour, the rise of digital board games, and the post-effects of the COVID-19 pandemic, all of which have had a significant impact on the development directions of the industry. We also investigate the innovation opportunities for domestic developers and the growing importance of sustainability considerations in board game production.

   Read less
3. Y-LIChess: Live and Interactive Over-The-Board Chess Recognition and Play with Yolo

   Gibran Benitez-Garcia and Hiroki Takahashi Economy and Finance, 2025

   Chess is widely played on computers, yet over-theboard (OTB) chess remains the official and preferred format for many players due to its tactile and immersive nature. Bridging digital and physical ... Read more

   Chess is widely played on computers, yet over-theboard (OTB) chess remains the official and preferred format for many players due to its tactile and immersive nature. Bridging digital and physical play requires accurate recognition of OTB positions. Prior research has explored modular pipelines for board localization, square occupancy, and piece classification, as well as single-stage detectors. While these approaches demonstrate strong accuracy in controlled conditions, they often accumulate errors across stages, face latency and robustness issues, and rarely support interactive play. Thus, in this work, we present Y-LIChess, a YOLO-based system for live, interactive OTB play with engines and online platforms. Y-LIChess employs semiautomatic calibration, event-triggered recognition, and legalityaware validation to ensure seamless, low-latency interaction. On our wood180 dataset, built with an active learning process to reduce manual annotation, Y-LIChess achieves 99.36 AP50 with only 0.21 % per-square error, reconstructs 100 % of boards within one mistake, and performs FEN reconstruction in  7 ms GPU, more than an order of magnitude faster than prior pipelines.

   Read less
4. Estimating the Difficulty of Chess Puzzles by Combining Fine-Tuned Maia-2 with Hand-Crafted and Engine Features

   Sebastian Björkqvist In 2025 40th International Conference on Image and Vision Computing New Zealand (IVCNZ), 2025

   A common way for chess players to practice tactical awareness is to solve chess puzzles, consisting of an initial position and a sequence of moves to achieve a winning position. This practice is mo... Read more

   A common way for chess players to practice tactical awareness is to solve chess puzzles, consisting of an initial position and a sequence of moves to achieve a winning position. This practice is more effective when puzzles are matched to the player’s skill level. In this work, we present an approach for estimating the difficulty of a chess puzzle using only the initial position and the sequence of correct moves. Our approach uses a fine-tuned modification of the Maia-2 model combined with a set of hand-crafted features and features extracted from chess engines such as Leela Chess Zero and Stockfish. All of these features are then used as input to a gradient boosted decision tree model that predicts the final rating of the puzzle. We applied our approach to the FedCSIS 2025 Challenge on Predicting Chess Puzzle Difficulty Part 2, where it achieved first place

   Read less
5. The bread emoji Team’s Submission to the 2025 FedCSIS Predicting Chess Puzzle Difficulty Challenge

   Tyler Woodruff, Luke Imbing, and Marco Cognetta In Proceedings of the 20th Conference on Computer Science and Intelligence Systems (FedCSIS), 2025

   We detail the bread emoji team’s submission to the FedCSIS 2025 Predicting Chess Puzzle Difficulty Challenge. Our solution revolved around improving our submission from the previous competition by ... Read more

   We detail the bread emoji team’s submission to the FedCSIS 2025 Predicting Chess Puzzle Difficulty Challenge. Our solution revolved around improving our submission from the previous competition by incorporating a new puzzle metadata feature and optimizing our implementation to allow for larger model ensembles and more stable training. Similar to our submission from last year, our system has two stages: learning a strong predictor for the Lichess dataset and then rescaling the distribution using an empirically-guided post-processing step to fit it to the smaller and noisier competition dataset. Our submission placed second with a  3.9% gap in mean squared error (MSE) from first place in the final evaluation.

   Read less
6. A Multi-Stage Framework for Chess Puzzle Difficulty Prediction

   Ling Cen, Jiahao Cen, Malin Song, and Zhuliang Yu In Proceedings of the 20th Conference on Computer Science and Intelligence Systems (FedCSIS), 2025

   Accurately estimating the difficulty of the chess puzzle is important for adaptive training systems, personalized recommendations, and large-scale content curation. Unlike engine evaluations optimi... Read more

   Accurately estimating the difficulty of the chess puzzle is important for adaptive training systems, personalized recommendations, and large-scale content curation. Unlike engine evaluations optimized for perfect play, this task involves modeling human-perceived solving difficulty, typically expressed by Glicko-2 ratings. We present a multi-stage framework developed for the FedCSIS 2025 Challenge. The method trains four rating-banded neural regression models in different Elo ranges to capture localized difficulty patterns and reduce bias from unbalanced data. Their predictions are combined with statistical attributes, including success probabilities, failure distributions, and solution length, through a feature-based regression stage to improve cross-range generalization. A final calibration step adjusts the output to statistically plausible rating levels, mitigating systematic prediction biases without adding computational complexity. An additional mask selection procedure was explored as part of the competition extension to identify 10% of the puzzles that are most likely to benefit from the refined evaluation. The proposed solution ranked in the final standings. These results demonstrate that a lightweight and interpretable regression pipeline can achieve competitive precision in modeling human-perceived chess puzzle difficulty.

   Read less
7. How Relevant Are Personas in Open-Source Software Development?

   Ahmed Chelly, Salma Hamza, and Javed Ali Khan In Proceedings of the 20th Conference on Computer Science and Intelligence Systems (FedCSIS), 2025

   Open-source software (OSS) projects, characterized by distributed development and volunteer contributions, face challenges in prioritizing user-centered design and usability. This difficulty arises... Read more

   Open-source software (OSS) projects, characterized by distributed development and volunteer contributions, face challenges in prioritizing user-centered design and usability. This difficulty arises because these projects are primarily driven by developers who focus on technical contributions. As a result, usability and user experience (UX) considerations are often neglected, leading to software that may not meet the needs of its broad and diverse users. To address this issue, we explore the potential of using user personas, which are fictional characters representing real user groups, to enhance user-centered design in OSS projects. Personas promote empathy and a deeper understanding of user needs, thereby improving alignment between developers and users. We conducted an experimental study on three OSS projects: Moodle, Lichess, and Audacity. Personas were created for each project and refined based on feedback from industry experts. Developers rated personas highly for credibility (86%), consistency(79%), and friendliness (86%), highlighting their relevance in OSS projects. A follow-up experiment with students confirmed these findings, with consistency (79%) demonstrating personas’ role in improving usability and aligning developers with user needs. While adoption remains limited due to technical priorities (only 14% of developers and 34% of students found personas useful and expressed willingness to adopt them), personas show significant potential to enhance user-centered design in OSS. Further research is needed to understand developers’ reluctance to adopt this technique and explore strategies to integrate personas more effectively into OSS workflows. This study’s novelty lies in its empirical exploration of personas within OSS, providing quantitative evidence of their effectiveness in improving usability and user-centered design.

   Read less
8. How to Scale Second-Order Optimization

   Zixi Chen, Shikai Qiu, Hoang Phan, Qi Lei, and Andrew Gordon Wilson Frontiers in Computer Science, 2025

   Several recently introduced deep learning optimizers inspired by second-order methods have shown promising speedups relative to the current dominant optimizer AdamW, particularly in relatively smal... Read more

   Several recently introduced deep learning optimizers inspired by second-order methods have shown promising speedups relative to the current dominant optimizer AdamW, particularly in relatively small-scale experiments. However, efforts to validate and replicate their successes have reported mixed results, with some finding quickly diminishing advantage over AdamW with scale. In this work, we investigate how to scale second-order optimizers to achieve optimal performance at scale. Through theoretical and empirical analysis, we derive scaling rules for hyperparameters such as learning rate and weight decay as we scale up model width and depth for a wide range of optimizers, including Shampoo, SOAP, and Muon, accounting for the impact of commonly used techniques such as blocking and grafting. For compute-optimal scaling, we find scaling independent weight decay as 1/width is nearly optimal across optimizers, and that second-order optimizers have a substantially larger optimal model size compared to AdamW for a fixed compute budget. Applying these scaling rules, we show Muon achieves close to 1.4x or higher speedup over AdamW in training transformer language models, while incorrect scaling can decrease the speedup from 1.4x to below 1.1x from 190M to 640M parameter models.

   Read less
9. Multi-Modal Deep Learning with Residual and Structure-Guided Refinement for Chess Puzzle Difficulty Prediction

   Junlin Chen, Cenru Liu, and Yujie Gao In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025

   The goal of FedCSIS 2025 Challenge is to build a model to predict the difficulty (measured as Lichess rating) of given chess puzzles. To address this task, we propose a three-stage joint visual–sta... Read more

   The goal of FedCSIS 2025 Challenge is to build a model to predict the difficulty (measured as Lichess rating) of given chess puzzles. To address this task, we propose a three-stage joint visual–statistical framework for predicting Glicko-based difficulty ratings. In the first stage, a convolutional model based on MobileNetV2 integrates FEN-rendered board images with structured features, including engine-predicted success probabilities, move count, and piece counts, to generate baseline predictions. The second stage employs LightGBM to perform residual refinement, explicitly learning the residual errors of the baseline predictions to correct systematic biases, particularly for extreme difficulty levels. Finally, a domain-informed refinement adjusts the outputs toward interpretable difficulty estimates derived from failure probability distributions and rating-bucket inflection points. Our model ranked 9th in the challenge. Experimental results show that residual refinement and domain-informed adjustment significantly reduce mean squared error compared to the baseline visual–statistical model.

   Read less
10. Strength Estimation and Human-Like Strength Adjustment in Games

    Chun Jung Chen, Chung-Chin Shih, and Ti-Rong Wu In Proceedings of the 20th Conference on Computer Science and Intelligence Systems (FedCSIS), 2025

    Strength estimation and adjustment are crucial in designing human-AI interactions, particularly in games where AI surpasses human players. This paper introduces a novel strength system, including a... Read more

    Strength estimation and adjustment are crucial in designing human-AI interactions, particularly in games where AI surpasses human players. This paper introduces a novel strength system, including a strength estimator (SE) and an SE-based Monte Carlo tree search, denoted as SE-MCTS, which predicts strengths from games and offers different playing strengths with human styles. The strength estimator calculates strength scores and predicts ranks from games without direct human interaction. SE-MCTS utilizes the strength scores in a Monte Carlo tree search to adjust playing strength and style. We first conduct experiments in Go, a challenging board game with a wide range of ranks. Our strength estimator significantly achieves over 80% accuracy in predicting ranks by observing 15 games only, whereas the previous method reached 49% accuracy for 100 games. For strength adjustment, SE-MCTS successfully adjusts to designated ranks while achieving a 51.33% accuracy in aligning to human actions, outperforming a previous state-of-the-art, with only 42.56% accuracy. To demonstrate the generality of our strength system, we further apply SE and SE-MCTS to chess and obtain consistent results. These results show a promising approach to strength estimation and adjustment, enhancing human-AI interactions in games. Our code is available at https://rlg.iis.sinica.edu.tw/papers/strength-estimator.

    Read less
11. Design of Electronic Chess Board Using Analogue Hall-effect Sensors for Piece Identification

    Justin Julius Chin Cheong, Praneel Bhatia, Matthias Krauledat, and Ronny Hartanto In The Thirteenth International Conference on Learning Representations, 2025

    This paper presents the design, implementation, and evaluation of an innovative electronic chess board leveraging Hall-effect sensors for chess piece identification. Current electronic chess boards... Read more

    This paper presents the design, implementation, and evaluation of an innovative electronic chess board leveraging Hall-effect sensors for chess piece identification. Current electronic chess boards employ diverse technologies such as RFID, resistive switches, optical sensors, and computer vision, each with varying complexity and cost. The proposed solution utilizes Hall-effect sensors to detect changes in magnetic flux density caused by magnets embedded in chess pieces.The design features a modular 4x4 grid of sensors integrated on printed circuit boards, where each piece is differentiated by a unique magnet-spacer configuration. Experimental investigations validate the relationship between magnetic flux density and distance, modelled theoretically and tested with a field strength meter and the PCB system. Results confirm the accurate detection of piece positions and classes, despite minor interference from adjacent pieces. A Python-based system processes the sensor data, translating chess moves into live updates on the Li-Chess platform.The system has displayed good results in piece position identification accurately. Further improvement including as spacer configurations are to be done to implement piece classification. Future work aims to explore enhanced micro-controller integration, offering a streamlined and accessible alternative to existing technologies.

    Read less
12. Human Chess: A Novel Searchless RL-based Chess Agent Capable of Multi-ELO Human-Like Play

    Prerit Choudhary, Rikhil Vagadia, and Ankush Dhawan In 2025 7th International Congress on Human-Computer Interaction, Optimization and Robotic Applications (ICHORA), 2025
13. The Advantage of Moving First in Amateur Online Chess

    Tyler Cook , 2025

    There is a long-held belief in the chess community that the player with the white pieces has an advantage in making the first move. This phenomenon has been observed repeatedly in over-the-board ga... Read more

    There is a long-held belief in the chess community that the player with the white pieces has an advantage in making the first move. This phenomenon has been observed repeatedly in over-the-board games between high-level players and professionals. However, less is known about the prevalence of white’s advantage in games played between amateurs in more casual settings. This article attempts to identify a first-move advantage in chess by examining a large database of amateur games played online on a dedicated chess website. Win rates are calculated for various rating levels, and the influence of opening move choice is also explored. These results can help determine whether there is an inherent first-move advantage in chess observable for all players in multiple settings, or if this effect is exclusively seen with players of high skill during games played in person.

    Read less
14. Understanding the learned look-ahead behavior of chess neural networks

    Diogo Cruz ICGA Journal, 2025

    We investigate the look-ahead capabilities of chess-playing neural networks, specifically focusing on the Leela Chess Zero policy network. We build on the work of Jenner et al. (2024) by analyzing ... Read more

    We investigate the look-ahead capabilities of chess-playing neural networks, specifically focusing on the Leela Chess Zero policy network. We build on the work of Jenner et al. (2024) by analyzing the model’s ability to consider future moves and alternative sequences beyond the immediate next move. Our findings reveal that the network’s look-ahead behavior is highly context-dependent, varying significantly based on the specific chess position. We demonstrate that the model can process information about board states up to seven moves ahead, utilizing similar internal mechanisms across different future time steps. Additionally, we provide evidence that the network considers multiple possible move sequences rather than focusing on a single line of play. These results offer new insights into the emergence of sophisticated look-ahead capabilities in neural networks trained on strategic tasks, contributing to our understanding of AI reasoning in complex domains. Our work also showcases the effectiveness of interpretability techniques in uncovering cognitive-like processes in artificial intelligence systems.

    Read less
15. Interpretation and Control of AI Model Behavior Through Direct Adjustment of Latent Representations

    Austin Davis CoRR, 2025

    This dissertation investigates the structures and mechanisms underpinning the latent space representations that emerge within Generative Pretrained Transformer (GPT) models. Addressing the bro... Read more

    This dissertation investigates the structures and mechanisms underpinning the latent space representations that emerge within Generative Pretrained Transformer (GPT) models. Addressing the broader goal of enhancing AI trustworthiness through transparency, accountability, and controllability, we focus on techniques to understand, quantify, and manipulate these latent space representations. Through a series of analyses, we examine several chess-playing GPT models as controlled testbeds, leveraging their structured decision space to explore emergent representations and decision-making processes. Key contributions include a mechanistic analysis of the attention heads and latent representations, the development of novel metrics for evaluating intervention outcomes, and the application of linear probe classifiers to decode and edit the model’s internal world representations. Analysis of the probe weight vectors reveals that the chess-playing GPT developed an emergent world model of the game that includes pieces, positions, and movement rules, and provides empirical support for the linear representation hypothesis–the idea that abstract concepts are encoded as specific directions in the model’s hidden state space. Complementary analysis of the hidden state vectors demonstrates that the model’s internal representations honor the Markovian property of chess. Experimental results demonstrate that linear interventions can causally steer GPT outputs while preserving their semantic validity. Drawing on the dose-response analogy from medicine, we vary both the strength and position of interventions, showing that output quality is maximized when intervention strength follows an exponentially decaying schedule across token positions. Similar experiments using sparse autoencoders in place of linear probes yielded significantly poorer performance. These results highlight the effectiveness of simple linear probes as valuable tools for interpretability and control.

    Read less
16. ChessMoveLLM: Large Language Models for Chess Next Move Prediction

    Kassim B. Diallo and Moulay A. Akhloufi , 2025

    Creating a chess engine using language models is challenging due to the complexity of understanding the logic and depth of moves. To solve this problem, we propose several approaches with different... Read more

    Creating a chess engine using language models is challenging due to the complexity of understanding the logic and depth of moves. To solve this problem, we propose several approaches with different types of training of different LLMs that already have some knowledge of tasks and not necessarily on chess, and adjust them for faster learning. We propose three groups of methods and five approaches to enhance state-of-the-art results. Our approaches will involve different types of database manipulation, the training of different types of networks and the application of a new, widely used approach, the RAG (Retrieval-Augmented Generation). We demonstrated the potential of networks and approaches with promising results for the first 2 methods, and we demonstrated with RAG in combination with FAISS (Facebook AI Similarity Search) [1], algebraic notation and GPT3.5-turbo-instruct that it is possible to increase the ability of networks to predict good chess moves. The first approach demonstrated that successive move generation benefits generalization and reduces illegal moves; for the second approach, we proved that the use of noiseless algebraic notation gives the network the possibility of progressing significantly before giving illegal moves. The second group of approaches, demonstrated that using the FEN (Forsyth-Edwards Notation), a textual graphical representation, for the prediction of move sequences to force context understanding could give promising results with a low number of epochs. Even with limitations in computation time and epochs, we were able to find promising results with a part of our dataset that can be improved with more epochs and data.

    Read less
17. Data Serialization and Persistence

    Adarsh Divakaran In SoutheastCon 2025, 2025
18. Beyond Perfect Play: A Combinatorial and Probabilistic Approach to Chess Endgame Strategy

    Divij Dogra , 2025

    Chess is a complex game that requires deep strategic thinking, pattern recognition, calculations, and creative problem-solving. Modeling strategic decision-making in chess endgames poses a unique c... Read more

    Chess is a complex game that requires deep strategic thinking, pattern recognition, calculations, and creative problem-solving. Modeling strategic decision-making in chess endgames poses a unique challenge due to the game’s high complexity and the uncertainty of human play. In this paper, we propose a hybrid analytical framework that combines combinatorial game theory, graph-theoretic game-tree analysis, and probabilistic modeling to explore optimal strategies in simplified endgame positions. Using real examples including games from the 2024 FIDE World Chess Championship match between Gukesh and Ding, we construct and analyze game trees to evaluate strategies under probabilistic distributions of plausible human responses. Our approach extends traditional models by incorporating uncertainty into the decision-making process, allowing for a richer understanding of practical play in high-stakes scenarios. This framework offers a bridge between theoretical rigor and real-world applicability in chess and beyond.

    Read less
19. Quantifying Elicitation of Latent Capabilities in Language Models

    Elizabeth Donoway, Hailey Joren, Arushi Somani, Henry Sleight, Julian Michael, Michael R DeWeese, John Schulman, Ethan Perez, Fabien Roger, and Jan Leike The National High School Journal of Science, 2025
20. Generating Creative Chess Puzzles

    Xidong Feng, Vivek Veeriah, Marcus Chiam, Michael Dennis, Ryan Pachauri, Thomas Tumiel, Federico Barbero, Johan Obando-Ceron, Jiaxin Shi, Satinder Singh, Shaobo Hou, Nenad Tomašev, and Tom Zahavy In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025

    While Generative AI rapidly advances in various domains, generating truly creative, aesthetic, and counter-intuitive outputs remains a challenge. This paper presents an approach to tackle these dif... Read more

    While Generative AI rapidly advances in various domains, generating truly creative, aesthetic, and counter-intuitive outputs remains a challenge. This paper presents an approach to tackle these difficulties in the domain of chess puzzles. We start by benchmarking Generative AI architectures, and then introduce an RL framework with novel rewards based on chess engine search statistics to overcome some of those shortcomings. The rewards are designed to enhance a puzzle’s uniqueness, counter-intuitiveness, diversity, and realism. Our RL approach dramatically increases counter-intuitive puzzle generation by 10x, from 0.22% (supervised) to 2.5%, surpassing existing dataset rates (2.1%) and the best Lichess-trained model (0.4%). Our puzzles meet novelty and diversity benchmarks, retain aesthetic themes, and are rated by human experts as more creative, enjoyable, and counter-intuitive than composed book puzzles, even approaching classic compositions. Our final outcome is a curated booklet of these AI-generated puzzles, which is acknowledged for creativity by three world-renowned experts.

    Read less
21. Machine Learning Based Dynamic Difficulty Adaptation for Chess

    Mohamed Gamal, Amal Aboulhassan, and Yomna M.I. Hassan , 2025

    Chess enhances problem-solving and decisionmaking skills. Traditional chess often features static difficulty, which can frustrate novices or bore masters. Dynamic Difficulty Adaptation (DDA) addres... Read more

    Chess enhances problem-solving and decisionmaking skills. Traditional chess often features static difficulty, which can frustrate novices or bore masters. Dynamic Difficulty Adaptation (DDA) addresses this by adjusting the game’s challenge based on player performance, enabling personalized learning and engagement. DDA relies on game analytics by collecting and interpreting performance data such as move timing, accuracy, and positional evaluations compared to top engines. In chess, the structured gameplay and availability of engines like Stockfish offer an ideal environment for DDA. By integrating game analytics, DDA can apply adjustments responsive to the player’s current ability rather than relying on fixed difficulty presets. This approach tailors the gaming experience to individual skill levels, preventing novice frustration and keeping expert players challenged. Thus, game analytics enhances DDA in chess, creating an adaptive environment that caters to a broad spectrum of players.

    Read less
22. Investigating Experiential Effects in Online Chess using a Hierarchical Bayesian Analysis

    Adam Gee, Sydney O. Seese, James P. Curley, and Owen G. Ward In 2025 International Mobile, Intelligent, and Ubiquitous Computing Conference (MIUCC), 2025
23. The Effect of Transcranial Direct Current Stimulation on Risky Decision-Making of Student Chess Players Based on their Introverted and Extroverted Personality Traits

    Shahrouz Ghayebzadeh, Mehrdad Moharramzadeh, and Maryam Zoghi , 2025

    The aim of this research was to investigate the effects of transcranial direct current stimulation (tDCS) on risky decision-making in student chess players, taking into account their personality tr... Read more

    The aim of this research was to investigate the effects of transcranial direct current stimulation (tDCS) on risky decision-making in student chess players, taking into account their personality traits. In this study, 28 high school students who were active in chess and participated in provincial and national chess leagues were selected. Based on the NEO Five-Factor Inventory of personality traits, they were divided into two groups: 14 extroverted students (17 \pm 0.88) and 14 introverted students (16.5 \pm 1.02). Each participant attended three separate sessions in the laboratory, with a minimum 72-hour rest period between sessions. In each session, participants performed the Iowa Gambling Task and the Lichess computer game before any stimulation. They were then subjected to one of three conditions: right anodal/left cathodal, right cathodal/left anodal, or sham stimulation, for 20 minutes at 2 mA intensity over the dorsolateral prefrontal cortex. After the stimulation, participants repeated the Iowa Gambling Task and the Lichess computer game. Data analysis using two-way mixed ANOVA revealed a significant difference in the Iowa Gambling Task between right anodal/left cathodal and right cathodal/left anodal stimulation based on personality traits (p = 0.001). The findings of this study indicated that transcranial direct current stimulation had a differential effect on decision-making in chess players based on their personality traits. Specifically, the study’s results showed that extroverted players exhibited more risk-taking behaviour, while introverted players acted more cautiously.

    Read less
24. Investigating the capabilities of language models in puzzle reasoning: A survey and experimental analysis

    Panagiotis Giadikiaroglou Sport Psychology Studies, 2025

    Puzzle-solving has long served as a benchmark for evaluating artificial intelligence, testing a model’s ability to reason, infer, and strategize across complex problem spaces. Traditional AI and ma... Read more

    Puzzle-solving has long served as a benchmark for evaluating artificial intelligence, testing a model’s ability to reason, infer, and strategize across complex problem spaces. Traditional AI and machine learning methods, such as symbolic reasoning and reinforcement learning, have made notable strides in structured domains like board games and logic puzzles. However, as neural networks and, more recently, large language models (LLMs) have evolved, new possibilities have emerged for tackling a broader range of puzzle types, including those requiring nuanced commonsense reasoning, abstract pattern recognition, and complex multi-step calculations. LLMs, with their vast data-driven language capabilities, hold unique potential to bridge structured logical tasks and less formal, knowledge-based puzzles. Despite these advances, the current landscape of puzzle-solving with LLMs reveals both achievements and limitations, particularly when models are tasked with problems that demand interpretative reasoning and precise calculation. This thesis explores the evolving role of LLMs in solving such complex reasoning tasks, specifically focusing on their puzzle-solving capabilities. Divided into two main sections, the thesis first provides a comprehensive survey of recent advancements in LLM methodologies, covering diverse prompting techniques, neuro-symbolic approaches, and fine-tuning strategies for puzzles. Using a newly proposed taxonomy, puzzles are categorized into rule-based and rule-less types, with each category examined for its unique cognitive demands on LLMs. The second section presents experimental evaluations conducted on four datasets–two math-based datasets (GSM8K, SVAMP) and two puzzle-focused datasets (Game of 24 and RiddleSense). Various reasoning techniques, including Input-Output (IO) prompting, Chain-of-Thought (CoT), Least-to-Most (LtM), and Faithful-CoT methods, are employed to assess LLM performance. Models of varying scales, particularly smaller LLMs like Llama-3.1 family and Mistral, are tested across settings such as zero-shot, few-shot, and self-consistency to evaluate their efficacy in solving complex and multi-step reasoning tasks. The thesis provides critical insights into the performance limitations of current LLMs in puzzle-solving, particularly noting that advanced reasoning methods like Faithful-CoT and puzzle translation techniques yield inconsistent improvements with smaller models. Finally, it outlines future research directions, advocating for expanded dataset creation, neuro-symbolic integration, and advancements in puzzle generation. This thesis aims to deepen our understanding of LLMs’ reasoning abilities and highlight pathways to enhance their performance in complex cognitive tasks.

    Read less
25. AlphaDeepChess: chess engine based on alpha-beta pruning

    Juan Girón Herranz and Yi Wang Qiu , 2025

    Chess engines have played a fundamental role in the advancement of articial intelligence applied to the game since the mid-20th century. Today, Stocksh, the most powerful and open source chess engi... Read more

    Chess engines have played a fundamental role in the advancement of articial intelligence applied to the game since the mid-20th century. Today, Stocksh, the most powerful and open source chess engine, still relies on alpha-beta pruning, but also incorporates machine learning techniques. The goal of this project is to develop a chess engine capable of competing againstboth other engines and human players, using minimax with alpha-beta pruning as its core. Additionally, we analyze the impact of other classical algorithmic techniquessuch as transposition tables, iterative deepening, and a move generator based on magic bitboards. The chess engine has been uploaded to the Lichess platform, where AlphaDeepChess achieved an Elo rating of 1900 while running on a Raspberry Pi 5 equipped with a 2GB transposition table.

    Read less
26. Checkmating One, by Using Many: Combining Mixture of Experts with MCTS to Improve in Chess

    Felix Helfenstein, Johannes Czech, Jannis Blüml, Max Eisel, and Kristian Kersting , 2025

    In games like chess, strategy evolves dramatically across distinct phases - the opening, middlegame, and endgame each demand different forms of reasoning and decision-making. Yet, many modern chess... Read more

    In games like chess, strategy evolves dramatically across distinct phases - the opening, middlegame, and endgame each demand different forms of reasoning and decision-making. Yet, many modern chess engines rely on a single neural network to play the entire game uniformly, often missing opportunities to specialize. In this work, we introduce M2CTS, a modular framework that combines Mixture of Experts with Monte Carlo Tree Search to adapt strategy dynamically based on game phase. We explore three different methods for training the neural networks: Separated Learning, Staged Learning, and Weighted Learning. By routing decisions through specialized neural networks trained for each phase, M2CTS improves both computational efficiency and playing strength. In experiments on chess, M2CTS achieves up to +122 Elo over standard single-model baselines and shows promising generalization to multi-agent domains such as Pommerman. These results highlight how modular, phase-aware systems can better align with the structured nature of games and move us closer to human-like behavior in dividing a problem into many smaller units.

    Read less
27. Beyond Evaluation: Learning Contextual Chess Position Representations

    Ben Hull , 2025

    PDF

    This paper introduces ChessLM, a novel Transformer-based model designed to learn rich, contextual vector representations (embeddings) of chess positions. Moving beyond traditional chess engines foc... Read more

    This paper introduces ChessLM, a novel Transformer-based model designed to learn rich, contextual vector representations (embeddings) of chess positions. Moving beyond traditional chess engines focused on move evaluation, our approach is inspired by the success of self-supervised learning in Natural Language Processing. We adapt the Vision Transformer architecture and train the model using two self-supervised tasks on a large corpus of chess games: Masked Piece Prediction, which requires the model to infer masked pieces based on board context, and Move Difference Prediction, which involves predicting the number of moves between two positions from the same game. Qualitative analysis demonstrates the model’s ability to capture high-level structural and thematic similarities between positions across different game phases (opening, middlegame, endgame), such as shared pawn structures, material imbalances, and king safety patterns. However, the analysis also reveals limitations that would significantly impact our model’s ability to generate position evaluations. Despite this, the learned embeddings are shown to be effective for retrieving strategically or tactically similar positions, enabling applications such as intelligent chess puzzle suggestions. This work contributes a novel method for learning chess position representations that encode thematic information, opening avenues for future research in applying representation learning to various chess-related tasks beyond traditional evaluation.

    Read less
28. Methods and Tools for Teaching Chess in Higher Education

    Kifayet Huseynova and Aide Novruzova , 2025

    This paper examines pedagogical approaches and instructional tools for teaching chess in higher education. Chess instruction in universities can serve disciplinary goals (e.g., sport sciences, cogn... Read more

    This paper examines pedagogical approaches and instructional tools for teaching chess in higher education. Chess instruction in universities can serve disciplinary goals (e.g., sport sciences, cognitive psychology), cross-curricular goals (critical thinking, problem solving), and extra-curricular objectives (wellness, student engagement). Drawing on theoretical frameworks from constructivist and experiential learning, and on empirical literature about cognitive and educational effects of chess training, the paper presents a structured course design, recommended teaching methods, practical activities, digital and physical tools, assessment strategies, and implementation considerations. The aim is to provide instructors and programme designers with an evidence-informed, practical roadmap to develop effective, measurable, and scalable chess courses or modules that align with higher-education learning outcomes.

    Read less
29. Can Large Language Models Develop Strategic Reasoning? Post-training Insights from Learning Chess

    Dongyoon Hwang, Hojoon Lee, Jaegul Choo, Dongmin Park, and Jongho Park Porta Universorum, 2025

    While reinforcement learning (RL) for large language models (LLMs) has shown promise in mathematical reasoning, strategic reasoning for LLMs using RL remains largely unexplored. We investigate whet... Read more

    While reinforcement learning (RL) for large language models (LLMs) has shown promise in mathematical reasoning, strategic reasoning for LLMs using RL remains largely unexplored. We investigate whether LLMs can develop strategic reasoning capabilities through RL in chess. To this end, we leverage a chess-pretrained action-value network to provide dense reward on the LLM’s output move quality, which can be seen as a form of knowledge distillation. Our experiments show that our distillation-based dense rewards often outperform sparse binary rewards. However, surprisingly, all models plateau far below expert levels. We provide SFT and RL ablations on chess reasoning training and find evidence that this limitation stems from a deficit in the pretrained models’ internal understanding of chess-a deficit which RL alone may not be able to fully overcome. The code is available at this https URL: https://github.com/krafton-ai/Chess-R1

    Read less
30. Generalized Multi-Linear Models for Sufficient Dimension Reduction on Tensor Valued Predictors

    Daniel Kapla and Efstathia Bura CoRR, 2025

    We consider supervised learning (regression/classification) problems with tensor-valued input. We derive multi-linear sufficient reductions for the regression or classification problem by modeling ... Read more

    We consider supervised learning (regression/classification) problems with tensor-valued input. We derive multi-linear sufficient reductions for the regression or classification problem by modeling the conditional distribution of the predictors given the response as a member of the quadratic exponential family. We develop estimation procedures of sufficient reductions for both continuous and binary tensor-valued predictors. We prove the consistency and asymptotic normality of the estimated sufficient reduction using manifold theory. For continuous predictors, the estimation algorithm is highly computationally efficient and is also applicable to situations where the dimension of the reduction exceeds the sample size. We demonstrate the superior performance of our approach in simulations and real-world data examples for both continuous and binary tensor-valued predictors.

    Read less
31. Binary Matching of Android and iOS Apps

    Alexander Keusch , 2025

    Android and iOS are the two dominant mobile operating systems in the rapidly expanding smartphone market, serving billions of users worldwide. Both platforms feature extensive app stores with milli... Read more

    Android and iOS are the two dominant mobile operating systems in the rapidly expanding smartphone market, serving billions of users worldwide. Both platforms feature extensive app stores with millions of applications available for download. While security measures are in place to prevent the distribution of malicious or vulnerable apps, instances of malware have still been discovered in both stores. These incidents highlight a significant security risk that threatens user privacy and highlights the urgent need for advanced research in detecting malicious code and security vulnerabilities in mobile applications. As a step toward addressing this challenge, this thesis explores the feasibility of using binary similarity detection techniques to identify similarities between Android and iOS applications. To achieve this, we designed and implemented a novel analysis pipeline specifically tailored for comparing Android apps (compiled into binary OAT files) with iOS binaries. This pipeline enables the automatic identification of matches between corresponding applications across both platforms. The pipeline comprises several key stages, including preparing the apps and their third-party libraries for binary analysis, disassembling the binaries using both IDA Pro and Ghidra to account for potential variations introduced by different disassemblers, and conducting similarity analysis on the disassembly results. To assess the effectiveness of our pipeline, we conducted a comprehensive analysis on a dataset of 100 cross-platform apps. Our findings indicate that current binary similarity analysis methods have limitations in directly identifying cross-platform similarities between applications. However, we demonstrated that incorporating third-party libraries into the analysis significantly enhances similarity detection and can help to provide meaningful insights. This highlights the crucial role that third-party libraries play in cross-platform app analysis. Additionally, we found that the choice of disassembler has a significant impact on analysis results. Notably, no single disassembler proved to be clearly superior, as both exhibited their own strengths and limitations. Ultimately, this study offers valuable insights into the challenges and potential of cross-platform binary app analysis using traditional binary diffing techniques, laying a strong foundation for future research in this evolving field.

    Read less
32. A Modular Voice-Controlled IoT Architecture for Screenless Real-Time Interaction

    Ojas Khamele, Amruta Lambe, and Praveen Pawar , 2025

    The growing adoption of the Internet of Things (IoT) highlights the need for intuitive, accessible, and screenless modes of interaction. Voice interfaces, combining speech-to-text (STT) and text-to... Read more

    The growing adoption of the Internet of Things (IoT) highlights the need for intuitive, accessible, and screenless modes of interaction. Voice interfaces, combining speech-to-text (STT) and text-to-speech (TTS) processing, provide a natural mechanism for controlling IoT systems while enabling inclusive user experiences. This paper proposes a modular architecture for voice-controlled IoT applications, designed for reusability across domains with minimal adaptation. The architecture integrates embedded hardware, cloud-based speech services, and real-time feedback mechanisms, forming a flexible pipeline for voice- driven interactions. To demonstrate feasibility, the architecture is implemented in the context of online blindfold chess through integration with the Lichess API. The system supports realtime gameplay using only spoken input and audio feedback, providing an accessible interface for visually impaired users and a valuable tool for blindfold chess training. Evaluation of the prototype indicates an STT accuracy of 85% after optimizations, average move execution delay of 3.5 seconds, and end-to-end round-trip latency below 8 seconds. These results validate the practicality of the design and position it as a reusable template for robust, voice-driven IoT applications.

    Read less
33. Collaborative versus Individual Chess Puzzle Solving

    Alex Knopps In 2025 IEEE 17th International Conference on Computational Intelligence and Communication Networks (CICN), 2025

    The 2024 Chessable Research Awards had five student winners, including Alex Knopps, the author of this guest blog post. Knopps explores whether solving chess puzzles alone or with a partner leads t... Read more

    The 2024 Chessable Research Awards had five student winners, including Alex Knopps, the author of this guest blog post. Knopps explores whether solving chess puzzles alone or with a partner leads to better outcomes. His research also accounted for the difficulty of puzzles. The results indicate that there wasn’t much difference between the number of correctly solved chess puzzles in both individual and collaborative settings. However, working in a group led to fewer errors.

    Read less
34. LLM CHESS: Benchmarking Reasoning and Instruction-Following in LLMs through Chess

    Sai Kolasani, Maxim Saplin, Nicholas Crispino, Kyle Montgomery, Jared Davis, Matei Zaharia, Chi Wang, and Chenguang Wang , 2025

    We introduce LLM CHESS, an evaluation framework designed to probe the generalization of reasoning and instruction-following abilities in large language models (LLMs) through extended agentic intera... Read more

    We introduce LLM CHESS, an evaluation framework designed to probe the generalization of reasoning and instruction-following abilities in large language models (LLMs) through extended agentic interaction in the domain of chess. We rank over 50 open and closed source models by playing against a random opponent using a range of behavioral metrics, including win and loss rates, move quality, move legality, hallucinated actions, and game duration. For a subset of models, we derive an Elo estimate by playing against a chess engine with variably configured skill. Despite the simplicity of the instruction-following task and the weakness of the opponent, many state-of-the-art models struggle to complete games or achieve consistent wins. Similar to other benchmarks on complex reasoning tasks, our experiments reveal a clear separation between reasoning and non-reasoning models. However, unlike existing static benchmarks, the stochastic and dynamic nature of LLM CHESS uniquely reduces overfitting and memorization while preventing benchmark saturation. To support future work on evaluating reasoning and instruction-following in LLMs, we release our experimental framework, a public leaderboard, and a dataset of associated games. Our code is available athttps://github.com/LLM-CHESS/llm_chess.

    Read less
35. Synthetic Data Generation for Screen Time and App Usage

    Gustavo Kruger, Nikhil Sachdeva, and Michael Sobolev In Workshop on Foundations of Reasoning in Language Models at NeurIPS 2025, 2025

    Smartphone usage data can provide valuable insights for understanding interaction with technology and human behavior. However, collecting large-scale, in-the-wild smartphone usage logs is challengi... Read more

    Smartphone usage data can provide valuable insights for understanding interaction with technology and human behavior. However, collecting large-scale, in-the-wild smartphone usage logs is challenging due to high costs, privacy concerns, under representative user samples and biases like non-response that can skew results. These challenges call for exploring alternative approaches to obtain smartphone usage datasets. In this context, large language models (LLMs) such as Open AI’s ChatGPT present a novel approach for synthetic smartphone usage data generation, addressing limitations of real-world data collection. We describe a case study on how four prompt strategies influenced the quality of generated smartphone usage data. We contribute with insights on prompt design and measures of data quality, reporting a prompting strategy comparison combining two factors, prompt level of detail (describing a user persona, describing the expected results characteristics) and seed data inclusion (with versus without an initial real usage example). Our findings suggest that using LLMs to generate structured and behaviorally plausible smartphone use datasets is feasible for some use cases, especially when using detailed prompts. Challenges remain in capturing diverse nuances of human behavioral patterns in a single synthetic dataset, and evaluating tradeoffs between data fidelity and diversity, suggesting the need for use-case-specific evaluation metrics and future research with more diverse seed data and different LLM models.

    Read less
36. Policies of Multiple Skill Levels for Better Strength Estimation in Games

    Kyota Kuboki, Tatsuyoshi Ogawa, Chu-Hsuan Hsueh, Shi-Jim Yen, and Kokolo Ikeda , 2025

    Accurately estimating human skill levels is crucial for designing effective human-AI interactions so that AI can provide appropriate challenges or guidance. In games where AI players have beaten to... Read more

    Accurately estimating human skill levels is crucial for designing effective human-AI interactions so that AI can provide appropriate challenges or guidance. In games where AI players have beaten top human professionals, strength estimation plays a key role in adapting AI behavior to match human skill levels. In a previous state-of-the-art study, researchers have proposed a strength estimator trained using human players’ match data. Given some matches, the strength estimator computes strength scores and uses them to estimate player ranks (skill levels). In this paper, we focus on the observation that human players’ behavior tendency varies according to their strength and aim to improve the accuracy of strength estimation by taking this into account. Specifically, in addition to strength scores, we obtain policies for different skill levels from neural networks trained using human players’ match data. We then combine features based on these policies with the strength scores to estimate strength. We conducted experiments on Go and chess. For Go, our method achieved an accuracy of 80% in strength estimation when given 10 matches, which increased to 92% when given 20 matches. In comparison, the previous state-of-the-art method had an accuracy of 71% with 10 matches and 84% with 20 matches, demonstrating improvements of 8-9%. We observed similar improvements in chess. These results contribute to developing a more accurate strength estimation method and to improving human-AI interaction.

    Read less
37. Optimizing AI-Driven Chess Bots: Strategies for Balancing Performance, Accuracy, and Computational Efficiency

    Girish Kumar D, K S Shiva Kumar, P Pani Rama Prasad, Sangamesh C Jalade, C T M Praveen Kumar, and Subhashree D C CoRR, 2025
38. Exploring resource-rational planning under time pressure in online chess

    Ionatan Kuperwajs, Evan Russek, Lisa Schut, Yotam Sagiv, Marcelo G Mattar, Wei Ji Ma, and Tom Griffiths In 2025 4th International Conference on Distributed Computing and Electrical Circuits and Electronics (ICDCECE), 2025

    Human planning is incredibly efficient. Even in complex situations with many possible courses of action, people are able to make good decisions. Recent proposals suggest that a primary contributor ... Read more

    Human planning is incredibly efficient. Even in complex situations with many possible courses of action, people are able to make good decisions. Recent proposals suggest that a primary contributor to this efficiency is the intelligent use of cognitive resources, but how people allocate these resources under time constraints is not fully understood. In this work, we conduct a resource-rational analysis of planning in a large data set of online chess games. We first demonstrate that players spent more time thinking when they had more time to do so, and that this effect was especially prevalent when computation was more valuable. Then, we show that additional time spent planning resulted in better selected moves when one existed, and compare between signals of general and immediate time pressure. Finally, we highlight the role of expertise in this setting. Our results provide evidence that people make resource-rational choices when planning under time pressure.

    Read less
39. Computer Vision for Chess Game Automation

    Samir Lemeš, Mirhad Kolić, and Edin Tabak Proceedings of the Annual Meeting of the Cognitive Science Society, 2025

    In the modern age of computing and technology, computer vision has become a key aspect of numerous innovations and solutions that make everyday life easier. Object recognition in images is one area... Read more

    In the modern age of computing and technology, computer vision has become a key aspect of numerous innovations and solutions that make everyday life easier. Object recognition in images is one area where computer vision can contribute to significant improvements. Playing chess, one of the oldest and most challenging intellectual games requires concentration and tactical thinking. In computer vision, automating the recognition of chessboards and figures can contribute to developing advanced chess applications, help users analyse games, or even enable a game with a computer based on real-world chess setups. This paper describes how the system for recognising chessboards and figures using computer vision techniques was developed and implemented. Through analysing existing methods, designing a new algorithm, and experimental evaluation, the goal is to create an accurate and efficient system that can recognise chess positions and individual figures based on images.

    Read less
40. Characterizing Chess Player Styles with Neural Network Embeddings from Stockfish

    Victor Lequenne In New Technologies, Development and Application VIII, 2025

    This thesis investigates how neural networks can be used to analyze and compare chess player styles. Using the last layer of Stockfish’s neural network, we process positions from historical World C... Read more

    This thesis investigates how neural networks can be used to analyze and compare chess player styles. Using the last layer of Stockfish’s neural network, we process positions from historical World Chess Championships (1886–2024), as well as the 2024 World Blitz and Rapid Championships. We apply dimensionality reduction (PCA, t-SNE, MDS) and clustering (K-means) to build style-based player maps, measuring similarities through Jensen–Shannon divergence. Results show consistent stylistic signatures for some players–such as Firouzja and Dubov but may fail with others e.g. Nepomniachtchi. We also confirmed the evolution of chess player’s still accros time. Though the approach is promising, limitations remain, especially in data size and the influence of forced lines. Still, this work lays a foundation for future stylometry studies and applications in AI driven chess training

    Read less
41. A Stacking-Based Ensemble Approach for Predicting Chess Puzzle Difficulty

    Alan Liang, Cenzhi Liu, Kai Wang, and Ethan Liu , 2025

    FedCSIS 2025 competition is to predict the difficulty of chess puzzles, we present a structured multi-stage regression pipeline developed for the FedCSIS 2025 Challenge. The approach consists of th... Read more

    FedCSIS 2025 competition is to predict the difficulty of chess puzzles, we present a structured multi-stage regression pipeline developed for the FedCSIS 2025 Challenge. The approach consists of three stages: (i) four Elo-banded base models trained on separate rating ranges to capture localized difficulty semantics and mitigate bias in imbalanced datasets; (ii) a feature-level stacking ensemble combining base predictions with structural attributes, such as success probabilities, failure distributions, and solution length, to enhance cross-band generalization; and (iii) a lightweight post-hoc residual correction to reduce systematic prediction biases. Additionally, an uncertainty-aware mask-based evaluation is introduced to identify the 10% most challenging puzzles for extended scoring. Our method achieved competitive results, ranking 7th in the final leaderboard, while maintaining low computational cost. These findings demonstrate that lightweight, interpretable models, when combined with structural reasoning and uncertainty estimation, can rival more complex deep-learning approaches. This study highlights the potential of structured machine learning pipelines for scalable, human-centric chess puzzle analytics.

    Read less
42. ChessArena: A Chess Testbed for Evaluating Strategic Reasoning Capabilities of Large Language Models

    Jincheng Liu, Sijun He, Jingjing Wu, Xiangsen Wang, Yang Chen, Zhaoqi Kuang, Siqi Bao, and Yuan Yao In Proceedings of the 20th Conference on Computer Science and Intelligence Systems (FedCSIS), 2025

    Recent large language models (LLMs) have shown strong reasoning capabilities. However, a critical question remains: do these models possess genuine reasoning skills particularly complex strategic r... Read more

    Recent large language models (LLMs) have shown strong reasoning capabilities. However, a critical question remains: do these models possess genuine reasoning skills particularly complex strategic reasoning or are they primarily excelling at sophisticated pattern recognition within their training data? To address this question, this paper presents a chess testbed, ChessArena, to evaluate the strategic reasoning capabilities of LLMs. Chess requires complex strategic reasoning capabilities including long-term planning, strict rule comprehension, and multi-turn conversation memorization. Specifically, ChessArena is a competitive framework where LLMs play against each other, under four different play modes. The testbed is equipped with a ranking algorithm and a leaderboard. The testbed can also evaluate fine-grained capabilities including basic understanding, move selection, and puzzle solving. Over 13 LLMs with different modes are evaluated in ChessArena, playing over 800 games. The results reveal significant shortcomings in current LLMs: no model can beat Maia-1100 (a chess engine at human amateur level), while some even failed to defeat a random player that selects moves arbitrarily. We also present a strong baseline to the testbed: our fine-tuned Qwen3-8B substantially improved performance, approaching much larger state-of-the-art reasoning models.

    Read less
43. Hybrid Boosting and Multi-Modal Fusion for Chess Puzzle Difficulty Prediction

    Ming Liu, Junye Wang, Yinghan Hu, Xiaolin Yang, and Defu Lin , 2025

    The FedCSIS 2025 Challenge on Predicting Chess Puzzle Difficulty tasked participants with estimating puzzle ratings directly from board states and solution sequences, without relying on human solve... Read more

    The FedCSIS 2025 Challenge on Predicting Chess Puzzle Difficulty tasked participants with estimating puzzle ratings directly from board states and solution sequences, without relying on human solver statistics. We propose a three-stage hybrid framework integrating gradient-boosting regressors, a multi-modal neural network, and an XGBoost stacking ensemble. The boosting stage modeled handcrafted structural features derived from FEN and engine metadata, while the multi-modal network jointly learned from structured features and image-rendered chessboards to capture positional and tactical patterns. The residual-based stacking stage explicitly modeled prediction errors to correct systematic biases and enhance performance, particularly for high-difficulty puzzles. Our method achieved a competitive performance, ranking 7th in the preliminary stage and 8th in the final leaderboard. These results demonstrate that combining interpretable boosting models with visual-tactical deep representations and meta-learning provides a robust and computationally efficient alternative to large-scale transformer-based approaches.

    Read less
44. A Neural Network Approach to Chess Cheat Detection

    Maksim Iavich and Zura Kevanishvili In Proceedings of the 20th Conference on Computer Science and Intelligence Systems (FedCSIS), 2025

    With the development of chess engines, cheating online has never been easier, resulting in a need for more robust and accurate detection systems. This paper presents a novel approach to chess cheat... Read more

    With the development of chess engines, cheating online has never been easier, resulting in a need for more robust and accurate detection systems. This paper presents a novel approach to chess cheater detection that combines conventional chess engines and neural networks to help identify which games are authentically played by humans and which show signs of extraneous intervention. By utilizing Stockfish to measure centipawn loss and its mathematical derivatives, we can measure deviations from typical computer-generated moves much like in conventional anti-cheat systems. Additionally, the neural network Maia, designed specifically to mimic human play, transmutes centipawn loss data to highlight deviations from human style. This dual-measurement system addresses the limitations of the given traditional anti-cheat systems, which face the issue of distinguishing between strong human players and those using engines. The collected metadata is analyzed using a sequential neural network, which identifies patterns of fair play violation. Our approach offers a robust solution for maintaining the integrity of online chess by accurately detecting and preventing cheating.

    Read less
45. Re-evaluating metamorphic testing of chess engines: A replication study

    Axel Martin, Djamel Eddine Khelladi, Théo Matricon, and Mathieu Acher In Information and Software Technologies, 2025

    Context: This study aims to confirm, replicate and extend the findings of a previous article entitled ”Metamorphic Testing of Chess Engines” that reported inconsistencies in the analyses provided b... Read more

    Context: This study aims to confirm, replicate and extend the findings of a previous article entitled ”Metamorphic Testing of Chess Engines” that reported inconsistencies in the analyses provided by Stockfish, the most widely used chess engine, for transformed chess positions that are fundamentally identical. Initial findings, under conditions strictly identical to those of the original study, corroborate the reported inconsistencies. Objective: However, the original article considers a specific dataset (including randomly generated chess positions, end-games, or checkmate problems) and very low analysis depth (10 plies11A ply refers to a single turn taken by one player in a game. Two plies, one from each player, together constitute a complete move., corresponding to 5 moves). These decisions pose threats that limit generalizability of the results, but also their practical usefulness both for chess players and maintainers of Stockfish. Thus, we replicate the original study. Methods: We consider this time (1) positions derived from actual chess games, (2) analyses at appropriate and larger depths, and (3) different versions of Stockfish. We conduct novel experiments on thousands of positions, employing significantly deeper searches. Results: The replication results show that the Stockfish chess engines demonstrate significantly greater consistency in its evaluations. The metamorphic relations are not as effective as in the original article, especially on realistic chess positions. We also demonstrate that, for any given position, there exists a depth threshold beyond which further increases in depth do not result in any evaluation differences for the studied metamorphic relations. We perform an in-depth analysis to identify and clarify the implementation reasons behind Stockfish’s inconsistencies when dealing with transformed positions. Conclusion: A first concrete result is thus that metamorphic testing of chess engines is not yet an effective technique for finding faults of Stockfish. Another result is the lessons learned through this replication effort: metamorphic relations must be verified in the context of the domain’s specificities; without such contextual validation, they may lead to misleading or irrelevant conclusions; changes in parameters and input dataset can drastically alter the effectiveness of a testing method.

    Read less
46. Practice Structure Predicts Skill Growth in Online Chess: A Behavioral Modeling Approach

    Luı́s Meireles, Tiago Mendes-Neves, and João Moreira Information and Software Technology, 2025

    Skill acquisition is central to developing expertise, yet the behavioral mechanisms that separate more successful learners from less successful ones remain poorly understood. Using a large naturali... Read more

    Skill acquisition is central to developing expertise, yet the behavioral mechanisms that separate more successful learners from less successful ones remain poorly understood. Using a large naturalistic dataset of about one million online chess games played by  \hspace0.167em820 individuals over three years (2013–2015), we built an interpretable machine learning model to classify learners based only on behavioral features. Learners were labeled as “fast learners” or “not fast learners” based on normalized monthly Elo progression, adjusted for both starting rating and the increasing difficulty of improving at higher levels. We engineered time-sensitive features across four behavioral dimensions: practice structure, challenge level, strategic exploration (measured via move-sequence entropy), and tactical efficiency (the number of rounds needed to reach a 70% win probability in games eventually won). A logistic regression model trained on the five strongest predictors - optimal challenge steady magnitude, optimal challenge late slope, entropy steady magnitude, optimal challenge mean, and tactical efficiency mean - achieved an F1 of 0.68 and an AUC of 0.78. Coefficients showed that average tactical efficiency was a strong predictor of fast learning, whereas the role of challenge-level features was less clear. To explore this, we fitted a linear regression with average tactical efficiency (as a proxy for expertise) as the dependent variable. This model explained 53% of the variance (R\ensuremath^2 = 0.53, RMSE\hspace0.167em=\hspace0.167em0.05) and revealed optimal challenge as the strongest predictor. This suggests that well-calibrated challenge levels are key to differences in chess performance.

    Read less
47. Out-of-distribution Tests Reveal Compositionality in Chess Transformers

    Anna Mészáros, Patrik Reizinger, and Ferenc Huszár , 2025

    Chess is a canonical example of a task that requires rigorous reasoning and long-term planning. Modern decision Transformers - trained similarly to LLMs - are able to learn competent gameplay, but ... Read more

    Chess is a canonical example of a task that requires rigorous reasoning and long-term planning. Modern decision Transformers - trained similarly to LLMs - are able to learn competent gameplay, but it is unclear to what extent they truly capture the rules of chess. To investigate this, we train a 270M parameter chess Transformer and test it on out-of-distribution scenarios, designed to reveal failures of systematic generalization. Our analysis shows that Transformers exhibit compositional generalization, as evidenced by strong rule extrapolation: they adhere to fundamental syntactic rules of the game by consistently choosing valid moves even in situations very different from the training data. Moreover, they also generate high-quality moves for OOD puzzles. In a more challenging test, we evaluate the models on variants including Chess960 (Fischer Random Chess) - a variant of chess where starting positions of pieces are randomized. We found that while the model exhibits basic strategy adaptation, they are inferior to symbolic AI algorithms that perform explicit search, but gap is smaller when playing against users on Lichess. Moreover, the training dynamics revealed that the model initially learns to move only its own pieces, suggesting an emergent compositional understanding of the game.

    Read less
48. Pretraining Transformers for Chess Puzzle Difficulty Prediction

    Szymon Miłosz , 2025

    This paper presents our third-place solution for the FedCSIS 2025 Challenge: Predicting Chess Puzzle Difficulty - Second Edition. Building on our prior GlickFormer architecture, we develop a transf... Read more

    This paper presents our third-place solution for the FedCSIS 2025 Challenge: Predicting Chess Puzzle Difficulty - Second Edition. Building on our prior GlickFormer architecture, we develop a transformer-based approach featuring a novel multitask pretraining strategy that combines masked-square reconstruction with solution policy prediction. Our spatial-only architecture directly embeds solution moves, eliminating temporal modules, while integrating human-centric priors through Maia-2 engine solve-rate predictions. Evaluated on the Lichess puzzle corpus, our approach reduces validation MSE by 30.4% compared to from-scratch training and achieves competitive results (test MSE: 55.9k) despite distribution shifts in the competition environment.\hspace0.6em

    Read less
49. Observational Analysis of Mistakes in Chess Initiation, Using Decision Trees

    Jorge Miranda, Javier Arana, Daniel Lapresa, and M. Teresa Anguera In Proceedings of the 20th Conference on Computer Science and Intelligence Systems (FedCSIS), 2025
50. Adaptive decision making in the wild: a case study of chess

    Supratik Mondal and Jakub Traczyk International Journal of Computer Science in Sport, 2025

    Chess is a game of strategic thinking and time management, where a player can lose a game on time despite making all the best moves. Finding the best move is a deliberate and energy-intensive proce... Read more

    Chess is a game of strategic thinking and time management, where a player can lose a game on time despite making all the best moves. Finding the best move is a deliberate and energy-intensive process in a game where players are often under time pressure. Therefore, players who can balance this trade-off will have a significant advantage. The current study explores such instances where winning is contingent on how well players balance their accuracy under time pressure. We found that winning players, compared to their opponents, followed a more adaptive decision strategy–they made more theoretical best moves (i.e., accurate moves) in highly critical positions. However, the accuracy difference between the opponents was very similar in less critical positions. We conclude that winning players have a better understanding of when and how to allocate their limited resources efficiently, even when controlling for differences in skill levels, compared to their opponents.

    Read less
51. Estimation of partial rankings from sparse, noisy comparisons

    Sebastian Morel-Balbi and Alec Kirkley Thinking & Reasoning, 2025

    PDF

    Ranking items from pairwise comparisons is common in domains ranging from sports to consumer preferences. Statistical inference-based methods, such as the Bradley–Terry model, have emerged as flexi... Read more

    Ranking items from pairwise comparisons is common in domains ranging from sports to consumer preferences. Statistical inference-based methods, such as the Bradley–Terry model, have emerged as flexible and powerful tools to tackle ranking in empirical data. However, in situations with limited and/or noisy comparisons, it is often challenging to confidently distinguish item performance based on evidence available in the data. Most ranking methods nevertheless force a complete ordering, suggesting a meaningful distinction when there is none. Here, we introduce a principled nonparametric Bayesian framework for learning partial rankings—rankings with ties—that infers distinctions between items only when supported by the evidence. We develop a fast agglomerative algorithm for Maximum a Posteriori (MAP) inference under this framework and evaluate its performance on a range of synthetic and real-world datasets, finding that it often yields a more parsimonious and reliable summary of the data than traditional ranking approaches, particularly in sparse observational settings.

    Read less
52. Mining Security Documentation Practices in OpenAPI Descriptions

    Diana Carolina Muñoz Hurtado, Souhaila Serbout, and Cesare Pautasso Communications Physics, 2025

    Security is an integral requirement of any trustworthy software architecture, particularly critical for application programming interfaces (APIs). In this paper, we survey security documentation pr... Read more

    Security is an integral requirement of any trustworthy software architecture, particularly critical for application programming interfaces (APIs). In this paper, we survey security documentation practices, specifically API security schemes related to authentication and authorization, by mining a large collection of OpenAPI descriptions retrieved from open-source GitHub repositories. Our study focuses on detecting existing security schemes and evaluating their prevalence and positioning within API descriptions. We distinguish whether security schemes are introduced locally (at the path or operation level) or globally (for the entire API). Our analysis highlights scenarios where security schemes are featured in APIs in different proportions over time, thus tracking whether the API documentation tends to include more (or less) security details as the API evolves.

    Read less
53. A Comparison of the Effects of Model Adaptation Techniques on Large Language Models for Non-Linguistic and Linguistic Tasks

    Khoa Nguyen, Sadia Jahan, and Rocky Slavin , 2025

    Generative large language models (LLMs) have revolutionized natural language processing (NLP) by demonstrating exceptional performance in interpreting and generating human language. There has been ... Read more

    Generative large language models (LLMs) have revolutionized natural language processing (NLP) by demonstrating exceptional performance in interpreting and generating human language. There has been some exploration of their application to non-linguistic tasks, which could lead to significant advancements in fields that rely heavily on structured data and specialized knowledge. However, there has been limited direct comparison of the effects of model adaptation techniques for non-linguistic compared to linguistic tasks with LLMs. To this end, the work in this paper investigates the effects of fine-tuning and few-shot learning on pre-trained LLMs for non-linguistic tasks using chess puzzles as a case study task. We compare the impact of fine-tuning and few-shot learning on models performing the same task represented in both chess notation (i.e., non-linguistic data) and natural language descriptions of the same chess notations (i.e., natural language data). Our experiments with Mixtral-8x7B-v0.1 and Meta-Llama-3-70B resulted in a 5% lower average increase in performance after fine-tuning for non-linguistic tasks compared to linguistic tasks. Similarly, few-shot learning on pre-trained models exhibited a 3% lower average increase in performance for on non-linguistic tasks compared to linguistic tasks. Furthermore, few-shot learning on fine-tuned models resulted in a significant accuracy drop, particularly for Mixtral, with a 24.82% decrease for non-linguistic tasks. These results suggest that fine-tuning and few-shot learning for generative LLMs have stronger effects on linguistic tasks and their data than for non-linguistic.

    Read less
54. Soft Prompts for Evaluation: Measuring Conditional Distance of Capabilities

    Ross Nordby In Proceedings of the 40th ACM/SIGAPP Symposium on Applied Computing, Catania International Airport, Catania, Italy, 2025

    To help evaluate and understand the latent capabilities of language models, this paper introduces an approach using optimized input embeddings, or ’soft prompts,’ as a metric of conditional distanc... Read more

    To help evaluate and understand the latent capabilities of language models, this paper introduces an approach using optimized input embeddings, or ’soft prompts,’ as a metric of conditional distance between a model and a target behavior. The technique aims to facilitate latent capability discovery as a part of automated red teaming/evaluation suites and to provide quantitative feedback about the accessibility of potentially concerning behaviors in a way that may scale to powerful future models, including those which may otherwise be capable of deceptive alignment. An evaluation framework using soft prompts is demonstrated in natural language, chess, and pathfinding, and the technique is extended with generalized conditional soft prompts to aid in constructing task evaluations.

    Read less
55. Chess Rating Estimation from Moves and Clock Times Using a CNN-LSTM

    Michael Omori and Prasad Tadepalli , 2025

    PDF

    Current chess rating systems update ratings incrementally and may not always accurately reflect a player’s true strength at all times, especially for rapidly improving players or very rusty players... Read more

    Current chess rating systems update ratings incrementally and may not always accurately reflect a player’s true strength at all times, especially for rapidly improving players or very rusty players. To overcome this, we explore a method to estimate player ratings directly from game moves and clock times. We compiled a benchmark dataset from Lichess with over one million games, encompassing various time controls and including move sequences and clock times. Our model architecture comprises a CNN to learn positional features, which are then integrated with clock-time data into a Bidirectional LSTM, predicting player ratings after each move. The model achieved an MAE of 182 rating points on the test data. Additionally, we applied our model to the 2024 IEEE Big Data Cup Chess Puzzle Difficulty Competition dataset, predicted puzzle ratings and achieved competitive results. This model is the first to use no hand-crafted features to estimate chess ratings and also the first to output a rating prediction after each move. Our method highlights the potential of using move-based rating estimation for enhancing rating systems and potentially other applications such as cheating detection.

    Read less
56. Leveraging Systems and Control Theory for Social Robotics: A Model-Based Behavioral Control Approach to Human-Robot Interaction

    Maria Morão Patrı́cio and Anahita Jamshidnejad In Computers and Games, 2025

    Social robots (SRs) should autonomously interact with humans, while exhibiting proper social behaviors associated to their role. By contributing to health-care, education, and companionship, SRs wi... Read more

    Social robots (SRs) should autonomously interact with humans, while exhibiting proper social behaviors associated to their role. By contributing to health-care, education, and companionship, SRs will enhance life quality. However, personalization and sustaining user engagement remain a challenge for SRs, due to their limited understanding of human mental states. Accordingly, we leverage a recently introduced mathematical dynamic model of human perception, cognition, and decision-making for SRs. Identifying the parameters of this model and deploying it in behavioral steering system of SRs allows to effectively personalize the responses of SRs to evolving mental states of their users, enhancing long-term engagement and personalization. Our approach uniquely enables autonomous adaptability of SRs by modeling the dynamics of invisible mental states, significantly contributing to the transparency and awareness of SRs. We validated our model-based control system in experiments with 10 participants who interacted with a Nao robot over three chess puzzle sessions, 45 - 90 minutes each. The identified model achieved a mean squared error (MSE) of 0.067 (i.e., 1.675% of the maximum possible MSE) in tracking beliefs, goals, and emotions of participants. Compared to a model-free controller that did not track mental states of participants, our approach increased engagement by 16% on average. Post-interaction feedback of participants (provided via dedicated questionnaires) further confirmed the perceived engagement and awareness of the model-driven robot. These results highlight the unique potential of model-based approaches and control theory in advancing human-SR interactions.

    Read less
57. Inferring Piece Value in Chess and Chess Variants

    Steven Pav , 2025

    We use logistic regression to estimate the value of the pieces in standard chess and several chess variants, namely Chess 960, Atomic chess, Antichess, and Horde chess. We perform our regressions o... Read more

    We use logistic regression to estimate the value of the pieces in standard chess and several chess variants, namely Chess 960, Atomic chess, Antichess, and Horde chess. We perform our regressions on several years of data from Lichess, the free and open-source internet chess server. We use the published player ratings to control for the confounding effect of differential player skill. We adjust for the attenuation bias in regressions due to the noise in observed ratings. We find that major piece values, relative to the value of a pawn, are fairly consistent with historical valuation systems. However we find slightly higher value to bishops than knights. We find that piece values are smaller, in absolute value, in Atomic and Antichess than standard chess. We also present approximate values of the pieces to equalize odds when players of varying skill face off.

    Read less
58. Automatically Augmenting GitHub Issues with Informative User Reviews

    Arthur Pilone, Marco Raglianti, Michele Lanza, Fabio Kon, and Paulo Meirelles , 2025

    Development teams for mobile applications can receive thousands of user reviews daily. At the same time, these developers use different communication channels, such as the GitHub issue tracker. Alt... Read more

    Development teams for mobile applications can receive thousands of user reviews daily. At the same time, these developers use different communication channels, such as the GitHub issue tracker. Although GitHub issues are accessible and manageable for developers, their content often differs starkly from what users write in app reviews. Issues may lack steps to reproduce bugs or insights that justify the priority of new feature requests. The sheer volume of user reviews for a popular app, combined with their heterogeneity and varying quality, makes manual integration into issue trackers unfeasible. We present an approach that automatically augments GitHub issues with informative user reviews to bridge the gap between user feedback and developer-managed issues. Using a state-of-the-art large language model (LLM), our approach automatically retrieves user reviews with high semantic textual similarity (STS) to the issue content and suggests reviews that augment developers’ understanding of the issue. In this paper, we present large-scale quantitative and qualitative analyses to assess the feasibility of enriching development workflows with user-written information. Using over 37,000 issues and 750,000 reviews from 19 popular Free/Libre/Open Source Software (FLOSS) mobile applications, our approach augments 3,017 (8%) issues with 7,287 (1%) potentially informative reviews. In addition to providing insights into user-reported bugs and feature requests, the information from these matches points toward a novel and promising way to leverage user reviews for concerted app evolution.

    Read less
59. Programmatic Representation Learning with Language Models

    Gabriel Poesia and Georgia Gabriela Sampaio In 2025 International Conference on Software Maintenance and Evolution (ICSME), 2025
60. TDHook: A Lightweight Framework for Interpretability

    Yoann Poupart , 2025

    Interpretability of Deep Neural Networks (DNNs) is a growing field driven by the study of vision and language models. Yet, some use cases, like image captioning, or domains like Deep Reinforcement ... Read more

    Interpretability of Deep Neural Networks (DNNs) is a growing field driven by the study of vision and language models. Yet, some use cases, like image captioning, or domains like Deep Reinforcement Learning (DRL), require complex modelling, with multiple inputs and outputs or use composable and separated networks. As a consequence, they rarely fit natively into the API of popular interpretability frameworks. We thus present TDHook, an open-source, lightweight, generic interpretability framework based on ‘tensordict‘ and applicable to any ‘torch‘’ model. It focuses on handling complex composed models which can be trained for Computer Vision, Natural Language Processing, Reinforcement Learning or any other domain. This library features ready-to-use methods for attribution, probing and a flexible get-set API for interventions, and is aiming to bridge the gap between these method classes to make modern interpretability pipelines more accessible. TDHook is designed with minimal dependencies, requiring roughly half as much disk space as ‘transformer_lens‘, and, in our controlled benchmark, achieves up to a \texttimes2 speed-up over ‘captum‘ when running integrated gradients for multi-target pipelines on both CPU and GPU. In addition, to value our work, we showcase concrete use cases of our library with composed interpretability pipelines in Computer Vision (CV) and Natural Language Processing (NLP), as well as with complex models in DRL.

    Read less
61. A metacognitive appraisal of quitting

    Hariharan Purohit and Nisheeth Srivastava , 2025

    Stopping decisions are frequently modeled as decisions to switch to alternative activities once the current activity stops being adequately rewarding, such as in optimal foraging theory, as well as... Read more

    Stopping decisions are frequently modeled as decisions to switch to alternative activities once the current activity stops being adequately rewarding, such as in optimal foraging theory, as well as more recent metacognitive models. However, the sense of stopping and making decisions in such frameworks is highly platonic, with both decisions and stopping actions occurring instantaneously. In contrast, the phenomenology of quitting actions that one is undertaking appears to be temporally extended and metacognitively challenging. We study the metacognitive covariates of quitting decisions made by chess players using a large database of chess games sourced from an online chess portal. Our analysis reveals that players tend to persevere when they are playing against stronger opponents and after having played poor moves. We also find that a history of quitting games makes players more likely to quit in future games, but that having recently quit in a game offers some protective effect against quitting. Finally, we find that quitting a game makes it more likely that a player will play a game again soon. We place these results in the context of modeling quitting as a metacognitive choice affected by multiple competing goals.

    Read less
62. ’Sounds like a skill issue’: what makes you quit at chess?

    Hariharan Purohit and Nisheeth Srivastava Proceedings of the Annual Meeting of the Cognitive Science Society, 2025
63. Fluid Roles for Close-Knit Gaming: Households Playing Digital Games

    Heidi Rautalahti, Rongjun Ma, Amel Bourdoucen, Yajing Wang, and Janne Lindqvist , 2025

    Households increasingly play and engage with video games. We examined how households play video games among 20 interviewees coming from varied and familial households. Our study focused on interact... Read more

    Households increasingly play and engage with video games. We examined how households play video games among 20 interviewees coming from varied and familial households. Our study focused on interactions, examining how gaming influences daily household dynamics. Previous studies have focused mainly on the impact on relationships. Looking at households led us to observe fluid role dynamics around gaming. Our findings map the stages of how households play games from gaining plausible momentum, actions, conversations, and roles taken during game sessions, and reflections after gaming. Our findings highlight a novel role of the Gamer Host leading the game session and attending to everyone’s enjoyment. Our observations exemplify the supportive and positive social outcomes close-knit gaming can afford and implications for achieving harmonious gaming in households. Our findings tie to prospects on communal and social aspects on technology use providing new perspectives on user experiences in an immediate social environment.

    Read less
64. EloMetrics: Advanced Outcome Prediction for Chess Matches with Elo Ratings and Logistic Regression

    Ma. Julianna Re-an DG. Reyes, Eirnan Dicreto, Emmanuel Gabriel D. Santos, Daniella Franxene P. Limbag, and Gabriel Avelino Sampedro Proc. ACM Hum.-Comput. Interact., 2025

    PDF

    Chess is a complex game characterized by diverse strategies and time constraints, making quick decision-making essential for success. While Elo ratings are widely recognized as indicators of player... Read more

    Chess is a complex game characterized by diverse strategies and time constraints, making quick decision-making essential for success. While Elo ratings are widely recognized as indicators of player skill, the predictability of match outcomes based solely on these ratings remains a challenge. The study aims to develop a model for accurately predicting the outcome of chess games using logistic regression, focusing on Elo rating differences and the number of moves in each game. The dataset includes over 20,000 games from lichess.org, and only games with decisive outcomes (excluding draws) were used. The research categorizes Elo ratings into various classes and evaluates model performance across these ranges. The model achieves a predictive accuracy of 68.18%, demonstrating the significance of Elo ratings and move counts in determining game results. Performance metrics, including precision, recall, and Fl-score, further validate the model’s effectiveness. The study concludes that while Elo ratings and move count are strong predictors of chess outcomes, further refinement is needed to improve performance at high-ranking skill levels. The model performs well for experts but loses accuracy in master-level games, reflecting higher skill-level complexities. The insights gained from this research contribute to a deeper understanding of predictive modeling in chess, sug-gesting potential avenues for further investigation into additional Influencing factors.

    Read less
65. Coordination in Multi-Agent LLM Systems: The Role of a Question-Asking Agent in Guiding Collaborative Consensus

    Keon Roohani In 2025 International Conference on Electronics, Information, and Communication (ICEIC), 2025

    PDF

    Coordination, debate, and reflection have shown promising improvements in multi-agent Large Language Model (LLM) task performance. Inspired by the role of questioning in human group reasoning, this... Read more

    Coordination, debate, and reflection have shown promising improvements in multi-agent Large Language Model (LLM) task performance. Inspired by the role of questioning in human group reasoning, this research introduces a novel component to multi-agent LLM systems: a Question-Asking Agent (QAA) that guides collaboration through targeted, uncertainty-reducing questions. The QAA selects questions based on Expected Information Gain (EIG), a metric used to quantify the value of information a question may provide. To evaluate the impact of the QAA, a multi-agent LLM system was implemented and tested on the chess game state tracking task, a benchmark problem that challenges LLMs to maintain consistent reasoning across a sequential input. The system included generic agents collaborating through dialogue and a QAA generating questions using template-based formulations with calculable EIG. Experiments were conducted across 15 configurations varying the number of agents (1-5) and QAA strategy (none, random, EIG-driven). Results show that the QAA with EIG consistently improved system accuracy compared to both the baseline and the random-question QAA. Additionally, increasing the number of agents showed improvements across all QAA strategies. This study demonstrates that EIG-guided questioning can significantly improve reasoning performance in multi-agent LLM systems. These findings open new directions for enhancing coordination, interpretability, and performance in multi-agent LLM settings across a range of structured reasoning tasks beyond chess.

    Read less
66. Beyond Workload: Paving the Road for the Next Generation of Implicit Prefrontal Cortex Based Brain-Computer Interfaces

    Matthew Russell , Worcester, MA, USA, 2025

    The rapidly evolving field of Human-Computer Interaction (HCI) faces a fundamental constraint: the limited bandwidth of information exchange between users and computing systems. One promising appro... Read more

    The rapidly evolving field of Human-Computer Interaction (HCI) faces a fundamental constraint: the limited bandwidth of information exchange between users and computing systems. One promising approach to increasing this bandwidth is implicit interaction: a paradigm in which applications modify their state based on information gleaned from users, rather than direct input. Within the context of reading such information from human neural signals, this concept is formally recognized as implicit Brain-Computer Interfaces (implicit BCI). My work focuses on implicit BCIs which measure the prefrontal cortex (PFC); early prototypes have successfully leveraged the PFC to approximate mental workload, but much is left to be understood about the full potential of this region. Through three research projects spanning two brain measurement modalities, this dissertation makes targeted contributions to this area of research. With functional Near-Infrared Spectroscopy (fNIRS), I explore two facets of PFC activation demonstrated in functional Magnetic Resonance Imaging (fMRI)-based neuroscience research which are underexplored in applied contexts: episodic memory and brain-network based classification; in the first project, I study the measurable effects of episodic and working memory within the context of using Large Language Models (LLMs), and in the second project I develop a real-time implicit BCI designed to differentiate between different brain networks. The third project benchmarks low-cost EEG in three studies which distinguish brain states based on different factors: quality of moves made during chess playing, workload levels within standard cognitive psychology tasks, and cognitive states during the tasks. For all studies I use Linear Mixed Models (LMM) to observe macro patterns in the data, and machine learning to explore potential for implicit BCI. Results indicate that, in addition to the well-understood concept of measuring singular aspects of consciousness across a gradient (e.g. workload), promising potential exists for leveraging the PFC towards classification across tasks which engage different cognitive processes, both with fNIRS and low-cost EEG. Further, careful consideration of “noise” in implicit BCI introduces a new idea: Human-Sensor-Computer Interaction (HSCI). Taken together, this dissertation provides relevant context to inform the next generation of Human-Sensor-Computer systems, including PFC-based interfaces stretching past workload, and beyond.

    Read less
67. Decoding Chess Puzzle Play and Standard Cognitive Tasks for BCI: A Low-Cost EEG Study

    Matthew Russell, Samuel Youkeles, William Xia, Kenny Zheng, Aman Shah, and Robert J. K. Jacob , 2025
68. E-valuator: Reliable Agent Verifiers with Sequential Hypothesis Testing

    Shuvom Sadhuka, Drew Prinster, Clara Fannjiang, Gabriele Scalia, Aviv Regev, and Hanchen Wang CoRR, 2025

    Agentic AI systems execute a sequence of actions, such as reasoning steps or tool calls, in response to a user prompt. To evaluate the success of their trajectories, researchers have developed veri... Read more

    Agentic AI systems execute a sequence of actions, such as reasoning steps or tool calls, in response to a user prompt. To evaluate the success of their trajectories, researchers have developed verifiers, such as LLM judges and process-reward models, to score the quality of each action in an agent’s trajectory. Although these heuristic scores can be informative, there are no guarantees of correctness when used to decide whether an agent will yield a successful output. Here, we introduce e-valuator, a method to convert any black-box verifier score into a decision rule with provable control of false alarm rates. We frame the problem of distinguishing successful trajectories (that is, a sequence of actions that will lead to a correct response to the user’s prompt) and unsuccessful trajectories as a sequential hypothesis testing problem. E-valuator builds on tools from e-processes to develop a sequential hypothesis test that remains statistically valid at every step of an agent’s trajectory, enabling online monitoring of agents over arbitrarily long sequences of actions. Empirically, we demonstrate that e-valuator provides greater statistical power and better false alarm rate control than other strategies across six datasets and three agents. We additionally show that e-valuator can be used for to quickly terminate problematic trajectories and save tokens. Together, e-valuator provides a lightweight, model-agnostic framework that converts verifier heuristics into decisions rules with statistical guarantees, enabling the deployment of more reliable agentic systems.

    Read less
69. Domain Obedient Deep Learning

    Soumadeep Saha , 2025

    Deep learning, a family of data-driven artificial intelligence techniques, has shown immense promise in a plethora of applications, and it has even outpaced experts in several domains. However, unl... Read more

    Deep learning, a family of data-driven artificial intelligence techniques, has shown immense promise in a plethora of applications, and it has even outpaced experts in several domains. However, unlike symbolic approaches to learning, these methods fall short when it comes to abiding by and learning from pre-existing established principles. This is a significant deficit for deployment in critical applications such as robotics, medicine, industrial automation, etc. For a decision system to be considered for adoption in such fields, it must demonstrate the ability to adhere to specified constraints, an ability missing in deep learning-based approaches. Exploring this problem serves as the core tenet of this dissertation. This dissertation starts with an exploration of the abilities of conventional deep learning-based systems vis-à-vis domain coherence. A large-scale rule-annotated dataset is introduced to mitigate the pronounced lack of suitable constraint adherence evaluation benchmarks, and with its aid, the rule adherence abilities of vision systems are analyzed. Additionally, this study probes language models to elicit their performance characteristics with regard to domain consistency. Examination of these language models with interventions illustrates their ineptitude at obeying domain principles, and a mitigation strategy is proposed. This is followed by an exploration of techniques for imbuing deep learning systems with domain constraint information. Also, a comprehensive study of standard evaluation metrics and their blind spots pertaining to domain-aware performance estimation is undertaken. Finally, a novel technique to enforce constraint compliance in models without training is introduced, which pairs a search strategy with large language models to achieve cutting-edge performance. A key highlight of this dissertation is the emphasis on addressing pertinent real-world problems with scalable and practicable solutions. We hope the results presented here pave the way for wider adoption of deep learning-based systems in pivotal situations with enhanced confidence in their trustworthiness.

    Read less
70. The Memory Premium

    Yuval Salant, Jorg L Spenkuch, and David Almog , 2025

    We explore the role of memory for choice behavior in unfamiliar environments. Using a unique data set, we document that decision makers exhibit a "memory premium." They tend to choose in-memory al... Read more

    We explore the role of memory for choice behavior in unfamiliar environments. Using a unique data set, we document that decision makers exhibit a "memory premium." They tend to choose in-memory alternatives over out-of-memory ones, even when the latter are objectively better. Consistent with well-established regularities regarding the inner workings of human memory, the memory premium is associative, subject to interference and repetition effects, and decays over time. Even as decision makers gain familiarity with the environment, the memory premium remains economically large. Our results imply that the ease with which past experiences come to mind plays an important role in shaping choice behavior.

    Read less
71. Iterative Inference in a Chess-Playing Neural Network

    Elias Sandmann, Sebastian Lapuschkin, and Wojciech Samek , 2025

    Do neural networks build their representations through smooth, gradual refinement, or via more complex computational processes? We investigate this by extending the logit lens to analyze the policy... Read more

    Do neural networks build their representations through smooth, gradual refinement, or via more complex computational processes? We investigate this by extending the logit lens to analyze the policy network of Leela Chess Zero, a superhuman chess engine. We find strong monotonic trends in playing strength and puzzle-solving ability across layers, yet policy distributions frequently follow non-smooth trajectories. Evidence for this includes correct puzzle solutions that are discovered early but subsequently discarded, move rankings that remain poorly correlated with final outputs, and high policy divergence until late in the network. These findings contrast with the smooth distributional convergence typically observed in language models.

    Read less
72. The role of expertise, impulsivity, and preference for intuition on decision quality

    Robin Schrödter, Katrin Heyers, Jan Birkemeyer, and Stefanie Klatt In Mechanistic Interpretability Workshop at NeurIPS 2025, 2025

    Decision-making researchers often face a trade-off when conducting controlled laboratory experiments, as these can limit the ability to identify stable relationships between decision-making quality... Read more

    Decision-making researchers often face a trade-off when conducting controlled laboratory experiments, as these can limit the ability to identify stable relationships between decision-making quality and individual differences, such as expertise or personality traits. This study introduces an innovative paradigm that leverages the objective assessment capabilities of artificial intelligence in a naturalistic online chess setting. Ninety-four participants evaluated tactical chess positions and identified optimal moves under various time control conditions, followed by personality assessments. Expertise, measured by online Elo rating, emerged as a key predictor, accounting for 55 % of the variance in decision quality and 61 % in evaluation accuracy, underscoring the precision of the chosen approach. The study also highlights the significant impact of time control on decision-making. Additionally, the paradigm shows promise in examining the interplay between personality factors and decision-making processes, with a notable correlation between higher impulsivity scores and faster response times. However, high impulsivity was not associated with reduced decision quality, raising questions about the validity of this measurement. Overall, the results suggest that the chess paradigm, accessible to a broad sample due to the widespread appeal of online chess, provides a powerful tool that combines laboratory precision with real-world relevance.

    Read less
73. Bridging the human–AI knowledge gap through concept discovery and transfer in AlphaZero

    Lisa Schut, Nenad Tomašev, Thomas McGrath, Demis Hassabis, Ulrich Paquet, and Been Kim Personality and Individual Differences, 2025

    As AI systems become more capable, they may internally represent concepts outside the sphere of human knowledge. This work gives an end-to-end example of unearthing machine-unique knowledge in the ... Read more

    As AI systems become more capable, they may internally represent concepts outside the sphere of human knowledge. This work gives an end-to-end example of unearthing machine-unique knowledge in the domain of chess. We obtain machine-unique knowledge from an AI system (AlphaZero) by a method that finds novel yet teachable concepts and show that it can be transferred to human experts (grandmasters). In particular, the new knowledge is learned from internal mathematical representations without a priori knowing what it is or where to start. The produced knowledge from AlphaZero (new chess concepts) is then taught to four grandmasters in a setting where we can quantify their learning, showing that machine-guided discovery and teaching is possible at the highest human level. AI systems have attained superhuman performance across various domains. If the hidden knowledge encoded in these highly capable systems can be leveraged, human knowledge and performance can be advanced. Yet, this internal knowledge is difficult to extract. Due to the vast space of possible internal representations, searching for meaningful new conceptual knowledge can be like finding a needle in a haystack. Here, we introduce a method that extracts new chess concepts from AlphaZero, an AI system that mastered chess via self-play without human supervision. Our method excavates vectors that represent concepts from AlphaZero’s internal representations using convex optimization, and filters the concepts based on teachability (whether the concept is transferable to another AI agent) and novelty (whether the concept contains information not present in human chess games). These steps ensure that the discovered concepts are useful and meaningful. For the resulting set of concepts, prototypes (chess puzzle–solution pairs) are presented to experts for final validation. In a preliminary human study, four top chess grandmasters (all former or current world chess champions) were evaluated on their ability to solve concept prototype positions. All grandmasters showed improvement after the learning phase, suggesting that the concepts are at the frontier of human understanding. Despite the small scale, our result is a proof of concept demonstrating the possibility of leveraging knowledge from a highly capable AI system to advance the frontier of human knowledge; a development that could bear profound implications and shape how we interact with AI systems across many applications.

    Read less
74. Learning in online chess increases with more time spent thinking and diversity of experience

    Lisa Schut, Evan Russek, Ionatan Kuperwajs, Marcelo G Mattar, Wei Ji Ma, and Tom Griffiths Proceedings of the National Academy of Sciences, 2025

    What factors of our learning experiences enable us to best acquire complex skills? Recent ideas from artificial intelligence point to two such factors: (1) a balance of real experience with simulat... Read more

    What factors of our learning experiences enable us to best acquire complex skills? Recent ideas from artificial intelligence point to two such factors: (1) a balance of real experience with simulated experience acquired during planning itself, and (2) appropriate diversity in training examples. To test whether these factors influence the development of human expertise, we analyzed data from 1,873 chess players on the online platform Lichess, each of whom played hundreds to thousands of games over months to years. We found that both the time spent planning before moves and the diversity of opening positions encountered predict skill improvement over time. These findings suggest that principles shaping the development of expertise in artificial intelligence systems may also apply to human learning.

    Read less
75. Human-Aligned Chess AI: A Multitask Transformer for Humanlike Decision-Making

    Easwar Gnana Hari Sekar and Roger Jin In , 2025
76. Stress and Strategic Decision Making

    Benjamin G. Serpell, Blair T. Crewther, Phillip J. Fourie, Stephen P. J. Goodman, and Christian J. Cook In 2025 IEEE Conference on Artificial Intelligence (CAI), 2025

    Psychology and social science research offer some promising work in the field of decision-making science. However, given the qualitative nature of much of this research, understanding some physiolo... Read more

    Psychology and social science research offer some promising work in the field of decision-making science. However, given the qualitative nature of much of this research, understanding some physiological bases of decision-making may assist by providing more objectivity. The purpose of this study, therefore, was to explore hormonal and neurophysiological biomarkers of stress relative to strategic decision making, with and without an accompanying exercise stress.

    Read less
77. Seqret: Mining Rule Sets from Event Sequences

    Aleena Siji, Joscha Cüppers, Osman Ali Mian, and Jilles Vreeken Adaptive Human Behavior and Physiology, 2025
78. A Behavior-Based Knowledge Representation Improves Prediction of Players’ Moves in Chess by 25%

    Benny Skidanov, Daniel Erbesfeld, Gera Weiss, and Achiya Elyasaf , 2025
79. Beyond Accuracy: A Geometric Stability Analysis of Large Language Models in Chess Evaluation

    Xidan Song, Weiqi Wang, Ruifeng Cao, and Qingya Hu , 2025

    The evaluation of Large Language Models (LLMs) in complex reasoning domains typically relies on performance alignment with ground-truth oracles. In the domain of chess, this standard manifests as a... Read more

    The evaluation of Large Language Models (LLMs) in complex reasoning domains typically relies on performance alignment with ground-truth oracles. In the domain of chess, this standard manifests as accuracy benchmarks against strong engines like Stockfish. However, high scalar accuracy does not necessarily imply robust conceptual understanding. This paper argues that standard accuracy metrics fail to distinguish between genuine geometric reasoning and the superficial memorization of canonical board states. To address this gap, we propose a Geometric Stability Framework, a novel evaluation methodology that rigorously tests model consistency under invariant transformations-including board rotation, mirror symmetry, color inversion, and format conversion. We applied this framework to a comparative analysis of six state-of-the-art LLMs including GPT-5.1, Claude Sonnet 4.5, and Kimi K2 Turbo, utilizing a dataset of approximately 3,000 positions. Our results reveal a significant Accuracy-Stability Paradox. While models such as GPT-5.1 achieve near-optimal accuracy on standard positions, they exhibit catastrophic degradation under geometric perturbation, specifically in rotation tasks where error rates surge by over 600%. This disparity suggests a reliance on pattern matching over abstract spatial logic. Conversely, Claude Sonnet 4.5 and Kimi K2 Turbo demonstrate superior dual robustness, maintaining high consistency across all transformation axes. Furthermore, we analyze the trade-off between helpfulness and safety, identifying Gemini 2.5 Flash as the leader in illegal state rejection (96.0%). We conclude that geometric stability provides an orthogonal and essential metric for AI evaluation, offering a necessary proxy for disentangling reasoning capabilities from data contamination and overfitting in large-scale models.

    Read less
80. Towards Piece-by-Piece Explanations for Chess Positions with SHAP

    Francesco Spinnato , 2025

    Contemporary chess engines offer precise yet opaque evaluations, typically expressed as centipawn scores. While effective for decision-making, these outputs obscure the underlying contributions of ... Read more

    Contemporary chess engines offer precise yet opaque evaluations, typically expressed as centipawn scores. While effective for decision-making, these outputs obscure the underlying contributions of individual pieces or patterns. In this paper, we explore adapting SHAP (SHapley Additive exPlanations) to the domain of chess analysis, aiming to attribute a chess engines evaluation to specific pieces on the board. By treating pieces as features and systematically ablating them, we compute additive, per-piece contributions that explain the engines output in a locally faithful and human-interpretable manner. This method draws inspiration from classical chess pedagogy, where players assess positions by mentally removing pieces, and grounds it in modern explainable AI techniques. Our approach opens new possibilities for visualization, human training, and engine comparison. We release accompanying code and data to foster future research in interpretable chess AI.

    Read less
81. Maia-2: a unified model for human-AI alignment in chess

    Zhenwei Tang, Difan Jiao, Reid McIlroy-Young, Jon Kleinberg, Siddhartha Sen, and Ashton Anderson In Proceedings of AI4HGI 2025, the First Workshop on Artificial Intelligence for Human-Game Interaction at the 28th European Conference on Artificial Intelligence (ECAI 2025), 2025

    There are an increasing number of domains in which artificial intelligence (AI) systems both surpass human ability and accurately model human behavior. This introduces the possibility of algorithmi... Read more

    There are an increasing number of domains in which artificial intelligence (AI) systems both surpass human ability and accurately model human behavior. This introduces the possibility of algorithmically-informed teaching in these domains through more relatable AI partners and deeper insights into human decision-making. Critical to achieving this goal, however, is coherently modeling human behavior at various skill levels. Chess is an ideal model system for conducting research into this kind of human-AI alignment, with its rich history as a pivotal testbed for AI research, mature superhuman AI systems like AlphaZero, and precise measurements of skill via chess rating systems. Previous work in modeling human decision-making in chess uses completely independent models to capture human style at different skill levels, meaning they lack coherence in their ability to adapt to the full spectrum of human improvement and are ultimately limited in their effectiveness as AI partners and teaching tools. In this work, we propose a unified modeling approach for human-AI alignment in chess that coherently captures human style across different skill levels and directly captures how people improve. Recognizing the complex, non-linear nature of human learning, we introduce a skill-aware attention mechanism to dynamically integrate players’ strengths with encoded chess positions, enabling our model to be sensitive to evolving player skill. Our experimental results demonstrate that this unified framework significantly enhances the alignment between AI and human players across a diverse range of expertise levels, paving the way for deeper insights into human decision-making and AI-guided teaching tools. Our implementation is available https://github.com/CSSLab/maia2.

    Read less
82. Is Elo Rating Reliable? A Study Under Model Misspecification

    Shange Tang, Yuanhao Wang, and Chi Jin In Proceedings of the 38th International Conference on Neural Information Processing Systems, Vancouver, BC, Canada, 2025
83. SEAM: Semantically Equivalent Across Modalities Benchmark for Vision-Language Models

    Zhenwei Tang, Difan Jiao, Blair Yang, and Ashton Anderson , 2025

    Evaluating whether vision-language models (VLMs) reason consistently across representations is challenging because modality comparisons are typically confounded by task differences and asymmetric i... Read more

    Evaluating whether vision-language models (VLMs) reason consistently across representations is challenging because modality comparisons are typically confounded by task differences and asymmetric information. We introduce SEAM, a benchmark that pairs semantically equivalent inputs across four domains that have existing standardized textual and visual notations. By employing distinct notation systems across modalities, in contrast to OCR-based image-text pairing, SEAM provides a rigorous comparative assessment of the textual-symbolic and visual-spatial reasoning capabilities of VLMs. Across 21 contemporary models, we observe systematic modality imbalance: vision frequently lags language in overall performance, despite the problems containing semantically equivalent information, and cross-modal agreement is relatively low. Our error analysis reveals two main drivers: textual perception failures from tokenization in domain notation and visual perception failures that induce hallucinations. We also show that our results are largely robust to visual transformations. SEAM establishes a controlled, semantically equivalent setting for measuring and improving modality-agnostic reasoning.

    Read less
84. East Meets East: Inside The 2024 World Chess Championship In Singapore

    Olimpiu G Urcan In Second Conference on Language Modeling, 2025

    This book presents a compelling account of the historic FIDE world chess championship match between Ding Liren of China and Gukesh Dommaraju of India, held in Singapore from November 25 to December... Read more

    This book presents a compelling account of the historic FIDE world chess championship match between Ding Liren of China and Gukesh Dommaraju of India, held in Singapore from November 25 to December 13, 2024. Sponsored by Google, the 14-game match marked several significant milestones: the first All-Asian world chess championship, the first to be hosted in Singapore, and the momentous crowning of the youngest undisputed world chess champion in history. Through detailed reports, in-depth analyses of each game, and a curated collection of over a hundred quality photographs, the book explores the players’ remarkable journeys, offering insights into their distinct playing styles, rigorous preparation, and the psychological challenges they faced both on and off the board. Set against the dynamic cultural backdrop of Singapore, it also examines the growing prominence of Asia in the world of chess. Combining strategic depth, expert commentary, and behind-the-scenes perspectives, this work provides an engaging narrative of resilience, rivalry, and excellence, making it an indispensable read for chess enthusiasts and sports aficionados.

    Read less
85. Evaluating In Silico Creativity: An Expert Review of AI Chess Compositions

    Vivek Veeriah, Federico Barbero, Marcus Chiam, Xidong Feng, Michael Dennis, Ryan Pachauri, Thomas Tumiel, Johan Obando-Ceron, Jiaxin Shi, Shaobo Hou, Satinder Singh, Nenad Tomašev, and Tom Zahavy , 2025

    The rapid advancement of Generative AI has raised significant questions regarding its ability to produce creative and novel outputs. Our recent work investigates this question within the domain of ... Read more

    The rapid advancement of Generative AI has raised significant questions regarding its ability to produce creative and novel outputs. Our recent work investigates this question within the domain of chess puzzles and presents an AI system designed to generate puzzles characterized by aesthetic appeal, novelty, counter-intuitive and unique solutions. We briefly discuss our method below and refer the reader to the technical paper for more details. To assess our system’s creativity, we presented a curated booklet of AI-generated puzzles to three world-renowned experts: International Master for chess compositions Amatzia Avni, Grandmaster Jonathan Levitt, and Grandmaster Matthew Sadler. All three are noted authors on chess aesthetics and the evolving role of computers in the game. They were asked to select their favorites and explain what made them appealing, considering qualities such as their creativity, level of challenge, or aesthetic design.

    Read less
86. Explore the Reasoning Capability of LLMs in the Chess Testbed

    Shu Wang, Lei Ji, Renxi Wang, Wenxiao Zhao, Haokun Liu, Yifan Hou, and Ying Nian Wu , 2025

    Reasoning is a central capability of human intelligence. In recent years, with the advent of large-scale datasets, pretrained large language models have emerged with new capabilities, including rea... Read more

    Reasoning is a central capability of human intelligence. In recent years, with the advent of large-scale datasets, pretrained large language models have emerged with new capabilities, including reasoning. However, these models still struggle with long-term, complex reasoning tasks, such as playing chess. Based on the observation that expert chess players employ a dual approach combining long-term strategic play with short-term tactical play along with language explanation, we propose improving the reasoning capability of large language models in chess by integrating annotated strategy and tactic. Specifically, we collect a dataset named MATE, which consists of 1 million chess positions with candidate moves annotated for strategy and tactics. We finetune the LLaMA-3-8B model and compare it against state-of-the-art commercial language models in the task of selecting better chess moves. Our experiments show that our models perform better than GPT, Claude, and Gemini models. We find that language explanations can enhance the reasoning capability of large language models.

    Read less
87. ChessQA: Evaluating Large Language Models for Chess Understanding

    Qianfeng Wen, Zhenwei Tang, and Ashton Anderson In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers), 2025

    Chess provides an ideal testbed for evaluating the reasoning, modeling, and abstraction capabilities of large language models (LLMs), as it has well-defined structure and objective ground truth whi... Read more

    Chess provides an ideal testbed for evaluating the reasoning, modeling, and abstraction capabilities of large language models (LLMs), as it has well-defined structure and objective ground truth while admitting a wide spectrum of skill levels. However, existing evaluations of LLM ability in chess are ad hoc and narrow in scope, making it difficult to accurately measure LLM chess understanding and how it varies with scale, post-training methodologies, or architecture choices. We present ChessQA, a comprehensive benchmark that assesses LLM chess understanding across five task categories (Structural, Motifs, Short Tactics, Position Judgment, and Semantic), which approximately correspond to the ascending abstractions that players master as they accumulate chess knowledge, from understanding basic rules and learning tactical motifs to correctly calculating tactics, evaluating positions, and semantically describing high-level concepts. In this way, ChessQA captures a more comprehensive picture of chess ability and understanding, going significantly beyond the simple move quality evaluations done previously, and offers a controlled, consistent setting for diagnosis and comparison. Furthermore, ChessQA is inherently dynamic, with prompts, answer keys, and construction scripts that can evolve as models improve. Evaluating a range of contemporary LLMs, we find persistent weaknesses across all five categories and provide results and error analyses by category. We will release the code, periodically refreshed datasets, and a public leaderboard to support further research.

    Read less
88. Multi-Source Feature Fusion and Neural Embedding for Predicting Chess Puzzle Difficulty

    Haitao Xiao, Daiyuan Yu, Xuegang Wen, Le Chen, and Kun Fu , 2025

    Estimating the difficulty of chess puzzles provides a rich testbed for studying human–computer interaction and adaptive learning. Building on recent advances and the FedCSIS 2025 Challenge, we addr... Read more

    Estimating the difficulty of chess puzzles provides a rich testbed for studying human–computer interaction and adaptive learning. Building on recent advances and the FedCSIS 2025 Challenge, we address the task of predicting chess puzzle difficulty ratings using a multi-source representation approach. Our approach integrates pre-trained neural embeddings of board states, solution move sequences, and engine-derived success probabilities. These heterogeneous features are fused via dedicated embedding and projection layers, followed by a multi-layer perceptron regressor. Post-processing calibration and model ensemble further enhance robustness and generalization. Experiments on the FedCSIS 2025 dataset demonstrate that our method effectively leverages both structural and empirical information, achieving strong predictive performance. Our approach achieved fifth place on the final official leaderboard, highlighting the effectiveness of combining neural representations with domain-specific probabilistic features for robust chess puzzle difficulty prediction.

    Read less
89. Mixture of Experts Made Intrinsically Interpretable

    Xingyi Yang, Constantin Venhoff, Ashkan Khakzar, Christian Schroeder Witt, Puneet K. Dokania, Adel Bibi, and Philip Torr In Proceedings of the 20th Conference on Computer Science and Intelligence Systems (FedCSIS), 2025
90. Implicit Search via Discrete Diffusion: A Study on Chess

    Jiacheng Ye, Zhenyu Wu, Jiahui Gao, Zhiyong Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong , 2025

    In the post-AlphaGo era, there has been a renewed interest in search techniques such as Monte Carlo Tree Search (MCTS), particularly in their application to Large Language Models (LLMs). This renew... Read more

    In the post-AlphaGo era, there has been a renewed interest in search techniques such as Monte Carlo Tree Search (MCTS), particularly in their application to Large Language Models (LLMs). This renewed attention is driven by the recognition that current next-token prediction models often lack the ability for long-term planning. Is it possible to instill search-like abilities within the models to enhance their planning abilities without relying on explicit search? We propose DiffuSearch , a model that does \textitimplicit search by looking into the future world via discrete diffusion modeling. We instantiate DiffuSearch on a classical board game, Chess, where explicit search is known to be essential. Through extensive controlled experiments, we show DiffuSearch outperforms both the searchless and explicit search-enhanced policies. Specifically, DiffuSearch outperforms the one-step policy by 19.2% and the MCTS-enhanced policy by 14% on action accuracy. Furthermore, DiffuSearch demonstrates a notable 30% enhancement in puzzle-solving abilities compared to explicit search-based policies, along with a significant 540 Elo increase in game-playing strength assessment. These results indicate that implicit search via discrete diffusion is a viable alternative to explicit search over a one-step policy. All codes are publicly available at \hrefhttps://github.com/HKUNLP/DiffuSearchhttps://github.com/HKUNLP/DiffuSearch

    Read less
91. Biological Processing Units: Leveraging an Insect Connectome to Pioneer Biofidelic Neural Architectures

    Siyu Yu, Zihan Qin, Tingshan Liu, Beiya Xu, R. Jacob Vogelstein, Jason Brown, and Joshua T. Vogelstein In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025, 2025
92. General search techniques without common knowledge for imperfect-information games, and application to superhuman Fog of War chess

    Brian Hu Zhang and Tuomas Sandholm , 2025

    Since the advent of AI, games have served as progress benchmarks. Meanwhile, imperfect-information variants of chess have existed for over a century, present extreme challenges, and have been the f... Read more

    Since the advent of AI, games have served as progress benchmarks. Meanwhile, imperfect-information variants of chess have existed for over a century, present extreme challenges, and have been the focus of significant AI research. Beyond calculation needed in regular chess, they require reasoning about information gathering, the opponent’s knowledge, signaling, etc. The most popular variant, Fog of War (FoW) chess (aka. dark chess) is a recognized challenge problem in AI after superhuman performance was reached in no-limit Texas hold’em poker. We present Obscuro, the first superhuman AI for FoW chess. It introduces advances to search in imperfect-information games, enabling strong, scalable reasoning. Experiments against the prior state-of-the-art AI and human players – including the world’s best – show that Obscuro is significantly stronger. FoW chess is the largest (by amount of imperfect information) turn-based game in which superhuman performance has been achieved and the largest game in which imperfect-information search has been successfully applied.

    Read less
93. Human-Aligned Chess With a Bit of Search

    Yiming Zhang, Athul Paul Jacob, Vivian Lai, Daniel Fried, and Daphne Ippolito , 2025

    Chess has long been a testbed for AI’s quest to match human intelligence, and in recent years, chess AI systems have surpassed the strongest humans at the game. However, these systems are not human... Read more

    Chess has long been a testbed for AI’s quest to match human intelligence, and in recent years, chess AI systems have surpassed the strongest humans at the game. However, these systems are not human-aligned; they are unable to match the skill levels of all human partners or model human-like behaviors beyond piece movement. In this paper, we introduce Allie, a chess-playing AI designed to bridge the gap between artificial and human intelligence in this classic game. Allie is trained on log sequences of real chess games to model the behaviors of human chess players across the skill spectrum, including non-move behaviors such as pondering times and resignations In offline evaluations, we find that Allie exhibits humanlike behavior: it outperforms the existing state-of-the-art in human chess move prediction and ponders at critical positions. The model learns to reliably assign reward at each game state, which can be used at inference as a reward function in a novel time-adaptive Monte-Carlo tree search (MCTS) procedure, where the amount of search depends on how long humans would think in the same positions. Adaptive search enables remarkable skill calibration; in a large-scale online evaluation against players with ratings from 1000 to 2600 Elo, our adaptive search method leads to a skill gap of only 49 Elo on average, substantially outperforming search-free and standard MCTS baselines. Against grandmaster-level (2500 Elo) opponents, Allie with adaptive search exhibits the strength of a fellow grandmaster, all while learning exclusively from humans.

    Read less
94. Predicting Human Chess Moves: An AI Assisted Analysis of Chess Games Using Skill-group Specific n-gram Language Models

    Daren Zhong, Dingcheng Huang, and Clayton Greenberg In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025, 2025

    Chess, a deterministic game with perfect information, has long served as a benchmark for studying strategic decision-making and artificial intelligence. Traditional chess engines or tools for analy... Read more

    Chess, a deterministic game with perfect information, has long served as a benchmark for studying strategic decision-making and artificial intelligence. Traditional chess engines or tools for analysis primarily focus on calculating optimal moves, often neglecting the variability inherent in human chess playing, particularly across different skill levels. To overcome this limitation, we propose a novel and computationally efficient move prediction framework that approaches chess move prediction as a behavioral analysis task. The framework employs n-gram language models to capture move patterns characteristic of specific player skill levels. By dividing players into seven distinct skill groups, from novice to expert, we trained separate models using data from the open-source chess platform Lichess. The framework dynamically selects the most suitable model for prediction tasks and generates player moves based on preceding sequences. Evaluation on real-world game data demonstrates that the model selector module within the framework can classify skill levels with an accuracy of up to 31.7% when utilizing early game information (16 half-moves). The move prediction framework also shows substantial accuracy improvements, with our Selector Assisted Accuracy being up to 39.1% more accurate than our benchmark accuracy. The computational efficiency of the framework further enhances its suitability for real-time chess analysis.

    Read less
95. FedCSIS 2025 knowledgepit.ai Competition: Predicting Chess Puzzle Difficulty Part 2 & A Step Toward Uncertainty Contests

    Jan Zyśko, Michał Ślęzak, Dominik Ślęzak, and Maciej Świechowski , 2025

    We summarize the results of the FedCSIS 2025 machine learning competition organized on the knowledgepit.ai platform. We recall the competition’s goals corresponding to estimations of the chess puzz... Read more

    We summarize the results of the FedCSIS 2025 machine learning competition organized on the knowledgepit.ai platform. We recall the competition’s goals corresponding to estimations of the chess puzzle difficulty levels, we refer to the winning solutions, and we also compare the scope of this year’s competition (and particularly the data available to competition participants) with its previous edition associated with the IEEE BigData 2024 conference. Finally, we discuss the new functionality of the knowledgepit.ai platform, which enables competition participants to submit additional "uncertainty masks" reflecting their assessment of test cases that are mostly problematic for their machine learning models.

    Read less

2024

1. Unleashing Artificial Cognition: Integrating Multiple AI Systems

   Muntasir Adnan, Buddhi Gamage, Zhiwei Xu, Damith Chandana Herath, and Carlos C. N. Kuhn In Proceedings of the 20th Conference on Computer Science and Intelligence Systems (FedCSIS), 2025

   In this study, we present an innovative fusion of language models and query analysis techniques to unlock cognition in artificial intelligence. The introduced open-source AI system seamlessly integ... Read more

   In this study, we present an innovative fusion of language models and query analysis techniques to unlock cognition in artificial intelligence. The introduced open-source AI system seamlessly integrates a Chess engine with a language model, enabling it to predict moves and provide strategic explanations. Leveraging a vector database to achieve retrievable answer generation, our AI system elucidates its decision-making process, bridging the gap between machine’s computational cognition and human-like understanding. Our choice of Chess as the demonstration environment underscores the versatility of our approach. Beyond Chess, our system holds promise for diverse applications, from medical diagnostics to financial forecasting. Our AI system is available at https://github.com/TheOpenSI/CoSMIC

   Read less
2. Advancing Chess Engine Design with Elo-Integrated Evaluation (EIE)

   Paul-Andrei Aldea, Pranav Bangarbale, and Jerry Li In Australasian Conference on Information Systems, ACIS 2024, Canberra, Australia, December 4-6, 2024, 2024
3. Wasm-R3: Record-Reduce-Replay for Realistic and Standalone WebAssembly Benchmarks

   Doehyun Baek, Jakob Getz, Yusung Sim, Daniel Lehmann, Ben L. Titzer, Sukyoung Ryu, and Michael Pradel , 2024

   WebAssembly (Wasm for short) brings a new, powerful capability to the web as well as Edge, IoT, and embedded systems. Wasm is a portable, compact binary code format with high performance and robust... Read more

   WebAssembly (Wasm for short) brings a new, powerful capability to the web as well as Edge, IoT, and embedded systems. Wasm is a portable, compact binary code format with high performance and robust sandboxing properties. As Wasm applications grow in size and importance, the complex performance characteristics of diverse Wasm engines demand robust, representative benchmarks for proper tuning. Stopgap benchmark suites, such as PolyBenchC and libsodium, continue to be used in the literature, though they are known to be unrepresentative. Porting of more complex suites remains difficult because Wasm lacks many system APIs and extracting real-world Wasm benchmarks from the web is difficult due to complex host interactions. To address this challenge, we introduce Wasm-R3, the first record and replay technique for Wasm. Wasm-R3 transparently injects instrumentation into Wasm modules to record an execution trace from inside the module, then reduces the execution trace via several optimizations, and finally produces a replay module that is executable standalone without any host environment-on any engine. The benchmarks created by our approach are (i) realistic, because the approach records real-world web applications, (ii) faithful to the original execution, because the replay benchmark includes the unmodified original code, only adding emulation of host interactions, and (iii) standalone, because the replay benchmarks run on any engine. Applying Wasm-R3 to web-based Wasm applications in the wild demonstrates the correctness of our approach as well as the effectiveness of our optimizations, which reduce the recorded traces by 99.53% and the size of the replay benchmark by 9.98%. We release the resulting benchmark suite of 27 applications, called Wasm-R3-Bench, to the community, to inspire a new generation of realistic and standalone Wasm benchmarks.

   Read less
4. Skill vs. Chance Quantification for Popular Card & Board Games

   Tathagata Banerjee, Anushka De, Subhamoy Maitra, and Diganta Mukherjee Proc. ACM Program. Lang., 2024

   This paper presents a data-driven statistical framework to quantify the role of skill in games, addressing the long-standing question of whether success in a game is predominantly driven by skill o... Read more

   This paper presents a data-driven statistical framework to quantify the role of skill in games, addressing the long-standing question of whether success in a game is predominantly driven by skill or chance. We analyze player level data from four popular games Chess, Rummy, Ludo, and Teen Patti, using empirical win statistics across varying levels of experience. By modeling win rate as a function of experience through a regression framework and employing empirical bootstrap resampling, we estimate the degree to which outcomes improve with repeated play. To summarize these dynamics, we propose a flexible skill score that emphasizes learning over initial performance, aligning with practical and regulatory interpretations of skill. Our results reveal a clear ranking, with Chess showing the highest skill component and Teen Patti the lowest, while Rummy and Ludo fall in between. The proposed framework is transparent, reproducible, and adaptable to other game formats and outcome metrics, offering potential applications in legal classification, game design, and player performance analysis.

   Read less
5. Towards Robust and Cooperative Learning Algorithms

   Mark Beliaev CoRR, 2024

   Recent progress in machine learning has been fueled by increasing scale, enabling breakthroughs in domains such as image generation, natural language understanding, and decision-making. While ... Read more

   Recent progress in machine learning has been fueled by increasing scale, enabling breakthroughs in domains such as image generation, natural language understanding, and decision-making. While tremendous improvements have been realized for low-risk applications like chat completion and recommendation, there are fundamental challenges that need to be addressed for high-risk applications like robotics and security. Specifically, this thesis is motivated by the following issues: (1) Collecting large datasets often relies on crowdsourcing, which inevitably introduces heterogeneous and potentially suboptimal data. (2) Applications that require coordination between multiple agents are often coupled with risk, making it challenging to use decentralized learning algorithms. (3) While neural networks have demonstrated tremendous predictive capabilities, they are inherently fragile to adversarial attacks. In this thesis, we present algorithms for combatting these challenges within the domains of Reinforcement Learning and Adversarial Machine Learning. The first contribution introduces two algorithms for imitation learning in suboptimal settings, demonstrating that by modeling the suboptimalities present, we can improve the learning framework. The second contribution proposes a mechanism to encourage cooperation in multi-agent reinforcement learning, demonstrating that by allowing agents to gift part of their reward, we can promote prosocial behavior in a decentralized fashion. The third contribution develops a robust classification algorithm designed for sparse attacks, demonstrating that by extending our theoretical insights from idealized settings, we can combat adversarial attacks in neural network classifiers. We test our proposed algorithms across applications in image-classification, game-playing, and robotics. Together, these contributions aim to advance the deployment of machine learning in real-world scenarios by considering practical problem settings, presenting novel theoretical and algorithmic insights, and validating these findings empirically.

   Read less
6. Solution Path Heuristics for Predicting Difficulty and Enjoyment Ratings of Roguelike Level Segments

   Colan Biemer and Seth Cooper , 2024

   When generating levels, algorithmically evaluating the results is essential. In this paper, we looked at predicting a level’s difficulty and enjoyment. Past work has approached this problem for puz... Read more

   When generating levels, algorithmically evaluating the results is essential. In this paper, we looked at predicting a level’s difficulty and enjoyment. Past work has approached this problem for puzzle games like Sudoku by analyzing the characteristics of the initial level, the solved level, and the process that led to that solution. In this work, we examined a set of heuristics for Roguelike levels and their solutions, and their relationship to subjective player ratings of the levels. We gathered ratings of difficulty and enjoyment of levels in a study with 143 players. We ran an ablation study on the set of heuristics to find the best combination of heuristics for predicting difficulty and enjoyment with a linear regression model, and found solution path-based heursitics performed well. However, these models did not outperform a simple baseline for predicting enjoyment. Jaccard similarity on paths–a method we have not seen used in the field of game AI–was a useful predictor of difficulty. Testing proximity to enemies across a solution path is the only heuristic needed to predict how enjoyable a level will be.

   Read less
7. Estimating the Puzzlingness of Chess Puzzles

   Sebastian Björkqvist In Proceedings of the 19th International Conference on the Foundations of Digital Games, Worcester, MA, USA, 2024

   PDF

   Solving chess puzzles, which require finding a certain sequence of moves to achieve a winning position (such as checkmate or a significant material advantage), is a commonly used method for improvi... Read more

   Solving chess puzzles, which require finding a certain sequence of moves to achieve a winning position (such as checkmate or a significant material advantage), is a commonly used method for improving chess skills, particularly tactical awareness. Estimating the puzzlingness–i.e., the difficulty–of a chess puzzle is challenging and typically involves showing the puzzle to multiple players and measuring their success rate. In this work, we present a method for predicting the difficulty of chess puzzles using only the initial position and the sequence of correct moves. We generate multiple features, including both hand-crafted features and those extracted from chess engines such as Maia, Leela Chess Zero, and Stockfish. We also train a residual neural network to directly predict the Glicko-2 puzzle rating. The output of this neural network, along with the other generated features, serves as input to a gradient boosting decision tree model to predict the final Glicko-2 rating. Our model was applied to the IEEE BigData 2024 Cup competition on Predicting Chess Puzzle Difficulty, where it achieved third place.

   Read less
8. Decoding Chess Mastery: A Mechanistic Analysis of a Chess Language Transformer Model

   Austin L. Davis and Gita Sukthankar In 2024 IEEE International Conference on Big Data (BigData), 2024

   Mechanistic interpretability (MI) studies aim to identify the specific neural pathways that underlie decision-making in neural networks. Here we analyze both the horizontal and vertical information... Read more

   Mechanistic interpretability (MI) studies aim to identify the specific neural pathways that underlie decision-making in neural networks. Here we analyze both the horizontal and vertical information flows of a chess-playing transformer. This paper introduces a new taxonomy of chessboard attention patterns that synchronize to guide move selection. Our findings show that the early layers of the chess transformer correctly identify moves that are highly ranked by the final layer. Experiments conducted on human chess players laid the foundation for much of our current understanding of human problem-solving, cognition, and visual memory. We believe that the study of chess language transformers may be an equally fruitful research area for AGI systems.

   Read less
9. Hidden Pieces: An Analysis of Linear Probes for GPT Representation Edits

   Austin L Davis and Gita Sukthankar In Artificial General Intelligence: 17th International Conference, AGI 2024, Seattle, WA, USA, August 13–16, 2024, Proceedings, SEATTLE, WA, USA, 2024

   PDF

   Probing classifiers are a technique for understanding and modifying the operation of neural networks in which a smaller classifier is trained to use the model’s internal representation to learn a p... Read more

   Probing classifiers are a technique for understanding and modifying the operation of neural networks in which a smaller classifier is trained to use the model’s internal representation to learn a probing task. Similar to a neural electrode array, probing classifiers help both discern and edit the internal representation of a neural network. This paper evaluates the use of probing classifiers to modify the internal hidden state of a chess-playing transformer. The weights of the learned linear classifiers are very informative and can be used to reliably delete pieces from the board showing that the model internally maintains an editable emergent representation of game state.

   Read less
10. Generative Reinforcement Learning with Transformers

    Gregoire Deletang, Anian Ruoss, Li Kevin Wenliang, Elliot Catt, Tim Genewein, Jordi Grau-Moya, Marcus Hutter, and Joel Veness In 2024 International Conference on Machine Learning and Applications (ICMLA), 2024

    PDF

    In reinforcement learning, Transformers have been shown to be powerful models for multi-task policy distillation and, to a lesser extent, policy improvement via return interventions within framewor... Read more

    In reinforcement learning, Transformers have been shown to be powerful models for multi-task policy distillation and, to a lesser extent, policy improvement via return interventions within frameworks such as Decision Transformers. These recent results are somewhat atypical for reinforcement learning, as they do not rely on the learning of a value function, which is usually at the heart of most traditional approaches. In this paper, we explore a principled approach to purely generative value function approximation with Transformers, opening the way for existing techniques to be applied for policy improvement. Importantly, unlike other RL methods, this generative approach allows us to kickstart the learning process by fine-tuning strong pretrained state predictors, such as foundation models, substantially shortening the training time. We showcase the potential of our approach by constructing an action-value function for chess that can play at the level of an expert human and over 400 Elo stronger than direct behavioural cloning.

    Read less
11. Easy2Hard-Bench: Standardized Difficulty Labels for Profiling LLM Performance and Generalization

    Mucong Ding, Chenghao Deng, Jocelyn Choo, Zichu Wu, Aakriti Agrawal, Avi Schwarzschild, Tianyi Zhou, Tom Goldstein, John Langford, A. Anandkumar, and Furong Huang , 2024

    While generalization over tasks from easy to hard is crucial to profile language models (LLMs), the datasets with fine-grained difficulty annotations for each problem across a broad range of comple... Read more

    While generalization over tasks from easy to hard is crucial to profile language models (LLMs), the datasets with fine-grained difficulty annotations for each problem across a broad range of complexity are still blank. Aiming to address this limitation, we present Easy2Hard-Bench, a consistently formatted collection of 6 benchmark datasets spanning various domains, such as mathematics and programming problems, chess puzzles, and reasoning questions. Each problem within these datasets is annotated with numerical difficulty scores. To systematically estimate problem difficulties, we collect abundant performance data on attempts to each problem by humans in the real world or LLMs on the prominent leaderboard. Leveraging the rich performance data, we apply well-established difficulty ranking systems, such as Item Response Theory (IRT) and Glicko-2 models, to uniformly assign numerical difficulty scores to problems. Moreover, datasets in Easy2Hard-Bench distinguish themselves from previous collections by a higher proportion of challenging problems. Through extensive experiments with six state-of-the-art LLMs, we provide a comprehensive analysis of their performance and generalization capabilities across varying levels of difficulty, with the aim of inspiring future research in LLM generalization. The datasets are available at https://huggingface.co/datasets/furonghuang-lab/Easy2Hard-Bench.

    Read less
12. Turing Tests in Chess: An Experiment Revealing the Role of Human Subjectivity

    Yke Bauke Eisma, Robin Koerts, and Joost de Winter In NeurIPS 2024, 2024

    With the growing capabilities of AI, technology is increasingly able to match or even surpass human performance. In the current study, focused on the game of chess, we investigated whether chess pl... Read more

    With the growing capabilities of AI, technology is increasingly able to match or even surpass human performance. In the current study, focused on the game of chess, we investigated whether chess players could distinguish if they were playing against a human or a computer, and how they achieved this. A total of 24 chess players each played eight 5+0 Blitz games from different starting positions. They played against (1) a human, (2) Maia, a neural network-based chess engine trained to play in a human-like manner, (3) Stockfish 16, the best chess engine available, downgraded to play at a lower level, and (4) Stockfish 16 at its maximal level. The opponent’s move time was fixed at 10 seconds. During the game, participants verbalized their thoughts, and after each game, they indicated by means of a questionnaire whether they thought they had played against a human or a machine and if there were particular moves that revealed the nature of the opponent. The results showed that Stockfish at the highest level was usually correctly identified as an engine, while Maia was often incorrectly identified as a human. The moves of the downgraded Stockfish were relatively often labeled as ‘strange’ by the participants. In conclusion, the Turing test, as applied here in a domain where computers can perform superhumanly, is essentially a test of whether the chess computer can devise suboptimal moves that correspond to human moves, and not necessarily a test of computer intelligence.

    Read less
13. Information based explanation methods for deep learning agents—with applications on large open-source chess models

    Patrik Hammersborg and Inga Strümke Computers in Human Behavior Reports, 2024

    With large chess-playing neural network models like AlphaZero contesting the state of the art within the world of computerised chess, two challenges present themselves: the question of how to expla... Read more

    With large chess-playing neural network models like AlphaZero contesting the state of the art within the world of computerised chess, two challenges present themselves: the question of how to explain the domain knowledge internalised by such models, and the problem that such models are not made openly available. This work presents the re-implementation of the concept detection methodology applied to AlphaZero, by using large, open-source chess models with comparable performance. We obtain results similar to those achieved when applying this methodology to AlphaZero, while relying solely on open-source resources. We also present a novel explainable AI (XAI) method, which is guaranteed to highlight exhaustively and exclusively the information used by the explained model. This method generates visual explanations tailored to domains characterised by discrete input spaces, as is the case for chess. Our presented method has the desirable property of controlling the information flow between any input vector and the given model, which in turn provides strict guarantees regarding what information is used by the trained model during inference. We demonstrate the viability of our method by applying it to standard }}8 \backslashtimes 8}}chess, using large open-source chess models.

    Read less
14. Detecting Fair Play Violations in Chess Using Neural Networks

    Maksim Iavich and Zura Kevanishvili Scientific Reports, 2024
15. Evidence of Learned Look-Ahead in a Chess-Playing Neural Network

    Erik Jenner, Shreyas Kapur, Vasil Georgiev, Cameron Allen, Scott Emmons, and Stuart Russell In Proceedings of 29th International Conference Information Society and University Studies, 2024

    PDF

    Do neural networks learn to implement algorithms such as look-ahead or search "in the wild"? Or do they rely purely on collections of simple heuristics? We present evidence of learned look-ahead in... Read more

    Do neural networks learn to implement algorithms such as look-ahead or search "in the wild"? Or do they rely purely on collections of simple heuristics? We present evidence of learned look-ahead in the policy and value network of Leela Chess Zero, the currently strongest deep neural chess engine. We find that Leela internally represents future optimal moves and that these representations are crucial for its final output in certain board states. Concretely, we exploit the fact that Leela is a transformer that treats every chessboard square like a token in language models, and give three lines of evidence: (1) activations on certain squares of future moves are unusually important causally; (2) we find attention heads that move important information "forward and backward in time," e.g., from squares of future moves to squares of earlier ones; and (3) we train a simple probe that can predict the optimal move 2 turns ahead with 92% accuracy (in board states where Leela finds a single best line). These findings are clear evidence of learned look-ahead in neural networks and might be a step towards a better understanding of their capabilities.

    Read less
16. Personalized recommendation of chess puzzles

    Aryan Karn, Chinmay Anil Biradar, Aryan Puranik, Attili Krishna Kireeti, and R Jayashree In Advances in Neural Information Processing Systems, 2024

    In this paper, we address the problem of finding similar chess puzzles to a given query puzzle using a dataset of one million puzzles. We approach this problem through an information retrieval (IR)... Read more

    In this paper, we address the problem of finding similar chess puzzles to a given query puzzle using a dataset of one million puzzles. We approach this problem through an information retrieval (IR) perspective. Chess positions can be compared in mainly two aspects, positional similarity and dynamic similarity. We experimented by comparing chess positions solely based on the string representation. However, to capture static and dynamic features of a position, we create an index of features for the chess puzzles which include static and dynamic information regarding the best move for the given puzzles. The features of the query chess puzzle can be compared to the puzzles in the dataset to find the most similar chess puzzle. To optimize the search, the index features can be stored in a separate database to avoid repetitive calculations of mobility and similarity between FENs. Our approach incorporates algorithms such as cosine similarity, Levenshtein distance, and feature encoding to enhance the accuracy and efficiency of puzzle retrieval.

    Read less
17. Emergent World Models and Latent Variable Estimation in Chess-Playing Language Models

    Adam Karvonen In Computer Science Engineering: Proceedings of the 1st International Conference on Computing and Intelligent Information Systems (ICCIIS 2024), Bangalore, India, 19-20th April, 2024 Volume 1, 2024
18. Measuring Progress in Dictionary Learning for Language Model Interpretability with Board Game Models

    Adam Karvonen, Benjamin Wright, Can Rager, Rico Angell, Jannik Brinkmann, Logan Smith, Claudio Mayrink Verdun, David Bau, and Samuel Marks CoRR, 2024

    Poster Slides

    What latent features are encoded in language model (LM) representations? Recent work on training sparse autoencoders (SAEs) to disentangle interpretable features in LM representations has shown sig... Read more

    What latent features are encoded in language model (LM) representations? Recent work on training sparse autoencoders (SAEs) to disentangle interpretable features in LM representations has shown significant promise. However, evaluating the quality of these SAEs is difficult because we lack a ground-truth collection of interpretable features which we expect good SAEs to identify. We thus propose to measure progress in interpretable dictionary learning by working in the setting of LMs trained on Chess and Othello transcripts. These settings carry natural collections of interpretable features–for example, “there is a knight on F3”–which we leverage into metrics for SAE quality. To guide progress in interpretable dictionary learning, we introduce a new SAE training technique, -annealing, which demonstrates improved performance on our metric.

    Read less
19. Learning from rewards and social information in naturalistic strategic behavior

    Ionatan Kuperwajs, Bas Opheusden, Evan Russek, and Tom Griffiths In Advances in Neural Information Processing Systems, 2024

    Acting intelligently in complex environments poses a challenging learning problem: faced with many different situations and possible actions, how do people learn which action to take in each situat... Read more

    Acting intelligently in complex environments poses a challenging learning problem: faced with many different situations and possible actions, how do people learn which action to take in each situation? While traditional laboratory-based experiments have been used to study specific learning mechanisms, these experiments often employ relatively simple tasks conducted over a short period of time. Thus, it is unclear to what extent these mechanisms are used in the significantly more complex and temporally extended environments people encounter in their everyday lives. To understand the processes by which people learn policies to guide their decisions, we investigate the opening strategies of novice online chess players over their first months of play. We use a large online data set consisting of 2,499,783 games, providing us with the necessary scale to explore learning mechanisms in a complex setting. In particular, we focus on two types of learning: reinforcement learning, or learning from rewards given repeated experiences, and social learning, or learning from the actions of others. We show that players’ choices are modulated by both game outcomes and observing their opponents’ actions, and that they exhibit important hallmarks of adaptive decision-making such as exploration and expertise. Our results provide evidence that people use sophisticated learning algorithms in naturalistic strategic behavior.

    Read less
20. A Few-Shot Embedding Learning Approach for Predicting the Behavior of Individual Chess Players

    Lennart August , 2024

    This study presents a few-shot embedding learning approach to predict the behavior of individual chess players, based on only 100 games. Traditional models have relied on extensive datasets, often ... Read more

    This study presents a few-shot embedding learning approach to predict the behavior of individual chess players, based on only 100 games. Traditional models have relied on extensive datasets, often requiring thousands of games to achieve accurate move predictions. In contrast, our method leverages a limited number of games to generate dense vector representations, or embeddings, that capture a player’s unique style. We trained a neural network to create these embeddings and used them to predict subsequent moves. Our results indicate that the embedding model performs well across various player sets and can accurately identify players even at scale among a great player population, picking out players with 84% accuracy from among 100k candidates. There are indications that including information on the clock situation during the game improves the embedding process, although our findings are inconclusive. Despite these limitations, our approach shows promise in making personalized chess training more accessible and highlights the potential for embedding learning in human-centered AI applications. Future work will aim to refine both the embedding and move prediction models and explore its application in other domains.

    Read less
21. Predicting Chess Puzzle Difficulty with Transformers

    Szymon Miłosz and Paweł Kapusta , 2024
22. Measuring the Digital Welfare of Online Social Systems

    Lillio Mok In IEEE International Conference on Big Data, Big Data 2024, Washington DC, USA, December 15-18, 2024, 2024
23. Predictive Analysis of Chess Player Performance: An Analysis of Factors Influencing Competitive Success Using Machine Learning Techniques

    Amar Mujagić, Adnan Mujagić, and Dželila Mehanović , 2024

    The world of competitive chess has long been a captivating arena for intellectual competition, where human intelligence, strategic thinking, and long-term planning converge. This study delves into ... Read more

    The world of competitive chess has long been a captivating arena for intellectual competition, where human intelligence, strategic thinking, and long-term planning converge. This study delves into the intricate web of factors that influence a chess player’s competitive success through the lens of predictive modeling and machine learning techniques.

    Read less
24. Discovering High-Quality Chess Puzzles Through One Billion Plays with Offline Reinforcement Learning

    Allen Nie, Anirudhan Badrinath, Nicholas Tomlin, Timothy Dai, Carissa Yip, Rose E Wang, Emma Brunskill, and Christopher J Piech In Advanced Technologies, Systems, and Applications IX, 2024
25. An alternative chess rating model based on latent variables

    Patrick O’Rourke , 2024
26. Sparse but Strategic: Quantitative Insights into Chess Middlegame Complexity

    Justin Ong, Merim Bilalić, and Nemanja Vaci , 2024

    PDF

    Introduction: Quantifying signals in large, sparse datasets is challenging, as noise and redundant features often obscure informative patterns. Chess middlegames, with their dynamic complexity and ... Read more

    Introduction: Quantifying signals in large, sparse datasets is challenging, as noise and redundant features often obscure informative patterns. Chess middlegames, with their dynamic complexity and endless possibilities, provide a testbed for exploring such challenges. Building on 12 studies that identified three categories of chess complexity–difficulty, optionality, and rarity–we propose a novel method to quantify chess pattern complexity. Methods: A complexity scoring system was developed, integrating centipawn change, variability, engine search depth, and inverse document frequency of constellations to measure difficulty, optionality, and rarity. From a large dataset of chess games, we extracted 106,810 unique 11-ply constellations that appeared at least 20 times across 2,235,823 games. The chess engine was used to evaluate chess decisions when answering on constellations, and a multinomial logistic regression model was employed to predict move quality. Results: The model demonstrated good predictive power with a weighted F1 score of 0.75 and a ROC AUC of 0.82. It correctly classified over 70% of blunders, mistakes, and optimal moves, although it has less success in differentiating inaccuracies from optimal moves. Discussion: Our approach establishes a flexible framework for quantifying complexity in large, sparse datasets, demonstrating applicability beyond chess. By integrating different metrics, we achieve high accuracy in predicting move quality during chess middlegames. Beyond supporting chess education by identifying constellations tied to specific move types, this methodology could be valuable in contexts where understanding the interplay of dynamic patterns and their outcomes is essential for deriving actionable insights.

    Read less
27. Empirical Evaluation of Concept Probing for Game-Playing Agents

    A\dhalsteinn Pálsson and Yngvi Björnsson Research Square, 2024
28. "Hunt Takes Hare": Theming Games Through Game-Word Vector Translation

    Younès Rabii and Michael Cook In ECAI 2024 - 27th European Conference on Artificial Intelligence, 19-24 October 2024, Santiago de Compostela, Spain - Including 13th Conference on Prestigious Applications of Intelligent Systems (PAIS 2024), 2024

    A game’s theme is an important part of its design – it conveys narrative information, rhetorical messages, helps the player intuit strategies, aids in tutorialisation and more. Thematic elements of... Read more

    A game’s theme is an important part of its design – it conveys narrative information, rhetorical messages, helps the player intuit strategies, aids in tutorialisation and more. Thematic elements of games are notoriously difficult for AI systems to understand and manipulate, however, and often rely on large amounts of hand-written interpretations and knowledge. In this paper we present a technique which connects game embeddings, a recent method for modelling game dynamics from log data, and word embeddings, which models semantic information about language. We explain two different approaches for using game embeddings in this way, and show evidence that game embeddings enhance the linguistic translations of game concepts from one theme to another, opening up exciting new possibilities for reasoning about the thematic elements of games in the future.

    Read less
29. Pairwise Learning to Rank for Chess Puzzle Difficulty Prediction

    Andry Rafaralahy In Proceedings of the 19th International Conference on the Foundations of Digital Games, Worcester, MA, USA, 2024
30. The Temporal Differences in Chess Between ADHD and Neurotypical Individuals

    Benjamin Rosales In IEEE International Conference on Big Data, Big Data 2024, Washington DC, USA, December 15-18, 2024, 2024

    PDF

    This research investigates temporal differences in chess gameplay between ADHD and neurotypical players, analyzing over 9,800 games across various skill levels and time controls. The study reveals ... Read more

    This research investigates temporal differences in chess gameplay between ADHD and neurotypical players, analyzing over 9,800 games across various skill levels and time controls. The study reveals distinct patterns in time management and decision-making, with significant variations observed across different game phases and complexity levels.

    Read less
31. Amortized planning with large-scale transformers: a case study on chess

    Anian Ruoss, Grégoire Delétang, Sourabh Medapati, Jordi Grau-Moya, Li Kevin Wenliang, Elliot Catt, John Reid, Cannada A. Lewis, Joel Veness, and Tim Genewein , 2024

    PDF Poster

    This paper uses chess, a landmark planning problem in AI, to assess transformers’ performance on a planning task where memorization is futile – even at a large scale. To this end, we release ChessB... Read more

    This paper uses chess, a landmark planning problem in AI, to assess transformers’ performance on a planning task where memorization is futile – even at a large scale. To this end, we release ChessBench, a large-scale benchmark dataset of 10 million chess games with legal move and value annotations (15 billion data points) provided by Stockfish 16, the state-of-the-art chess engine. We train transformers with up to 270 million parameters on ChessBench via supervised learning and perform extensive ablations to assess the impact of dataset size, model size, architecture type, and different prediction targets (state-values, action-values, and behavioral cloning). Our largest models learn to predict action-values for novel boards quite accurately, implying highly non-trivial generalization. Despite performing no explicit search, our resulting chess policy solves challenging chess puzzles and achieves a surprisingly strong Lichess blitz Elo of 2895 against humans (grandmaster level). We also compare to Leela Chess Zero and AlphaZero (trained without supervision via self-play) with and without search. We show that, although a remarkably good approximation of Stockfish’s search-based algorithm can be distilled into large-scale transformers via supervised learning, perfect distillation is still beyond reach, thus making ChessBench well-suited for future research.

    Read less
32. Moves Based Prediction of Chess Puzzle Difficulty with Convolutional Neural Networks

    Dymitr Ruta, Ming Liu, and Ling Cen In Proceedings of the 38th International Conference on Neural Information Processing Systems, Vancouver, BC, Canada, 2024
33. VALUED - Vision and Logical Understanding Evaluation Dataset

    Soumadeep Saha, Saptarshi Saha, and Utpal Garain In IEEE International Conference on Big Data, Big Data 2024, Washington DC, USA, December 15-18, 2024, 2024
34. Optimizing language models for chess : the impact of custom notation and Elo-based fine-tuning

    Lars Schmid and Jerome Maag Journal of Data-centric Machine Learning Research, 2024

    This research investigates the potential for large language models to learn to generate valid chess moves solely through pre-training on chess game data. The primary objective of this study is to i... Read more

    This research investigates the potential for large language models to learn to generate valid chess moves solely through pre-training on chess game data. The primary objective of this study is to investigate the impact of custom notation systems and tokenisation methods specifically designed for use with chess games. The study aims to improve the models’ understanding of game states and move sequences by developing and implementing a custom notation system, xLAN+. This notation system incorporates additional information about captures, checks and checkmates in order to increase the performance of the models. Furthermore, the strategic gameplay capabilities of the models are to be enhanced by fine-tuning them on datasets filtered by Elo rating. This approach postulates that filtering games by skill level can help models develop a deeper understanding of strategic gameplay, thereby improving their ability to generate high-quality moves. The research uses two different architectures, Transformer, based on OpenAI’s GPT-2 configuration, and Mamba, a State Space Model (SSM) optimised for long sequence processing. Initial findings indicate that the custom notation xLAN+ significantly improves the models’ ability to generate valid moves and maintain game state accuracy over extended sequences. The comparison between GPT-2 and Mamba reveals that while both architectures can learn chess rules and generate plausible moves, the SSM offers slight advantages in handling long-range dependencies and maintaining game context. This project demonstrates the potential of language models to learn complex tasks like chess through data-driven approaches, paving the way for their application in other strategic and decision-making domains.

    Read less
35. Estimating Chess Puzzle Difficulty Without Past Game Records Using a Human Problem-Solving Inspired Neural Network Architecture

    Anan Schütt, Tobias Huber, and Elisabeth André , 2024
36. Mastering Board Games by External and Internal Planning with Language Models

    John Schultz, Jakub Adamek, Matej Jusup, Marc Lanctot, Michael Kaisers, Sarah Perrin, Daniel Hennes, Jeremy Shar, Cannada Lewis, Anian Ruoss, Tom Zahavy, Petar Veličković, Laurel Prince, Satinder Singh, Eric Malmi, and Nenad Tomašev In IEEE International Conference on Big Data, Big Data 2024, Washington DC, USA, December 15-18, 2024, 2024
37. Bad Crypto: Chessography and Weak Randomness of Chess Games

    Martin Stanek , 2024
38. Massively Parallel Inverse Block-sorting Transforms for bzip2 Decompression on GPUs

    André Weißenberger and Bertil Schmidt , 2024

    Lossless data compression has evolved into an indispensable tool for reducing data transfer times in heterogeneous systems. However, performing decompression on host systems can create performance ... Read more

    Lossless data compression has evolved into an indispensable tool for reducing data transfer times in heterogeneous systems. However, performing decompression on host systems can create performance bottlenecks. Accelerator libraries, such as nvCOMP, address this problem by providing custom GPU-enabled versions of some general-purpose compression methods, including Snappy, ZStandard and gzip. However, popular bzip-like compression schemes, which rely on block-sorting transforms have not yet been integrated since their fine-grained parallelization is challenging. With a focus on decompression, we propose novel techniques for fine-grained parallelization of inverse Burrows-Wheeler transform (iBWT) and inverse Move-to-Front (iMTF) transform on GPUs, which enable efficient processing of bzip2-based archives on CUDA-enabled accelerators for the first time. Consequently, we present the first fully GPU-enabled bzip2 decompression pipeline as a use case for the proposed algorithms. Our experimental results reveal speedups of up to 6.1x over a multicore CPU implementation for iBWT, and throughput rates of up to 2400 MB/s for combined iBWT and iMTF on an A100 GPU. For decompression of bzip2 archives, a throughput of over 11.62 GB/s is achieved on a DGX H100 server. The source code of our parallel decoder implementation is available at https://github.com/weissenberger/bzip2gpu.

    Read less
39. The bread emoji Team’s Submission to the IEEE BigData 2024 Cup: Predicting Chess Puzzle Difficulty Challenge

    Tyler Woodruff, Oleg Filatov, and Marco Cognetta In Proceedings of the 53rd International Conference on Parallel Processing, Gotland, Sweden, 2024
40. Predicting User Perception of Move Brilliance in Chess

    Kamron Zaidi and Michael Guerzhoy In IEEE International Conference on Big Data, Big Data 2024, Washington DC, USA, December 15-18, 2024, 2024

    PDF

    AI research in chess has been primarily focused on producing stronger agents that can maximize the probability of winning. However, there is another aspect to chess that has largely gone unexamined... Read more

    AI research in chess has been primarily focused on producing stronger agents that can maximize the probability of winning. However, there is another aspect to chess that has largely gone unexamined: its aesthetic appeal. Specifically, there exists a category of chess moves called “brilliant” moves. These moves are appreciated and admired by players for their high intellectual aesthetics. We demonstrate the first system for classifying chess moves as brilliant. The system uses a neural network, using the output of a chess engine as well as features that describe the shape of the game tree. The system achieves an accuracy of 79% (with 50% base-rate), a PPV of 83%, and an NPV of 75%. We demonstrate that what humans perceive as "brilliant" moves is not merely the best possible move. We show that a move is more likely to be predicted as brilliant, all things being equal, if a weaker engine considers it lower-quality (for the same rating by a stronger engine). Our system opens the avenues for computer chess engines to (appear to) display human-like brilliance, and, hence, creativity.

    Read less
41. Transcendence: Generative Models Can Outperform The Experts That Train Them

    Edwin Zhang, Vincent Zhu, Naomi Saphra, Anat Kleiman, Benjamin L. Edelman, Milind Tambe, Sham M. Kakade, and Eran Malach In Proceedings of the 15th International Conference on Computational Creativity, 2024

    Generative models are trained with the simple objective of imitating the conditional probability distribution induced by the data they are trained on. Therefore, when trained on data generated by h... Read more

    Generative models are trained with the simple objective of imitating the conditional probability distribution induced by the data they are trained on. Therefore, when trained on data generated by humans, we may not expect the artificial model to outperform the humans on their original objectives. In this work, we study the phenomenon of transcendence: when a generative model achieves capabilities that surpass the abilities of the experts generating its data. We demonstrate transcendence by training an autoregressive transformer to play chess from game transcripts, and show that the trained model can sometimes achieve better performance than all players in the dataset. We theoretically prove that transcendence can be enabled by low-temperature sampling, and rigorously assess this claim experimentally. Finally, we discuss other sources of transcendence, laying the groundwork for future investigation of this phenomenon in a broader setting.

    Read less
42. IEEE Big Data Cup 2024 Report: Predicting Chess Puzzle Difficulty at KnowledgePit.ai

    Jan Zyśko, Maciej Świechowski, Sebastian Stawicki, Katarzyna Jagieła, Andrzej Janusz, and Dominik Ślęzak In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024, 2024

    We summarize the results of the IEEE BigData 2024 Cup: Predicting Chess Puzzle Difficulty – a data science competition organized at the knowledgepit.ai platform in association with the IEEE BigData... Read more

    We summarize the results of the IEEE BigData 2024 Cup: Predicting Chess Puzzle Difficulty – a data science competition organized at the knowledgepit.ai platform in association with the IEEE BigData 2024 conference. We describe the competition goal and tie it to existing research on human-computer interaction, focusing on task difficulty estimation and aligning human and AI behavior. We explain how we acquired and processed the data, separately for training and testing datasets. We review submitted solutions and evaluate their performance by comparing them to a simple benchmark. We explain how the achieved rating differences translate to user experience when solving chess puzzles. We further explore the concept of chess puzzle difficulty by replicating competition results with puzzle ratings obtained using chess bots only. We conclude with a summary of our findings and directions for future studies, as well as an invitation to the next edition of the competition in 2025.

    Read less

2023

1. Making Superhuman AI More Human in Chess

   Daniel Barrish, Steve Kroon, and Brink Merwe In 2024 IEEE International Conference on Big Data (BigData), 2024

   Computer chess research has traditionally focused on creating the strongest possible chess engine. Recently, however, attempts have been made to create engines that mimic the playing strength and s... Read more

   Computer chess research has traditionally focused on creating the strongest possible chess engine. Recently, however, attempts have been made to create engines that mimic the playing strength and style of human players. Our research proposes enhancements of models developed in this vein that more accurately imitate master-level players, as well as improve the prediction accuracy of existing models on weaker players. Our proposed enhancements are simple to apply by post-processing the output of existing chess engines. The performance of our enhancements was evaluated and compared using two metrics, prediction accuracy and average centipawn loss. We found that using an ensemble model over search depths maximised prediction accuracy, while an evaluation window filtering approach was preferable with respect to average centipawn loss.

   Read less
2. Can artificial intelligence identify creativity?: An empirical study

   William Bart In Advances in Computer Games - 18th International Conference, ACG 2023, Virtual Event, November 28-30, 2023, Revised Selected Papers, 2023

   This article reports an investigation of the extent to which a chess program with an artificial intelligence component (i.e., Stockfish with NNUE) can identify 10 chess moves that are recognized as... Read more

   This article reports an investigation of the extent to which a chess program with an artificial intelligence component (i.e., Stockfish with NNUE) can identify 10 chess moves that are recognized as outstanding chess moves. Stockfish with NNUE was able to identify seven of the ten moves. Although Stockfish with NNUE is a very powerful chess program, it has some limitations in identifying creative chess moves. There is a discussion of those limitations.

   Read less
3. On the Limitations of the Elo, Real-World Games are Transitive, not Additive

   Quentin Bertrand, Wojciech Marian Czarnecki, and Gauthier Gidel Journal of Creativity, 2023

   PDF

   The Elo score has been extensively used to rank players by their skill or strength in competitive games such as chess, go, or StarCraft II. The Elo score implicitly assumes games have a strong addi... Read more

   The Elo score has been extensively used to rank players by their skill or strength in competitive games such as chess, go, or StarCraft II. The Elo score implicitly assumes games have a strong additive–hence transitive–component. In this paper, we investigate the challenge of identifying transitive components in games. As a starting point, we show that the Elo score provably fails to extract the transitive component of some elementary transitive games. Based on this observation, we propose an alternative ranking system which properly extracts the transitive components in these games. Finally, we conduct an in-depth empirical validation on real-world game payoff matrices: it shows significant prediction performance improvements compared to the Elo score.

   Read less
4. Quantifying human performance in chess

   Sandeep Chowdhary, Iacopo Iacopini, and Federico Battiston In International Conference on Artificial Intelligence and Statistics, 25-27 April 2023, Palau de Congressos, Valencia, Spain, 2023

   From sports to science, the recent availability of large-scale data has allowed to gain insights on the drivers of human innovation and success in a variety of domains. Here we quantify human perfo... Read more

   From sports to science, the recent availability of large-scale data has allowed to gain insights on the drivers of human innovation and success in a variety of domains. Here we quantify human performance in the popular game of chess by leveraging a very large dataset comprising of over 120 million games between almost 1 million players. We find that individuals encounter hot streaks of repeated success, longer for beginners than for expert players, and even longer cold streaks of unsatisfying performance. Skilled players can be distinguished from the others based on their gaming behaviour. Differences appear from the very first moves of the game, with experts tending to specialize and repeat the same openings while beginners explore and diversify more. However, experts experience a broader response repertoire, and display a deeper understanding of different variations within the same line. Over time, the opening diversity of a player tends to decrease, hinting at the development of individual playing styles. Nevertheless, we find that players are often not able to recognize their most successful openings. Overall, our work contributes to quantifying human performance in competitive settings, providing a first large-scale quantitative analysis of individual careers in chess, helping unveil the determinants separating elite from beginner performance.

   Read less
5. Quantifying the complexity and similarity of chess openings using online chess community data

   Giordano De Marzo and Vito D. P. Servedio Scientific Reports, 2023

   Chess is a centuries-old game that continues to be widely played worldwide. Opening Theory is one of the pillars of chess and requires years of study to be mastered. In this paper, we use the games... Read more

   Chess is a centuries-old game that continues to be widely played worldwide. Opening Theory is one of the pillars of chess and requires years of study to be mastered. In this paper, we use the games played in an online chess platform to exploit the “wisdom of the crowd” and answer questions traditionally tackled only by chess experts. We first define a relatedness network of chess openings that quantifies how similar two openings are to play. Using this network, we identify communities of nodes corresponding to the most common opening choices and their mutual relationships. Furthermore, we demonstrate how the relatedness network can be used to forecast future openings players will start to play, with back-tested predictions outperforming a random predictor. We then apply the Economic Fitness and Complexity algorithm to measure the difficulty of openings and players’ skill levels. Our study not only provides a new perspective on chess analysis but also opens the possibility of suggesting personalized opening recommendations using complex network theory.

   Read less
6. ChessGPT: Bridging Policy Learning and Language Modeling

   Xidong Feng, Yicheng Luo, Ziyan Wang, Hongrui Tang, Mengyue Yang, Kun Shao, David Mguni, Yali Du, and Jun Wang Scientific Reports, 2023
7. Determining Chess Piece Values Using Machine Learning

   Aditya Gupta, Christopher Grattoni, and Arnav Gupta In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023, 2023

   This paper attempts to generate point values for chess pieces, as alternatives to the commonly accepted chess piece values. We use a database of over a million online chess games to heuristically d... Read more

   This paper attempts to generate point values for chess pieces, as alternatives to the commonly accepted chess piece values. We use a database of over a million online chess games to heuristically determine the value of a chess piece, by using material imbalances to predict game results. We then explore how piece values change when we analyze material imbalances at various stages of a chess game. As further exploration, we determine what practical values chess pieces and imbalances have at various rating ranges. This creates practical data that players of varying rating can use to aid in chess calculation, as opposed to the rigid values that are typically accepted.</p>

   Read less
8. Building a Natural Language Chess Engine with Pretraining and Instruction Fine-Tuning

   Bowen Jiang Journal of Student Research, 2023

   Although pretrained large language models (LLMs) can generate convincing natural language about games like chess, they lack positional and contextual knowledge and as such are poor game-playing age... Read more

   Although pretrained large language models (LLMs) can generate convincing natural language about games like chess, they lack positional and contextual knowledge and as such are poor game-playing agents. In this project, I utilize language pretaining; instruction fine-tuning, an additional training regimen with chess-specific tasks presented in natural language; and chain-of-thought prompting, a natural language description of problem reasoning prepended to the answer of a problem, to improve the performance of LLMs at chess move generation (validity/legality and quality of moves). I show that fine-tuned GPT-2-XL, a 1.5B parameter LLM, performs favorably well at move generation compared to ChatGPT with few-shot learning; I also validate the additional benefits of chain-of-thought prompting compared to plain prompts in ChatGPT while highlighting tradeoffs between the quality of natural language and the quality of chess when more verbose prompts are used in the smaller GPT-2-XL.

   Read less
9. Merging Neural Networks with Traditional Evaluations in Crazyhouse

   Anei Makovec, Johanna Pirker, and Matej Guid , 2023

   In the intricate landscape of game-playing algorithms, Crazyhouse stands as a complex variant of chess where captured pieces are reintroduced, presenting unique evaluation challenges. This paper ex... Read more

   In the intricate landscape of game-playing algorithms, Crazyhouse stands as a complex variant of chess where captured pieces are reintroduced, presenting unique evaluation challenges. This paper explores a hybrid approach that combines traditional evaluation functions with neural network-based evaluations, seeking an optimal balance in performance. Through rigorous experimentation, including self-play, matchups against a variant of the renowned program, Go-deep experiments, and score deviations, we present compelling evidence for the effectiveness of a weighted sum of both evaluations. Remarkably, in our experiments, the combination of 75% neural network and 25% traditional evaluation consistently emerged as the most effective choice. Furthermore, we introduce the use of Best-Change rates, which have previously been associated with evaluation quality, in the context of Monte Carlo tree search-based algorithms.

   Read less
10. Improving the Strength of Human-Like Models in Chess

    Saumik Narayanan, Kassa Korley, Chien-Ju Ho, and Siddhartha Sen In Advances in Computer Games: 18th International Conference, ACG 2023, Virtual Event, November 28–30, 2023, Revised Selected Papers, Siegen, Germany, 2023

    PDF

    Designing AI systems that capture human-like behavior has attracted growing attention in applications where humans may want to learn from, or need to collaborate with, these AI systems. Many existi... Read more

    Designing AI systems that capture human-like behavior has attracted growing attention in applications where humans may want to learn from, or need to collaborate with, these AI systems. Many existing works in designing human-like AI have taken a supervised learning approach that learns from data of human behavior, with the goal of creating models that can accurately predict human behavior. While this approach has shown success in capturing human behavior at different skill levels and even identifying individual behavioral styles, it also suffers from the drawback of mimicking human mistakes. Moreover, existing models only capture a snapshot of human behavior, leaving the question of how to improve them—e.g., from one human skill level to a stronger one—largely unanswered. Using chess as an experimental domain, we investigate the question of teaching an existing human-like model to be stronger using a data-efficient curriculum, while maintaining the model’s human similarity. To achieve this goal, we extend the concept of curriculum learning to settings with multiple labeling strategies, allowing us to vary both the curriculum (dataset) and the teacher (labeling strategy). We find that the choice of teacher has a strong impact on both playing strength and human similarity; for example, a teacher that is too strong can be less effective at improving playing strength and degrade human similarity more rapidly. We also find that the choice of curriculum can impact these metrics, but to a smaller extent; for example, training on a curriculum of human mistakes provides only a marginal benefit over training on a random curriculum. Finally, we show that our strengthened models achieve human similarity on datasets corresponding to their strengthened level of play, suggesting that our curriculum training methodology is improving them in human-like steps.

    Read less
11. Investigation of the Sicilian Defense: Winning rates and strategic discrimination

    Ziming Song , 2023
12. Can higher social status of competitors cause decision makers to commit more errors?

    Li Qian Tay Interdisciplinary Humanities and Communication Studies, 2023
13. Predicting Chess Player Rating Based on a Single Game

    Tim Tijhuis, Paris Mavromoustakos Blom, and Pieter Spronck In Proceedings of the Annual Meeting of the Cognitive Science Society, 2023
14. A Method for Estimating Online Chess Game Player Ratings with Decision Tree

    Habuki Yamada, Nobuko Kishi, Masato Oguchi, and Miyuki Nakano In IEEE Conference on Games, CoG 2023, Boston, MA, USA, August 21-24, 2023, 2023
15. Chess Piece Image Recognition Using Transfer Learning, Simple Neural Network, and Convolutional Neural Network

    Gabriel Yohanes, Mario Nursalim, Nicholas, and Felix Indra Kurniadi In IEEE International Conference on Big Data and Smart Computing, BigComp 2023, Jeju, Republic of Korea, February 13-16, 2023, 2023
16. Diversifying AI: Towards Creative Chess with AlphaZero

    Tom Zahavy, Vivek Veeriah, Shaobo Hou, Kevin Waugh, Matthew Lai, Edouard Leurent, Nenad Tomasev, Lisa Schut, Demis Hassabis, and Satinder Singh In 2023 4th International Conference on Artificial Intelligence and Data Sciences (AiDAS), 2023

    In recent years, Artificial Intelligence (AI) systems have surpassed human intelligence in a variety of computational tasks. However, AI systems, like humans, make mistakes, have blind spots, hallu... Read more

    In recent years, Artificial Intelligence (AI) systems have surpassed human intelligence in a variety of computational tasks. However, AI systems, like humans, make mistakes, have blind spots, hallucinate, and struggle to generalize to new situations. This work explores whether AI can benefit from creative decision-making mechanisms when pushed to the limits of its computational rationality. In particular, we investigate whether a team of diverse AI systems can outperform a single AI in challenging tasks by generating more ideas as a group and then selecting the best ones. We study this question in the game of chess, the so-called drosophila of AI. We build on AlphaZero (AZ) and extend it to represent a league of agents via a latent-conditioned architecture, which we call AZ_db. We train AZ_db to generate a wider range of ideas using behavioral diversity techniques and select the most promising ones with sub-additive planning. Our experiments suggest that AZ_db plays chess in diverse ways, solves more puzzles as a group and outperforms a more homogeneous team. Notably, AZ_db solves twice as many challenging puzzles as AZ, including the challenging Penrose positions. When playing chess from different openings, we notice that players in AZ_db specialize in different openings, and that selecting a player for each opening using sub-additive planning results in a 50 Elo improvement over AZ. Our findings suggest that diversity bonuses emerge in teams of AI agents, just as they do in teams of humans and that diversity is a valuable asset in solving computationally hard problems.

    Read less

2022

1. A Practical Algorithm for Chess Unwinnability

   Miguel Ambrona CoRR, 2023

   PDF

   The FIDE Laws of Chess establish that if a player runs out of time during a game, they lose unless there exists no sequence of legal moves that ends in a checkmate by their opponent, in which case ... Read more

   The FIDE Laws of Chess establish that if a player runs out of time during a game, they lose unless there exists no sequence of legal moves that ends in a checkmate by their opponent, in which case the game is drawn. The problem of determining whether or not a given chess position is unwinnable for a certain player has been considered intractable by the community and, consequently, chess servers do not apply the above rule rigorously, thus unfairly classifying many games. We propose, to the best of our knowledge, the first algorithm for chess unwinnability that is sound, complete and efficient for practical use. We also develop a prototype implementation and evaluate it over the entire Lichess Database (containing more than 3 billion games), successfully identifying all unfairly classified games in the database.

   Read less
2. Deep learning for computer chess (part 1)

   Manav Arora In 11th International Conference on Fun with Algorithms, FUN 2022, May 30 to June 3, 2022, Island of Favignana, Sicily, Italy, 2022

   This report encompasses the implementation of two state-of-the-art machine learning algorithms for evaluating chess positions. The first algorithm makes use of artificial neural networks and manual... Read more

   This report encompasses the implementation of two state-of-the-art machine learning algorithms for evaluating chess positions. The first algorithm makes use of artificial neural networks and manual feature representation thus closely following the implementation and architecture of Matthew Lai’s Giraffe. Giraffe learns to play chess largely by self-play and derives its own rules based on the data [1]. Giraffe was implemented as a 7 class classification problem on a dataset of over 10,000 grandmaster level games. Four different implementations of Giraffe were explored covering two different architectures and the effects of regularization on the model performance. The second algorithm implemented goes through an unsupervised learning phase to perform feature extraction followed by a supervised learning phase thus replicating David Eli’s DeepChess. DeepChess evaluates chess positions using a deep neural network without any a priori knowledge regarding the rules of chess. DeepChess is implemented as a siamese network of two disjoint deep belief networks connected to each other by fully connected layers [2]. This architecture was implemented as a binary classification problem on the same dataset as Giraffe and also on a larger dataset of LiChess games. Different implementations of DeepChess covering different training methodologies and parameter sets were executed.

   Read less
3. Towards the automatic synthesis of interpretable chess tactics

   Abhijeet Krishnan and Chris Martens , 2022

   PDF Slides

   State-of-the-art reinforcement learning agents are capable of outperforming human experts at games like chess, Go, and StarCraft II. These agents do not simply take advantage of their digital hardw... Read more

   State-of-the-art reinforcement learning agents are capable of outperforming human experts at games like chess, Go, and StarCraft II. These agents do not simply take advantage of their digital hardware in being able to react and calculate faster than humans, but employ better strategies that lead to more victories. Interpreting these strategies would give human players valuable insight into how to improve their play. In this preliminary work, we propose a symbolic sub-policy model for playing chess. Inspired by chess tactics, our model attempts to incorporate domain knowledge to improve interpretability. We adapt patterns learned by an inductive logic programming system called PAL to derive our model. We contribute a divergence metric to evaluate our model against a random baseline, and find a set of tactics that is able to suggest moves of similar playing strength to a human beginner. Finally, we propose a computational evaluation scheme for the model by augmenting an off-the-shelf engine with it.

   Read less
4. Synthesizing interpretable chess tactics from player games

   Abhijeet Krishnan and Chris Martens In Proceedings of the Explainable Agency in Artificial Intelligence Workshop, 36th AAAI Conference on Artificial Intelligence, 2022

   PDF

   Competitive games admit a wide variety of player strategies and emergent, domain-specific concepts that are not obvious from an examination of their rules. Expert agents trained on these games demo... Read more

   Competitive games admit a wide variety of player strategies and emergent, domain-specific concepts that are not obvious from an examination of their rules. Expert agents trained on these games demonstrate many useful strategies, but these are difficult for human players to understand and adopt. Algorithmically revealing these strategies could help players develop a better model for making decisions that lead to victories. This paper presents a method for the automatic discovery of player-oriented strategies for chess. We present a formal model for chess strategies, inspired by documented chess tactics, that uses first-order logic clauses for representation. Our system uses inductive logic programming to learn human-interpretable strategies for playing chess in the form of our tactic model. Given minimal background knowledge and training data drawn from real games, our system is able to learn tactics that generalize to a large number of positions. We show that these tactics cover a large number of real-world positions and produce moves that outperform a random player.

   Read less
5. Towards Transparent Cheat Detection in Online Chess: An Application of Human and Computer Decision-Making Preferences

   Thijs Laarhoven and Aditya Ponukumati In Proceedings of the Workshop on Artificial Intelligence for Strategy Games (SG) and Esports Analytics (EA), 18th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, 2022

   Online game providers face the challenge of preventing malicious users (cheaters) from breaking the rules and winning games through illegal means. This issue in particular plagues the online c... Read more

   Online game providers face the challenge of preventing malicious users (cheaters) from breaking the rules and winning games through illegal means. This issue in particular plagues the online chess scene, where the strongest algorithms have long surpassed the world’s best players - any cheater can beat the best human players through computer assistance. Moreover, recent developments in AI-based chess engines have opened the door to even more human-like engines, which are increasingly able to mimic legitimate human players. Unfortunately, because major chess websites do not discuss their cheat detection mechanisms publicly, there is limited scientific literature on how to tackle the pervasive problem of cheating in online chess. Certainly, there is no way to validate whether these mechanisms actually work. We take a first step towards formalizing a proper cheat detection framework for online chess by leveraging a large-scale statistical examination of human and computer decision-making tendencies over millions of chess games played online. Although cheaters are not engines (computer players) but centaurs (computer-assisted human players), the insights into computer play serve as a useful guideline for finding the strongest indicators of cheating. We then demonstrate how these findings may distinguish legitimate human players from cheaters in an automated, rules-based manner. Additionally, we argue that the status quo of hiding cheat detection mechanisms from the public eye is dangerous to the integrity of the game, and that cheat detection is foremost a service to society instead of a competitive advantage for chess websites to attract more users. Consistent with Kerckhoffs’ paradigm, we believe that the benefits of an open discussion on cheat detection far outweigh the potential drawbacks of cheaters learning about these methods.

   Read less
6. Gambits: Theory and evidence

   Shiva Maharaj, Nick Polson, and Christian Turk In Computers and Games - International Conference, CG 2022, Virtual Event, November 22-24, 2022, Revised Selected Papers, 2022

   Abstract Gambits are central to human decision-making. Our goal is to provide a theory of Gambits. A Gambit is a combination of psychological and technical factors designed to disrupt predictable p... Read more

   Abstract Gambits are central to human decision-making. Our goal is to provide a theory of Gambits. A Gambit is a combination of psychological and technical factors designed to disrupt predictable play. Chess provides an environment to study gambits and behavioral game theory. Our theory is based on the Bellman optimality path for sequential decision-making. This allows us to calculate the Q\ Q \\-values of a Gambit where material (usually a pawn) is sacrificed for dynamic play. On the empirical side, we study the effectiveness of a number of popular chess Gambits. This is a natural setting as chess Gambits require a sequential assessment of a set of moves (a.k.a. policy) after the Gambit has been accepted. Our analysis uses Stockfish 14.1 to calculate the optimal Bellman Q\ Q \\-values, which fundamentally measures if a position is winning or losing. To test whether Bellman’s equation holds in play, we estimate the transition probabilities to the next board state via a database of expert human play. This then allows us to test whether the Gambiteer is following the optimal path in his decision-making. Our methodology is applied to the popular Stafford and reverse Stafford (a.k.a. Boden–Kieretsky–Morphy) Gambit and other common ones including the Smith-Morra, Goring, Danish and Halloween Gambits. We build on research in human decision-making by proving an irrational skewness preference within agents in chess. We conclude with directions for future research.

   Read less
7. Learning Models of Individual Behavior in Chess

   Reid McIlroy-Young, Russell Wang, Siddhartha Sen, Jon M. Kleinberg, and Ashton Anderson Applied Stochastic Models in Business and Industry, 2022

   Video

   AI systems that can capture human-like behavior are becoming increasingly useful in situations where humans may want to learn from these systems, collaborate with them, or engage with them as partn... Read more

   AI systems that can capture human-like behavior are becoming increasingly useful in situations where humans may want to learn from these systems, collaborate with them, or engage with them as partners for an extended duration. In order to develop human-oriented AI systems, the problem of predicting human actions—as opposed to predicting optimal actions—has received considerable attention. Existing work has focused on capturing human behavior in an aggregate sense, which potentially limits the benefit any particular individual could gain from interaction with these systems. We extend this line of work by developing highly accurate predictive models of individual human behavior in chess. Chess is a rich domain for exploring human-AI interaction because it combines a unique set of properties: AI systems achieved superhuman performance many years ago, and yet humans still interact with them closely, both as opponents and as preparation tools, and there is an enormous corpus of recorded data on individual player games. Starting with Maia, an open-source version of AlphaZero trained on a population of human players, we demonstrate that we can significantly improve prediction accuracy of a particular player’s moves by applying a series of fine-tuning methods. Furthermore, our personalized models can be used to perform stylometry—predicting who made a given set of moves—indicating that they capture human decision-making at an individual level. Our work demonstrates a way to bring AI systems into better alignment with the behavior of individual people, which could lead to large improvements in human-AI interaction.

   Read less
8. Check Mate: A Sanity Check for Trustworthy AI

   Sascha Mücke and Lukas Pfahler In KDD ’22: The 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Washington, DC, USA, August 14 - 18, 2022, 2022

   Methods of Explainable AI (XAI) try to illuminate the decision making process of complex Machine Learning models by generating explanations. However, for most real-world data there is no “groundtru... Read more

   Methods of Explainable AI (XAI) try to illuminate the decision making process of complex Machine Learning models by generating explanations. However, for most real-world data there is no “groundtruth” explanation, which makes evaluating the correctness of XAI methods and model decisions difficult. Often visual assessment or anecdotal evidence is the only type of evaluation. In this work we propose to use the game of chess as a source of “near ground-truth” (NGT) explanations, which XAI methods can be compared against using various metrics, serving as a “sanity check”. We demonstrate this process in an experiment with a deep convolutional neural network, to which we apply a range of commonly used XAI methods. As our main contribution, we publish our data set of 30 million chess positions along with their NGT explanations for free use in XAI research.

   Read less
9. Time spent thinking in online chess reflects the value of computation

   Evan Russek, Daniel Acosta-Kane, Bas Opheusden, Marcelo Mattar, and Tom Griffiths In Proceedings of the LWDA 2022 Workshops: FGWM, FGKD, and FGDB, Hildesheim (Germany), Oktober 5-7th, 2022, 2022

   Although artificial intelligence systems can now outperform humans in a variety of domains, they still lag behind in the ability to arrive at good solutions to problems using limited resources. Rec... Read more

   Although artificial intelligence systems can now outperform humans in a variety of domains, they still lag behind in the ability to arrive at good solutions to problems using limited resources. Recent proposals have suggested that the key to this cognitive efficiency is intelligent selection of the situations in which computational resources are spent. We tested this hypothesis in the domain of complex planning by analyzing how humans managed time available for thinking in over 12 million online chess matches. We found that players spent more time thinking in board positions where planning was more beneficial. This effect was greater in stronger players, and additionally strengthened by considering only the information available to the player at the time of choice. Finally, we found that the qualitative features of this relationship were consistent with a policy that considers the empirically-measured cost of spending time in chess. This provides evidence that human efficiency is supported by intelligent selection of when to apply computation.

   Read less
10. Complexity and Satisficing: Theory with Evidence from Chess

    Yuval Salant and Jorg L Spenkuch PsyArXiv, 2022

    We develop a satisficing model of choice in which the available alternatives differ in their inherent complexity. We assume–and experimentally validate–that complexity leads to errors in the percep... Read more

    We develop a satisficing model of choice in which the available alternatives differ in their inherent complexity. We assume–and experimentally validate–that complexity leads to errors in the perception of alternatives’ values. The model yields sharp predictions about the effect of complexity on choice probabilities, some of which qualitatively contrast with those of maximization-based choice models. We confirm the predictions of the satisficing model–and thus reject maximization–in a novel data set with information on hundreds of millions of real-world chess moves by highly experienced players. These findings point to the importance of complexity and satisficing for decision making outside of the laboratory.

    Read less
11. Measuring the Non-Transitivity in Chess

    Ricky Sanjaya, Jun Wang, and Yaodong Yang , 2022
12. Dataset Related Experimental Investigation of Chess Position Evaluation Using a Deep Neural Network

    Dawid Wieczerzak and Pawel Czarnul Algorithms, 2022
13. Topological Data Analysis in chess

    Jakub Zelek In Parallel Processing and Applied Mathematics - 14th International Conference, PPAM 2022, Gdansk, Poland, September 11-14, 2022, Revised Selected Papers, Part I, 2022

2021

1. Automatic Recognition of Similar Chess Motifs

   Miha Bizjak and Matej Guid , 2022

   We present a novel method to find chess positions similar to a given query position from a collection of chess games. We consider not only the static similarity resulting from the arrangement of ch... Read more

   We present a novel method to find chess positions similar to a given query position from a collection of chess games. We consider not only the static similarity resulting from the arrangement of chess pieces, but also the dynamic similarity involving the recognition of chess motifs and the tactical, dynamic aspects of position similarity. By encoding chess tactical problems as text documents, we use information retrieval techniques to enable efficient approximate searches. We have also developed a method for automatically generating tactical puzzles from a collection of chess games. We have experimentally shown the importance of including both static and dynamic features for successful recognition of similar chess motifs. The experiments have clearly shown that dynamic similarity plays a very important role in the evaluation of the similarity of chess motifs by both the program and chess experts.

   Read less
2. Improving the Chess Elo System With Process Mining

   Niels Bos In Advances in Computer Games: 17th International Conference, ACG 2021, Virtual Event, November 23–25, 2021, Revised Selected Papers, 2021

   Over the last decade, the amount of data generated by software applications e.g. information systems, websites, mobile applications etc. has increased tremendously. Process mining, a subdiscipline ... Read more

   Over the last decade, the amount of data generated by software applications e.g. information systems, websites, mobile applications etc. has increased tremendously. Process mining, a subdiscipline of data science, uses this data to analyse and improve processes. In this research, the possibilities of process mining on chess event logs are explored, to ultimately improve the chess Elo system. The chess Elo system is a widely used and well accepted rating system. The Elo system is, however, flawed in multiple ways. Two major flaws of the Elo system, are its incapability to review a player?s strength and the excessive time needed to gain the appropriate Elo rating. This research explores the potential of process mining to identify chess expertise. To be more specific, multiple process mining techniques are applied on chess event logs, and the generated process models are analysed to identify chess expertise. This research presents a method to analyse the differences between high and low rated players. This is achieved by comparing process models generated from high and low rated chess games. The results show that by comparing the process models differences between high and low rated players can be observed. Process mining is therefore a promising approach to improve the Elo system and might be applicable to other software too. However, only the first twelve moves of a game were used. To gain more insight into the differences between high and low rated players, the mid and end games should be included in the event logs as well. Future research should be conducted with more chess games added to the event logs to increase the validity.

   Read less
3. Predicting Chess Opening Through Modelling Of Chess Opponents

   Debarpan Bose Chowdhury and Banashree Sen , 2021

   PDF

   A chess opening is the preliminary stage of a chess game which typically consists of moves from formerly analysed openings. Opening strategy plays a crucial role in the entire game and decides the ... Read more

   A chess opening is the preliminary stage of a chess game which typically consists of moves from formerly analysed openings. Opening strategy plays a crucial role in the entire game and decides the destiny of the middlegame and endgame. Here in this article, we attempted to introduce a method to predict the opening moves of a specific opponent. The technique analyses the past games played by a specific player to discover the most probable opening that the player is going to play in the subsequent games. The overall performance of this method is analysed and demonstrated by taking several factors such as transience, average turn and opponent’s response in a given situation into consideration. This comparative study enables us to acquire the knowledge about the opening preferences of an opponent, which gives a strategic advantage to a chess player and helps them to develop a game strategy from the beginning of the game. In this article, we attempted to give an overview of how to predict the most favoured opening of a player and how we can utilise it for our benefit

   Read less
4. A lightweight approach for predicting errors in chess matches

   Giovanni Comarela and Davi Silva Webology (ISSN: 1735-188X), 2021

   PDF

   Chess is becoming more popular and accessible by the day. For ins- tance, online chess enables matches between players from different parts of the world, bringing new ways of learning the game and ... Read more

   Chess is becoming more popular and accessible by the day. For ins- tance, online chess enables matches between players from different parts of the world, bringing new ways of learning the game and interacting with other Web users. With this growth in popularity, there is a possibility to empower amateur players with rich computer analysis and tools, which may assist them in their learning process. One of the ways to analyze chess matches is through the study of errors. In this context, we present a new approach for the task of error pre- diction in chess. Our motivation is that knowing when players are likely to make a mistake is knowing what types of situations lead to difficulties in making the right decision. To that end, we add an abstraction layer to the already studied error prediction task, providing graph-based features to the machine learning models. Our results show a increase in the accuracy of the tested models, im- proving the results obtained in recent studies.

   Read less
5. Improving AlphaZero Using Monte-Carlo Graph Search

   Johannes Czech, Patrick Korus, and Kristian Kersting In Anais do XVIII Encontro Nacional de Inteligência Artificial e Computacional, Evento Online, 2021

   PDF

   The AlphaZero algorithm has been successfully applied in a range of discrete domains, most notably board games. It utilizes a neural network that learns a value and policy function to guide the exp... Read more

   The AlphaZero algorithm has been successfully applied in a range of discrete domains, most notably board games. It utilizes a neural network that learns a value and policy function to guide the exploration in a Monte-Carlo Tree Search. Although many search improvements such as graph search have been proposed for Monte-Carlo Tree Search in the past, most of them refer to an older variant of the Upper Confidence bounds for Trees algorithm that does not use a policy for planning. We improve the search algorithm for AlphaZero by generalizing the search tree to a directed acyclic graph. This enables information flow across different subtrees and greatly reduces memory consumption. Along with Monte-Carlo Graph Search, we propose a number of further extensions, such as the inclusion of Epsilon-Greedy exploration, a revised terminal solver and the integration of domain knowledge as constraints. In our empirical evaluations, we use the CrazyAra engine on chess and crazyhouse as examples to show that these changes bring significant improvements to AlphaZero.

   Read less
6. Who’s on First?: Probing the Learning and Representation Capabilities of Language Models on Deterministic Closed Domains

   David Demeter and Doug Downey In Proceedings of the Thirty-First International Conference on Automated Planning and Scheduling, ICAPS 2021, Guangzhou, China (virtual), August 2-13, 2021, 2021

   The capabilities of today’s natural language processing systems are typically evaluated using large datasets of curated questions and answers. While these are critical benchmarks of progress, they ... Read more

   The capabilities of today’s natural language processing systems are typically evaluated using large datasets of curated questions and answers. While these are critical benchmarks of progress, they also suffer from weakness due to artificial distributions and incomplete knowledge. Artifacts arising from artificial distributions can overstate language model performance, while incomplete knowledge limits fine-grained analysis. In this work, we introduce a complementary benchmarking approach based on SimPlified Language Activity Traces (SPLAT). SPLATs are corpora of language encodings of activity in some closed domain (we study traces from chess and baseball games in this work). SPLAT datasets use naturally-arising distributions, allow the generation of question-answer pairs at scale, and afford complete knowledge in their closed domains. We show that language models of three different architectures can answer questions about world states using only verb-like encodings of activity. Our approach is extensible to new language models and additional question-answering tasks.

   Read less
7. Risk-taking in adversarial games: What can 1 billion online chess games tell us?

   Cameron Holdaway and Ed Vul In Proceedings of the 25th Conference on Computational Natural Language Learning, 2021
8. Classification of Chess Games: An Exploration of Classifiers for Anomaly Detection in Chess

   Masudul Hoque In Proceedings of the 43rd Annual Meeting of the Cognitive Science Society, CogSci 2021, virtual, July 26-29, 2021, 2021
9. Detecting Individual Decision-Making Style: Exploring Behavioral Stylometry in Chess

   Reid McIlroy-Young, Yu Wang, Siddhartha Sen, Jon Kleinberg, and Ashton Anderson , 2021

   Video Poster Slides

   The advent of machine learning models that surpass human decision-making ability in complex domains has initiated a movement towards building AI systems that interact with humans. Many building blo... Read more

   The advent of machine learning models that surpass human decision-making ability in complex domains has initiated a movement towards building AI systems that interact with humans. Many building blocks are essential for this activity, with a central one being the algorithmic characterization of human behavior. While much of the existing work focuses on aggregate human behavior, an important long-range goal is to develop behavioral models that specialize to individual people and can differentiate among them.To formalize this process, we study the problem of behavioral stylometry, in which the task is to identify a decision-maker from their decisions alone. We present a transformer-based approach to behavioral stylometry in the context of chess, where one attempts to identify the player who played a set of games. Our method operates in a few-shot classification framework, and can correctly identify a player from among thousands of candidate players with 98% accuracy given only 100 labeled games. Even when trained on amateur play, our method generalises to out-of-distribution samples of Grandmaster players, despite the dramatic differences between amateur and world-class players. Finally, we consider more broadly what our resulting embeddings reveal about human style in chess, as well as the potential ethical implications of powerful methods for identifying individuals from behavioral data.

   Read less
10. The Complementary Nature of Perceived and Actual Time Spent Online in Measuring Digital Well-being

    Lillio Mok and Ashton Anderson In Advances in Neural Information Processing Systems, 2021

    As online platforms become ubiquitous, there is growing concern that their use can potentially lead to negative outcomes in users’ personal lives, such as disrupted sleep and impacted social relati... Read more

    As online platforms become ubiquitous, there is growing concern that their use can potentially lead to negative outcomes in users’ personal lives, such as disrupted sleep and impacted social relationships. A central question in the literature studying these problematic effects is whether they are associated with the amount of time users spend on online platforms. This is often addressed by either analyzing self-reported measures of time spent online, which are generally inaccurate, or using objective metrics derived from server logs or tracking software. Nonetheless, how the two types of time measures comparatively relate to problematic effects – whether they complement or are redundant with each other in predicting problematicity – remains unknown. Additionally, transparent research into this question is hindered by the literature’s focus on closed platforms with inaccessible data, as well as selective analytical decisions that may lead to reproducibility issues.In this work, we investigate how both self-reported and data-derived metrics of time spent relate to potentially problematic effects arising from the use of an open, non-profit online chess platform. These effects include disruptions to sleep, relationships, school and work performance, and self-control. To this end, we distributed a gamified survey to players and linked their responses with publicly-available game logs. We find problematic effects to be associated with both self-reported and data-derived usage measures to similar degrees. However, analytical models incorporating both self-reported and actual time explain problematic effects significantly more effectively than models with either type of measure alone. Furthermore, these results persist across thousands of possible analytical decisions when using a robust and transparent statistical framework. This suggests that the two methods of measuring time spent measure contain distinct, complementary information about problematic usage outcomes and should be used in conjunction with each other.

    Read less
11. Online Chess Social Networks

    Eva Nolan and Valentin Scognamillo Proc. ACM Hum.-Comput. Interact., 2021

    The goal of this research is to analyze the structure of the network of chess players that play on Lichess.org. We aim to understand the way that Lichess randomizes player pairings and how closely ... Read more

    The goal of this research is to analyze the structure of the network of chess players that play on Lichess.org. We aim to understand the way that Lichess randomizes player pairings and how closely related players are in order to better understand the relationship between ranking and pairing systems. We will also observe the behavior of players with respect to the game types that they play to see if this influences player groupings. Another hope of this project is to do an exploratory analysis of standout players and those that choose their opponents, rather than let the algorithm choose for them.

    Read less
12. Cheat Detection on Online Chess Games using Convolutional and Dense Neural Network

    Reyhan Patria, Sean Favian, Anggoro Caturdewa, and Derwin Suhartono , 2021

    PDF

    With the widespread use of chess engines cheating in chess has become easier than ever, especially in online chess. Cheating obviously brings a negative impact to the sport. However, research on th... Read more

    With the widespread use of chess engines cheating in chess has become easier than ever, especially in online chess. Cheating obviously brings a negative impact to the sport. However, research on the topic on cheat detection in chess is still scarcely found. Thus, this paper will discuss data and algorithms that can be used to develop cheat detection tools to analyze games. For data, there are analyzed data and unanalyzed data from online chess games whereas for the algorithm that will be explored there are convolutional neural network (CNN) and densely connected neural network. The results from the experiment using the CNN algorithm are better than the densely connected neural network for detecting if the player is cheating or not. Meanwhile for the data, using either unanalyzed and analyzed data doesn’t change the best performing neural network, but it was found using the analyzed data still boosts the accuracy of both neural networks.

    Read less
13. Revealing Game Dynamics via Word Embeddings of Gameplay Data

    Younès Rabii and Michael Cook In 2021 4th International Seminar on Research of Information Technology and Intelligent Systems (ISRITI), 2021
14. Can You Learn an Algorithm? Generalizing from Easy to Hard Problems with Recurrent Networks

    Avi Schwarzschild, Eitan Borgnia, Arjun Gupta, Furong Huang, Uzi Vishkin, Micah Goldblum, and Tom Goldstein In Proceedings of the Seventeenth AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, AIIDE 2021, virtual, October 11-15, 2021, 2021
15. Datasets for Studying Generalization from Easy to Hard Examples

    Avi Schwarzschild, Eitan Borgnia, Arjun Gupta, Arpit Bansal, Zeyad Emam, Furong Huang, Micah Goldblum, and Tom Goldstein In Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, 2021
16. Watching a Language Model Learning Chess

    Andreas Stöckl CoRR, 2021

    We analyse how a transformer-based language model learns the rules of chess from text data of recorded games. We show how it is possible to investigate how the model capacity and the available numb... Read more

    We analyse how a transformer-based language model learns the rules of chess from text data of recorded games. We show how it is possible to investigate how the model capacity and the available number of training data influence the learning success of a language model with the help of chess-specific metrics. With these metrics, we show that more games used for training in the studied range offers significantly better results for the same training time. However, model size does not show such a clear influence. It is also interesting to observe that the usual evaluation metrics for language models, predictive accuracy and perplexity, give no indication of this here. Further examination of trained models reveals how they store information about board state in the activations of neuron groups, and how the overall sequence of previous moves influences the newly-generated moves.

    Read less

2020

1. Learning to Play the Chess Variant Crazyhouse Above World Champion Level With Deep Neural Networks and Human Data

   Johannes Czech, Moritz Willig, Alena Beyer, Kristian Kersting, and Johannes Fürnkranz In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), 2021

   PDF

   Deep neural networks have been successfully applied in learning the board games Go, chess, and shogi without prior knowledge by making use of reinforcement learning. Although starting from zero kno... Read more

   Deep neural networks have been successfully applied in learning the board games Go, chess, and shogi without prior knowledge by making use of reinforcement learning. Although starting from zero knowledge has been shown to yield impressive results, it is associated with high computationally costs especially for complex games. With this paper, we present CrazyAra which is a neural network based engine solely trained in supervised manner for the chess variant crazyhouse. Crazyhouse is a game with a higher branching factor than chess and there is only limited data of lower quality available compared to AlphaGo. Therefore, we focus on improving efficiency in multiple aspects while relying on low computational resources. These improvements include modifications in the neural network design and training configuration, the introduction of a data normalization step and a more sample efficient Monte-Carlo tree search which has a lower chance to blunder. After training on 569537 human games for 1.5 days we achieve a move prediction accuracy of 60.4%. During development, versions of CrazyAra played professional human players. Most notably, CrazyAra achieved a four to one win over 2017 crazyhouse world champion Justin Tan (aka LM Jann Lee) who is more than 400 Elo higher rated compared to the average player in our training set. Furthermore, we test the playing strength of CrazyAra on CPU against all participants of the second Crazyhouse Computer Championships 2017, winning against twelve of the thirteen participants. Finally, for CrazyAraFish we continue training our model on generated engine games. In 10 long-time control matches playing Stockfish 10, CrazyAraFish wins three games and draws one out of 10 matches.

   Read less
2. Leveraging rationales to improve human task performance

   Devleena Das and Sonia Chernova Frontiers Artif. Intell., 2020

   Machine learning (ML) systems across many application areas are increasingly demonstrating performance that is beyond that of humans. In response to the proliferation of such models, the field of E... Read more

   Machine learning (ML) systems across many application areas are increasingly demonstrating performance that is beyond that of humans. In response to the proliferation of such models, the field of Explainable AI (XAI) has sought to develop techniques that enhance the transparency and interpretability of machine learning methods. In this work, we consider a question not previously explored within the XAI and ML communities: Given a computational system whose performance exceeds that of its human user, can explainable AI capabilities be leveraged to improve the performance of the human? We study this question in the context of the game of Chess, for which computational game engines that surpass the performance of the average player are widely available. We introduce the Rationale-Generating Algorithm, an automated technique for generating rationales for utility-based computational methods, which we evaluate with a multi-day user study against two baselines. The results show that our approach produces rationales that lead to statistically significant improvement in human task performance, demonstrating that rationales automatically generated from an AI’s internal task model can be used not only to explain what the system is doing, but also to instruct the user and ultimately improve their task performance.

   Read less
3. Aligning Superhuman AI with Human Behavior: Chess as a Model System

   Reid McIlroy-Young, Siddhartha Sen, Jon M. Kleinberg, and Ashton Anderson In Proceedings of the 25th International Conference on Intelligent User Interfaces, Cagliari, Italy, 2020
4. Explain Your Move: Understanding Agent Actions Using Specific and Relevant Feature Attribution

   Nikaash Puri, Sukriti Verma, Piyush Gupta, Dhruv Kayastha, Shripad Deshmukh, Balaji Krishnamurthy, and Sameer Singh In KDD ’20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, CA, USA, August 23-27, 2020, 2020

   PDF

   As deep reinforcement learning (RL) is applied to more tasks, there is a need to visualize and understand the behavior of learned agents. Saliency maps explain agent behavior by highlighting the fe... Read more

   As deep reinforcement learning (RL) is applied to more tasks, there is a need to visualize and understand the behavior of learned agents. Saliency maps explain agent behavior by highlighting the features of the input state that are most relevant for the agent in taking an action. Existing perturbation-based approaches to compute saliency often highlight regions of the input that are not relevant to the action taken by the agent. Our proposed approach, SARFA (Specific and Relevant Feature Attribution), generates more focused saliency maps by balancing two aspects (specificity and relevance) that capture different desiderata of saliency. The first captures the impact of perturbation on the relative expected reward of the action to be explained. The second downweighs irrelevant features that alter the relative expected rewards of actions other than the action to be explained. We compare SARFA with existing approaches on agents trained to play board games (Chess and Go) and Atari games (Breakout, Pong and Space Invaders). We show through illustrative examples (Chess, Atari, Go), human studies (Chess), and automated evaluation methods (Chess) that SARFA generates saliency maps that are more interpretable for humans than existing approaches. For the code release and demo videos, see: https://nikaashpuri.github.io/sarfa-saliency/.

   Read less
5. A Comparative Study of Chess Online Educational Products

   Qian Dong and Rong Miao In International Conference on Learning Representations, 2020

   With the development of technology, more and more online educational products emerge in chess, which makes it difficult for different users to choose from. It’s important to develop methodologies t... Read more

   With the development of technology, more and more online educational products emerge in chess, which makes it difficult for different users to choose from. It’s important to develop methodologies to assist different levels chess players to learn in varies environment. List method and rubric evaluation has been conducted, and advice has been put forward based on this approach. The results show that chess online educational products were rich in content and full featured, which could be divided into four categories: overall ecology, video tutorial, tactical training, live broadcast product. However, products still need to improve in product positioning and user experience to promote the development of chess online education.

   Read less

2019

1. Chess Position Identification using Pieces Classification Based on Synthetic Images Generation and Deep Neural Network Fine-Tuning

   Afonso Sá Delgado Neto and Rafael Mendes Campello In Blended Learning. Education in a Smart Learning Environment: 13th International Conference, ICBL 2020, Bangkok, Thailand, August 24–27, 2020, Proceedings, Bangkok, Thailand, 2020

   Chess pieces recognition using computer vision is a problem generally approached in various ways, with different kinds of results and complexity. Deep learning is a state of the art approach to sol... Read more

   Chess pieces recognition using computer vision is a problem generally approached in various ways, with different kinds of results and complexity. Deep learning is a state of the art approach to solve problems on image recognition although facing necessity of huge data sets. This paper discusses a method to identify synthetically generated chess images on Blender using its Python API via fine-tuning of VGG16 convolutional network obtaining close to 97% accuracy on piece classification. Possible applications include automated record of real chess games and real-time play between online players using real boards.

   Read less
2. Deep learning investigation for chess player attention prediction using eye-tracking and game data

   Justin Le Louedec, Thomas Guntz, James L. Crowley, and Dominique Vaufreydaz In 2019 21st Symposium on Virtual and Augmented Reality (SVR), 2019

   PDF

   This article reports on an investigation of the use of convolutional neural networks to predict the visual attention of chess players. The visual attention model described in this article has been ... Read more

   This article reports on an investigation of the use of convolutional neural networks to predict the visual attention of chess players. The visual attention model described in this article has been created to generate saliency maps that capture hierarchical and spatial features of chessboard, in order to predict the probability fixation for individual pixels Using a skip-layer architecture of an autoencoder, with a unified decoder, we are able to use multiscale features to predict saliency of part of the board at different scales, showing multiple relations between pieces. We have used scan path and fixation data from players engaged in solving chess problems, to compute 6600 saliency maps associated to the corresponding chess piece configurations. This corpus is completed with synthetically generated data from actual games gathered from an online chess platform. Experiments realized using both scan-paths from chess players and the CAT2000 saliency dataset of natural images, highlights several results. Deep features, pretrained on natural images, were found to be helpful in training visual attention prediction for chess. The proposed neural network architecture is able to generate meaningful saliency maps on unseen chess configurations with good scores on standard metrics. This work provides a baseline for future work on visual attention prediction in similar contexts.

   Read less
3. Playing Chess at a Human Desired Level and Style

   Hanan Rosemarin and Ariel Rosenfeld In Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications, ETRA 2019, Denver , CO, USA, June 25-28, 2019, 2019

   Human chess players prefer training with human opponents over chess agents as the latter are distinctively different in level and style than humans. Chess agents designed for human-agent play are c... Read more

   Human chess players prefer training with human opponents over chess agents as the latter are distinctively different in level and style than humans. Chess agents designed for human-agent play are capable of adjusting their level, however their style is not aligned with that of human players. In this paper, we propose a novel approach for designing such agents by integrating the theory of chess players’ decision-making with a state-of-the-art Monte Carlo Tree Search (MCTS) algorithm. We demonstrate the benefits of our approach using two sets of analyses. Quantitatively, we establish that the agents attain their desired Elo ratings. Qualitatively, through a Turing-inspired test with a human chess expert, we show that our agents are indistinguishable from human players.

   Read less
4. Training a Convolutional Neural Network to Evaluate Chess Positions

   Joel Vikström In Proceedings of the 7th International Conference on Human-Agent Interaction, HAI 2019, Kyoto, Japan, October 06-10, 2019, 2019

   Convolutional neural networks are typically applied to image analysis problems. We investigate whether a simple convolutional neural network can be trained to evaluate chess positions by means of p... Read more

   Convolutional neural networks are typically applied to image analysis problems. We investigate whether a simple convolutional neural network can be trained to evaluate chess positions by means of predicting Stockfish (an existing chess engine) evaluations. Publicly available data from lichess.org was used, and we obtained a final MSE of 863.48 and MAE of 12.18 on our test dataset (with labels ranging from -255 to +255). To accomplish better results, we conclude that a more capable model architecture must be used.

   Read less

2018

1. The role of emotion in problem solving: first results from observing chess

   Thomas Guntz, James L. Crowley, Dominique Vaufreydaz, Raffaella Balzarini, and Philippe Dessus , 2019

   PDF

   In this paper we present results from recent experiments that suggest that chess players associate emotions to game situations and reactively use these associations to guide search for planning and... Read more

   In this paper we present results from recent experiments that suggest that chess players associate emotions to game situations and reactively use these associations to guide search for planning and problem solving. We report on a pilot experiment with multi-modal observation of human experts engaged in solving challenging problems in Chess. Our results confirm that cognitive processes have observable correlates in displays of emotion and fixation, and that these displays can be used to evaluate models of cognitive processes. They also revealed an unexpected observation of rapid changes in emotion as players attempt to solve challenging problems. In this paper, we propose a cognitive model to explain our observations, and describe initial results from a second experiment designed to test this model.

   Read less
2. Learning to Evaluate Chess Positions with Deep Neural Networks and Limited Lookahead

   Matthia Sabatelli, Francesco Bidoia, Valeriu Codreanu, and Marco Wiering In Proceedings of the Workshop on Modeling Cognitive Processes from Multimodal Data, Boulder, Colorado, 2018

   In this paper we propose a novel supervised learning approach for training Artificial Neural Networks (ANNs) to evaluate chess positions. The method that we present aims to train different ANN arch... Read more

   In this paper we propose a novel supervised learning approach for training Artificial Neural Networks (ANNs) to evaluate chess positions. The method that we present aims to train different ANN architectures to understand chess positions similarly to how highly rated human players do. We investigate the capabilities that ANNs have when it comes to pattern recognition, an ability that distinguishes chess grandmasters from more amateurplayers. We collect around 3,000,000 different chess positions played by highly skilled chess players and label them with the evaluation function of Stockfish, one of the strongest existing chess engines. We create 4 different datasets from scratch that are used for different classification and regression experiments. The results show how relatively simple Multilayer Perceptrons (MLPs) outperform Convolutional Neural Networks (CNNs) in all the experiments that we have performed. We also investigate two different board representations, the first one representing if a piece is present on the board or not, and the second one in which we assign a numerical value to the piece according to its strength. Our results show how the latter input representation influences the performances of the ANNs negatively in almost all experiments.

   Read less
3. Analysis of Chess Playing Skills on Mathematics Learning Outcomes Junior Athletes Raja Kombi Trenggalek Chess Club

   Andika Yogi Setiawan and Henri Gunawan Pratama In 7th International Conference on Pattern Recognition Applications and Methods, 2018

   This study aims to determine the results of the analysis of chess playing skills on mathematics learning outcomes for junior athletes of the Raja Kombi Trenggalek chess club. The research meth... Read more

   This study aims to determine the results of the analysis of chess playing skills on mathematics learning outcomes for junior athletes of the Raja Kombi Trenggalek chess club. The research method used is a qualitative descriptive method with a quantitative approach. Participants in this study were 8 junior athletes of Raja Kombi Trenggalek chess club. Data collection techniques using interviews, skills results and documentation. The data analysis in this study used the mean and percentage formula. After analyzing the data, the results of this study concluded that in this study, the average score of playing chess skills was 85.00, then the average score of mathematics learning outcomes was 86.25. This is of course the higher the level of achievement of skills or intellectual intelligence, the higher the level of problem solving such as in learning mathematics. This is also influenced by motor and psychological aspects as a support for intelligence skills that affect the thinking of athletes. Then from the data analysis it can be said that the higher the level of achievement of chess playing skills, the higher the level of problem solving as in learning mathematics

   Read less