Chess engine project I'm working on

I made a fork of the GPL licensed Stocfkish chess engine, to explore some ideas I had, drawing inspiration from metaphysical realism, Platonic forms and information theory.




Because of my paradigm and background being so out of the mainstream (philosophy and theology) in typical chess engine development spaces, my project has met opposition, even personal attacks. I was really naive I guess to think that "collaboration" and "new ideas" wouldn't be welcomed in the Open Source space common to those kinds of communities. But human sinfulness tends to infiltrate all institutions, perhaps even at inception. My project has been criticized simply because I only changed a little bit of code in the source, and used a different conceptual framework for training the engine. The goal is to create a more human-aligned engine for chess game analysis, that can be used to produce analysis that elucidates strategic and positional themes, rather than the dominant paradigm in open-source chess engines, of seeing chess as a purely mathematical optimization problem, with the output a series of superhuman forcing tactics alien to how humans actually play chess. And so far the kind of data I we are generating raises deeper questions I think are worth exploring. My hypothesis, that a conceptually pure approach to chess evaluation networks, trained on the equivalent of hundreds of billions of exploratory playouts, should give a clearer strategic vision of chess than merely calculating strong forcing lines, has been confirmed. And this unsettles some people with entrenched interests in certain communities, because the underlying logic is alien to how they understand what a chess engine should be for.

rocknanchor said:

Personally, having stepped away from chess for decades, I still consider myself to approach the subject as a novice. My question, in consideration of your post, nothing in the OP touches on that weighty subject of both human and machine depth of being able to eradicate oneself (or itself) from a tide of the opener's initial plays, namely extensive memory of the openings? Many grandmasters have raised the issue of memory vs thought. What am I missing, or is this some other thought than approaching the traditional pastime?

Click to expand...

Deep study of opening lines isn't that important until a player gets quite advanced. Some people might learn a few trick openings and get good at them (the Fried Liver Attack, etc.), but memorizing lines at the novice or club level rarely helps you improve beyond a certain point.

Based on what I've studied about chess improvement, the most evidence-based approach is having annotated games of chess masters that you can follow along with and study, and just playing alot of chess against humans (preferably). Even at lower levels, like novice levels, humans can play moves that may be suboptimal in terms of theory, but still cause practical problems.

Tactical chess puzzles are of limited usefulness. Engine analysis, IMO, is stil in its infancy in terms of human alignment, and that's what I'm trying to improve upon.

linux.poet said:

I dig it, and if I was going back into chess, I would be using this engine for evaluations of my games.

I run kubuntu Linux and typically use ChessX for running evaluations. Which linux package should I download (ax2 or bmi) in order to get started?

Click to expand...

AVX2 should run on all newer processors that are less than 10-15 years old. BMI2 is better if you have Intel processors, and should work on newer AMD processors as well. The differences in performance are really quite small, especially given that the engine is already really quite efficient.

We are still working on the project: it's still a work in progress. We're still experimenting with further changes to the underlying Stockfish 16 based architecture. We are also looking at ways to modify the open source GUI Lucas Chess to better utilize Theoria for game analysis. It will work with any UCI compatible GUI of course, but we are looking for ways to optimize the output for chess improvement.

Early peak evaluation stability with Theoria occurs with 250,00-500,000 nodes/move, or about 200-300 miliseconds/move or so, depending on how you choose to allocate compute resources. Beyond that point, improvements in analysis are diminishing returns, and there's really no point in using more than 1 million nodes/move, and little point in using more than 500,000 nodes per move for practical purposes.

Our research also shows that Stockfish 16 on its own is a surprisingly good engine for positional chess analysis, only somewhat less efficient than Theoria. Stockfish 17.1 has an entirely different character, though, when it comes to evaluation quality- it reduces chess to a series of mostly tactical, forcing moves. Based on my research with Lc0, I have reason to think this might just reflects limitations on Stockfish 17's approach, and might not reflect "objectivity". Lc0 running on beefier GPU hardware, with larger networks, might well continue to be humanly interpretable: the real limitation is current graphics card memory size, storage capacity, and the time to train a network.

I set up a custom project in Claude to analyze my own games. I gave it a few classic chess strategy books (Aron Nimzowitsch's My System and Jeremy Silman's works) and I showed it a bunch of my own games and asked it to look for patterns that needed improvement. The results were surprising and not what I expected, that's partly what lead to my interest in developing better chess analysis in the first place.