Chess robot that detects moves from a phone camera and drives a LEGO NXT over Bluetooth, no PC in the loop
retained text
The whole loop runs on an Android phone sitting on a stand above the board. It captures a frame after you press the end-of-turn button, compares the board state to the previous one to work out what you played, runs the position through a minimax engine written for the project, and pushes the resulting move to the NXT brick over Bluetooth. Vision and search both run on the phone, and the full rules are handled, castling, en passant and promotion included. The NXT side runs an NXC program that drives the axes and the gripper. Design decisions that mattered: \- comparing a photo before and after the move instead of trying to classify each piece. I only need to know which squares changed, which is a far easier problem than recognising a knight from a bishop \- capture is two moves, not one: take the opponent piece off the board, then play. It sounds obvious but it doubles the number of ways the arm can collide with something \- the gripper tolerates a slightly misplaced piece, better along one axis than the other, but a piece sitting right on the edge of its square makes it crash, and the queen is the least forgiving \- lighting and shadows still fool it on some positions, so the app lets you enter the move manually instead of failing the game I would like to make the detection more robust and the honest bottleneck is data: I only have photos of my own board, in my own lighting.
document fetched 2026-09-26 00:37 · canonical https://reddit.com/r/robotics/comments/1wpy287/chess_robot_that_detects_moves_from_a_phone