MICHAEL RENCHECK
Single Player Foosball
Automating a mini-foosball table to compete against a human opponent. An overhead camera is used to track the position of the ball and then that information is used to control the rods. The camera is also used to detect the approximate angle of a rod by tracking the profile of the players.
The design philosophy behind this project was to create an automated system that was not dependant on special balls, players, or any specific table design. This holds true for both the physical design as well as the software configuration.
The software platform was developed on top of ROS to assist with visualizations and currently uses a Linux device as the main controller. The OpenCV library was used for the computer vision tasks in the project. Each rod uses two stepper motors to control the two degrees of freedom of each rod and each motor is controlled with a stepper controller over USB.
A deeper dive into the software used, all source code, and information for how to replicate this project is hosted on my Github.
The table comes with 3 control schemes:
-
Open Loop position control
-
Feedback rotational control
-
Infinite Spin
The first mode uses the stepper controllers' internal calculation of the position of each motor based on the issued step commands. This method is able to provide a competitive gameplay experience. However, if the ball causes the rod to jam while the motor is moving, this will stall the motor and throw off the controller's internal position count. During gameplay, this was only ever an issue with the rotation motors.
The second mode uses computer vision to detect the angle of the players and correct for any inaccuracies between the actual position and the assumed position. This is able to achieve competitive gameplay, but is still a little experimental and needs some refinement.
The third mode commands the rods to continually spin instead of discretely kicking.
The difficulty of the gameplay can be easily adjusted by:
-
Lowering the quality of the ball detection
-
Lowering the velocity limits of the motors
-
Decreasing the kicking thresholds
Each of these methods is easily configured without having to modify any code and will make the table easier to play against.
Build Process Photos
Future Development
Achieve near-perfect defense. Currently, the ball detection pipeline struggles to detect a ball that is moving at high-speeds. The camera is providing frames at a sufficient rate, but the issue lies in the fidelity of the ball in the image. The classifier favors being able to see the pattern on the ball for reliable detection, so when the image of the ball is blurry, the classifier is not able to consistently see the ball. The other drawback for this table is that training the classifier on images of a blurry, white ball leads the classifier to pick out the white players as the most likely candidate as opposed to the actual ball.
Increase the quality of the player angle feedback. Also, the player angle feedback currently uses background subtraction as the main technique for identifying the contour of a player. This method limits the player's range of motion based on the perspective of the camera and can be affected by the ball and moving reflections. This pipeline might benefit from using additional rod states from using a classification pipeline similar to the ball detection pipeline.
Increase the consistency of the player angle feedback control. The speed of the controller is faster than the processing loop so the kick is stopped in a wide range of positions. This results in varying quality of localizing the rod angle.
Continue to improve the gameplay strategy of the robot by experimenting with reinforcement learning in a sim-to-real application. The software package is ready to be adapted to a PyBullet environment where various discrete action space RL algorithms can be experimented with.