Breakout

The Breakout demo is the classic paddle-and-ball game. A paddle slides along the bottom to keep a bouncing ball in play, and the ball smashes a wall of bricks at the top. The genetic algorithm evolves a small neural network to control the paddle. Early on the paddle barely moves and the ball slips past at once. After a few generations the paddle starts shadowing the ball and the rallies stretch out.

It lives alongside Flappy Bird, Snake, the Self-Driving Car, and the Dino Runner under the same Live Agent demo. Use the tabs at the top of the page to switch between them.

The problem¶

The network has one job: keep the ball alive by getting the paddle under it. It reads five inputs, the ball's position and velocity and the paddle's position, and picks one of three moves: left, stay, or right. The largest output wins.

The network never sees the bricks. They break on their own whenever the ball reaches them, so the wall comes down as a side effect of good paddle play, not something the network has to aim at. Keeping the agent's job this simple is what makes it reliable to learn while the screen still fills with breaking bricks.

Fitness rewards how long the ball stays in play, with a bonus for every brick broken and every paddle hit. A miss ends the run. The ball speeds up a little with each paddle hit, so long rallies get gradually harder, and clearing the whole wall drops in a fresh one so a strong policy just keeps going.

How a run looks¶

Click Start. The current best network plays a run live, and a panel in the corner shows the three move scores each step with the chosen one highlighted. The chart on the right tracks the best score for each cycle.

Generation zero leaves the paddle parked and loses the ball almost immediately. Within a few cycles the paddle begins to track the ball across the court, and the rallies grow long enough to bring the whole wall down.

Playing it yourself¶

Switch the mode toggle from Agent to You to take over. Slide the paddle with the left and right arrow keys, or the on-screen left and right buttons on a touch screen. Switch back to Agent at any time and the evolved network picks up from where you left off.

Settings worth trying¶

Open Settings to tune the genetic algorithm for the next run:

• Population size. More candidates per generation means more variety to select from, at the cost of a slower cycle.
• Generations per cycle. How much evolving happens between each live run you watch.
• Mutation rate. How much the networks are randomly perturbed each generation. A little keeps the search exploring; too much stops it from settling on a good paddle.

Breakout records and shares like the other demos. See Sharing and recording for how to capture a clip of a long rally.