Gem Collector

The Gem Collector demo puts an agent on a small grid with a gem somewhere on it. Reach the gem and a new one appears in another spot, so the agent keeps navigating. The genetic algorithm evolves a small neural network to steer. Monsters patrol the grid, and touching one ends the run. Early agents wander off, circle, or walk straight into a monster. After a few generations they head for each gem, route around the blocks and slip past the monsters.

It lives alongside the other live-agent games (Flappy Bird, Snake, the Self-Driving Car, Dino Runner, Breakout, Pong and Frog Crossing) under the same Live Agent demo. Use the tabs at the top of the page to switch between them.

The Problem¶

The grid is 11 by 11 cells and the agent starts near the top-left corner. Single-cell blocks are scattered across it, 12 by default. They are always spaced apart, never touching each other even at a corner, so no cell can ever be boxed in. Every Start lays the blocks out in a fresh random arrangement, which then stays the same for that run.

Two monsters patrol the grid by default. Each one walks back and forth along a straight stretch of its own row or column, from block to block or edge to edge, moving one cell every other step, so a quick agent can slip past. Their paths never cross the starting cell, and they are laid out with the blocks, so they too stay the same for a run. Touching a monster, or swapping places with one, ends the run.

The agent moves one cell up, right, down or left each step. The network reads six inputs: whether each of the four neighbouring cells is blocked, and whether the gem is to the left or right and above or below. The grid edge counts as blocked, and so does the cell the agent just came from, unless going back is the only way out. One hidden layer of eight neurons feeds the four move outputs, and the largest one among the open moves wins, so the agent never walks into a block or the edge. The network has no memory and doesn't know where on the grid it is, so it has to learn a general way of reaching a gem rather than a fixed route. Treating the way back as closed is what stops it from stepping back and forth beside a block, since without memory it can't otherwise tell where it just was.

The agent also senses danger: for each of the four moves, whether it would walk into a monster on the next step. Danger isn't fed into the network. Instead the agent has one more gene, its fear of monsters, and each dangerous move's score drops by that much (times four). A random agent can start out fearless, or even drawn toward monsters, so evolution has to find a healthy fear. When every other open move is dangerous, the way back opens up so the agent can always back away.

Fitness gives a big bonus for each gem, a little for every step that gets closer and a little less for every step that moves away. A run ends when the time is up, when a monster catches the agent, after 80 steps without reaching a gem, or as soon as the agent starts going round in a loop. Each explorer is tested on two runs with different gems, each as long as the two-minute game you watch, so a champion has already shown it keeps collecting for the whole game. Because the network has no memory, coming back to the same cell from the same direction while chasing the same gem means it would repeat that loop forever, so there's no point waiting.

How a Run Looks¶

Click Start. The current best network plays a run live, and a panel in the corner shows the four move scores each step with the chosen one starred, the closed moves crossed out, and moves that walk into a monster marked with !. Fading footprints behind the explorer show the last few cells it walked, so detours around blocks are easy to follow. The chart on the right tracks the best score, which is the number of gems collected, for each cycle.

The gems move to new spots every cycle while the blocks stay put, and each cycle trains and replays on the same gem positions. Reaching a new set of gems each time is how you can tell the agent learned to navigate rather than memorise one path.

Playing It Yourself¶

Switch the mode toggle from Agent to You, or press M, to take over. Move with the arrow keys or the joystick over the game. The agent only moves when you press a direction, and holding a key or the joystick keeps it moving. The joystick appears wherever you touch or click the game, so you can play with your thumb in whichever spot is comfortable, and it stays there for next time. When the game ends, tap the game (or press Space, or Play again) to start a new round straight away. In You mode there is no idle time-out, so you can stop and think as long as you like. Press P to pause or resume and Esc to reset. Switch back to Agent at any time and the evolved network picks up from where you left off.

Settings Worth Trying¶

Open Settings to tune the next run. These only affect Agent mode:

• Number of blocks. From 0 (an open grid) to 25. More blocks means more routing around obstacles.
• Number of monsters. From 0 (no monsters) to 4. The default is 2.
• Population size. More agents per generation means more variety to select from, at the cost of a slower cycle. The default is 30.
• Generations per cycle. How much evolving happens between each live run you watch, from 1 to 15. The default is 2, so you can watch it improve over the cycles: the first runs collect a handful of gems and often end in a loop or in a monster's path, and by the later cycles the best explorer keeps going for the whole two minutes and collects about 110. In our tests with two monsters (24 runs, 10 cycles each), no first-cycle champion lasted the full two minutes, about half of the fifth-cycle champions did, and 62 of the 72 champions of the last three cycles did. Raise it to see a strong agent right away.
• Mutation rate. How much the networks are randomly perturbed each generation. A little keeps the search exploring; too much stops it from settling on a good policy.

What the Achievement Rewards¶

Signed-in users unlock the Gem Collector achievement the first time an evolved agent collects at least five gems in one of the runs you watch in Agent mode. Your own runs in You mode don't count toward it.

Gem Collector records and shares like the other demos. See Sharing and recording for how to capture a clip of a good run.

For more on how the demos catalog works, see What the Demos Do.