Self-steering scooter project preview

Self-steering scooter

May 2022

Project

Reinforcement learning final project at UT Austin

Built with
  • MuJoCo
  • Python
  • Gym
  • Stable Baselines3

Rental scooters were always lying on the sidewalks around campus. I wondered if one could ride itself back to a charger. For my reinforcement learning final at UT Austin, I built a scooter in MuJoCo and gave the policy control of the handlebars. At 12 mph, with a reward for staying upright, it learned to ride in one big circle.

On the left, the untrained scooter tips over in half a second. On the right, the same task after 72,000 training steps.

I put a target 50 meters away and pointed the scooter in the opposite direction. After days of training it still couldn’t make the U-turn. I started it sideways instead. Once it learned the 90-degree turn, it could learn the U-turn too. Training on just those two turns got it steering toward the target from angles it hadn’t practiced.

Overlaid scooter paths from the origin to a red target 50 meters to the left. One green path veers off to the lower right.
After a final round of training with random starting directions, I ran 200 trials with the same target. The scooter got within 4 meters of it 199 times. The green path at the bottom right is the one that missed.

Then I randomized both the starting point and the destination. The same policy reached a target 50 to 100 meters away in all 200 trials. For the final demo I stopped resetting the simulation. Each time the scooter reached a waypoint, I gave it the next one and let it keep riding.

A continuous path across the simulated floor changes color at each waypoint.
One uninterrupted ride. Each color is the path to the next waypoint.