How Rovers Use Machine Learning to Navigate Mars and the Moon

9 min readOct 12, 2023

It’s not news to anyone that Elon Musk and SpaceX plan on sending humans to Mars. The Artemis program run by NASA plans on sending people back to the moon as well. Recently, companies and governments alike have been trying to return to space as quickly as possible. Science fiction stories have glorified the idea of colonizing Mars and the moon, and many people want those stories to become a reality.

But how will people get things done on Mars? It’s a treacherous place. Mars lacks a magnetic field and an atmosphere. On Earth, the magnetic field protects us from solar radiation, and the ozone layer filters out much of the dangerous UV light that would harm our skin. Neither of those things are on Mars, so the whole planet is a giant radioactive desert that does everything it possibly can to kill you. Not only that, but the planet experiences many dust storms every year. Every few years there’s a global dust storm that blocks sunlight throughout the planet for days or even weeks.

How Do We Get Stuff Done on Mars? Rovers!

So it’s clear that Mars isn’t the safest place on Earth. It’s not close to Earth, literally and figuratively. Being outside a heavily shielded Martian habitat puts an astronaut at risk of being harmed by solar radiation. Luckily, we already have a solution to this problem. Rovers are already in use on Mars and on the Moon, and will likely be vital in upcoming space missions. The astronauts will need a way to conduct experiments remotely from a safe habitat and also transport goods from one base to another. Rovers would be perfect for this if it weren’t for one thing.

Rovers are slow. Laughably slow. Every day a team at NASA manually gives both rovers instructions for the next day. These instructions normally include movement and choosing which experiments to run. This process is a hassle for engineers at NASA since they could spend their time doing something else.

The Perseverance Rover only usually travels at about 4 centimeters per second. There isn’t anything physically stopping the Perseverance or Curiosity rovers from moving faster than 4 centimeters per second. NASA capped off the speed there to avoid crashing the rover or getting it stuck. Using a rover would be so simple if the rovers could just drive themselves without any human intervention. A human (or AI) could find an area of interest, tell the rover to go there, and the rover would do it. It’s the Mars equivalent of using a GPS to go to the nearest restaurant instead of coming up with the route yourself with a map. The map might be cheaper, but it wastes a lot of time.

AutoNav and eNav: Navigating Mars with Machine Learning

Machine learning is the solution to getting the rover to move by itself — and faster. Machine learning is, in simple terms, teaching a computer how to do a specific task.

But what is AutoNav and eNav exactly? AutoNav is essentially the Tesla autopilot of Mars rovers. It takes images of its surroundings, analyzes hazards around the rover, builds a 2.5D elevation map of the terrain, comes up with a bunch of different potential paths it could take to get past the terrain, then evaluates each path and finally picks the best one. eNav (Enhanced AutoNav) is the system that sorts the giant list of paths and then runs the Approximate Clearance Evaluation (ACE) algorithm to determine how safe each path is.

TL;DR — eNav finds the best path and AutoNav does everything else.

ENav and the ACE Algorithm

Let’s take a look at the whole process of planning out the rover’s movement. First, the rover captures pictures of the area around it using stereo cameras. These images are turned into a 3D point cloud sent to the eNav. The eNav updates a 2.5D height map centered on the rover. The 2.5D height map is a grid of cells with X-Y coordinates, where each cell has a height value. Once the height map has been updated, a machine-learning algorithm analyzes the terrain and produces a cost map. The cost map rates how dangerous each cell would be. Higher values are worse, and lower values are better.

Now that the rover has a cost map, it can start generating paths. In the image above, the paths are represented by the lines or arcs coming out of the rover. The algorithm generates a huge suite of potential paths. The rover can only move in straight lines, arcs, or turn in place. It must stop each time it steers the wheels since it cannot steer and drive. Each path in the algorithm is given a ranking. Paths with fewer steering changes have a lower cost. The actual ranking is the average of many different factors evaluated by the algorithms.

Once the paths are each given a cost and sorted, the ACE (Approximate Clearance Evaluation) evaluates each path in order. It takes factors such as the height of the wheels, the rover’s attitude, suspension angles, clearance from the terrain, and a few others. If these factors break any limits set by NASA engineers, that path is given an infinite cost. An infinite cost means the rover isn’t allowed to go there. If the path does not break any limits, the cost is determined by how close any bound is to the limits. Once the algorithm finds a path with a finite cost, it will continue analyzing other paths until it reaches a certain threshold. It then returns the lowest cost path validated by the ACE algorithm.

Moving the Rover

After the rover finds a valid path, it starts moving. It won’t move much though. The rover only executes the first part of any path. This can be the first meter of an arc or straight line or the first 30 degrees of a turn in place. It then takes more images and restarts the planning cycle. This method of moving the rover is in place to ensure the rover does not break during its long journey. There are no engineers on Mars to fix the rover if anything happens to it, so it has to move as safely as possible to avoid damaging any vital parts.

While the rover is executing one maneuver, it will also be running the planning cycle to find the next maneuver. If the rover can find a new maneuver to make before it stops, it can immediately transition to the next maneuver. If it can’t find a valid path in time, it will stop until it can find a solution. This is called “overthinking” (like what you do on a math test). Overthinking normally happens when there’s a poor ranking of paths, or when there are many ACE evaluations taking a lot of time. It’s pretty bad for the rover, since it wastes time, increases wear and tear on the wheels, and increases the risk of the mission.

ENav as a whole takes up a bunch of computing power too. The old Curiosity rover has to stop and evaluate the terrain every few meters because it can’t make decisions fast enough to continually move. The Perseverance rover is a massive step for space robotics and machine learning since it’s able to make decisions fast enough to drive and use AutoNav at the same time. However, Perseverance still relies on people for much of its movement.

Wait, I Zoned Out… How Does It Work Again?

Rover grabs pictures of the surroundings.
Makes an elevation map with the pictures.
Sends the map to eNav, which creates a cost map.
Potential paths are generated using the cost map.
ACE algorithm gives each path a cost.
The rover performs the first maneuver of the best path.
Repeat until the rover reaches its destination.

Sustainability and Future Space Missions

So the Perseverance and Curiosity rovers use a combination of human commands and AutoNav to traverse the Martian surface, but what about future rovers? Is this method of controlling rovers sustainable? If we want to send people to Mars or the Moon, will we realistically be able to manually give commands to every robot? What about the time it takes them to move? Perseverance and Curiosity are extremely slow, especially during the AutoNav cycle. Missions will be extremely slow if we have to wait for the rovers to transport goods from one colony to another at 4 centimeters per second.

Of course, the answer has been in front of us all this time. AutoNav is a fairly reliable system. Maximizing usage of AutoNav while minimizing human input would relieve many people of controlling the rover. If we want to continue space exploration of Mars and the Moon, we need advanced machine learning systems that can quickly make decisions and perform operations autonomously.

We can take this one step further. The future of rovers is not limited to ML controlling the movements. Machine learning algorithms could theoretically create long mission paths that span thousands of meters with sufficient mapping data. The concept of Google Maps for Mars was suggested at the virtual AI Systems Summit put on by Kisaco Research, where Shreyansh Daftry gave a talk about machine learning systems in rover navigation. Instead of a person coming up with the general route, and the rover following that route, the person could just give the rover a destination, and it would go there.

In addition to these better ML systems, we could make cheaper rovers without all the science instruments specifically for transporting goods and moving things around outside a Martian habitat. The image below (generated by Canva’s AI image generator) shows a rover carrying some crates of goods. A real supply rover probably wouldn't look like this, but it’s a good example of the general concept. Better machine learning could be implemented with lighter, cheaper rovers like this to greatly improve the speed of travel, increasing the speed at which astronauts can complete experiments or missions.

LunarNav: Navigating the Moon

LunarNav is a concept for determining where a rover is on the moon by using images of craters around it. The general concept goes like this: A rover will take stereo images of the area around it. A machine learning algorithm will then identify craters and their sizes. It will finally compare the craters to a database of the craters on the moon to pinpoint its own location, and then come up with a route accordingly. This method works because the moon has been extensively mapped over the last 60 years since the space race.

CREDIT: Shreyansh Daftry, Jet Propulsion Laboratory, California Institute of Technology

Shreyansh Daftry (the Google Maps on Mars guy) is a researcher working at the Jet Propulsion Laboratory on LunarNav. He and his team are currently working on machine learning algorithms that can accurately detect craters on the moon's surface. This includes in shadows, since future lunar missions may go into dark parts of the moon. One such mission is Endurance. The idea is that a rover would be sent to the south pole of the moon. It would then collect samples in the South Pole-Aitken Impact Basin, traveling over 2000 km along the way. For this mission to work, the rover needs to be able to navigate thousands of kilometers of the lunar surface, in light and dark. Shreyansh Daftry and his team at JPL and CalTech are working on LunarNav to supply the Endurance mission with proper navigation systems.

Key Takeaways

Mars and the Moon are really dangerous, so we want rovers to do work for us instead
Rovers move pretty slowly now because people manually control them and safety is a priority
We can speed up rovers with improved machine-learning algorithms that can make decisions for people instead of with people
Future missions to Mars and the Moon, like the Endurance mission, will need the improved algorithms mentioned above.
Cheaper rovers designed for moving goods or construction could be designed to be lighter and move faster.
Machine learning in rovers is SICK!!!

If you liked this article, maybe you should hit the like button and follow me to read more of my stuff. Also, make sure to follow me on Twitter at @AyaanNaha!