By Adityaveer Raswan, Staff Software Engineer, Autonomy, Agtonomy --
Over the past nine years, I've built motion planning systems across three autonomous domains. First at Waymo, where I developed the planning algorithms that navigate robotaxis through San Francisco traffic. Then at Waabi, working on autonomous long-haul trucking. Now I lead motion planning at Agtonomy, where we're building the software that lets tractors and sprayers drive themselves through orchards and vineyards.
Every time I changed industries, people asked me the same question: Does any of what you built before actually work here? The short answer is yes, but the interesting answer is where it works and where it doesn't. Equipment manufacturers investing in autonomy need to understand both.
The Core Architecture Transfers Directly
Every autonomous vehicle, whether it's a robotaxi in San Francisco or a tractor in a walnut orchard, runs the same basic loop
- Perceive the surroundings using cameras, lidar, or radar, predict where obstacles and other agents might go next
- Plan a trajectory that avoids collisions while completing the task
- Translate that plan into steering, throttle, and braking commands.
That loop reflects the physics of moving through space safely, and it doesn't change when you swap asphalt for dirt. The on-road AV industry spent 15 years and tens of billions of dollars refining it, and off-road autonomy doesn't need to reinvent it from scratch.
What this means in practice is that vendors coming from the AV world aren't starting over when they enter agriculture or construction. The foundational architecture is proven, and the question is really about how each step gets adapted for a specific operating environment.
Where On-Road Experience Pays Off
Safety engineering. The on-road industry developed rigorous methods for making sure autonomous systems fail safely: operational design domains that define exactly where and when a system can operate, safety envelopes that constrain what a planner is allowed to do regardless of what its algorithms want, and redundant systems so that a single sensor failure doesn't turn into an accident. These frameworks took years and billions of dollars to mature, and they apply directly to off-road equipment. An autonomous sprayer working three feet from farm workers in a vineyard needs the same layered safety thinking that Waymo built for a robotaxi passing three feet from a pedestrian on Market Street.
Planning under uncertainty. Even in a controlled agricultural environment, things go wrong. A row of trees doesn't match the expected spacing. An irrigation line has been moved overnight. A worker steps out from behind a vine row. A branch hangs lower than expected after last night's storm. The on-road AV industry invested heavily in planning algorithms that handle surprise gracefully. The machine doesn't just plan for what it expects to happen. It maintains backup plans, updates its trajectory as new information arrives, and knows when conditions have degraded enough that it should stop and call a remote operator. That discipline of always asking "what if I'm wrong about what I think I see" is one of the most transferable things the AV industry built.
Simulation and testing. Waymo has logged over 200 million fully autonomous miles on public roads, but its vehicles have driven more than 20 billion miles in simulation. Think about that ratio: for every mile on an actual street, the system was stress-tested against roughly a hundred simulated ones. The AV industry built environments where you can replay real scenarios, inject hazards that would be reckless to stage on a public road, and verify that the planning algorithms respond correctly every time. Off-road autonomy needs the same discipline. You can't roll a tractor through every combination of terrain, weather, and obstacle with a human standing nearby to see what happens. But you can simulate degraded GPS in a canyon, a fallen branch blocking a row, a sudden slope change after rain, and verify your system handles them before it ever touches a customer's field.
The simulation toolbox itself is evolving rapidly. Techniques like Gaussian splatting and neural rendering can now reconstruct photorealistic 3D environments from a handful of real sensor passes, giving you a high-fidelity virtual version of an actual orchard or mine site that a planning system can train against. Foundation world models are learning to predict how a scene will change in response to the vehicle's own actions, which means the simulator doesn't just replay what happened, it can generate plausible "what-if" scenarios the vehicle has never seen. The practical implication is that simulation fidelity is no longer the bottleneck it was five years ago. The tools exist to test off-road autonomy at a level of realism that used to be exclusive to companies with billion-dollar budgets.
Where Off-Road Has to Diverge
Not everything from the highway survives the trip to the field. There are fundamental differences worth watching for when evaluating autonomy solutions.
No maps, no GPS, no guarantees. A robotaxi navigates using high-definition maps that encode every lane boundary, traffic signal, stop sign, and curb, and it knows where it is within centimeters by matching what its sensors see against that pre-built world. Agriculture doesn't have any of that. There are no lane markings in an orchard, no traffic signals at the end of a row, no standardized intersection geometry. A freshly pruned vineyard in March looks nothing like the same vineyard in August when the canopy closes overhead. And the localization problem is just as bad: GPS signals degrade under heavy tree canopy, the kind you find in every commercial orchard and vineyard. In canyons, quarries, and near large metallic structures on construction sites, GPS can drop out entirely or jump by several meters without warning. The ground itself shifts. Wheel ruts, mud, standing water, and crop growth reshape the visual landscape constantly.
An on-road system solves localization once during map-building and then just matches against that map forever. An off-road system must solve it continuously, often with degraded inputs. That means relying more on the machine's own sensors to track its movement distance and direction—essentially counting its own steps through the field—and less on any fixed external reference. It means fusing multiple imperfect signals and knowing which ones to trust when they disagree. A localization system that works flawlessly in a flat, open field may fail completely under dense canopy. This is not a commodity feature. It's one of the primary technical differentiators between autonomy providers in off-road environments.
The task matters as much as the driving. A robotaxi has one job: get a passenger from point A to point B safely and comfortably. An agricultural machine must do that while also completing a precise operation. A sprayer needs to cover every row evenly, a mower needs full field coverage without gaps or overlaps, and a harvesting platform needs to position itself relative to a specific tree at a specific angle. The planning system can't just avoid obstacles. It must execute the agricultural operation correctly while avoiding them. This coupling between task planning and motion planning is something the on-road world never had to solve, and it's where much of the real engineering challenge lives in off-road autonomy. Newer approaches like vision-language-action models, which combine visual understanding with natural-language task descriptions to generate machine actions, and learned cost functions, where the planner's sense of "what makes a good trajectory" is trained from expert operator behavior rather than hand-coded by an engineer, are starting to close this gap, and they're worth watching closely.
Different vehicles, different physics. A robotaxi is a passenger car with well-characterized, predictable dynamics. Agricultural and construction equipment is a different universe. A compact orchard tractor handles nothing like a 40-ton articulated haul truck. Skid-steer loaders pivot in place; Ackermann-steered tractors sweep wide arcs. Tracked equipment grips differently from wheeled equipment on the same slope. A machine working a 15-degree grade in wet clay must account for slip and slide forces that simply don't exist on dry pavement. Planning algorithms need to be tuned, and sometimes fundamentally redesigned, for each machine type and the conditions in which it operates. Manufacturers should be skeptical of any autonomy provider who claims a single software stack runs identically across every platform without significant adaptation work.
What This Means for Equipment Manufacturers
Off-road autonomy is neither a solved problem nor an unsolvable one. The core architecture, safety methodology, and simulation approaches all transfer from on-road. But map-free navigation, GPS-denied localization, task-coupled planning, and diverse vehicle dynamics each require fresh engineering that the on-road world never had to do.
For manufacturers evaluating autonomy partners or building in-house capability, a few pointed questions cut through marketing claims quickly. How does the system know where it is when GPS drops out under canopy? How does it handle the actual operation, not just driving from one point to another? Has it been validated on your specific machine type and in your actual operating conditions, or just on a demo unit in ideal weather? And critically, what happens when the system encounters something it wasn't designed for, because every system eventually does?
The autonomous vehicle industry spent more than 15 years and tens of billions of dollars building technology that works on city streets. The off-road equipment industry doesn't need to repeat that journey. But it does need to be clear-eyed about which pieces transfer and which problems demand fresh engineering, built for the realities of mud, canopy, and machines that weigh more than a house.
Adityaveer Raswan leads the motion planning team at AEM member company Agtonomy, where he builds autonomous navigation systems for agricultural equipment. With over nine years of experience across robotaxis (Waymo), autonomous trucking (Waabi), and agricultural autonomy (Agtonomy), he brings a cross-domain perspective to off-road motion planning. He is a contributor to AEM's AI Subcommittee, a participant in the AMCC, and a contributing partner at OpenAutonomy.com.
