The Hard Part of Fire-Tracking Drones Isn't the Loss Curve

arXiv:2609.10433 trains multiple UAVs to track simulated wildfire perimeters with deep reinforcement learning. The evidence it offers that this works is internal to training: loss trends converge, reward improves, and navigation patterns become more consistent, including the fire-boundary tracking the summary names.

The summary reports only a training run and no baseline. It never reports perimeter error, the distance between the edge a policy is tracking and the true fire edge. A simulator holds that true perimeter by construction and updates it every step, so computing the error is a subtraction against ground truth, not a measurement campaign. Without perimeter error, the result is a training curve, not a measurement.

Following an edge is a different job from mapping a field

Coverage planning is the older problem, and it has a hand-written answer: frontier-based exploration. Build an occupancy grid, mark every cell free, occupied, or unknown, and drive the robot toward the nearest boundary between known-free and unknown space. Repeat until nothing is unknown. That is a rule, not a learned policy. You can read it, test it, and explain exactly why it stalled.

Tracking a fire perimeter is a different objective wearing similar clothes. The target is a contour, it moves, and its shape changes between the moment you observe it and the moment you act on the observation. A policy that tracks it runs a loop: fuse the local sensor observations into an estimate of where the edge is now, predict where it will have moved by the time the aircraft arrives, and command a path that intercepts the predicted contour. The prediction is what does the work. Without one, the agent flies to where the fire was when it last looked, which is the stale-edge failure: it locks onto an edge the fire has already left and misses a new finger. A coverage planner rewarded for reducing unknown space will happily fly a clean lawnmower pattern across a fire that has already run two kilometres up a drainage in a different direction.

That table describes the design space, not this paper. The summary suggests that environmental structure and reward design influence policy effectiveness. The summary does not report which reward terms or observation space were used.

Reward shaping is where the engineering actually is

With a sparse reward, paid only when the full perimeter is mapped, nearly every rollout returns zero. A policy gradient method, which is one that improves a policy by estimating which actions raise expected reward, takes its update from that return, so a run of zero-reward episodes produces a gradient that is zero or dominated by noise. Getting a useful update requires random exploration to stumble onto a complete perimeter, and that becomes exponentially less likely as the episode lengthens. So practitioners shape the reward: pay for proximity to the contour, pay for alignment with it, pay a penalty for revisiting cells already seen. Each of those terms buys learnability and sells correctness.

Give an agent reward proportional to the number of boundary pixels it observes and it will eventually find the densest patch of boundary and hover there. The reward is satisfied. The mission is not. This is the standard failure mode of the approach, and it is why a converged loss curve is weak evidence about a system. Optimisation converging means the policy got good at the reward you wrote. It says nothing about whether the reward describes the job.

Why multiple agents is the part that is genuinely hard

Single-agent coverage in a simulator is largely a solved exercise. Multi-agent is not, for three structural reasons.

• Partial observability. Each aircraft sees a slice of the environment. The fire edge outside its sensor cone does not exist to it except as inference from what it has already seen.
• Non-stationarity. Every other agent is learning at the same time, so the environment each policy is optimising against shifts underneath it. The convergence guarantees that hold for single-agent value iteration assume a stationary environment, and that assumption is false here.
• Credit assignment. A team reward tells you the squad did well. It does not tell you which aircraft's decision earned it, and gradient estimates get noisy fast as the fleet grows.

The standard fix for the second and third is centralized training with decentralized execution — general knowledge about the field rather than something this summary reports. During training a single critic sees the joint state and the joint action and estimates the value of the team's combined move. That one number is factored back down into per-agent contributions, usually one of two ways: score each agent's action against a counterfactual baseline, which asks what the team would have got had that agent acted differently and charges it the difference, or learn a mixing function that combines per-agent values into the joint value under a monotonicity constraint, so that improving one agent's value cannot make the team's worse. Either way each policy receives a gradient attributable to its own action rather than the squad's, which is the whole point: only a critic that sees everyone can say who to blame, and it exists only during training.

At runtime the critic is gone and each policy sees only its own observations. Coordination is still there, learned implicitly. The observation each agent conditions on carries cues about its teammates: relative positions, how recently each region was covered, the shared reward history. Each policy learns the action that is the best response to what it expects the others to do. No explicit communication channel is needed for that, which is why the approach survives a radio link that drops behind a ridge.

Multi-agent here does not mean what it means in LLM tooling

Multi-agent reinforcement learning is decentralized control with a shared objective and a shared reward signal. That is a different thing from the multi-agent pattern in LLM tooling, where several language models pass messages to each other or a planner delegates to sub-agents. The agents here are control policies, the coordination is learned, and the question worth asking is whether the learned coordination beats what a rule would give you.

What the evidence supports

Converging loss and rising reward are diagnostics of a training run. Fire-boundary tracking that becomes more consistent across training is a simulation behaviour, not a field result.

It is not a fielded capability. The environment is simulated, and the summary does not report which physical effects the simulator models or which it leaves out, so how much of the policy's behaviour depends on that choice is unknown. With no baseline reported, there is also no way to tell whether the learned policy beats a scripted frontier planner on the same maps, which is the comparison that would matter to anyone writing flight software.

What would change my mind

The number is perimeter error, defined operationally: the mean distance between the edge the policy is tracking and the true fire edge the simulator already holds, averaged over flight minutes, with the maximum gap reported separately. Mean distance on its own hides the failure that matters, because a policy that tracks well on average but loses the edge for thirty seconds at a time is not tracking. The second number is the same measurement with wind and sensor noise switched on, against a scripted baseline, on maps the policy never trained on. A margin that survives both is a real result, and the reward-design work behind it becomes worth reading closely.