Docs  /  A* pathfinding × reinforcement learning

A* pathfinding × reinforcement learning

Classic A* planning is integrated with every MathNN RL mode for grid navigation. Planning gives optimal (or near-optimal) paths and cost-to-go potentials; RL learns reactive policies that can imitate, hybridize, or explore beyond the planner.

Components

Module Role
Pathfinding.hpp/.cpp A* core: 4/8-connected grids, weighted A*, cost fields, graph A*, cost-to-go, path→actions
PathfindingRL.hpp/.cpp GridEnv (RLEnv), teacher, shaping, train-all-modes, hybrid/imitate
Curriculum id pathfind_astar Smokeable via rl curriculum smoke pathfind_astar

A* features

  • Binary occupancy grids and non-negative cost fields
  • Heuristics: Manhattan (octile with diagonals), Euclidean, Chebyshev, Zero (Dijkstra)
  • Weighted A* (weight > 1 trades optimality for speed)
  • Graph A* for abstract graphs
  • Cost-to-go field from the goal (Dijkstra) for potential-based reward shaping
  • Action map: 0=wait, 1=N, 2=S, 3=W, 4=E (matches game_simulator)

RL integration modes

Mode What happens
Teacher Always A* greedy on cost-to-go (baseline)
Learn Pure RL (DQN family, actor–critic, …)
Hybrid With probability (p), take A* action; else policy
Imitate Supervised Q warm-start from A* actions, then RL
Potential shaping (r \leftarrow r + \gamma\varphi(s') - \varphi(s)), (\varphi=-\mathrm{dist}_{A*})
Sparse / dense Goal bonus only, or Euclidean progress

RL algorithms trained (pathfind rl train all)

  • dqn
  • dueling-dqn
  • actor-critic
  • bandit
  • hierarchical
  • multihead-dueling
  • dqn+curiosity (world-model surprise scale)
  • dueling-hybrid-astar
  • dqn+imitate-astar

Commands

pathfind demo [4|8] [manhattan|euclidean|chebyshev|dijkstra]
pathfind from 0 0 to 9 9 4 manhattan 1.0
pathfind graph demo
pathfind cost-to-go

pathfind rl demo
pathfind rl teacher 20
pathfind rl train dqn 80
pathfind rl train all 40
pathfind rl imitate dqn 50
pathfind rl hybrid dueling-dqn 50 0.35
pathfind rl shape potential|sparse|dense
pathfind rl compare 20

rl curriculum smoke pathfind_astar 8 80
rl curriculum domain pathfind_astar
nn math train pathfind_astar   # if routed via MakeSmokeEnv + TrainDQN

Observation (12-D)

Agent pose (norm), goal delta, 4-way blocked flags, cost-to-go, time fraction, teacher action hint, potential (\varphi).

Safety

Synthetic maze only. For live robots/Minecraft, treat A* as a plan proposal behind shields and dry-run—same deployment ladder as other navigation RL.

Files

  • Pathfinding.cpp / Pathfinding.hpp
  • PathfindingRL.cpp / PathfindingRL.hpp
  • RLCurriculumOrchestrator.cpp (pathfind_astar)
  • Movement::FollowPath still uses demo A*
Generated from the project markdown docs on 2026-07-24. This is a static, self-contained site.