Agent Author's Guide

Related: RL hub, MathNN, RL Curriculum, ComputerRL (reference implementation).

The RL agents in _AI share a common anatomy. This guide distills the pattern so a new domain agent can be added consistently.

Anatomy

Neural policy slot

Register a MathNN policy through _RLAgentNet under a stable agent id, then select and observe through the bridge:

_RLAgentNet::Enable("myagent", true);
_RLAgentNet::EnsureConfigured("myagent", STATE, ACTIONS, 64, 1e-3, 0.99, "dueling-dqn");
_MathNNAgentBridge::SelectAction("myagent", state, epsilon);
_MathNNAgentBridge::ObserveTransition("myagent", s, a, r, ns, terminal);

Training happens online inside Observe while the agent is in a learning mode. Inspect with rl net status / rl net loss myagent.

Modes

off | assist | train | eval | explore

explore is the autonomous, curiosity-driven mode (novelty bonus + elevated epsilon); eval runs the current policy; train learns from transitions.

Safety

Wrap action selection in a safety shield, score policy rules into the reward, gate execution on _SafetyEthicsTrust::ApproveBotAction, and default to dry-run. See The Safety & Ethics Stack.

Checkpoints

Persist to a file and to MySQL via _RLCheckpointMySQL under a table key and profile, so a checkpoint trained on one node can be loaded on another.

Command surface & Lua

myagent rl on|assist|train|eval|explore|off
myagent rl status|features|actions|explain|dashboard
myagent rl scenario <name> [ticks]
myagent rl checkpoint tag|list|rollback | save|load
RegisterLuaGlobals(L)   // expose status/set_mode/scenario to Lua

Testing

Extract the pure logic (state build, reward, shield, novelty) into a *Core.hpp with assertion tests, following the other tested cores, so the policy glue can be iterated safely.

Checklist

1. Perception → state vector.
2. Action set + names.
3. Reward (verifiable) + curiosity for explore.
4. Neural slot via _RLAgentNet.
5. Safety shield + ethics gate + dry-run.
6. Checkpoints (file + MySQL).
7. Command handler + catalog + Lua globals.
8. Tested Core + a scenario runner.