The Safety & Ethics Stack
Related: Be Good (Ethics), Gaming Ethics Gate, ComputerRL, Risk Mitigation Plan.
Agents in _AI can drive real input devices, so guardrails are layered rather than singular. An action must survive every layer below before it reaches the operating system.
The layers
| # | Layer | Where | Effect |
|---|---|---|---|
| 1 | Dry-run | per-agent flag | Recommend/log only; no real input is emitted. |
| 2 | Safety shield | per-agent applySafetyShield | Rewrites a selected action to a safe one based on perceived state. |
| 3 | Policy rules | per-agent rule table | Reward penalties when a trigger is not answered by the required action. |
| 4 | Ethics gate | _SafetyEthicsTrust::ApproveBotAction | Central approve/deny before execution. |
| 5 | Admission | Active Directory group check | Gates who may run _AI commands at all. |
Shield example: ComputerRL
The Computer RL shield classifies the active dialog and forces confirmation on high-severity classes before the older keyword checks:
system_elevation / credential_or_payment / destructive -> request_confirm
risky_text on a click/type action -> request_confirm
error dialog on a click -> wait
cursor stuck on a repeated click -> scrollStructural dialog classification (window title + recognized screen text) is stronger than a flat “risky word” substring scan, which both over- and under-triggers.
Verified outcomes, not proxies
Reward signals prefer verification over proxies: a goal is credited only when its content is stably present on screen, not merely when an external status flag flips. This mirrors the verifiable-reward stance used elsewhere in the RL stack.
Data-side protections
- Memory encryption — sensitive memory is sealed at rest (AES-GCM, OS-sealed key).
- Redaction — credential/PII regions should be masked before recognized text enters logs or replay buffers.
How an agent composes them
- Select an action (heuristic / Q-table / neural).
- Pass it through the safety shield (may rewrite it).
- Score policy-rule compliance into the reward.
- Gate execution on the ethics check.
- Execute only if dry-run is off.
Operator controls
<agent> rl shield on|off
<agent> rl dry-run on|off
computer rl policy status|seed|list|add
computer rl explain # last choice + reward + shield reason