Docs / Applying Cognitive Layers to MathNN and all RL
Applying Cognitive Layers to MathNN and all RL
This document brainstorms (and maps) how the attention / curiosity / drives stack applies across MathNN and every RL path in _AugmentedIntelligence.
1. MathNN applications
| Area | How to apply layers | Status / hook |
|---|---|---|
| Dense / conv nets | Use attention block outputs as features; curiosity weights which layer activations to read | _Attention already in architectures; wire layers focus over activation keys later |
| Transformer demos | nn attn-demo = pure attention; layers attn-demo wraps same |
Done demos |
| Curiosity attention | Memory slots as keys; novelty from episodic mem | nn curiosity-demo + layers curiosity-demo |
| Tutor / solution steps | Attend over step history (nn math attn tutor) |
MathNNMathExtended |
| Curriculum | High drive.curiosity → harder phase; high fatigue → easier | MathNNLearning curiosity_scale |
| World-model rollouts | Novelty of imagined next-state embeddings | RLAgentNet world model curiosity already |
| Embedding spaces | Observe embeddings into layers memory when indexing |
Call ObserveEmbedding from TF/semantic path |
| Dropout schedule | Map fatigue → higher dropout (regularize when tired) | Future |
| Multi-task heads | Attention over task tokens | Future architecture |
Commands to combine today
nn attn-demo
nn curiosity-demo
layers mode explore
layers cycle 0.8 0.3
layers rl <nn_agent_id>
Brainstorm next MathNN features
nn attend <layer>— dump soft weights over hidden units.- Curiosity-gated training — up-weight batches with high prediction error.
- Drive-conditioned hyperparams — lr ∝ confidence; batch size ∝ attention.
- Slot memory module — shared with
_CuriosityAttention::EpisodicMemory. - Attention regularization loss — entropy bonus on weights in explore mode.
2. RL applications (all major tracks)
2.1 Core RL (RLExplore, RLAlgorithms, RLFeatures)
| Mechanism | Application |
|---|---|
_IntrinsicCuriosity |
Already count-based; also layers curiosity bonus <state> for shared table |
| ε-greedy / Boltzmann | Scale ε with drives.curiosity and inverse confidence |
| UCB / Thompson | Bandit arms = skills/goals; novelty from layers |
| Reward shaping | layers shape r i / _CuriosityAttention::ShapeReward |
| Safe shield | Mode safety → only shield-legal actions; attention on ethics |
layers mode explore
layers rl cartpole
rl cartpole 50 # existing; curiosity flags in workflows
2.2 Neural RL (RLAgentNet, MathNNLearning)
| Hook | Application |
|---|---|
SetCuriosityScale |
layers rl <agent> pushes 0.02 * beta_eff |
| Curriculum phases | Explore mode → advance curriculum; fatigue → hold phase |
| Imagine / world model | Observe imagined embeddings; bonus if novel |
| Align / low curiosity | Body recover → layers safety → low β → align mode |
2.3 Minecraft / Gaming / Life / Driving / Knowledge / Politics RL
| Domain | Attention | Curiosity |
|---|---|---|
| Minecraft | Attend mobs, ore goal, chat hazards | Explore unseen chunks; count visit cells |
| Gaming FPS | Attend enemies/threats | Peek novel angles carefully (shield) |
| LifeRL | Attend calendar/stress goals | Explore routines when restorative |
| DrivingRL | Attend lane/obstacles | Curiosity low in safety mode |
| KnowledgeRL | Attend uncertain claims | Info-gain over quiz items |
| PoliticalRL | Attend stakeholders | Explore policies under ethics shield |
| Space Engineers | Attend damage/power | Explore build variants |
| Child/parenting RL | Social attention mode | Gentle novelty |
Unified pattern
each RL step:
novelty = f(state_visit, pred_error, layers.memory)
layers.drives step novelty success
beta = layers.beta_eff
r' = r + beta * intrinsic
if layers.mode == safety: action = shield(action)
focus = layers.focus(state_summary) # optional auxiliary
2.4 Distributed / offline / RLAIF
| Idea | Application |
|---|---|
| Peer trust dynamics | Attend trustworthy peers more (MathDynamics trust) |
| Offline datasets | Curiosity over rare transitions for prioritization |
| RLAIF | Attention over preference pairs; curiosity for disagreement cases |
| PBT | Mutate β and capacity as population genes |
3. Cross-cutting recipes
| Recipe | MathNN | RL |
|---|---|---|
focus_work |
mode task, low β | ε↓, shield on |
explore_desk |
mode explore, high β | ε↑, count bonus |
safety_watch |
mode safety | shield strict, β≈0 |
recover_narrow |
capacity 1–2, fatigue high | freeze train, exploit only |
Wire later as:
layers recipe focus_work|explore_desk|safety_watch|recover_narrow
4. Implementation checklist (done vs next)
| Item | Done |
|---|---|
Unified layers / attention / curiosity commands |
Yes |
| Curiosity-biased focus over candidates | Yes |
| Drives ODE step | Yes |
| Count curiosity + shape reward | Yes |
| Body link for β | Yes |
| Body equilibrium calls cycle | Yes |
layers rl <agent> → SetCuriosityScale |
Yes |
| Per-domain auto-hook every Minecraft step | Next |
| ε schedule from drives | Next |
| Prioritized replay by novelty | Next |
| PerceptionBus → auto candidates | Next |
5. Example session
layers selftest
layers seed
layers mode explore
layers cycle 0.7 0.4 what next
layers rl minecraft
nn curiosity-demo
body equilibrium
layers mode safety
layers focus
Generated from the project markdown docs on 2026-07-24. This is a static, self-contained site.