MathNN

MathNN is the native C++ neural-network subsystem inside _AugmentedIntelligence. It is designed for small-to-medium in-process models where the application needs local learning without sending every update to TensorFlow, CUDA, or a remote policy server.

The main current use case is reinforcement learning: MathNN can act as a local value network, policy network, or DQN Q-network for systems such as Minecraft, driving RL, gaming RL, strategy agents, Life RL, and math tutoring domains.

RL hub: ReinforcementLearning.md · System map: AugmentedIntelligenceSystems.md · Life RL: LifeRL.md · Research paper (layers × gaming RL): MathNN_Layers_Are_What_You_Need.md · Field catalog: MathNNFields.md · Multi-level access map: MathNNAccessHub.md (nn access map)

What MathNN Is

MathNN is a compact neural-network framework implemented in the repository's C++ codebase. The dense network core supports analytic backpropagation, optimizers, checkpoint save/load, and command-line demos.

MathNN is not meant to replace large model training stacks. Use CUDA, TensorFlow, TensorRT, ONNX Runtime, llama.cpp, or a remote policy server for large pretrained models, high-throughput vision models, LLMs, or heavy transformer training.

Core Files

File	Role
MathNN.hpp / MathNN.cpp	Dense neural network core, activations, losses, optimizers, dropout, save/load.
MathNNLayers.hpp / MathNNLayers.cpp	Conv2D, max pooling, layer norm, RNN, LSTM forward path, and attention forward path.
MathNNDevice.hpp / MathNNDevice.cpp	CPU/GPU/NPU/APU/Remote placement, probe, bench, agent overrides.
MathNNLearning.hpp / MathNNLearning.cpp	Domain learning functions: prefer, transfer, distill, scale, federate, …
MathNNFeatures.hpp / MathNNFeatures.cpp	User-facing nn commands, Lua registration, and _NNPolicy.

See also: MathNNLearningFunctions.md · MathDE.md · BuildConfigurations.md · RealityConstruct.md · SpaceEngineersRL.md | RLNeural.hpp / RLNeural.cpp | In-process learners: DQN, dueling DQN, actor-critic, bandit, hierarchical, multi-head dueling, reward model, world model. | | RLAgentNet.hpp / RLAgentNet.cpp | Per-agent neural policy registry with aux reward/world models, checkpoint meta, Q/loss/feedback commands. | | LifeRL.hpp / LifeRL.cpp | Advisory multi-domain "life" RL on MathNN (sleep, focus, chores, …). |

What It Can Do

Dense Networks

The _NN class can build stacked dense networks:

• Add dense layers with configurable input/output sizes.
• Choose layer activations.
• Run forward inference with Predict.
• Train with supervised TrainStep.
• Save and load model weights.
• Report parameter count and network summaries.

Supported activations:

• Linear
• ReLU
• LeakyReLU
• Sigmoid
• Tanh
• GELU
• Softmax

Supported losses:

• MSE
• CrossEntropy
• BCE

Supported optimizers:

• SGD
• Momentum
• Adam

Additional dense-network features:

• Inverted dropout during training.
• Fused stable gradients for softmax plus cross entropy.
• Fused stable gradients for sigmoid plus binary cross entropy.
• Xavier/Glorot-style initialization for dense layers.

Architecture Layers

MathNNLayers adds lower-level neural building blocks:

Component	Capability
_Conv2D	Forward pass, backward pass, trainable weights, explicit channel-first image tensors.
_MaxPool2D	Forward pass and backward pass with cached argmax locations.
_LayerNorm	Forward pass, backward pass, trainable gamma/beta.
_RNNCell	Elman RNN forward steps plus sequence training with backprop-through-time.
_LSTMCell	Forward/inference path. Local BPTT training is not exposed; route larger LSTM training to a remote trainer.
_Attention	Scaled dot-product attention and multi-head attention forward/inference, with optional causal masking.

Tensor convention for image-like layers is flat row-major storage with explicit shapes. Images use channels-first layout: C * H * W.

RL Policy Adapter

_NNPolicy wraps _NN as a value or policy network:

• ActionValues(state) returns Q-values for all actions.
• GreedyAction(state) picks the highest-valued action.
• UpdateQ(state, action, target) trains one selected action toward a target.
• Save(path) and Load(path) persist the policy.

The default policy shape is:

state_dim -> hidden ReLU -> hidden ReLU -> action_dim Linear

This is enough for many DQN-style value functions where the network predicts one Q-value per action.

In-Process DQN

_NNQLearner builds a local DQN around _NNPolicy:

• Online network.
• Target network.
• Replay buffer.
• Epsilon-greedy action selection.
• TD target training: reward + gamma * max(Q(next_state)).
• Target network sync.
• Checkpoint save/load.

_RLNeural::TrainDQN provides a training loop over any RLEnv implementation. That makes MathNN reusable across game, robotics, driving, and knowledge environments when those environments expose state, action, reward, and terminal signals.

Per-Agent Neural Registry

_RLAgentNet lets individual agents opt into a MathNN-backed learner without rewriting their existing heuristic or tabular paths.

Primary modes: dqn, dueling-dqn, actor-critic, bandit, hierarchical, multihead-dueling, worldmodel (dueling + curiosity), reward-model (bandit + preference head).

Auxiliary (combinable):

• Reward model: rl net rm <agent> on, rl net reward <agent> good|bad, rl net blend <agent> 0.7
• World model / curiosity: rl net wm <agent> on, rl net curiosity <agent> 0.05

Supported agent examples:

• minecraft, driving, gaming_fps, strategy
• life, life_sleep_routine, life_focus_blocks, …
• math tutoring domains, human, cybernetic, desktop_support

Checkpoints write a .meta sidecar (mode, dims) plus optional .rm / .wm files.

Life RL

Advisory multi-domain daily-living RL (LifeRL). Synthetic smoke by default; sensitive domains (mood, budget, social) are bandit/preference oriented and never auto-execute external side effects. See docs/LifeRL.md.

Commands

Mathematics RL Domains (MathNN)

A full mathematical field catalog (~55 domains) trains MathNN DQN policies over safe synthetic curriculum environments (category mathematics). Each field maps tutoring actions (simplify, prove, integrate, optimize, …) into the shared eight-value observation used by the RL curriculum smoke envs.

See MathNNFields.md for the complete taxonomy (foundations, algebra, analysis, geometry/topology, discrete, applied, and cross-disciplinary mathematical sciences).

Group	Example domain IDs
School / core	math_arithmetic, math_algebra, math_geometry, math_calculus
Advanced calculus / DE	math_vector_calculus, math_differential_equations, math_partial_differential_equations
Algebra / discrete	math_linear_algebra, math_abstract_algebra, math_number_theory, math_graph_theory
Analysis / foundations	math_real_analysis, math_complex_analysis, math_topology, math_logic
Applied	math_numerical_analysis, math_optimization, math_control_theory, math_game_theory
Cross-field	math_mathematical_physics, math_mathematical_finance, math_mathematical_biology

nn math domains
nn math domain math_algebra
nn math domain topology
nn math smoke mathematics 4 48
nn math train math_algebra 200
nn math train pde 150
nn math net enable math_algebra
rl net save math_algebra checkpoints/math_algebra_mathnn.txt
rl curriculum domains mathematics
rl curriculum smoke math_trigonometry 8 78

Aliases include short names (algebra, pde, linalg, stats, topology, …) and umbrella mathematics / math to smoke all math fields.

MathNN Demos

Run these in the _AugmentedIntelligence command interface:

nn status
nn demo
nn conv-demo
nn rnn-demo
nn attn-demo

Command behavior:

Command	What it tests
nn status	Confirms the MathNN command surface is registered.
nn demo	Trains a small dense network on XOR with BCE and Adam. This is the main smoke test for dense forward/backprop/training.
nn conv-demo	Runs Conv2D and MaxPool over an 8x8 input and prints output shapes.
nn rnn-demo	Trains a small RNN echo task and prints loss movement.
nn attn-demo	Runs multi-head causal attention over a small sequence.

Lua surface:

ai_nn_xor_demo()

This runs the XOR demo and returns the final loss.

RLAgentNet Commands

rl net status
rl net modes
rl net enable <agent>
rl net disable <agent>
rl net mode <agent> <dqn|dueling-dqn|actor-critic|bandit|hierarchical|multihead-dueling|worldmodel|reward-model>
rl net save <agent> <path>
rl net load <agent> <path>
rl net q <agent> [top_k]
rl net loss <agent>
rl net reward <agent> good|bad|<n>
rl net blend <agent> <alpha>
rl net curiosity <agent> <scale>
rl net wm <agent> on|off
rl net rm <agent> on|off

Examples:

rl net enable driving
rl net mode gaming_fps actor-critic
rl net wm minecraft on
rl net curiosity minecraft 0.05
rl net reward gaming_fps good
rl net save driving checkpoints/driving_mathnn.txt
rl net load driving checkpoints/driving_mathnn.txt
rl cartpole 50 dueling curiosity
life rl smoke sleep_routine 12
life rl suggest focus_blocks

Minecraft MathNN Commands

The Minecraft DQN subsystem can use MathNN through its neural-net mode:

minecraft dqn net on
minecraft dqn net off
minecraft dqn net status
minecraft dqn net save <path>
minecraft dqn net load <path>

Executive Functions (high-level setup steps each train/eval tick):

minecraft dqn plan on
minecraft dqn plan detail
minecraft dqn plan next 2
minecraft dqn plan status

Typical Minecraft startup sequence:

minecraft dqn observe
minecraft dqn dry-run
minecraft dqn plan on
minecraft dqn goal mine iron_ore
minecraft dqn net on
minecraft dqn net status
minecraft dqn train 1000

Use short training runs first. Stop and inspect behavior before letting it run for long periods:

minecraft dqn off
minecraft dqn status
minecraft dqn net save minecraft_mathnn_checkpoint.txt

How MathNN Fits Into RL

A MathNN-backed RL agent uses this loop:

1. The domain converts the current world into a numeric state vector.
2. MathNN predicts Q-values for available actions.
3. The agent chooses an action, often epsilon-greedy during training.
4. The environment executes the action.
5. The domain computes reward, next state, and terminal status.
6. _NNQLearner stores the transition in replay memory.
7. Training samples a minibatch and updates the online network.
8. The target network is periodically synchronized.
9. A checkpoint is saved after useful behavior appears.

The most important part is the state/reward/action design. MathNN can only learn from the signals it is given. If the state vector does not show walls, velocity, health, objective distance, inventory, or other useful facts, the network cannot learn those facts. If the reward function pays the wrong behavior, the network will optimize the wrong behavior.

Current Capability Matrix

Area	Status	Notes
Dense MLP inference	Available	_NN::Predict.
Dense MLP training	Available	_NN::TrainStep.
Activations	Available	Linear, ReLU, LeakyReLU, Sigmoid, Tanh, GELU, Softmax.
Losses	Available	MSE, cross entropy, BCE.
Optimizers	Available	SGD, Momentum, Adam.
Dropout	Available	Training-time inverted dropout.
Save/load	Available	Native MathNN checkpoint format.
Conv2D	Available	Forward/backward implemented; gradient-check before production training.
MaxPool2D	Available	Forward/backward implemented.
LayerNorm	Available	Forward/backward implemented.
RNN	Available	Elman RNN with BPTT sequence training.
LSTM	Forward only	Use remote trainer for full local sequence training needs.
Attention	Forward only	Scaled dot-product and multi-head inference helpers.
DQN	Available	_NNQLearner with online/target nets and replay.
Dueling DQN	Available	_NNDuelingQLearner V+A.
Actor-critic	Available	_NNActorCriticLearner policy+value.
Bandit	Available	_NNBanditLearner.
Hierarchical RL	Available	skill manager + worker DQN.
Multi-head dueling	Available	skill-group advantages.
Reward model	Available	preference BCE + blend into Observe.
World model	Available	Δs+reward predict + curiosity bonus.
Life RL	Available	advisory multi-domain (life rl …).
Per-agent neural policies	Available	RLAgentNet opt-in registry.
LLM behavior	Not MathNN	Use the LLM subsystem; MathNN can score states/actions, not generate rich language.
Large CUDA training	Not MathNN	Use CUDA/TensorFlow/remote training for large models.

What MathNN Is Good For

Use MathNN for:

• Local DQN value functions.
• Small supervised models.
• RL action scoring.
• Agent-specific policies.
• Lightweight experiments that should compile into _AugmentedIntelligence.
• Fast smoke tests for learning loops without external services.
• Simple RNN sequence experiments.
• Small perception experiments using Conv2D and pooling.

Use another subsystem for:

• LLM chat and reasoning.
• Image generation.
• Large CNNs or transformers.
• Production-scale CUDA training.
• Model import/export pipelines such as ONNX.
• High-volume offline training jobs.

Practical Training Guidance

Start with the demos before training a domain:

nn status
nn demo
nn conv-demo
nn rnn-demo
nn attn-demo

Then validate the target domain without learning:

minecraft dqn observe
minecraft dqn dry-run

For any RL domain, verify:

• State values change when the world changes.
• Legal actions match what the agent can actually do.
• Rewards increase for intended progress.
• Terminal events are marked correctly.
• Safety shields and stop commands work.

Only after that should you enable MathNN training:

rl net enable <agent>

or for Minecraft:

minecraft dqn net on
minecraft dqn train 1000

Save checkpoints when behavior improves:

rl net save <agent> checkpoints/<agent>_mathnn.txt
minecraft dqn net save minecraft_mathnn_checkpoint.txt

Troubleshooting

Problem	Likely cause	What to check
nn status is not recognized	Command dispatcher is not routing nn commands.	Check _MathNNFeatures::HandleSpeechCommand registration.
nn demo loss does not decrease	Training path, optimizer, or numeric issue.	Rebuild, rerun, and compare XOR predictions.
RL agent never improves	Bad state/reward/action wiring.	Inspect observations, rewards, terminal flags, and action execution before tuning the network.
Minecraft walks into walls	State/action safety issue, not just a network issue.	Use minecraft dqn observe, minecraft dqn wall-guard on, and staged arenas.
Checkpoint load fails	Wrong path, incompatible model shape, or inactive agent.	Enable/configure the same agent and use a matching checkpoint.
Training is unstable	Learning rate, reward scale, or exploration is too aggressive.	Lower learning rate, normalize rewards, reduce action space, and train in shorter sessions.

Safety Notes

MathNN can select actions automatically when connected to an RL domain. Treat live-control domains differently from simulations:

• Prefer simulation or dry-run first.
• Keep explicit stop commands available.
• Use safety shields for driving, robotics, Minecraft, and desktop control.
• Keep high-stakes domains in advisory, simulated, or human-approved modes.
• Do not deploy learned behavior directly into real-world control without independent validation.

Short Summary

MathNN gives _AugmentedIntelligence a local neural-learning layer. It can train small dense networks, run several neural layer demos, act as a DQN value network, and provide opt-in neural policies for RL agents. Its strongest current role is local reinforcement learning support, especially when paired with carefully designed states, rewards, legal actions, and safety guards.

Multi-device placement (CPU / GPU / NPU / APU)

ext nn device status nn device list nn device refresh nn device bench all 256 nn device set global train gpu nn device set global infer npu nn device set global dtype fp16 nn device set agent minecraft train gpu infer gpu nn device set agent life infer npu nn device resolve space_engineers nn device clear agent life

• CPU � MatrixAcceleration AVX/AVX-512 GEMM (default, correctness).
• GPU � CUDA probe + hybrid staged buffers (host GEMM until cuBLAS kernel lands).
• NPU � env hints (OPENVINO_DEVICE, QNN_SDK_ROOT, DML_VISIBLE_DEVICES); placeholder GEMM + ONNX EP path later.
• APU � hybrid policy (CPU train path + optional iGPU/NPU prefs).
• Remote � AI_REMOTE_CUDA_HOST for HardwareAcceleration workers.

Agent status (nn agent status <id>) includes device_train / device_infer resolved devices.