How to extend _AugmentedIntelligence with new capabilities, and how the multi-node mesh stack works. This is the developer counterpart to the user-facing install/usage docs.

Two extension surfaces:

• Native C++ modules — compiled into the host, each exposing a typed command and (optionally) Lua globals.
• Lua scripts — orchestrators that drive the runtime through the registered Lua API, without recompiling.

The mesh systems (NetMesh, NetFeatures, NetLLM) are themselves ordinary modules that follow the same contract, so understanding the module contract explains the whole system.

Part 1 — The module contract

Every extensible subsystem is a class named _Something with static entry points. The minimal contract:

[cpp]
#pragma once
// Apache 2.0 header …
#include “AugmentedIntelligence.hpp”      // the platform umbrella header
using namespace std;

class _MyModule
{
public:
    // Optional lifecycle
    static void Init();
    static void Shutdown();
    static string GetStatus();

    // Typed-command entry point. `command` is the whitespace-tokenized input;
    // `start_index` is where this module’s arguments begin.
    static void HandleSpeechCommand(const vector<string>& command, size_t start_index = 0);

    // Expose scripting helpers to the embedded Lua VM.
    static void RegisterLuaGlobals(lua_State* L);
};

The tested-core pattern (recommended)

Most numeric/logic subsystems split into:

File	Role	Buildable in isolation?
MyModuleCore.hpp	header-only pure logic, std-only	✅ compiles under the g++ test harness
MyModule.cpp	glue: command parsing, Lua, DB/LLM/GUI calls	❌ needs the full dependency stack (MSVC)
suite_mymodule.cpp	behavioural tests over the core	✅ runs offline

The pure core is unit-tested offline via tests/run_math_tests.ps1 (a MinGW g++ harness with a shim AugmentedIntelligence.hpp), while the glue is validated by static review + the real MSVC build. Keeping the testable logic in a header-only *Core.hpp is the single most useful convention in this codebase.

Part 2 — Adding a typed command

Typed commands are the primary control plane. Dispatch is an additive block in SpeechCommands.cpp (myCallback), where each umbrella claims a top-level verb and returns early:

[cpp]
// SpeechCommands.cpp — v11 build-cycle command umbrellas
if (!command.empty() && command[0] == “net”)   { _NetworkingFeatures::HandleSpeechCommand(command, 0); return vector<string>(); }
if (!command.empty() && command[0] == “rl”)    { _RLFeatures::HandleSpeechCommand(command, 0);        return vector<string>(); }
if (!command.empty() && command[0] == “nn”)    { _MathNNFeatures::HandleSpeechCommand(command, 0);    return vector<string>(); }
// … calc, algebra, de, mlapp, …

Steps

1. Create MyModule.hpp / MyModule.cpp implementing HandleSpeechCommand.
2. Include the header in SpeechCommands.cpp and add one dispatch line to the umbrella block:

   “`cpp

   if (!command.empty() && command[0] == “mymod”) { _MyModule::HandleSpeechCommand(command, 0); return vector<string>(); }

   “`

• Register the source in _AugmentedIntelligence.vcxproj (<ClInclude> + <ClCompile>) and .vcxproj.filters.
• (Optional) Add a suite_mymodule.cpp + core header and wire it into tests/run_math_tests.ps1.

Sub-command umbrella pattern

For a family of related commands, dispatch on the next token — exactly how net fans out:

[cpp]
void _NetworkingFeatures::HandleSpeechCommand(const vector<string>& command, size_t start_index)
{
    const string selector = command.size() > 1 ? command[1] : “”;
    if (selector == “features”) { _NetFeatures::HandleSpeechCommand(command, start_index + 2); return; }
    if (selector == “mesh”)     { _NetMesh::HandleSpeechCommand(command, start_index + 2);     return; }
    if (selector == “llm”)      { _NetLLM::HandleSpeechCommand(command, start_index + 2);      return; }
    // else: print usage
}

command is the tokenized line (command[0] = verb); parse your arguments starting at start_index.

Part 3 — Exposing a Lua API

Scripts run in the embedded Lua VM. To make a module scriptable:

• Implement RegisterLuaGlobals(lua_State* L) using lua_register and the C-API push/check helpers:

   “`cpp

   static int l_mymod_do(lua_State* L) {

       const char* arg = luaL_checkstring(L, 1);

       lua_pushnumber(L, / result /);

       return 1;  // number of return values

   }

   void _MyModule::RegisterLuaGlobals(lua_State* L) {

       lua_register(L, “mymod_do”, l_mymod_do);

   }

   “`

• Hook it in to the host’s Lua init — the registration list in RSIAgent.cpp:

   “`cpp

   // RSIAgent.cpp — registerExtendedLuaGlobals(lua_State* L)

   _HardwareAcceleration::RegisterLuaGlobals(L);

   _PerceptionBus::RegisterLuaGlobals(L);

   // … add:

   _MyModule::RegisterLuaGlobals(L);

   “`

Existing Lua surfaces you can build on

Prefix	From	Purpose
ai_nn_*	MathNN	build/train/query native neural nets
ai_reason_*	ReasoningEngine	assert facts/rules, prove goals, resolve gaps
perception / memory globals	PerceptionBus, MemoryKnowledge, CrossModalEpisodicEmbedder	read the fused perception summary, read/write memory

Lua orchestrators live in lua_scripts/ (e.g. the RL orchestrators). They’re loaded at runtime — no recompile needed to iterate on behavior.

Part 4 — Mesh systems

The mesh turns a single always-on node into a self-organizing fleet. Entry point: the net command → _NetworkingFeatures → three modules.

net features <…>   -> _NetFeatures   (security / resilience / observability)
net mesh <…>       -> _NetMesh        (discovery / gossip / relay / federation)
net llm <…>        -> _NetLLM         (distributed LLM routing)

4a. NetMesh — fleet fabric

Built on the existing RemotePeer registry ( _Networking remote-commands channel).

Capability	What it does
UDP discovery beacon	LAN auto-discovery (StartDiscovery, default port 7711, 5 s interval); EncodeBeacon/ParseBeacon.
Heartbeat / gossip	Liveness via RecordHeartbeat + BuildGossipDigest/MergeGossipDigest; PruneStale drops nodes older than 30 s.
Node registry	Node { name, host, port=7710, tags, last_seen_ms, load, healthy }. Query with HealthyNodesWithTag(“gpu:4090”).
Store-and-forward	Enqueue/Drain reliable broadcast across flaky links (TTL’d).
Relay / NAT traversal	StartRelay (default port 7712) forwards `<token>\	<peer>\	<command> after authorizing the token for scope relay`.
RSI leaderboard	_RsiLeaderboard — nodes publish routing-weight improvements; the best config gossips fleet-wide.
Federated memory	_FederatedMemory — Last-Writer-Wins episodic memory merge across nodes (push/pull via persist/load hooks).
Sensor ingest	_SensorIngest — versioned binary packet (AISP magic; Audio/Video/Depth/Activity/Screen) so any node/phone feeds the world model via SetConsumer.

4b. NetFeatures — security, resilience, observability

Dependency-light (std + OpenSSL, already linked). Hardens the remote-commands channel.

Class	Purpose
_CapabilityToken	Signed, scoped, expiring grants (base64url(payload).hmac) — replaces the all-or-nothing shared token. Issue / Verify / Authorize(token, scope, …).
_NetCrypto	Base64, HMAC-SHA256, SHA-1 (WebSocket accept), constant-time compare, random tokens.
_RateLimiter	Token-bucket per peer/identity.
_AuditLog	Append-only, hash-chained log — edits/deletions are detectable (Verify).
_KillSwitch	Global safe-mode (severs actuation + mesh) and per-peer quarantine; PeerAllowed. Tie into the ethics gate.
_Backoff	Exponential backoff with full jitter for reconnect loops.
_Metrics	Counters/gauges exposed as Prometheus text at /metrics.
_LogRing	Bounded in-memory log so peers can network logs <n> remotely.
_CredentialBroker	Scope-gated secret store — nodes fetch DB/API creds at runtime instead of baking them in source.
_WebSocket	RFC6455 handshake + frame encode/decode for live dashboard push.

4c. NetLLM — distributed inference

Class	Purpose
_LlmPool	Health-checked backend pool; EWMA latency ranking, failover + cooldown. PickBest(tag) chooses the fastest healthy backend matching e.g. model:llama3.
_InferenceRouter	Routes a request Local (a pool backend) or Peer (a fleet node that has the model + a free GPU), executing via an injected local invoker or RemoteCommandsSendToPeer.
_LlmEnsemble	Fan-out to several backends + Reconcile (first / majority). Speculative decoding is a design hook.
_LlmStreamingRelay	Proxy token-by-token SSE from a remote Ollama back to the requester (stub — needs a streaming HTTP client).

Ports summary

Port	Use
7710	Remote commands (node default)
7711	UDP discovery beacon
7712	Mesh relay / rendezvous
7720	Secondary remote channel (firewall-opened by the installer)

Security model

Capability tokens (scoped + expiring) replace a single shared secret, every relayed command is authorized for a scope, the kill-switch/quarantine severs compromised peers, and the audit log is tamper-evident. Heavier transports are declared but stubbed until their library is wired — each returns a clear not-implemented status:

Stub	Dependency to wire	Design
_SecureChannel	OpenSSL TLS 1.3 / Noise (libsodium)	wrap socket send/recv in SSL; pin peer public keys
_KerberosAuth	Windows SSPI (secur32)	SPNEGO negotiate → AD group → capability scopes
_WebRtcEgress	libdatachannel / libwebrtc	SDP over remote-commands; stream perception frames
_MqttBridge	paho.mqtt.c / mosquitto	publish ai/<node>/perception/<type>; subscribe sensors into _SensorIngest

Part 5 — Wiring status & how to finish it

• Typed commands (`net …`) are wired into the host dispatcher (SpeechCommands.cpp), and _NetworkingFeatures::InitAll() starts the three modules.
• Lua globals: _NetworkingFeatures::RegisterLuaGlobals(L) is implemented but confirm it’s called from your Lua init (RSIAgent.cpp registerExtendedLuaGlobals) — add it alongside the other module registrations if you want the net_* helpers in scripts.
• Stubs (_SecureChannel, _KerberosAuth, _WebRtcEgress, _MqttBridge, _LlmStreamingRelay) compile and return a not-implemented status until their third-party library is linked.

See NETWORKING_FEATURES.md (repo root) for the full brainstorm-to-code status matrix of every mesh feature.

Checklist: ship a new module

• [ ] MyModuleCore.hpp (pure, testable) + MyModule.{hpp,cpp} (glue), Apache header, #include “AugmentedIntelligence.hpp”.
• [ ] HandleSpeechCommand implemented; dispatch line added to SpeechCommands.cpp.
• [ ] RegisterLuaGlobals implemented (if scriptable); call added to RSIAgent.cpp.
• [ ] <ClInclude>/<ClCompile> added to .vcxproj and .vcxproj.filters.
• [ ] suite_mymodule.cpp + core wired into tests/run_math_tests.ps1; run_math_tests.ps1 green.
• [ ] Builds in a real MSVC configuration (Release-GPU-AVX2 or a CPU variant).

Related documents

• NETWORKING_FEATURES.md — mesh feature status matrix (source of truth)
• AGENT_ORCHESTRATION.md (AGENT_ORCHESTRATION.md) — multi-agent / Lua orchestration
• MathNN.md (MathNN.md) — native neural-net framework (ai_nn_* Lua API)
• ReinforcementLearning.md (ReinforcementLearning.md) — RL hub (curriculum, MathNN, domains)
• AugmentedIntelligenceSystems.md (AugmentedIntelligenceSystems.md) — whole-app system map
• FileSystemUpdate.md (FileSystemUpdate.md) — background FS inventory + MySQL
• LifeRL.md (LifeRL.md) — advisory life-domain RL
• Windows-Install.md (Windows-Install.md) / Ubuntu-Install.md (Ubuntu-Install.md) — build & run

---

Filed under: Uncategorized - @ July 18, 2026 6:30 am