Remote GPU offload for _AugmentedIntelligence: multi-worker pool, session tensors,

async jobs, hybrid matmul, GUI ops strip, and a Python worker.

ENV defaults: EnvironmentVariables.md (EnvironmentVariables.md)

Components

Piece	Path
Client	RemoteCuda.hpp / RemoteCuda.cpp
Worker	tools/remote_cuda_worker.py
Device placement	MathNNDevice kind remote
HA RPC compat	HardwareAcceleration::RemoteInfer + /infer
GUI	Ops strip summary via _RemoteCuda::OpsStripSummary
Commands	cuda remote … (also remote cuda …)

Commands

[text]
cuda remote status
cuda remote ping [worker_id]
cuda remote hello [worker_id]
cuda remote bench [size] [worker_id]
cuda remote set host <url…> [url2…]
cuda remote set token <tok>
cuda remote set timeout <ms>
cuda remote set enabled on|off
cuda remote job list|status <id>|cancel <id|all>|enqueue <op>
cuda remote tensor list|clear [name]|put [name]
cuda remote checkpoint push <local> [remote_name]
cuda remote checkpoint pull <remote> [local]
cuda remote allreduce
cuda remote ode [model]
cuda remote refresh

Worker quick start

[bat]
set AI_REMOTE_CUDA_TOKEN=dev-token
set AI_REMOTE_CUDA_PORT=7720
python tools\remote_cuda_worker.py –host 0.0.0.0 –port 7720

Client:

[bat]
set AI_REMOTE_CUDA_HOST=127.0.0.1
set AI_REMOTE_CUDA_PORT=7720
set AI_REMOTE_CUDA_TOKEN=dev-token

Then: cuda remote ping · cuda remote bench 256 · nn device set global train remote

Protocol

• Prefer `POST /v1/<op>` with JSON { “op”, “token”, “payload”: {…} } 
• Fallback `POST /infer` (HardwareAcceleration style) 
• Auth: Authorization: Bearer <token> or body “token” 
• Ops: ping, hello, bench, matmul, allreduce, checkpoint_*, dda_volume, ode_batch, stubs for fft/conv2d/…

Behaviour guarantees

1. Local fallback — matmul/bench always produce local results if remote fails. 
2. Kill-switch — safe mode cancels queued/running remote jobs. 
3. Pool — AI_REMOTE_CUDA_HOSTS least-inflight pick. 
4. Hybrid matmul — remote handshake + local GEMM for correctness (full tensor ship only for tiny shapes).

Implemented brainstorm map

Brainstorm item	Status
Worker daemon	Python HTTP worker
Op catalog	ping/hello/bench/matmul/allreduce/checkpoint/volume/ode + stubs
Session tensors	Client-side registry (cuda remote tensor)
Capability hello	Worker /v1/hello VRAM/CUDA fields
Placement policy	nn device set … remote
Async jobs	Background pump + cuda remote job
All-reduce lite	RPC ack + local vector
Checkpoint meta	push/pull metadata JSON
ODE batch	RPC + local catalog fallback
Volume stage	RPC stub for RC
Multi-worker pool	HOSTS list
VRAM fields	From hello/bench
Priority queues	Job priority field
Preemption	CancelAll on kill-switch
GUI ops strip	Remote summary
cuda remote * commands	Wired in SpeechCommands
Cloud adapters	Deferred (use HOSTS to any HTTP worker)

Deferred / next

• Full float tensor upload for large matmul on worker GPU 
• gRPC + TLS/mTLS 
• Real federated gradient merge across multi-node 
• Cloud marketplace adapters (RunPod/etc.) as URL presets 

---

Filed under: Uncategorized - @ July 18, 2026 6:50 am