Remote CUDA — Implementation

Remote GPU offload for _AugmentedIntelligence: multi-worker pool, session tensors, async jobs, hybrid matmul, GUI ops strip, and a Python worker.

ENV defaults: EnvironmentVariables.md

Components

Commands

cuda remote status
cuda remote ping [worker_id]
cuda remote hello [worker_id]
cuda remote bench [size] [worker_id]
cuda remote set host <url...> [url2...]
cuda remote set token <tok>
cuda remote set timeout <ms>
cuda remote set enabled on|off
cuda remote job list|status <id>|cancel <id|all>|enqueue <op>
cuda remote tensor list|clear [name]|put [name]
cuda remote checkpoint push <local> [remote_name]
cuda remote checkpoint pull <remote> [local]
cuda remote allreduce
cuda remote ode [model]
cuda remote refresh

Worker quick start

set AI_REMOTE_CUDA_TOKEN=dev-token
set AI_REMOTE_CUDA_PORT=7720
python tools\remote_cuda_worker.py --host 0.0.0.0 --port 7720

Client:

set AI_REMOTE_CUDA_HOST=127.0.0.1
set AI_REMOTE_CUDA_PORT=7720
set AI_REMOTE_CUDA_TOKEN=dev-token

Then: cuda remote ping · cuda remote bench 256 · nn device set global train remote

Protocol

• Prefer POST /v1/<op> with JSON { "op", "token", "payload": {...} }
• Fallback POST /infer (HardwareAcceleration style)
• Auth: Authorization: Bearer <token> or body "token"
• Ops: ping, hello, bench, matmul, allreduce, checkpoint_*, dda_volume, ode_batch, stubs for fft/conv2d/…

Behaviour guarantees

1. Local fallback — matmul/bench always produce local results if remote fails.
2. Kill-switch — safe mode cancels queued/running remote jobs.
3. Pool — AI_REMOTE_CUDA_HOSTS least-inflight pick.
4. Hybrid matmul — remote handshake + local GEMM for correctness (full tensor ship only for tiny shapes).

Implemented brainstorm map

Deferred / next

• Full float tensor upload for large matmul on worker GPU
• gRPC + TLS/mTLS
• Real federated gradient merge across multi-node
• Cloud marketplace adapters (RunPod/etc.) as URL presets