{"id":2058,"date":"2026-07-18T06:50:30","date_gmt":"2026-07-18T13:50:30","guid":{"rendered":"http:\/\/macdaddy4sure.ai\/?p=2058"},"modified":"2026-07-18T06:50:30","modified_gmt":"2026-07-18T13:50:30","slug":"remote-cuda-implementation","status":"publish","type":"post","link":"http:\/\/macdaddy4sure.ai\/index.php\/2026\/07\/18\/remote-cuda-implementation\/","title":{"rendered":"Remote CUDA \u2014 Implementation"},"content":{"rendered":"\n

Remote GPU offload for _AugmentedIntelligence: multi-worker pool, session tensors,<\/p>\n\n\n\n

async jobs, hybrid matmul, GUI ops strip, and a Python worker.<\/p>\n\n\n\n

ENV defaults:<\/strong> EnvironmentVariables.md (EnvironmentVariables.md)<\/p>\n\n\n\n

Components<\/h2>\n\n\n\n
Piece<\/strong><\/td>Path<\/strong><\/td><\/tr>
Client<\/td>RemoteCuda.hpp \/ RemoteCuda.cpp<\/td><\/tr>
Worker<\/td>tools\/remote_cuda_worker.py<\/td><\/tr>
Device placement<\/td>MathNNDevice kind remote<\/td><\/tr>
HA RPC compat<\/td>HardwareAcceleration::RemoteInfer + \/infer<\/td><\/tr>
GUI<\/td>Ops strip summary via _RemoteCuda::OpsStripSummary<\/td><\/tr>
Commands<\/td>cuda remote \u2026 (also remote cuda \u2026)<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

Commands<\/h2>\n\n\n\n

[text]
<\/em>cuda remote status
cuda remote ping [worker_id]
cuda remote hello [worker_id]
cuda remote bench [size] [worker_id]
cuda remote set host <url…> [url2…]
cuda remote set token <tok>
cuda remote set timeout <ms>
cuda remote set enabled on|off
cuda remote job list|status <id>|cancel <id|all>|enqueue <op>
cuda remote tensor list|clear [name]|put [name]
cuda remote checkpoint push <local> [remote_name]
cuda remote checkpoint pull <remote> [local]
cuda remote allreduce
cuda remote ode [model]
cuda remote refresh<\/p>\n\n\n\n

Worker quick start<\/h2>\n\n\n\n

[bat]
<\/em>set AI_REMOTE_CUDA_TOKEN=dev-token
set AI_REMOTE_CUDA_PORT=7720
python tools\\remote_cuda_worker.py –host 0.0.0.0 –port 7720<\/p>\n\n\n\n

Client:<\/p>\n\n\n\n

[bat]
<\/em>set AI_REMOTE_CUDA_HOST=127.0.0.1
set AI_REMOTE_CUDA_PORT=7720
set AI_REMOTE_CUDA_TOKEN=dev-token<\/p>\n\n\n\n

Then: cuda remote ping \u00b7 cuda remote bench 256 \u00b7 nn device set global train remote<\/p>\n\n\n\n

Protocol<\/h2>\n\n\n\n
    \n
  • Prefer `POST \/v1\/<op>`<\/strong> with JSON { “op”, “token”, “payload”: {…} }\u00a0<\/li>\n\n\n\n
  • Fallback `POST \/infer`<\/strong> (HardwareAcceleration style)\u00a0<\/li>\n\n\n\n
  • Auth: Authorization: Bearer <token> or body “token”\u00a0<\/li>\n\n\n\n
  • Ops: ping, hello, bench, matmul, allreduce, checkpoint_*, dda_volume, ode_batch, stubs for fft\/conv2d\/\u2026<\/li>\n<\/ul>\n\n\n\n

    Behaviour guarantees<\/h2>\n\n\n\n
      \n
    1. Local fallback<\/strong> \u2014 matmul\/bench always produce local results if remote fails.\u00a0<\/li>\n\n\n\n
    2. Kill-switch<\/strong> \u2014 safe mode cancels queued\/running remote jobs.\u00a0<\/li>\n\n\n\n
    3. Pool<\/strong> \u2014 AI_REMOTE_CUDA_HOSTS least-inflight pick.\u00a0<\/li>\n\n\n\n
    4. Hybrid matmul<\/strong> \u2014 remote handshake + local GEMM for correctness (full tensor ship only for tiny shapes).<\/li>\n<\/ol>\n\n\n\n

      Implemented brainstorm map<\/h2>\n\n\n\n
      Brainstorm item<\/strong><\/td>Status<\/strong><\/td><\/tr>
      Worker daemon<\/td>Python HTTP worker<\/td><\/tr>
      Op catalog<\/td>ping\/hello\/bench\/matmul\/allreduce\/checkpoint\/volume\/ode + stubs<\/td><\/tr>
      Session tensors<\/td>Client-side registry (cuda remote tensor)<\/td><\/tr>
      Capability hello<\/td>Worker \/v1\/hello VRAM\/CUDA fields<\/td><\/tr>
      Placement policy<\/td>nn device set \u2026 remote<\/td><\/tr>
      Async jobs<\/td>Background pump + cuda remote job<\/td><\/tr>
      All-reduce lite<\/td>RPC ack + local vector<\/td><\/tr>
      Checkpoint meta<\/td>push\/pull metadata JSON<\/td><\/tr>
      ODE batch<\/td>RPC + local catalog fallback<\/td><\/tr>
      Volume stage<\/td>RPC stub for RC<\/td><\/tr>
      Multi-worker pool<\/td>HOSTS list<\/td><\/tr>
      VRAM fields<\/td>From hello\/bench<\/td><\/tr>
      Priority queues<\/td>Job priority field<\/td><\/tr>
      Preemption<\/td>CancelAll on kill-switch<\/td><\/tr>
      GUI ops strip<\/td>Remote summary<\/td><\/tr>
      cuda remote * commands<\/td>Wired in SpeechCommands<\/td><\/tr>
      Cloud adapters<\/td>Deferred (use HOSTS to any HTTP worker)<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

      Deferred \/ next<\/h2>\n\n\n\n
        \n
      • Full float tensor upload for large matmul on worker GPU\u00a0<\/li>\n\n\n\n
      • gRPC + TLS\/mTLS\u00a0<\/li>\n\n\n\n
      • Real federated gradient merge across multi-node\u00a0<\/li>\n\n\n\n
      • Cloud marketplace adapters (RunPod\/etc.) as URL presets\u00a0<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"

        Remote GPU offload for _AugmentedIntelligence: multi-worker pool, session tensors, async jobs, hybrid matmul, GUI ops strip, and a Python worker. ENV defaults: EnvironmentVariables.md (EnvironmentVariables.md) Components Piece Path Client RemoteCuda.hpp \/ RemoteCuda.cpp Worker tools\/remote_cuda_worker.py Device placement MathNNDevice kind remote HA RPC compat HardwareAcceleration::RemoteInfer + \/infer GUI Ops strip summary via _RemoteCuda::OpsStripSummary Commands cuda remote \u2026 (also […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-2058","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"http:\/\/macdaddy4sure.ai\/index.php\/wp-json\/wp\/v2\/posts\/2058","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/macdaddy4sure.ai\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/macdaddy4sure.ai\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/macdaddy4sure.ai\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/macdaddy4sure.ai\/index.php\/wp-json\/wp\/v2\/comments?post=2058"}],"version-history":[{"count":1,"href":"http:\/\/macdaddy4sure.ai\/index.php\/wp-json\/wp\/v2\/posts\/2058\/revisions"}],"predecessor-version":[{"id":2059,"href":"http:\/\/macdaddy4sure.ai\/index.php\/wp-json\/wp\/v2\/posts\/2058\/revisions\/2059"}],"wp:attachment":[{"href":"http:\/\/macdaddy4sure.ai\/index.php\/wp-json\/wp\/v2\/media?parent=2058"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/macdaddy4sure.ai\/index.php\/wp-json\/wp\/v2\/categories?post=2058"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/macdaddy4sure.ai\/index.php\/wp-json\/wp\/v2\/tags?post=2058"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}