Docs / TensorRT in `_AI`

TensorRT in `_AI`

Local engines (TensorRT.cpp) + registry / async detect (TrtRegistry) + remote tensorrt_detect on the Remote CUDA worker.

Build

Config	TensorRT
`Release-GPU-AVX2-Tensor`	`AI_USE_TENSORRT` + nvinfer libs
`Release-GPU-AVX512-Tensor`	same + AVX-512

/p:TensorRtDir=C:\path\to\TensorRT

Default: third_party/tensorrt.

Without Tensor configs, _TrtRegistry still runs: status, remote fallback, job queue.

Commands

trt status | list
trt load od_default | unload <id> | active <id>
trt register <id> <engine> <onnx> [kind]
trt build <onnx> <engine>
trt detect <image> [model]
trt enqueue <image> [model]
trt job list|status <id>|cancel <id>
trt bench [model] [n]
trt fallback

Aliases: tensorrt … (legacy) and trt ….

Models (defaults)

id	Role
`od_default`	`tensorrt_engine_path` / ONNX settings
`od_remote`	`HardwareAcceleration::RemoteDetectFrame`
`yolo_fast`	optional path under cache dir
`depth_midas`	optional depth engine

Fallback chain

local TRT session → remote tensorrt_detect → empty (Vision TFLite still used by Vision path)

Worker

tools/remote_cuda_worker.py implements stub tensorrt_detect with one box so clients can test remote OD without TRT on the worker. Replace with real TRT Python/C++ for production.