Dear ImGui — Fitting More Features

Brainstorm for expanding the Win32 + DirectX 11 dashboard (Gui.cpp / Gui.hpp)

without turning the main thread into an unmaintainable monolith.

Current surface (already sketched in code): camera/OD panes, audio waveform,

thread terminals, HUD, command palette, timeline, node graph, inspector,

profiles, plugin panels, quick controls.

Design principles

One data bus, many views — widgets only read thread-safe snapshots; never

call long work on the ImGui thread.

Commands stay commands — GUI buttons enqueue the same strings as typed /

speech (SubmitCommandFromGui / CommandQueue).

Docking + layouts — use ImGui docking (and optional imgui.ini layouts)

so dense features stay discoverable.

Lazy panels — register panels in a catalog; only tick visible windows.
Feature flags — match build configs (AI_CPU_ONLY, CUDA, etc.) so panels

hide when subsystems are absent.

Information architecture

Zone	Role	Examples
Ops strip	Always-on status	CPU/GPU, queue depth, RSI, MySQL, agent count
Command center	Input + history	Palette, autocomplete, favorites, macros
Perception	Sensors	Cameras, OD table, audio, depth grid
Cognition	AI state	MathNN nets, Life RL, curriculum, eval pass rate
Worlds	External sims	Minecraft, SE, RealityConstruct voxels
Math lab	DE/calc/algebra	Catalog runner, trajectory plots, PINN loss
Agents	Runtime	Lua agents, red/blue, terminals per thread
Dev	Diagnostics	Logs, metrics, compile-test launcher, settings

Suggested top-level: dockspace with named layouts: Ops, Train, World, Math, Debug.

Feature ideas by density ROI

A. Command & control (high ROI)

Feature	Fit
Command bar with autocomplete	Fuzzy match SeedCatalog / command_batch verbs; Tab complete
Macro buttons	User-defined rows of de catalog run decay etc.
Command history ring	Up/down + search; re-run; pin
Queue visualizer	CommandQueue depth, in-flight, worker state when threading on
Dry-run toggle	Global “don’t actuate bots” mirror of policy flags

B. Live telemetry (high ROI)

Feature	Fit
Sparkline strip	Loss, reward, residual_rms, FPS, audio level
RSI traffic light	Green/yellow/red from _RsiStabilityMonitor
EvalHarness pass-rate gauge	Last run_label from MySQL or file
Device probe panel	MathNNDevice CPU/GPU/NPU placement
Thread heat map	Which threads wrote last; stuck detection

C. Math lab panels (medium ROI, unique)

Feature	Fit
ODE catalog browser	List models; Run; plot y(t) with ImPlot
Phase portrait	2D state scatter for van der Pol / Lotka
PINN training card	Steps, data loss, residual, checkpoint path
Calc playground	Expr input → derivative/integral numeric table
Algebra matrix pad	Small matrix edit → expm / eig / cond
Sensitivity readout	de sensitivity results as table

Use ImPlot (companion to ImGui) for trajectories; keep samples capped

(e.g. 2k points) from ring buffers filled by worker threads.

D. RL / Life / curriculum

Feature	Fit
Curriculum board	Domains, smoke status, last reward
Life RL dashboard	Focus/energy twin gauges (from catalog twins)
Checkpoint browser	List checkpoints/*.net, load/compare
Experience store peek	MathNNExperience counts by domain
Shadow vs promote	Learning-function pipeline status

E. Worlds & bridges

Feature	Fit
Minecraft bridge card	Port 8765, last status_response fixture fields
SE bridge card	Energy/mass/thrust dry-run buttons
RealityConstruct slice view	2D slice of voxels + DDA ray overlay
Protocol fixture viewer	Load tests/fixtures/** for offline UI demos

F. Perception upgrades

Feature	Fit
Multi-cam tabs	Already partial; add picture-in-picture
OD filter chips	Class / score threshold sliders
Depth occupancy grid	Heatmap from DepthOccupancyEstimator
Spectrogram zoom	Click-to-cue audio buffer
Snapshot gallery	Recent stills with OCR/OD annotations

G. Agent & Lua studio

Feature	Fit
Agent list	Runtime profiles, start/stop, script path
Lua scratch pad	Run snippet with ai_* globals (sandboxed)
Red/blue exercise panel	Scenario picker + log
Plugin panel host	Dynamic ImGui callbacks from plugins (ABI careful)

H. Accessibility & UX density

Feature	Fit
Docking layouts	Save/load layouts/ops.ini
Command-centric search (Ctrl+K)	Single palette for commands, panels, docs
Font scale / high-contrast theme	Operator preference
Notification toast stack	Errors, eval fails, RSI trips
Help drawer	Embed snippets from docs/*.md (truncated)

How to fit without clutter

1. Panel registry

[text]
struct GuiPanel {
id, title, category, default_open, draw_fn, enabled_fn
};

DrawPluginPanelsWindow already leans this way via _UiRuntime::GetPanels() —

extend that registry for first-party panels too.

2. Background workers only

ODE runs, PINN fit, RL smoke → CommandQueue or std::async
UI reads shared_ptr<const Snapshot> updated under mutex
Never block RenderFrame on MySQL or network

3. Progressive disclosure

Collapsed “Advanced” trees for matrix pads / raw JSON
Novice layout: Command + Camera + Status
Expert layout: Math lab + Agents + Bridges

4. ImPlot + tables over free text

Prefer:

ImGui::BeginTable for OD / catalog / eval results
ImPlot::PlotLine for trajectories
Color-coded chips for status

over multi-kilobyte TextUnformatted log dumps (keep those in Terminals).

5. Time budgets

Per frame:

UI logic < 2 ms
Texture upload budget for cameras
Defer spectrogram FFT to audio thread

6. Optional modules

Compile-time:

AI_GUI_MATHLAB — math panels
AI_GUI_WORLDS — MC/SE/RC
AI_GUI_IMPLOT — plot dependency

Keeps Linux/headless builds free of DX11.

Implementation phases (suggested)

Phase	Deliverable
P0	Dockspace + layout save; unified Ctrl+K palette; ops strip (queue, RSI, device)
P1	Command history + macros; EvalHarness pass-rate; checkpoint list
P2	ImPlot math lab (catalog run + plot); PINN card
P3	Bridge cards + fixture viewer; RealityConstruct 2D slice
P4	Agent/Lua scratch; plugin ImGui host; toast notifications
P5	Theme pack, help drawer, accessibility

Risks

Risk	Mitigation
UI thread deadlock with CoutCapture	GUI never holds locks while printing; drain logs async
Stale snapshots	Version counters; “updated Xs ago” labels
Too many windows	Layout presets; max 1 auto-popup
Plugin ABI breaks	C-style draw callback table with version field
GPU texture thrash	Cap camera resolution in UI; throttle refresh

Implementation status (shipped)

Implemented in GuiDashboard.hpp / GuiDashboard.cpp, wired from Gui::RunGui:

Feature	Status
Main menu + layout presets (Ops/Train/World/Math/Debug/All)	Done
Ops strip (RSI, queue, device, eval, model count)	Done
Math lab (catalog plot via ImGui::PlotLines, PINN residual, calc/algebra)	Done
World bridges + protocol fixture viewer	Done
RealityConstruct XZ slice view	Done
Agents / Lua scratch enqueue	Done
Checkpoints browser	Done
Macros + command history	Done
Command queue panel	Done
EvalHarness pass-rate gauge	Done
Learning / experience commands	Done
Theme / font scale / high-contrast	Done
Help drawer	Done
Toast stack	Done
Extended palette verbs (de/calc/algebra/…)	Done
imgui_ai_dashboard.ini layout persistence	Done

Not in this ImGui build: docking branch (ImGuiConfigFlags_DockingEnable). Layouts are presets + free-floating windows instead of a dockspace.

How to use

Launch with –gui (or existing GUI thread path).
View menu → pick a layout.
Ctrl+K → command palette (includes math/de verbs).
Math Lab → pick model → Run catalog integrate (short in-process plot) or Enqueue.
Toasts appear top-right for RSI/eval/math events.

Macdaddy4sure's Blog