Train a program-icon classifier from .exe files for Computer RL and desktop

automation: extract Windows icons → augment → TensorFlow SavedModel + LiteRT

(TFLite).

Why

Computer RL already tracks windows, OCR text, and generic object detections.

Recognizing which app icon is under the cursor or on the taskbar/desktop

gives a stronger app_family / target signal (Chrome vs VS Code vs Explorer).

Pipeline

[text]
scan .exe dirs
    → extract associated icons (Shell32)
    → dedupe identical artwork
    → augment (bg, scale, color, blur)
    → train CNN (default) or MobileNetV2
    → export SavedModel + model.tflite + desktop_icons.names

Script:

[text]
scripts/train_desktop_icon_classifier.py
scripts/train_desktop_icon_classifier.ps1

Train

Default (Local Programs + Program Files, cap 300 EXEs):

[powershell]
cd “E:\_AugmentedIntelligence\_src\_AugmentedIntelligence_v11.0 – Typed Commands”
.\scripts\train_desktop_icon_classifier.ps1

Custom directory:

[powershell]
.\scripts\train_desktop_icon_classifier.ps1 `
  -ScanDir “C:\Program Files”,”C:\Users\Temp\AppData\Local\Programs” `
  -MaxExes 500 `
  -Epochs 20 `
  -AugPerClass 64

Python:

[bat]
python scripts/train_desktop_icon_classifier.py –scan-dir “C:\Program Files” –max-exes 400 –epochs 20

Reuse extracted icons only:

[bat]
python scripts/train_desktop_icon_classifier.py –skip-extract –epochs 25

Outputs

Default root:

[text]
C:\_AugmentedIntelligence\tensorflow_models\desktop_icons\
  icons\                  per-class PNGs (icon_00 + aug_*)
  desktop_icons.names     class labels (line index = class id)
  labels.json             label + source EXE paths
  manifest.json           metrics + export paths
  saved_model\            TF SavedModel (frozen export)
  saved_model_tf\         copy for TF C API path
  model.tflite            LiteRT / TFLite
  model_float32.tflite    LiteRT float32
  keras_model.keras
  history.json
  README.md

Re-export only (if Keras exists but TFLite/SavedModel missing):

[powershell]
python scripts/export_desktop_icon_models.py

Runtime (C++ LiteRT)

DesktopIconClassifier loads model.tflite + desktop_icons.names.

[text]
desktop icon status
desktop icon enable
desktop icon file C:\_AugmentedIntelligence\tensorflow_models\desktop_icons\icons\chrome\icon_00.png
computer rl icon status
computer icon labels 20

Settings:

[text]
desktop_icon_classifier_enabled=true
tflite_desktop_icon_model=C:/_AugmentedIntelligence/tensorflow_models/desktop_icons/model.tflite
tflite_desktop_icon_labels=C:/_AugmentedIntelligence/tensorflow_models/desktop_icons/desktop_icons.names
tensorflow_desktop_icon_model=C:/_AugmentedIntelligence/tensorflow_models/desktop_icons/saved_model_tf/

Inference contract

Field	Value
Input name	image
Shape	[1, 64, 64, 3] (default –image-size 64)
Dtype	float32
Scale	RGB [0, 1]
Output	probs softmax over classes
Labels	desktop_icons.names (one label per line)

Settings paths (see AugmentedIntelligence.cpp):

[text]
tflite_desktop_icon_model   = C:/_AugmentedIntelligence/tensorflow_models/desktop_icons/model.tflite
tflite_desktop_icon_labels  = C:/_AugmentedIntelligence/tensorflow_models/desktop_icons/desktop_icons.names
tensorflow_desktop_icon_model = C:/_AugmentedIntelligence/tensorflow_models/desktop_icons/saved_model_tf/

Computer RL integration notes

1. Crop candidate icon regions from the desktop/taskbar capture (OpenCV).
2. Resize crop → 64×64, convert BGR→RGB, /255.0.
3. Run LiteRT interpreter (same pattern as LoL skin classifier in GamingLoL.cpp).
4. Map argmax → desktop_icons.names[i] → feature or action target label.
5. Prefer dry-run in Computer RL before actuating clicks on recognized icons.

Tips

• Installer/updater EXEs (setup, unins, vcredist, …) are skipped by name.
• Identical icons collapse to one class; extra EXE paths stay in labels.json.
• For more classes, raise –max-exes and re-run (or –skip-extract after a wider extract).
• Use –backbone cnn for 64px icons (default). Use mobilenet only with –image-size 96+.

Safety

Local personal models only. Do not ship third-party proprietary icon artwork

as a public dataset; keep icons/ on the workstation that owns the software.

---

Filed under: Uncategorized - @ July 18, 2026 6:29 am