{"id":1970,"date":"2026-07-18T06:29:51","date_gmt":"2026-07-18T13:29:51","guid":{"rendered":"http:\/\/macdaddy4sure.ai\/?p=1970"},"modified":"2026-07-18T06:29:51","modified_gmt":"2026-07-18T13:29:51","slug":"desktop-icon-vision-computer-desktop-rl","status":"publish","type":"post","link":"http:\/\/macdaddy4sure.ai\/index.php\/2026\/07\/18\/desktop-icon-vision-computer-desktop-rl\/","title":{"rendered":"Desktop Icon Vision (Computer \/ Desktop RL)"},"content":{"rendered":"\n

Train a program-icon classifier<\/strong> from .exe files for Computer RL and desktop<\/p>\n\n\n\n

automation: extract Windows icons \u2192 augment \u2192 TensorFlow SavedModel + LiteRT<\/p>\n\n\n\n

(TFLite).<\/p>\n\n\n\n

Why<\/h2>\n\n\n\n

Computer RL already tracks windows, OCR text, and generic object detections.<\/p>\n\n\n\n

Recognizing which app icon<\/strong> is under the cursor or on the taskbar\/desktop<\/p>\n\n\n\n

gives a stronger app_family \/ target signal (Chrome vs VS Code vs Explorer).<\/p>\n\n\n\n

Pipeline<\/h2>\n\n\n\n

[text]
<\/em>scan .exe dirs
    \u2192 extract associated icons (Shell32)
    \u2192 dedupe identical artwork
    \u2192 augment (bg, scale, color, blur)
    \u2192 train CNN (default) or MobileNetV2
    \u2192 export SavedModel + model.tflite + desktop_icons.names<\/p>\n\n\n\n

Script:<\/p>\n\n\n\n

[text]
<\/em>scripts\/train_desktop_icon_classifier.py
scripts\/train_desktop_icon_classifier.ps1<\/p>\n\n\n\n

Train<\/h2>\n\n\n\n

Default (Local Programs + Program Files, cap 300 EXEs):<\/p>\n\n\n\n

[powershell]
<\/em>cd “E:\\_AugmentedIntelligence\\_src\\_AugmentedIntelligence_v11.0 – Typed Commands”
.\\scripts\\train_desktop_icon_classifier.ps1<\/p>\n\n\n\n

Custom directory:<\/p>\n\n\n\n

[powershell]
<\/em>.\\scripts\\train_desktop_icon_classifier.ps1 `
  -ScanDir “C:\\Program Files”,”C:\\Users\\Temp\\AppData\\Local\\Programs” `
  -MaxExes 500 `
  -Epochs 20 `
  -AugPerClass 64<\/p>\n\n\n\n

Python:<\/p>\n\n\n\n

[bat]
<\/em>python scripts\/train_desktop_icon_classifier.py –scan-dir “C:\\Program Files” –max-exes 400 –epochs 20<\/p>\n\n\n\n

Reuse extracted icons only:<\/p>\n\n\n\n

[bat]
<\/em>python scripts\/train_desktop_icon_classifier.py –skip-extract –epochs 25<\/p>\n\n\n\n

Outputs<\/h2>\n\n\n\n

Default root:<\/p>\n\n\n\n

[text]
<\/em>C:\\_AugmentedIntelligence\\tensorflow_models\\desktop_icons\\
  icons\\                  per-class PNGs (icon_00 + aug_*)
  desktop_icons.names     class labels (line index = class id)
  labels.json             label + source EXE paths
  manifest.json           metrics + export paths
  saved_model\\            TF SavedModel (frozen export)
  saved_model_tf\\         copy for TF C API path
  model.tflite            LiteRT \/ TFLite
  model_float32.tflite    LiteRT float32
  keras_model.keras
  history.json
  README.md<\/p>\n\n\n\n

Re-export only (if Keras exists but TFLite\/SavedModel missing):<\/p>\n\n\n\n

[powershell]
<\/em>python scripts\/export_desktop_icon_models.py<\/p>\n\n\n\n

Runtime (C++ LiteRT)<\/h2>\n\n\n\n

DesktopIconClassifier loads model.tflite + desktop_icons.names.<\/p>\n\n\n\n

[text]
<\/em>desktop icon status
desktop icon enable
desktop icon file C:\\_AugmentedIntelligence\\tensorflow_models\\desktop_icons\\icons\\chrome\\icon_00.png
computer rl icon status
computer icon labels 20<\/p>\n\n\n\n

Settings:<\/p>\n\n\n\n

[text]
<\/em>desktop_icon_classifier_enabled=true
tflite_desktop_icon_model=C:\/_AugmentedIntelligence\/tensorflow_models\/desktop_icons\/model.tflite
tflite_desktop_icon_labels=C:\/_AugmentedIntelligence\/tensorflow_models\/desktop_icons\/desktop_icons.names
tensorflow_desktop_icon_model=C:\/_AugmentedIntelligence\/tensorflow_models\/desktop_icons\/saved_model_tf\/<\/p>\n\n\n\n

Inference contract<\/h2>\n\n\n\n
Field<\/strong><\/td>Value<\/strong><\/td><\/tr>
Input name<\/td>image<\/td><\/tr>
Shape<\/td>[1, 64, 64, 3] (default –image-size 64)<\/td><\/tr>
Dtype<\/td>float32<\/td><\/tr>
Scale<\/td>RGB [0, 1]<\/strong><\/td><\/tr>
Output<\/td>probs softmax over classes<\/td><\/tr>
Labels<\/td>desktop_icons.names (one label per line)<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

Settings paths (see AugmentedIntelligence.cpp):<\/p>\n\n\n\n

[text]
<\/em>tflite_desktop_icon_model   = C:\/_AugmentedIntelligence\/tensorflow_models\/desktop_icons\/model.tflite
tflite_desktop_icon_labels  = C:\/_AugmentedIntelligence\/tensorflow_models\/desktop_icons\/desktop_icons.names
tensorflow_desktop_icon_model = C:\/_AugmentedIntelligence\/tensorflow_models\/desktop_icons\/saved_model_tf\/<\/p>\n\n\n\n

Computer RL integration notes<\/h2>\n\n\n\n
    \n
  1. Crop candidate icon regions from the desktop\/taskbar capture (OpenCV).<\/li>\n\n\n\n
  2. Resize crop \u2192 64\u00d764, convert BGR\u2192RGB, \/255.0.<\/li>\n\n\n\n
  3. Run LiteRT interpreter (same pattern as LoL skin classifier in GamingLoL.cpp).<\/li>\n\n\n\n
  4. Map argmax \u2192 desktop_icons.names[i] \u2192 feature or action target label.<\/li>\n\n\n\n
  5. Prefer dry-run<\/strong> in Computer RL before actuating clicks on recognized icons.<\/li>\n<\/ol>\n\n\n\n

    Tips<\/h2>\n\n\n\n
      \n
    • Installer\/updater EXEs (setup, unins, vcredist, \u2026) are skipped by name.<\/li>\n\n\n\n
    • Identical icons collapse to one class; extra EXE paths stay in labels.json.<\/li>\n\n\n\n
    • For more classes, raise –max-exes and re-run (or –skip-extract after a wider extract).<\/li>\n\n\n\n
    • Use –backbone cnn for 64px icons (default). Use mobilenet only with –image-size 96+.<\/li>\n<\/ul>\n\n\n\n

      Safety<\/h2>\n\n\n\n

      Local personal models only. Do not ship third-party proprietary icon artwork<\/p>\n\n\n\n

      as a public dataset; keep icons\/ on the workstation that owns the software.<\/p>\n","protected":false},"excerpt":{"rendered":"

      Train a program-icon classifier from .exe files for Computer RL and desktop automation: extract Windows icons \u2192 augment \u2192 TensorFlow SavedModel + LiteRT (TFLite). Why Computer RL already tracks windows, OCR text, and generic object detections. Recognizing which app icon is under the cursor or on the taskbar\/desktop gives a stronger app_family \/ target signal […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-1970","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"http:\/\/macdaddy4sure.ai\/index.php\/wp-json\/wp\/v2\/posts\/1970","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/macdaddy4sure.ai\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/macdaddy4sure.ai\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/macdaddy4sure.ai\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/macdaddy4sure.ai\/index.php\/wp-json\/wp\/v2\/comments?post=1970"}],"version-history":[{"count":1,"href":"http:\/\/macdaddy4sure.ai\/index.php\/wp-json\/wp\/v2\/posts\/1970\/revisions"}],"predecessor-version":[{"id":1971,"href":"http:\/\/macdaddy4sure.ai\/index.php\/wp-json\/wp\/v2\/posts\/1970\/revisions\/1971"}],"wp:attachment":[{"href":"http:\/\/macdaddy4sure.ai\/index.php\/wp-json\/wp\/v2\/media?parent=1970"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/macdaddy4sure.ai\/index.php\/wp-json\/wp\/v2\/categories?post=1970"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/macdaddy4sure.ai\/index.php\/wp-json\/wp\/v2\/tags?post=1970"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}