_AugmentedIntelligence Minecraft Integration
_AugmentedIntelligence_ integrates with Minecraft Java Edition servers in two ways:
- The preferred server deployment is the Spigot plugin bridge in minecraft_spigot_plugin/.
- The legacy fallback is RCON.
Documentation map:
[text]
docs/MinecraftPluginOperations.md = quick operator reference, config, commands, troubleshooting
docs/MinecraftPluginTrainingGuide.md = full DQN/client-control/MathNN training curriculum
docs/ReinforcementLearning.md = RL hub (MathNN, curriculum, Life RL, Minecraft)
docs/MathNN.md = in-process neural policies (`rl net`, learners)
docs/GameModBridges.md = SourceMod/Quake FPS bridge setup
minecraft_spigot_plugin/README.md = plugin build/deploy quick reference
The Spigot bridge streams structured observations and events to _AugmentedIntelligence, then receives safe allowlisted actions for Minecraft DQN and server automation. This is better than RCON for a headless server because the app receives typed player/world state instead of parsing command output.
Spigot Plugin Bridge
Build the plugin:
[powershell]
cd minecraft_spigot_plugin
gradle jar
Copy build/libs/AugmentedIntelligenceSpigotBridge-2.0.0.jar into the Minecraft server plugins/ directory and restart the server. Edit plugins/AugmentedIntelligenceBridge/config.yml so the token matches _AugmentedIntelligence.
Start the _AugmentedIntelligence bridge:
[text]
minecraft bridge configure 0.0.0.0 8765 change-me true
minecraft bridge start
minecraft bridge status
For the current EARTH/JUPITER lab, the plugin on JUPITER should connect to
EARTH:
[yaml]
bridge:
host: “10.0.0.151”
port: 8765
token: “change-me”
Plugin commands:
[text]
/ai help
/ai status
/ai connect
/ai disconnect
/ai reload
/ai observe
/ai monitor
/ai remote <_AI typed command>
/ai debug [player]
/ai explain [player]
/ai action 1 @p
/ai dryrun on
/ai chat status
/ai chat on
/ai chat scope private
/ai chat transcript [player] 12
/ai safety on
/ai movement status
/ai goal list
/ai arena status
/ai replay status
/ai metrics status
/ai ml status
/ai ml systems
/ai ml train replay
/ai kit miner
/ai path
/ai qvalues
/ai integrations status
The plugin sends JSON-line messages over TCP:
[json]
{“type”:”observation”,”player”:”Steve”,”x”:0.0,”y”:64.0,”z”:0.0,”health”:20.0,”food”:20,”nearby_hostiles”:1}
{“type”:”event”,”event”:”block_break”,”player”:”Steve”,”reward”:0.03,”terminal”:false}
{“type”:”remote_command”,”sender”:”Steve”,”player”:”Steve”,”command”:”minecraft dqn status”,”source”:”spigot_admin”}
When plugin integrations are enabled, observations and events also include fields such as:
[json]
{“plugin_integrations_plugins”:”Vault,WorldGuard,Quests”,”plugin_integrations_capabilities”:”economy,permissions,regions,quests”,”plugin_vault”:true,”plugin_worldguard”:true,”integration_quests_present”:true}
Use these from _AI as high-level context: economy state may matter for Jobs/Vault goals, WorldGuard/claim plugins explain build restrictions, Citizens/MythicMobs can signal NPC or custom mob behavior, and Quests/Towny/Factions can define social or objective-driven rewards. The Spigot plugin only detects and reports these integrations; heavy policy learning still runs in _AI.
Stock market + federal government (built-in, not softdeps): see docs/MinecraftMarketAndGovernment.md and docs/ThreeBranches.md.
Commands: /ai market …, /ai gov … on the server; minecraft market …, minecraft gov … on _AI.
Three branches: legislative (vote/conference), executive (sign/veto), judicial (judicial_review/court).
Multi-agent civics: lua_scripts/MC-Federal-Government-Agents.lua (agent type mc_federal).
Jobs Reborn: soft API via JobsService — /ai jobs status|list|player, bridge type:jobs, events jobs_payment / jobs_levelup.
Faction ideologies: /ai ideology list|set|get|can_claim|can_war — kits in faction_ideologies.yml (see docs/FactionIdeologies.md).
The app sends safe action messages back:
[json]
{“type”:”action”,”action_id”:6,”label”:”smooth_sprint_forward”,”player”:”@p”,”source”:”minecraft_dqn”}
Chat uses the same JSON-lines bridge. The plugin sends rich chat prompts:
[json]
{“type”:”chat”,”player”:”Steve”,”message”:”what should I do next?”,”scope”:”private”,”reply_scope”:”private”,”intent”:”chat”,”urgency”:”normal”,”active_goal”:”survive_60″,”selected_item”:”stone_pickaxe”,”nearby_hostiles”:1}
_AI sends replies back with delivery metadata:
[json]
{“type”:”chat”,”target”:”Steve”,”message”:”Get food first, then return to your lit base path.”,”scope”:”private”,”urgency”:”normal”,”source”:”minecraft_llm”}
The plugin sends delivery results back to _AI:
[json]
{“type”:”chat_result”,”id”:”1″,”ok”:true,”message”:”ok”,”target”:”Steve”,”scope”:”private”,”delivered”:1}
Memory/control messages are explicit:
[json]
{“type”:”chat_control”,”command”:”clear_memory”,”player”:”Steve”}
Admin remote commands let an operator run _AI typed commands from Minecraft:
[text]
/ai remote minecraft bridge status
/ai remote minecraft dqn status
/ai remote minecraft bridge chat say @a hello from _AI
minecraft bridge remote status
minecraft bridge remote off
minecraft bridge remote on
The plugin gates /ai remote behind augmentedintelligence.admin and, by default, operator status. _AI also has its own bridge-side switch: minecraft bridge remote off rejects remote command envelopes even if the plugin sends them.
Safe actions are intentionally allowlisted. DQN action IDs 0 through 25 cover observe, smooth movement, jump, sprint, turning, target-block breaking, target-block placement, hotbar selection, jump movement, diagonal movement, and look up/down. Utility/admin actions use high IDs: 90 retreat, 91 extinguish, 92 return_to_base, 93 coach_ping, 94 clear_weather, 95 set_day, and 96 save_world.
The plugin also supports action IDs 26 through 43 for server-side survival
helpers: sneak, swim up/down, eat food, select best tool, place torch, equip
shield, use item, open/close door, look at nearby hostile/item/player, pick up
nearest item, craft basic tools, smelt ore, sleep at night, return safe zone,
and return to arena center. Actions 11 through 19 are client hotbar actions
and only have an effect when _AI is controlling the local Minecraft client.
The plugin can also forward explicit player chat prompts to _AugmentedIntelligence and receive LLM chat replies. By default, only messages that start with !ai or mention @ai are forwarded. Chat payloads include player state, active goal, arena, safety, nearby players/hostiles, nearest memory, preference, and recent transcript:
[text]
!ai what should I do next?
/ai chat scope private
/ai chat ask how do I get unstuck?
/ai chat transcript [player] 12
minecraft bridge chat status
minecraft bridge chat metrics
minecraft bridge chat cooldown 3
minecraft bridge chat system survival_coach
minecraft bridge chat say @a hello from _AI
The plugin also includes lightweight server-side ML telemetry:
[text]
/ai ml behavior [player]
/ai ml bc [player]
/ai ml reward [player]
/ai ml stuck [player]
/ai ml nav [player]
/ai ml planner [player]
/ai ml player [player]
/ai ml roles
/ai ml anomaly
/ai ml grief [player]
/ai ml coach [player]
/ai ml death [player]
/ai ml memory [player]
/ai ml quest [player]
/ai ml council [player]
/ai ml export
These commands summarize replay data, demonstrations, reward feedback, stuck states, navigability, player intent, cooperative roles, anomaly/grief signals, session coaching context, and death reviews. Heavy neural learning remains in _AugmentedIntelligence: /ai ml mathnn [player] and /ai ml vision request [player] send bridge requests for MathNN training/checkpoint work or client-side screenshot/OCR/object perception.
Plugin integration operations:
[text]
/ai integrations status
/ai integrations scan
/ai integrations list
/ai integrations payload [player]
/ai integrations observe on|off
/ai integrations events on|off
/ai integrations commands on|off
/ai integrations args on|off
Player plugin command observation is off by default. Enable it when you want _AI to learn from command-level activity around quests, jobs, towns, factions, NPCs, claims, regions, plots, islands, warps, and homes. Leave arguments off unless you intentionally want command text captured.
Server Setup
RCON remains available as a fallback. Edit the Minecraft server server.properties file:
[properties]
enable-rcon=true
rcon.port=25575
rcon.password=replace-with-a-strong-password
Restart the Minecraft server after changing those values. Keep the RCON port firewalled; RCON is an admin control channel.
If the Spigot plugin is installed and connected, use the bridge for DQN
observations/actions and keep RCON as an emergency/admin fallback.
Configure _AugmentedIntelligence
From the typed command prompt:
[text]
minecraft configure 127.0.0.1 25575 replace-with-a-strong-password
That writes minecraft_rcon_config.local.txt in the working directory. You can also use environment variables:
[powershell]
$env:AI_MINECRAFT_RCON_HOST = “127.0.0.1”
$env:AI_MINECRAFT_RCON_PORT = “25575”
$env:AI_MINECRAFT_RCON_PASSWORD = “replace-with-a-strong-password”
The Spigot bridge writes/reads:
[text]
minecraft_spigot_bridge.local.txt
Useful bridge commands:
[text]
minecraft bridge configure 0.0.0.0 8765 YOUR_LONG_SHARED_TOKEN true
minecraft bridge start
minecraft bridge status
minecraft bridge chat status
minecraft bridge chat metrics
minecraft bridge chat clear [player]
Commands
[text]
minecraft status
minecraft test
minecraft list
minecraft seed
minecraft say hello from AugmentedIntelligence
minecraft tell PlayerName check the north gate
minecraft time set day
minecraft weather clear
minecraft gamemode creative PlayerName
minecraft tp PlayerName 0 80 0
minecraft rcon difficulty normal
minecraft script minecraft_commands.txt
Minecraft DQN
The Minecraft DQN module prefers the Spigot bridge when connected and falls back to RCON when it is not. It can also play through the local Minecraft client by sending keyboard/mouse input with minecraft dqn control client.
[text]
minecraft bridge start
minecraft dqn status
minecraft dqn player @p
minecraft dqn observe
minecraft dqn control client
minecraft dqn client status
minecraft dqn client focus on
minecraft dqn client hold 900
minecraft dqn client mine-hold 1800
minecraft dqn client use-hold 350
minecraft dqn client mine
minecraft dqn client place
minecraft dqn client slot 1
minecraft dqn client look-up
minecraft dqn unstuck on
minecraft dqn wall-guard on
minecraft dqn train 1000
minecraft dqn dry-run
minecraft dqn qvalues
minecraft dqn explain 10
minecraft dqn step
minecraft dqn interval 2000
minecraft dqn reward 0.5
minecraft dqn reward -1 terminal
minecraft dqn net on
minecraft dqn net status
minecraft dqn save
minecraft dqn load
dry-run prints the selected action without sending it to the server. step observes the current state, chooses an action, applies it through the Spigot bridge when connected or RCON as fallback, observes the next state, computes reward, and trains only when the mode is train.
train [interval_ms] sets DQN mode to training and starts an autonomous background loop that repeatedly runs step. Use minecraft dqn off, minecraft dqn eval, or minecraft dqn observe to stop the background loop. The default interval is 1000 ms, and minecraft dqn interval <milliseconds> changes the delay between autonomous steps.
By default, DQN actions are sent through the server bridge/RCON. To make the DQN play through the Minecraft client running on the _AI machine, use minecraft dqn control client. Client control sends short keyboard/mouse actions to a foreground Minecraft window while still using the Spigot bridge for observations and rewards. Use minecraft dqn client focus on if you want _AI to bring the Minecraft window to the foreground before sending input.
minecraft dqn client hold <milliseconds> controls how long movement keys stay down. Sprint holds W plus left control for that duration. minecraft dqn client mine-hold <milliseconds> controls how long left mouse is held for the client_mine_hold action, so the bot can actually break blocks instead of only tapping attack. minecraft dqn client use-hold <milliseconds> controls how long right mouse is held for the client_place_hold action. Use minecraft dqn client mine or minecraft dqn client action 9 to manually test mining, minecraft dqn client place or minecraft dqn client action 10 to manually test right-click block placement/use, and minecraft dqn client slot <1-9> to test hotbar selection. minecraft dqn client release forces held movement or mouse input up.
For server-driven movement, the plugin can smooth movement actions instead of teleporting the player one block at a time:
[text]
/ai movement status
/ai movement smooth on
/ai movement ticks 8
/ai movement speed 0.28
Additional plugin-side controls:
[text]
/ai follow <target-player> [follower]
/ai hud on
/ai safety build-zone on
/ai replay file
/ai kit miner
/ai path
/ai route base mine
/ai mob spawn zombie 1
/ai curriculum start flatwalk
/ai teach good
/ai metrics actions
/ai ml planner
/ai ml stuck
/ai kit miner
/ai path
/ai arena build <name> hallway
/ai arena build <name> wall
/ai arena build <name> steps
/ai arena build <name> bridge
/ai arena build <name> pit
/ai arena build <name> obstacle
/ai arena build <name> random
/ai model list
/ai model preview <name> [scale] [material]
/ai model build <name> [scale] [material]
/ai model status
/ai model cancel
The bridge observation now includes target block, target entity, placeable-face, hotbar slot, selected item count, inventory tool flags, food/block availability, wood/cobblestone/torch counts, armor, held-item durability, weather, path obstruction reasons, yaw/pitch, and local block-grid signals. Old DQN checkpoints with smaller state/action counts can still load; new weights start untrained.
OBJ/MTL Model Building
The Spigot plugin can build a Wavefront .obj mesh, with optional .mtl
diffuse colors, directly into the Minecraft world. Copy files to the server:
[text]
plugins/AugmentedIntelligenceBridge/models/model_name.obj
plugins/AugmentedIntelligenceBridge/models/model_name.mtl
Enable the feature in plugins/AugmentedIntelligenceBridge/config.yml only when
you are ready to place blocks:
[yaml]
model-build:
allow-world-edit: true
max-blocks: 25000
max-dimension: 256
blocks-per-tick: 500
Then run from Minecraft:
[text]
/ai reload
/ai model list
/ai model preview model_name 1.0
/ai model build model_name 1.0
The build is centered on the executing player, starts at the player’s Y level,
and places blocks over multiple ticks. Use
/ai model build model_name 1.5 stone to override all .mtl colors with one
block type. Always run preview before build on large models.
The recommended training order is:
[text]
1. verify bridge observations and replay logging
2. manually test client movement, looking, mining, placing, and hotbar selection
3. train flat arena locomotion with single steps first
4. train wall and hallway recovery with wall_guard and unstuck enabled
5. train steps, pits, target-block mining, then block placement
6. train resource goals such as collect_wood and mine_iron
7. add survival pressure, return-to-base, follow-guided data, and mixed obstacle arenas
8. move to short open-survival trials only after arena behavior is stable
For the detailed curriculum with promotion criteria and rollback rules, see docs/MinecraftPluginTrainingGuide.md.
For the concise operational command reference, see docs/MinecraftPluginOperations.md.
minecraft dqn wall-guard on prevents client movement actions from being selected when the plugin reports the matching direction as blocked, such as blocked_forward=true.
Training transitions are appended to minecraft_dqn_transitions.csv under gaming_rl_log_directory for offline review. minecraft dqn net on enables the shared in-process neural DQN learner for the Minecraft agent; without it, the built-in compact DQN model is used.
Dangerous raw commands require explicit confirmation:
[text]
minecraft confirm stop
minecraft confirm op PlayerName
minecraft script confirm admin_commands.txt
Aliases:
[text]
mc status
gaming minecraft list
Script Files
A script file is one RCON command per line. Blank lines and lines starting with # are ignored.
[text]
# minecraft_commands.txt
say [AI] starting maintenance window
time set day
weather clear
save-all
Filed under: Uncategorized - @ July 18, 2026 6:43 am