titor/mimiclaw
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: update for ReAct tool use, web_search, and build-time config

Browse Source 
Update READMEs with config file setup (Option A/B), tool section,
set_search_key command, and touch-before-build note. Update
ARCHITECTURE.md with ReAct data flow, tools module map, non-streaming
API protocol, and config priority. Mark tool use items done in TODO.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
crispyberry
2026-02-07 00:37:49 +08:00
parent 0e1da79b74
commit e04254fa94
4 changed files with 271 additions and 146 deletions
Download Patch File Download Diff File Expand all files Collapse all files

191
docs/ARCHITECTURE.md
View File

@@ -20,41 +20,46 @@ Telegram App (User)
│ │ Poller │ └────────┬─────────┘ │
│ │ (Core 0) │ │ │
│ └─────────────┘ ▼ │
│ ┌──────────────┐ │
│ ┌─────────────┐ │ Agent Loop │ │
│ │ WebSocket │──────▶│ (Core 1) │ │
│ │ Server │ │ │ │
│ │ (:18789) │ │ Context ──▶ LLM Proxy │
│ └─────────────┘ │ Builder (HTTPS) │
│ └──────┬───────┘ │
│ ┌─────────────┐ │ │
│ │ Serial CLI │ ▼ │
│ │ (Core 0) │ ┌──────────────┐ │
│ └─────────────┘ │ Outbound Queue│ │
│ └──────┬───────┘ │
│ │ │
│ ┌──────▼───────┐ │
│ │ Outbound │ │
│ │ Dispatch │ │
│ │ (Core 0) │ │
│ └──┬────────┬──┘ │
│ │ │ │
│ Telegram WebSocket │
│ sendMessage send │
│ │
│ ┌──────────────────────────────────────────┐ │
│ │ SPIFFS (12 MB) │ │
│ │ /spiffs/config/ SOUL.md, USER.md │ │
│ │ /spiffs/memory/ MEMORY.md, YYYY-MM-DD │ │
│ │ /spiffs/sessions/ tg_<chat_id>.jsonl │ │
│ └──────────────────────────────────────────┘ │
└──────────────────────────────────────────────────┘
│ ┌────────────────────────┐ │
│ ┌─────────────┐ │ Agent Loop │ │
│ │ WebSocket │─▶│ (Core 1) │ │
│ │ Server │ │ │ │
│ │ (:18789) │ │ Context ──▶ LLM Proxy │ │
│ └─────────────┘ │ Builder (HTTPS) │ │
│ │ ▲ │ │ │
│ ┌─────────────┐ │ │ tool_use? │ │
│ │ Serial CLI │ │ │ ▼ │ │
│ │ (Core 0) │ │ Tool Results ◀─ Tools │ │
│ └─────────────┘ │ (web_search)│ │
│ └──────────┬─────────────┘ │
│ │ │
│ ┌──────▼───────┐ │
│ │ Outbound Queue│ │
│ └──────┬───────┘ │
│ │ │
│ ┌──────▼───────┐ │
│ │ Outbound │ │
│ │ Dispatch │ │
│ │ (Core 0) │ │
│ └──┬────────┬──┘ │
│ │ │ │
│ Telegram WebSocket │
│ sendMessage send │
│ │
│ ┌──────────────────────────────────────────┐ │
│ │ SPIFFS (12 MB) │ │
│ │ /spiffs/config/ SOUL.md, USER.md │ │
│ │ /spiffs/memory/ MEMORY.md, YYYY-MM-DD │ │
│ │ /spiffs/sessions/ tg_<chat_id>.jsonl │ │
│ └──────────────────────────────────────────┘ │
└───────────────────────────────────────────────────┘
│
│ Anthropic Messages API (HTTPS + SSE)
│ Anthropic Messages API (HTTPS)
│ + Brave Search API (HTTPS)
▼
┌───────────┐
│ Claude API │
└───────────┘
┌───────────┐ ┌──────────────┐
│ Claude API │ │ Brave Search │
└───────────┘ └──────────────┘
```
---
@@ -67,12 +72,18 @@ Telegram App (User)
3. Message pushed to Inbound Queue (FreeRTOS xQueue)
4. Agent Loop (Core 1) pops message:
a. Load session history from SPIFFS (JSONL)
b. Build system prompt (SOUL.md + USER.md + MEMORY.md + recent notes)
c. Build messages array (history + current message)
d. Call Claude API via HTTPS (SSE streaming)
e. Accumulate streamed response tokens
f. Save user + assistant messages to session file
g. Push response to Outbound Queue
b. Build system prompt (SOUL.md + USER.md + MEMORY.md + recent notes + tool guidance)
c. Build cJSON messages array (history + current message)
d. ReAct loop (max 10 iterations):
i. Call Claude API via HTTPS (non-streaming, with tools array)
ii. Parse JSON response → text blocks + tool_use blocks
iii. If stop_reason == "tool_use":
- Execute each tool (e.g. web_search → Brave Search API)
- Append assistant content + tool_result to messages
- Continue loop
iv. If stop_reason == "end_turn": break with final text
e. Save user message + final assistant text to session file
f. Push response to Outbound Queue
5. Outbound Dispatch (Core 0) pops response:
a. Route by channel field ("telegram" → sendMessage, "websocket" → WS frame)
6. User receives reply
@@ -85,7 +96,9 @@ Telegram App (User)
```
main/
├── mimi.c Entry point — app_main() orchestrates init + startup
├── mimi_config.h All compile-time constants in one place
├── mimi_config.h All compile-time constants + build-time secrets include
├── mimi_secrets.h Build-time credentials (gitignored, highest priority)
├── mimi_secrets.h.example Template for mimi_secrets.h
│
├── bus/
│ ├── message_bus.h mimi_msg_t struct, queue API
@@ -100,14 +113,20 @@ main/
│ └── telegram_bot.c Long polling loop, JSON parsing, message splitting
│
├── llm/
│ ├── llm_proxy.h llm_chat() API
│ └── llm_proxy.c Anthropic Messages API, SSE stream parser
│ ├── llm_proxy.h llm_chat() + llm_chat_tools() API, tool_use types
│ └── llm_proxy.c Anthropic Messages API (non-streaming), tool_use parsing
│
├── agent/
│ ├── agent_loop.h Agent task init/start
│ ├── agent_loop.c Main processing loop: inbound → context → LLM → outbound
│ ├── agent_loop.c ReAct loop: LLM call → tool execution → repeat
│ ├── context_builder.h System prompt + messages builder API
│ └── context_builder.c Reads bootstrap files + memory, assembles prompt
│ └── context_builder.c Reads bootstrap files + memory + tool guidance
│
├── tools/
│ ├── tool_registry.h Tool definition struct, register/dispatch API
│ ├── tool_registry.c Tool registration, JSON schema builder, dispatch by name
│ ├── tool_web_search.h Web search tool API
│ └── tool_web_search.c Brave Search API via HTTPS (direct + proxy)
│
├── memory/
│ ├── memory_store.h Long-term + daily memory API
@@ -125,7 +144,7 @@ main/
│
├── cli/
│ ├── serial_cli.h CLI init API
│ └── serial_cli.c esp_console REPL with 14 commands
│ └── serial_cli.c esp_console REPL with 15 commands
│
└── ota/
├── ota_manager.h OTA update API
@@ -206,17 +225,20 @@ Session files are JSONL (one JSON object per line):
## NVS Configuration
| Namespace | Key | Description |
|---------------|--------------|-----------------------------------------|
| `wifi_config` | `ssid` | WiFi SSID |
| `wifi_config` | `password` | WiFi password |
| `tg_config` | `bot_token` | Telegram Bot API token |
| `llm_config` | `api_key` | Anthropic API key |
| `llm_config` | `model` | Model ID (default: claude-opus-4-6) |
| `proxy_config`| `host` | HTTP proxy hostname/IP |
| `proxy_config`| `port` | HTTP proxy port |
| Namespace | Key | Description |
|-----------------|--------------|-----------------------------------------|
| `wifi_config` | `ssid` | WiFi SSID |
| `wifi_config` | `password` | WiFi password |
| `tg_config` | `bot_token` | Telegram Bot API token |
| `llm_config` | `api_key` | Anthropic API key |
| `llm_config` | `model` | Model ID (default: claude-opus-4-6) |
| `proxy_config` | `host` | HTTP proxy hostname/IP |
| `proxy_config` | `port` | HTTP proxy port |
| `search_config` | `api_key` | Brave Search API key |
All configured via Serial CLI commands: `wifi_set`, `set_tg_token`, `set_api_key`, `set_model`, `set_proxy`, `clear_proxy`.
**Configuration priority**: `mimi_secrets.h` (build-time) > NVS (CLI-set) > defaults.
All configurable via Serial CLI or build-time config file (`mimi_secrets.h`).
---
@@ -260,33 +282,50 @@ Client `chat_id` is auto-assigned on connection (`ws_<fd>`) but can be overridde
Endpoint: `POST https://api.anthropic.com/v1/messages`
Request format (Anthropic-native, not OpenAI):
Request format (Anthropic-native, non-streaming, with tools):
```json
{
"model": "claude-opus-4-6",
"max_tokens": 4096,
"stream": true,
"system": "<system prompt>",
"tools": [
{
"name": "web_search",
"description": "Search the web for current information.",
"input_schema": {"type": "object", "properties": {"query": {"type": "string"}}, "required": ["query"]}
}
],
"messages": [
{"role": "user", "content": "Hello"},
{"role": "assistant", "content": "Hi!"},
{"role": "user", "content": "How are you?"}
{"role": "user", "content": "What's the weather today?"}
]
}
```
Key difference from OpenAI: `system` is a top-level field, not inside the `messages` array.
SSE streaming response events:
```
event: content_block_delta
data: {"type":"content_block_delta","delta":{"type":"text_delta","text":"Hello"}}
event: message_stop
data: {"type":"message_stop"}
Non-streaming JSON response:
```json
{
"id": "msg_xxx",
"type": "message",
"role": "assistant",
"content": [
{"type": "text", "text": "Let me search for that."},
{"type": "tool_use", "id": "toolu_xxx", "name": "web_search", "input": {"query": "weather today"}}
],
"stop_reason": "tool_use"
}
```
The SSE parser in `llm_proxy.c` accumulates `text_delta` tokens into a response buffer.
When `stop_reason` is `"tool_use"`, the agent loop executes each tool and sends results back:
```json
{"role": "assistant", "content": [<text + tool_use blocks>]}
{"role": "user", "content": [{"type": "tool_result", "tool_use_id": "toolu_xxx", "content": "..."}]}
```
The loop repeats until `stop_reason` is `"end_turn"` (max 10 iterations).
---
@@ -301,13 +340,14 @@ app_main()
├── memory_store_init() Verify SPIFFS paths
├── session_mgr_init()
├── wifi_manager_init() Init WiFi STA mode + event handlers
├── http_proxy_init() Load proxy config from NVS
├── telegram_bot_init() Load bot token from NVS
├── llm_proxy_init() Load API key + model from NVS
├── http_proxy_init() Load proxy config (secrets > NVS)
├── telegram_bot_init() Load bot token (secrets > NVS)
├── llm_proxy_init() Load API key + model (secrets > NVS)
├── tool_registry_init() Register tools, build tools JSON
├── agent_loop_init()
├── serial_cli_init() Start REPL (works without WiFi)
│
├── wifi_manager_start() Connect using NVS credentials
├── wifi_manager_start() Connect (secrets > NVS credentials)
│ └── wifi_manager_wait_connected(30s)
│
└── [if WiFi connected]
@@ -330,6 +370,7 @@ If WiFi credentials are missing or connection times out, the CLI remains availab
| `set_tg_token <TOKEN>` | Save Telegram bot token |
| `set_api_key <KEY>` | Save Anthropic API key |
| `set_model <MODEL_ID>` | Set LLM model identifier |
| `set_search_key <KEY>` | Save Brave Search API key |
| `set_proxy <HOST> <PORT>` | Set HTTP CONNECT proxy |
| `clear_proxy` | Remove proxy, use direct connection |
| `memory_read` | Print MEMORY.md contents |
@@ -340,22 +381,24 @@ If WiFi credentials are missing or connection times out, the CLI remains availab
| `restart` | Reboot the device |
| `help` | List all available commands |
> **Note**: CLI-set values are stored in NVS but are overridden by `mimi_secrets.h` build-time values if set.
---
## Nanobot Reference Mapping
| Nanobot Module | MimiClaw Equivalent | Notes |
|-----------------------------|--------------------------------|------------------------------|
| `agent/loop.py` | `agent/agent_loop.c` | Simplified: no tool use loop |
| `agent/context.py` | `agent/context_builder.c` | Loads SOUL.md + USER.md + memory |
| `agent/loop.py` | `agent/agent_loop.c` | ReAct loop with tool use |
| `agent/context.py` | `agent/context_builder.c` | Loads SOUL.md + USER.md + memory + tool guidance |
| `agent/memory.py` | `memory/memory_store.c` | MEMORY.md + daily notes |
| `session/manager.py` | `memory/session_mgr.c` | JSONL per chat, ring buffer |
| `channels/telegram.py` | `telegram/telegram_bot.c` | Raw HTTP, no python-telegram-bot |
| `bus/events.py` + `queue.py`| `bus/message_bus.c` | FreeRTOS queues vs asyncio |
| `providers/litellm_provider.py` | `llm/llm_proxy.c` | Direct Anthropic API only |
| `config/schema.py` | `mimi_config.h` + NVS | Compile-time + NVS storage |
| `config/schema.py` | `mimi_config.h` + `mimi_secrets.h` + NVS | Build-time secrets > NVS |
| `cli/commands.py` | `cli/serial_cli.c` | esp_console REPL |
| `agent/tools/*` | *(not yet implemented)* | See TODO.md |
| `agent/tools/*` | `tools/tool_registry.c` + `tool_web_search.c` | web_search via Brave API |
| `agent/subagent.py` | *(not yet implemented)* | See TODO.md |
| `agent/skills.py` | *(not yet implemented)* | See TODO.md |
| `cron/service.py` | *(not yet implemented)* | See TODO.md |

71
docs/TODO.md
View File

@@ -7,11 +7,8 @@
## P0 — Core Agent Capabilities
### [ ] Tool Use Loop (multi-turn agent iteration)
- **nanobot**: `loop.py` L167-210 — while loop calls LLM, checks `response.has_tool_calls`, executes tools, feeds results back into messages, repeats until LLM stops calling tools (max 20 iterations)
- **MimiClaw**: `agent_loop.c` only makes a single LLM call (one-shot), cannot use any tools
- **Scope**: Need to parse Anthropic API `tool_use` content blocks, implement tool execution loop
- **Note**: Anthropic tool_use format differs from OpenAI — uses content blocks, not function_call
### [x] ~~Tool Use Loop (multi-turn agent iteration)~~
- Implemented: `agent_loop.c` ReAct loop with `llm_chat_tools()`, max 10 iterations, non-streaming JSON parsing
### [ ] Memory Write via Tool Use (agent-driven memory persistence)
- **openclaw**: Agent uses standard `write`/`edit` tools to write `MEMORY.md` and `memory/YYYY-MM-DD.md`; system prompt instructs agent to persist important information; pre-compaction memory flush triggers a silent agent turn to save durable memories before context window limit
@@ -19,20 +16,13 @@
- **Scope**: Expose `memory_write` and `memory_append_today` as tool_use tools for Claude; add system prompt guidance on when to persist memory; optionally add pre-compaction flush (trigger memory save when session history nears `MIMI_SESSION_MAX_MSGS`)
- **Depends on**: Tool Use Loop
### [ ] Tool Registry + Built-in Tools
- **nanobot**: `tools/registry.py` — dynamic tool registration/execution, `tools/base.py` defines abstract Tool base class
- **nanobot built-in tools**:
- `read_file` — read files (`tools/filesystem.py`)
- `write_file` — write files
- `edit_file` — edit files
- `list_dir` — list directory
- `exec` — execute shell commands (`tools/shell.py`)
- `web_search` — web search (`tools/web.py`)
- `web_fetch` — fetch web pages
- `message` — send message to user (`tools/message.py`)
- `spawn` — launch subagent (`tools/spawn.py`)
- **MimiClaw**: No tool system at all
- **Recommendation**: Reasonable tool subset for ESP32: `read_file`, `write_file`, `list_dir` (SPIFFS), `message`. Shell/web not suitable for MCU
### [x] ~~Tool Registry + web_search Tool~~
- Implemented: `tools/tool_registry.c` — tool registration, JSON schema builder, dispatch by name
- Implemented: `tools/tool_web_search.c` — Brave Search API via HTTPS (direct + proxy support)
### [ ] More Built-in Tools
- **nanobot built-in tools** not yet ported: `read_file`, `write_file`, `edit_file`, `list_dir`, `message`
- **Recommendation**: Reasonable tool subset for ESP32: `read_file`, `write_file`, `list_dir` (SPIFFS), `message`, `memory_write`
### [ ] Subagent / Spawn Background Tasks
- **nanobot**: `subagent.py` — SubagentManager spawns independent agent instances with isolated tool sets and system prompts, announces results back to main agent via system channel
@@ -77,10 +67,8 @@
- **MimiClaw**: `context_builder.c` only reads last 3 days
- **Recommendation**: Make configurable, but mind token budget
### [ ] System Prompt Tool Guidance
- **nanobot**: `context.py` L74-101 — includes current time, workspace path, tool usage instructions
- **MimiClaw**: Has current time, but lacks tool usage guide and workspace description
- **Depends on**: Tool Use implementation
### [x] ~~System Prompt Tool Guidance~~
- Implemented: `context_builder.c` includes tool usage guidance in system prompt
### [ ] Message Metadata (media, reply_to, metadata)
- **nanobot**: `bus/events.py` — InboundMessage has media, metadata fields; OutboundMessage has reply_to
@@ -116,10 +104,9 @@
- **MimiClaw**: Not implemented
- **Recommendation**: Requires extra HTTPS request to Whisper API: download Telegram voice -> forward -> get text
### [ ] YAML Config File System
- **nanobot**: `config/loader.py` + `config/schema.py` — Pydantic config validation, YAML config support
- **MimiClaw**: All configuration via NVS key-value storage
- **Recommendation**: Current NVS approach is suitable for MCU, no change needed
### [x] ~~Build-time Config File~~
- Implemented: `mimi_secrets.h` — build-time credentials with highest priority over NVS/CLI
- Replaces need for YAML config; suitable for MCU workflow
### [ ] WebSocket Gateway Protocol Enhancement
- **nanobot**: Gateway port 18790 + richer protocol
@@ -150,32 +137,34 @@
- [x] Telegram Bot long polling (getUpdates)
- [x] Message Bus (inbound/outbound queues)
- [x] Agent Loop basic flow (single LLM call)
- [x] Claude API (Anthropic Messages API + SSE streaming)
- [x] Context Builder (system prompt + bootstrap files + memory)
- [x] Agent Loop with ReAct tool use (multi-turn, max 10 iterations)
- [x] Claude API (Anthropic Messages API, non-streaming, tool_use protocol)
- [x] Tool Registry + web_search tool (Brave Search API)
- [x] Context Builder (system prompt + bootstrap files + memory + tool guidance)
- [x] Memory Store (MEMORY.md + daily notes)
- [x] Session Manager (JSONL per chat_id, ring buffer history)
- [x] WebSocket Gateway (port 18789, JSON protocol)
- [x] Serial CLI (esp_console, 14 commands)
- [x] HTTP CONNECT Proxy (Telegram + Claude API via proxy tunnel)
- [x] Serial CLI (esp_console, 15 commands)
- [x] HTTP CONNECT Proxy (Telegram + Claude API + Brave Search via proxy tunnel)
- [x] OTA Update
- [x] WiFi Manager (NVS credentials, exponential backoff)
- [x] SPIFFS storage
- [x] NVS configuration (token, API key, model)
- [x] Build-time config (`mimi_secrets.h`, highest priority over NVS)
- [x] NVS configuration (token, API key, model, search key)
---
## Suggested Implementation Order
```
1. Tool Use Loop + Tool Registry <- this determines whether the agent is truly "intelligent"
1. [done] Tool Use Loop + Tool Registry + web_search
2. Memory Write via Tool Use <- makes the agent actually remember
3. Built-in Tools (read_file, write_file, message)
3. Telegram Allowlist (allow_from) <- security essential
4. Bootstrap File Completion (AGENTS.md, TOOLS.md)
5. Subagent (simplified)
6. Telegram Markdown -> HTML
7. Media Handling
8. Cron / Heartbeat
9. Other enhancements
4. Telegram Allowlist (allow_from) <- security essential
5. Bootstrap File Completion (AGENTS.md, TOOLS.md)
6. Subagent (simplified)
7. Telegram Markdown -> HTML
8. Media Handling
9. Cron / Heartbeat
10. Other enhancements
```