1 changed files with 35 additions and 19 deletions
@@ -54,8 +54,8 @@ Build **stub-service** — a Go application that provides:
 │         ▼                   ▼                    ▼                  │
 │  ┌─────────────┐   ┌──────────────┐   ┌─────────────────┐         │
 │  │  Stub Store │   │ Request Log  │   │  LLM Adapter    │         │
-│  │  (SQLite /  │   │   (ring buf) │   │  (Anthropic /   │         │
+│  │  (SQLite /  │   │   (ring buf) │   │  (local CPU /   │         │
-│  │   bbolt)    │   │              │   │   OpenAI)       │         │
+│  │   bbolt)    │   │              │   │   Ollama)       │         │
 │  └─────────────┘   └──────────────┘   └─────────────────┘         │
 └─────────────────────────────────────────────────────────────────────┘
 ```
@@ -113,17 +113,23 @@ Persistence layer for imposters, stubs, and request logs.
 #### 6. LLM Adapter (`internal/llm`)
-Translates natural-language descriptions into stub configurations.
+Translates natural-language descriptions into stub configurations using a **local, CPU-only** LLM. No external API calls — everything runs on-premise, keeping data private and eliminating network latency / cloud costs.
 - **Interface:** A `StubGenerator` interface that takes a text prompt and returns a structured stub definition.
- **Implementation:** HTTP client calling the Anthropic Messages API (Claude) or OpenAI Chat Completions API, configurable via environment variables.
+- **Runtime:** [Ollama](https://ollama.com/) runs as a sidecar container (or host-installed binary). It serves an OpenAI-compatible HTTP API on `localhost:11434` and manages model downloads, quantization, and inference. No GPU required.
- **Prompt strategy:** System prompt contains the stub JSON schema and examples. User message is the natural-language description. The LLM returns a JSON stub definition that is validated before being applied.
+- **Model selection (CPU, ≤ 60 s per generation):**
  - **Primary: `qwen2.5-coder:1.5b-instruct-q4_K_M`** (~1 GB RAM) — strong code/JSON generation for its size; comfortably produces a stub JSON within 15–40 s on a 4-core CPU.
  - **Fallback: `phi3:mini-4k-instruct-q4_K_M`** (3.8 B, ~2.3 GB RAM) — higher quality output when the host has ≥ 8 GB RAM; generation stays under 60 s on 4+ cores.
  - The model is configurable via `LLM_MODEL` env var. Any Ollama-compatible GGUF model can be substituted.
 - **Prompt strategy:** System prompt contains the stub JSON schema and 2-3 few-shot examples. User message is the natural-language description. Response is constrained to JSON via Ollama's `format: "json"` parameter. The result is validated against the stub schema before use.
 - **Performance budget:** Generation must complete within **60 seconds** wall-clock time. The adapter enforces a context-based timeout; if the model exceeds it, the request fails with a clear error rather than hanging. Recommended: keep prompts under 512 tokens input and request ≤ 1024 tokens output.
 - **Flow:**
  1. User types: *"Return 200 with a JSON user profile when GET /api/users/123 is called"*
-  2. LLM adapter sends the prompt with schema context
+  2. LLM adapter sends the prompt to the local Ollama API (`POST /api/generate`) with schema context and `format: "json"`
  3. Response is parsed and validated against the stub schema
  4. If valid, the stub is presented to the user for confirmation in the Web UI
  5. On confirmation, the stub is saved via the Management API
 - **Graceful degradation:** If Ollama is unavailable or the model is not pulled, the `/generate` endpoint returns `503 Service Unavailable` with a diagnostic message. All other stub-service functionality remains unaffected — the LLM feature is strictly optional.
 #### 7. Request Log (`internal/log`)
@@ -143,7 +149,7 @@ In-memory ring buffer of recent requests per imposter, with optional persistence
 | HTTP framework | `net/http` (stdlib) + [chi](https://github.com/go-chi/chi) router | Minimal dependency, chi adds route params and middleware without abstracting stdlib |
 | Web UI | Templ + htmx + Pico CSS | No JS build toolchain, server-rendered, interactive via htmx |
 | Database | SQLite (modernc.org/sqlite) | Pure Go, zero CGO, rich SQL queries, single file |
-| LLM client | Direct HTTP (Anthropic SDK or `net/http`) | Avoid heavy SDK deps; a thin wrapper suffices |
+| LLM runtime | [Ollama](https://ollama.com/) sidecar (local CPU) | No cloud dependency, data stays on-premise, zero API cost; `qwen2.5-coder:1.5b` generates stubs in < 60 s on CPU |
 | Config | Environment variables + optional YAML file | 12-factor, simple for Docker |
 | Logging | `log/slog` (stdlib) | Structured logging, zero deps |
 | Testing | `testing` + [testify](https://github.com/stretchr/testify) | Assertions, standard tooling |
@@ -414,10 +420,9 @@ stub-service/
 │   │       └── 001_init.sql
 │   ├── server/                  # Dynamic stub HTTP listeners
 │   │   └── listener.go
-│   ├── llm/                     # LLM integration
+│   ├── llm/                     # LLM integration (local, CPU-only)
-│   │   ├── generator.go         # StubGenerator interface
+│   │   ├── generator.go         # StubGenerator interface + timeout logic
-│   │   ├── anthropic.go         # Anthropic/Claude implementation
+│   │   └── ollama.go            # Ollama HTTP client implementation
 │   │   └── openai.go            # OpenAI implementation
 │   ├── ui/                      # Web UI
 │   │   ├── templates/           # Templ files
 │   │   ├── static/              # CSS, minimal JS (htmx)
@@ -474,12 +479,23 @@ services:
      STUB_ADMIN_PORT: "8080"
      STUB_DATA_DIR: "/data"
      STUB_LOG_LEVEL: "debug"
-      LLM_PROVIDER: "anthropic"
+      LLM_OLLAMA_URL: "http://ollama:11434"
-      LLM_API_KEY: "${LLM_API_KEY}"
+      LLM_MODEL: "qwen2.5-coder:1.5b-instruct-q4_K_M"
-      LLM_MODEL: "claude-sonnet-4-6"
+      LLM_TIMEOUT: "60s"
    depends_on:
      ollama:
        condition: service_started
  ollama:
    image: ollama/ollama:latest
    volumes:
      - ollama-models:/root/.ollama
    # CPU-only — no GPU runtime required
    # Pull the model on first start: docker compose exec ollama ollama pull qwen2.5-coder:1.5b-instruct-q4_K_M
 volumes:
  stub-data:
  ollama-models:
 ```
 **Configuration (environment variables):**
@@ -491,9 +507,9 @@ volumes:
 | `STUB_LOG_LEVEL` | `info` | Log level (debug, info, warn, error) |
 | `STUB_PORT_RANGE_START` | `4545` | Start of port range for stub listeners |
 | `STUB_PORT_RANGE_END` | `4600` | End of port range for stub listeners |
-| `LLM_PROVIDER` | (none) | LLM provider: `anthropic` or `openai` |
+| `LLM_OLLAMA_URL` | `http://localhost:11434` | Ollama API base URL |
-| `LLM_API_KEY` | (none) | LLM API key |
+| `LLM_MODEL` | `qwen2.5-coder:1.5b-instruct-q4_K_M` | Ollama model tag for stub generation |
-| `LLM_MODEL` | `claude-sonnet-4-6` | Model to use for stub generation |
+| `LLM_TIMEOUT` | `60s` | Maximum wall-clock time for a single generation request |
 ---
@@ -504,7 +520,7 @@ volumes:
 - **Single binary** — no runtime dependencies beyond the OS; trivial to deploy.
 - **Embedded Web UI** — no separate frontend build/deploy pipeline; Templ + htmx keeps it all in Go.
 - **SQLite** — zero operational overhead, no external database to manage, easy backup (copy a file).
- **LLM integration** — significantly lowers the barrier for creating complex stubs, especially for QA engineers unfamiliar with JSONPath/regex predicates.
+- **Local LLM integration** — significantly lowers the barrier for creating complex stubs, especially for QA engineers unfamiliar with JSONPath/regex predicates. Runs entirely on-premise (CPU-only, no GPU), so sensitive request/response data never leaves the network and there are zero API costs.
 - **Mountebank-compatible concepts** — teams familiar with Mountebank can transfer their mental model (imposters, stubs, predicates).
 - **Port-per-imposter isolation** — each mock service gets its own port, matching how real services are addressed.
@@ -512,7 +528,7 @@ volumes:
 - **Port range management** — dynamic port allocation requires firewall/Docker port-range configuration.
 - **SQLite single-writer** — under very heavy concurrent management API load, SQLite's single-writer lock could be a bottleneck. Acceptable for a testing/dev tool.
- **LLM dependency** — the generation feature requires external API access and incurs cost. It is optional — the service works fully without it.
+- **LLM resource usage** — the Ollama sidecar with a 1.5 B model requires ~1 GB RAM and consumes noticeable CPU during generation (~15–40 s on 4 cores). Larger fallback models (3.8 B) need ~2.3 GB RAM. The feature is optional — the service works fully without Ollama running.
 - **No multi-protocol support initially** — only HTTP/HTTPS. TCP/SMTP stubs (which Mountebank supports) are deferred to a future iteration.
 ### Risks