## llm.* — host LLM access

*Host API · llm.**

Borrow the host's **user-configured** LLM provider for stateless single-shot completions. Your bundle never sees an API key, never hard-codes a model name, and the request is billed against the user's existing quota — `modelPreferences` only nudges the selector between providers the user *actually enabled*.

**Path:** Bundle (postMessage) → `POST /api/v1/anna-apps/runtime/rpc` → `_h_llm_complete` (dispatcher) → `SqlAlchemyWindowStore.llm_complete` → `_decode_or_mint_complete` lazy-mints an `AnnaAppSession` (kind=`complete`) cached by `complete:{window_uuid}` → `app_llm_facade.complete` → `LLMProviderService.select_model_by_preferences` → LangChain `ChatOpenAI`.

**ACL.** The only dispatcher gate is `host_api_allows` for the `llm` namespace: `manifest.ui.host_api.llm` (a `string[]`) must contain the method — `complete` / `stream` / `embed` — or `*`. `manifest.permissions[]` is display/audit metadata and is NEVER checked.

**MCP-shaped.** Both request (`messages`, `maxTokens`, `temperature`, `modelPreferences`, `systemPrompt`, `stopSequences`) and response (`role`, `content:{type:'text', text}`, `model`, `stopReason`, `usage`, `_meta`) mirror MCP `sampling/createMessage` so the same payload works against MCP samplers too.

**Quota model.**
• Per-call cap `quota_caps.max_tokens_per_call` (default **4096**) — `maxTokens > cap` is **silently clamped down**; read back `usage.outputTokens` to detect.
• Pre-flight `check_quota_available` + post-call `check_and_consume_quota_with_redis` both raise `APP_QUOTA_EXCEEDED` (the latter raises *after* the model returns, dropping the response — never double-charges).
• `_meta.quotaConsumed` reports the actual debited amount (provider cost × `cost_multiplier`).

**Stateless.** No conversation state between calls. The lazy-minted `app_session_uuid` is reused server-side only for token caching, NOT for prompt history — callers MUST resend full `messages` every call. For multi-turn / tool-using workflows use `agent.*` instead.

**Non-streaming.** The `complete` response arrives in a single chunk after the provider finishes. For token-by-token UI use `llm.stream` (same request shape, async-iterable `RpcStream`; SDK ≥ 0.10.0 / dispatcher ≥ 0.12.0) or `agent.session.run` (LangGraph streaming over `rpc.stream`). SDK per-namespace timeout default is **180 000 ms**.

- **llm.complete** — MCP-shaped single-shot completion. `messages` required (array, non-empty). Effective output cap = `min(maxTokens ∥ 8192, model output cap, 8192 sync ceiling)` — echoed in `_meta.maxTokens` (`{requested, effective, limitedBy}`); long outputs belong on `llm.stream` (SDK ≥ 0.17.0 auto-routes `maxTokens > 8192` over the streaming channel). Provider + model resolved server-side from `modelPreferences ∩ user-grant`. Returns `{role, content:{type:'text', text}, model, stopReason:'endTurn'|'maxTokens'|'contentFilter'|'toolUse', usage:{inputTokens, outputTokens, totalTokens}, _meta:{provider, latencyMs, quotaConsumed, appSessionUuid, maxTokens}}`. Errors: `APP_NOT_GRANTED`, `APP_INVALID_REQUEST`, `APP_QUOTA_EXCEEDED` (worst-case precheck; data carries `requiredCu`/`remainingCu`/`requestedMaxTokens`), `APP_PROVIDER_ERROR`.  `MCP-shaped` `stateless` `non-streaming` `sync cap 8192`

- **llm.stream** — Streaming L1 completion (SDK ≥ 0.10.0 / dispatcher ≥ 0.12.0). Same request shape as `llm.complete` but returns an async-iterable stream: `{event:'model_token', text}` frames followed by a terminal `{event:'complete', role, content, model, stopReason, usage, _meta}` over the shared `rpc.stream` topic. Output ceiling 65 536 tokens (vs 8 192 sync); terminal frames with > ~48 KB accumulated text omit `content.text` (`_meta.textOmitted` + `_meta.textLength`) — accumulate the token frames. Grant via `manifest.ui.host_api.llm` containing `stream` (or `*`); same quota model as `complete`. Old hosts return `-32601` — feature-detect and fall back to `llm.complete`.  `MCP-shaped` `streaming` `cap 65536`

---

## Detailed reference

### llm.complete

*Host API method · iframe surface*

Stateless single-shot chat completion against the **user's chosen** LLM provider / model — your bundle never sees an API key, never hard-codes a model name, and the request is billed against the user's existing quota and provider routing. MCP-shaped: the request mirrors `sampling/createMessage` (`messages`, `maxTokens`, `temperature`, `modelPreferences`, `systemPrompt`, `stopSequences`) and the response mirrors the MCP result (`role`, `content: {type:'text', text}`, `model`, `stopReason`, `usage`).

**Signature**

```ts
anna.llm.complete(args: {messages: Array<{role:'user'|'assistant'|'system', content: string|object}>, maxTokens?: number, temperature?: number, modelPreferences?: {hints?: object, costPriority?: number, speedPriority?: number, intelligencePriority?: number}, systemPrompt?: string, stopSequences?: string[], metadata?: object}, opts?: {timeoutMs?: number}) => Promise<{role:'assistant', content:{type:'text', text:string}, model:string, stopReason:string, usage:{inputTokens:number, outputTokens:number, totalTokens:number}, _meta?:{provider:string|null, latencyMs:number, quotaConsumed:number, appSessionUuid:string}}>
```

**Direction:** Bundle (iframe) → Host RPC → app_llm_facade.complete → LangChain ChatOpenAI → provider

**Parameters**

- `messages` (Array<{role, content}>, required) — Ordered chat transcript to complete. `role` is `user` / `assistant` / `system`. `content` is a string, an MCP-shaped block, or an array of blocks. Supported blocks: `{type:'text', text}` and `{type:'image', data:'<raw base64 or data: URI>', mimeType:'image/png'}` (or `{type:'image', url:'https://...'}`). Text-only content is flattened into a LangChain `HumanMessage` / `AIMessage` / `SystemMessage` by `_assemble_lc_messages`; image blocks become OpenAI-style `image_url` content parts sent to the provider.
    - `APP_INVALID_REQUEST` if missing, non-array, or empty.
    - Order is preserved verbatim — caller is responsible for trimming context to the model's window.
    - **Image blocks**: `data` accepts raw base64 (with `mimeType`, default `image/png`) or a full `data:` URI; `url` accepts HTTP(S) and is SSRF-validated server-side. Max 20 MB per image; MIME must be one of `image/jpeg|png|gif|webp|bmp|svg+xml`.
    - Text and images may be mixed in one content array, or placed in separate consecutive `user` messages — both are supported.
    - Image blocks are only allowed in `user` messages; in `system` / `assistant` messages they raise `APP_INVALID_REQUEST`.
    - If the resolved model is not multimodal the call fails fast with `APP_MODEL_NOT_VISION_CAPABLE` (`-32019`) — pick a vision model via `modelPreferences.hints`. Images are never silently dropped.
    - Unknown block types (`{type:'audio'}` etc.) raise `APP_INVALID_REQUEST` — they are not silently ignored.
- `maxTokens` (number (int), optional) — Hard cap on output tokens. The effective cap is resolved per call as `min(maxTokens ∥ 8192, model output cap, path ceiling)` — path ceiling is **8 192** for `llm.complete` (held-HTTP sync channel) and **65 536** for `llm.stream`. There is no per-app grant knob — caps are platform constants ∩ model capability.
    - `APP_INVALID_REQUEST` if `≤ 0`.
    - Clamping is visible: the response `_meta.maxTokens` echoes `{requested, effective, limitedBy: 'request'|'model'|'transport'|'default'}`.
    - For outputs beyond 8 192 tokens use `llm.stream` (ceiling 65 536). With `@anna-ai/app-runtime` ≥ 0.17.0, `llm.complete` with `maxTokens > 8192` is automatically delivered over the streaming channel and resolves with the identical single-result shape.
    - Pre-flight quota check is worst-case: a call may be rejected with `APP_QUOTA_EXCEEDED` (429) when the balance cannot cover the *effective* cap — the error `data` carries `{requiredCu, remainingCu, requestedMaxTokens}` so the app can lower `maxTokens` and retry.
- `temperature` (number, optional) — Sampling temperature passed through to the provider. Default `0.7` when omitted.
- `modelPreferences` ({hints?, costPriority?, speedPriority?, intelligencePriority?}, optional) — Soft routing hints consumed by `LLMProviderService.select_model_by_preferences`. The user's grant ALWAYS overrides — preferences only break ties between providers the user actually enabled.
    - `hints` is an arbitrary dict (e.g. `{family: 'claude'}`); the selector pattern-matches against `LLMModel.metadata`.
    - Priorities are floats in `[0, 1]`; higher means "more important".
    - If no eligible model is available for the user, the call raises `APP_PROVIDER_ERROR` (`no eligible model available`).
- `systemPrompt` (string, optional) — Optional system prompt prepended as a `SystemMessage` before `messages`. Use this for app-specific persona / format instructions instead of injecting into `messages[0]`.
- `stopSequences` (string[], optional) — Forwarded as the provider's `stop` parameter. Caller must trim — the host does not validate length / count limits.
- `metadata` (object, optional) — Free-form caller-side tagging. NOT forwarded to the provider; reserved for future telemetry.

### llm.stream

*Host API method · iframe surface · streaming*

Streaming variant of `llm.complete` — same request shape, but the SDK returns an async-iterable `RpcStream`: `for await` yields `{event:'model_token', text}` incremental frames, and the final `{event:'complete', role, content, model, usage, _meta}` frame mirrors the `llm.complete` response exactly. Generation runs on the worker fleet (same single-producer dispatch as `agent.session.run`); frames arrive via the window's `rpc.stream` event channel and are reassembled by the SDK — your bundle just iterates.

**Signature**

```ts
anna.llm.stream(args: {messages: Array<{role, content}>, maxTokens?: number, temperature?: number, modelPreferences?: object, systemPrompt?: string, stopSequences?: string[], metadata?: object}, opts?: {timeoutMs?: number}) => RpcStream<{event:'model_token', text:string} | {event:'complete', role:'assistant', content:{type:'text',text:string}, model:string, usage:object, _meta?:object}>
```

**Direction:** Bundle (iframe) → Host RPC → worker-dispatched app_llm_facade.stream_completion → frames pushed back over `rpc.stream` events

**Parameters**

- `(all)` (—, optional) — Identical to `llm.complete`: `messages` (required; same text + image block support and vision gating), `maxTokens` (effective cap = `min(maxTokens ∥ 8192, model output cap, 65 536 stream ceiling)` — echoed in the terminal frame's `_meta.maxTokens`), `temperature`, `modelPreferences`, `systemPrompt`, `stopSequences`, `metadata`. See the `llm.complete` entry for full per-param semantics — the request travels the same validation path.