OpenWikivercel-labs/openwiki

Chat API

Purpose and Scope

The Chat API is the server boundary that turns a repository question from the web app into an eve conversation session and then exposes that session as a stream. It is designed as a two-step interaction: first the client submits a repository URL, current message, and optional short history to create a chat run; then the client opens a streaming endpoint keyed by the returned session id. This separation keeps request validation, repository normalization, rate limiting, and eve authentication on the server while allowing the browser to reconnect to the stream with a starting event index.

Sources: app/api/chat/route.ts, app/api/chat/stream/[sessionId]/route.ts

In OpenWiki terminology, a repository chat is not a generic chat completion. The agent rebuilds source context for the requested public GitHub repository, prepares a repository-specific prompt, and starts an eve conversation with state that records the repository revision and sandbox file selection. The API route therefore accepts user-facing chat data, but the actual assistant run is delegated to the repo-message eve route. The streaming route is a proxy to the corresponding eve session stream, preserving the response body and important stream headers rather than interpreting each event itself.

Sources: app/api/chat/route.ts, agent/lib/repo-message/run-repo-message.ts

Relevant Source Files

  • app/api/chat/route.ts - Implements POST /api/chat, validates the request body, normalizes the GitHub repository URL, enforces chat rate limits, starts the repo-message eve route, and returns the session metadata needed for streaming.
  • app/api/chat/stream/[sessionId]/route.ts - Implements GET /api/chat/stream/[sessionId], validates sessionId and startIndex, proxies the eve session stream, and preserves stream response headers.
  • app/api/eve-client.ts - Re-exports the shared eve URL and server-header helpers used by the app API routes.
  • lib/chat-rate-limit.ts - Defines the chat rate-limit switch, default limits, environment-variable parsing, reservation call, error type, and client-facing rate-limit response code.
  • agent/lib/repo-message/run-repo-message.ts - Starts the repository chat run inside the agent, builds source context, selects sandbox files, creates the prompt, and sends an eve conversation session.

Route Reference

POST /api/chat accepts a JSON body with repoUrl, message, and optional history. The route rejects non-JSON bodies with status 400 before schema validation. The schema requires repoUrl to be a URL, message to be a non-empty trimmed string no longer than 8,000 characters, and history to contain at most eight prior messages. Each history item must have role assistant or user, and each history content field is trimmed, non-empty, and capped at 4,000 characters. These limits give the agent enough continuity for a short conversation while bounding request size before any repository or model work begins.

Sources: app/api/chat/route.ts

After schema validation, POST /api/chat parses the repository URL as a public GitHub repository. A URL that is syntactically valid but not accepted as a public GitHub repo produces a 400 response with a public-repository-specific error. The parsed value is also canonicalized before it is sent to eve: the route forwards repoUrl: repo.url rather than trusting the raw input string. Successful creation returns HTTP 202 with a repo object containing owner, name, and canonical url, plus a session object from eve. The session contains sessionId, continuationToken, and streamIndex, with streamIndex starting at the value returned by the agent.

Sources: app/api/chat/route.ts

GET /api/chat/stream/[sessionId] is the companion streaming endpoint. It requires a non-empty sessionId path parameter and accepts an optional startIndex query parameter. startIndex must match a non-negative integer pattern, so clients can resume from a known event index but cannot pass arbitrary stream cursor strings. The route fetches the eve session stream with the incoming request signal, which lets client disconnects propagate to the upstream request. It returns the upstream response body with cache-control copied from eve or defaulted to no-store, and content-type copied from eve or defaulted to application/x-ndjson. Sources: app/api/chat/stream/[sessionId]/route.ts

Both chat routes export maxDuration = 800, indicating they are expected to remain open long enough for model-backed work and streaming. The create route does not stream its final assistant answer directly; it only starts the eve run and returns session metadata. The stream route does not create new work; it attaches to an existing session. This contract is important for UI code because retries should be routed differently: retrying POST /api/chat may create a new conversation, while retrying the stream with a startIndex should resume reading events for the same session.

Sources: app/api/chat/route.ts, app/api/chat/stream/[sessionId]/route.ts

Request, Response, and Error Contracts

The normal create-request shape is compact and intentionally stable: repoUrl identifies the repository, message is the latest user turn, and history carries a small transcript window. A successful response has status 202 rather than 200 because the answer generation continues asynchronously through the eve session stream. The client should treat the returned session.sessionId as the primary key for opening /api/chat/stream/{sessionId} and may retain continuationToken if it needs to display or persist session metadata. The initial streamIndex is returned so the client has an explicit starting point for stream consumption.

Sources: app/api/chat/route.ts

The create route has several explicit failure modes. Invalid JSON returns 400 with Expected a JSON request body. Invalid schema input returns the first zod issue message or a generic invalid-body message. Invalid repository parsing returns 400 with Expected a public GitHub repository URL. If the embedded eve route cannot be reached, the route returns 502 with an error telling local developers to start the app with pnpm dev, plus a detail string derived from the thrown error. If eve responds with a non-OK status, OpenWiki converts that into a 502 and forwards the parsed error field when present.

Sources: app/api/chat/route.ts

The stream route uses a thinner error contract. An empty session id returns 400, an invalid startIndex returns 400, and failure to open the eve stream returns 502 with a detail message. Once the upstream stream is opened, the route passes through the upstream status and status text. This means client-side stream handling should be prepared for both JSON error responses before a stream exists and non-200 stream responses from eve after the proxy request succeeds.

Sources: app/api/chat/stream/[sessionId]/route.ts

Rate Limiting and Storage Integration

Chat rate limiting is enforced before the eve run is started. enforceChatRateLimit is called with the parsed repository full name and the incoming request, allowing the limit reservation to combine repository scope with a hashed client identity. The rate-limit module is enabled by default: OPENWIKI_CHAT_RATE_LIMIT_ENABLED only disables it when set to a falsey non-enabled value, while 1, true, yes, and on explicitly enable it. If all configured limits are set to zero, enforcement returns without reserving a message attempt.

Sources: app/api/chat/route.ts, lib/chat-rate-limit.ts

The default chat limits are 40 messages per client per hour, 200 messages per client per day, and 600 global messages per hour. These values are read from OPENWIKI_CHAT_RATE_LIMIT_CLIENT_HOURLY, OPENWIKI_CHAT_RATE_LIMIT_CLIENT_DAILY, and OPENWIKI_CHAT_RATE_LIMIT_GLOBAL_HOURLY; invalid, missing, blank, or negative values fall back to the defaults. The storage layer performs the actual reservation through reserveChatMessageAttempt, while getClientKeyHash derives the client key from the request. Because the route also handles storageConfigurationErrorResponse, storage misconfiguration can be surfaced as an application response instead of being mistaken for a model or eve failure.

Sources: lib/chat-rate-limit.ts, app/api/chat/route.ts

When the reservation is denied, ChatRateLimitError carries the numeric limit, ISO-like resetAt, retryAfterSeconds, and scope. The create route serializes that error with code chat_rate_limited, includes the same fields in the JSON response, and sets the Retry-After header. The message text is tailored to the scope: global pressure says the instance is busy, daily client exhaustion says today's chat limit has been reached, and hourly client throttling says several chat messages were sent recently. Clients can use code for programmatic handling and retryAfter or the header for cooldown UI.

Sources: lib/chat-rate-limit.ts, app/api/chat/route.ts

Eve Session and Agent Execution Flow

OpenWiki routes call eve through helpers re-exported by app/api/eve-client.ts. The create route uses getOpenWikiEveUrl(request, "repo-message") and getEveServerHeaders() to invoke the embedded eve route with server-side authentication headers. The stream route uses getEveSessionStreamUrl({ request, sessionId, startIndex }) with the same server-header helper. Keeping these helpers behind app/api/eve-client.ts gives route handlers a small API surface and avoids duplicating eve URL construction across chat endpoints.

Sources: app/api/eve-client.ts, app/api/chat/route.ts, app/api/chat/stream/[sessionId]/route.ts

Inside the agent, startRepoMessage is the run function that backs the repo-message route. It parses the GitHub repository URL, obtains a GitHub snapshot, reads chat context snippets, selects sandbox file paths, and creates a repository-message prompt. It then calls send with mode: "conversation", a continuation token prefixed with openwiki-chat:, and state created from the commit SHA, default branch, canonical repo URL, and selected paths. The returned eve session id and continuation token are passed back to the app route, which relays them to the browser.

Sources: agent/lib/repo-message/run-repo-message.ts, app/api/chat/route.ts

The sandbox selection logic is intentionally bounded. It starts with paths already used as chat context, then adds files from the repository inventory that are no larger than 180,000 bytes until the set reaches 36 paths. Files are sorted by a repository-oriented priority score: root README.md, package.json, tsconfig.json, entrypoint-like files, docs, examples, package directories, source directories, and app directories are favored, while tests and dot-directories are deprioritized. This means chat answers are grounded in a practical cross-section of source files without loading an entire repository into the sandbox.

Sources: agent/lib/repo-message/run-repo-message.ts

Implementation Notes for Clients

A typical client should call POST /api/chat with the latest user message and a trimmed history window, check for 400 validation errors or 429 rate-limit responses, and only open the stream after receiving 202. The streaming URL should include startIndex when reconnecting after a disconnect so already-read events are not replayed unnecessarily. Because the stream endpoint defaults to newline-delimited JSON content, consumers should parse events incrementally and handle temporary stream-open failures separately from create-request failures.

Sources: app/api/chat/route.ts, app/api/chat/stream/[sessionId]/route.ts

For server operators, the main operational controls are the chat rate-limit environment variables and the eve connectivity implied by the helper calls. If local development returns the embedded-eve 502 message, the create route is specifically indicating that the app should be started in the mode that serves the eve route, such as pnpm dev. If users receive rate-limit responses, tune the client hourly, client daily, and global hourly limits rather than changing the route contract. For adjacent implementation details, read the repository chat UI page for browser persistence and reconnect behavior, and the agent architecture page for how eve channels and sessions are wired.

Ask a question...