OpenWikivercel-labs/openwiki

Application Architecture

Purpose and Scope

OpenWiki is organized as a product application rather than a standalone library: the repository combines a public Next.js interface, route handlers, storage-backed repository records, and an eve agent that performs long-running documentation work. The README defines the user-facing promise as generating a living, source-grounded wiki for any public GitHub repository, with docs-style navigation, repository chat, featured wiki prerendering, daily refresh scheduling, and built-in public-generation rate limiting. That framing is important for contributors because most source modules serve one of two jobs: present the current wiki to readers, or coordinate background generation so the current wiki can be created and refreshed safely.

Sources: README.md

The architecture can be read as four cooperating layers. The web layer renders the landing page, repository wiki pages, navigation, and chat experiences. The API layer accepts repository, markdown, chat, indexing, and maintenance requests, then validates and delegates them. The shared library layer contains reusable repository, storage, parsing, rate-limit, and configuration helpers. The agent layer uses eve to plan outlines, generate pages, publish artifacts, and stream chat or tool-call results back to callers. Keeping these layers separate lets the UI remain responsive while generation and refresh work can run through agent sessions and persisted job state.

Sources: README.md, agent/agent.ts, agent/lib/run-message.ts

Relevant Source Files

  • README.md — Defines the product promise, deployment model, runtime services, local workflow, and the main OpenWiki capabilities that the codebase implements.
  • app/layout.tsx — Provides the root Next.js document shell, global CSS, font setup, theme initialization script, analytics, speed insights, and top-level theme provider.
  • app/page.tsx — Implements the static home route by loading featured repository cards and rendering the repository-entry experience plus footer.
  • agent/agent.ts — Declares the eve agent entrypoint and selects the model used by OpenWiki agent work.
  • agent/lib/run-message.ts — Converts eve readable stream events into a final assistant reply and captured tool-call records, including failure handling.
  • lib/github-repository.ts — Encapsulates GitHub repository metadata lookup, public-repository checks, default-branch commit resolution, and GitHub request headers.

System-to-Code Mapping

The root application shell lives in app/layout.tsx. It imports global styles, installs the Geist font as the --font-sans variable, declares page metadata, and wraps all route children in ThemeProvider. The file also injects a small pre-hydration theme script that reads the openwiki-theme cookie, resolves dark, light, or system, updates the document classes, sets colorScheme, and chooses a background color before React hydrates. That means visual state is treated as application infrastructure rather than a per-page concern, and every route receives the same analytics, speed insights, theme context, and base document styling.

Sources: app/layout.tsx

The landing route in app/page.tsx is intentionally small. It marks the page as force-static with no revalidation, loads featured repository cards through getFeaturedRepositoryCards with storage-configuration fallback enabled, and renders RepositoryHome inside Suspense followed by OpenWikiFooter. Architecturally, the home page is the entry point for repository discovery and creation, but it does not embed indexing logic directly. Instead, it prepares initial presentation data and lets UI components and API routes handle user actions such as entering a GitHub URL, opening a featured repository, or starting generation.

Sources: app/page.tsx

The agent entrypoint in agent/agent.ts is deliberately compact: it calls defineAgent from eve and supplies the model returned by getOpenWikiAgentModel. This places model selection behind a configuration helper while keeping the exported agent definition stable for eve runtime discovery. In the broader system, route handlers and job orchestration code can treat the agent as a service that receives repository tasks and emits stream events, while the agent implementation can evolve through instructions, tools, and model configuration without changing every caller.

Sources: agent/agent.ts

agent/lib/run-message.ts is the adapter between eve's event stream protocol and OpenWiki's application expectations. Its public helpers read a ReadableStream<unknown> until a terminal event, remember the last completed text message, collect tool calls requested by the model, attach tool results when they arrive, and throw on session failure. The file defines a specific RunMessageMissingReplyError for sessions that complete without text, which gives callers a clearer operational signal than a generic null result. This adapter is used conceptually wherever OpenWiki needs a deterministic reply from an asynchronous agent run.

Sources: agent/lib/run-message.ts

Execution Flow

A typical repository-generation path starts at the web UI, where a user pastes or visits a public GitHub repository. The README states that OpenWiki intentionally rejects private repositories on public deployments and recommends GITHUB_TOKEN to improve public GitHub API reliability. The shared GitHub helper enforces that policy at the metadata boundary: it fetches repository profile data from GitHub, rejects private repositories, normalizes owner avatar URLs, reads star count and description, resolves the default branch, and then requests the current default-branch commit SHA. That commit SHA becomes the source identity for deciding what should be indexed or refreshed.

Sources: README.md, lib/github-repository.ts

Repository existence checks are also centralized in lib/github-repository.ts. githubRepositoryExists calls the GitHub REST repository endpoint with OpenWiki's headers, returns false for 404, falls back to a GitHub page HEAD request when the API returns 403, rejects private repositories when metadata says private: true, and throws a generic verification error for other unexpected responses. The fallback is a practical architecture choice: public deployments may hit GitHub API limits, but they still need a safe way to distinguish a public repository page from a missing repository without treating private repositories as eligible input.

Sources: lib/github-repository.ts

Once a repository is accepted, the API and job orchestration layers can start or reuse indexing work while the UI shows progress or an existing wiki. The official route evidence shows this architectural pattern repeatedly: API handlers validate input, translate public requests into repository references, enforce rate limits or storage configuration checks, and then delegate to shared helpers or the eve backend. In the same style, chat requests send a normalized repository URL, current message, and optional history to an eve endpoint, while markdown export reads already-published wiki artifacts rather than rerunning generation. The route layer is therefore a boundary, not the owner of generation intelligence.

Sources: README.md, agent/lib/run-message.ts

API Components and Shared Boundaries

OpenWiki's public routes should be understood as clients of the same underlying repository model. Repository creation and lookup depend on parsing a GitHub URL, checking whether the repository is public and reachable, storing or retrieving metadata, and deciding whether indexing should start. Chat depends on a previously indexed source context and streams work through the eve agent. Markdown export depends on published wiki artifacts. Featured repository pages depend on preconfigured metadata and static rendering support. These surfaces differ in HTTP shape, but they share one architectural rule: validate at the edge, then call narrowly scoped helpers that can be tested and reused.

Sources: README.md, lib/github-repository.ts

The shared GitHub helper illustrates the expected boundary design. It exposes small functions with precise responsibilities: getGitHubRepositoryMetadata composes profile lookup and commit resolution; getGitHubRepositoryProfile returns public profile fields without the commit; getGitHubDefaultBranchCommitSha resolves the current branch SHA; and githubRepositoryExists performs a lightweight eligibility check. The helper also hides request details such as the GitHub media type, user-agent: openwiki, and optional bearer authorization from GITHUB_TOKEN. That keeps route handlers and indexing code from duplicating network-policy details.

Sources: lib/github-repository.ts

The eve stream adapter provides a similar boundary on the agent side. Callers do not need to inspect every raw event type themselves; they can call readRunMessage when only text is needed, or readRunMessageWithToolCalls when auditing tool activity matters. Internally, the adapter recognizes message.completed, actions.requested, action.result, session.failed, turn.completed, session.waiting, and session.completed. Tool outputs are normalized to strings, including JSON stringification for non-string outputs. That gives the rest of the app a compact reply-plus-tool-calls contract while preserving enough detail for logs and debugging.

Sources: agent/lib/run-message.ts

Implementation Details

The global layout demonstrates how user experience concerns are handled centrally. The app defaults to the dark theme, but the initialization script honors a saved light or system preference before hydration to prevent a flash of the wrong theme. It writes both classes and data-openwiki-theme-preference, which lets CSS and React state agree about the selected mode. By placing Analytics and SpeedInsights in the root body, OpenWiki also treats production telemetry as part of the application shell. Page and component authors can focus on repository workflows without reimplementing document-level behavior.

Sources: app/layout.tsx

The home page demonstrates a complementary pattern for rendering strategy. It is static because the public landing experience should load quickly and because featured repository cards can be fetched with a graceful fallback if storage is not configured. The Suspense wrapper allows the repository home component tree to participate in asynchronous rendering without making the file responsible for loading states. This is consistent with the larger architecture: pages assemble source-backed data and UI components, while operational details such as repository storage, indexing jobs, and agent calls live behind route handlers or library functions.

Sources: app/page.tsx

Next Steps

When changing OpenWiki, start by identifying which layer owns the behavior. UI changes usually belong under the app component tree and should consume existing route or library contracts. Repository validation, GitHub metadata, and public/private checks should go through lib/github-repository.ts instead of ad hoc fetch calls. Agent-response handling should use the stream adapter rather than re-parsing eve events. For deeper implementation work, continue with the pages on the wiki-generation pipeline, repository API, chat API, indexing engine, and navigation/layout components so that changes stay aligned with the existing boundaries.

Sources: README.md, app/layout.tsx, app/page.tsx, agent/agent.ts, agent/lib/run-message.ts, lib/github-repository.ts

Ask a question...