Source Grounding and Citations
Purpose and Scope
OpenWiki is designed to generate a living wiki whose pages are grounded in repository evidence rather than generic project guesses. The README presents this as the product promise: a Next.js application backed by an eve agent that plans a documentation-style outline, writes source-cited pages, publishes the result, and keeps repositories fresh as their source changes. Source grounding is therefore not a cosmetic feature. It is the contract that lets a generated page explain a system, workflow, or API while giving readers concrete files they can inspect when they need to verify the claim or continue implementation work.
Sources: README.md
The grounding model has three layers. First, indexing builds a useful inventory of repository files and reads a bounded set of high-signal context snippets. Second, prompt assembly summarizes the repository map, selected files, documentation candidates, public surface candidates, and official documentation hints so outline and page agents can plan from evidence. Third, the page generator is instructed to write human-readable documentation with visible source paths and structured citations, while avoiding raw file lists or line-by-line commentary. The result should read like first-party developer documentation, but remain traceable to the repository snapshot used for generation.
Sources: agent/lib/indexing/context.ts, agent/lib/indexing/prompt.ts, agent/subagents/page_generator/instructions.md
Relevant Source Files
- README.md — Establishes OpenWiki’s user-facing promise: source-grounded repository wikis with page-level citations and repository chat using the same indexed source context.
- agent/lib/indexing/source-paths.ts — Defines the filters that exclude internal planning documentation from public wiki evidence selection.
- agent/lib/indexing/context.ts — Selects priority files from the prepared repository workspace and reads bounded snippets for indexing context.
- agent/lib/indexing/prompt.ts — Builds the parent indexing prompt from repository metadata, file inventory, documentation hints, public surface candidates, and official docs index data.
- agent/subagents/page_generator/instructions.md — Defines the page generator output contract, including section structure, prose quality, citation objects, and source-backed writing expectations.
Useful File Inventory
The indexing context reader starts from the prepared repository workspace and its file inventory. It does not attempt to feed every file into the agent. Instead, it selects priority context paths, limits the number of files, limits file size, and truncates each snippet before adding it to the context list. This keeps the agent focused on files that usually explain a project well: README files, package manifests, workspace configuration, TypeScript and JavaScript entry points, tests, examples, docs, workflows, templates, and paths containing source directories. The score-based ordering gives top-level README and package files especially high priority because they often define the project’s public contract.
Sources: agent/lib/indexing/context.ts
A second important behavior is negative selection. OpenWiki intentionally filters out internal planning documentation before using docs-like files as public wiki evidence. The source-path helper normalizes slashes and lowercases paths, then treats documentation areas such as active work, completed work, drafts, notes, plans, research, scratch space, temporary folders, and todo material as internal. It also detects file names that look like feedback notes, gap analyses, quality runs, research plans, implementation plans, workflow plans, porting notes, scratchpads, or todos. This prevents generated public documentation from treating project management artifacts as authoritative user-facing docs.
Sources: agent/lib/indexing/source-paths.ts, agent/lib/indexing/context.ts
Prompt Assembly and Evidence Shape
The indexing prompt is where the selected evidence becomes an agent-readable task. It tells the indexing run that the repository has already been hydrated into a sandbox and then supplies repository identity, URL, default branch, commit SHA, useful file counts, skipped file counts, a wiki depth target, workspace location, and a manifest path. It also includes a repository map, an important file inventory sample, documentation source candidates, first-party documentation information architecture hints, an official docs index, and public surface candidates. These sections steer the model toward reader journeys and system explanations rather than a directory-by-directory summary.
Sources: agent/lib/indexing/prompt.ts
The prompt also sets qualitative boundaries that matter for source grounding. It directs the indexing agent to generate a real first-party documentation tree, prefer reader journeys over repository topology, use first-party docs as primary reader-facing evidence when they exist, and use implementation source to deepen or verify pages. It repeats that internal planning and status docs should be ignored for public wiki pages. For large repositories with limited user-facing docs, it encourages mature system sections such as architecture, package ecosystem, runtime behavior, routing, caching, development infrastructure, testing, continuous integration, examples, and glossary material when the evidence supports them.
Sources: agent/lib/indexing/prompt.ts
Page-Level Citation Contract
The page generator instructions define the final documentation contract. Every generated page is expected to be a source-grounded wiki page with purpose-first prose, clear sections, helpful tables where they improve scanability, and citations to relevant source files. The instructions explicitly warn against line-by-line code commentary and against turning pages into raw file lists. That distinction is important: a source path is support for a synthesized explanation, not the explanation itself. Readers should come away understanding the system or workflow, while citations show where the explanation was derived from.
Sources: agent/subagents/page_generator/instructions.md
The required output shape includes a page slug, title, markdown lines, structured citation objects, related pages, and coverage notes. The citation objects carry repository-relative paths plus start and end line numbers, with null accepted when line numbers are unknown. The instructions also require a page to start with a title heading and include a relevant source files section. In practice, OpenWiki pages should pair visible source lines in the prose with matching structured citations so rendered pages are readable while downstream consumers can still reason over citation metadata.
Sources: agent/subagents/page_generator/instructions.md
System-to-Code Mapping
| Concept | Source support | How it contributes |
|---|---|---|
| Product promise | README.md | Defines source-grounded wikis, page-level citations, repository chat, publishing, and refresh behavior. |
| Internal-doc filtering | agent/lib/indexing/source-paths.ts | Prevents planning and scratch documentation from shaping public wiki pages. |
| Context selection | agent/lib/indexing/context.ts | Chooses high-signal files, caps file count and size, and truncates snippets for safe prompt use. |
| Indexing prompt | agent/lib/indexing/prompt.ts | Converts the repository snapshot into the evidence bundle used to plan and draft the wiki. |
| Page output contract | agent/subagents/page_generator/instructions.md | Specifies prose expectations, sections, markdown line output, citations, related pages, and coverage notes. |
Practical Authoring Flow
When reading or extending OpenWiki’s indexing behavior, begin with the inventory and filtering rules. Ask whether a file should be considered public evidence, private planning material, or low-value context. Then inspect the context reader’s priority logic to understand whether the file can be sampled for the indexing prompt. From there, follow prompt assembly to see how the evidence is presented to the agent. Finally, compare the page generator instructions against generated output: each page should make claims that can be traced to repository paths, include enough explanatory prose to be useful, and disclose any important uncertainty in coverage notes.
Sources: agent/lib/indexing/source-paths.ts, agent/lib/indexing/context.ts, agent/lib/indexing/prompt.ts, agent/subagents/page_generator/instructions.md
A useful mental model is that OpenWiki separates evidence selection from documentation writing. Selection is deterministic and bounded: the code chooses candidate files, scores them, reads snippets, and formats prompt sections. Writing is agentic but constrained: the prompt and subagent instructions define quality targets, page structure, citation shape, and the expectation that source paths support synthesized claims. This separation lets maintainers improve grounding quality in targeted places. For example, changing priority path rules affects what the agent sees, while changing page instructions affects how the agent presents and cites that evidence.
Sources: agent/lib/indexing/context.ts, agent/lib/indexing/prompt.ts, agent/subagents/page_generator/instructions.md
Next Steps
Use this page when reviewing whether a generated wiki is adequately grounded. If a page feels generic, trace the issue backward: first check whether the page cites meaningful files, then whether those files were available in the outline page source paths, then whether the indexing prompt exposed enough repository map, docs, and public surface information. For broader context, read the wiki generation pipeline to see how outline planning and page generation fit together, and read official docs outlines to understand how first-party documentation structure influences the generated navigation.
Sources: README.md, agent/lib/indexing/prompt.ts, agent/subagents/page_generator/instructions.md