Repository API

Purpose and Scope

The repository API is the public HTTP surface that lets the OpenWiki web app list known repositories, create or look up a repository from a GitHub URL, start indexing when needed, retrieve a generated wiki, show featured repository cards, and search GitHub for repository suggestions. These routes sit at the boundary between user-facing repository discovery and the indexing engine. They intentionally keep the browser workflow simple: the client can submit a GitHub URL or search query, while the server validates inputs, checks storage, verifies public GitHub repositories, applies public-generation controls, and delegates long-running wiki generation to the indexing helper.

Sources: app/api/repositories/route.ts, app/api/repositories/[owner]/[repo]/route.ts, app/api/repositories/featured/route.ts, app/api/repositories/search/route.ts, app/api/repositories/indexing.ts

This page is a route-reference view of the repository API rather than a guide to the full indexing pipeline. It focuses on concrete endpoints, request shapes, response patterns, and the source-level decisions that matter when integrating a UI or debugging repository creation. The important design constraint is that OpenWiki accepts public GitHub repositories only: repository URLs are parsed as public GitHub refs, unknown repositories are verified against GitHub before storage is updated, and search results filter out private repositories before returning suggestions to the client.

Relevant Source Files

app/api/repositories/route.ts - Implements GET /api/repositories for listing stored repositories and POST /api/repositories for repository creation, lookup, rate-limit enforcement, and indexing delegation.
app/api/repositories/[owner]/[repo]/route.ts - Implements GET /api/repositories/{owner}/{repo} for loading the stored wiki payload for a specific repository owner/name pair.
app/api/repositories/featured/route.ts - Implements GET /api/repositories/featured for dynamic, no-store featured repository card data.
app/api/repositories/search/route.ts - Implements GET /api/repositories/search?q=... by querying GitHub Search, normalizing results, ranking matches, and returning a compact repository card shape.
app/api/repositories/indexing.ts - Re-exports startRepositoryIndexing and RepositoryIndexingStart from the shared indexing module so the repository creation route can start or refresh indexing without embedding engine internals.

Endpoint Reference

Method and path	Request input	Success response	Important failure responses
`GET /api/repositories`	None	`{ repositories }` from storage	Storage configuration errors are converted to a JSON response when recognized.
`POST /api/repositories`	JSON body with `repoUrl` and optional `force`	Existing indexed repositories return `{ job: null, repository }`; unindexed or forced repositories delegate to indexing.	`400` for malformed JSON or invalid URL, `404` when GitHub cannot find the repository, `502` when GitHub verification fails, plus configured creation-disabled and rate-limit responses.
`GET /api/repositories/{owner}/{repo}`	Dynamic route params `owner` and `repo`	`{ wiki }`	`404` with `Repository not found.` when storage has no wiki for that pair.
`GET /api/repositories/featured`	None	`{ repositories }` card array	Storage configuration errors are converted to a JSON response when recognized.
`GET /api/repositories/search?q=...`	Query string `q`	`{ repositories }` with up to nine normalized results	`502` when GitHub Search does not return an OK response.

The top-level repository route uses Zod to define the creation contract. The body must parse as JSON and must contain a repoUrl accepted by z.url; force is optional and defaults to false. After schema validation, the handler calls the project GitHub URL parser, and a second invalid-URL branch returns the same public message, Expected a public GitHub repository URL. This two-step validation is useful because a syntactically valid URL is not necessarily a GitHub repository reference in the format OpenWiki can index.

Sources: app/api/repositories/route.ts

Creation and Indexing Flow

A POST /api/repositories request first normalizes the requested repository into an owner, name, and full name. It then checks whether the full name is configured as a featured repository and whether public repository creation has been disabled. Featured repositories are treated specially for rate limiting: the route’s local enforceGenerationRateLimitOnce helper returns immediately for configured featured repositories, and it also ensures that the same request reserves the generation quota at most once. That matters because a single request can both create a missing repository row and later start an indexing job.

Sources: app/api/repositories/route.ts

The handler next looks up the repository by full name in storage. If the row does not exist, creation may be blocked by the OPENWIKI_DISABLE_REPOSITORY_CREATION policy exposed through isRepositoryCreationDisabled. When creation is allowed, the route verifies the public repository through githubRepositoryExists. A negative verification produces a 404 so clients can distinguish a real missing GitHub repository from a storage or generation problem. A verification exception becomes a 502, which communicates that OpenWiki could not complete its upstream GitHub check.

Sources: app/api/repositories/route.ts

When the repository does not yet exist and GitHub verification succeeds, the route enforces the generation rate limit before calling upsertRepository. A RepositoryGenerationRateLimitError is translated by the route into the project’s repository-generation rate-limited response, while storage configuration failures are passed through storageConfigurationErrorResponse. This split keeps expected public-operational states in JSON and lets unexpected exceptions surface normally. If the repository already has currentIndexedRevisionId and the caller did not set force, the route returns immediately with job: null, avoiding unnecessary regeneration of an already indexed wiki.

Sources: app/api/repositories/route.ts

If the repository is not indexed or the caller requests force, the route checks the creation-disabled switch again before delegating to startRepositoryIndexing. The helper is imported from a local module that re-exports the shared library entry point and the associated RepositoryIndexingStart type. That small indirection keeps the route source focused on HTTP validation and policy decisions, while the indexing library remains the owner of job creation, repository revision handling, and the agent-backed generation workflow.

Sources: app/api/repositories/route.ts, app/api/repositories/indexing.ts

Fetching Repository Wikis

The owner/repo route is deliberately narrow. It receives dynamic route params as a promise, awaits owner and repo, and calls getRepositoryWiki with { owner, name: repo }. If storage returns null, the route responds with 404 and Repository not found. Otherwise it returns the full wiki payload as { wiki }. This makes it the direct API counterpart to repository wiki pages: clients that already know an owner and repository name can fetch the generated wiki without repeating GitHub URL parsing or triggering repository creation.

Sources: app/api/repositories/[owner]/[repo]/route.ts

Because this route is read-only and does not start indexing, callers should not use it as the first step for unknown repositories. The creation route is responsible for checking whether the repository exists in GitHub and deciding whether an index job should be started. The wiki-fetch route assumes that storage is already the source of truth for a published wiki. In practice, UI flows often submit through POST /api/repositories first, then navigate to or fetch the wiki once repository metadata and indexing state are available.

Featured and Search APIs

The featured route is dynamic and explicitly returns cache-control: no-store. It calls getFeaturedRepositoryCards and wraps the result as { repositories }. Storage configuration failures are converted to a response using the same helper used by the top-level repository route. The no-store behavior is important for a featured list because card metadata may be refreshed independently of page builds, and the API should not serve stale HTTP-cached JSON when storage has newer descriptions, avatars, URLs, or star counts.

Sources: app/api/repositories/featured/route.ts

The search route is a GitHub-backed suggestion endpoint. It reads q from the query string, trims it, and returns an empty repository list until the query has at least two characters. For real searches, it calls https://api.github.com/search/repositories with per_page=30 and a query scoped to in:name,full_name fork:false. The fetch uses next: { revalidate: 600 }, so matching search calls can be cached by Next.js for ten minutes while still keeping the app responsive to popular repeated searches.

Sources: app/api/repositories/search/route.ts

Search normalization is intentionally defensive. A GitHub item is discarded if it is private or if full_name and html_url are not strings. Remaining items are converted into OpenWiki’s compact result shape: nullable description, fullName, normalized iconSrc, repoUrl, and nullable starCount. Results are then ranked before being returned. Exact full-name matches rank first, exact repository-name matches come next, followed by full-name prefix matches, repository-name prefix matches, and finally all other GitHub-ranked items. The response is capped to nine repositories so the UI can present a compact suggestion list.

Sources: app/api/repositories/search/route.ts

Authentication, Rate Limits, and Operational Behavior

These repository routes do not use a separate authenticated session boundary in the supplied source. Instead, their public safety controls come from URL validation, public GitHub verification, private-result filtering, the repository creation-disabled switch, storage configuration handling, and generation rate limiting. The search route optionally adds an authorization: Bearer ... header when GITHUB_TOKEN is present, but the token is used only as a server-side GitHub API credential. The default GitHub headers also set accept: application/vnd.github+json and user-agent: openwiki.

Sources: app/api/repositories/search/route.ts, app/api/repositories/route.ts

For implementers, the main rule is to choose the endpoint that matches the user intent. Use search when the user is typing or discovering repositories. Use POST /api/repositories when the user has chosen a GitHub repository URL and the app may need to create storage state or start indexing. Use GET /api/repositories/{owner}/{repo} when the wiki should already exist and the client needs the stored wiki payload. Use featured for the curated home-page card list. For deeper behavior, continue with the indexing jobs, repository search and creation, and wiki routing pages.

Ask a question...