Testing Experience
Purpose and Scope
VS Code presents testing as a first-class developer workflow: tests can be discovered by extensions, executed from the Test Explorer, debugged with the normal debugger, and reported inline or in result panels. In this repository, the same ideas appear in the product code and in the repository’s own test suites. The supplied source evidence focuses on Copilot and diagnostic-driven behavior, which is useful because it shows how VS Code features are verified at several levels: pure unit tests, VS Code API mocks, tool registration tests, simulated diagnostics providers, and telemetry-oriented service tests.
For users, “testing experience” means the editor can coordinate language-specific test providers, show results in context, and help with setup or failures through Copilot. For contributors, it means each feature needs tests that prove both the public behavior and the integration boundaries. The Copilot tests in this page validate error classification, diagnostic collection, diagnostic push notifications, prompt settings, and debug logging. Together they illustrate the contract VS Code expects from testing-related features: stable inputs, explicit outputs, debounced event handling, predictable defaults, and safe behavior when data is empty or unknown.
Sources: extensions/copilot/src/platform/remoteSearch/test/node/codeOrDocsSearchErrors.spec.ts, extensions/copilot/test/simulation/diagnosticProviders/index.ts, extensions/copilot/src/extension/chat/vscode-node/test/chatDebugFileLoggerService.spec.ts, extensions/copilot/src/extension/chatSessions/copilotcli/vscode-node/test/diagnosticsChanged.spec.ts, extensions/copilot/src/extension/chatSessions/copilotcli/vscode-node/test/getDiagnostics.spec.ts, extensions/copilot/src/extension/completions-core/vscode-node/lib/src/prompt/test/defaultDiagnosticSettings.test.ts
Relevant Source Files
extensions/copilot/src/platform/remoteSearch/test/node/codeOrDocsSearchErrors.spec.tsverifies that remote code and documentation search failures are mapped into the expected error classes.extensions/copilot/test/simulation/diagnosticProviders/index.tsdefines the known diagnostic providers used by Copilot simulation tests, including TypeScript, ESLint, Python, Roslyn, C++, and Ruff providers.extensions/copilot/src/extension/chat/vscode-node/test/chatDebugFileLoggerService.spec.tstests a chat debug file logger service with OpenTelemetry span and span-event inputs.extensions/copilot/src/extension/chatSessions/copilotcli/vscode-node/test/diagnosticsChanged.spec.tsverifies the Copilot CLI diagnostic change push notification path using mocked VS Code diagnostics APIs.extensions/copilot/src/extension/chatSessions/copilotcli/vscode-node/test/getDiagnostics.spec.tsverifies theget_diagnosticsMCP-style tool contract for retrieving VS Code diagnostics.extensions/copilot/src/extension/completions-core/vscode-node/lib/src/prompt/test/defaultDiagnosticSettings.test.tsverifies parsing and defaulting behavior for diagnostic settings used in prompt context.
Core Testing Concepts
The official VS Code testing model is extension-driven: a testing extension discovers tests, publishes them as test items, and lets users run or debug those tests from editor surfaces. A test provider may represent a single assertion, a suite, or a hierarchy of cases, and it decides when discovery happens. The repository evidence mirrors that extensibility model by treating diagnostics, tools, and chat services as explicit contracts that can be exercised without a full manual UI session. Rather than assuming a feature works because it is visible in the workbench, the tests isolate the observable API behavior that the workbench, Copilot, or an extension depends on.
Diagnostics are an important bridge between testing, language intelligence, and AI-assisted workflows. The simulation provider registry exposes named providers such as tsc, tscIgnoreImportErrors, eslint, pyright, pylint, roslyn, cpp, and ruff, then routes getDiagnostics calls through the selected provider. That design lets simulations ask a uniform question—what diagnostics exist for these files—while preserving implementation differences between TypeScript, Python, C#, C++, and linting tools. In testing terms, it creates a stable harness around language-specific behavior, which is the same principle used by Test Explorer providers for framework-specific test discovery.
Sources: extensions/copilot/test/simulation/diagnosticProviders/index.ts
System-to-Code Mapping
| Testing concern | Repository evidence | What it proves |
|---|---|---|
| Remote search failure handling | codeOrDocsSearchErrors.spec.ts | Known search and repository errors become typed error classes, while unknown failures remain general errors. |
| Diagnostic simulation | diagnosticProviders/index.ts | Multiple language and linter providers can be selected through a common provider ID contract. |
| Chat debug logging | chatDebugFileLoggerService.spec.ts | Completed spans and emitted span events can be modeled with test helpers and a fake OpenTelemetry service. |
| Diagnostic push notifications | diagnosticsChanged.spec.ts | VS Code diagnostic changes are listened to, debounced, transformed, and broadcast as diagnostics_changed. |
| Diagnostic retrieval tool | getDiagnostics.spec.ts | The get_diagnostics tool registers correctly and returns URI-scoped or workspace-wide diagnostics. |
| Prompt diagnostic defaults | defaultDiagnosticSettings.test.ts | JSON settings parse safely and fall back to expected defaults for missing or invalid fields. |
The error mapping tests show a classic unit-test pattern: feed a constructor known protocol values and assert the resulting runtime type. Search repository errors such as inaccessible organization, unavailable documentation embeddings, and not-indexed states are expected to produce specific subclasses. General search errors such as max retries and endpoint access are handled separately. This matters for the testing experience because test failures should indicate the exact boundary that broke: server protocol interpretation, user access, indexing state, or generic fallback behavior. Typed errors make downstream UI and Copilot responses easier to validate.
Sources: extensions/copilot/src/platform/remoteSearch/test/node/codeOrDocsSearchErrors.spec.ts
Execution Flow
The diagnostic change tests demonstrate an event-driven flow that resembles how editor features react to workspace state. The test mocks the vscode module, captures the callback passed to languages.onDidChangeDiagnostics, then triggers the callback with mock URIs. After a fake timer advances, the mocked HTTP server is expected to broadcast a diagnostics_changed notification containing URI strings and normalized diagnostics. A separate test sends rapid changes and verifies that only one broadcast occurs, documenting an important runtime constraint: diagnostic notifications are debounced so consumers are not flooded during active editing or rebuilds.
The diagnostic retrieval tests cover the pull side of the same system. A mock MCP server receives a registered get_diagnostics tool, and tests assert that the tool exists before exercising its handler. When a URI is supplied, the handler returns diagnostics for that file. When no URI is supplied, it returns all diagnostics from the mocked VS Code API. The tests also verify severity normalization from VS Code’s numeric diagnostic severities into strings such as error, warning, information, and hint. That is the kind of precise contract a tool consumer needs before it can automate fixes or summarize failures.
Sources: extensions/copilot/src/extension/chatSessions/copilotcli/vscode-node/test/diagnosticsChanged.spec.ts, extensions/copilot/src/extension/chatSessions/copilotcli/vscode-node/test/getDiagnostics.spec.ts
Implementation Details
Several tests deliberately construct small fake services instead of booting the full workbench. The chat debug logger spec builds completed span objects with attributes for chat sessions, models, token counts, and tool calls, then drives a TestOTelService that exposes onDidCompleteSpan and onDidEmitSpanEvent events. This style keeps service tests deterministic while still preserving the shape of production telemetry inputs. It also shows that Copilot testing is not limited to visible chat responses: it includes the trace and logging surfaces needed to debug sessions after execution.
Prompt diagnostic settings are tested as a parsing boundary. DefaultDiagnosticSettings.from() returns undefined for null, undefined, empty strings, invalid JSON, and empty JSON objects. Valid JSON can provide warnings, maxLineDistance, and maxDiagnostics, with accepted warning modes including yes, no, and yesIfNoErrors. The tests also verify defensive defaults: invalid or missing warning values become no, and invalid distances fall back to a safe default. That behavior keeps prompt construction predictable even when configuration input is malformed or incomplete.
Sources: extensions/copilot/src/extension/chat/vscode-node/test/chatDebugFileLoggerService.spec.ts, extensions/copilot/src/extension/completions-core/vscode-node/lib/src/prompt/test/defaultDiagnosticSettings.test.ts
Practical Contributor Workflow
When adding or changing a VS Code testing-related feature, start by identifying the contract boundary. If the code maps a protocol payload into internal classes, write direct unit tests like the search error suite. If the feature listens to VS Code APIs, mock the relevant vscode entrypoints and assert the emitted output, as the diagnostic notification tests do. If the feature registers a tool or command, first assert registration, then call the handler with representative empty, scoped, and multi-file inputs. This produces failures that explain which public behavior changed rather than only that an implementation detail moved.
For AI-assisted testing paths, keep diagnostics and telemetry explicit. Diagnostics should include URI, file path where relevant, severity, source, optional code, and range, because those are the fields tools and chat agents use to reason about failures. Telemetry-oriented tests should use representative spans and events rather than opaque mocks, because debugging agents depends on session IDs, operation names, model names, and tool names being preserved. After changing a provider, notification, or prompt setting, run the closest unit or integration test first, then expand outward to the feature-level or smoke workflow that exercises the visible VS Code surface.
Next Steps
Read the debugging and launch configuration pages if the next task is to run tests under breakpoints. Read the Copilot and AI pages if the work involves diagnostics flowing into chat, tools, or agent sessions. Extension authors should also connect this repository evidence to the public Testing API model: implement discovery with a test controller, publish test items, and report results in a way that VS Code can display consistently in Test Explorer, editor decorations, and result views.