Repository navigation
feat: add general experimentation infrastructure and telemetry attribution - #1873
Stella Huang (StellaHuang95) wants to merge 4 commits into
Conversation
…ution Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
f03f8ca to
411e024
Compare
Prevent cached assignments from being reused when resolved audience parameters such as the VS Code language change. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Follow-up on the resolved cache-key thread: I think In
Those responses share this
The same mismatch applies to Could the cache identity include the SDK-provided built-in targeting dimensions as well? A regression test with only |
Invalidate cached assignments when VS Code's built-in TAS targeting values change, including language, editor version, and application name. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Eleanor Boyd (@eleanorjboyd) Thanks for catching this. Addressed in The cache namespace now includes the SDK-provided built-in targeting context—normalized VS Code application version, app/build name, legacy MachineId, and language—in addition to the caller-configured resolved assignment parameters. I also added the requested regression coverage using an identity-only configuration and verified that language, VS Code version, and app-name changes each produce a cache miss. Validation passed with lint, TypeScript compilation, 37 focused experimentation tests, and the full unit suite (2,638 passing, 6 pending). |
Summary
Prepare reusable experimentation infrastructure for the Python Environments extension without enabling an experiment or changing feature settings.
vscode-tas-clientwith one activation-owned service, validated publisher configuration, isolatedglobalStatecaching, defaulted treatment queries, bounded HTTPS requests, and consent-aware cancellation and disposal.abexp.assignmentcontextto ordinary events, explicitly reported errors, and automatically captured exceptions. Preserve consent, data cleaning, capture-time attribution, and existing event names without duplicate automatic-error reporting.Scope and rollout
The publisher-owned
experimentationblock is intentionally absent. This build makes no extension TAS requests; existing consented telemetry continues, with anotConfiguredinitialization diagnostic. No feature setting is exposed or enabled, and no A/B test is launched.Update TypeScript to support the current TAS SDK declarations and
@vscode/extension-telemetryto 1.5.2 for its public automatic-error logger options. Generated native tooling and local verification artifacts are excluded.Validation
Re-run for this PR:
npm run lintnpm run compile-testsnpm run compilegit diff --cached --checkThe implementation session also exercised Windows smoke/integration/discovery workflows, an isolated environment/package lifecycle, installed release VSIX behavior, and real-SDK/telemetry paths with controlled responses. Those broader checks were not repeated during this PR handoff.
Follow-up and limits
Live rollout still requires an approved endpoint, randomization identity and targeting contract, integration review, metrics/scorecard setup, and A/A verification. MachineId is not assumed to be DevDeviceId.
Production TAS authentication/proxies, non-Windows and remote hosts, and long-duration polling remain unverified. The earlier session reproduced independent multi-root selection and shutdown issues on the unchanged base commit; unrelated production behavior is deliberately outside this change.