[yarikoptic-gitmate]

Summary

This adds a "Compatibility with dandi-archive instances" section to DEVELOPMENT.md, plus a one-line pointer in CLAUDE.md. The section says:

• Support for a new dandi-archive feature must be backward compatible with prior dandi-archive versions, in particular those deployed by the DANDI and EMBER instances (production and sandbox). Then dandi-cli can be released before dandi-archive releases or deploys the feature, and tested against sandbox instances that already have it. No synchronized releases of dandi-cli and dandi-archive are needed, and no synchronized deployments across all instances.
• How to decide whether a server supports a feature:
  1. by server_info["version"] if the releasing dandi-archive version is known (forks may report versions that cannot be compared);
  2. otherwise, and most likely, by one-time sensing, with the result remembered.
• Rules for the conditional, following current practice:
  • a comment that refers to the dandi-archive issue/PR and says when the fallback can be removed;
  • log the fallback at DEBUG or INFO;
  • test both paths;
  • remove the fallback once all known instances support the feature.
• Examples in the code, each linked to its lines in the 0.81.0 release.

This first step is documentation only, describing how this is done today. A follow-up PR will propose a registry of such conditionals (see "Next steps").

Ongoing: BULK_DELETE (WiP)

• Use the bulk asset deletion endpoint for all asset deletion #1960 (draft) sends all asset deletion through POST /dandisets/{id}/versions/{version}/assets/bulk-delete/. That endpoint comes from Add bulk asset deletion endpoint dandi-archive#2920, which is open and not in any release (dandi-archive is at 1.0.19). On a 404 that does not list missing assets, Use the bulk asset deletion endpoint for all asset deletion #1960 falls back to per-asset DELETE and remembers that on the DandiAPIClient.

• It shows the limits of sensing:

  • the 404 is ambiguous: an old server and a nonexistent version give the same response;
  • missing assets are reported only in a message string;
  • the result is remembered per client object, not per server.

  A features entry in /api/info/, or a structured 404 body (missing_asset_ids, which Use the bulk asset deletion endpoint for all asset deletion #1960 already accepts), would remove the ambiguity.

• Only support bulk asset deletion dandi-archive#2930 proposes removing the single-asset DELETE. Per this guideline the client keeps its fallback until all instances have bulk delete. The safer order is expand/contract: the server keeps the old endpoint while clients migrate.

Historical precedents

Lockstep (2021–2023, before LINC and EMBER). Server changes were followed right away by a client release, and older clients broke:

• Update for the version /info endpoint in dandi-api #575: version /info/ endpoint
• Include "dandiset" field in /uploads/initialize/ requests #863: embargo, dandiset in /uploads/initialize/
• Update client for change in Zarr entries API #1175: Zarr entries API
• Update for new Zarr upload procedure; remove no longer needed service-scripts #1202 and Update for fix to Zarr upload procedure #1215: new Zarr upload. 0.49.0 broke once the md5 encoding changed on the server.
• Add get_assets_by_glob() #981 (get_assets_by_glob) was held about 2.5 months until the server shipped its endpoint.

Server-driven gates in /api/info/:

• cli-minimal-version / cli-bad-versions were introduced in Add a get_instance() function #186 (redirector, 2020) and moved into /api/info/ with Add netlify plugin and redirection dandi-archive#1018.
  • The minimum was raised 0.14.2 → 0.51.0 → 0.60.0 → 0.74.0 (Boost minimal client version to be 0.51.0 since needed for fresh dandischema dandi-archive#1542, Require v0.60.0 of dandi-cli dandi-archive#1878, Display the correct CLI and required python version in the GUI dandi-archive#2714).
  • These gates only guard a server against old clients. They cannot protect a new client from an older server.
• Schema version:
  1. exact match required (Make upload() fail if client & server schema versions are not in sync #724);
  2. relaxed using the server's allowed_schema_versions (Allow for client to provide records with newer COMPATIBLE or older (upgradable) version of schema to the server #1732; field added in expose "allowed_schema_versions" to /info (/server-info) dandi-archive#2624 / Expose allowed schema versions at /info/ endpoint dandi-archive#2625);
  3. outgoing metadata downgraded to the server's schema (Downgrade outgoing metadata to server's schema version on upload #1898, Provide "downgrade" functionality to migrate and make dandi-cli use migrate! dandi-schema#343).
• Before that, dandischema was pinned and unpinned while servers lagged: Do not allow dandischema 0.10.1 - new schema not yet supported by dandi-archive #1419 / Revert "Do not allow dandischema 0.10.1 - new schema not yet supporteed by dandi-archive" #1420, Temporarily restrict dandischema requirement to < 0.10.2 #1458 / Revert "Temporarily restrict dandischema requirement to < 0.10.2" #1459, New release to unwind schema pin? #1635 / Unrestrict dandischema version #1637. Support multiple versions of target dandischema #1571 (support multiple target schemas) is still open.
• A "features" API was proposed in API call which would return information on supported features dandi-archive#274 (2020). It was closed in 2021 as "an issue for the CLI to solve" once the API reports its version. API semver was deferred in Any breaking change to the API should introduce a new .minor boost version tag dandi-archive#368.

Sensing and fallbacks in the client:

• _get_instance() tries /server-info and falls back to /api/info/ on 404 (Accept URLs as Dandi instance specifiers #1298).
• Zarr upload sends x-amz-tagging only if the presigned URL signs it (Add tagging to zarr upload if embargoed #1685).
  • A version check was rejected because LINC reported version as 0.0.post6586+g39387b8.
  • A TODO to make this strict "once LINC and EMBER update their API to match dandi-archive >= 0.12.14" is still on master.
• A pre-existing Zarr is detected by either the old or the new server error message (Add support for updated error text upon Zarr name conflicts #1848).
• Zarr multipart upload:
  • Zarr multipart upload tune ups: condition on server capability, fix progress reporting, harmonize variable names #1846 probed /uploads/zarr/initialize/ once per server for Use multi-part upload on zarr chunks larger than 5GB #1839.
  • In Use multi-part upload on zarr chunks larger than 5GB #1839 an alternative was proposed: raise cli-minimal-version on production and list the new client in cli-bad-versions on downstream instances. It was not adopted, because it makes other instances react to every client change. This guideline takes that position.
  • The successor, Add support for zarr multipart upload #1925 (draft), reads upload_type from the server's response, so an older server simply does not return the field.
• Paginated listings fall back to sequential next following (dandi/dandiapi.py, Fetch paginated results concurrently #1021). DANDI_PAGINATION_DISABLE_FALLBACK is an existing env var precedent for controlling a fallback.

Coordinated deployment instead of a fallback:

• Don't include x-amz-acl header in zarr upload #1710 (stop sending x-amz-acl): the merge was held about 3 weeks until EMBER confirmed its forks were synced.
• The staging → sandbox rename (Rename dandiarchive staging => sandbox #1668, Bring back dandi-staging to provide more graceful transition for 3rd party libraries #1694, Add support for all known instances in short URL format, removed dandi-staging and renamed EMBER* ones into EMBER-DANDI* #1750).

CI gap:

• dandi-cli integration tests always use the latest dandiarchive/dandiarchive-api image. A client PR that depends on an unmerged server PR therefore stays red (Use multi-part upload on zarr chunks larger than 5GB #1839, Use the bulk asset deletion endpoint for all asset deletion #1960), and older-server code paths are covered only by mocks.
• [TEST] CI: cross-repo PR matrix vs dandi-archive#2784 #1851 (draft) prototypes testing against a server PR named in the PR description.

Existing conditionals: removable now?

Links are to the 0.81.0 release. The verdicts are based on the source of the branches the instances deploy: upstream master, aplbrain/dandi-archive prod for EMBER, and lincbrain/linc-archive release for LINC. I could not query /api/info/ on the live servers from the environment where this was prepared, so please check before acting on it.

Code	Accommodates	Verdict
_get_instance(), /server-info → /api/info/ (dandi/utils.py)	GUI URLs (Netlify proxy) vs bare API hosts	keep
allowed_schema_versions fallback in check_schema_version() (dandi/dandiapi.py)	servers older than dandi-archive 0.19.0	keep: LINC lacks the field. The TODO about the field name (dandi/dandi-archive#2624) can go, since dandi/dandi-archive#2625 settled the name.
strict x-amz-tagging check, commented out (dandi/files/zarr.py)	servers before 0.12.14	EMBER is past it; LINC is not, so keep
old "Zarr already exists" text in mkzarr (dandi/files/zarr.py)	DRF before the upgrade in dandi-archive 1.0.0	keep: EMBER and LINC still send it, and upstream still raises it on a race
dandi-staging check in get_instance() (dandi/utils.py)	the removed instance id	candidate: unreachable for string ids, because known_instances[dandi_id] raises KeyError first
retry of downloads on HTTP 400 (dandi/download.py)	Girder (#87)	candidate: Girder support was removed in 2021, unless S3 400s merit a retry

To verify: if LINC still serves schema 0.6.9, check_schema_version() raises on dandi upload to LINC for every dandischema that dandi-cli allows. 0.6.9 is not in ALLOWED_TARGET_SCHEMAS, and it is in a different MAJOR.MINOR band than client schemas 0.7.0 and 0.8.0.

How other projects handle this

• DataLad and git-annex: AnnexRepo._check_version_kludges() computes named, version-dependent flags once and caches them. It has a "maybe" state for development snapshots. Flags are removed in batches when the minimum version is raised (fa58b48). A cron-only CI job tests the minimum git-annex version.
• Matrix: /versions returns spec versions plus unstable_features flags. Flags are dropped once a feature is in the spec, and an absent flag means unsupported. matrix-js-sdk's featureSupportResolver is a registry mapping each feature to its unstable flag(s) and the stable spec version, which is close to the registry proposed under "Next steps".
• Nextcloud: the OCS capabilities endpoint lists versioned per-app capabilities. The desktop client's bulkUpload() reads dav.bulkupload and can be forced on or off with NEXTCLOUD_BULK_UPLOAD=0/1. This is the closest analogue to BULK_DELETE plus a kill switch.
• Docker: the client negotiates the API version lazily, once per client. DOCKER_API_VERSION pins it and is meant only for when you must, or for debugging.
• git protocol v2: servers advertise capabilities, and the agent (version) string "MUST NOT be used to programmatically assume the presence or absence of particular features". The protocol.version knob mattered in practice: v2 was made the default in 2.26, demoted in 2.27 for rough edges, and restored in 2.29.
• OCI distribution: a client MUST fall back to the tag scheme when the referrers API returns 404. oras-go caches a tri-state support flag per repository. It tells a "repository unknown" 404 apart from "API unsupported", which is the same ambiguity Use the bulk asset deletion endpoint for all asset deletion #1960 faces.
• Kubernetes: kubectl caches discovery per host with a 6 h TTL. It falls back to client-side validation when the server lacks fieldValidation. Both feature gates and the skew policy define when old paths can go.
• BioBlend and Galaxy: BioBlend branches on /api/version and runs a weekly CI matrix over about 10 Galaxy releases.
• OpenStack microversions: server-side version pinning is too heavy for us. The Nova spec also warns that a per-deployment extension list became "a poor man's versioning mechanism", so a features list should be curated and pruned.
• Enforcing removal:
  • pip's deprecated(gone_in=...) turns into an error once pip reaches that version;
  • the deprecation package has fail_if_not_removed;
  • Django uses grep-able RemovedInDjangoXXWarning comments;
  • Chromium flags have an expiry milestone.

Next steps (follow-up PRs, not here)

• dandi-cli registry of server features. Each entry has a CODENAME, the dandi-archive PR, base_version (the version the feature is developed on, so older servers need no sensing), optional released_version, and a sensing function. Results are resolved once per server, keyed by API URL rather than per DandiAPIClient.
• DANDI_DEVEL_DISABLE_SERVER_FEATURES=BULK_DELETE,... to force the fallback when a feature turns out to be buggy.
• A scheduled (e.g. weekly) test over known_instances that reports which features each instance supports. It fails when a fallback is supported everywhere and can be removed. Keep in mind that scheduled workflows are disabled after 60 days of repository inactivity.
• dandi-archive: a features entry in /api/info/ generated from code. Keep old endpoints while clients migrate (Only support bulk asset deletion dandi-archive#2930).
• CI against the dandi-archive versions the instances deploy, not only the latest image.

🤖 Generated with Claude Code

https://claude.ai/code/session_014sJtxzEPbSxycxetp4DquU

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.82%. Comparing base (0d5f96e) to head (2939a09).

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1963      +/-   ##
==========================================
+ Coverage   90.76%   90.82%   +0.06%     
==========================================
  Files          92       92              
  Lines       14159    14159              
==========================================
+ Hits        12851    12860       +9     
+ Misses       1308     1299       -9     

Flag	Coverage Δ
unittests	90.82% <ø> (+0.06%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[yarikoptic]

keep it more factual to the moment and with better refs to the code for humans

[yarikoptic]

@waxlamp @jjnesbitt @naglepuff and @dandi/dandi-archive at large: I am initiating these guidelines so far within dandi-cli but hope we could distill a solution which we would implement between both dandi-archive and dandi-cli in a more robust fashion. See this PR description (alert: claude-composed but quite informative! yet to be fully curated but looked sensible, might be worth migrating parts into doc for the next stage if we decide to proceed on "centralized" solution) for more history/pointers etc than this brief addition to DEVELOPMENT.md provides. Also please review this text snippet on either anything to change to reflect our agreement from the recent Zoom session on Tue.