Migrate from databricks-sdk-go to sdk-go by denik · Pull Request #7070 · databricks/cli

denik · 2026-10-15T11:09:24Z

Changes

Migrate the CLI from github.com/databricks/databricks-sdk-go to the new github.com/databricks/sdk-go (one module per service, e.g. github.com/databricks/sdk-go/jobs/v2).

The old SDK is still used for authentication and profile resolution only (config, credentials/u2m, useragent, httplog); sdk-go itself depends on it for U2M. Every API call goes through sdk-go clients or, where sdk-go has no client, through a small hand-written REST layer.

Building blocks

libs/sdktypes/<module>/<vN> (generated by internal/sdktypesgen, ./task generate-sdktypes): JSON-tagged mirrors of the sdk-go model types. They are derived from sdk-go's unexported wire.go, with flattened oneofs, re-exported enums and generated <T>ToSDK/<T>FromSDK converters. sdk-go model types have no JSON tags and protobuf-style oneofs, so everything that is serialized uses the mirrors: bundle config, direct engine state, --json flags, output rendering and the test server. Fields are ordered by JSON name, like the previous SDK, so output order is unchanged.
libs/sdkclients: *sdkclients.Clients replaces both *databricks.WorkspaceClient and *databricks.AccountClient. It has one field per sdk-go module and is built from the CLI's resolved config.Config.
libs/apiclient: the shared HTTP transport and raw REST client. It keeps the previous SDK's behaviour where sdk-go differs:
- User-Agent composition.
- Retries on 429/503 honouring Retry-After.
- Inactivity-based HTTP timeout (DATABRICKS_BUNDLE_HTTP_TIMEOUT_SECONDS).
- Debug request logging.
- Normalization of non-JSON error bodies.
libs/apierrs: error classification with the previous SDK's semantics (Is(err, ErrNotFound), ErrorCode, the missing-object reclassification). It also strips sdk-go's databricks-api error: prefix from user-visible messages.
Hand-written clients for APIs sdk-go does not have: libs/workspaceapi, libs/bundledeployments, libs/qualitymonitors, libs/sqllegacy.
internal/cligen: retargeted to sdk-go. cmd/workspace and cmd/account are regenerated; 192 cli.json methods without an sdk-go equivalent call libs/apiclient directly.
bundle/config/resources and bundle/direct: rewritten on top of the mirror types.
bundle/internal/sdkmeta: keeps the bundle JSON schema keyed by the old type names, so the Python classes do not change.

ForceSendFields removed

sdk-go expresses "set to zero" with pointers, so all ForceSendFields support is gone (libs/structs, libs/dyn/convert, marshal wrappers). Two changes preserve the old zero-value behaviour:

An explicitly set zero scalar on API types (for example parent_path: "") survives typed round trips.
DropEmptyStrings keeps empty strings on required fields.

Behaviour changes and known gaps

Request bodies: sdk-go repeats path parameters in request bodies, for example request_object_type/request_object_id, name, full_name_arg, dashboard_id, pipeline_id and securable_*. Recorded requests in acceptance tests change accordingly. Recorded bodies are now written with sorted keys, so key-order noise is gone.
Oneof conflicts: sdk-go rejects requests and responses that set more than one member of a oneof. Bundle configs that set both autoscale and num_workers now get a warning, and num_workers is dropped. Other conflicts, such as two task types in one task or app env value together with value_from, are reported as only one of X, Y can be set.
Postgres field masks: sdk-go's FieldMask rejects oneof group names (spec.expiration, spec.suspension) that the Postgres API requires in update_mask. Bundle deploy sends those updates through libs/apiclient. databricks postgres update-{branch,endpoint,project} cannot use group names; use databricks api patch instead.
Fields missing from sdk-go:
- Model serving workload_type.
- Pipeline cluster init_scripts and cluster_log_conf destinations other than dbfs/s3.
- The --purpose flag on credentials commands.
- Hidden --page-size on quality-monitor-v2 list-quality-monitor.
- clusters list --sort-by.
- Lakeview create etag.
- --read-mask on agent-kit memory commands.
- workspace-conf get-status --keys.
SCIM v1 commands (users, groups, service-principals) are thin wrappers over the v2 commands. They gain --limit.
Explicit nulls: a null on a pointer field (for example run_job_task: with no value) can't be represented, so the "required field job_id" warning for that case is gone.
Plan output: bundle plan -o json no longer prints zero-valued remote fields such as "remote": "" or embed_credentials: false.
Error messages from request construction lose the failed during request visitor: prefix.