Skip to content

Add support for breakglass for immutable folders - #6774

Open
andrewnester wants to merge 17 commits into
mainfrom
feat/immutable/breakglass
Open

Add support for breakglass for immutable folders#6774
andrewnester wants to merge 17 commits into
mainfrom
feat/immutable/breakglass

Conversation

@andrewnester

@andrewnester andrewnester commented Sep 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Changes

Add support for breakglass for immutable folders

Why

Immutable folders can be "break-glassed" in an emergency. In this case, CLI should prevent a normal deploy if the previously used snapshot was broken. With --force, it should allow to deploy to a new suffixed path and the deployments after should continue as normal

Note: there is no REST API endpoint to check the status of folder yet so we use GraphQL one for now

Tests

Added an acceptance test

@github-actions

github-actions Bot commented Sep 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Approval status: pending

/acceptance/bundle/ - needs approval

11 files changed
Suggested: @denik
Also eligible: @janniklasrose, @pietern, @shreyas-goenka, @anton-107, @lennartkats-db

/bundle/ - needs approval

7 files changed
Suggested: @denik
Also eligible: @janniklasrose, @pietern, @shreyas-goenka, @anton-107, @lennartkats-db

General files (require maintainer)

4 files changed
Based on git history:

  • @denik -- recent work in libs/testserver/, .nextchanges/bundles/, bundle/direct/dresources/

Any maintainer (@anton-107, @denik, @pietern, @shreyas-goenka, @simonfaltum, @renaudhartert-db, @janniklasrose, @lennartkats-db, @rugpanov, @rclarey) can approve all areas.
See OWNERS for ownership rules.

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Sep 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 779156f

Run: 39569649343

Env ❌​FAIL ✅​pass 🙈​skip Time
❌​ aws linux-2core-8gb 20 280 18 9:41
❌​ aws-windows-latest-4core-16gb 20 282 16 19:55
❌​ azure linux-2core-8gb 20 279 18 9:19
❌​ azure-windows-latest-4core-16gb 20 281 16 20:50
❌​ gcp linux-2core-8gb 18 280 19 8:44
❌​ gcp-windows-latest-4core-16gb 18 282 17 18:05
20 interesting tests: 20 FAIL
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
❌​ TestAccept ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/deploy/immutable/basic ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/deploy/immutable/basic/DMS= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/deploy/immutable/breakglass ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/deploy/immutable/breakglass/DMS= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/deploy/immutable/can-manage ❌​F ❌​F ❌​F ❌​F 🙈​s 🙈​s
❌​ TestAccept/bundle/deploy/immutable/can-manage/DMS= ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/deploy/immutable/destroy-redeploy ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/deploy/immutable/destroy-redeploy/DMS= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/deploy/immutable/dev-mode ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/deploy/immutable/dev-mode/DMS= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/deploy/immutable/legacy-state ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/deploy/immutable/legacy-state/DMS= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/deploy/immutable/missing-snapshot ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/deploy/immutable/missing-snapshot/DMS= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/deploy/immutable/no-artifacts ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/deploy/immutable/no-artifacts/DMS=/READPLAN= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/deploy/immutable/no-artifacts/DMS=/READPLAN=1 ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/deploy/immutable/permissions-change ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/deploy/immutable/permissions-change/DMS= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F

pietern
pietern reviewed Sep 25, 2026
View reviewed changes
Comment thread .nextchanges/bundles/immutable-breakglass.md Outdated
Comment thread bundle/deploy/snapshot/upload.go
return 0, nil
}

entry, ok := b.DeploymentBundle.StateDB.GetResourceEntry(resources.SnapshotKey)

@pietern pietern Sep 25, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This reads state and makes a determination. Shouldn't that live in a DoRead of the snapshot?

I.e. seeing a different generation, or a "broken_glass: true" on DoRead can trigger recreation.

@andrewnester andrewnester Sep 25, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a couple of blockers why it is not in DoRead, mainly the --force flag, which is not available at the resource level

It's also conceptually similar to existing CheckDashboardsModifiedRemotely check we have

Comment thread libs/snapshot/client.go
// ContentSubdir is the fixed subfolder the snapshot API appends to a caller-supplied
// relative_path to form the immutable content path (e.g. "<root>/<relative_path>/snapshot").
// The content path is what the create API echoes back and what InspectSnapshot inspects.
const ContentSubdir = "snapshot"

@pietern pietern Sep 25, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this happen?

@andrewnester andrewnester Sep 25, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's related to the fact that when we upload zip, we create 2 folders: one snapshot which can be broken glass and one is .internal which is not and contains a break glass marker and some other metadata

Comment thread libs/snapshot/client.go Outdated
@andrewnester
andrewnester requested review from a team as code owners September 25, 2026 10:11
@github-actions github-actions Bot added the DABs DABs related issues label Sep 25, 2026
pietern
pietern reviewed Sep 25, 2026
View reviewed changes
Comment thread bundle/deploy/snapshot/upload.go
Comment thread libs/snapshot/client.go Outdated
@andrewnester
andrewnester requested a review from pietern October 13, 2026 15:14
@github-actions

github-actions Bot commented Oct 20, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Commit: 779156fa9 | Workflow run

Regression test report

Selected 12 changed tests (limit=20, 0 not selected)

16 tests: regression=16 unreleased=0 coverage=0 failing=0

test branch main (0a32c93) latest (v1.21.0)
TestAccept/bundle/deploy/immutable/basic/DMS= ✅ ❌ ❌
TestAccept/bundle/deploy/immutable/breakglass/DMS= ✅ ❌ ❌
TestAccept/bundle/deploy/immutable/can-manage/DMS= ✅ ❌ ❌
TestAccept/bundle/deploy/immutable/destroy-redeploy/DMS= ✅ ❌ ❌
TestAccept/bundle/deploy/immutable/dev-mode/DMS= ✅ ❌ ❌
TestAccept/bundle/deploy/immutable/legacy-state/DMS= ✅ ❌ ❌
TestAccept/bundle/deploy/immutable/missing-snapshot/DMS= ✅ ❌ ❌
TestAccept/bundle/deploy/immutable/no-artifacts/DMS=/READPLAN= ✅ ❌ ❌
TestAccept/bundle/deploy/immutable/no-artifacts/DMS=/READPLAN=1 ✅ ❌ ❌
TestAccept/bundle/deploy/immutable/permissions-change/DMS= ✅ ❌ ❌
TestAccept/bundle/dms/immutable-folder/DMS=true/READPLAN= ✅ ❌ ❌
TestAccept/bundle/dms/immutable-folder/DMS=true/READPLAN=1 ✅ ❌ ❌
TestAccept/bundle/refschema/DMS= ✅ ❌ ❌
TestAccept/bundle/refschema/DMS=true ✅ ❌ ❌
TestAccept/bundle/resources/apps/immutable/DMS= ✅ ❌ ❌
TestAccept/bundle/resources/apps/immutable/DMS=true ✅ ❌ ❌
TestAccept/bundle/deploy/immutable/basic/DMS=: branch ✅ | main (0a32c93) ❌ | latest (v1.21.0) ❌

main (0a32c93):

acceptance_test.go:1307: Diff:
        --- bundle/deploy/immutable/basic/output.txt
        +++ /tmp/TestAcceptbundledeployimmutablebasicDMS=2890332949/002/output.txt
        @@ -15,7 +15,7 @@
         [
           {
             "notebook_task": {
        -      "notebook_path": "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot/files/src/notebook"
        +      "notebook_path": "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]/files/src/notebook"
             },
             "task_key": "notebook_task"
           },
        @@ -30,7 +30,7 @@
           {
             "environment_key": "env",
             "spark_python_task": {
        -      "python_file": "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot/files/src/main.py"
        +      "python_file": "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]/files/src/main.py"
             },
             "task_key": "spark_python_task"
           }
        @@ -42,7 +42,7 @@
             "environment_key": "env",
             "spec": {
               "dependencies": [
        -        "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot/artifacts/.internal/immutable-0.0.1-py3-none-any.whl"
        +        "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]/artifacts/.internal/immutable-0.0.1-py3-none-any.whl"
               ],
               "environment_version": "4"
             }
        @@ -54,8 +54,8 @@
           "action": "create",
           "new_state": {
             "value": {
        -      "relative_path": "[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]",
        -      "full_path": "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot",
        +      "relative_path": "[SNAPSHOT_HASH]/[SNAPSHOT_HASH]",
        +      "full_path": "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]",
               "bundle_id": "[SNAPSHOT_HASH]",
               "acl": [
                 {
        @@ -63,142 +63,27 @@
                   "permission_level": "CAN_READ"
                 }
               ],
        -      "can_manage": [
        -        {
        -          "user_name": "[USERNAME]"
        -        }
        -      ],
        -      "zip_path": "[TEST_TMP_DIR]/.databricks/bundle/default/snapshots/[SNAPSHOT_HASH]-[UUID].zip"
        +      "zip_path": "[TEST_TMP_DIR]/.databricks/bundle/default/snapshots/[SNAPSHOT_HASH].zip"
             }
           }
         }
         
         >>> [CLI] bundle deploy
         Building python_artifact...
        -Created internal_immutable_snapshots.immutable
        -Created jobs.my_job
        -Files: 0 uploaded, 0 deleted
        -Resources: 2 created, 0 changed, 0 deleted, 0 unchanged
        +Error: cannot create resources.internal_immutable_snapshots.immutable: snapshot upload: No stub found for pattern: POST /api/2.0/repos/snapshots (501 <nil>)
         
        ->>> jq .settings.tasks job.json
        -"/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot/files/src/main.py"
        +Endpoint: POST [DATABRICKS_URL]/api/2.0/repos/snapshots
        +HTTP Status: 501 Not Implemented
        +API error_code: <nil>
        +API message: No stub found for pattern: POST /api/2.0/repos/snapshots
         
        ->>> jq .settings.tasks job.json
        -"/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot/files/src/notebook"
        +Error: cannot create resources.jobs.my_job: dependency failed: resources.internal_immutable_snapshots.immutable
         
        ->>> jq .settings.environments[0].spec.dependencies job.json
        -[
        -  "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot/artifacts/.internal/immutable-0.0.1-py3-none-any.whl"
        -]
        -
        ->>> title Check snapshot plan after deploy
        -
        -=== Check snapshot plan after deploy
        ->>> [CLI] bundle plan -o json
        -Building python_artifact...
        -
        ->>> jq .plan["resources.internal_immutable_snapshots.immutable"] plan.json
        -{
        -  "action": "recreate",
        -  "new_state": {
        -    "value": {
        -      "relative_path": "[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]",
        -      "full_path": "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot",
        -      "bundle_id": "[SNAPSHOT_HASH]",
        -      "acl": [
        -        {
        -          "user_name": "[USERNAME]",
        -          "permission_level": "CAN_READ"
        -        }
        -      ],
        -      "can_manage": [
        -        {
        -          "user_name": "[USERNAME]"
        -        }
        -      ],
        -      "zip_path": "[TEST_TMP_DIR]/.databricks/bundle/default/snapshots/[SNAPSHOT_HASH]-[UUID].zip"
        -    }
        -  },
        -  "remote_state": {
        -    "relative_path": "[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]",
        -    "full_path": "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot"
        -  },
        -  "changes": {
        -    "acl": {
        -      "action": "skip",
        -      "reason": "missing_in_remote",
        -      "old": [
        -        {
        -          "user_name": "[USERNAME]",
        -          "permission_level": "CAN_READ"
        -        }
        -      ],
        -      "new": [
        -        {
        -          "user_name": "[USERNAME]",
        -          "permission_level": "CAN_READ"
        -        }
        -      ],
        -      "remote": null
        -    },
        -    "bundle_id": {
        -      "action": "skip",
        -      "reason": "missing_in_remote",
        -      "old": "[SNAPSHOT_HASH]",
        -      "new": "[SNAPSHOT_HASH]",
        -      "remote": ""
        -    },
        -    "can_manage": {
        -      "action": "skip",
        -      "reason": "missing_in_remote",
        -      "old": [
        -        {
        -          "user_name": "[USERNAME]"
        -        }
        -      ],
        -      "new": [
        -        {
        -          "user_name": "[USERNAME]"
        -        }
        -      ]
        -    },
        -    "full_path": {
        -      "action": "recreate",
        -      "reason": "immutable",
        -      "old": "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot",
        -      "new": "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot",
        -      "remote": "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot"
        -    },
        -    "relative_path": {
        -      "action": "recreate",
        -      "reason": "immutable",
        -      "old": "[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]",
        -      "new": "[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]",
        -      "remote": "[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]"
        -    },
        -    "zip_path": {
        -      "action": "recreate",
        -      "reason": "immutable",
        -      "old": "[TEST_TMP_DIR]/.databricks/bundle/default/snapshots/[SNAPSHOT_HASH]-[UUID].zip",
        -      "new": "[TEST_TMP_DIR]/.databricks/bundle/default/snapshots/[SNAPSHOT_HASH]-[UUID].zip",
        -      "remote": ""
        -    }
        -  }
        -}
        -
        ->>> [CLI] bundle deploy
        -Building python_artifact...
        -Recreated internal_immutable_snapshots.immutable
        -Updated jobs.my_job
         Files: 0 uploaded, 0 deleted
        -Resources: 1 created, 1 changed, 1 deleted, 0 unchanged
         
        -=== Check that job uses a new snapshot: Snapshot changed as expected
        -
         >>> [CLI] bundle destroy --auto-approve
        -The following resources will be deleted:
        -  delete resources.jobs.my_job
        -
         All files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/test-bundle-immutable-[UNIQUE_NAME]/default
         
        -Destroy: 1 deleted
        +Destroy: 0 deleted
        +
        +Exit code: 1
        
    acceptance_test.go:1126: Test produced unexpected files:
        plan.json
FAIL

latest (v1.21.0): same output as main (0a32c93)

TestAccept/bundle/deploy/immutable/breakglass/DMS=: branch ✅ | main (0a32c93) ❌ | latest (v1.21.0) ❌

main (0a32c93):

acceptance_test.go:1307: Diff:
        --- bundle/deploy/immutable/breakglass/output.txt
        +++ /tmp/TestAcceptbundledeployimmutablebreakglassDMS=1167916243/002/output.txt
        @@ -1,42 +1,20 @@
         
         === Deploy the immutable snapshot
         >>> [CLI] bundle deploy
        -Created internal_immutable_snapshots.immutable
        -Created jobs.my_job
        -Files: 0 uploaded, 0 deleted
        -Resources: 2 created, 0 changed, 0 deleted, 0 unchanged
        -First snapshot path: /Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot/files/src/main.py
        +Error: cannot create resources.internal_immutable_snapshots.immutable: snapshot upload: No stub found for pattern: POST /api/2.0/repos/snapshots (501 <nil>)
         
        -=== Break the glass out of band
        ->>> [CLI] api post /api/2.0/snapshots:breakGlass --json {"snapshot_content_path": "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot"}
        -true
        +Endpoint: POST [DATABRICKS_URL]/api/2.0/repos/snapshots
        +HTTP Status: 501 Not Implemented
        +API error_code: <nil>
        +API message: No stub found for pattern: POST /api/2.0/repos/snapshots
         
        -=== Deploy without --force fails
        ->>> musterr [CLI] bundle deploy
        -Error: the last deployment was modified in the workspace. Changes might be overwritten by this deployment. To confirm it, use the --force flag
        +Error: cannot create resources.jobs.my_job: dependency failed: resources.internal_immutable_snapshots.immutable
         
         Files: 0 uploaded, 0 deleted
         
        -=== Deploy with --force moves to a new snapshot
        ->>> [CLI] bundle deploy --force
        -Recreated internal_immutable_snapshots.immutable
        -Updated jobs.my_job
        -Files: 0 uploaded, 0 deleted
        -Resources: 1 created, 1 changed, 1 deleted, 0 unchanged
        -Second snapshot path: /Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot/files/src/main.py
        -OK: --force created a new snapshot path
        -
        -=== Redeploy reuses the new snapshot
        ->>> [CLI] bundle deploy
        -Files: 0 uploaded, 0 deleted
        -Resources: 0 created, 0 changed, 0 deleted, 2 unchanged
        -Third snapshot path: /Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot/files/src/main.py
        -OK: new snapshot reused on redeploy
        -
         >>> [CLI] bundle destroy --auto-approve
        -The following resources will be deleted:
        -  delete resources.jobs.my_job
        -
         All files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/test-bundle-immutable-breakglass-[UNIQUE_NAME]/default
         
        -Destroy: 1 deleted
        +Destroy: 0 deleted
        +
        +Exit code: 1

latest (v1.21.0): same output as main (0a32c93)

TestAccept/bundle/deploy/immutable/can-manage/DMS=: branch ✅ | main (0a32c93) ❌ | latest (v1.21.0) ❌

main (0a32c93):

server.go:281: No handler for URL: /api/2.0/repos/snapshots
        Body: [2208 bytes] --fb5a2b8f95db53ceb2a488e5fd2f5ff596bf2c981efcc5af98d92ee17560
        Content-Disposition: form-data; name="snapshot_id"
        
        9c492bc95344722f15c8f517a34e595d91372b0a137fc871d24df9211af2634a
        --fb5a2b8f95db53ceb2a488e5fd2f5ff596bf2c981efcc5af98d92ee17560
        Content-Disposition: form-data; name="bundle_id"
        
        c4390ac68c4280f2453d19b4a56719da35e14607b5b958699a3051bcae2653b0
        --fb5a2b8f95db53ceb2a488e5fd2f5ff596bf2c981efcc5af98d92ee17560
        Content-Disposition: form-data; name="access_control_list"
        
        [{"user_name":"aaaaaaaa-bbbb-4ccc-dddd-eeeeeeeeeeee","permission_level":"CAN_READ"},{"user_name":"deco-test-user@databricks.com","permission_level":"CAN_READ"}]
        --fb5a2b8f95db53ceb2a488e5fd2f5ff596bf2c981efcc5af98d92ee17560
        Content-Disposition: form-data; name="file"; filename="snapshot.zip"
        Content-Type: application/zip
        
        PK����������!(��������������	�files/databricks.ymlUT����Cm8|�͎�0���~��w��.>-Z������q�0񟺝���GN�����K���ewc�����0��JR�jj��X����&�1��a�k����p񡓑џ�^�u����b�)U-��`�9���*��T!�^��$
��h�`���d��Oǿ����(�v�Փ�z���_=V�ػA�.G��$��h���n�����r���\0�
�[�e��Mz1�@��)ȃ-s��d����u��h�/�} �۝��E�i[�O���sN����)M�e����+MR�=�oAv"�����~����PK���OH���������PK����������!(��������������	�files/databricks.yml.tmplUT����Cm8|��n� ���<�*ʕ��Y�P����&9#�lZbX�`�~�
        �M�F�\l�~3,4=�-
        �@ڡ��)���ƹ>��"o5q�I�!_����I���K��#`4�)k[Jn�u���Q@�=2�0:���
        ���V���V����$����F5/�>��Q���U�}�{f,b�}lqj��f���Q]}3�����X��S��E|2T����t�T��'U�����{E������6��ۭӆ6a��!
&z*�3��4��я���������U����x����b���PK��"3����������PK����������!(��������������	�files/src/main.pyUT����Cm8�'���print("Hello from Spark Python Task!")
        ��PK�����'.���'���PK����������!(������������"�	�.databricks/snapshot-metadata.jsonUT����Cm8�W���{"permissions_hash":"8cbff342214bd1b8d3d6120b10cd8a2e62c6a04998aa7eb1a94ede0f07487789"}��PK������^���W���PK������������!(�OH�����������	���������������files/databricks.ymlUT����Cm8PK������������!("3������������	�����������V���files/databricks.yml.tmplUT����Cm8PK������������!(���'.���'�����	���������������files/src/main.pyUT����Cm8PK������������!(����^���W���"�	���������������.databricks/snapshot-metadata.jsonUT����Cm8PK����������<���������
        --fb5a2b8f95db53ceb2a488e5fd2f5ff596bf2c981efcc5af98d92ee17560--
        
        
        For acceptance tests, add this to test.toml:
        [[Server]]
        Pattern = "POST /api/2.0/repos/snapshots"
        Response.Body = '<response body here>'
        # Response.StatusCode = <response code if not 200>
    acceptance_test.go:1307: Diff:
        --- bundle/deploy/immutable/can-manage/output.txt
        +++ /tmp/TestAcceptbundledeployimmutablecan-manageDMS=1028998772/002/output.txt
        @@ -13,23 +13,19 @@
         See https://docs.databricks.com/dev-tools/bundles/permissions.html to learn more about permission configuration.
           in databricks.yml:8:3
         
        -Created internal_immutable_snapshots.immutable
        -Created jobs.my_job
        -Created jobs.my_job.permissions
        -Files: 0 uploaded, 0 deleted
        -Resources: 3 created, 0 changed, 0 deleted, 0 unchanged
        +Error: cannot create resources.internal_immutable_snapshots.immutable: snapshot upload: No stub found for pattern: POST /api/2.0/repos/snapshots (501 <nil>)
         
        -=== Check can_manage_principals sent to the snapshot create API
        ->>> print_requests.py //api/2.0/snapshots
        -[
        -  {
        -    "service_principal_name": "[USERNAME]"
        -  },
        -  {
        -    "user_name": "deco-test-user@databricks.com"
        -  }
        -]
        +Endpoint: POST [DATABRICKS_URL]/api/2.0/repos/snapshots
        +HTTP Status: 501 Not Implemented
        +API error_code: <nil>
        +API message: No stub found for pattern: POST /api/2.0/repos/snapshots
         
        +Error: cannot create resources.jobs.my_job: dependency failed: resources.internal_immutable_snapshots.immutable
        +
        +Error: cannot create resources.jobs.my_job.permissions: dependency failed: resources.internal_immutable_snapshots.immutable
        +
        +Files: 0 uploaded, 0 deleted
        +
         >>> [CLI] bundle destroy --auto-approve
         Recommendation: permissions section should explicitly include the current deployment identity '[USERNAME]' or one of its groups
         If it is not included, CAN_MANAGE permissions are only applied if the present identity is used to deploy.
        @@ -43,9 +37,8 @@
         See https://docs.databricks.com/dev-tools/bundles/permissions.html to learn more about permission configuration.
           in databricks.yml:8:3
         
        -The following resources will be deleted:
        -  delete resources.jobs.my_job
        -
         All files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/test-bundle-immutable-can-manage-[UNIQUE_NAME]/default
         
        -Destroy: 1 deleted
        +Destroy: 0 deleted
        +
        +Exit code: 1

latest (v1.21.0):

server.go:281: No handler for URL: /api/2.0/repos/snapshots
        Body: [2197 bytes] --4ce29b9245ceb8dd219a17a7423079dbcab16eb518098705bd388646e2e0
        Content-Disposition: form-data; name="snapshot_id"
        
        f7656eb62f90ca726efc0fbe17c46643aaaf1bb0ac86c404e76fe677af66d4ca
        --4ce29b9245ceb8dd219a17a7423079dbcab16eb518098705bd388646e2e0
        Content-Disposition: form-data; name="bundle_id"
        
        b5c5c2c05b8421dc1dc9bf59d2939eb73eb4199d906de00bbea413e7832f8fc5
        --4ce29b9245ceb8dd219a17a7423079dbcab16eb518098705bd388646e2e0
        Content-Disposition: form-data; name="access_control_list"
        
        [{"user_name":"aaaaaaaa-bbbb-4ccc-dddd-eeeeeeeeeeee","permission_level":"CAN_READ"},{"user_name":"deco-test-user@databricks.com","permission_level":"CAN_READ"}]
        --4ce29b9245ceb8dd219a17a7423079dbcab16eb518098705bd388646e2e0
        Content-Disposition: form-data; name="file"; filename="snapshot.zip"
        Content-Type: application/zip
        
        PK����������!(��������������	�files/databricks.ymlUT����Cm8|��n�0���~��w��ҋOEU�Sy�k�,ʼn��]'%o_���!�dkfv�y�1����H��@!)z���q,��������������SǘS>��e����|�J�i ��R�P+o���CKl��HJ
�ы����4��(������n���T�������%������-�lػ^�.G��$���\���|���v�Y���ƹ�.RA���W�*؞f�2 �v��1'[�[
        �=sg�\���d`��v��>���.Fi�S]�� �I=Y�p��,T��A��&�p���J������PK��G��M�������PK����������!(��������������	�files/databricks.yml.tmplUT����Cm8|��j�0���z�%��ғn����
mⳐ�M�X?f%����$8�r��ofw��������ڡ��1�Y�ƹ!��"o��N{��|]W��Z��(%c��#��>i���#����I@���둜�������������ReQ������I�E���?�z/��,��'c�1��e�94�/���94����nʆ��t����YP�N�b��S��~�WY�����������(`���n�6~�O����P��>�B�#{B��^D�V�>�Y�F�|c����������PK��"3����������PK����������!(��������������	�files/src/main.pyUT����Cm8*(��+�P�H���WH+��U�.H,�V��,���S�I,�VT��������PK�����'-���'���PK����������!(������������"�	�.databricks/snapshot-metadata.jsonUT����Cm8���
�1���]:�p����(4w��D����}ǻ>���z=�~z��6t����!Gbh�9� �V�SMZӁ��]*Ѝ+�6�����������PK������S���W���PK������������!(G��M���������	���������������files/databricks.ymlUT����Cm8PK������������!("3������������	�����������W���files/databricks.yml.tmplUT����Cm8PK������������!(���'-���'�����	���������������files/src/main.pyUT����Cm8PK������������!(����S���W���"�	���������������.databricks/snapshot-metadata.jsonUT����Cm8PK����������<���������
        --4ce29b9245ceb8dd219a17a7423079dbcab16eb518098705bd388646e2e0--
        
        
        For acceptance tests, add this to test.toml:
        [[Server]]
        Pattern = "POST /api/2.0/repos/snapshots"
        Response.Body = '<response body here>'
        # Response.StatusCode = <response code if not 200>
    acceptance_test.go:1307: Diff:
        --- bundle/deploy/immutable/can-manage/output.txt
        +++ /tmp/TestAcceptbundledeployimmutablecan-manageDMS=1212669755/002/output.txt
        @@ -13,23 +13,19 @@
         See https://docs.databricks.com/dev-tools/bundles/permissions.html to learn more about permission configuration.
           in databricks.yml:8:3
         
        -Created internal_immutable_snapshots.immutable
        -Created jobs.my_job
        -Created jobs.my_job.permissions
        -Files: 0 uploaded, 0 deleted
        -Resources: 3 created, 0 changed, 0 deleted, 0 unchanged
        +Error: cannot create resources.internal_immutable_snapshots.immutable: snapshot upload: No stub found for pattern: POST /api/2.0/repos/snapshots (501 <nil>)
         
        -=== Check can_manage_principals sent to the snapshot create API
        ->>> print_requests.py //api/2.0/snapshots
        -[
        -  {
        -    "service_principal_name": "[USERNAME]"
        -  },
        -  {
        -    "user_name": "deco-test-user@databricks.com"
        -  }
        -]
        +Endpoint: POST [DATABRICKS_URL]/api/2.0/repos/snapshots
        +HTTP Status: 501 Not Implemented
        +API error_code: <nil>
        +API message: No stub found for pattern: POST /api/2.0/repos/snapshots
         
        +Error: cannot create resources.jobs.my_job: dependency failed: resources.internal_immutable_snapshots.immutable
        +
        +Error: cannot create resources.jobs.my_job.permissions: dependency failed: resources.internal_immutable_snapshots.immutable
        +
        +Files: 0 uploaded, 0 deleted
        +
         >>> [CLI] bundle destroy --auto-approve
         Recommendation: permissions section should explicitly include the current deployment identity '[USERNAME]' or one of its groups
         If it is not included, CAN_MANAGE permissions are only applied if the present identity is used to deploy.
        @@ -43,9 +37,8 @@
         See https://docs.databricks.com/dev-tools/bundles/permissions.html to learn more about permission configuration.
           in databricks.yml:8:3
         
        -The following resources will be deleted:
        -  delete resources.jobs.my_job
        -
         All files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/test-bundle-immutable-can-manage-[UNIQUE_NAME]/default
         
        -Destroy: 1 deleted
        +Destroy: 0 deleted
        +
        +Exit code: 1
TestAccept/bundle/deploy/immutable/destroy-redeploy/DMS=: branch ✅ | main (0a32c93) ❌ | latest (v1.21.0) ❌

main (0a32c93):

acceptance_test.go:1307: Diff:
        --- bundle/deploy/immutable/destroy-redeploy/output.txt
        +++ /tmp/TestAcceptbundledeployimmutabledestroy-redeployDMS=1096163402/002/output.txt
        @@ -1,58 +1,20 @@
         
         === Deploy the immutable snapshot
         >>> [CLI] bundle deploy
        -Created internal_immutable_snapshots.immutable
        -Created jobs.my_job
        -Files: 0 uploaded, 0 deleted
        -Resources: 2 created, 0 changed, 0 deleted, 0 unchanged
        -First snapshot path: /Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot/files/src/main.py
        +Error: cannot create resources.internal_immutable_snapshots.immutable: snapshot upload: No stub found for pattern: POST /api/2.0/repos/snapshots (501 <nil>)
         
        -=== Destroy the bundle
        ->>> [CLI] bundle destroy --auto-approve
        -The following resources will be deleted:
        -  delete resources.jobs.my_job
        +Endpoint: POST [DATABRICKS_URL]/api/2.0/repos/snapshots
        +HTTP Status: 501 Not Implemented
        +API error_code: <nil>
        +API message: No stub found for pattern: POST /api/2.0/repos/snapshots
         
        -All files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/test-bundle-immutable-redeploy-[UNIQUE_NAME]/default
        +Error: cannot create resources.jobs.my_job: dependency failed: resources.internal_immutable_snapshots.immutable
         
        -Destroy: 1 deleted
        -
        -=== Deploy again moves to a new snapshot instead of the one left behind
        ->>> [CLI] bundle deploy
        -Created internal_immutable_snapshots.immutable
        -Created jobs.my_job
         Files: 0 uploaded, 0 deleted
        -Resources: 2 created, 0 changed, 0 deleted, 0 unchanged
        -Second snapshot path: /Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot/files/src/main.py
        -OK: deployment moved to a new snapshot
         
        -=== Redeploy reuses the new snapshot
        ->>> [CLI] bundle deploy
        -Files: 0 uploaded, 0 deleted
        -Resources: 0 created, 0 changed, 0 deleted, 2 unchanged
        -Third snapshot path: /Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot/files/src/main.py
        -OK: new snapshot reused on redeploy
        -
        -=== Destroy and deploy again moves to yet another snapshot
         >>> [CLI] bundle destroy --auto-approve
        -The following resources will be deleted:
        -  delete resources.jobs.my_job
        -
         All files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/test-bundle-immutable-redeploy-[UNIQUE_NAME]/default
         
        -Destroy: 1 deleted
        +Destroy: 0 deleted
         
        ->>> [CLI] bundle deploy
        -Created internal_immutable_snapshots.immutable
        -Created jobs.my_job
        -Files: 0 uploaded, 0 deleted
        -Resources: 2 created, 0 changed, 0 deleted, 0 unchanged
        -Fourth snapshot path: /Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot/files/src/main.py
        -OK: deployment moved to a new snapshot
        -
        ->>> [CLI] bundle destroy --auto-approve
        -The following resources will be deleted:
        -  delete resources.jobs.my_job
        -
        -All files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/test-bundle-immutable-redeploy-[UNIQUE_NAME]/default
        -
        -Destroy: 1 deleted
        +Exit code: 1

latest (v1.21.0): same output as main (0a32c93)

TestAccept/bundle/deploy/immutable/dev-mode/DMS=: branch ✅ | main (0a32c93) ❌ | latest (v1.21.0) ❌

main (0a32c93):

acceptance_test.go:1307: Diff:
        --- bundle/deploy/immutable/dev-mode/output.txt
        +++ /tmp/TestAcceptbundledeployimmutabledev-modeDMS=795581928/002/output.txt
        @@ -18,21 +18,20 @@
         Validation OK!
         
         >>> [CLI] bundle deploy -t dev
        -Created internal_immutable_snapshots.immutable
        -Created jobs.my_job
        -Files: 0 uploaded, 0 deleted
        -Resources: 2 created, 0 changed, 0 deleted, 0 unchanged
        +Error: cannot create resources.internal_immutable_snapshots.immutable: snapshot upload: No stub found for pattern: POST /api/2.0/repos/snapshots (501 <nil>)
         
        ->>> [CLI] jobs get [MY_JOB_ID]
        -{
        -  "name": "[dev [USERNAME]] my job",
        -  "python_file": "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot/files/src/main.py"
        -}
        +Endpoint: POST [DATABRICKS_URL]/api/2.0/repos/snapshots
        +HTTP Status: 501 Not Implemented
        +API error_code: <nil>
        +API message: No stub found for pattern: POST /api/2.0/repos/snapshots
         
        ->>> [CLI] bundle destroy -t dev --auto-approve
        -The following resources will be deleted:
        -  delete resources.jobs.my_job
        +Error: cannot create resources.jobs.my_job: dependency failed: resources.internal_immutable_snapshots.immutable
         
        +Files: 0 uploaded, 0 deleted
        +
        +>>> [CLI] bundle destroy -t dev --auto-approve
         All files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/test-bundle-immutable-dev-mode-[UNIQUE_NAME]/dev
         
        -Destroy: 1 deleted
        +Destroy: 0 deleted
        +
        +Exit code: 1

latest (v1.21.0): same output as main (0a32c93)

TestAccept/bundle/deploy/immutable/legacy-state/DMS=: branch ✅ | main (0a32c93) ❌ | latest (v1.21.0) ❌

main (0a32c93):

acceptance_test.go:1307: Diff:
        --- bundle/deploy/immutable/legacy-state/output.txt
        +++ /tmp/TestAcceptbundledeployimmutablelegacy-stateDMS=4045615794/002/output.txt
        @@ -1,34 +1,20 @@
         
         === Deploy an immutable snapshot
         >>> [CLI] bundle deploy
        -Created internal_immutable_snapshots.immutable
        -Created jobs.my_job
        -Files: 0 uploaded, 0 deleted
        -Resources: 2 created, 0 changed, 0 deleted, 0 unchanged
        +Error: cannot create resources.internal_immutable_snapshots.immutable: snapshot upload: No stub found for pattern: POST /api/2.0/repos/snapshots (501 <nil>)
         
        -=== Rewrite the state the way a CLI without this feature wrote it
        -=== Deploying on top of that state succeeds and moves to the current layout
        ->>> [CLI] bundle deploy
        -Recreated internal_immutable_snapshots.immutable
        -Updated jobs.my_job
        -Files: 0 uploaded, 0 deleted
        -Resources: 1 created, 1 changed, 1 deleted, 0 unchanged
        -Snapshot path: /Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot/files/src/main.py
        -OK: deployed to the current snapshot layout
        +Endpoint: POST [DATABRICKS_URL]/api/2.0/repos/snapshots
        +HTTP Status: 501 Not Implemented
        +API error_code: <nil>
        +API message: No stub found for pattern: POST /api/2.0/repos/snapshots
         
        -=== The check still applies to a deployment carried over from the older layout
        ->>> [CLI] api post /api/2.0/snapshots:breakGlass --json {"snapshot_content_path": "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot"}
        -true
        +Error: cannot create resources.jobs.my_job: dependency failed: resources.internal_immutable_snapshots.immutable
         
        ->>> musterr [CLI] bundle deploy
        -Error: the last deployment was modified in the workspace. Changes might be overwritten by this deployment. To confirm it, use the --force flag
        -
         Files: 0 uploaded, 0 deleted
         
         >>> [CLI] bundle destroy --auto-approve
        -The following resources will be deleted:
        -  delete resources.jobs.my_job
        -
         All files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/test-bundle-immutable-legacy-[UNIQUE_NAME]/default
         
        -Destroy: 1 deleted
        +Destroy: 0 deleted
        +
        +Exit code: 1

latest (v1.21.0): same output as main (0a32c93)

TestAccept/bundle/deploy/immutable/missing-snapshot/DMS=: branch ✅ | main (0a32c93) ❌ | latest (v1.21.0) ❌

main (0a32c93):

acceptance_test.go:1307: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1307
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1077
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:655
        	Error:      	Not equal: 
        	            	expected: "\n=== Deploy an immutable snapshot\n>>> [CLI] bundle deploy\nCreated internal_immutable_snapshots.immutable\nCreated jobs.my_job\nFiles: 0 uploaded, 0 deleted\nResources: 2 created, 0 changed, 0 deleted, 0 unchanged\n\n=== Point the state at a snapshot that is not there\n=== Deploying still works instead of failing on the snapshot that is not there\n>>> [CLI] bundle deploy\nRecreated internal_immutable_snapshots.immutable\nUpdated jobs.my_job\nFiles: 0 uploaded, 0 deleted\nResources: 1 created, 1 changed, 1 deleted, 0 unchanged\n\n>>> [CLI] bundle destroy --auto-approve\nThe following resources will be deleted:\n  delete resources.jobs.my_job\n\nAll files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/test-bundle-immutable-missing-[UNIQUE_NAME]/default\n\nDestroy: 1 deleted\n"
        	            	actual  : "\n=== Deploy an immutable snapshot\n>>> [CLI] bundle deploy\nError: cannot create resources.internal_immutable_snapshots.immutable: snapshot upload: No stub found for pattern: POST /api/2.0/repos/snapshots (501 <nil>)\n\nEndpoint: POST [DATABRICKS_URL]/api/2.0/repos/snapshots\nHTTP Status: 501 Not Implemented\nAPI error_code: <nil>\nAPI message: No stub found for pattern: POST /api/2.0/repos/snapshots\n\nError: cannot create resources.jobs.my_job: dependency failed: resources.internal_immutable_snapshots.immutable\n\nFiles: 0 uploaded, 0 deleted\n\n>>> [CLI] bundle destroy --auto-approve\nAll files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/test-bundle-immutable-missing-[UNIQUE_NAME]/default\n\nDestroy: 0 deleted\n\nExit code: 1\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -3,22 +3,19 @@
        	            	 >>> [CLI] bundle deploy
        	            	-Created internal_immutable_snapshots.immutable
        	            	-Created jobs.my_job
        	            	+Error: cannot create resources.internal_immutable_snapshots.immutable: snapshot upload: No stub found for pattern: POST /api/2.0/repos/snapshots (501 <nil>)
        	            	+
        	            	+Endpoint: POST [DATABRICKS_URL]/api/2.0/repos/snapshots
        	            	+HTTP Status: 501 Not Implemented
        	            	+API error_code: <nil>
        	            	+API message: No stub found for pattern: POST /api/2.0/repos/snapshots
        	            	+
        	            	+Error: cannot create resources.jobs.my_job: dependency failed: resources.internal_immutable_snapshots.immutable
        	            	+
        	            	 Files: 0 uploaded, 0 deleted
        	            	-Resources: 2 created, 0 changed, 0 deleted, 0 unchanged
        	            	-
        	            	-=== Point the state at a snapshot that is not there
        	            	-=== Deploying still works instead of failing on the snapshot that is not there
        	            	->>> [CLI] bundle deploy
        	            	-Recreated internal_immutable_snapshots.immutable
        	            	-Updated jobs.my_job
        	            	-Files: 0 uploaded, 0 deleted
        	            	-Resources: 1 created, 1 changed, 1 deleted, 0 unchanged
        	            	 
        	            	 >>> [CLI] bundle destroy --auto-approve
        	            	-The following resources will be deleted:
        	            	-  delete resources.jobs.my_job
        	            	-
        	            	 All files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/test-bundle-immutable-missing-[UNIQUE_NAME]/default
        	            	 
        	            	-Destroy: 1 deleted
        	            	+Destroy: 0 deleted
        	            	 
        	            	+Exit code: 1
        	            	+
        	Test:       	TestAccept/bundle/deploy/immutable/missing-snapshot/DMS=
        	Messages:   	bundle/deploy/immutable/missing-snapshot/output.txt vs /tmp/TestAcceptbundledeployimmutablemissing-snapshotDMS=1426572305/002/output.txt

latest (v1.21.0): same output as main (0a32c93)

TestAccept/bundle/deploy/immutable/no-artifacts/DMS=/READPLAN=: branch ✅ | main (0a32c93) ❌ | latest (v1.21.0) ❌

main (0a32c93):

acceptance_test.go:1307: Diff:
        --- bundle/deploy/immutable/no-artifacts/output.txt
        +++ /tmp/TestAcceptbundledeployimmutableno-artifactsDMS=READPLAN=301955975/002/output.txt
        @@ -15,16 +15,16 @@
           {
             "notebook_task": {
               "base_parameters": {
        -        "path": "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot/files/some_path"
        +        "path": "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]/files/some_path"
               },
        -      "notebook_path": "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot/files/src/notebook"
        +      "notebook_path": "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]/files/src/notebook"
             },
             "task_key": "notebook_task"
           },
           {
             "environment_key": "env",
             "spark_python_task": {
        -      "python_file": "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot/files/src/main.py"
        +      "python_file": "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]/files/src/main.py"
             },
             "task_key": "spark_python_task"
           }
        @@ -35,8 +28,8 @@
           "action": "create",
           "new_state": {
             "value": {
        -      "relative_path": "[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]",
        -      "full_path": "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot",
        +      "relative_path": "[SNAPSHOT_HASH]/[SNAPSHOT_HASH]",
        +      "full_path": "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]",
               "bundle_id": "[SNAPSHOT_HASH]",
               "acl": [
                 {
        @@ -44,101 +37,21 @@
                   "permission_level": "CAN_READ"
                 }
               ],
        -      "can_manage": [
        -        {
        -          "user_name": "[USERNAME]"
        -        }
        -      ],
        -      "zip_path": "[TEST_TMP_DIR]/.databricks/bundle/default/snapshots/[SNAPSHOT_HASH]-[UUID].zip"
        +      "zip_path": "[TEST_TMP_DIR]/.databricks/bundle/default/snapshots/[SNAPSHOT_HASH].zip"
             }
           }
         }
         
         >>> [CLI] bundle deploy
        -Created internal_immutable_snapshots.immutable
        -Created jobs.my_job
        -Files: 0 uploaded, 0 deleted
        -Resources: 2 created, 0 changed, 0 deleted, 0 unchanged
        +Error: cannot create resources.internal_immutable_snapshots.immutable: snapshot upload: No stub found for pattern: POST /api/2.0/repos/snapshots (501 <nil>)
         
        ->>> jq .settings.tasks job.json
        -"/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot/files/src/main.py"
        +Endpoint: POST [DATABRICKS_URL]/api/2.0/repos/snapshots
        +HTTP Status: 501 Not Implemented
        +API error_code: <nil>
        +API message: No stub found for pattern: POST /api/2.0/repos/snapshots
         
        ->>> jq .settings.tasks job.json
        -"/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot/files/src/notebook"
        +Error: cannot create resources.jobs.my_job: dependency failed: resources.internal_immutable_snapshots.immutable
         
        ->>> jq .settings.tasks job.json
        -"/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot/files/some_path"
        -
        ->>> title Check snapshot plan after deploy: 
        -
        -=== Check snapshot plan after deploy: 
        ->>> [CLI] bundle plan -o json
        -
        ->>> jq .plan["resources.internal_immutable_snapshots.immutable"] plan.json
        -{
        -  "action": "skip",
        -  "remote_state": {
        -    "relative_path": "[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]",
        -    "full_path": "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot"
        -  },
        -  "changes": {
        -    "acl": {
        -      "action": "skip",
        -      "reason": "missing_in_remote",
        -      "old": [
        -        {
        -          "user_name": "[USERNAME]",
        -          "permission_level": "CAN_READ"
        -        }
        -      ],
        -      "new": [
        -        {
        -          "user_name": "[USERNAME]",
        -          "permission_level": "CAN_READ"
        -        }
        -      ],
        -      "remote": null
        -    },
        -    "bundle_id": {
        -      "action": "skip",
        -      "reason": "missing_in_remote",
        -      "old": "[SNAPSHOT_HASH]",
        -      "new": "[SNAPSHOT_HASH]",
        -      "remote": ""
        -    },
        -    "can_manage": {
        -      "action": "skip",
        -      "reason": "missing_in_remote",
        -      "old": [
        -        {
        -          "user_name": "[USERNAME]"
        -        }
        -      ],
        -      "new": [
        -        {
        -          "user_name": "[USERNAME]"
        -        }
        -      ]
        -    },
        -    "zip_path": {
        -      "action": "skip",
        -      "reason": "missing_in_remote",
        -      "old": "[TEST_TMP_DIR]/.databricks/bundle/default/snapshots/[SNAPSHOT_HASH]-[UUID].zip",
        -      "new": "[TEST_TMP_DIR]/.databricks/bundle/default/snapshots/[SNAPSHOT_HASH]-[UUID].zip",
        -      "remote": ""
        -    }
        -  }
        -}
        -
        ->>> [CLI] bundle deploy
         Files: 0 uploaded, 0 deleted
        -Resources: 0 created, 0 changed, 0 deleted, 2 unchanged
        -Snapshot did not change as expected
         
        ->>> [CLI] bundle destroy --auto-approve
        -The following resources will be deleted:
        -  delete resources.jobs.my_job
        -
        -All files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/test-bundle-immutable-no-artifacts-[UNIQUE_NAME]/default
        -
        -Destroy: 1 deleted
        +Exit code: 1
        
    acceptance_test.go:1126: Test produced unexpected files:
        plan.json

latest (v1.21.0): same output as main (0a32c93)

TestAccept/bundle/deploy/immutable/no-artifacts/DMS=/READPLAN=1: branch ✅ | main (0a32c93) ❌ | latest (v1.21.0) ❌

main (0a32c93):

acceptance_test.go:1307: Diff:
        --- bundle/deploy/immutable/no-artifacts/output.txt
        +++ /tmp/TestAcceptbundledeployimmutableno-artifactsDMS=READPLAN=1521910736/002/output.txt
        @@ -15,16 +15,16 @@
           {
             "notebook_task": {
               "base_parameters": {
        -        "path": "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot/files/some_path"
        +        "path": "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]/files/some_path"
               },
        -      "notebook_path": "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot/files/src/notebook"
        +      "notebook_path": "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]/files/src/notebook"
             },
             "task_key": "notebook_task"
           },
           {
             "environment_key": "env",
             "spark_python_task": {
        -      "python_file": "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot/files/src/main.py"
        +      "python_file": "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]/files/src/main.py"
             },
             "task_key": "spark_python_task"
           }
        @@ -35,8 +28,8 @@
           "action": "create",
           "new_state": {
             "value": {
        -      "relative_path": "[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]",
        -      "full_path": "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot",
        +      "relative_path": "[SNAPSHOT_HASH]/[SNAPSHOT_HASH]",
        +      "full_path": "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]",
               "bundle_id": "[SNAPSHOT_HASH]",
               "acl": [
                 {
        @@ -44,101 +37,21 @@
                   "permission_level": "CAN_READ"
                 }
               ],
        -      "can_manage": [
        -        {
        -          "user_name": "[USERNAME]"
        -        }
        -      ],
        -      "zip_path": "[TEST_TMP_DIR]/.databricks/bundle/default/snapshots/[SNAPSHOT_HASH]-[UUID].zip"
        +      "zip_path": "[TEST_TMP_DIR]/.databricks/bundle/default/snapshots/[SNAPSHOT_HASH].zip"
             }
           }
         }
         
         >>> [CLI] bundle deploy
        -Created internal_immutable_snapshots.immutable
        -Created jobs.my_job
        -Files: 0 uploaded, 0 deleted
        -Resources: 2 created, 0 changed, 0 deleted, 0 unchanged
        +Error: cannot create resources.internal_immutable_snapshots.immutable: snapshot upload: No stub found for pattern: POST /api/2.0/repos/snapshots (501 <nil>)
         
        ->>> jq .settings.tasks job.json
        -"/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot/files/src/main.py"
        +Endpoint: POST [DATABRICKS_URL]/api/2.0/repos/snapshots
        +HTTP Status: 501 Not Implemented
        +API error_code: <nil>
        +API message: No stub found for pattern: POST /api/2.0/repos/snapshots
         
        ->>> jq .settings.tasks job.json
        -"/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot/files/src/notebook"
        +Error: cannot create resources.jobs.my_job: dependency failed: resources.internal_immutable_snapshots.immutable
         
        ->>> jq .settings.tasks job.json
        -"/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot/files/some_path"
        -
        ->>> title Check snapshot plan after deploy: 
        -
        -=== Check snapshot plan after deploy: 
        ->>> [CLI] bundle plan -o json
        -
        ->>> jq .plan["resources.internal_immutable_snapshots.immutable"] plan.json
        -{
        -  "action": "skip",
        -  "remote_state": {
        -    "relative_path": "[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]",
        -    "full_path": "/Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot"
        -  },
        -  "changes": {
        -    "acl": {
        -      "action": "skip",
        -      "reason": "missing_in_remote",
        -      "old": [
        -        {
        -          "user_name": "[USERNAME]",
        -          "permission_level": "CAN_READ"
        -        }
        -      ],
        -      "new": [
        -        {
        -          "user_name": "[USERNAME]",
        -          "permission_level": "CAN_READ"
        -        }
        -      ],
        -      "remote": null
        -    },
        -    "bundle_id": {
        -      "action": "skip",
        -      "reason": "missing_in_remote",
        -      "old": "[SNAPSHOT_HASH]",
        -      "new": "[SNAPSHOT_HASH]",
        -      "remote": ""
        -    },
        -    "can_manage": {
        -      "action": "skip",
        -      "reason": "missing_in_remote",
        -      "old": [
        -        {
        -          "user_name": "[USERNAME]"
        -        }
        -      ],
        -      "new": [
        -        {
        -          "user_name": "[USERNAME]"
        -        }
        -      ]
        -    },
        -    "zip_path": {
        -      "action": "skip",
        -      "reason": "missing_in_remote",
        -      "old": "[TEST_TMP_DIR]/.databricks/bundle/default/snapshots/[SNAPSHOT_HASH]-[UUID].zip",
        -      "new": "[TEST_TMP_DIR]/.databricks/bundle/default/snapshots/[SNAPSHOT_HASH]-[UUID].zip",
        -      "remote": ""
        -    }
        -  }
        -}
        -
        ->>> [CLI] bundle deploy
         Files: 0 uploaded, 0 deleted
        -Resources: 0 created, 0 changed, 0 deleted, 2 unchanged
        -Snapshot did not change as expected
         
        ->>> [CLI] bundle destroy --auto-approve
        -The following resources will be deleted:
        -  delete resources.jobs.my_job
        -
        -All files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/test-bundle-immutable-no-artifacts-[UNIQUE_NAME]/default
        -
        -Destroy: 1 deleted
        +Exit code: 1
        
    acceptance_test.go:1126: Test produced unexpected files:
        plan.json

latest (v1.21.0): same output as main (0a32c93)

TestAccept/bundle/deploy/immutable/permissions-change/DMS=: branch ✅ | main (0a32c93) ❌ | latest (v1.21.0) ❌

main (0a32c93):

acceptance_test.go:1307: Diff:
        --- bundle/deploy/immutable/permissions-change/output.txt
        +++ /tmp/TestAcceptbundledeployimmutablepermissions-changeDMS=1292320514/002/output.txt
        @@ -1,52 +1,20 @@
         
         === Deploy without permissions
         >>> [CLI] bundle deploy
        -Created internal_immutable_snapshots.immutable
        -Created jobs.my_job
        -Files: 0 uploaded, 0 deleted
        -Resources: 2 created, 0 changed, 0 deleted, 0 unchanged
        -First snapshot path: /Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot/files/src/main.py
        +Error: cannot create resources.internal_immutable_snapshots.immutable: snapshot upload: No stub found for pattern: POST /api/2.0/repos/snapshots (501 <nil>)
         
        -=== Add permissions and redeploy
        ->>> [CLI] bundle deploy
        -Recommendation: permissions section should explicitly include the current deployment identity '[USERNAME]' or one of its groups
        -If it is not included, CAN_MANAGE permissions are only applied if the present identity is used to deploy.
        +Endpoint: POST [DATABRICKS_URL]/api/2.0/repos/snapshots
        +HTTP Status: 501 Not Implemented
        +API error_code: <nil>
        +API message: No stub found for pattern: POST /api/2.0/repos/snapshots
         
        -Consider using a adding a top-level permissions section such as the following:
        +Error: cannot create resources.jobs.my_job: dependency failed: resources.internal_immutable_snapshots.immutable
         
        -  permissions:
        -    - user_name: [USERNAME]
        -      level: CAN_MANAGE
        -
        -See https://docs.databricks.com/dev-tools/bundles/permissions.html to learn more about permission configuration.
        -  in databricks.yml:22:3
        -
        -Created jobs.my_job.permissions
        -Recreated internal_immutable_snapshots.immutable
        -Updated jobs.my_job
         Files: 0 uploaded, 0 deleted
        -Resources: 2 created, 1 changed, 1 deleted, 0 unchanged
        -Second snapshot path: /Workspace/Users/[UUID]/.snapshots/[SNAPSHOT_HASH]/[SNAPSHOT_HASH]-[UUID]/snapshot/files/src/main.py
         
        -=== Verify snapshot path changed after permissions change
        -OK: snapshot path changed as expected
        -
         >>> [CLI] bundle destroy --auto-approve
        -Recommendation: permissions section should explicitly include the current deployment identity '[USERNAME]' or one of its groups
        -If it is not included, CAN_MANAGE permissions are only applied if the present identity is used to deploy.
        -
        -Consider using a adding a top-level permissions section such as the following:
        -
        -  permissions:
        -    - user_name: [USERNAME]
        -      level: CAN_MANAGE
        -
        -See https://docs.databricks.com/dev-tools/bundles/permissions.html to learn more about permission configuration.
        -  in databricks.yml:22:3
        -
        -The following resources will be deleted:
        -  delete resources.jobs.my_job
        -
         All files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/test-bundle-immutable-perms-[UNIQUE_NAME]/default
         
        -Destroy: 1 deleted
        +Destroy: 0 deleted
        +
        +Exit code: 1

latest (v1.21.0): same output as main (0a32c93)

6 more failure details omitted, see the full report in the run.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@denik denik Awaiting requested review from denik
@janniklasrose janniklasrose Awaiting requested review from janniklasrose
@pietern pietern Awaiting requested review from pietern

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

DABs DABs related issues

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@andrewnester @eng-dev-ecosystem-bot @pietern