Add nineteen sdkharness conformance checks: large files, copy, URLs, locks, encryption, keys, bucket configuration

.sdkharness/README.md

@@ -5,7 +5,7 @@
 assertions in `tests/`; the harness builds this checkout, starts a fresh local B2 simulator,
 and retains and reports the evidence.
 
-Scenarios so far: `health/golden-path` and eight `conformance` and sixteen `resilience` scenarios (below). `tests/run-health` runs
+Scenarios so far: `health/golden-path` and 27 `conformance` (one per capability the SDK claims) and sixteen `resilience` scenarios (below). `tests/run-health` runs
 `tests/health/GoldenPath.java` as a single-file Java program that authorizes, uploads a small
 file, downloads it and compares the bytes, lists it, deletes it and confirms it is gone, then
 prints one result line:
@@ -35,6 +35,36 @@ deletion, and a deletion that fails turns the verdict into a `FAIL` (`cleanup --
 | `files.hide` | hide marker is a new version; not in the default listing, visible in the version listing; 404 by name; the original still downloads by id |
 | `bucket.crud` | create, duplicate name refused, list, update type and info (revision grows), delete refused while non-empty, delete |
 | `files.metadata` | by id (`b2_get_file_info`) and by name (an HTTP HEAD of the download URL: the cell is `partial`); the check records the SDK's own requests and asserts that boundary |
+| `files.server_side_copy` | `b2_copy_file` sends no bytes (no `b2_upload_file`), same sha1 under a new fileId, the COPY and REPLACE metadata directives, a ranged copy |
+| `large.multipart` | `uploadLargeFile` of three parts' worth: one start, more than one `b2_upload_part`, one finish, contentSha1 `none`, `large_file_sha1` carried, bytes intact |
+| `large.concurrent_parts` | the same eight-part file on a one-thread and a four-thread executor: the SDK's own HTTP client counts `b2_upload_part` requests in flight (1 with one worker, more than 1 with four; wall time is only a `NOTE`) |
+| `large.resume` | a first upload interrupted after two parts, then `finishUploadingLargeFile` on the unfinished file: no new `b2_start_large_file`, only the missing parts, same fileId, bytes intact; run with and without a known sha1 (see the note below) |
+| `large.compose_mixed` | one large file from local bytes, a copied byte range and a whole copied file (`storeLargeFile` with `B2UploadingPartStorer` and `B2CopyingPartStorer`), equal to their ordered concatenation |
+| `urls.native_download` | `getDownloadByNameUrl` is a native `/file/<bucket>/<name>` URL (no `X-Amz-*`); 401 without a token, 200 with a download authorization, 401 outside its prefix and after it expires |
+| `client.progress` | a `B2UploadListener` on a large upload (one worker): it fires, every part's bytes and the running total never go down, every part ends SUCCEEDED, the total equals the byte count |
+| `lock.bucket_default` | a default retention of governance, 1 day on a lock-enabled bucket: read back, and an uploaded file inherits a retention of about now + 1 day |
+| `lock.per_file_retention` | governance retention now + 2 days on a file: read back, and a delete without bypass is refused (401 `access_denied`) and leaves the file |
+| `lock.legal_hold` | hold on: read back and the delete is refused; hold off: read back and the delete succeeds |
+| `lock.bypass_governance` | under governance retention a delete is refused (401 `access_denied`), `setBypassGovernance(true)` deletes the version |
+| `enc.sse_b2` | an SSE-B2 upload reports SSE-B2 / AES256 and downloads as plaintext; a bucket default of SSE-B2 applies to a plain upload |
+| `enc.sse_c` | an SSE-C upload reports SSE-C / AES256; download with the key returns the plaintext and the key's MD5; without it and with another key it is refused (403 for a wrong key) |
+| `keys.crud` | a bucket-restricted key is created (secret returned), listed (no secret anywhere in what the SDK returns), used by a second client, deleted (gone, and its secret no longer authorizes) |
+| `bucket.cors` | typed `B2CorsRule`: set at create, replaced by update, cleared; read back field for field |
+| `bucket.lifecycle` | `daysFromHidingToDeleting` 1 round trips with no other field populated; a replacement list replaces; an empty list clears |
+| `bucket.notification_rules` | a webhook rule with a signing secret is set, read back (the secret unchanged) and deleted |
+| `bucket.inbound_webhook` | offline: a payload signed in the check with the documented HMAC is accepted and parsed by `B2EventNotification.parse`; a wrong secret, a changed payload and a malformed signature are refused; a test event parses |
+| `bucket.replication_config` | a source rule (destination bucket, prefix, priority, flags) with a replication key is set and read back field for field |
+
+`large.resume` reports one SDK behavior as a `FAIL` whose reason starts with `FINDING`. When the content source knows its sha1, the
+SDK stores it as the file info `large_file_sha1`; `finishUploadingLargeFile` then compares the request's file info
+with the stored one after removing the key `X-Bz-Info-large_file_sha1` (`B2Headers.LARGE_FILE_SHA1`,
+`B2LargeFileUploader.throwIfLargeFileVersionDoesntSeemToMatchRequest`), but the stored key is
+`large_file_sha1` (`B2Headers.LARGE_FILE_SHA1_INFO_NAME`), so the same request that started the file is refused with a
+`mismatch` error. The check does not work around that refusal: the scenario fails with the finding as its reason (the
+harness detects, it does not fix the SDK) and passes once the SDK accepts the same request.
+
+Every check that locks files releases them on exit (legal hold off, governance retention bypassed); none uses
+compliance mode, which nothing could lift.
 
 A failing scenario prints one `SDKHARNESS_RESULT ... FAIL` line with the step and the reason. Details are
 formatted to one capped printable line from exception class names, status codes and the check's own

.sdkharness/tests.tsv

@@ -8,6 +8,25 @@ conformance	files.delete_version	simulator	./.sdkharness/tests/run-conformance
 conformance	files.hide	simulator	./.sdkharness/tests/run-conformance
 conformance	bucket.crud	simulator	./.sdkharness/tests/run-conformance
 conformance	files.metadata	simulator	./.sdkharness/tests/run-conformance
+conformance	files.server_side_copy	simulator	./.sdkharness/tests/run-conformance
+conformance	large.multipart	simulator	./.sdkharness/tests/run-conformance
+conformance	large.concurrent_parts	simulator	./.sdkharness/tests/run-conformance
+conformance	large.resume	simulator	./.sdkharness/tests/run-conformance
+conformance	large.compose_mixed	simulator	./.sdkharness/tests/run-conformance
+conformance	urls.native_download	simulator	./.sdkharness/tests/run-conformance
+conformance	client.progress	simulator	./.sdkharness/tests/run-conformance
+conformance	lock.bucket_default	simulator	./.sdkharness/tests/run-conformance
+conformance	lock.per_file_retention	simulator	./.sdkharness/tests/run-conformance
+conformance	lock.legal_hold	simulator	./.sdkharness/tests/run-conformance
+conformance	lock.bypass_governance	simulator	./.sdkharness/tests/run-conformance
+conformance	enc.sse_b2	simulator	./.sdkharness/tests/run-conformance
+conformance	enc.sse_c	simulator	./.sdkharness/tests/run-conformance
+conformance	keys.crud	simulator	./.sdkharness/tests/run-conformance
+conformance	bucket.cors	simulator	./.sdkharness/tests/run-conformance
+conformance	bucket.lifecycle	simulator	./.sdkharness/tests/run-conformance
+conformance	bucket.notification_rules	simulator	./.sdkharness/tests/run-conformance
+conformance	bucket.inbound_webhook	simulator	./.sdkharness/tests/run-conformance
+conformance	bucket.replication_config	simulator	./.sdkharness/tests/run-conformance
 resilience	upload.retry_503	simulator	./.sdkharness/tests/run-resilience
 resilience	upload.retry_500	simulator	./.sdkharness/tests/run-resilience
 resilience	upload.retry_408	simulator	./.sdkharness/tests/run-resilience

.sdkharness/tests/conformance/BucketCors.java

@@ -0,0 +1,82 @@
+// Copyright 2026, Backblaze Inc. All Rights Reserved.
+// License https://www.backblaze.com/using_b2_code.html
+
+import com.backblaze.b2.client.structures.B2Bucket;
+import com.backblaze.b2.client.structures.B2CorsRule;
+import com.backblaze.b2.client.structures.B2UpdateBucketRequest;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+
+/**
+ * bucket.cors. Run by ../run-conformance (see ../../README.md).
+ *
+ * <p>Every field sent is a documented rule field (corsRuleName, allowedOrigins, allowedOperations, allowedHeaders,
+ * exposeHeaders, maxAgeSeconds), and b2_download_file_by_name and b2_download_file_by_id are documented
+ * allowedOperations values. Rules can be "set when you create the bucket with b2_create_bucket, or updated on an
+ * existing bucket using b2_update_bucket", and b2_list_buckets returns corsRules
+ * (https://www.backblaze.com/docs/cloud-storage-cross-origin-resource-sharing-rules,
+ * https://www.backblaze.com/apidocs/b2-update-bucket, https://www.backblaze.com/apidocs/b2-list-buckets).
+ *
+ * <p>This SDK models a rule as the typed {@code B2CorsRule} (the card's cell is "yes", not "partial"). Asserted: a
+ * bucket created with one rule reports it, field for field, when listed; updating the bucket to a different pair of
+ * rules replaces the set (the read-back is exactly the new pair); updating to an empty list removes every rule.
+ */
+public class BucketCors {
+    public static void main(String[] args) {
+        Support.run("bucket.cors", () -> {
+            final B2CorsRule first = B2CorsRule.builder("sdkharness-first", Arrays.asList("https://example.com", "https://app.example.com"),
+                            new HashSet<>(Arrays.asList("b2_download_file_by_name", "b2_download_file_by_id")), 3600)
+                    .setAllowedHeaders(Arrays.asList("range", "authorization"))
+                    .setExposeHeaders(Arrays.asList("x-bz-content-sha1"))
+                    .build();
+
+            Support.step("create with a rule");
+            final B2Bucket bucket = Support.bucket(false, b -> b.setCorsRules(Collections.singletonList(first)));
+            Support.step("read back the created rule");
+            assertRules("rules on create", Collections.singletonList(first), bucket.getCorsRules());
+            assertRules("rules when listed", Collections.singletonList(first), listed(bucket).getCorsRules());
+
+            Support.step("replace the rules");
+            final B2CorsRule second = B2CorsRule.builder("sdkharness-second", Collections.singletonList("*"),
+                    new HashSet<>(Collections.singletonList("b2_download_file_by_name")), 60).build();
+            final B2CorsRule third = B2CorsRule.builder("sdkharness-third", Collections.singletonList("https://other.example.com"),
+                    new HashSet<>(Collections.singletonList("b2_download_file_by_id")), 120)
+                    .setAllowedHeaders(Collections.singletonList("*")).build();
+            final B2Bucket replaced = Support.client.updateBucket(B2UpdateBucketRequest.builder(bucket).setCorsRules(Arrays.asList(second, third)).build());
+            assertRules("rules after the update", Arrays.asList(second, third), listed(bucket).getCorsRules());
+
+            Support.step("remove the rules");
+            // updateBucket is optimistic: each update names the revision it was made from, so it starts from the last response.
+            Support.client.updateBucket(B2UpdateBucketRequest.builder(replaced).setCorsRules(Collections.emptyList()).build());
+            Support.eq("rules after clearing", 0, listed(bucket).getCorsRules().size());
+        });
+    }
+
+    private static B2Bucket listed(B2Bucket bucket) throws Exception {
+        final B2Bucket found = Support.client.getBucketOrNullByName(bucket.getBucketName());
+        Support.check(found != null, "the bucket is not listed");
+        return found;
+    }
+
+    /** Field for field, in order. */
+    private static void assertRules(String what, List<B2CorsRule> expected, List<B2CorsRule> actual) {
+        Support.eq(what + ": number of rules", expected.size(), actual.size());
+        for (int i = 0; i < expected.size(); i++) {
+            final B2CorsRule e = expected.get(i);
+            final B2CorsRule a = actual.get(i);
+            Support.eq(what + ": corsRuleName", e.getCorsRuleName(), a.getCorsRuleName());
+            Support.eq(what + ": allowedOrigins of " + e.getCorsRuleName(), e.getAllowedOrigins(), a.getAllowedOrigins());
+            Support.eq(what + ": allowedOperations of " + e.getCorsRuleName(), new HashSet<>(e.getAllowedOperations()), new HashSet<>(a.getAllowedOperations()));
+            Support.eq(what + ": allowedHeaders of " + e.getCorsRuleName(), nullToEmpty(e.getAllowedHeaders()), nullToEmpty(a.getAllowedHeaders()));
+            Support.eq(what + ": exposeHeaders of " + e.getCorsRuleName(), nullToEmpty(e.getExposeHeaders()), nullToEmpty(a.getExposeHeaders()));
+            Support.eq(what + ": maxAgeSeconds of " + e.getCorsRuleName(), e.getMaxAgeSeconds(), a.getMaxAgeSeconds());
+        }
+    }
+
+    private static List<String> nullToEmpty(List<String> list) {
+        return list == null ? Collections.emptyList() : list;
+    }
+}

.sdkharness/tests/conformance/BucketInboundWebhook.java

@@ -0,0 +1,105 @@
+// Copyright 2026, Backblaze Inc. All Rights Reserved.
+// License https://www.backblaze.com/using_b2_code.html
+
+import com.backblaze.b2.client.exceptions.B2SignatureVerificationException;
+import com.backblaze.b2.client.structures.B2EventNotification;
+import com.backblaze.b2.client.structures.B2EventNotificationEvent;
+
+import javax.crypto.Mac;
+import javax.crypto.spec.SecretKeySpec;
+import java.nio.charset.StandardCharsets;
+
+/**
+ * bucket.inbound_webhook. Run by ../run-conformance (see ../../README.md).
+ *
+ * <p>The X-Bz-Event-Notification-Signature value "comprises the signature version, always v1, followed by an equals
+ * sign =, followed by the lowercase hex-encoded signature value", and to verify it "create an HMAC SHA-256 digest
+ * using the hmacSha256SigningSecret ... as the key, with the request payload as the data. The resulting hex-encoded
+ * HMAC digest value must match the signature in the header." The payload's event fields are the documented ones
+ * (https://www.backblaze.com/docs/cloud-storage-event-notifications-reference-guide).
+ *
+ * <p>The SDK's part of this capability is parsing and verifying what a service receives, so nothing here needs a
+ * server round trip: a payload shaped like the documented example is signed in this check with its own HMAC code (not
+ * the SDK's), then given to {@code B2EventNotification.parse}. Asserted: with the right secret the payload is accepted
+ * and every documented event field is read back; with another secret, with the payload changed after signing, and
+ * with a signature that is not v1 hex, it is refused with {@code B2SignatureVerificationException}; a test event (no
+ * object fields) parses.
+ */
+public class BucketInboundWebhook {
+    private static final String PAYLOAD = "{\"events\":[{"
+            + "\"accountId\":\"sdkharness-account\",\"bucketId\":\"sdkharness-bucket-id\",\"bucketName\":\"sdkharness-conf-bucket\","
+            + "\"eventId\":\"sdkharness-event-1\",\"eventTimestamp\":1761000000000,\"eventType\":\"b2:ObjectCreated:Upload\","
+            + "\"eventVersion\":1,\"matchedRuleName\":\"sdkharness-rule\",\"objectName\":\"st/new.txt\",\"objectSize\":1234,"
+            + "\"objectVersionId\":\"sdkharness-version-1\"}]}";
+    private static final String TEST_PAYLOAD = "{\"events\":[{"
+            + "\"accountId\":\"sdkharness-account\",\"bucketId\":\"sdkharness-bucket-id\",\"bucketName\":\"sdkharness-conf-bucket\","
+            + "\"eventId\":\"sdkharness-event-2\",\"eventTimestamp\":1761000000001,\"eventType\":\"b2:TestEvent\","
+            + "\"eventVersion\":1,\"matchedRuleName\":\"sdkharness-rule\"}]}";
+    private static final String SECRET = "*******";
+
+    public static void main(String[] args) {
+        Support.run("bucket.inbound_webhook", () -> {
+            Support.step("accept a correctly signed payload");
+            final byte[] body = PAYLOAD.getBytes(StandardCharsets.UTF_8);
+            final B2EventNotification accepted = B2EventNotification.parse(body, sign(body, SECRET), SECRET);
+            Support.eq("number of events", 1, accepted.getEvents().size());
+            final B2EventNotificationEvent event = accepted.getEvents().get(0);
+            Support.eq("accountId", "sdkharness-account", event.getAccountId());
+            Support.eq("bucketId", "sdkharness-bucket-id", event.getBucketId());
+            Support.eq("bucketName", "sdkharness-conf-bucket", event.getBucketName());
+            Support.eq("eventId", "sdkharness-event-1", event.getEventId());
+            Support.eq("eventTimestamp", 1761000000000L, event.getEventTimestamp());
+            Support.eq("eventType", "b2:ObjectCreated:Upload", event.getEventType());
+            Support.eq("eventVersion", 1, event.getEventVersion());
+            Support.eq("matchedRuleName", "sdkharness-rule", event.getMatchedRuleName());
+            Support.eq("objectName", "st/new.txt", event.getObjectName());
+            Support.eq("objectSize", 1234L, event.getObjectSize());
+            Support.eq("objectVersionId", "sdkharness-version-1", event.getObjectVersionId());
+
+            Support.step("the string overload agrees");
+            Support.eq("event id through the string overload", "sdkharness-event-1",
+                    B2EventNotification.parse(PAYLOAD, sign(body, SECRET), SECRET).getEvents().get(0).getEventId());
+
+            Support.step("refuse a wrong secret");
+            refused("a payload verified with another secret", () -> B2EventNotification.parse(body, sign(body, SECRET), "another-secret"));
+
+            Support.step("refuse a changed payload");
+            final byte[] changed = PAYLOAD.replace("st/new.txt", "st/evil.txt").getBytes(StandardCharsets.UTF_8);
+            refused("a payload changed after signing", () -> B2EventNotification.parse(changed, sign(body, SECRET), SECRET));
+
+            Support.step("refuse a signature that is not v1 hex");
+            refused("a signature without the v1= prefix", () -> B2EventNotification.parse(body, sign(body, SECRET).substring(3), SECRET));
+            refused("an empty signature", () -> B2EventNotification.parse(body, "", SECRET));
+
+            Support.step("a test event");
+            final byte[] test = TEST_PAYLOAD.getBytes(StandardCharsets.UTF_8);
+            final B2EventNotificationEvent testEvent = B2EventNotification.parse(test, sign(test, SECRET), SECRET).getEvents().get(0);
+            Support.eq("test event type", "b2:TestEvent", testEvent.getEventType());
+            Support.eq("test event object name", null, testEvent.getObjectName());
+        });
+    }
+
+    private interface Parse {
+        void run() throws Exception;
+    }
+
+    private static void refused(String what, Parse parse) throws Exception {
+        try {
+            parse.run();
+        } catch (B2SignatureVerificationException expected) {
+            return;
+        }
+        Support.fail(what + " was accepted");
+    }
+
+    /** The documented signature: "v1=" and the lowercase hex of HMAC-SHA256(secret, payload). Not the SDK's code. */
+    private static String sign(byte[] payload, String secret) throws Exception {
+        final Mac mac = Mac.getInstance("HmacSHA256");
+        mac.init(new SecretKeySpec(secret.getBytes(StandardCharsets.UTF_8), "HmacSHA256"));
+        final StringBuilder hex = new StringBuilder("v1=");
+        for (byte b : mac.doFinal(payload)) {
+            hex.append(String.format("%02x", b));
+        }
+        return hex.toString();
+    }
+}