Syncing Redaction Artifacts to S3 Object Lock

This walkthrough provisions a lock-enabled versioned S3 bucket and uploads a single redacted PDF alongside its audit manifest under a compliance-mode retain-until date, then proves the lock stuck by reading it back with head_object. It is the hands-on companion to WORM Storage Synchronization and focuses on the per-object upload path rather than bucket-wide policy.

Prerequisites & Version Matrix #Permalink to this section

The calling IAM principal needs s3:CreateBucket, s3:PutBucketVersioning, s3:PutObject, s3:PutObjectRetention, and s3:GetObject (which covers head_object). Grant s3:PutObjectRetention explicitly — without it, PUTs that carry a retention header are denied even though ordinary uploads succeed.

Step 1: Create a Lock-Enabled, Versioned Bucket #Permalink to this section

Object Lock can only be turned on at creation, and it forces versioning on. Enable both up front; there is no path to add either to an existing bucket.

import boto3

s3 = boto3.client("s3", region_name="us-east-1")
BUCKET = "matter-4471-redaction-worm"

# ObjectLockEnabledForBucket implicitly and permanently enables versioning.
s3.create_bucket(Bucket=BUCKET, ObjectLockEnabledForBucket=True)

# Belt-and-suspenders: confirm versioning is Enabled before writing anything.
status = s3.get_bucket_versioning(Bucket=BUCKET).get("Status")
assert status == "Enabled", f"versioning must be Enabled, got {status!r}"

Step 2: Upload the Redacted PDF and Its Manifest #Permalink to this section

Upload both artifacts with the same compliance-mode retain-until date so the document and the record proving how it was redacted expire together. Compute the date in UTC to keep it unambiguously in the future.

import hashlib
from datetime import datetime, timedelta, timezone

def put_locked(key: str, path: str, retain_until: datetime) -> str:
    with open(path, "rb") as fh:
        body = fh.read()
    digest = hashlib.sha256(body).hexdigest()  # store hash, never log raw bytes
    s3.put_object(
        Bucket=BUCKET,
        Key=key,
        Body=body,
        ObjectLockMode="COMPLIANCE",
        ObjectLockRetainUntilDate=retain_until,
        Metadata={"sha256": digest},
    )
    return digest

# Six-year retention window, anchored to UTC to stay strictly in the future.
retain_until = datetime.now(timezone.utc) + timedelta(days=365 * 6)
pdf_hash = put_locked("2026/07/deposition-0012.redacted.pdf",
                      "deposition-0012.redacted.pdf", retain_until)
manifest_hash = put_locked("2026/07/deposition-0012.manifest.json",
                           "deposition-0012.manifest.json", retain_until)

Step 3: Verify the Lock via head_object #Permalink to this section

Reading the metadata back confirms S3 accepted and persisted the retention, rather than trusting the PUT returned without error. A silent failure here — an object stored with no lock because a header was dropped by a proxy or an SDK misconfiguration — is exactly the gap opposing counsel probes during a spoliation challenge, so treat the read-back as mandatory, not optional.

head = s3.head_object(Bucket=BUCKET, Key="2026/07/deposition-0012.redacted.pdf")
print(head["ObjectLockMode"], head["ObjectLockRetainUntilDate"])
# -> COMPLIANCE 2032-07-15 12:00:00+00:00

Compliance Checkpoint #Permalink to this section

Storing the redacted output in a compliance-mode object lock satisfies SEC Rule 17a-4(f)(2)(ii)(A), which mandates preserving records “in a non-rewriteable, non-erasable format.” Because compliance mode denies deletion and shortening to every principal including the account root, the stored deposition and its manifest meet that non-erasable bar for the full retention period, and the co-located SHA-256 digests give an examiner an independent integrity check against the Cryptographic Audit Manifests ledger.

Gotchas #Permalink to this section

• Object Lock demands versioning at creation. If the bucket was made without ObjectLockEnabledForBucket=True, retention PUTs fail with InvalidRequest. You cannot bolt it on later — create a fresh bucket and re-sync.
• Compliance mode cannot be shortened, even by root. A retain-until date set too far out cannot be walked back; pick the window deliberately from the statutory schedule before uploading, not after.
• A missing s3:PutObjectRetention permission looks like a code bug. Plain uploads succeed while locked ones return AccessDenied. Check the IAM policy before debugging the payload.
• Naive datetimes drift. A retain-until built from local time can land in the past under clock skew and be rejected; always use datetime.now(timezone.utc).

Verification & Testing #Permalink to this section

This assertion fails loudly if the retention did not attach, giving a deterministic gate for a smoke test:

from datetime import datetime, timezone

head = s3.head_object(Bucket=BUCKET, Key="2026/07/deposition-0012.redacted.pdf")
assert head["ObjectLockMode"] == "COMPLIANCE"
assert head["ObjectLockRetainUntilDate"] > datetime.now(timezone.utc)
print("PASS: object is locked until", head["ObjectLockRetainUntilDate"].date())
# -> PASS: object is locked until 2032-07-15

Related #Permalink to this section

Set account-wide defaults and reconcile windows in Enforcing Retention Policies on WORM Buckets, review the architecture and failure modes in WORM Storage Synchronization, and feed these uploads from the workers in Async Batch Processing Pipelines. Part of the Immutable Audit Trails & Chain of Custody field guide.