feat: per-part sha256 checksums on multipart uploads#122
Merged
mobileoverlord merged 1 commit intomainfrom Apr 2, 2026
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adds per-part SHA-256 checksums to multipart artifact uploads so each uploaded part can be integrity-checked by S3 (via x-amz-checksum-sha256) and later finalized with per-part checksums included in the /complete payload.
Changes:
- Compute a SHA-256 checksum for each ~50 MiB part and base64-encode it.
- Send the checksum as
x-amz-checksum-sha256on each presigned PUT request (host-side and container-side upload paths). - Include
checksum_sha256on eachCompletedPartin the complete request payload.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
src/commands/connect/upload.rs |
Computes per-part SHA-256 checksums in both upload paths and attaches them to completed part metadata. |
src/commands/connect/client.rs |
Extends multipart completion payload (CompletedPart) and adds x-amz-checksum-sha256 header to part PUT requests. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
lee-reinhardt
force-pushed
the
lee-eng-1663
branch
from
f0bb919 to
01b53e8
Compare
compute per-part sha256 checksums during artifact discovery and send them with create_runtime so the api can store them on artifact_blobs and sign them into presigned upload urls. s3 validates each part inline at upload time, catching corruption immediately. - host path: single-pass compute_sha256_with_parts reads 50 MiB chunks, producing both whole-file hex hash and per-part base64 checksums - container path: discovery script uses dd + openssl to compute per-part checksums inside the build container - send part_size in artifact params so api stores the cli's chunk size on the blob - upload phase: verifies computed checksum matches pre-computed value from discovery before uploading each part - x-amz-checksum-sha256 header sent on every presigned PUT - checksum_sha256 included in complete request for api forwarding to s3 CompleteMultipartUpload - 7 new tests: compute_sha256_with_parts (single/multi part, whole-file hash, independent verification), artifact param serialization, completed part serialization
lee-reinhardt
force-pushed
the
lee-eng-1663
branch
from
01b53e8 to
ad00ba2
Compare
mobileoverlord
approved these changes
Apr 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Per-part SHA-256 checksums for multipart uploads, computed during artifact discovery and sent with
create_runtimeso the API can sign them into presigned URLs. S3 validates each part inline at upload time.compute_sha256_with_parts()computes whole-file hex SHA-256 and per-part base64 checksums in a single read pass (50 MiB chunks)dd+openssl dgst -sha256 -binary | openssl base64 -Ato compute per-part checksums inside the build containerpart_checksumsandpart_sizeincreate_runtimerequest (part_checksumsskipped if empty for backward compat)x-amz-checksum-sha256header sent on every presigned PUTchecksum_sha256included per part in complete request for API forwarding to S3CompleteMultipartUploadAPI dependency — deploy API first
Requires peridio/avocado-connect-mono-repo#350 deployed before merging. The API must sign
x-amz-checksum-sha256into presigned URLs and setChecksumAlgorithm: SHA256onCreateMultipartUpload. Without the API changes, S3 rejects uploads withAccessDenied: headers not signed.Test coverage (7 new tests)
compute_sha256_with_parts: single part, multiple parts, whole-file hash correctness, independent part checksum verificationArtifactParamserialization: includespart_checksums/part_sizewhen present, omitspart_checksumswhen emptyCompletedPartserialization: includeschecksum_sha256E2E verified (local k3d + MinIO)
SignatureDoesNotMatchcargo build— cleancargo test— all pass (749 + 756 + integration suites)