fleet/server/datastore/s3
Anthony Maxwell 41fd8409e9
Feat: Perform S3 Uploads with Upload Manager (#32010)
# Overview

This PR implements the S3 upload manager under-the-hood of our
`datastore/s3` client's `Put()` method.

# Description

As surfaced by #31667, the current S3 implementation utilizes the `PUT`
operation which means services, such as MinIO, take issue with attempted
uploads that are [too
large](https://github.com/minio/minio/blob/master/cmd/streaming-signature-v4.go#L260).
The `PUT` operation can also present challenges in memory-constrained
environments as the entire upload target is read into memory before it's
shipped.

# Notes

- See the `TODO` comment section, there's more cool stuff we can and
should do with this in the future!

# Standard Pull Request Details

## Testing

- [x] QA'd all new/changed functionality manually

For unreleased bug fixes in a release candidate, one of:

- [x] Confirmed that the fix is not expected to adversely impact load
test results

---------

Signed-off-by: Illbjorn <am@hades.so>
2025-08-20 15:20:53 -04:00
..
bootstrap_package.go Migrate from aws-sdk-go v1 to v2 (#30308) 2025-06-30 17:45:39 -03:00
bootstrap_package_test.go Migrate from aws-sdk-go v1 to v2 (#30308) 2025-06-30 17:45:39 -03:00
carves.go Migrate from aws-sdk-go v1 to v2 (#30308) 2025-06-30 17:45:39 -03:00
carves_test.go report errors that can occur during file carving (#8972) 2022-12-09 13:21:30 -03:00
common_file_store.go Feat: Perform S3 Uploads with Upload Manager (#32010) 2025-08-20 15:20:53 -04:00
s3.go Fix STS assume role in aws-sdk-go v2 (#30699) 2025-07-10 13:00:27 -03:00
software_installer.go Migrate from aws-sdk-go v1 to v2 (#30308) 2025-06-30 17:45:39 -03:00
software_installer_test.go Feat: Perform S3 Uploads with Upload Manager (#32010) 2025-08-20 15:20:53 -04:00
testing_utils.go Migrate from aws-sdk-go v1 to v2 (#30308) 2025-06-30 17:45:39 -03:00