fleet/server
Anthony Maxwell 41fd8409e9
Feat: Perform S3 Uploads with Upload Manager (#32010)
# Overview

This PR implements the S3 upload manager under-the-hood of our
`datastore/s3` client's `Put()` method.

# Description

As surfaced by #31667, the current S3 implementation utilizes the `PUT`
operation which means services, such as MinIO, take issue with attempted
uploads that are [too
large](https://github.com/minio/minio/blob/master/cmd/streaming-signature-v4.go#L260).
The `PUT` operation can also present challenges in memory-constrained
environments as the entire upload target is read into memory before it's
shipped.

# Notes

- See the `TODO` comment section, there's more cool stuff we can and
should do with this in the future!

# Standard Pull Request Details

## Testing

- [x] QA'd all new/changed functionality manually

For unreleased bug fixes in a release candidate, one of:

- [x] Confirmed that the fix is not expected to adversely impact load
test results

---------

Signed-off-by: Illbjorn <am@hades.so>
2025-08-20 15:20:53 -04:00
..
archtest Basic Android MDM on/off functionality (#26309) 2025-02-18 09:43:11 -06:00
authz Add backend APIs for adding, deleting and listing secret variables (#31936) 2025-08-14 19:33:47 -03:00
aws_common Fix STS assume role in aws-sdk-go v2 (#30699) 2025-07-10 13:00:27 -03:00
bindata
config feat: allow different cisa url to be provided (#31728) 2025-08-13 13:35:45 -05:00
contexts Allow certain licenses to disable telemetry (#29093) 2025-05-22 14:27:07 -04:00
cron Add SCEP endpoint for host identity. (#30589) 2025-07-11 11:44:07 -03:00
datastore Feat: Perform S3 Uploads with Upload Manager (#32010) 2025-08-20 15:20:53 -04:00
errorstore Fixed potential panic in error handler when Redis is down. (#31643) 2025-08-06 17:14:31 +02:00
fleet 4498 empty software (#31940) 2025-08-19 10:38:53 -04:00
goose Add gosimple linter (#23250) 2024-10-29 14:17:51 -05:00
health 🧹 friday cleanup party: substitute deprecated import of go-kit (#19774) 2024-06-17 10:27:31 -03:00
launcher Updating golangci-lint to 1.61.0 (#22973) 2024-10-18 12:38:26 -05:00
live_query Fix flakey livequery test (#21666) 2024-08-29 10:03:45 -06:00
logging Fix STS assume role in aws-sdk-go v2 (#30699) 2025-07-10 13:00:27 -03:00
mail Fix STS assume role in aws-sdk-go v2 (#30699) 2025-07-10 13:00:27 -03:00
mdm IdP Authentication before BYOD (#32017) 2025-08-18 18:31:53 +02:00
mock IdP Authentication before BYOD (#32017) 2025-08-18 18:31:53 +02:00
policies Allow users of Fleet in Primo mode to access Software automations and Failing policy ticket & webhook automations (#30865) 2025-07-17 15:53:31 -07:00
ptr Added better handling of timeout and insufficient permissions errors in NDES SCEP proxy. (#23654) 2024-11-11 14:57:28 -06:00
pubsub Updating golangci-lint to 1.61.0 (#22973) 2024-10-18 12:38:26 -05:00
service Don't flood the terminal with binary output when downloading pkg (#32081) 2025-08-20 12:16:53 -04:00
shellquote Updating golangci-lint to 1.61.0 (#22973) 2024-10-18 12:38:26 -05:00
sso Make sure SAML requests always ask for email format NameID (#31008) 2025-07-17 14:25:33 -04:00
test Updated SQL modes in tests to match production. (#31445) 2025-08-03 08:18:13 +02:00
variables Added verification support for $FLEET_VAR_HOST_UUID (#31777) 2025-08-11 14:47:55 +02:00
version Move external dependency fleetdm/kolide-kit to monorepo (#15861) 2024-01-02 18:22:52 -03:00
vulnerabilities feat: allow different cisa url to be provided (#31728) 2025-08-13 13:35:45 -05:00
webhooks Allow users of Fleet in Primo mode to access Software automations and Failing policy ticket & webhook automations (#30865) 2025-07-17 15:53:31 -07:00
websocket
worker Batch script cron schedule (#31808) 2025-08-14 14:44:47 -04:00
utils.go Fix CleanupExpiredHosts to prevent deletion of DEP-assigned hosts (#28313) 2025-04-18 12:49:03 -05:00
utils_test.go feature: target profiles by labels (#16202) 2024-01-26 11:00:58 -05:00