mirror of
https://github.com/fleetdm/fleet
synced 2026-05-23 17:08:53 +00:00
e8999e8ddb
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #35007 # Details This PR implements a new `StreamHosts` service method and utilizes this method in `listHostsEndpoint`. The purpose of this method is to allow the "list hosts" endpoint to stream a JSON response down one host at a time (more or less), so that the entire JSON payload doesn't have to sit in memory all at once. The response can be very large when software and users are populated using the optional `populate_software` and `populate_users` querystring options. **Notes:** * The `ListHosts` database method currently returns the entire result set at once. The assumption with this iteration is that it was populating the extra data in each host that was pushing the response to untenable sizes. If we're still seeing OOM issues after this PR, we can refactor to also use an iterator to stream rows from the database. * The `listHostsEndpoint` method calls `StreamHosts` to get a host iterator, but also creates its own iterator to supply to the response, because we (sometimes) need to populate `Labels` on each host response object. The `StreamHosts` method only returns `Host`s not `HostResponse`s, hence the need for the wrapper. * A new response type `StreamHostsResponse` was created because other methods utilize the existing `ListHostsResponse`. We could keep just `ListHostsResponse` and toggle how it responds based on whether an iterator is provided, but keeping the types separate is simpler to reason about and test. # Checklist for submitter If some of the following don't apply, delete the relevant line. - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [ ] Added/updated automated tests - [X] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [X] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Hosts listing API now streams results incrementally for faster initial responses and better performance on large inventories or complex queries. * Host details (optional labels, software and MDM info) are progressively enriched and delivered as available, allowing partial results while remaining data loads. * Streaming provides clearer partial-error reporting so partial results can be returned even when some enrichments fail. <sub>✏️ Tip: You can customize this high-level summary in your review settings.</sub> <!-- end of auto-generated comment: release notes by coderabbit.ai -->
1 line
88 B
Text
1 line
88 B
Text
- Implemented streaming for the `GET /hosts` ("list hosts") API to improve performance.
|