Elgato_dark/fleet
1
0
Fork 0
mirror of https://github.com/fleetdm/fleet synced 2026-05-21 16:08:47 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 129
fleet/docs/Contributing
History
Lucas Manuel Rodriguez 6415f062c6
Reduce size of DistributedQueryResult to improve live query performance (#11882) 
This was found while working on #10957.

When running a live query, a lot of unused host data is stored in Redis
and sent on every live query result message via websockets. The frontend
and fleetctl just need `id`, `hostname` and `display_name`. (This
becomes worse every time we add new fields to the `Host` struct.)

Sample of one websocket message result when running `SELECT * from
osquery_info;`:

size in `main`: 2234 bytes
```
a["{\"type\":\"result\",\"data\":{\"distributed_query_execution_id\":57,\"host\":
{\"created_at\":\"2023-05-22T12:14:11Z\",\"updated_at\":\"2023-05-23T12:31:51Z\",
\"software_updated_at\":\"0001-01-01T00:00:00Z\",\"id\":106,\"detail_updated_at\":\"2023-05-23T11:50:04Z\",
\"label_updated_at\":\"2023-05-23T11:50:04Z\",\"policy_updated_at\":\"1970-01-02T00:00:00Z\",
\"last_enrolled_at\":\"2023-05-22T12:14:12Z\",
\"seen_time\":\"2023-05-23T09:52:23.876311-03:00\",\"refetch_requested\":false,
\"hostname\":\"lucass-macbook-pro.local\",\"uuid\":\"BD4DFA10-E334-41D9-8136-D2163A8FE588\",\"platform\":\"darwin\",\"osquery_version\":\"5.8.2\",\"os_version\":\"macOS 13.3.1\",\"build\":\"22E261\",\"platform_like\":\"darwin\",\"code_name\":\"\",
\"uptime\":91125000000000,\"memory\":34359738368,\"cpu_type\":\"x86_64h\",\"cpu_subtype\":\"Intel x86-64h Haswell\",\"cpu_brand\":\"Intel(R) Core(TM) i7-1068NG7 CPU @ 2.30GHz\",\"cpu_physical_cores\":4,\"cpu_logical_cores\":8,\"hardware_vendor\":\"Apple Inc.\",\"hardware_model\":\"MacBookPro16,2\",\"hardware_version\":\"1.0\",
\"hardware_serial\":\"0DPQR4HMD1FZ\",
\"computer_name\":\"Lucas’s MacBook Pro\",\"public_ip\":\"\",
\"primary_ip\":\"192.168.0.230\",\"primary_mac\":\"68:2f:67:8e:b6:1f\",
\"distributed_interval\":1,\"config_tls_refresh\":60,\"logger_tls_period\":10,\"team_id\":null,
\"pack_stats\":null,\"team_name\":null,
\"gigs_disk_space_available\":386.23,\"percent_disk_space_available\":40,
\"issues\":{\"total_issues_count\":0,\"failing_policies_count\":0},
\"mdm\":{\"enrollment_status\":null,\"server_url\":null,\"name\":\"\",\"encryption_key_available\":false},
\"status\":\"online\",\"display_text\":\"lucass-macbook-pro.local\",\"display_name\":\"Lucas’s MacBook Pro\"},
\"rows\":[{\"build_distro\":\"10.14\",\"build_platform\":\"darwin\",
\"config_hash\":\"b7ee9363a7c686e76e99ffb122e9c5241a791e69\",\"config_valid\":\"1\",
\"extensions\":\"active\",\"host_display_name\":\"Lucas’s MacBook Pro\",
\"host_hostname\":\"lucass-macbook-pro.local\",\"instance_id\":\"cde5de81-344b-4c76-b1c5-dae964fdd4f2\",\"pid\":\"8370\",\"platform_mask\":\"21\",\"start_time\":\"1684757652\",
\"uuid\":\"BD4DFA10-E334-41D9-8136-D2163A8FE588\",
\"version\":\"5.8.2\",\"watcher\":\"8364\"}],\"error\":null}}"]
```

vs. size of the message result on this branch: 675 bytes
```
a["{\"type\":\"result\",\"data\":{\"distributed_query_execution_id\":59,
\"host\":{\"id\":106,\"hostname\":\"lucass-macbook-pro.local\",
\"display_name\":\"Lucas’s MacBook Pro\"},
\"rows\":[{\"build_distro\":\"10.14\",\"build_platform\":\"darwin\",
\"config_hash\":\"f80dee827635db39077a458243379b3ad63311fd\",
\"config_valid\":\"1\",\"extensions\":\"active\",\"host_display_name\":\"Lucas’s MacBook Pro\",
\"host_hostname\":\"lucass-macbook-pro.local\",
\"instance_id\":\"cde5de81-344b-4c76-b1c5-dae964fdd4f2\",\"pid\":\"8370\",\"platform_mask\":\"21\",
\"start_time\":\"1684757652\",\"uuid\":\"BD4DFA10-E334-41D9-8136-D2163A8FE588\",\"version\":\"5.8.2\",
\"watcher\":\"8364\"}]}}"]
```

Manual tests included running with an old fleetctl running with a new
fleet server, and vice-versa, a new fleetctl running against an old
fleet server.

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
  - ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
2023-05-25 08:11:53 -03:00
..
Adding-new-endpoints.md Docs: Fix broken links in documentation (#11424) 2023-04-28 16:04:33 -05:00
API-for-contributors.md Reduce size of DistributedQueryResult to improve live query performance (#11882) 2023-05-25 08:11:53 -03:00
API-Versioning.md Replace references to /api/latest/fleet with /api/v1/fleet (#7464) 2022-08-31 09:51:25 -05:00
Automatically-generating-UI-component-boilerplate.md Clean up docs for generate (#11687) 2023-05-16 13:40:55 -07:00
Building-Fleet.md Docs: Fix broken links in documentation (#11424) 2023-04-28 16:04:33 -05:00
Committing-Changes.md change relative links in documentation (#8441) 2022-10-26 18:26:49 -05:00
Configuration-for-contributors.md Remove packs from docs (#9452) 2023-01-20 16:47:24 -06:00
FAQ.md Docs: Fix broken FAQ link (#11405) 2023-04-28 10:34:20 -05:00
Fleet-UI-Testing.md Documentation: Update testing docs with instructions how to run live QA Wolf e2e tests (#11318) 2023-04-26 11:21:21 -04:00
Migrations.md change relative links in documentation (#8441) 2022-10-26 18:26:49 -05:00
Orbit-development-and-release-strategy.md Docs: add missing pageOrderInSection meta tag (#8104) 2022-10-06 11:04:36 -05:00
README.md Run-Locally-Built-Orbit (#9359) 2023-01-19 10:31:24 -06:00
Releasing-Fleet.md Update Releasing-Fleet.md (#11106) 2023-04-24 11:42:44 -05:00
Run-Locally-Built-Orbit.md Add mTLS support to fleetd (#11319) 2023-04-27 08:44:39 -03:00
Seeding-Data.md Documentation: Seeding data links to creating test hosts (#8131) 2022-10-10 14:08:18 -04:00
Simulate-slow-network.md Add guide on how to simulate slow connections to redis and mysql (#9140) 2023-01-03 17:29:40 -03:00
Testing-and-local-development.md Docs: remove issuer_uri from end_user_authentication (#11639) 2023-05-23 10:59:28 -07:00

README.md

Contributing

Building Fleet

Learn about building the code, development infrastructure, and database migrations.

Build And Run Orbit From Local Code

Building and running your modified code.

Testing

Look at Fleet's full test suite and integration tests.

Migrations

Learn about creating and updating database migrations.

Committing changes

Learn how to merge changes into the codebase.

Releasing Fleet

Learn the process for how to release Fleet.

Seeding data

Learn how to add fake data to your development instance.

API for contributors

Get to grips with Fleet API routes. This documentation is helpful for developing or contributing to Fleet.

FAQ

Find commonly asked questions and answers about contributing to Fleet as part of our community.