mirror of
https://github.com/fleetdm/fleet
synced 2026-05-21 16:08:47 +00:00
810eb58b95
#10292, #12554 When scanning tens of thousands of files for permissions, using the `find` command exposed as a fleetd table is more performant than trying to use the `file` table. This change caused the watchdog to *stop* killing osquery because of exceeding memory or CPU limit. - [X] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or docs/Contributing/API-for-contributors.md)~ - ~[ ] Documented any permissions changes~ - ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements)~ - ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for new osquery data ingestion features.~ - [X] Added/updated tests - [X] Manual QA for all new/changed functionality - For Orbit and Fleet Desktop changes: - [X] Manual QA must be performed in the three main OSs, macOS, Windows and Linux. - ~[ ] Auto-update manual QA, from released version of component to new version (see [tools/tuf/test](../tools/tuf/test/README.md)).~ |
||
|---|---|---|
| .. | ||
| gnuplot_osqueryd_cpu_memory.sh | ||
| README.md | ||
Load test of osquery queries in macOS
Following are the steps to load test osquery on macOS. The purpose is to know the impact of Fleet provided queries on real devices.
At the time of writing the changes to add watchog logging needed for this script are under review: https://github.com/osquery/osquery/pull/8070. You will have to build osqueryd from source code.
Requirements
- Install gnuplot and ripgrep:
brew install gnuplot ripgrep
- Tooling to build osqueryd from source (at the time of writing this is needed), see https://osquery.readthedocs.io/en/stable/development/building/.
Build fleetd_tables
We are going to use the fleetd tables as an extension so that it is also monitored by the watchdog.
make fleetd-tables-darwin-universal
sudo cp fleetd_tables_darwin_universal.ext /usr/local/osquery_extensions/fleetd_tables.ext
echo "/usr/local/osquery_extensions/fleetd_tables.ext" > /tmp/extensions.load
Run osquery
The following assumes a Fleet server instance running and listening at
localhost:8080.
sudo ENROLL_SECRET=<...> ./osquery/osqueryd \
--verbose=true \
--tls_dump=true \
--pidfile=/Users/luk/osqueryd/osquery.pid \
--database_path=/Users/luk/osqueryd/osquery.db \
--logger_path=/Users/luk/osqueryd/osquery_log \
--host_identifier=instance \
--tls_server_certs=/Users/luk/fleetdm/git/fleet/tools/osquery/fleet.crt \
--enroll_secret_env=ENROLL_SECRET \
--tls_hostname=localhost:8080 \
--enroll_tls_endpoint=/api/v1/osquery/enroll \
--config_plugin=tls \
--config_tls_endpoint=/api/v1/osquery/config \
--config_refresh=60 \
--disable_distributed=false \
--distributed_plugin=tls \
--distributed_tls_max_attempts=10 \
--distributed_tls_read_endpoint=/api/v1/osquery/distributed/read \
--distributed_tls_write_endpoint=/api/v1/osquery/distributed/write \
--logger_plugin=tls,filesystem \
--logger_tls_endpoint=/api/v1/osquery/log \
--disable_carver=false \
--carver_disable_function=false \
--carver_start_endpoint=/api/v1/osquery/carve/begin \
--carver_continue_endpoint=/api/v1/osquery/carve/block \
--carver_block_size=2000000 \
--extensions_autoload=/tmp/extensions.load
--allow_unsafe \
--enable_watchdog_debug \
--distributed_denylist_duration 0 \
--enable_extensions_watchdog 2>&1 | tee /tmp/osqueryd.log
Render CPU and memory usage
./tools/loadtest/osquery/macos/gnuplot_osqueryd_cpu_memory.sh
The horizontal red line is the configured CPU usage limit (hardcoded to
1200msin thegnuplot_osqueryd_cpu_memory.sh)