Elgato_dark/fleet
1
0
Fork 0
mirror of https://github.com/fleetdm/fleet synced 2026-05-22 16:39:01 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 140
19881 commits 3323 branches 575 tags 2.3 GiB
Commit graph

60 commits

Author SHA1 Message Date
Ian Littman
f91aa591b0
Target Redis 6 everywhere rather than a mix of 5 and 6 (#35373) 
Redis 5 has been EOL for a few years, and didn't get updates for the
latest high-severity CVEs. We're already using 6 in most places
(fleetctl preview, recommended reference architectures, managed cloud
environments) so it's safe to set 6 as the new minimum.

Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
 2025-11-10 17:05:44 -06:00
Jonathan Rudenberg
48760fec58
Add support for reading private_key from AWS Secrets Manager (#31134) 
Adds support for reading server `private_key` from AWS Secrets Manager.
Combined with #31075, this should allow removing all common sensitive
secrets from the environment/config (if I missed any let me know). This
works with localstack for local development (set
`AWS_ENDPOINT_URL=$LOCALSTACK_URL`, `AWS_ACCESS_KEY_ID=test`, and
`AWS_SECRET_ACCESS_KEY=test`).

I did not include config options for `AWS_ACCESS_KEY_ID` and
`AWS_SECRET_ACCESS_KEY` because they are a bad practice vs role
credentials and defeat the purpose of this feature which is to remove
secrets from the environment/config.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Scott Gress <scott@fleetdm.com>
 2025-09-09 16:56:35 -05:00
Victor Lyuboslavsky
98954e8113
Make MySQL test port configurable (#31782) 
Fixes #31781 

Only test-related changes.

I found this useful for agentic AI workflows. For example, you have an
AI agent debugging/rerunning a test. Meanwhile, you can spin up another
`mysql_test` instance in another workarea and work there in parallel.
 2025-08-12 15:45:43 +02:00
Lucas Manuel Rodriguez
404f0d3ac0
Migrate from aws-sdk-go v1 to v2 (#30308) 
#29482

[Migrate to the AWS SDK for Go
v2](https://docs.aws.amazon.com/sdk-for-go/v2/developer-guide/migrate-gosdk.html)
documents how to migrate codebases.

QA on features that use AWS SDK Go:
- Bootstrap package:
  - upload:  ✅
  - download: ✅
  - cleanup: ✅
- Software (upload, download, installation, etc.) ✅
  - Cloudfront: Luckly, this feature was already using aws-sdk-go-v2.
- Carves ✅
- Logging:
	- Firehose ✅
	- Kinesis ✅
- Lambda ✅ (tested result logs to a lambda function on our AWS Dogfood
account)
- Email:
	- Amazon SES TODO ⚠️ (this is what Dogfood uses and a few customers)
- We cannot easily test locally, we can use dogfood or load testing
(AWS) environments.

---

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [ ] Manual QA for all new/changed functionality
 2025-06-30 17:45:39 -03:00
Lucas Manuel Rodriguez
148d914f01
Optimize software_titles query to use indexes (#25722) 
For #25160.

Measured improvement by splitting the MySQL query into two queries to
use the indexes more efficiently:
- ~8s vs ~100ms for ~30k entries in software_titles for ~1.7k incoming
software without bundle_identifier (linux software).
- ~1.64s vs ~2ms for 25k entries in software_titles and ~500 incoming
new software with bundle_identifier + ~200 new software without
bundle_identifier (macOS software).

---

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [X] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [X] Manual QA for all new/changed functionality
 2025-01-23 15:48:21 -03:00
Lucas Manuel Rodriguez
963cc7e22c
Automatic install custom packages (#25021) 
#24385

Some docs change here: https://github.com/fleetdm/fleet/pull/25026.

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
 2024-12-27 15:10:28 -03:00
Lucas Manuel Rodriguez
f6b2ec8793
Remove unused toxiproxy service (#21554) 
Everyone running this thing by default when doing `docker compose up`
does not make sense. I added it and used once a few years ago.
 2024-08-26 13:28:51 -03:00
Dante Catalfamo
5e1a3d03ae
MySQL 8.0 Migration (#20225) 
#17249
 2024-07-22 16:27:36 -04:00
Lucas Manuel Rodriguez
28ca463d13
iOS/iPadOS as platforms/labels (#20126) 
#19963 

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added/updated tests
- [X] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [X] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [X] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [X] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [X] Manual QA for all new/changed functionality

---

# API changes for dashboard UI changes

## Main dashboard page

`GET /api/latest/fleet/host_summary?low_disk_space=32` (see
`ios`/`ipados` platforms and `iOS`/`iPadOS` labels)
```json
{
  "totals_hosts_count": 9,
  "online_count": 0,
  "offline_count": 9,
  "mia_count": 0,
  "missing_30_days_count": 0,
  "new_count": 0,
  "all_linux_count": 2,
  "low_disk_space_count": 3,
  "builtin_labels": [
    {
      "id": 1,
      "name": "macOS 14+ (Sonoma+)",
      "description": "macOS hosts with version 14 and above",
      "label_type": "builtin"
    },
    {
      "id": 7,
      "name": "All Hosts",
      "description": "All hosts which have enrolled in Fleet",
      "label_type": "builtin"
    },
    {
      "id": 8,
      "name": "macOS",
      "description": "All macOS hosts",
      "label_type": "builtin"
    },
    {
      "id": 9,
      "name": "Ubuntu Linux",
      "description": "All Ubuntu hosts",
      "label_type": "builtin"
    },
    {
      "id": 10,
      "name": "CentOS Linux",
      "description": "All CentOS hosts",
      "label_type": "builtin"
    },
    {
      "id": 11,
      "name": "MS Windows",
      "description": "All Windows hosts",
      "label_type": "builtin"
    },
    {
      "id": 12,
      "name": "Red Hat Linux",
      "description": "All Red Hat Enterprise Linux hosts",
      "label_type": "builtin"
    },
    {
      "id": 13,
      "name": "All Linux",
      "description": "All Linux distributions",
      "label_type": "builtin"
    },
    {
      "id": 14,
      "name": "chrome",
      "description": "All Chrome hosts",
      "label_type": "builtin"
    },
    {
      "id": 15,
      "name": "iOS",
      "description": "All iOS hosts",
      "label_type": "builtin"
    },
    {
      "id": 16,
      "name": "iPadOS",
      "description": "All iPadOS hosts",
      "label_type": "builtin"
    }
  ],
  "platforms": [
    {
      "platform": "darwin",
      "hosts_count": 3
    },
    {
      "platform": "ios",
      "hosts_count": 1
    },
    {
      "platform": "ipados",
      "hosts_count": 1
    },
    {
      "platform": "rhel",
      "hosts_count": 1
    },
    {
      "platform": "ubuntu",
      "hosts_count": 1
    },
    {
      "platform": "windows",
      "hosts_count": 2
    }
  ]
}
```

## After selecting a platform

`GET /api/latest/fleet/host_summary?platform=ios&low_disk_space=100`
(similar with `ipados`)
```json
{
  "totals_hosts_count": 1,
  "online_count": 0,
  "offline_count": 1,
  "mia_count": 0,
  "missing_30_days_count": 0,
  "new_count": 0,
  "all_linux_count": 0,
  "low_disk_space_count": 1,
  "builtin_labels": [
    {
      "id": 1,
      "name": "macOS 14+ (Sonoma+)",
      "description": "macOS hosts with version 14 and above",
      "label_type": "builtin"
    },
    {
      "id": 7,
      "name": "All Hosts",
      "description": "All hosts which have enrolled in Fleet",
      "label_type": "builtin"
    },
    {
      "id": 8,
      "name": "macOS",
      "description": "All macOS hosts",
      "label_type": "builtin"
    },
    {
      "id": 9,
      "name": "Ubuntu Linux",
      "description": "All Ubuntu hosts",
      "label_type": "builtin"
    },
    {
      "id": 10,
      "name": "CentOS Linux",
      "description": "All CentOS hosts",
      "label_type": "builtin"
    },
    {
      "id": 11,
      "name": "MS Windows",
      "description": "All Windows hosts",
      "label_type": "builtin"
    },
    {
      "id": 12,
      "name": "Red Hat Linux",
      "description": "All Red Hat Enterprise Linux hosts",
      "label_type": "builtin"
    },
    {
      "id": 13,
      "name": "All Linux",
      "description": "All Linux distributions",
      "label_type": "builtin"
    },
    {
      "id": 14,
      "name": "chrome",
      "description": "All Chrome hosts",
      "label_type": "builtin"
    },
    {
      "id": 15,
      "name": "iOS",
      "description": "All iOS hosts",
      "label_type": "builtin"
    },
    {
      "id": 16,
      "name": "iPadOS",
      "description": "All iPadOS hosts",
      "label_type": "builtin"
    }
  ],
  "platforms": [
    {
      "platform": "ios",
      "hosts_count": 1
    }
  ]
}
```

### To populate list of MDM solutions of a selected platform

`GET /api/latest/fleet/hosts/summary/mdm\?platform=ios` (similar with
`ipados`)

```json
{
  "counts_updated_at": "2024-06-27T21:56:45Z",
  "mobile_device_management_enrollment_status": {
    "enrolled_manual_hosts_count": 0,
    "enrolled_automated_hosts_count": 1,
    "pending_hosts_count": 0,
    "unenrolled_hosts_count": 0,
    "hosts_count": 1
  },
  "mobile_device_management_solution": [
    {
      "id": 1,
      "name": "Fleet",
      "server_url": "https://lucas-fleet.ngrok.app/mdm/apple/mdm",
      "hosts_count": 1
    }
  ]
}
```

### To populate OS versions of a selected platform

`GET /api/latest/fleet/os_versions?platform=ipados` (similar with `ios`)
```json
{
  "meta": {
    "has_next_results": false,
    "has_previous_results": false
  },
  "count": 1,
  "counts_updated_at": "2024-06-27T21:36:12Z",
  "os_versions": [
    {
      "os_version_id": 7,
      "hosts_count": 1,
      "name": "iPadOS 17.5.1",
      "name_only": "iPadOS",
      "version": "17.5.1",
      "platform": "ipados",
      "vulnerabilities": []
    }
  ]
}
```

## Filtering hosts by the two new `iOS`/`iPadOS` labels

Works the same as with other labels.
 2024-07-08 18:05:29 -03:00
Victor Lyuboslavsky
f761827850
Move CalculateAggregatedPerfStatsPercentiles reads to the replica (#19206) 
Move CalculateAggregatedPerfStatsPercentiles reads to the replica
#18838

I manually tested the aggregated query stats change by using a read
replica.
https://github.com/fleetdm/fleet/blob/main/tools/mysql-replica-testing/README.md
- But set `master_delay=0` due to issue
https://github.com/fleetdm/fleet/issues/19272
- Run a saved query as a live query, and see that its stats in
`aggregated_stats` table are updated.


# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
 2024-05-31 07:08:31 -05:00
Lucas Manuel Rodriguez
b7aac2cfab
Fix bundle_identifier for common software in osquery-perf (#19023) 
bundle_identifier should be unique for the common software (should not
have hostname name in it).
 2024-05-24 09:26:42 -03:00
Lucas Manuel Rodriguez
1afb015f6c
Test DB migrations with Percona XtraDB MySQL server 5.7.25 (#16320) 
#15881

This PR adds a script to test DB migrations with Percona XtraDB 5.7.25.

PS: To run this test before we merge this PR to `main` you will need to
change step 2 (`Make sure to be on latest main`), instead of `main` use
this branch `15881-test-migrations-with-percona`.
 2024-01-29 14:21:37 -03:00
Tim Lee
0557f10ac5
14729 smtp settings validation for TLS (#15029) 2023-11-21 11:48:21 -07:00
Lucas Manuel Rodriguez
9facf144dc
Schedule queries on hosts even if automations off (given we now store results) (#14411) 
This is ready to go.

---------

Co-authored-by: Tim Lee <timlee@fleetdm.com>
 2023-10-11 15:20:06 -03:00
Luke Heath
79a9451952
Set MySQL table-definition-cache to 2048 (#13864) 2023-09-11 15:04:43 -05:00
Roberto Dip
a23d208b1d
gate DEP enrollment behind SSO when configured (#11309) 
#10739

Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com>
Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>
 2023-04-27 09:43:20 -03:00
Martin Angers
9aab3d628c
Move Redis cluster docker yml to separate file (#11162) 2023-04-12 15:14:28 -04:00
Lucas Manuel Rodriguez
40265d0e6f
Fix SMTP e-mail send when SMTP server has credentials (#10758) 
#9609

This PR also fixes #10777.

The issue is: We were using `svc.AppConfig` instead of
`svc.ds.AppConfig` to retrieve the SMTP credentials.
`svc.AppConfig` obfuscates credentials, whereas `svc.ds.AppConfig` does
not.
To help prevent this from happening again I've renamed `svc.AppConfig`
to `svc.AppConfigObfuscated`.
I've also added a new test SMTP server
(https://github.com/axllent/mailpit) that supports Basic Authentication
and tests that make use of it to catch these kind of bugs (the tests are
executed when running `go test` with `MAIL_TEST=1`).

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
  - ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
 2023-03-28 15:23:15 -03:00
Lucas Manuel Rodriguez
78df7b6f70
Add guide on how to simulate slow connections to redis and mysql (#9140) 
* Add guide on how to simulate slow connections to redis and mysql

* Add pageOrderInSection
 2023-01-03 17:29:40 -03:00
Lucas Manuel Rodriguez
9191f4ce66
Add Apple MDM functionality (#7940) 
* WIP

* Adding DEP functionality to Fleet

* Better organize additional MDM code

* Add cmdr.py and amend API paths

* Fix lint

* Add demo file

* Fix demo.md

* go mod tidy

* Add munki setup to Fleet

* Add diagram to demo.md

* Add fixes

* Update TODOs and demo.md

* Fix cmdr.py and add TODO

* Add endpoints to demo.md

* Add more Munki PoC/demo stuff

* WIP

* Remove proposals from PoC

* Replace prepare commands with fleetctl commands

* Update demo.md with current state

* Remove config field

* Amend demo

* Remove Munki setup from MVP-Dogfood

* Update demo.md

* Add apple mdm commands (#7769)

* fleetctl enqueue mdm command

* fix deps

* Fix build

Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>

* Add command to upload installers

* go mod tidy

* fix subcommands help

There is a bug in urfave/cli where help text is not generated properly when subcommands
are nested too deep.

* Add support for installing apps

* Add a way to list enrolled devices

* Add dep listing

* Rearrange endpoints

* Move DEP routine to schedule

* Define paths globally

* Add a way to list enrollments and installers

* Parse device-ids as comma-separated string

* Remove unused types

* Add simple commands and nest under enqueue-command

* Fix simple commands

* Add help to enqueue-command

* merge apple_mdm database

* Fix commands

* update nanomdm

* Split nanomdm and nanodep schemas

* Set 512 MB in memory for upload

* Remove empty file

* Amend profile

* Add sample commands

* Add delete installers and fix bug in DEP profile assigning

* Add dogfood.md deployment guide

* Update schema.sql

* Dump schema with MySQL 5

* Set default value for authenticate_at

* add tokens to enrollment profiles

When a device downloads an MDM enrollment profile, verify the token passed
as a query parameter. This ensures untrusted devices don't enroll with
our MDM server.

- Rename enrollments to enrollment profiles. Enrollments is used by nano
  to refer to devices that are enrolled with MDM
- Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles
- Generate a token for authentication when creating an enrollment profile
- Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token=

* remove mdm apple server url

* update docs

* make dump-test-schema

* Update nanomdm with missing prefix table

* Add docs and simplify changes

* Add changes file

* Add method docs

* Fix compile and revert prepare.go changes

* Revert migration status check change

* Amend comments

* Add more docs

* Clarify storage of installers

* Remove TODO

* Remove unused

* update dogfood.md

* remove cmdr.py

* Add authorization tests

* Add TODO comment

* use kitlog for nano logging

* Add yaml tags

* Remove unused flag

* Remove changes file

* Only run DEP routine if MDM is enabled

* Add docs to all new exported types

* Add docs

* more nano logging changes

* Fix unintentional removal

* more nano logging changes

* Fix compile test

* Use string for configs and fix config test

* Add docs and amend changes

* revert changes to basicAuthHandler

* remove exported BasicAuthHandler

* rename rego authz type

* Add more information to dep list

* add db tag

* update deps

* Fix schema

* Remove unimplemented

Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com>
Co-authored-by: Michal Nicpon <michal@fleetdm.com>
 2022-10-05 19:53:54 -03:00
Zach Wasserman
db7d1c5bf5
Strongly discourage use of MariaDB (#7699) 2022-09-23 11:18:34 -07:00
Lucas Manuel Rodriguez
142f19f286
Turn mysql event scheduler off (#6640) 2022-07-13 13:50:58 -03:00
Lucas Manuel Rodriguez
22382c757c
Add better support for M1 to docker-compose.yml and add fleet-dev target to Makefile (#6432) 
* Add better support for M1 to docker-compose.yml and add fleet-dev target

* Amend comment
 2022-06-29 14:38:23 -03:00
Lucas Manuel Rodriguez
fc7650c4f8
Fix mock tests set test name (#6345) 
* Set mock test on live query mocked tests

* Use MySQL 5.7 as default in docker-compose.yml
 2022-06-23 11:34:52 -03:00
Zach Wasserman
26eae438f6
Document supported MySQL versions (#5421) 
- Pin versions in development and CI.
 2022-04-27 16:21:16 -07:00
Martin Angers
fd0cd153ce
Fix SSO paths to always use /v1/ instead of /latest/ (#5246) 2022-04-20 12:46:45 -04:00
Martin Angers
90b15071a4
Introduce API version 2022-04, deprecate use of /global in paths (#4731) 2022-04-05 11:35:53 -04:00
Zach Wasserman
35523017c9
Make migrations compatible with GTID replication (#2615) 
* Make migrations compatible with GTID replication

Fixes an issue some deployments encountered when migrations used a
statement that is unsupported in GTID replication mode (#2462).

Local dev MySQL now enforces this consistency, so it should be easier to
maintain compatibility going forward.

* Update docker-compose formatting

* if exists
 2021-10-21 07:46:02 -03:00
Martin Angers
eda6d60be8
Add replicas to the redis docker-compose setup. (#2511) 2021-10-18 08:33:26 -04:00
Benjamin Edwards
16c5823692
add support for minio backend file carving (#2448) 
* add support for minio backend file carving
* add changes file
Co-authored-by: Zach Wasserman <zach@fleetdm.com>
 2021-10-12 15:32:06 -04:00
Martin Angers
9a0871a2f1
Address issues related to Redis Cluster support (#1885) 
Closes #1847 .
 2021-09-01 16:32:57 -04:00
Benjamin Edwards
c18214be37
add logging settings to config api response (#1467) 
- add docker-compose file for locally testing aws dependencies
- update firehose & kinesis configs to optionally supply endpoint url override
- serialize `logging` field in appconfig api response
 2021-07-30 11:45:49 -04:00
Tomas Touceda
a6cff7ea89
Migrate all mysql tests to the new form (#1408) 
* Migrate all mysql tests to the new form

* Only dump sql if MYSQL_TEST is on

* Removing parallel until we get rid of this code

* Move TestMain to an actual _test file

* A little experiment with tmpfs to speed up the db

* Let's make sure the dump.sql file is also in ram
 2021-07-19 18:20:31 -03:00
Zach Wasserman
8be9068c1c
Fix migration for MariaDB compatibility (#1280) 
Give unique names to each new foreign key in the migrations.

Fixes #1279
 2021-07-01 15:59:31 -07:00
Tomas Touceda
8e2db1fecf
For platform on mysql so it runs in Apple M1 macbooks (#1274) 2021-07-01 13:09:51 -03:00
Zach Wasserman
8c97b36764
Set up Cypress testing for Teams/Tiers (#1005) 
- Update names/roles of users in `make e2e-setup`.
- Update test SSO user info.
- Add Cypress commands for seeding users/Teams.
- Stub Cypress tests for team/tier matrix.
 2021-06-09 11:56:59 -07:00
Zach Wasserman
6860dec81a
Update flags for development cAdvisor (#679) 
Updates to a minimal set of permissions for working cAdvisor.

Tested with Docker Desktop 3.3.1 on macOS 11.2.3
 2021-04-23 17:58:23 -07:00
Zach Wasserman
e961cfe0c3
Batch updates to host seen time (#633) 
Instead of synchronously updating the seen_time column for a host on an update, batch these updates to be written together every 1 second.

This results in a ~33% reduction in MySQL CPU usage in a local test with 4,000 simulated hosts and MySQL running in Docker.
 2021-04-12 16:22:22 -07:00
Zach Wasserman
9f71fcf440
Speed up MySQL tests (#585) 
Improves MySQL test time (on my 2020 MBP) to ~18s from ~125s.

- Use separate databases for each test to allow parallelization.
- Run migrations only once at beginning of tests and then reload
  generated schema.
- Add `--innodb-file-per-table=OFF` for ~20% additional speedup.
 2021-04-03 11:42:27 -07:00
Zach Wasserman
ee580bf811
Add ability to test SSO locally (#552) 
- Use IdP in Docker container.
- Document process.
 2021-03-29 13:50:39 -07:00
Zach Wasserman
c5f931bb54
Bump development docker-compose Redis to version 5 (#414) 2021-03-08 14:36:10 -08:00
Zach Wasserman
8fcd14b394
Add --dev flag and change docker defaults (#251) 
- Add --dev flag that will set default flag values. This simplifies the
  invocation of Fleet in a development environment.
- Change defaults in docker-compose to use `fleet` in place of `kolide`.
-  Skip prompt in `prepare db` when `--dev` specified.
- Update developer documentation.

Updates to MySQL configuration in docker-compose.yml may require
existing development containers and volumes to be deleted (this will
delete data in MySQL):

```shell
docker-compose rm -sf
docker volume rm fleet_mysql-persistent-volume
```

Closes #170
 2021-02-01 18:14:16 -08:00
Zach Wasserman
6cbd10965c
Add dev infrastructure and docs for Prometheus monitoring (#33) 
- Set up a simple example of Prometheus monitoring in the development
  docker-compose.yml.
- Add documentation for configuring Prometheus.
 2020-11-12 19:06:56 -08:00
Brendan Shaklovitz
87137252de
docker-compose QoL improvements (#2319) 
* Use YAML anchors to avoid repeating config blocks
* Use docker volumes to persist data for mysql
* Allow setting `FLEET_SERVER` (fixes #2127) when using the docker-compose file to spin up multiple osquery clients
 2020-10-09 10:10:33 -07:00
Zachary Wasserman
2bfcecf0d3
Add CAdvisor container to dev docker-compose (#2263) 
This container is useful for monitoring stats of other containers.
 2020-07-21 15:36:11 -07:00
Austin Burnett
59efb495ca Add automatic host expiration capability (#2117) 
When configured, this feature will delete hosts that have not checked in after the specified number of days.

Closes #1860.
 2019-10-16 16:35:17 -07:00
Zachary Wasserman
07e7d336dd Enable slow query log on dev DBs (#1437) 
With this change, MySQL will log "slow" queries to the `mysql.slow_log` table.
 2017-03-20 17:02:44 -07:00
Zachary Wasserman
c0d87cb69a Use persistent MySQL for local server, tmpfs MySQL for tests (#1245) 2017-02-17 09:15:10 -08:00
John Murphy
56195b0768 Changed mysql version to match that of circle.yaml (#520) 2016-11-21 21:59:48 +08:00
Zachary Wasserman
1d06ad80c8 Use tmpfs with dev MySQL instances (#483) 
This results in ~100x speedup in MySQL tests.
 2016-11-15 10:06:33 -08:00