Commit graph

406 commits

Author SHA1 Message Date
Roberto Dip
e630fabf89
token rotation for fleet desktop (#7779)
This implements what's described in detail here https://github.com/fleetdm/fleet/blob/main/proposals/fleet-desktop-token-rotation.md
2022-10-10 17:15:35 -03:00
Frank Sievertsen
e9f7066d87
7135 host display name (#7873) 2022-10-08 08:57:46 -04:00
Lucas Manuel Rodriguez
9191f4ce66
Add Apple MDM functionality (#7940)
* WIP

* Adding DEP functionality to Fleet

* Better organize additional MDM code

* Add cmdr.py and amend API paths

* Fix lint

* Add demo file

* Fix demo.md

* go mod tidy

* Add munki setup to Fleet

* Add diagram to demo.md

* Add fixes

* Update TODOs and demo.md

* Fix cmdr.py and add TODO

* Add endpoints to demo.md

* Add more Munki PoC/demo stuff

* WIP

* Remove proposals from PoC

* Replace prepare commands with fleetctl commands

* Update demo.md with current state

* Remove config field

* Amend demo

* Remove Munki setup from MVP-Dogfood

* Update demo.md

* Add apple mdm commands (#7769)

* fleetctl enqueue mdm command

* fix deps

* Fix build

Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>

* Add command to upload installers

* go mod tidy

* fix subcommands help

There is a bug in urfave/cli where help text is not generated properly when subcommands
are nested too deep.

* Add support for installing apps

* Add a way to list enrolled devices

* Add dep listing

* Rearrange endpoints

* Move DEP routine to schedule

* Define paths globally

* Add a way to list enrollments and installers

* Parse device-ids as comma-separated string

* Remove unused types

* Add simple commands and nest under enqueue-command

* Fix simple commands

* Add help to enqueue-command

* merge apple_mdm database

* Fix commands

* update nanomdm

* Split nanomdm and nanodep schemas

* Set 512 MB in memory for upload

* Remove empty file

* Amend profile

* Add sample commands

* Add delete installers and fix bug in DEP profile assigning

* Add dogfood.md deployment guide

* Update schema.sql

* Dump schema with MySQL 5

* Set default value for authenticate_at

* add tokens to enrollment profiles

When a device downloads an MDM enrollment profile, verify the token passed
as a query parameter. This ensures untrusted devices don't enroll with
our MDM server.

- Rename enrollments to enrollment profiles. Enrollments is used by nano
  to refer to devices that are enrolled with MDM
- Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles
- Generate a token for authentication when creating an enrollment profile
- Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token=

* remove mdm apple server url

* update docs

* make dump-test-schema

* Update nanomdm with missing prefix table

* Add docs and simplify changes

* Add changes file

* Add method docs

* Fix compile and revert prepare.go changes

* Revert migration status check change

* Amend comments

* Add more docs

* Clarify storage of installers

* Remove TODO

* Remove unused

* update dogfood.md

* remove cmdr.py

* Add authorization tests

* Add TODO comment

* use kitlog for nano logging

* Add yaml tags

* Remove unused flag

* Remove changes file

* Only run DEP routine if MDM is enabled

* Add docs to all new exported types

* Add docs

* more nano logging changes

* Fix unintentional removal

* more nano logging changes

* Fix compile test

* Use string for configs and fix config test

* Add docs and amend changes

* revert changes to basicAuthHandler

* remove exported BasicAuthHandler

* rename rego authz type

* Add more information to dep list

* add db tag

* update deps

* Fix schema

* Remove unimplemented

Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com>
Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 19:53:54 -03:00
Juan Fernandez
812d3c85de
Fixes various bugs with NVD vulnerability detection (#7963)
- Improved NVD CPE matching process.
- Fixed bug with the 'software/<id>' endpoint not showing the generated_cpe value.
2022-10-04 07:04:48 -04:00
Martin Angers
8eee0b35f3
Add validations for agent options' command-line flags section. (#7979) 2022-10-03 08:29:41 -04:00
Roberto Dip
3a349bb07b
fix data race in TestAutomationsSchedule (#7952) 2022-09-26 17:19:38 -03:00
Martin Angers
1fd6844bb0
Make osquery-perf aware of kubequery so that hosts are not reported as such erroneously (#7949) 2022-09-26 15:39:39 -04:00
gillespi314
639b85e47b
Refactor cron worker to schedule package (#7886) 2022-09-23 11:45:48 -05:00
Martin Angers
db24cf32fa
Add new low_disk_space URL parameter to the GET /hosts endpoint (and GET /hosts/count) (#7853) 2022-09-21 15:16:31 -04:00
gillespi314
34688f531a
Refactor webhooks cron to new schedule package (#7840) 2022-09-20 14:26:36 -05:00
Juan Fernandez
f78dc8febd
Use directories to organize msrc assets (#7851)
Use directories to organize MSRC assets
2022-09-20 14:12:19 -04:00
Martin Angers
478b4d3f69
Validate team and appconfig payloads, with dry-run and force modes (#7731) 2022-09-19 13:53:44 -04:00
gillespi314
6a3d9959fc
Refactor vulnerabilities cron to scheduler package (#7650) 2022-09-16 10:08:51 -05:00
Juan Fernandez
765031d962
Bug 6717: Duplicate calls on vulnerability third party integrations. (#7698)
If the same vulnerability was present in more than one software, then duplicated third party integration calls were being issued.
2022-09-13 10:41:52 -04:00
Juan Fernandez
b60d535d4a
Feature 7084: Add new EE endpoint for Fleet Desktop (#7530)
Added new EE endpoint, that is meant to be used by Fleet Desktop only. The new endpoint will return the number of failed policies.
2022-09-12 15:37:38 -04:00
Juan Fernandez
98e4d6e0bc
Feature 7394: Use MSRC parser to generate security bulletin artifacts (#7491)
Generate security artifacts using the MSRC parser.
2022-09-12 15:17:13 -04:00
Juan Fernandez
b6d084deec
Feature 7559: Include 3rd party severity scores in the vulnerability web-hook payload (#7581)
Premium users should receive vulnerability scores as part of the web-hook payload.
2022-09-06 15:12:17 -04:00
Michal Nicpon
0709d1bc5c
improve vuln cpe matching on macos (#6985)
* add cpe translations
* fix matching on target_sw
2022-09-01 10:02:07 -06:00
gillespi314
8157359f84
Ingest Windows host display version for hosts table (#7403) 2022-08-30 10:16:48 -05:00
Roberto Dip
eeefe2fab9
enable controlled rollout of features by teams (#7408) 2022-08-30 08:13:09 -03:00
Martin Angers
f602ea3446
Provide Munki issues for MacOS hosts (#7280) 2022-08-29 14:40:16 -04:00
Roberto Dip
f9bb0b3dc0
adjust osquery_perf to respect config settings (#7418)
users, software_macos and software_linux queries can be optionally disabled in the fleet server via the enable_host_users and enable_software_inventory config settings, however osquery_perf is always running them.

this adjusts the code accordingly to only execute them if the server asks.
2022-08-29 13:34:40 -03:00
Juan Fernandez
0572c621d5
Fix osquery-perf Ubuntu software path issue (#7417)
Fixed issue with software fixture path used in osquery-perf
2022-08-26 15:43:06 -04:00
Juan Fernandez
3048a07fd1
Feature 7076: Ingest installed windows updates (#7138)
* Ingest installed Windows updates and store them in the windows_updates table. 
* Added config option for enabling/disabling Windows update ingestion and Windows OS vuln. detection.
2022-08-26 14:55:03 -04:00
Roberto Dip
9a1d2066bb
deprecate host_settings in favor of features (#7358)
Related to #7312, this makes use of the changes introduced in #7353 to rename host_settings to features while keeping backwards compatibility.
2022-08-25 13:41:50 -03:00
Martin Angers
1ed6512c5b
Implement retries when building MSI on Apple M1 architecture (#7367) 2022-08-24 14:52:32 -04:00
Martin Angers
dfd7d3f244
Add activity types for applied team spec and edited agent options (#7355) 2022-08-24 08:32:45 -04:00
Juan Fernandez
7e1e69c9bc
Fixed flaky vulnerability CRON tests (#7311)
* Fixed flaky vuln cron tests

* Fixed typo

* Remove sub-test
2022-08-23 08:22:08 -03:00
gillespi314
7fb109e6b3
Handle errors for duplicate os records (#7294) 2022-08-22 14:34:00 -05:00
Zachary Winnerman
2103de275a
improve error handling in vulnerabilities cron (#7276)
* improve error handling in vulnerabilities cron

* fix tests

* Use errHandler and go mod tidy

* Add dep that got removed by mod tidy

* add dsl to tools

* Add changes file

Co-authored-by: Michal Nicpon <michal@fleetdm.com>
Co-authored-by: Tomas Touceda <chiiph@gmail.com>
2022-08-18 16:53:58 -03:00
gillespi314
90f57f4849
Enhance API endpoints with host operating systems info (#7154) 2022-08-12 14:23:25 -05:00
Zach Wasserman
acf60189ff
Add message for fleetctl when prepare commands are used (#7152) 2022-08-11 12:26:52 -07:00
Martin Angers
c8cdddf0ea
Update /macadmins endpoints to include MDM name and aggregate count (#7137) 2022-08-10 15:15:01 -04:00
Roberto Dip
fc8c15c0d1
add application config setting to enable JIT provisioning (#7140)
As part of #7053, this adds a config setting to enable JIT provisioning.
2022-08-10 15:15:35 -03:00
gillespi314
e2194be61c
Add schedule package and refactor cron jobs for cleanups, aggregations, and usage statistics (#6618) 2022-08-10 11:00:56 -05:00
gillespi314
3a88afaf98
Add new database tables and queries for host operating systems (#6920) 2022-08-09 13:34:41 -05:00
Lucas Manuel Rodriguez
6dcff28be0
Move specs parsing functionality to a new pkg/spec package (#7050) 2022-08-05 19:07:32 -03:00
Martin Angers
012869f971
Fleet Premium: de-anonymize usage stats (#7013) 2022-08-03 14:44:34 -04:00
Martin Angers
c1d38598e2
Prevent removing team enroll secrets when applying team specs without new secrets (#6890) 2022-08-02 09:51:03 -04:00
Gabriel Hernandez
f61a17bd9b
add google analytics to sandbox instances (#6941)
* add google analytics to sandbox instances

* Add serverType variable to frontend handler

* update version of html-webpack-plugin

Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>
2022-08-01 11:27:12 +01:00
Roberto Dip
780db937de
pass a pointer to config instead of the whole config to goroutines (#6823) 2022-07-26 12:27:30 -03:00
Roberto Dip
d63f56f8c0
add support for notarization in fleetdm/fleetctl images (#6818)
#6674
2022-07-25 20:06:10 -03:00
gillespi314
cc34585a47
Add statistics for number of hosts that have not submitted results for distributed queries (#6495) 2022-07-20 20:54:10 -05:00
Michal Nicpon
d4be5ad2a1
add upgrade tests (#6596)
* add upgrade tests

* fix lint issues

go.mod

* remove req.cnf

* revert unrelated changes

* make version configurable in test

* fix golangci-lint ruleguard issue

Related to https://github.com/go-critic/go-critic/issues/1152
Need to have github.com/quasilyte/go-ruleguard/dsl

* fix lint issues

* fix

* clean up docker-compose.yml

* fix http request

* add readme

* fix lint issues

* address feedback

* fix

* add platform

* address feedback

* run go fmt
2022-07-19 15:11:51 -06:00
Eng Zer Jun
1ab171faf3
test: use T.Setenv to set env vars in tests (#6714)
This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.Setenv
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-07-18 14:22:28 -03:00
Roberto Dip
69f8f2a73b
add API endpoints to retrieve pre-built installers (#6672)
Rel: #6365, this adds a new endpoint to check and download pre-built installers.
2022-07-18 13:44:30 -03:00
Roberto Dip
86366b2ed8
add a new S3 datastore to retrieve pre-built packages (#6631)
Related to #6365, this extends the datastore/s3 package to retrieve installers from S3 according to the conventions listed in the parent issue. This also includes:

- A minor refactor to decouple Carves-related functionality from the core S3 functionality
- Set-up to run tests using minio (only enabled via the FILE_STORAGE_TEST env flag)
2022-07-14 14:14:24 -03:00
Roberto Dip
08cd206de7
only send vuln configs to goroutine (#6650)
In https://github.com/fleetdm/fleet/pull/6630 I added a new config for packaging, but when I started the server I got:

```
~/fleet $ ./build/fleet serve --dev --dev_license
level=info ts=2022-07-13T21:36:06.055998Z component=redis mode=standalone
fatal error: newproc: function arguments too large for new goroutine

runtime stack:
runtime.throw({0x103d85259, 0x37})
	/Users/roperzh/.gvm/gos/go1.17/src/runtime/panic.go:1198 +0x54
runtime.newproc1(0x104569a30, 0x14000ffda28, 0x7f8, 0x140000001a0, 0x103bc85a4)
	/Users/roperzh/.gvm/gos/go1.17/src/runtime/proc.go:4299 +0x4d0
runtime.newproc.func1()
	/Users/roperzh/.gvm/gos/go1.17/src/runtime/proc.go:4255 +0x4c
runtime.systemstack()
	/Users/roperzh/.gvm/gos/go1.17/src/runtime/asm_arm64.s:230 +0x6c

goroutine 1 [running]:
runtime.systemstack_switch()
	/Users/roperzh/.gvm/gos/go1.17/src/runtime/asm_arm64.s:187 +0x8 fp=0x14000ffd9c0 sp=0x14000ffd9b0 pc=0x102b60958
runtime.newproc(0x7f8, 0x104569a30)
	/Users/roperzh/.gvm/gos/go1.17/src/runtime/proc.go:4254 +0x54 fp=0x14000ffda10 sp=0x14000ffd9c0 pc=0x102b38034
main.runCrons({0x1045cd500, 0x14000262990}, {0x104611a38, 0x14000262930}, 0x14000298190, {0x1045a0480, 0x140009c9f20}, {{{0x103d1adc4, 0x3}, {0x103d2b188, ...}, ...}, ...}, ...)
	/Users/roperzh/fleet/cmd/fleet/serve.go:694 +0x2c4 fp=0x14000ffe260 sp=0x14000ffda10 pc=0x103bc85a4
main.createServeCmd.func1(0x1400027ca00, {0x140000bcb40, 0x0, 0x2})
```

With my local changes, `serve.go:694` is this line:

7559988000/cmd/fleet/serve.go (L685-L686)

After passing only a subset of the config the issue was solved.
2022-07-14 08:04:29 -03:00
Roberto Dip
6faee84f57
allow to configure a default global enroll secret (#6609)
Related to #6365 this adds a new config to set a global enroll token that will be used by the server.
2022-07-12 19:12:10 -03:00
Roberto Dip
f7dd8c86cd
implement a docker image to package orbit natively in Linux (#6504)
Related to #6364 and #6363, this:

- Adds a new Docker image, `fleetdm/fleetctl` equipped with all necessary dependencies to build Fleet-osquery binaries for all platforms
- Modifies the package generation logic to special case this scenario via an environment variable `FLEETCTL_NATIVE_TOOLING`
- Adds a new GitHub workflow to test this

There are more details in the README, but part of the special-casing logic is in place to output the binaries to a folder named `build` when they are run with `FLEETCTL_NATIVE_TOOLING`, this is so we can persist the binary generated by the docker container via a bind mount:

```bash
docker run -v "$(pwd):/build" fleetdm/fleetctl package --type=msi
```

To test this changeset, I have generated packages for all platforms, both via the new Docker image and via the classic `fleetctl package`.
2022-07-11 09:49:13 -03:00