Commit graph

83 commits

Author SHA1 Message Date
Roberto Dip
1ef19e4ff2 Merge remote-tracking branch 'origin/main' into feat-software-installers 2024-05-15 19:55:02 -03:00
Roberto Dip
d383876a3c
fix issues installing software in windows (#19048)
for #19039 and #19041 this:

- fixes the install/remove scripts to read the env variable the proper
way
- truncates output before storing in the databse in case its longer than
MySQL's TEXT size

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-05-15 19:39:42 -03:00
Roberto Dip
ad94dff814
installer report and rollback fixes (#19046)
for https://github.com/fleetdm/fleet/issues/19020

- Fixes the rollback logic to get the right script for the software
being installed
- Fixes the messages displayed in the install results

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-05-15 19:18:35 -03:00
Jahziel Villasana-Espinoza
01898fd176
fix: typos in scripts (#19045)
Feature cleanup

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
2024-05-15 16:36:31 -04:00
Martin Angers
1def5b2ddf
Add support for software installers in fleetctl gitops (#18990) 2024-05-14 16:58:58 -04:00
Dante Catalfamo
71c0026168
Orbit software installer flow (#18797)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).

---------

Co-authored-by: Roberto Dip <rroperzh@gmail.com>
2024-05-14 17:25:35 -03:00
Roberto Dip
3a31262353
add CLI and endpoints to set software via fleetctl apply (#18876)
for #18325

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
2024-05-14 15:06:33 -03:00
Tim Lee
ae00942d5c
Add backoff and retry to vuln downloads (#18855)
This adds a 3min max retry interval when downloading vulnerability
assets. The need was discovered initially due to frequent test failures
when downloading an unstable Ubuntu Kinetic OVAL feed.
2024-05-09 11:38:12 -06:00
Lucas Manuel Rodriguez
a1f77e01e5
Add support for escaping $ in gitops yamls (#18845)
#18467

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- ~[ ] If database migrations are included, checked table schema to
confirm autoupdate~
- ~For database migrations:~
- ~[ ] Checked schema for all modified table for columns that will
auto-update timestamps during migration.~
- ~[ ] Confirmed that updating the timestamps is acceptable, and will
not cause unwanted side effects.~
- ~[ ] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).~
- [X] Manual QA for all new/changed functionality
  - ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
2024-05-09 12:29:16 -03:00
Roberto Dip
e5283ef2a8 Merge remote-tracking branch 'origin/main' into feat-software-installers 2024-05-07 13:37:29 -03:00
Roberto Dip
37fe905f96
missing validations and tweaks to default scripts (#18780)
This adds two things:

- when implementing the CLI, I found [a
panel](https://www.figma.com/file/oQl2oQUG0iRkUy0YOxc307/%2314921-Deploy-security-agents-to-macOS%2C-Windows%2C-and-Linux-hosts?type=design&node-id=779-29335&mode=design&t=Y27cbj7DdhUEGJko-4)
in the Figma file with validations that I missed
- explicit shebang for bash scrips (requested by product) and removed a
comment that will be user facing for exe files.
2024-05-07 13:02:08 -03:00
Roberto Dip
bd3c0a1e9a
adjust logic to get default scripts (#18719)
This tweaks the logic to get default install/remove scripts to delegate
the variable replacement to `fleetd`
2024-05-06 11:41:31 -03:00
Victor Lyuboslavsky
4f4800be19
GitOps remove teams (#18640)
#16677 

Improvements to `fleetctl gitops` command:
- Added the ability to pass multiple files, like `fleetctl gitops -f
file1 -f file2`, where the first file must be the global configuration
- Added the ability to remove teams that were not specified in team
configs using the switch `--delete-other-teams`
- When passing a global config and team config during initial
configuration, the `org_settings.mdm.apple_bm_default_team` value can be
set to match the team that will be created by the provided team config.

After these changes are released to prod, we can update
https://github.com/fleetdm/fleet-gitops to use the new switches: #18692

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-05-03 08:03:00 -05:00
Roberto Dip
123fdc72b0
add endpoint to send software installer requests (#18711)
#18334

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-05-02 18:00:06 -03:00
Roberto Dip
92e540aee5
add scripts to add/remove software (#18649)
for:

- https://github.com/fleetdm/fleet/issues/18314
- https://github.com/fleetdm/fleet/issues/18315
- https://github.com/fleetdm/fleet/issues/18317
- https://github.com/fleetdm/fleet/issues/18316

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-05-01 14:15:59 -03:00
Martin Angers
7d014f9fad
Extract metadata from installers part 2 (#18608) 2024-04-30 11:20:53 -04:00
Gabriel Hernandez
7983e91933 Merge branch 'main' into feat-software-installers 2024-04-30 11:40:55 +01:00
Roberto Dip
05719633a1
add consistent MDM host lifecycle management (#18510)
The mantra for MDM lifecycle events is:

> - Noah: When MDM is turned on, install fleetd, bootstrap package (if
DEP),
> and profiles. Don't clear host vitals (everything you see on the Host
>      details page)
>    - Noah: On re-enrollment, don't clear host vitals.
>    - Noah: On lock and wipe, don't clear host vitals.
>    - Noah: On delete, clear host vitals.

This addresses issues:

- https://github.com/fleetdm/fleet/issues/17243
- https://github.com/fleetdm/fleet/issues/17481
- https://github.com/fleetdm/fleet/issues/17292
- https://github.com/fleetdm/fleet/issues/18030
- https://github.com/fleetdm/fleet/issues/18031
2024-04-29 16:43:15 -03:00
Martin Angers
563d55c218
Software installers: extract metadata from installers (part 1) (#18509) 2024-04-29 09:13:36 -04:00
Lucas Manuel Rodriguez
1b26129465
Delete expired activities in cleanup cron (#18441)
#16989

API draft PR: #17809.
UI draft PR: #18243.

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- ~[ ] If database migrations are included, checked table schema to
confirm autoupdate~
- ~For database migrations:~
- ~[ ] Checked schema for all modified table for columns that will
auto-update timestamps during migration.~
- ~[ ] Confirmed that updating the timestamps is acceptable, and will
not cause unwanted side effects.~
- ~[ ] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).~
- [X] Manual QA for all new/changed functionality
  - ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
2024-04-26 07:22:31 -03:00
Roberto Dip
e39f5bce3b
implement logic to sign apple profiles (#16490)
#10418
2024-04-18 18:01:37 -03:00
Roberto Dip
f0ad942a57
implement status reports for DDM commands (#17831)
for #17408
2024-03-26 10:40:35 -03:00
Sarah Gillespie
3d73174e90
Add macOS DDM protocol endpoints for tokens, declaration-items, and declaration/.../... (#17679) 2024-03-18 17:48:07 -05:00
Roberto Dip
f5cf156653
improve mdmtest package to handle any kind of command (#17673)
it delegates any extra unmarshaling to the caller. We might consider
building our own types in the future instead of relying on micromdm, but
these are used only for tests right now.
2024-03-15 17:10:48 -03:00
Sarah Gillespie
48b31a02ae
Add DDM service struct, basic handlers, and test client (#17671) 2024-03-15 15:20:15 -03:00
Lucas Manuel Rodriguez
e9a464e0cf
Add exponential backoff to orbit enroll retries (#17368)
#16594

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [X] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [X] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2024-03-13 07:57:00 -03:00
Victor Lyuboslavsky
02de6b5695
Adding full unicode/emoji support for team/policy names. (#17163)
#17027 
Added Unicode and emoji support for policy and team names.

I have the manual test steps in the issue:
https://github.com/fleetdm/fleet/issues/17027

# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-02-27 12:55:05 -06:00
Jahziel Villasana-Espinoza
272ce3187f
feat: move scep dependency inside the monorepo (#16988)
Following the pattern set up by Martin and Lucas for similar PRs. Can be
reviewed by commits:

1. move scep directory into monorepo
2. update import paths
3. update go.mod, go.sum
4. fix golint errors in scep package
5. skip a failing test that's been broken for a while
6. fix that failing test

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Manual QA for all new/changed functionality

Also verified that our test suite runs the `scep` tests.
2024-02-22 13:13:46 -05:00
Victor Lyuboslavsky
e4d5e27dd9
fleetctl gitops (#16535)
Add `fleetctl gitops` command for #13643 

Code review video:
https://www.loom.com/share/7941c51c709b44ccafd618dd05837d99?sid=27b923d7-1393-4396-bac7-30616b2d6de9

fleet-gitops PR that also needs review:
https://github.com/fleetdm/fleet-gitops/pull/26

Working global/team gitops configs that can be used for testing:
https://github.com/fleetdm/fleet-gitops/tree/victor/fixing-configs

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-02-09 13:34:57 -06:00
Roberto Dip
2d4c1b079a
improvements for troubleshooting workflow for MDM commands (#16654)
for #16416
2024-02-07 16:24:03 -03:00
Lucas Manuel Rodriguez
5360029d67
Allow custom osquery database on fleetd (#16554)
#16014

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
- [x] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2024-02-05 09:41:06 -03:00
Martin Angers
fc3304c902
Move nanomdm dependency in monorepo (#16015)
#15557 

Following the precedent that Lucas used for other similar PRs, the best
way to review is probably by commits.

* The first one simply copies over the files from the fork to the
monorepo
* Second one adjusts all import paths
* Third one tidies up the `go.mod` files
* Last one fixes the linter issues in the nanomdm package

# Checklist for submitter

- ~~Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.~~ (not a user-visible change)
- [x] Manual QA for all new/changed functionality (ran test suite,
re-generated mocks)

I also verified that our Go test suite did run the newly moved `nanomdm`
package steps:

```
ok  	github.com/fleetdm/fleet/v4/server/mdm/nanomdm/cryptoutil	0.003s	coverage: 0.0% of statements in github.com/fleetdm/fleet/v4/...
ok  	github.com/fleetdm/fleet/v4/server/mdm/nanomdm/mdm	0.005s	coverage: 46.2% of statements in github.com/fleetdm/fleet/v4/...
ok  	github.com/fleetdm/fleet/v4/server/mdm/nanomdm/service/certauth	1.320s	coverage: 20.7% of statements in github.com/fleetdm/fleet/v4/...
ok  	github.com/fleetdm/fleet/v4/server/mdm/nanomdm/storage/file	0.007s	coverage: 24.1% of statements in github.com/fleetdm/fleet/v4/...
```
2024-01-11 23:28:48 -03:00
Roberto Dip
3f302a79b4
fix log with next retry time once max retries are exceeded (#16026) 2024-01-10 17:53:30 -03:00
Roberto Dip
3b00b70786
fix unreleased bugs for the increased script timeout (#15897)
for https://github.com/fleetdm/fleet/issues/15196. The main problem was
that we have two timeouts:

1. The timeout used by the host to kill the script execution
2. The timeout used by the server to wait for the script results

Before the changes in https://github.com/fleetdm/fleet/pull/15779, the
server timeout was longer than the host timeout, but we inadvertently
set both values to 5 minutes, which breaks the logic we have to handle
both kinds of timeouts.
2024-01-04 10:26:13 -03:00
Roberto Dip
8957078d75
Add backoff functionality for fleetd updates (#15489)
related to #14176, `fleetd` will now retry 3 times and wait 24 hours to
try again for an specific update.
2023-12-08 19:43:56 -03:00
Roberto Dip
b07fbdc1d7
implement profile verification for windows (#15348)
for #14426.

In order to prevent import cycles and be able to use some type
definitions/constants I followed the same pattern we did for macOS by
creating a new package named `syncml`. This makes the changelog look
bigger than it actually is, so I split it into two commits to make it
easier to review:

-
[d7c233d](d7c233d54c)
moves the relevant bits to this new package
-
[7531a07](7531a0742b)
implements profile verification
2023-11-30 09:17:07 -03:00
Martin Angers
2f927df4f0
Implement Windows OS Updates (feature branch). (#15359) 2023-11-29 11:07:24 -05:00
Roberto Dip
5f313c8972
fix various bugs after testing windows MDM profiles (#15264)
for #13281
2023-11-22 10:48:28 -03:00
Roberto Dip
9cf30a9131
Feat windows msmdm (#14837)
for #13069

---------

Co-authored-by: Marcos Oviedo <marcos@fleetdm.com>
Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>
2023-11-01 11:13:12 -03:00
guangwu
33858d7301
chore: remove refs to deprecated io/ioutil (#14485)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- [ ] Documented any permissions changes (docs/Using
Fleet/manage-access.md)
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [ ] Added/updated tests
- [ ] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).

Signed-off-by: guoguangwu <guoguangwu@magic-shield.com>
2023-10-27 15:28:54 -03:00
Roberto Dip
9ffa11c25d
Feat: saved scripts (#14409)
For #9537
2023-10-10 19:00:45 -03:00
Marcos Oviedo
f0d77ab3db
Merging Bitlocker feature branch (#14350)
This relates to #12577

---------

Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>
Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2023-10-06 19:04:33 -03:00
gillespi314
e08bb000c9
Update nanomdm dependency (#12721)
Updates include:
- Fix issues where `GetBootstrapToken` returned `500` instead of no data
and no error per Apple MDM
[documentation](https://developer.apple.com/documentation/devicemanagement/get_bootstrap_token)
- Incorporate additional updates from the upstream nanomdm repo
2023-08-21 11:07:57 -03:00
Roberto Dip
902e064d04
fix issues with migration flow (#13297)
For #13094
2023-08-14 09:56:59 -03:00
Martin Angers
e323a3d881
Consider an empty EULA pdf file the same as an invalid one, returning 400 Bad Request (#12542) 2023-06-28 08:19:42 -04:00
Lucas Manuel Rodriguez
bb3b21b574
Add TestMDMClient to simulate MDM clients in osquery-perf (#11672)
#11528

osquery-perf simulated hosts enroll and are identified as manually
enrolled. (Enrolling as DEP requires more work, e.g. a new mocked Apple
DEP endpoint).

Given that these are simulated MDM clients, they cannot be woken up with
push notifications. Instead, these check for new commands to execute
every 10 seconds (which is not realistic, but could serve as a good
loadtesting exercise).

I will now start setting up the loadtest environment with MDM enabled
and configured to test this.

- ~[ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.~
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
  - ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
2023-05-12 13:50:20 -03:00
Roberto Dip
653bbec5f1
cli: try to infer the bootstrap package name from the URL too (#11571)
#11570
2023-05-11 10:36:28 -03:00
Roberto Dip
11356b2f15
add CRUD for EULA (#11274)
https://github.com/fleetdm/fleet/issues/10741
2023-05-02 10:09:33 -03:00
Lucas Manuel Rodriguez
7dadec3ecf
Add mTLS support to fleetd (#11319)
#7970

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [x] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2023-04-27 08:44:39 -03:00
Martin Angers
582e85c876
Add support for the mdm.macos_setup.macos_setup_assistant key in fleetctl, API (#11296) 2023-04-25 09:36:01 -04:00