Commit graph

93 commits

Author SHA1 Message Date
Roberto Dip
7b20060350
cover edge case for alternative zoom installer (#20286)
Zoom offers two installers:

- Zoom for IT admins (already covered previously)
- "Regular" Zoom (covered here)

This tweaks the logic made as part of #19144 to ensure we cover both

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-10 08:05:09 -03:00
Lucas Manuel Rodriguez
2875a9dbb8
Fixes to fleetctl debug connection and TLS certs documentation (#20166)
#6085

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-09 14:04:23 -03:00
Roberto Dip
196f761b6d
use the latest fleetd-base version for MDM commands (#20078)
for #19176

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-06-28 16:35:07 -03:00
Roberto Dip
8f7e2f5214
improve extraction of name and version, and add bundle identifier for pkg installers (#19838)
part 1 of #19144
2024-06-25 10:55:24 -03:00
Roberto Dip
1cc13a09fb
🧹 friday cleanup party: substitute deprecated import of go-kit (#19774)
`go-kit/kit/log` was deprecated and generating warnings

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Manual QA for all new/changed functionality
2024-06-17 10:27:31 -03:00
Lucas Manuel Rodriguez
904e8a6825
Added server_settings.query_report_cap (#19692)
#19600

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
2024-06-14 12:24:01 -03:00
Victor Lyuboslavsky
05eb338561
Enable gitops to create teams with no enroll secrets, or clear enroll secrets for an existing team (#19616)
Enable gitops to create teams with no enroll secrets, or clear enroll
secrets for an existing team
#19332 

`fleetctl apply` also gains this extra functionality. In `fleetctl
apply` secrets will not be change if one of the following:
- secrets is missing from yml
- They are blank in yml, like: `secrets:`
- They are null in yml, like: `secrets: null`

They will only be cleared with `fleetctl apply` if the user explicitly
sets them to an empty array, like:
- `secrets: []`

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-06-11 13:20:32 -05:00
Lucas Manuel Rodriguez
3dbdbc1bcf
Add support for iOS/iPadOS to osquery-perf (#19522)
#18119

- [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.

Sample on how to simulate 50 iPads and 50 iPhones:
```sh
go run ./cmd/osquery-perf -host_count 100 -os_templates iphone_14.6.tmpl:50,ipad_13.18.tmpl:50 -mdm_scep_challenge <...>
```
2024-06-10 17:02:35 -03:00
Lucas Manuel Rodriguez
f8cee672e0
Support environment variables in config profiles (#18891)
#17309

I added some missing env var replacement tests for policies, queries,
etc.

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
2024-05-28 13:44:43 -03:00
Tim Lee
6f008b40f2
Fix backoff in vuln stream logging (#19148) 2024-05-28 08:13:46 -06:00
Roberto Dip
1ef19e4ff2 Merge remote-tracking branch 'origin/main' into feat-software-installers 2024-05-15 19:55:02 -03:00
Roberto Dip
d383876a3c
fix issues installing software in windows (#19048)
for #19039 and #19041 this:

- fixes the install/remove scripts to read the env variable the proper
way
- truncates output before storing in the databse in case its longer than
MySQL's TEXT size

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-05-15 19:39:42 -03:00
Roberto Dip
ad94dff814
installer report and rollback fixes (#19046)
for https://github.com/fleetdm/fleet/issues/19020

- Fixes the rollback logic to get the right script for the software
being installed
- Fixes the messages displayed in the install results

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-05-15 19:18:35 -03:00
Jahziel Villasana-Espinoza
01898fd176
fix: typos in scripts (#19045)
Feature cleanup

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
2024-05-15 16:36:31 -04:00
Martin Angers
1def5b2ddf
Add support for software installers in fleetctl gitops (#18990) 2024-05-14 16:58:58 -04:00
Dante Catalfamo
71c0026168
Orbit software installer flow (#18797)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).

---------

Co-authored-by: Roberto Dip <rroperzh@gmail.com>
2024-05-14 17:25:35 -03:00
Roberto Dip
3a31262353
add CLI and endpoints to set software via fleetctl apply (#18876)
for #18325

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
2024-05-14 15:06:33 -03:00
Tim Lee
ae00942d5c
Add backoff and retry to vuln downloads (#18855)
This adds a 3min max retry interval when downloading vulnerability
assets. The need was discovered initially due to frequent test failures
when downloading an unstable Ubuntu Kinetic OVAL feed.
2024-05-09 11:38:12 -06:00
Lucas Manuel Rodriguez
a1f77e01e5
Add support for escaping $ in gitops yamls (#18845)
#18467

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- ~[ ] If database migrations are included, checked table schema to
confirm autoupdate~
- ~For database migrations:~
- ~[ ] Checked schema for all modified table for columns that will
auto-update timestamps during migration.~
- ~[ ] Confirmed that updating the timestamps is acceptable, and will
not cause unwanted side effects.~
- ~[ ] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).~
- [X] Manual QA for all new/changed functionality
  - ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
2024-05-09 12:29:16 -03:00
Roberto Dip
e5283ef2a8 Merge remote-tracking branch 'origin/main' into feat-software-installers 2024-05-07 13:37:29 -03:00
Roberto Dip
37fe905f96
missing validations and tweaks to default scripts (#18780)
This adds two things:

- when implementing the CLI, I found [a
panel](https://www.figma.com/file/oQl2oQUG0iRkUy0YOxc307/%2314921-Deploy-security-agents-to-macOS%2C-Windows%2C-and-Linux-hosts?type=design&node-id=779-29335&mode=design&t=Y27cbj7DdhUEGJko-4)
in the Figma file with validations that I missed
- explicit shebang for bash scrips (requested by product) and removed a
comment that will be user facing for exe files.
2024-05-07 13:02:08 -03:00
Roberto Dip
bd3c0a1e9a
adjust logic to get default scripts (#18719)
This tweaks the logic to get default install/remove scripts to delegate
the variable replacement to `fleetd`
2024-05-06 11:41:31 -03:00
Victor Lyuboslavsky
4f4800be19
GitOps remove teams (#18640)
#16677 

Improvements to `fleetctl gitops` command:
- Added the ability to pass multiple files, like `fleetctl gitops -f
file1 -f file2`, where the first file must be the global configuration
- Added the ability to remove teams that were not specified in team
configs using the switch `--delete-other-teams`
- When passing a global config and team config during initial
configuration, the `org_settings.mdm.apple_bm_default_team` value can be
set to match the team that will be created by the provided team config.

After these changes are released to prod, we can update
https://github.com/fleetdm/fleet-gitops to use the new switches: #18692

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-05-03 08:03:00 -05:00
Roberto Dip
123fdc72b0
add endpoint to send software installer requests (#18711)
#18334

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-05-02 18:00:06 -03:00
Roberto Dip
92e540aee5
add scripts to add/remove software (#18649)
for:

- https://github.com/fleetdm/fleet/issues/18314
- https://github.com/fleetdm/fleet/issues/18315
- https://github.com/fleetdm/fleet/issues/18317
- https://github.com/fleetdm/fleet/issues/18316

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-05-01 14:15:59 -03:00
Martin Angers
7d014f9fad
Extract metadata from installers part 2 (#18608) 2024-04-30 11:20:53 -04:00
Gabriel Hernandez
7983e91933 Merge branch 'main' into feat-software-installers 2024-04-30 11:40:55 +01:00
Roberto Dip
05719633a1
add consistent MDM host lifecycle management (#18510)
The mantra for MDM lifecycle events is:

> - Noah: When MDM is turned on, install fleetd, bootstrap package (if
DEP),
> and profiles. Don't clear host vitals (everything you see on the Host
>      details page)
>    - Noah: On re-enrollment, don't clear host vitals.
>    - Noah: On lock and wipe, don't clear host vitals.
>    - Noah: On delete, clear host vitals.

This addresses issues:

- https://github.com/fleetdm/fleet/issues/17243
- https://github.com/fleetdm/fleet/issues/17481
- https://github.com/fleetdm/fleet/issues/17292
- https://github.com/fleetdm/fleet/issues/18030
- https://github.com/fleetdm/fleet/issues/18031
2024-04-29 16:43:15 -03:00
Martin Angers
563d55c218
Software installers: extract metadata from installers (part 1) (#18509) 2024-04-29 09:13:36 -04:00
Lucas Manuel Rodriguez
1b26129465
Delete expired activities in cleanup cron (#18441)
#16989

API draft PR: #17809.
UI draft PR: #18243.

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- ~[ ] If database migrations are included, checked table schema to
confirm autoupdate~
- ~For database migrations:~
- ~[ ] Checked schema for all modified table for columns that will
auto-update timestamps during migration.~
- ~[ ] Confirmed that updating the timestamps is acceptable, and will
not cause unwanted side effects.~
- ~[ ] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).~
- [X] Manual QA for all new/changed functionality
  - ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
2024-04-26 07:22:31 -03:00
Roberto Dip
e39f5bce3b
implement logic to sign apple profiles (#16490)
#10418
2024-04-18 18:01:37 -03:00
Roberto Dip
f0ad942a57
implement status reports for DDM commands (#17831)
for #17408
2024-03-26 10:40:35 -03:00
Sarah Gillespie
3d73174e90
Add macOS DDM protocol endpoints for tokens, declaration-items, and declaration/.../... (#17679) 2024-03-18 17:48:07 -05:00
Roberto Dip
f5cf156653
improve mdmtest package to handle any kind of command (#17673)
it delegates any extra unmarshaling to the caller. We might consider
building our own types in the future instead of relying on micromdm, but
these are used only for tests right now.
2024-03-15 17:10:48 -03:00
Sarah Gillespie
48b31a02ae
Add DDM service struct, basic handlers, and test client (#17671) 2024-03-15 15:20:15 -03:00
Lucas Manuel Rodriguez
e9a464e0cf
Add exponential backoff to orbit enroll retries (#17368)
#16594

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [X] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [X] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2024-03-13 07:57:00 -03:00
Victor Lyuboslavsky
02de6b5695
Adding full unicode/emoji support for team/policy names. (#17163)
#17027 
Added Unicode and emoji support for policy and team names.

I have the manual test steps in the issue:
https://github.com/fleetdm/fleet/issues/17027

# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-02-27 12:55:05 -06:00
Jahziel Villasana-Espinoza
272ce3187f
feat: move scep dependency inside the monorepo (#16988)
Following the pattern set up by Martin and Lucas for similar PRs. Can be
reviewed by commits:

1. move scep directory into monorepo
2. update import paths
3. update go.mod, go.sum
4. fix golint errors in scep package
5. skip a failing test that's been broken for a while
6. fix that failing test

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Manual QA for all new/changed functionality

Also verified that our test suite runs the `scep` tests.
2024-02-22 13:13:46 -05:00
Victor Lyuboslavsky
e4d5e27dd9
fleetctl gitops (#16535)
Add `fleetctl gitops` command for #13643 

Code review video:
https://www.loom.com/share/7941c51c709b44ccafd618dd05837d99?sid=27b923d7-1393-4396-bac7-30616b2d6de9

fleet-gitops PR that also needs review:
https://github.com/fleetdm/fleet-gitops/pull/26

Working global/team gitops configs that can be used for testing:
https://github.com/fleetdm/fleet-gitops/tree/victor/fixing-configs

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-02-09 13:34:57 -06:00
Roberto Dip
2d4c1b079a
improvements for troubleshooting workflow for MDM commands (#16654)
for #16416
2024-02-07 16:24:03 -03:00
Lucas Manuel Rodriguez
5360029d67
Allow custom osquery database on fleetd (#16554)
#16014

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
- [x] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2024-02-05 09:41:06 -03:00
Martin Angers
fc3304c902
Move nanomdm dependency in monorepo (#16015)
#15557 

Following the precedent that Lucas used for other similar PRs, the best
way to review is probably by commits.

* The first one simply copies over the files from the fork to the
monorepo
* Second one adjusts all import paths
* Third one tidies up the `go.mod` files
* Last one fixes the linter issues in the nanomdm package

# Checklist for submitter

- ~~Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.~~ (not a user-visible change)
- [x] Manual QA for all new/changed functionality (ran test suite,
re-generated mocks)

I also verified that our Go test suite did run the newly moved `nanomdm`
package steps:

```
ok  	github.com/fleetdm/fleet/v4/server/mdm/nanomdm/cryptoutil	0.003s	coverage: 0.0% of statements in github.com/fleetdm/fleet/v4/...
ok  	github.com/fleetdm/fleet/v4/server/mdm/nanomdm/mdm	0.005s	coverage: 46.2% of statements in github.com/fleetdm/fleet/v4/...
ok  	github.com/fleetdm/fleet/v4/server/mdm/nanomdm/service/certauth	1.320s	coverage: 20.7% of statements in github.com/fleetdm/fleet/v4/...
ok  	github.com/fleetdm/fleet/v4/server/mdm/nanomdm/storage/file	0.007s	coverage: 24.1% of statements in github.com/fleetdm/fleet/v4/...
```
2024-01-11 23:28:48 -03:00
Roberto Dip
3f302a79b4
fix log with next retry time once max retries are exceeded (#16026) 2024-01-10 17:53:30 -03:00
Roberto Dip
3b00b70786
fix unreleased bugs for the increased script timeout (#15897)
for https://github.com/fleetdm/fleet/issues/15196. The main problem was
that we have two timeouts:

1. The timeout used by the host to kill the script execution
2. The timeout used by the server to wait for the script results

Before the changes in https://github.com/fleetdm/fleet/pull/15779, the
server timeout was longer than the host timeout, but we inadvertently
set both values to 5 minutes, which breaks the logic we have to handle
both kinds of timeouts.
2024-01-04 10:26:13 -03:00
Roberto Dip
8957078d75
Add backoff functionality for fleetd updates (#15489)
related to #14176, `fleetd` will now retry 3 times and wait 24 hours to
try again for an specific update.
2023-12-08 19:43:56 -03:00
Roberto Dip
b07fbdc1d7
implement profile verification for windows (#15348)
for #14426.

In order to prevent import cycles and be able to use some type
definitions/constants I followed the same pattern we did for macOS by
creating a new package named `syncml`. This makes the changelog look
bigger than it actually is, so I split it into two commits to make it
easier to review:

-
[d7c233d](d7c233d54c)
moves the relevant bits to this new package
-
[7531a07](7531a0742b)
implements profile verification
2023-11-30 09:17:07 -03:00
Martin Angers
2f927df4f0
Implement Windows OS Updates (feature branch). (#15359) 2023-11-29 11:07:24 -05:00
Roberto Dip
5f313c8972
fix various bugs after testing windows MDM profiles (#15264)
for #13281
2023-11-22 10:48:28 -03:00
Roberto Dip
9cf30a9131
Feat windows msmdm (#14837)
for #13069

---------

Co-authored-by: Marcos Oviedo <marcos@fleetdm.com>
Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>
2023-11-01 11:13:12 -03:00
guangwu
33858d7301
chore: remove refs to deprecated io/ioutil (#14485)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- [ ] Documented any permissions changes (docs/Using
Fleet/manage-access.md)
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [ ] Added/updated tests
- [ ] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).

Signed-off-by: guoguangwu <guoguangwu@magic-shield.com>
2023-10-27 15:28:54 -03:00