Commit graph

348 commits

Author SHA1 Message Date
Jahziel Villasana-Espinoza
abd01f2428
enable specifying installers by SHA256 in gitops (#28349)
> For #23497

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-04-18 16:41:41 -04:00
Ian Littman
0f2f085812
Add more test coverage for Fleet-maintained apps (#27722)
For #27377.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Added/updated automated tests
2025-04-03 09:18:27 -05:00
Ian Littman
0293d99800
Remove default EXE install/uninstall scripts, require entering install/uninstall scripts on EXE upload (#27268)
For #27267.

Below is what's shown immediately after selecting an EXE:

<img width="1254" alt="image"
src="https://github.com/user-attachments/assets/a28d8565-de88-448a-bdbc-92aefc34ad55"
/>


TODO:

* Tests
* GitOps requirements changes
* Disabling add button/adding errors when required scripts aren't
specified

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Luke Heath <luke@fleetdm.com>
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
Co-authored-by: RachelElysia <rachel@fleetdm.com>
2025-03-31 13:52:06 -05:00
Scott Gress
59f96651b6
Update to Go 1.24.1 (#27506)
For #26713 

# Details

This PR updates Fleet and its related tools and binaries to use Go
version 1.24.1.

Scanning through the changelog, I didn't see anything relevant to Fleet
that requires action. The only possible breaking change I spotted was:

> As [announced](https://tip.golang.org/doc/go1.23#linux) in the Go 1.23
release notes, Go 1.24 requires Linux kernel version 3.2 or later.

Linux kernel 3.2 was released in January of 2012, so I think we can
commit to dropping support for earlier kernel versions.

The new [tools directive](https://tip.golang.org/doc/go1.24#tools) is
interesting as it means we can move away from using `tools.go` files,
but it's not a required update.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [X] Make sure fleetd is compatible with the latest released version of
Fleet
   - [x] Orbit runs on macOS  , Linux   and Windows. 
- [x] Manual QA must be performed in the three main OSs, macOS ,
Windows and Linux .
2025-03-31 11:14:09 -05:00
Sarah Gillespie
8d17956f7b
Skip bootstrap package and other setup items when renewing Apple MDM enrollment profiles (#27560) 2025-03-28 16:33:22 -05:00
Ian Littman
4a1e5340f0
Switch Fleet-maintained apps to use manifest-based structure (#27201)
For #26082.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-03-20 21:21:56 -05:00
Scott Gress
2a18f56655
Manage labels in GitOps (#27038)
For #24473 

# Checklist for submitter

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.

## Details

This PR adds the ability to manage labels via GitOps. Usage is as
follows:

* If a top-level `labels:` key is provided in the global YAML file
provided to GitOps, then any labels in this list will be created (if
using a new name) or updated (if using an existing name).
* If no top-level `labels:` key is provided, no changes will be made to
labels. This allows backwards-compatibility; customers won't blow away
all of their labels if they don't immediately use `labels:` in their
YAML

Additionally, some new validation has been added so that label usage is
checked prior to application. This means that when the gitops command is
run, it will verify that any labels referenced elsewhere in the YAML
(e.g. by software installers or mdm profiles) exist, and will bail with
an error message if they don't.

## Testing

**Test label deletion**
1. Add some labels via the UI
2. Run `fleetctl gitops --dry-run` with a default.yml file _without_
`labels:` in it, and verify that it doesn't say it will update or delete
any labels
2. Run `fleetctl gitops` with a default.yml file _without_ `labels:` in
it, and verify that it doesn't modify or remove your labels
4. Run `fleetctl gitops --dry-run` with a default.yml file with
`labels:` in it and nothing underneath, and verify that it says that it
will delete your labels
4. Run `fleetctl gitops` with a default.yml file with `labels:` in it
and nothing underneath, and verify that it removes all your labels

**Test label create/update**
1. Add a label "foo" via the UI
2. Run `fleetctl gitops --dry-run` with a default.yml file with two
`labels:` in it, one named "foo" and one named "bar". Verify that the
output says that one label will be created and one will be updated.
2. Run `fleetctl gitops` with a default.yml file with two `labels:` in
it, one named "foo" and one named "bar". Verify that the two labels now
exist in the UI with the configuration you specified.

**Test label usage**
1. Add a label "foo" in the UI.
1. Run `fleetctl gitops --dry-run` with a default.yml file _without_
`labels:` in it, where a software installer or mdm profile uses the
"foo" label via `labels_include_any`. Verify that the output doesn't
complain about unknown labels.
1. Run `fleetctl gitops --dry-run` with a default.yml file _with_
`labels:` in it with nothing underneath, and a software installer or mdm
profile uses the "foo" label via `labels_include_any`. Verify that the
output complains about unknown label "foo"
1. Run `fleetctl gitops --dry-run` with a default.yml file _with_
`labels:` in it with a "foo" label defined underneath, and a software
installer or mdm profile uses the "foo" label via `labels_include_any`.
Verify that the output doesn't complain about unknown labels.
2025-03-19 16:35:11 -05:00
Victor Lyuboslavsky
6b7d232522
Additional CA validation (#27169)
For #26623

- Updated `github.com/groob/plist` to `github.com/micromdm/plist` -- it
was renamed
- Added validation that restricts DigiCert Fleet variables to
`com.apple.security.pkcs12` payloads plus additional restrictions
- Added validation that restricts Custom SCEP Fleet variables to
`com.apple.security.scep` payloads plus additional restrictions
- Enabled multiple CAs (Fleet variables) to be present in an Apple MDM
profile. But each CA can only be used once. For example, we can have
DigiCert CA and Custom SCEP CA in one Apple profile.

# Checklist for submitter
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-03-19 08:27:55 -05:00
gillespi314
c5386b1290 Merge branch 'main' into feat-23235-host-certificates 2025-02-26 12:43:19 -06:00
Dante Catalfamo
a1e752341b
Only allow once instance of fleet desktop at once (#25821)
#25396

---------

Co-authored-by: Lucas Manuel Rodriguez <lucas@fleetdm.com>
2025-02-25 16:46:06 -05:00
Martin Angers
0adf67e538 Fix conflicts 2025-02-25 14:39:35 -05:00
Ian Littman
4abbb5e06e
Add automated test coverage for VPP, FMA auto-install (#26571)
For #26190. Code is already merged, and tests didn't find any issues.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Added/updated automated tests
2025-02-24 15:56:49 -06:00
Martin Angers
8477856886 FIx conflicts 2025-02-24 14:28:34 -05:00
Ian Littman
ce36352fcd
Allow automatic creation of software install policy for VPP and FMA apps in API (#26440)
For #26190. FMA is included here because the previous implementation was
client-side. QA'd manually. Follow-up PR soon with automated test coverage.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-02-21 18:08:48 -06:00
Sarah Gillespie
adf09f099d
Add MDM CertificateList command for host certificates feature (#26479) 2025-02-20 16:49:41 -06:00
Victor Lyuboslavsky
2eb5119efb
Clear bootstrap package and enrollment profile with GitOps (#26095)
For #25648 

Fixed issue where `fleetctl gitops` was NOT deleting macOS setup
experience bootstrap package and enrollment profile. GitOps should clear
all settings that are not explicitly set in YAML config files.

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-02-07 14:35:51 -06:00
Victor Lyuboslavsky
55423f67e2
Fixed parsing of relative paths for MDM profiles in gitops no-team.yml (#26046)
For #25770 

We already unmarshal macOS/Windows settings (added by Martin), so we
replace the path with an absolute file path and keep them unmarshalled
so they don't have to be re-unmarshalled later. Note: the custom
UnmarshalJSON method on these structs checks for (and handles) legacy
format (before labels were added).

Also some refactorings:
- extracted `extractControlsForNoTeam`
- reorganized `TestGitOpsBasicGlobalAndNoTeam` with subtests -- I did
not actually change functionality of this test

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-02-06 16:39:15 -06:00
Lucas Manuel Rodriguez
c81117b9b2
Add Fleet Desktop support for Wayland display sessions (#25998)
For #19043.

See the versions and distributions tested during development on the QA
notes of #19043.

---

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [X] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [X] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [X] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2025-02-05 14:00:13 -03:00
Tim Lee
d38d180357
Add labels and editing for VPP apps (#25979)
For #24609

---------

Co-authored-by: Jahziel Villasana-Espinoza <jahziel@fleetdm.com>
Co-authored-by: Jahziel Villasana-Espinoza <jahzielv@gmail.com>
2025-02-03 11:16:21 -06:00
Victor Lyuboslavsky
a7b5aee6c2
Allow Windows SessionID=0 (#25582)
For #25581 

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
2025-01-20 09:12:33 -06:00
Ian Littman
4f0a2e2af9
Add VPP install automation in GitOps (#25400)
For #23531.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-01-14 12:52:39 -06:00
Jahziel Villasana-Espinoza
cf3a3cfbd2
fix: use a new strategy for finding the app name in case the title is wrong (#25297)
> For #24873

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Ian Littman <iansltx@gmail.com>
2025-01-10 16:42:06 -05:00
Victor Lyuboslavsky
cbe44eefe9
Refactoring crypto code for future reuse. (#25148)
Refactoring crypto code for future reuse for #24869. No functional
changes.
2025-01-07 16:14:12 -06:00
Ian Littman
6014908ab6
Correct fix for long MSI interned strings (#25104)
h/t https://github.com/binref/refinery/issues/72, for #24720. No changes
file as this is an unreleased bug.

Also added output for version in the custom package parser tool.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
2025-01-02 14:43:36 -06:00
Ian Littman
5beeb248f7
Handle long interned strings in MSI parsing (#25079)
For #24720. Used
https://github.com/ChaelChu/msi-props-reader/blob/master/src/msiPropsReader.ts
as inspiration. Not sure why the shift is 17 bits rather than 16 here
but confirmed that 17 works and 16 doesn't.

Tested against both existing GDrive MSIs for regression testing, plus
the one mentioned in the ticket.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
2025-01-02 10:41:08 -06:00
Ian Littman
5a30b477c6
Fall back to FileVersion when an EXE installer has FileVersion but not ProductVersion (#25070)
For #23541

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
2024-12-31 14:28:15 -06:00
Lucas Manuel Rodriguez
963cc7e22c
Automatic install custom packages (#25021)
#24385

Some docs change here: https://github.com/fleetdm/fleet/pull/25026.

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
2024-12-27 15:10:28 -03:00
gillespi314
c78002747f Merge branch 'main' into feat-labels-scoped-software 2024-12-20 17:06:48 -06:00
Dante Catalfamo
effd3563c8
Add secrets software script support (#24912)
#24899
2024-12-20 17:17:18 -05:00
Ian Littman
1f3971701f
Bump max length for installer URLs supplied in GitOps to 4000 characters (#24942)
For #24917. Should be worth the extra byte per row for the varchar
field.

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
2024-12-20 11:58:21 -06:00
Gabriel Hernandez
9057bf62a3 Merge branch 'main' into feat-labels-scoped-software 2024-12-18 15:36:20 -06:00
Dante Catalfamo
c9bdae8fb3
Embedded secrets validation (#24624)
#24549

Co-authored-by: Victor Lyuboslavsky <victor.lyuboslavsky@gmail.com>
2024-12-17 17:14:12 -05:00
Martin Angers
79ac8fa4a1
SSVL: implement gitops support for labels include/exclude on software packages (#24663) 2024-12-17 14:28:17 -05:00
Victor Lyuboslavsky
48e3654d75
Adding secret support to profiles via gitops. (#24675)
#24547

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-12-13 15:41:23 -06:00
Victor Lyuboslavsky
ab172f07af
Adding secret support to scripts via gitops. (#24649)
#24547 

# Checklist for submitter

- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-12-11 15:05:48 -06:00
Ian Littman
746e800fcf
Error out while parsing when script entries in a GitOps YAML file are missing paths (#24639)
For #22244. Previously empty script entries would get parsed and then
cause a panic later in the process.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-12-11 07:54:25 -06:00
Victor Lyuboslavsky
afebfde63c
Improvements for select next Apple MDM command query. (#24128)
#23832 

[Loadtest
report](https://docs.google.com/document/d/1HafECokrZ3jnzRskxMtJwp4k1E2uBTbO9vfKEUtyykI/edit?tab=t.0)

# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-12-05 12:02:48 -06:00
Martin Angers
c4404d9d68
Windows MDM Migration: API, CLI and activities (#24141) 2024-11-26 11:52:56 -05:00
Martin Angers
f0e1dccc8a
Bugfix: use an HTTP client that supports proxies for APNS push notifications (#23988) 2024-11-25 09:45:38 -05:00
Victor Lyuboslavsky
687ce3a71a
Fixed parsing Opera PE self-extracting archive. (#23751)
#23540 

Tested installers from Google drive:
https://drive.google.com/drive/folders/1MZVwgZu5Af7JnSnKCFknRPGDXM29mOa6

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com>
2024-11-18 16:09:29 -06:00
Martin Angers
c3ebce9293
Improve memory usage of software installers parsing (#23596) 2024-11-12 09:28:08 -05:00
Dante Catalfamo
ddbf6f1f29
Add support for zstd deb packages (#23657)
#22891
2024-11-11 14:13:01 -05:00
Ian Littman
cfead6a35f
Fix path resolution for installer queries and scripts to always be relative to where the query file or script is referenced (#23502)
#22187

Similar fix to #22555: resolve paths at spec parsing time rather than
when trying to grab files

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-11-07 11:22:08 -06:00
Victor Lyuboslavsky
06a4c86b3b
Allow uploading PKG without Distribution.xml (#23590)
#23213
Use PackageInfo.xml if Distribution.xml does not exist in uploaded macOS
PKG.

This means we now support script-only packages:
<div>
<a href="https://www.loom.com/share/fb2f9fe93cb64f3aa1221f974ca0eb3a">
<p>[Demo] Install script-only macOS package on Fleet (#23213) - Watch
Video</p>
    </a>
<a href="https://www.loom.com/share/fb2f9fe93cb64f3aa1221f974ca0eb3a">
<img style="max-width:300px;"
src="https://cdn.loom.com/sessions/thumbnails/fb2f9fe93cb64f3aa1221f974ca0eb3a-4b035241497a6c22-full-play.gif">
    </a>
  </div>

# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-11-06 14:38:13 -06:00
Victor Lyuboslavsky
3cf99f8039
Enable staticcheck Go linter. (#23487)
#23486 

Linter: https://staticcheck.dev/
2024-11-05 11:16:24 -06:00
Lucas Manuel Rodriguez
6fc0577704
Fix flaky timing test (#23333)
![Screenshot 2024-10-29 at 11 54
54 AM](https://github.com/user-attachments/assets/e90926bd-8bd8-4739-ad50-602fad029898)

#22740
2024-10-29 14:13:17 -03:00
Martin Angers
37584adcf3
SE: add CLI automated tests for fleetctl gitops (#23125) 2024-10-23 16:57:56 -04:00
Martin Angers
d1c3b5b28e
SE: CLI setup experience changes (#22956) 2024-10-23 14:51:02 -04:00
Gabriel Hernandez
d5689dd0fe Merge branch 'main' into feat-setup-experience 2024-10-21 12:36:50 +01:00
Victor Lyuboslavsky
f85b6f776f
Updating golangci-lint to 1.61.0 (#22973) 2024-10-18 12:38:26 -05:00
Victor Lyuboslavsky
383d03cd6d
GitOps support for NDES SCEP proxy config. (#22918)
#22124

# Demo
<div>
<a href="https://www.loom.com/share/ad30efda816e4bf3abca0eab92b6e928">
      <p>[Demo] NDES SCEP proxy gitops #21955 - Watch Video</p>
    </a>
<a href="https://www.loom.com/share/ad30efda816e4bf3abca0eab92b6e928">
<img style="max-width:300px;"
src="https://cdn.loom.com/sessions/thumbnails/ad30efda816e4bf3abca0eab92b6e928-33b1f073da294133-full-play.gif">
    </a>
  </div>

# Checklist for submitter
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-10-16 11:12:48 -05:00
Dante Catalfamo
6af7c23a5b
Start/Enqueue setup experience (#22664)
#22379
2024-10-10 11:46:23 -04:00
Ian Littman
0a8a396643
Ensure scripts set in no-team.yml can be used in run-script actions for No Team (#22809)
For #22787

Also revises the spec check to explain that scripts have to be defined
"controls" when used in policies for the same team, with an explicit
call-out for no-team.yml since this fix doesn't support pulling scripts
from the global file. This is because parsing and script-matching
happens early enough that we can't throw an error in the part of the
code where we bail when controls are defined in both no-team and default
files.

To minimize diff size, we're both "passing-by-ref" and returning the
maps-by-team of scripts and installers, though the former would be
sufficient on its own.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- N/A Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests (sorta)
- [x] Manual QA for all new/changed functionality
2024-10-10 06:12:24 -05:00
Ian Littman
550de39a7c
Switch path resolution for scripts to happen when spec is parsed (#22782)
#22555

This resolves issues where no-team script paths are resolved relative to
the base rather than the file they're in.

There was a similar issue for software packages, which I also fixed.

This also fixes script-poolicy associations when scripts and policy
files automating those scripts need different relative paths.

Test file moves ensure that these fixes are tested.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-10-09 17:57:08 -05:00
Victor Lyuboslavsky
567803955e
NDES SCEP proxy backend (#22542)
#21955

<div>
<a href="https://www.loom.com/share/ba40b440502845d2861fd3ec7611bade">
<p>[Demo] Deploy SCEP certificates from Network Device Enrollment
Service (NDES) #21955 - Watch Video</p>
    </a>
<a href="https://www.loom.com/share/ba40b440502845d2861fd3ec7611bade">
<img style="max-width:300px;"
src="https://cdn.loom.com/sessions/thumbnails/ba40b440502845d2861fd3ec7611bade-84f2d88c9f5106c2-full-play.gif">
    </a>
  </div>

Note: A few remaining subtasks will be done in a follow-up PR. See
#22123 for a detailed list.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-10-09 13:47:27 -05:00
Roberto Dip
a9bd2fb7c1 Merge remote-tracking branch 'origin/main' into feat-fleet-app-library 2024-10-04 08:46:12 -03:00
Ian Littman
e4df7abb67
Backend build for script automation (#22472)
#22115, #22116

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

No changes file, as FE changes file covers the entire feature

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
Co-authored-by: Tim Lee <timlee@fleetdm.com>
2024-10-03 20:03:40 -05:00
Roberto Dip
9bf5027194 Merge remote-tracking branch 'origin/main' into feat-fleet-app-library 2024-10-03 14:56:18 -03:00
Lucas Manuel Rodriguez
fee21ae2eb
Use pkgutil approach to be more effective at uninstalling (#22618)
#22571

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Added/updated tests
- [X] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [X] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [X] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [X] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [X] Manual QA for all new/changed functionality
2024-10-03 14:21:23 -03:00
Lucas Manuel Rodriguez
862cd142a3
Add filter to default unintaller for pkgs to only remove .app folders (#22585)
#22571

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
2024-10-02 17:50:27 -03:00
Martin Angers
38fe6d9a43
Fix windows installer stuck in pending state forever (#22592) 2024-10-02 16:18:37 -04:00
Gabriel Hernandez
d8fda9f085 Merge branch 'main' into feat-fleet-app-library 2024-10-02 10:01:25 +01:00
Lucas Manuel Rodriguez
f8f24e0a80
Add support to upload RPM packages (#22502)
#22473

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.

---------

Co-authored-by: RachelElysia <71795832+RachelElysia@users.noreply.github.com>
Co-authored-by: Ian Littman <iansltx@gmail.com>
2024-10-01 13:02:13 -03:00
Gabriel Hernandez
35b4f20396 Merge branch 'main' into feat-fleet-app-library 2024-09-30 12:35:06 +01:00
Lucas Manuel Rodriguez
f52e0a0e8c
Fix to not panic if MSRC for current month is not available (#21749)
See #21745.

---------

Co-authored-by: Ian Littman <iansltx@gmail.com>
2024-09-27 16:59:43 -05:00
Gabriel Hernandez
65668bed61 Merge branch 'main' into feat-fleet-app-library 2024-09-27 13:25:21 +01:00
Victor Lyuboslavsky
e0faa14025
Updating scep package with latest fixes (#22372)
Updating scep package with changes up to
github.com/micromdm/scep@781f8042a79cabcf61a5e6c01affdbadcb785932
Fixes needed for NDES client for #21955 

Manually pulled in the recent changes. You can view the changes in the
remote like:
https://github.com/getvictor/scep/compare/fleet...micromdm%3Ascep%3Amain
2024-09-27 07:04:11 -05:00
Gabriel Hernandez
4d54dc8a67 Merge branch 'main' into feat-fleet-app-library 2024-09-19 10:44:38 +01:00
Lucas Manuel Rodriguez
90959dbc9f
Fixes for no-team.yml GitOps (#22210)
#22185

- [X] Manual QA for all new/changed functionality
2024-09-18 14:16:59 -03:00
Gabriel Hernandez
ba5a9ec99e Merge branch 'main' into feat-fleet-app-library 2024-09-18 16:10:12 +01:00
Victor Lyuboslavsky
ddbdce4ab9
Updated PS1 install/uninstall scripts to fail on error. (#22164) 2024-09-17 17:32:14 -05:00
Lucas Manuel Rodriguez
2d05f2450d
Minor fixes to policy software installations (#22148)
PR for the three fixes described in #22104.

- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
2024-09-17 13:30:27 -03:00
Gabriel Hernandez
7f02749c0d Merge branch 'main' into feat-fleet-app-library 2024-09-17 15:01:48 +01:00
RachelElysia
1c1ebef4ef
20404: Edit packages feature (#21812)
## Issue
Cerra #20404

## Description
- Add frontend/API backend for editing software packages. GitOps will be
a separate PR.

## More
- Please see subtasks for change lists
  - #21611 
  - #21613

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Manual QA for all new/changed functionality

Automated tests will follow in another PR.

---------

Co-authored-by: Ian Littman <iansltx@gmail.com>
Co-authored-by: Luke Heath <luke@fleetdm.com>
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
Co-authored-by: Victor Lyuboslavsky <victor.lyuboslavsky@gmail.com>
2024-09-17 08:40:47 -05:00
Gabriel Hernandez
c4127e0ae9 Merge branch 'main' into feat-fleet-app-library 2024-09-16 11:55:31 +01:00
Victor Lyuboslavsky
e0e6b2a938
Updating install/uninstall scripts. (#22083)
Work in progress for #20000 
The biggest change here is the update to `uninstall_exe.ps1` so that it
is not completely broken.

I'd like to get these changes onto main for testing while I switch to
working on unreleased bugs.

# Windows EXE testing notes (in progress)

## FileZilla
https://filezilla-project.org/download.php?platform=win64
In uninstall script, use /S as $uninstallArgs

## Firefox
Get the full installer like:
https://download.mozilla.org/?product=firefox-latest&os=win&lang=en-US
DO NOT get product=firefox-stub
In uninstall script, use -ms as $uninstallArgs
2024-09-13 17:06:49 -05:00
Roberto Dip
519ee09117
add new changes for BYOD and fix issues (#22079)
for #21019 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-09-13 14:53:05 -03:00
Gabriel Hernandez
dec951f9f6 Merge branch 'main' into feat-fleet-app-library 2024-09-13 13:54:10 +01:00
Victor Lyuboslavsky
b60ebbc63e
Added GitOps support for uninstall script. (#21969)
`fleetctl gitops` subtask for #20320

# Checklist for submitter

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-09-12 13:25:40 -05:00
Lucas Manuel Rodriguez
4c24729df0
Add policies for "No team" (#21972)
#21467

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added/updated tests
- [X] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [X] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [X] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [X] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [X] Manual QA for all new/changed functionality
2024-09-12 14:23:25 -03:00
Victor Lyuboslavsky
f5fb6ba75c
Fixes from BE code review. 2024-09-12 10:29:28 -05:00
Gabriel Hernandez
08ca8c3b3d Merge branch 'main' into feat-fleet-app-library 2024-09-12 10:59:24 +01:00
Victor Lyuboslavsky
5f846c8870
Merge remote-tracking branch 'origin/main' into 20320-feat-uninstall-packages 2024-09-11 09:35:04 -05:00
Victor Lyuboslavsky
93f12d0d85
Updated/tested uninstall scripts, except for EXE. 2024-09-10 15:28:28 -05:00
Roberto Dip
05818902cd
implement OTA enrollment (#21942)
for #21019

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-09-10 16:52:17 -03:00
Martin Angers
9abd5a59d0
Maintained Apps: define app list, implement ingestion (#21946) 2024-09-10 13:55:13 -04:00
Dante Catalfamo
05e249c951
Clearer type error for software VPP app IDs (#21760)
#20828

This is a fix specific to the VPP App ID field in gitops. It's a field
where users are likely to get the type wrong, so I've addressed this
separately before a larger effort.
2024-09-09 15:26:21 -04:00
Victor Lyuboslavsky
1cf6cab829
Fix TestExtractInstallerMetadata test fail. 2024-09-08 13:16:04 -05:00
Victor Lyuboslavsky
cb537db724
Merge branch 'main' into 20320-feat-uninstall-packages
# Conflicts:
#	server/datastore/mysql/schema.sql
#	server/fleet/software_installer.go
2024-09-08 12:22:36 -05:00
Victor Lyuboslavsky
3287a245de
Added pe_test.go 2024-09-08 12:00:43 -05:00
Victor Lyuboslavsky
dd674552e2
Fixed and added Go tests. 2024-09-07 08:07:22 -05:00
Lucas Manuel Rodriguez
270ff784d6
Add GitOps support for policy installers (#21826)
#20895

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added/updated tests
- [X] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [X] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [X] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [X] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [X] Manual QA for all new/changed functionality
2024-09-06 19:10:28 -03:00
Jacob Shandling
78c534b44f Merge branch 'victor/20320-backend' into 20320-ui-uninstall-packages 2024-09-06 10:41:23 -07:00
Victor Lyuboslavsky
7d47cd95d8
Fixed tests 2024-09-06 09:49:36 -05:00
Victor Lyuboslavsky
42905d3c97
Can run install scripts now. 2024-09-06 09:49:28 -05:00
Victor Lyuboslavsky
ea10d43e93
Added uninstall script to software_installers 2024-09-06 09:49:27 -05:00
Victor Lyuboslavsky
d7f13295e9
Parsing package IDs 2024-09-06 09:49:27 -05:00
jacobshandling
0cfbdc6f58
UI – Implement changes for package uninstall scripts in the add software modal (#21828)
## Addresses #21564 – see issue for task list
![Screenshot 2024-09-04 at 5 45
12 PM](https://github.com/user-attachments/assets/546401dd-b56e-4c39-baba-456dc844ee0f)
![Screenshot 2024-09-04 at 5 42
57 PM](https://github.com/user-attachments/assets/810ca450-0ddd-4258-96a5-bddb300ae19d)
![Screenshot 2024-09-04 at 5 45
02 PM](https://github.com/user-attachments/assets/32a19ce6-52c3-4772-ba53-00e50145bc85)
![Screenshot 2024-09-04 at 5 43
23 PM](https://github.com/user-attachments/assets/925843fb-6290-489b-a639-de1cbfba83fa)

- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
2024-09-05 11:11:14 -07:00
Ian Littman
e96c70e4c0
Pull xz'd goval-dictionary sqlite files to evaluate vulnerabilities on Amazon Linux hosts (#21506)
#20934

This is tied to https://github.com/fleetdm/vulnerabilities/pull/14; for
supported OS versions (currently Amazon Linux 1/2/2022/2023) we'll pull
XZ'd sqlite files from the vulnerabilities repo and query them to
determine what's vulnerable. See the associated issue for how I
self-QA'd this.

This replaced OVAL parsing for Amazon Linux 2, as we were using the
wrong data source there (Amazon has backported a bunch of fixes to their
own-named releases, so any RHEL fixes don't match).

Some checklist items are missing here; getting this set up in draft to
get code feedback now, and I'll push updates with e.g. docs changes, as
well ass an addition to the changes file.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
    - [x] Add tests to oval_platform
    - [x] Add sync_test
    - [x] Add database_test
- [x] Manual QA for all new/changed functionality
- [x] Update vulnerability management docs
2024-08-26 14:07:42 -05:00
Tim Lee
a6a9a2e1c2
no team software gitops (#20847)
#20464 

Adding gitops support for a top level `software` key to be used to
manage installable software into "no team".

- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality

---------

Co-authored-by: Victor Lyuboslavsky <victor.lyuboslavsky@gmail.com>
2024-08-05 14:39:10 -03:00
Martin Angers
134534a9d8
Special-case the name of the .exe installer for Notion as it uses non-standard naming (#20881) 2024-07-31 13:56:25 -04:00
Victor Lyuboslavsky
671fc628e7
Fetch installed apps from iPhone/iPad devices. (#20733)
Part 2 of https://github.com/fleetdm/fleet/issues/19447
- iOS and iPadOS user-installed apps are loaded into Fleet
- Added an additional identifier into software_titles table to
differentiate between iOS/iPadOS apps
- Updated nano queue timestamp precision

Note: TestIntegrationsMDM/TestVPPApps fails when run as part of the
suite, but passes standalone. I'd like to proceed with merging this PR,
and figure out the issue next week.

# Checklist for submitter

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Roberto Dip <rroperzh@gmail.com>
2024-07-28 16:17:27 +02:00
Jahziel Villasana-Espinoza
5b97d01982 chore: merge main 2024-07-24 16:50:43 -04:00
Lucas Manuel Rodriguez
70d4558448
Backend support for iOS/iPadOS OS updates (#20649)
#20469 and #20471

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
2024-07-24 14:34:23 -03:00
Dante Catalfamo
51e2979965
VPP GitOps Config (#20238)
Currently covers the ability to sync and verify config with fleet
server. Bulk API moved to its own ticket (#20278) while product decides
its capabilities
2024-07-10 14:53:03 -04:00
Roberto Dip
7b20060350
cover edge case for alternative zoom installer (#20286)
Zoom offers two installers:

- Zoom for IT admins (already covered previously)
- "Regular" Zoom (covered here)

This tweaks the logic made as part of #19144 to ensure we cover both

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-10 08:05:09 -03:00
Lucas Manuel Rodriguez
2875a9dbb8
Fixes to fleetctl debug connection and TLS certs documentation (#20166)
#6085

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-09 14:04:23 -03:00
Roberto Dip
196f761b6d
use the latest fleetd-base version for MDM commands (#20078)
for #19176

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-06-28 16:35:07 -03:00
Roberto Dip
8f7e2f5214
improve extraction of name and version, and add bundle identifier for pkg installers (#19838)
part 1 of #19144
2024-06-25 10:55:24 -03:00
Roberto Dip
1cc13a09fb
🧹 friday cleanup party: substitute deprecated import of go-kit (#19774)
`go-kit/kit/log` was deprecated and generating warnings

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Manual QA for all new/changed functionality
2024-06-17 10:27:31 -03:00
Lucas Manuel Rodriguez
904e8a6825
Added server_settings.query_report_cap (#19692)
#19600

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
2024-06-14 12:24:01 -03:00
Victor Lyuboslavsky
05eb338561
Enable gitops to create teams with no enroll secrets, or clear enroll secrets for an existing team (#19616)
Enable gitops to create teams with no enroll secrets, or clear enroll
secrets for an existing team
#19332 

`fleetctl apply` also gains this extra functionality. In `fleetctl
apply` secrets will not be change if one of the following:
- secrets is missing from yml
- They are blank in yml, like: `secrets:`
- They are null in yml, like: `secrets: null`

They will only be cleared with `fleetctl apply` if the user explicitly
sets them to an empty array, like:
- `secrets: []`

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-06-11 13:20:32 -05:00
Lucas Manuel Rodriguez
3dbdbc1bcf
Add support for iOS/iPadOS to osquery-perf (#19522)
#18119

- [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.

Sample on how to simulate 50 iPads and 50 iPhones:
```sh
go run ./cmd/osquery-perf -host_count 100 -os_templates iphone_14.6.tmpl:50,ipad_13.18.tmpl:50 -mdm_scep_challenge <...>
```
2024-06-10 17:02:35 -03:00
Lucas Manuel Rodriguez
f8cee672e0
Support environment variables in config profiles (#18891)
#17309

I added some missing env var replacement tests for policies, queries,
etc.

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
2024-05-28 13:44:43 -03:00
Tim Lee
6f008b40f2
Fix backoff in vuln stream logging (#19148) 2024-05-28 08:13:46 -06:00
Roberto Dip
1ef19e4ff2 Merge remote-tracking branch 'origin/main' into feat-software-installers 2024-05-15 19:55:02 -03:00
Roberto Dip
d383876a3c
fix issues installing software in windows (#19048)
for #19039 and #19041 this:

- fixes the install/remove scripts to read the env variable the proper
way
- truncates output before storing in the databse in case its longer than
MySQL's TEXT size

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-05-15 19:39:42 -03:00
Roberto Dip
ad94dff814
installer report and rollback fixes (#19046)
for https://github.com/fleetdm/fleet/issues/19020

- Fixes the rollback logic to get the right script for the software
being installed
- Fixes the messages displayed in the install results

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-05-15 19:18:35 -03:00
Jahziel Villasana-Espinoza
01898fd176
fix: typos in scripts (#19045)
Feature cleanup

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
2024-05-15 16:36:31 -04:00
Martin Angers
1def5b2ddf
Add support for software installers in fleetctl gitops (#18990) 2024-05-14 16:58:58 -04:00
Dante Catalfamo
71c0026168
Orbit software installer flow (#18797)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).

---------

Co-authored-by: Roberto Dip <rroperzh@gmail.com>
2024-05-14 17:25:35 -03:00
Roberto Dip
3a31262353
add CLI and endpoints to set software via fleetctl apply (#18876)
for #18325

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
2024-05-14 15:06:33 -03:00
Tim Lee
ae00942d5c
Add backoff and retry to vuln downloads (#18855)
This adds a 3min max retry interval when downloading vulnerability
assets. The need was discovered initially due to frequent test failures
when downloading an unstable Ubuntu Kinetic OVAL feed.
2024-05-09 11:38:12 -06:00
Lucas Manuel Rodriguez
a1f77e01e5
Add support for escaping $ in gitops yamls (#18845)
#18467

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- ~[ ] If database migrations are included, checked table schema to
confirm autoupdate~
- ~For database migrations:~
- ~[ ] Checked schema for all modified table for columns that will
auto-update timestamps during migration.~
- ~[ ] Confirmed that updating the timestamps is acceptable, and will
not cause unwanted side effects.~
- ~[ ] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).~
- [X] Manual QA for all new/changed functionality
  - ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
2024-05-09 12:29:16 -03:00
Roberto Dip
e5283ef2a8 Merge remote-tracking branch 'origin/main' into feat-software-installers 2024-05-07 13:37:29 -03:00
Roberto Dip
37fe905f96
missing validations and tweaks to default scripts (#18780)
This adds two things:

- when implementing the CLI, I found [a
panel](https://www.figma.com/file/oQl2oQUG0iRkUy0YOxc307/%2314921-Deploy-security-agents-to-macOS%2C-Windows%2C-and-Linux-hosts?type=design&node-id=779-29335&mode=design&t=Y27cbj7DdhUEGJko-4)
in the Figma file with validations that I missed
- explicit shebang for bash scrips (requested by product) and removed a
comment that will be user facing for exe files.
2024-05-07 13:02:08 -03:00
Roberto Dip
bd3c0a1e9a
adjust logic to get default scripts (#18719)
This tweaks the logic to get default install/remove scripts to delegate
the variable replacement to `fleetd`
2024-05-06 11:41:31 -03:00
Victor Lyuboslavsky
4f4800be19
GitOps remove teams (#18640)
#16677 

Improvements to `fleetctl gitops` command:
- Added the ability to pass multiple files, like `fleetctl gitops -f
file1 -f file2`, where the first file must be the global configuration
- Added the ability to remove teams that were not specified in team
configs using the switch `--delete-other-teams`
- When passing a global config and team config during initial
configuration, the `org_settings.mdm.apple_bm_default_team` value can be
set to match the team that will be created by the provided team config.

After these changes are released to prod, we can update
https://github.com/fleetdm/fleet-gitops to use the new switches: #18692

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-05-03 08:03:00 -05:00
Roberto Dip
123fdc72b0
add endpoint to send software installer requests (#18711)
#18334

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-05-02 18:00:06 -03:00
Roberto Dip
92e540aee5
add scripts to add/remove software (#18649)
for:

- https://github.com/fleetdm/fleet/issues/18314
- https://github.com/fleetdm/fleet/issues/18315
- https://github.com/fleetdm/fleet/issues/18317
- https://github.com/fleetdm/fleet/issues/18316

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-05-01 14:15:59 -03:00
Martin Angers
7d014f9fad
Extract metadata from installers part 2 (#18608) 2024-04-30 11:20:53 -04:00
Gabriel Hernandez
7983e91933 Merge branch 'main' into feat-software-installers 2024-04-30 11:40:55 +01:00
Roberto Dip
05719633a1
add consistent MDM host lifecycle management (#18510)
The mantra for MDM lifecycle events is:

> - Noah: When MDM is turned on, install fleetd, bootstrap package (if
DEP),
> and profiles. Don't clear host vitals (everything you see on the Host
>      details page)
>    - Noah: On re-enrollment, don't clear host vitals.
>    - Noah: On lock and wipe, don't clear host vitals.
>    - Noah: On delete, clear host vitals.

This addresses issues:

- https://github.com/fleetdm/fleet/issues/17243
- https://github.com/fleetdm/fleet/issues/17481
- https://github.com/fleetdm/fleet/issues/17292
- https://github.com/fleetdm/fleet/issues/18030
- https://github.com/fleetdm/fleet/issues/18031
2024-04-29 16:43:15 -03:00
Martin Angers
563d55c218
Software installers: extract metadata from installers (part 1) (#18509) 2024-04-29 09:13:36 -04:00
Lucas Manuel Rodriguez
1b26129465
Delete expired activities in cleanup cron (#18441)
#16989

API draft PR: #17809.
UI draft PR: #18243.

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- ~[ ] If database migrations are included, checked table schema to
confirm autoupdate~
- ~For database migrations:~
- ~[ ] Checked schema for all modified table for columns that will
auto-update timestamps during migration.~
- ~[ ] Confirmed that updating the timestamps is acceptable, and will
not cause unwanted side effects.~
- ~[ ] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).~
- [X] Manual QA for all new/changed functionality
  - ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
2024-04-26 07:22:31 -03:00
Roberto Dip
e39f5bce3b
implement logic to sign apple profiles (#16490)
#10418
2024-04-18 18:01:37 -03:00
Roberto Dip
f0ad942a57
implement status reports for DDM commands (#17831)
for #17408
2024-03-26 10:40:35 -03:00
Sarah Gillespie
3d73174e90
Add macOS DDM protocol endpoints for tokens, declaration-items, and declaration/.../... (#17679) 2024-03-18 17:48:07 -05:00
Roberto Dip
f5cf156653
improve mdmtest package to handle any kind of command (#17673)
it delegates any extra unmarshaling to the caller. We might consider
building our own types in the future instead of relying on micromdm, but
these are used only for tests right now.
2024-03-15 17:10:48 -03:00
Sarah Gillespie
48b31a02ae
Add DDM service struct, basic handlers, and test client (#17671) 2024-03-15 15:20:15 -03:00
Lucas Manuel Rodriguez
e9a464e0cf
Add exponential backoff to orbit enroll retries (#17368)
#16594

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [X] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [X] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2024-03-13 07:57:00 -03:00
Victor Lyuboslavsky
02de6b5695
Adding full unicode/emoji support for team/policy names. (#17163)
#17027 
Added Unicode and emoji support for policy and team names.

I have the manual test steps in the issue:
https://github.com/fleetdm/fleet/issues/17027

# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-02-27 12:55:05 -06:00
Jahziel Villasana-Espinoza
272ce3187f
feat: move scep dependency inside the monorepo (#16988)
Following the pattern set up by Martin and Lucas for similar PRs. Can be
reviewed by commits:

1. move scep directory into monorepo
2. update import paths
3. update go.mod, go.sum
4. fix golint errors in scep package
5. skip a failing test that's been broken for a while
6. fix that failing test

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Manual QA for all new/changed functionality

Also verified that our test suite runs the `scep` tests.
2024-02-22 13:13:46 -05:00
Victor Lyuboslavsky
e4d5e27dd9
fleetctl gitops (#16535)
Add `fleetctl gitops` command for #13643 

Code review video:
https://www.loom.com/share/7941c51c709b44ccafd618dd05837d99?sid=27b923d7-1393-4396-bac7-30616b2d6de9

fleet-gitops PR that also needs review:
https://github.com/fleetdm/fleet-gitops/pull/26

Working global/team gitops configs that can be used for testing:
https://github.com/fleetdm/fleet-gitops/tree/victor/fixing-configs

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-02-09 13:34:57 -06:00
Roberto Dip
2d4c1b079a
improvements for troubleshooting workflow for MDM commands (#16654)
for #16416
2024-02-07 16:24:03 -03:00
Lucas Manuel Rodriguez
5360029d67
Allow custom osquery database on fleetd (#16554)
#16014

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
- [x] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2024-02-05 09:41:06 -03:00
Martin Angers
fc3304c902
Move nanomdm dependency in monorepo (#16015)
#15557 

Following the precedent that Lucas used for other similar PRs, the best
way to review is probably by commits.

* The first one simply copies over the files from the fork to the
monorepo
* Second one adjusts all import paths
* Third one tidies up the `go.mod` files
* Last one fixes the linter issues in the nanomdm package

# Checklist for submitter

- ~~Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.~~ (not a user-visible change)
- [x] Manual QA for all new/changed functionality (ran test suite,
re-generated mocks)

I also verified that our Go test suite did run the newly moved `nanomdm`
package steps:

```
ok  	github.com/fleetdm/fleet/v4/server/mdm/nanomdm/cryptoutil	0.003s	coverage: 0.0% of statements in github.com/fleetdm/fleet/v4/...
ok  	github.com/fleetdm/fleet/v4/server/mdm/nanomdm/mdm	0.005s	coverage: 46.2% of statements in github.com/fleetdm/fleet/v4/...
ok  	github.com/fleetdm/fleet/v4/server/mdm/nanomdm/service/certauth	1.320s	coverage: 20.7% of statements in github.com/fleetdm/fleet/v4/...
ok  	github.com/fleetdm/fleet/v4/server/mdm/nanomdm/storage/file	0.007s	coverage: 24.1% of statements in github.com/fleetdm/fleet/v4/...
```
2024-01-11 23:28:48 -03:00
Roberto Dip
3f302a79b4
fix log with next retry time once max retries are exceeded (#16026) 2024-01-10 17:53:30 -03:00
Roberto Dip
3b00b70786
fix unreleased bugs for the increased script timeout (#15897)
for https://github.com/fleetdm/fleet/issues/15196. The main problem was
that we have two timeouts:

1. The timeout used by the host to kill the script execution
2. The timeout used by the server to wait for the script results

Before the changes in https://github.com/fleetdm/fleet/pull/15779, the
server timeout was longer than the host timeout, but we inadvertently
set both values to 5 minutes, which breaks the logic we have to handle
both kinds of timeouts.
2024-01-04 10:26:13 -03:00
Roberto Dip
8957078d75
Add backoff functionality for fleetd updates (#15489)
related to #14176, `fleetd` will now retry 3 times and wait 24 hours to
try again for an specific update.
2023-12-08 19:43:56 -03:00
Roberto Dip
b07fbdc1d7
implement profile verification for windows (#15348)
for #14426.

In order to prevent import cycles and be able to use some type
definitions/constants I followed the same pattern we did for macOS by
creating a new package named `syncml`. This makes the changelog look
bigger than it actually is, so I split it into two commits to make it
easier to review:

-
[d7c233d](d7c233d54c)
moves the relevant bits to this new package
-
[7531a07](7531a0742b)
implements profile verification
2023-11-30 09:17:07 -03:00
Martin Angers
2f927df4f0
Implement Windows OS Updates (feature branch). (#15359) 2023-11-29 11:07:24 -05:00
Roberto Dip
5f313c8972
fix various bugs after testing windows MDM profiles (#15264)
for #13281
2023-11-22 10:48:28 -03:00
Roberto Dip
9cf30a9131
Feat windows msmdm (#14837)
for #13069

---------

Co-authored-by: Marcos Oviedo <marcos@fleetdm.com>
Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>
2023-11-01 11:13:12 -03:00
guangwu
33858d7301
chore: remove refs to deprecated io/ioutil (#14485)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- [ ] Documented any permissions changes (docs/Using
Fleet/manage-access.md)
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [ ] Added/updated tests
- [ ] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).

Signed-off-by: guoguangwu <guoguangwu@magic-shield.com>
2023-10-27 15:28:54 -03:00
Roberto Dip
9ffa11c25d
Feat: saved scripts (#14409)
For #9537
2023-10-10 19:00:45 -03:00
Marcos Oviedo
f0d77ab3db
Merging Bitlocker feature branch (#14350)
This relates to #12577

---------

Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>
Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2023-10-06 19:04:33 -03:00
gillespi314
e08bb000c9
Update nanomdm dependency (#12721)
Updates include:
- Fix issues where `GetBootstrapToken` returned `500` instead of no data
and no error per Apple MDM
[documentation](https://developer.apple.com/documentation/devicemanagement/get_bootstrap_token)
- Incorporate additional updates from the upstream nanomdm repo
2023-08-21 11:07:57 -03:00
Roberto Dip
902e064d04
fix issues with migration flow (#13297)
For #13094
2023-08-14 09:56:59 -03:00
Martin Angers
e323a3d881
Consider an empty EULA pdf file the same as an invalid one, returning 400 Bad Request (#12542) 2023-06-28 08:19:42 -04:00
Lucas Manuel Rodriguez
bb3b21b574
Add TestMDMClient to simulate MDM clients in osquery-perf (#11672)
#11528

osquery-perf simulated hosts enroll and are identified as manually
enrolled. (Enrolling as DEP requires more work, e.g. a new mocked Apple
DEP endpoint).

Given that these are simulated MDM clients, they cannot be woken up with
push notifications. Instead, these check for new commands to execute
every 10 seconds (which is not realistic, but could serve as a good
loadtesting exercise).

I will now start setting up the loadtest environment with MDM enabled
and configured to test this.

- ~[ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.~
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
  - ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
2023-05-12 13:50:20 -03:00
Roberto Dip
653bbec5f1
cli: try to infer the bootstrap package name from the URL too (#11571)
#11570
2023-05-11 10:36:28 -03:00
Roberto Dip
11356b2f15
add CRUD for EULA (#11274)
https://github.com/fleetdm/fleet/issues/10741
2023-05-02 10:09:33 -03:00
Lucas Manuel Rodriguez
7dadec3ecf
Add mTLS support to fleetd (#11319)
#7970

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [x] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2023-04-27 08:44:39 -03:00
Martin Angers
582e85c876
Add support for the mdm.macos_setup.macos_setup_assistant key in fleetctl, API (#11296) 2023-04-25 09:36:01 -04:00
Juan Fernandez
d353a3deb4
Handle flaky vulnerability tests (#11262)
- Refactored some of the vulnerabilities tests to help with flakiness.
- Don't load NVD assets if local assets have a timestamp of today.
2023-04-21 19:37:29 -04:00
Roberto Dip
12b79d5e46
add pkg to inspect xar files (#11015)
Part of #10213 this will be used to do various validations in the
provided bootstrap package.
2023-04-05 20:49:02 -03:00
Lucas Manuel Rodriguez
d09252e1ea
Fix YAML and add item to error (#10503) 2023-03-15 08:50:13 -03:00
Lucas Manuel Rodriguez
e0cbc3aad7
Add automation for win-10 cis policies and fix yaml (#10289) 2023-03-03 16:11:04 -03:00
Lucas Manuel Rodriguez
d4a1b4d218
Add CIS checks for 2.9.X and add pmset table to fleetd (#9470)
#9253

- ~[ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.~
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [X] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~

---------

Co-authored-by: Sharon Katz <121527325+sharon-fdm@users.noreply.github.com>
2023-02-08 13:08:17 -03:00
Lucas Manuel Rodriguez
bfe27475c1
Fix cis yaml (#9531)
The PR fixes the broken yaml in main and adds checks so that it doesn't
happen again.
2023-01-27 14:58:00 -03:00
Lucas Manuel Rodriguez
ac22aadc13
Fleet server and tooling to use NETWORK_TEST_GITHUB_TOKEN when environment variable is set. (#9143)
* WIP

* Add more logging

* Check rate limit at end of action

* Add github client in more places

* Add new published firefox 93 vulnerabilities to tests

* Remove fmt printfs

* Restore CI check settings

* Readd newline
2023-01-03 14:56:11 -03:00
Martin Angers
f5fc316ad3
Add options for configuring APNS and SCEP keys/certs in Fleet server (#8730) 2022-12-05 10:22:56 -05:00
Frank Sievertsen
437b1832dd
fix RunWithNetRetry (#8590) 2022-11-07 16:31:10 +01:00
Martin Angers
8f21e026e3
Fix bug with fleetctl apply for teams, clear agent options only if key is present (#8508) 2022-11-01 15:22:45 -04:00
Martin Angers
e19191161e
Validate team spec unknown keys (#8265) 2022-10-24 08:49:44 -04:00
Lucas Manuel Rodriguez
9191f4ce66
Add Apple MDM functionality (#7940)
* WIP

* Adding DEP functionality to Fleet

* Better organize additional MDM code

* Add cmdr.py and amend API paths

* Fix lint

* Add demo file

* Fix demo.md

* go mod tidy

* Add munki setup to Fleet

* Add diagram to demo.md

* Add fixes

* Update TODOs and demo.md

* Fix cmdr.py and add TODO

* Add endpoints to demo.md

* Add more Munki PoC/demo stuff

* WIP

* Remove proposals from PoC

* Replace prepare commands with fleetctl commands

* Update demo.md with current state

* Remove config field

* Amend demo

* Remove Munki setup from MVP-Dogfood

* Update demo.md

* Add apple mdm commands (#7769)

* fleetctl enqueue mdm command

* fix deps

* Fix build

Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>

* Add command to upload installers

* go mod tidy

* fix subcommands help

There is a bug in urfave/cli where help text is not generated properly when subcommands
are nested too deep.

* Add support for installing apps

* Add a way to list enrolled devices

* Add dep listing

* Rearrange endpoints

* Move DEP routine to schedule

* Define paths globally

* Add a way to list enrollments and installers

* Parse device-ids as comma-separated string

* Remove unused types

* Add simple commands and nest under enqueue-command

* Fix simple commands

* Add help to enqueue-command

* merge apple_mdm database

* Fix commands

* update nanomdm

* Split nanomdm and nanodep schemas

* Set 512 MB in memory for upload

* Remove empty file

* Amend profile

* Add sample commands

* Add delete installers and fix bug in DEP profile assigning

* Add dogfood.md deployment guide

* Update schema.sql

* Dump schema with MySQL 5

* Set default value for authenticate_at

* add tokens to enrollment profiles

When a device downloads an MDM enrollment profile, verify the token passed
as a query parameter. This ensures untrusted devices don't enroll with
our MDM server.

- Rename enrollments to enrollment profiles. Enrollments is used by nano
  to refer to devices that are enrolled with MDM
- Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles
- Generate a token for authentication when creating an enrollment profile
- Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token=

* remove mdm apple server url

* update docs

* make dump-test-schema

* Update nanomdm with missing prefix table

* Add docs and simplify changes

* Add changes file

* Add method docs

* Fix compile and revert prepare.go changes

* Revert migration status check change

* Amend comments

* Add more docs

* Clarify storage of installers

* Remove TODO

* Remove unused

* update dogfood.md

* remove cmdr.py

* Add authorization tests

* Add TODO comment

* use kitlog for nano logging

* Add yaml tags

* Remove unused flag

* Remove changes file

* Only run DEP routine if MDM is enabled

* Add docs to all new exported types

* Add docs

* more nano logging changes

* Fix unintentional removal

* more nano logging changes

* Fix compile test

* Use string for configs and fix config test

* Add docs and amend changes

* revert changes to basicAuthHandler

* remove exported BasicAuthHandler

* rename rego authz type

* Add more information to dep list

* add db tag

* update deps

* Fix schema

* Remove unimplemented

Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com>
Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 19:53:54 -03:00
Martin Angers
ec75fb10b2
Fix CI tests for Go (#8079) 2022-10-04 15:48:14 -04:00
Lucas Manuel Rodriguez
a4d7e81475
Orbit to support environments with revoked enroll secrets (#8056)
* Support environments with revoked enroll secrets

* Add instructions on how to fix Orbit enroll

* Rename to last_recorded_error

* Add alternative instructions
2022-10-03 17:28:19 -03:00
Roberto Dip
15c93f02ea
add retry logic for native notarization and codesigning (#7806)
Related to #7130, this adds logic to retry native notarization up to three times if it fails for some reason.

Since we're adding retries in various places, I added a new package under pkg for this purpose.
2022-09-19 13:08:39 -03:00
Roberto Dip
05ddeade90
add back-end implementation for SSO JIT provisioning (#7182)
Related to #7053, this uses the SSO config added in #7140 to enable JIT provisioning for premium instances.
2022-08-15 14:42:33 -03:00
Lucas Manuel Rodriguez
6dcff28be0
Move specs parsing functionality to a new pkg/spec package (#7050) 2022-08-05 19:07:32 -03:00
Lucas Manuel Rodriguez
a336ed61e5
Add gotestfmt to improve test output and fix flaky tests (#6528) 2022-07-11 08:12:33 -03:00
Juan Fernandez
255b670ce0
Bug 6391: Added new error strings to network retry logic (#6392) 2022-06-28 08:56:38 -04:00
Lucas Manuel Rodriguez
3dd89bc0a1
Fix flaky test TestVulnerabilityDataStream (#6162) 2022-06-09 10:36:37 -03:00
Michal Nicpon
30a864c8d2
Sync CVE scores periodically (#5838) 2022-06-01 10:06:57 -06:00
Lucas Manuel Rodriguez
114b678e26
test: Add retry to the correct part that has sync enabled (#5688) 2022-05-11 10:11:51 -03:00
Lucas Manuel Rodriguez
fda79a8770
Run network test serially to prevent timeouts on Github CI (#5557)
* Run network test serially to prevent timeouts on Github CI

* Revert lint changes

* Add simple file lock

* Revert test change

* Clarify error check
2022-05-10 11:52:33 -03:00
Lucas Manuel Rodriguez
c82c580716
Orbit: Add Fleet Desktop support to Windows (#4873)
* Orbit: Add Fleet Desktop support to Windows

* Rename workflow, fix linux build

* Do not compile systray on linux

* nolint on unused

* Fix lint properly

* nolint both checkers

* Fix monitor logic in desktopRunner

* Fix interrupt and execute order
2022-04-01 17:28:51 -03:00
Michal Nicpon
d8d582760d
fix rename tmp file (#4862)
Renaming a temporary file to the final destination does not always work.
Specifically, if the source and destination paths are on different file
systems, you will get the following error

 invalid cross-device link

Instead, create temporary file in the destination directory.
2022-04-01 09:03:11 -06:00
Lucas Manuel Rodriguez
ecdfd627b6
Fleet Desktop MVP (#4530)
* WIP

* WIP2

* Fix orbit and fleetctl tests

* Amend macos-app default

* Add some fixes

* Use fleetctl updates roots command

* Add more fixes to Updater

* Fixes to app publishing and downloading

* Add more changes to support fleetctl cross generation

* Amend comment

* Add pkg generation to ease testing

* Make more fixes

* Add changes entry

* Add legacy targets (until our TUF system exposes the new app)

* Fix fleetctl preview

* Fix bool flag

* Fix orbit logic for disabled-updates and dev-mode

* Fix TestPreview

* Remove constant and fix zip-slip attack (codeql)

* Return unknown error

* Fix updater's checkExec

* Add support for executable signing in init_tuf.sh

* Try only signing orbit

* Fix init_tuf.sh targets, macos-app only for osqueryd

* Specify GOARCH to support M1s

* Add workflow to generate osqueryd.app.tar.gz

* Use 5.2.2 on init_tuf.sh

* Add unit test for tar.gz target

* Use artifacts instead of releases

* Remove copy paste residue

* Fleet Desktop Packaging WIP

* Ignore gosec warning

* Trigger on PR too

* Install Go in workflow

* Pass url parameter to desktop app

* Fix fleetctl package

* Final set of changes for v1 of Fleet Desktop

* Add changes

* PR fixes

* Fix CI build

* add larger menu bar icon

* Add transparency item

* Delete host_device_auth entry on host deletion

* Add SetTargetChannel

* Update white logo and add desktop to update runner

* Add fleet-desktop monitoring to orbit

* Define fleet-desktop app exec name

* Fix update runner creation

* Add API test before enabling the My device menu item

Co-authored-by: Zach Wasserman <zach@fleetdm.com>
2022-03-21 14:53:53 -03:00
Lucas Manuel Rodriguez
be72dc356c
Add CentOS parsing+post-processing to reduce false positives in vulnerability processing (#4037)
* Add CentOS parsing and post-processing in fleet

* Add tests and amend SyncCPEDatabase

* Add test for centosPostProcessing

* Changes from PR comments

* Amend software test

* Fix sync test

* Add index to source and vendor

* Use os.MkdirTemp

* Rearrange migrations

* Regenerate test schema

* Add support for testing migrations (#4112)

* Add support for testing migrations

* Rename migration in tests

* Changes suggested in PR

* Go mod tidy
2022-02-14 15:13:44 -03:00
Martin Angers
c997f853e5
Make creation of http.Client uniform across the codebase (#3097) 2021-11-24 15:56:54 -05:00
Martin Angers
69a4985cac
Use new error handling approach in other packages (#2954) 2021-11-22 09:13:26 -05:00
Zach Wasserman
e279dc1682
Implement fleetctl updates rotate (#2831)
Add support for updating keys used in the TUF update system.
2021-11-15 10:01:48 -08:00
Tomas Touceda
b11c6ffe31
Download and launch orbit for the current device when starting preview (#2437)
* Download and launch orbit for the current device when starting preview

* Add windows kill

* Fix imports

* Fix import

* Disable secure directory checks on windows for now

* Add missing import
2021-10-15 18:23:06 -03:00
Martin Angers
e61cb7e0db
Configure golangci-lint for the whole repository
Add a relatively minimal set of linters that raise safe and 
mostly un-opinionated issues with the code. It runs 
automatically on CI via a github action.
2021-08-24 13:35:03 -04:00
Martin Angers
a3714d2ed9
Add fleetctl debug connection command (#1706)
Adds the `fleetctl debug connection` command to investigate
connection issues to the fleet server.

Closes #1579 .
2021-08-24 08:50:03 -04:00