For #27476
# Checklist for submitter
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
# Details
This PR adds a new command `generate-gitops` to the `fleetctl` tool. The
purpose of this command is to output GitOps-ready files that can then be
used with `fleetctl-gitops`.
The general usage of the command is:
```
fleectl generate-gitops --dir /path/to/dir/to/add/files/to
```
By default, the outputted files will not contain sensitive data, but
will instead add comments where the data needs to be replaced by a user.
In cases where sensitive data is redacted, the tool outputs warnings to
the user indicating which keys need to be updated.
The tool uses existing APIs to gather data for use in generating
configuration files. In some cases new API client methods needed to be
added to support the tool:
* ListConfigurationProfiles
* GetProfileContents
* GetScriptContents
* GetSoftwareTitleByID
Additionally, the response for the /api/latest/fleet/software/batch
endpoint was updated slightly to return `HashSHA256` for the software
installers. This allows policies that automatically install software to
refer to that software by hash.
Other options that we may or may not choose to document at this time:
* `--insecure`: outputs sensitive data in plaintext instead of leaving
comments
* `--print`: prints the output to stdout instead of writing files
* `--key`: outputs the value at a keypath to stdout, e.g. `--key
agent_options.config`
* `--team`: only generates config for the specified team name
* `--force`: overwrites files in the given directory (defaults to false,
which errors if the dir is not empty)
# Technical notes
The command is implemented using a `GenerateGitopsCommand` type which
holds some state (like a list of software and scripts encountered) as
well as a Fleet client instance (which may be a mock instance for tests)
and the CLI context (containing things like flags and output writers).
The actual "action" of the CLI command calls the `Run()` method of the
`GenerateGitopsCommand` var, which delegates most of the work to other
methods like `generateOrgSettings()`, `generateControls()`, etc.
Wherever possible, the subroutines use reflection to translate Go struct
fields into JSON property names. This guarantees that the correct keys
are written to config files, and protects against the unlikely event of
keys changing.
When sensitive data is encountered, the subroutines call `AddComment()`
to get a new token to add to the config files. These tokens are replaced
with comments like `# TODO - Add your enrollment secrets here` in the
final output.
# Known issues / TODOs:
* The `macos_setup` configuration is not output by this tool yet. More
planning is required for this. In the meantime, if the tool detects that
`macos_setup` is configured on the server, it outputs a key with an
invalid value and prints a warning to the user that they'll need to
configure it themselves.
* `yara_rules` are not output yet. The tool adds a warning that if you
have Yara rules (which you can only upload via GitOps right now) that
you'll have to migrate them manually. Supporting this will require a new
API that we'll have to discuss the authz for, so punting on it for now.
* Fleet maintained apps are not supported by GitOps yet (coming in
https://github.com/fleetdm/fleet/issues/24469). In the meantime, this
tool will output a `fleet_maintained_apps` key and trigger a warning,
and GitOps will fail if that key is present.
---------
Co-authored-by: Lucas Manuel Rodriguez <lucas@fleetdm.com>
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
For https://github.com/fleetdm/fleet/issues/9943
This will help us avoid issues like this where the log message never
worked right:
https://github.com/fleetdm/fleet/pull/28296#discussion_r2047505191
Most of the changes are no-op type changes like removing unneeded
typecast or disabling gosec on reviewed lines of code
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [x] Make sure fleetd is compatible with the latest released version of
Fleet (see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/fleetd-development-and-release-strategy.md)).
- [x] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
For #26692.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
Changes file included in FE PR.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [ ] Make sure fleetd is compatible with the latest released version of
Fleet (see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/fleetd-development-and-release-strategy.md)).
- [ ] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
For https://github.com/fleetdm/fleet/issues/27984
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #26114
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
For #26071 and #26089
Added `macos_setup.manual_agent_install` boolean option. No validation
(pushed to another story due to complications caused by bug #28497)
Tests are failing due to vulnerability issues that are not related to
this feature. All tests were passing earlier.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
Adds renewal of Digicert certificates:
https://github.com/fleetdm/fleet/issues/26553 . Does not attempt to
renew custom SCEP or NDES. Also we aren't actually calling the DigiCert
renewal endpoint at this time because we don't believe we need to and we
can't necessarily do that as we weren't previously storing the serial
number however this change adds storage of the serial number.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
For #26148
remove gitops settings when deleting no-team.yml from the gitops repo.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
> For #23497
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
> For #24087
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
#27275 and #27274
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [x] Make sure fleetd is compatible with the latest released version of
Fleet (see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/fleetd-development-and-release-strategy.md)).
- [x] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
---------
Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>
For #27281
This PR adds `/api/{version}/fleet/scim/details` endpoint, along with
some frontend fixes.
# Checklist for submitter
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #27284
This PR:
- Adds SCIM as a fallback for username during macOS end user
authentication during setup experience
- Adds SCIM/endUsers details to host details
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #27287
This PR adds integration tests for SCIM API endpoints as well as some
bug fixes found by these tests.
# Checklist for submitter
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
For #27301
# Checklist for submitter
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added/updated automated tests
- [X] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [X] Manual QA for all new/changed functionality
# Details
This PR adds the ability to set/unset labels on policies via GitOps. It
builds on https://github.com/fleetdm/fleet/pull/27575 (back end for
policy labels) and updates the `PolicySpec` type and `ApplyPolicySpecs`
methods to update the `policy_labels` table where needed.
## Testing
1. Create a few labels in the UI
1. Create a global policy "foo" in the UI without labels
2. Create a global policy "bar" in the UI with labels
2. Create a global policy "baz" in the UI with labels
4. Use `fleetctl gitops` with a global .yml file, and under `policies:`
add "foo", "bar", "baz" and "boop".
* Add labels to "foo" with `labels_include_any:`
* Don't add `labels_include_any:` to "bar"
* Add labels to "baz" with `labels_include_any:`, but different labels
than what you added in the UI
* Add labels to "boop" with `labels_include_any:`
The expected outcome when viewing the queries in the UI (on the "edit
query" screen)
* Foo, Baz and Boop should have the labels specified in gitops
* Bar should have no labels
Repeat testing with _excluded_ labels.
---------
Co-authored-by: dantecatalfamo <dante.catalfamo@gmail.com>
Co-authored-by: Dante Catalfamo <43040593+dantecatalfamo@users.noreply.github.com>
For #27287
Video explaining the PR: https://www.youtube.com/watch?v=ZHgFUAvrPEI
This PR adds SCIM Users support for Okta. The goal is to first add
Users/Groups support so that the remaining backend SCIM work can be done
in parallel.
This PR does not include the following, which will be added in later PRs
- Changes file
- Groups support for Okta
- Full support for Entra ID
- Integration tests
# Checklist for submitter
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #26713
# Details
This PR updates Fleet and its related tools and binaries to use Go
version 1.24.1.
Scanning through the changelog, I didn't see anything relevant to Fleet
that requires action. The only possible breaking change I spotted was:
> As [announced](https://tip.golang.org/doc/go1.23#linux) in the Go 1.23
release notes, Go 1.24 requires Linux kernel version 3.2 or later.
Linux kernel 3.2 was released in January of 2012, so I think we can
commit to dropping support for earlier kernel versions.
The new [tools directive](https://tip.golang.org/doc/go1.24#tools) is
interesting as it means we can move away from using `tools.go` files,
but it's not a required update.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [X] Make sure fleetd is compatible with the latest released version of
Fleet
- [x] Orbit runs on macOS ✅ , Linux ✅ and Windows.
- [x] Manual QA must be performed in the three main OSs, macOS ✅,
Windows and Linux ✅.
> For issues found during QA
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
If the Fleet server is on a connection that can't pull ~150 Mbps down,
it'll time out before being able to add Microsoft Word for macOS, due to
the 100-second default timeout. This skips that behavior.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
> For #26662
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #27235.
The added logging might help troubleshooting #27235 which seems to only
happen in our dogfood environment.
- [X] Manual QA for all new/changed functionality
> For #27548
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
> For #26655
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [ ] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Ian Littman <iansltx@gmail.com>
For #26082.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #27035
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [ ] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
## Details
This PR adds an `author_id` column to the `labels` table, and adds the
associated properties to the `Label` and `LabelSpec` types. When a new
label is created via the UI or API, an author ID is set on the label if
one can be inferred from the context. Otherwise, the author ID is set to
`null`.
## Authz and Automated testing
Additional backend authorization logic is introduced in a follow-on PR,
https://github.com/fleetdm/fleet/pull/27089, because rconciling all of
the test updates between this PR and
https://github.com/fleetdm/fleet/pull/27038 was getting complicated.
## Manual Testing
* Tested in the UI by creating a new label on the Hosts page
* Tested via Gitops by merging this branch with
https://github.com/fleetdm/fleet/pull/27038 and doing `fleetctl gitops`
with a global config with `labels:` in it.
For #24473
This PR allows users to add / update / remove labels from queries via
Gitops.
## Testing
1. Create a few labels in the UI
1. Create a global query "foo" in the UI without labels
2. Create a global query "bar" in the UI with labels
2. Create a global query "baz" in the UI with labels
4. Use `fleetctl gitops` with a global .yml file, and under `queries:`
and "foo", "bar", "baz" and "boop".
* Add labels to "foo" with `labels_include_any:`
* Don't add `labels_include_any:` to "bar"
* Add labels to "baz" with `labels_include_any:`, but different labels
than what you added in the UI
* Add labels to "boop" with `labels_include_any:`
The expected outcome when viewing the queries in the UI (on the "edit
query" screen)
* Foo, Baz and Boop should have the labels specified in gitops
* Bar should have no labels
For #26622
This PR includes:
- Making DigiCert client a real service that can be overridden in tests
- GitOps support for DigiCert and Custom SCEP configs
# Checklist for submitter
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #27234.
- Improved logging in orbit to help us during troubleshooting.
- Added some documentation on how to grep for errors related to software
package installation in orbit.
- Added `took` to server request error logs (it was only present when
the request succeeds).
---
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [x] Make sure fleetd is compatible with the latest released version of
Fleet (see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/fleetd-development-and-release-strategy.md)).
- [x] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
For #24473
# Checklist for submitter
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
## Details
This PR adds the ability to manage labels via GitOps. Usage is as
follows:
* If a top-level `labels:` key is provided in the global YAML file
provided to GitOps, then any labels in this list will be created (if
using a new name) or updated (if using an existing name).
* If no top-level `labels:` key is provided, no changes will be made to
labels. This allows backwards-compatibility; customers won't blow away
all of their labels if they don't immediately use `labels:` in their
YAML
Additionally, some new validation has been added so that label usage is
checked prior to application. This means that when the gitops command is
run, it will verify that any labels referenced elsewhere in the YAML
(e.g. by software installers or mdm profiles) exist, and will bail with
an error message if they don't.
## Testing
**Test label deletion**
1. Add some labels via the UI
2. Run `fleetctl gitops --dry-run` with a default.yml file _without_
`labels:` in it, and verify that it doesn't say it will update or delete
any labels
2. Run `fleetctl gitops` with a default.yml file _without_ `labels:` in
it, and verify that it doesn't modify or remove your labels
4. Run `fleetctl gitops --dry-run` with a default.yml file with
`labels:` in it and nothing underneath, and verify that it says that it
will delete your labels
4. Run `fleetctl gitops` with a default.yml file with `labels:` in it
and nothing underneath, and verify that it removes all your labels
**Test label create/update**
1. Add a label "foo" via the UI
2. Run `fleetctl gitops --dry-run` with a default.yml file with two
`labels:` in it, one named "foo" and one named "bar". Verify that the
output says that one label will be created and one will be updated.
2. Run `fleetctl gitops` with a default.yml file with two `labels:` in
it, one named "foo" and one named "bar". Verify that the two labels now
exist in the UI with the configuration you specified.
**Test label usage**
1. Add a label "foo" in the UI.
1. Run `fleetctl gitops --dry-run` with a default.yml file _without_
`labels:` in it, where a software installer or mdm profile uses the
"foo" label via `labels_include_any`. Verify that the output doesn't
complain about unknown labels.
1. Run `fleetctl gitops --dry-run` with a default.yml file _with_
`labels:` in it with nothing underneath, and a software installer or mdm
profile uses the "foo" label via `labels_include_any`. Verify that the
output complains about unknown label "foo"
1. Run `fleetctl gitops --dry-run` with a default.yml file _with_
`labels:` in it with a "foo" label defined underneath, and a software
installer or mdm profile uses the "foo" label via `labels_include_any`.
Verify that the output doesn't complain about unknown labels.
For #26603
This PR includes:
- Refactoring of NDES/SCEP verify/timeout logic for easier testing (with
dependency injection)
- Custom SCEP configs
- saving/deleting/updating of encrypted custom SCEP challenges
- validation call to custom SCEP server to verify connection
- Custom SCEP activities
- unit and integration tests for all of the above
This PR does not include the following:
- Changes file (in later PR)
# Checklist for submitter
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
> For #26083
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Ian Littman <iansltx@gmail.com>
https://github.com/fleetdm/fleet/issues/21300
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #26977.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #26743
This PR fixes an issue where using `fleetctl gitops` unsets the "gitops
mode" settings in the UI. The code which prepares the config spec to
send to the "modify config" endpoint deliberately copies over the
current app settings for gitops mode to facilitate this. I updated an
existing test to verify the new behavior.
For #26603
This PR includes:
- DigiCert configs
- Some config code for custom SCEP, but not fully functional
- Validation tests for DigiCert Configs
- DigiCert activities (manually tested)
This PR does not include the following:
- Encryption of DigiCert API tokens -- this requires DB migration (in
next PR)
- Making an HTTP call to DigiCert API for validation (in later PR)
- Integration tests (in later PR)
- Changes file (in later PR)
# Checklist for submitter
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
For #26218
- Added `users_deleted` table to track user actions if the user was
actually deleted.
- Added enable/disable Android MDM activities
Note: I could not auto-generate fleet.Service mock because it has issues
with methods that don't return anything. I ended up using testify mock
instead.
# Checklist for submitter
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
For #26218
- Added `GET /api/_version_/fleet/android_enterprise/signup_sse`
endpoint and tests
- Fixed up handling of Android status reports with a deleted device. We
don't actually expect this to happen in production since the proxy
should delete the pubSub connection when the enterprise is deleted.
# Checklist for submitter
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
For #26219
Video explaining the PR: https://www.youtube.com/watch?v=t2hmpE7N-50
This PR includes:
- applying a default/dummy policy to Android enterprise
- Setting up PubSub topic and subscription to point at Fleet Server
- Generating device enrollment token
- Enrolling a device
Although some tests were added, this PR is mostly manually tested.
Integration tests are an outstanding task.
# Checklist for submitter
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
> no issue. trying to make loadtests with vulnerable software more
accurate. these are the sw versions with most vulns from dogfood.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Manual QA for all new/changed functionality
## For #26230
- Add `gitops` settings to app config
- GET and PATCH endpoint functionality to retrieve and modify these
settings
- generate activities for enabling and disabling GitOps mode
- Premium only
- Update tests
<img width="1355" alt="Screenshot 2025-02-17 at 9 22 13 AM"
src="https://github.com/user-attachments/assets/e03d7fd6-8795-4df2-9c8f-ffbcabd2d212"
/>
<img width="1575" alt="Screenshot 2025-02-17 at 9 24 33 AM"
src="https://github.com/user-attachments/assets/5bf963d4-6fb7-4586-8f6f-9e605c25bf06"
/>
## Checklist for submitter
- [x] Added/updated automated tests
- [ ] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
For #26218
Basic Android MDM on/off backend functionality. Manually tested.
The following env vars must be set:
```
FLEET_DEV_ANDROID_ENABLED=1
FLEET_DEV_ANDROID_SERVICE_CREDENTIALS=$(cat credentials.json)
FLEET_DEV_ANDROID_PUBSUB_TOPIC=projects/your-project/topics/your-topic
```
I picked https://github.com/go-json-experiment/json as the JSON library,
which seems like the safest option.
- will become json/v2 at some point
- currently used in production by other companies, like Tailscale
- well-maintained
- Some context here: https://github.com/fleetdm/fleet/issues/25512
Plan for next work:
- refactoring from 1st PR
- add pubsub with device enroll -> spec proxy for fleetdm.com
- come back to this sub-task to add tests and finish TODOs
# Checklist for submitter
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
> For #24222
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
Android scaffold code and refactorings
- Android packages intended to be decoupled from other Fleet code
Video explaining the PR: https://www.youtube.com/watch?v=cza-35Z9Wxk
# Checklist for submitter
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
> For #26177
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
For #25648
Fixed issue where `fleetctl gitops` was NOT deleting macOS setup
experience bootstrap package and enrollment profile. GitOps should clear
all settings that are not explicitly set in YAML config files.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #25770
We already unmarshal macOS/Windows settings (added by Martin), so we
replace the path with an absolute file path and keep them unmarshalled
so they don't have to be re-unmarshalled later. Note: the custom
UnmarshalJSON method on these structs checks for (and handles) legacy
format (before labels were added).
Also some refactorings:
- extracted `extractControlsForNoTeam`
- reorganized `TestGitOpsBasicGlobalAndNoTeam` with subtests -- I did
not actually change functionality of this test
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
for #24471
This PR adds missing support for updating a team's failing policy
webhook via GitOps:
* If `team_settings.webhook_settings.failing_policies_webhook` is not
set, the webhook will be disabled and all settings cleared.
* If `team_settings.webhook_settings.failing_policies_webhook` is set,
the webhook will be configured according to the specified settings.
A new test has been added to verify this functionality, and I tested it
locally using the `fleetctl gitops` command.
---------
Co-authored-by: Dante Catalfamo <43040593+dantecatalfamo@users.noreply.github.com>
For #24870 subtask
API changes doc: #25293
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [x] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
For #25160.
Measured improvement by splitting the MySQL query into two queries to
use the indexes more efficiently:
- ~8s vs ~100ms for ~30k entries in software_titles for ~1.7k incoming
software without bundle_identifier (linux software).
- ~1.64s vs ~2ms for 25k entries in software_titles and ~500 incoming
new software with bundle_identifier + ~200 new software without
bundle_identifier (macOS software).
---
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [X] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [X] Manual QA for all new/changed functionality
For #25533
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #23531.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
## For #25320
- [x] Added/updated automated tests
- [ ] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
# Changes
- orbit >= 1.38.0, when configured to connect to
https://tuf.fleetctl.com (existing fleetd deployments) will now connect
to https://updates.fleetdm.com and start using the metadata in path
`/opt/orbit/updates-metadata.json`.
- orbit >= 1.38.0, when configured to connect to some custom TUF (not
Fleet's TUFs) will copy `/opt/orbit/tuf-metadata.json` to
`/opt/orbit/updates-metadata.json` (if it doesn't exist) and start using
the latter.
- fleetctl `4.63.0` will now generate artifacts using
https://updates.fleetdm.com by default (or a custom TUF if
`--update-url` is set) and generate two (same file) metadata files
`/opt/orbit/updates-metadata.json` and the legacy one to support
downgrades `/opt/orbit/tuf-metadata.json`.
- fleetctl `4.62.0` when configured to use custom TUF (not Fleet's TUF)
will generate just the legacy metadata file
`/opt/orbit/tuf-metadata.json`.
## User stories
See "User stories" in
https://github.com/fleetdm/confidential/issues/8488.
- [x] Update `update.defaultRootMetadata` and `update.DefaultURL` when
the new repository is ready.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [X] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [X] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [X] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
For #24869
This subtask contains code to sign the CloudFront software installer and
bootstrap package URL using AWS SDK URL signer.
It works with the current bootstrap package delivery. For software
installers, fleetd will need to be modified to take advantage of this
URL in a future subtask (which will also include updated API contributor
docs).
My article on signed URLs, for context:
https://victoronsoftware.com/posts/cloudfront-signed-urls/
# Checklist for submitter
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
# Overview
This PR adds support for remote YARA queries to osquery-perf, so that
remote YARA queries can be load-tested.
# Details
The existing `runLiveQuery()` is updated to branch off into different
query running functions based on the content of the query. If the query
contains `from yara` and `sigurl`, then the new `runLiveYaraQuery()`
function is run which makes a request to the Fleet "get yara rules" API
before returning an appropriate response. Otherwise, the new
`RunLiveMockQuery()` function is run which includes the previous logic
for sending a mock response.
# Testing
I don't see any automated testing for osquery-perf, but I manually
tested in the following way:
1. Started osquery-perf with `go run agent.go`
2. Ran a live query on the new host using
```
SELECT * FROM yara where sigurl="https://localhost:8080/api/osquery/yara/rule1.yar"
```
and verified that the result was as-expected:
<img width="1642" alt="image"
src="https://github.com/user-attachments/assets/a2c9cacf-e28d-409c-8e83-1c82809b89c0"
/>
I also used a log in Fleet to verify that the "get yara rules" API was
really being called.
3. Ran another live query on the host using:
```
SELECT * FROM system_info"
```
and verified that the result was as expected:
<img width="1665" alt="image"
src="https://github.com/user-attachments/assets/a8d35389-f193-4902-badf-200d760fdf46"
/>
I also tested that sending a `sigurl` with the wrong host returns a
`live yara query failed because sigurl host did not match server
address` error
# Checklist for submitter
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
For #24868 (subtask)
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
#24900
This PR includes and depends on PR #25012, which should be
reviewed/merged before this one.
Windows profiles are not included in this PR due to issue #25030
This PR adds the following functionality: Apple config/DDM profile is
resent to the device when the profile contains secret variables, and the
values of those variables have changed. For example.
- Upload secret variables
- Upload profile
- Device gets profile
- Upload the same profile
- Nothing happens
- Upload a different secret variable value
- Upload the same profile
- Device gets updated profile
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Manual QA for all new/changed functionality
For #22875.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
For secrets subtask #24548
Fixed secret variables support in Apple configuration profiles.
# Checklist for submitter
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
for #24041
This PR addresses an issue that can cause Windows vulnerability checks
to fail (possibly causing false negatives). We determine whether a
vulnerability in an MSRC bulletin applies to any hosts in a Fleet
instance by attempting to matching the data in [each row of the
`operating_systems`
table](65e374c85c/cmd/fleet/cron.go (L297-L303))
with [at least one "product" in a
bulletin](e2d9a9016c/server/vulnerabilities/msrc/analyzer.go (L39)),
including [matching architecture and "display
version"](76f5baced9/server/vulnerabilities/msrc/parsed/product.go (L26-L39)).
However a subset of products listed in these bulletins do not include
the display version, so for example a host whose OS was listed as
`Microsoft Windows Server 2022 Datacenter 21H2` (21H2 being the "display
version") would match nothing in the bulletins because no listed Server
2022 products include "21H2" in their names.
The fix made here is to add relevant version info to the products list
when we do our ETL of the MSRC bulletins. The version info was gleaned
from https://en.wikipedia.org/wiki/List_of_Microsoft_Windows_versions.
We see logs related to this issue a lot, so cleaning this up will
alleviate some noise and infra costs as well.
## Addresses #21855 and all of its subtasks
**Frontend:**
- Update list queries API call to include pagination and filter-related
query params, including new `platform` param for filtering queries by
platforms they've been set to target
- Convert all filtering, sorting, and pagination functionality of the
Manage queries page from client-side to server-side
- Remove unneeded variable declarations / logic
- Various typing and naming improvements
**Server:**
- Add new `platform` `ListQueryOption`
- Update service and datastore level list queries logic to handle
filtering queries by targeted platform
- Update service and datastore level list queries logic to include
`meta` and `count` fields in addition to filtered/paginated queries
- [x] Changes file added for user-visible changes in `changes/`, `
- [x] Added/updated tests
- [x] update DB, integration
- [x] add integration (pagination)
- [x] add integration (platform filter)
- [x] add DB (pagination)
- [x] add DB (platform filter)
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
#23816
This fix may not completely fix the customer's issue. However, I'd like
to see if there are improvements from this fix combined with the
previous query optimization fix.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
#22790
Changes file is on the FE PR.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
for #22824
This PR splits up the `newMDMProfileManager` function into two functions
`newAppleMDMProfileManager` and `newWindowsMDMProfileManager`, the
latter containing the code to start the windows-specific profile
management job. This allows scheduling and scaling the Apple and Windows
profile management jobs separately.
Tested locally by checking that the jobs were scheduled at startup, and
then triggering the `mdm_apple_profile_manager` and
`mdm_windows_profile_manager` schedules via the `/trigger` API; nothing
blew up.
For #24132
Drops retry wait a bit as there doesn't seem to be a point in waiting
that long (I pulled feeds with a 10s retry interval in curl earlier).
Total time to pull feeds and build the CPE DB was ~4 hours this evening
(~6pm to ~10pm CST).
# Checklist for submitter
- [x] Manual QA for all new/changed functionality
Previously when passing labels to the query run endpoints that do not
exist, the labels would simply be ignored. Now the endpoint will return
an error indicating which labels are invalid. This change also affects
the `fleetctl query` command `--labels` flag.
https://github.com/fleetdm/fleet/issues/23015
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Ian Littman <iansltx@gmail.com>
## Addresses #22702, #23713, #23756, #23746, #23747, and #23876
_-Note that much of this code as is will render as expected only once
integrated with the backend or if manipulated manually for testing
purposes_
**Frontend**:
- Update banners on my device page, tests
- Build new logic for calling endpoint to trigger linux key escrow on
clicking `Create key`
- Add `CreateLinuxKeyModal` to inform user of next steps after clicking
`Create key`
- Update banners on host details page, tests
- Update the Controls > OS settings section with new logic related to
linux disk encryption
- Expect and include counts of Linux hosts in aggregate disk encryption
stats UI
- Add "Linux" column to the disk encryption table
- Show disk encryption related UI for supported Linux platforms
- TODO: confirm platform string matching functionality in manual e2e
testing
- Expand capabilities of `SectionHeader` component, apply to new UI
- Flash "missing private key" error, with clickable link, when trying to
update disk encryption enabled while no server private key is present.
- TODO: QA this once other endpoints on Controls > Disk encryption are
enabled even when MDM not turned on
- Update Disk encryption key modal copy
-Other TODO:
- Confirm when integrated with API:
- Aggregate disk encryption counts
- Disk encryption table Linux column
- Show disk encryption key action on host details page when expected
- Opens Disk encryption key modal, displays key as expected
**Backend**:
- For "No team" and teams, error when trying to update disk encryption
enabled while no server private key is present.
- Remove requirement of mdm being enabled for use of various endpoints
related to Linux disk encryption
- Update tests
_________
**Host details and my device page banners**
**Create key modal**
<img width="1799" alt="create-key-modal"
src="https://github.com/user-attachments/assets/81a55ccb-b6b9-4eb6-b2ff-a463c60724c0">
**Enabling disk encryption**
**Disk encryption: Fleet free**
<img width="1912" alt="free"
src="https://github.com/user-attachments/assets/9f9cace3-8955-47c2-87d9-24ff9387ac1a">
**Custom settings: turn on MDM**
<img width="1912" alt="turn on mdm"
src="https://github.com/user-attachments/assets/4d3ad47b-4035-4d93-86f0-dc2691b38bb4">
**Device status indicators**
**Encryption key action and modal**
- [x] Changes file added for user-visible changes in `changes/`
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
- [ ] Full e2e testing to do when integrated with backend
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
Co-authored-by: Ian Littman <iansltx@gmail.com>
#22810
# Demo
[](http://www.youtube.com/watch?v=le71QQ92suc)
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [x] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
#19696
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Manual QA for all new/changed functionality
> Related issue: #22359
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
Implements the Fleet side of #14899
- Add new endpoints to update and retrieve yara rules
- Add support in fleetctl for applying the rules
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements)
- [ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for new osquery data ingestion features.
- [x] Added/updated tests
- [ ] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes
- [x] If database migrations are included, checked table schema to confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will auto-update timestamps during migration.
- [ ] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
#22187
Similar fix to #22555: resolve paths at spec parsing time rather than
when trying to grab files
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
#23258 (see
[comment](https://github.com/fleetdm/fleet/issues/23258#issuecomment-2443304838)
for rationale)
Validated by removing the two places that would create the directory
(early in scanVulnerabilities in cron.go, partway through download in
download.go) and ensuring the test failed (timeout after 10s).
Both dir creations happen early in the vulns cron so I was able to
drastically tighten the timing on the periodic check on this test, so
this tests completes way quicker than before as an added benefit
(automatic test parallelism notwithstanding).
The panic recovery here theoretically shouldn't be necessary, as on a
passed test the context will get cancelled while syncing the CPE sqlite,
but is included to ensure the test doesn't flake if the implementation
of the vulnerabilities cron changes such that we _would_ get a panic by
cancelling the context this early.
# Checklist for submitter
- [x] Added/updated tests
#22740
Full QA is still a WIP but this is ready for review.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [x] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
#21715
This fix is needed for https://github.com/fleetdm/fleet-gitops CI to run
successfully against a non-premium Fleet server.
That CI test will cover this change.
# Checklist for submitter
- [x] Manual QA for all new/changed functionality
#23050
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
makes a fix to the enroll byod handler. The hander use to be set once
when the server started but this lead to a user receiving a 404 page
after the fleet instance setup was complete but was not restarted.
now the `/enroll` handler will check for the setup requirement itself.
There is some error handling when the setup is required. if the fleet
instance has been setup, they will get the enroll byod page.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
For #21955 (the story has a video demo of core functionality)
Follow up for PR #22542
# Checklist for submitter
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
For #22787
Also revises the spec check to explain that scripts have to be defined
"controls" when used in policies for the same team, with an explicit
call-out for no-team.yml since this fix doesn't support pulling scripts
from the global file. This is because parsing and script-matching
happens early enough that we can't throw an error in the part of the
code where we bail when controls are defined in both no-team and default
files.
To minimize diff size, we're both "passing-by-ref" and returning the
maps-by-team of scripts and installers, though the former would be
sufficient on its own.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- N/A Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests (sorta)
- [x] Manual QA for all new/changed functionality
