Found while working on #12696.
This was caught be a recent check added by @mostlikelee to `fleetctl
apply` (#13294).
Sample error:
```sh
$ fleetctl apply --context loadtest -f ee/cis/win-10/cis-policy-queries.yml
Error: applying policies: policy names must be globally unique. Please correct policy "CIS - Ensure 'Windows Firewall: Public: Firewall state' is set to 'On (recommended)'\n" and try again.
```
#11939
- This PR fixes typos in three CIS Windows queries (the queries were
failing with `invalid SQL syntax`).
- Also adds tooling to perform similar testing that we ran for macOS
(using `fleetd_tables` as an extension).
changelog for the version bump can be found here:
https://github.com/macadmins/osquery-extension/releases/tag/v0.0.15
related to #13158 as this fixes a bug in the extension causing `profiles
show --type enrollment` to be run almost every time we queried something
from the `mdm` table.
I couldn't find any new tables. Other than that, some dependencies were
updated on their repo as well, most notably `osquery/osquery-go`
relates to #12986
This is a fix for a case where the `os_versions` endpoint was returning
a 404 when searching for `no teams`.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
related to #12483, we have found out that in distributed scenarios, the
URL of the Puppet server used for the request is appended to the
identifier, and it can be different between `/preassign` and `/match`
calls.
to account for this, we're only grabbing the first 36 characters of the
identifier.
Prior to 4.35.0, some rows in the scheduled_query table might have a
`NULL` value due to a race condition with database replicas and the way
`ds.EnsureGlobalPack` and `ApplyPackSpecs` work together.
This is no longer the case, but some databases are left in weird states,
which were not accounted by this migration.
Chaning the migration in-place because that's the approach we took in
previous migrations with similar problems.
relates to #12907
implement adding an enroll secret when creating a new team with
`fleetctl apply` if none is provided.
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Added/updated tests
- [x] Manual QA for all new/changed functionality
This relates to #12600
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
- [X] Manual QA for all new/changed functionality
## Addresses #12999
<img width="1282" alt="Screenshot 2023-07-27 at 11 59 01 AM"
src="https://github.com/fleetdm/fleet/assets/61553566/b60d3b41-3d7b-4550-ba7c-8615bae085a6">
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
## Addresses #12646
### See issue for list of completed work
### Also see PR #12713 **notes for review** on that PR for help manually
testing this work in lieu of the completed API.
- [x] Changes file added for user-visible changes in `changes/`
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
## Addresses #12636
### See issue for list work done
### Notes for review:
- Because other work is based on this branch, TODOs / fixes are noted
here until the team comes to a strategy for merging all of the work:
- Add missing space in the Performance impact column "Undetermined"
tooltip text
- I'm having trouble confirming that the inherited queries table is
working right with the mock hard-coded data, though I did see it working
correctly previously. There's an issue with the page reverting to "All
teams" when trying to show the inherited table, though it does show the
table before re-rendering.
- This work is organized clearly by commit, so that might be a
manageable way to go through this code.
- Since the updated API for this work is not yet complete, this work can
be manually tested by either:
- Using mock API infrastructure, or
- in `ManageQueriesPage.tsx`, comment out the two `useQuery` calls and
add appropriate mock data. You can then modify any fields of interest to
test their related UI functionality. For example, lines 119 -242 might
read:
```
// const {
// data: curTeamEnhancedQueries,
// error: curTeamQueriesError,
// isFetching: isFetchingCurTeamQueries,
// refetch: refetchCurTeamQueries,
// } = useQuery<IListQueriesResponse, Error, IEnhancedQuery[]>(
// [{ scope: "queries", teamId: teamIdForApi }],
// () => queriesAPI.loadAll(teamIdForApi),
// {
// refetchOnWindowFocus: false,
// enabled: isRouteOk,
// select: (data) => data.queries.map(enhanceQuery),
// }
// );
// // If a team is selected, fetch inherited global queries as well
// const {
// data: globalEnhancedQueries,
// error: globalQueriesError,
// isFetching: isFetchingGlobalQueries,
// refetch: refetchGlobalQueries,
// } = useQuery<IListQueriesResponse, Error, IEnhancedQuery[]>(
// [{ scope: "queries", teamId: -1 }],
// () => queriesAPI.loadAll(),
// {
// refetchOnWindowFocus: false,
// enabled: isRouteOk && isAnyTeamSelected,
// select: (data) => data.queries.map(enhanceQuery),
// }
// );
const [
curTeamEnhancedQueries,
curTeamQueriesError,
isFetchingCurTeamQueries,
refetchCurTeamQueries,
] = useMemo(() => {
return [
[
{
created_at: "2023-06-08T15:31:35Z",
updated_at: "2023-06-08T15:31:35Z",
id: 2,
name: "test",
description: "",
query: "SELECT * FROM osquery_info;",
team_id: 43,
platform: "darwin",
min_osquery_version: "",
automations_enabled: true,
logging: "snapshot",
saved: true,
// interval: 300,
interval: 0,
observer_can_run: false,
author_id: 1,
author_name: "Jacob",
author_email: "jacob@fleetdm.com",
packs: [],
stats: {
// system_time_p50: 1,
// system_time_p95: null,
// user_time_p50: 1,
// user_time_p95: null,
// total_executions: 1,
},
performance: "Undetermined",
platforms: ["darwin"],
},
] as IEnhancedQuery[],
undefined,
false,
() => {
console.log("got the new queries");
},
];
}, []);
const [
globalEnhancedQueries,
globalQueriesError,
isFetchingGlobalQueries,
refetchGlobalQueries,
] = useMemo(() => {
return [
[
{
created_at: "2023-06-08T15:31:35Z",
updated_at: "2023-06-08T15:31:35Z",
id: 200,
name: "test",
description: "",
query: "SELECT * FROM osquery_info;",
team_id: null,
platform: "darwin",
min_osquery_version: "",
automations_enabled: true,
logging: "snapshot",
saved: true,
// interval: 300,
interval: 0,
observer_can_run: false,
author_id: 1,
author_name: "Jacob",
author_email: "jacob@fleetdm.com",
packs: [],
stats: {
// system_time_p50: 1,
// system_time_p95: null,
// user_time_p50: 1,
// user_time_p95: null,
// total_executions: 1,
},
performance: "Undetermined",
platforms: ["darwin"],
},
] as IEnhancedQuery[],
undefined,
false,
() => {
console.log("got the new inherited queries");
},
];
}, []);
```
- [x] Changes file added for user-visible changes in `changes/`
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
Related to #12608, this automatically sets the
`DeferForceAtUserLoginMaxBypassAttempts` property to `1` on the
FileVault profile that's generated by Fleet.
This changeset also includes a migration to modify old FileVault
profiles that already exist in the database, and by virtue of that a
`InstallProfile` command will be issued to hosts that already have FV
enabled. During testing we found:
1. This doesn't affect users with FV already installed, they silently
get the profile updated without any changes.
2. Since the profile needs to be re-delivered, it'll go through the full
"pending" -> "verifying" -> "verified" cycle.
This is related #12701
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
- [X] Manual QA for all new/changed functionality
This relates to #12263
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added/updated tests
---------
Co-authored-by: Roberto Dip <me@roperzh.com>
#10292, #12554
When scanning tens of thousands of files for permissions, using the
`find` command exposed as a fleetd table is more performant than trying
to use the `file` table. This change caused the watchdog to *stop*
killing osquery because of exceeding memory or CPU limit.
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [X] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
relates to #12289
Implements enable and disable Windows mdm activity UI
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
- [x] Manual QA for all new/changed functionality
## Addresses #11355, pt.1 (pt. 2 already fixed)
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
#10292
The query was processing *every* file under `/Applications/`, which
makes it super expensive both in CPU usage and Memory footprint. This
query was the main culprit of triggering worker process kills by the
watchdog.
On some runs it triggered CPU usage alerts:
```
7716:W0623 15:38:05.402959 221732864 watcher.cpp:415] osqueryd worker (72976) stopping:
Maximum sustainable CPU utilization limit 1200ms exceeded for 12 seconds
```
And on other runs it triggered memory usage alerts:
```
4431 W0626 07:28:50.868021 147312640 watcher.cpp:424] osqueryd worker (21453) stopping:
Memory limits exceeded: 214020096 bytes (limit is 200MB)
```
For the above logs I used a custom osqueryd branch to be able to print
more information: https://github.com/osquery/osquery/pull/8070
The metrics for the old query were CPU usage: ~4521 ms
```
435:level=warn ts=2023-06-26T09:58:29.665712Z query=fleet_policy_query_1233 queryTime=4521 memory=12226560 msg="distributed query performance is excessive" hostID=308 platform=darwin
```
With the new query, CPU usage: ~210 ms.
```
23893:level=debug ts=2023-06-26T18:06:08.242456Z query=fleet_policy_query_1233 queryTime=210 msg=stats memory=0 hostID=308 platform=darwin
```
Basically a ~20x improvement.
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- ~[ ] Added/updated tests~
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
## Addresses #11037
### Implement the `privacy_preferences` table for the Fleetd Chrome
extension. Columns correspond to the available properties of
[`chrome.privacy`](https://developer.chrome.com/docs/extensions/reference/privacy/).
Chrome on mac:
<img width="816" alt="Screenshot 2023-06-23 at 11 55 21 AM"
src="https://github.com/fleetdm/fleet/assets/61553566/a4700749-6325-442e-acf2-c14b1c9adf8f">
Chromebook with enterprise access (actual use case):
* Chromebook w/o enterprise access: as you can see, sometimes certain
APIs are not available - this error occurs because the expected API
object that would have a `get` method is actually `undefined` TODO – How
to handle this case given that we want to let errors bubble up to the
level at which Fleet can catch them? Maybe it would be nice to catch
such errors and send them up to the Fleet layer, and still allow the
loop to continue to populate the columns whose APIs _are_ available.
_Decision: catch API errors here to preserve functionality of the
remaining columns_
- [x] Changes file
- [x] Manual QA
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
This PR requires the Windows MDM configuration changes - This will be
updated next week
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- [x] Documented any permissions changes
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
relates to #11932
This improves the UI error messaging for AMB 400 errors
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
Issue reported by @jarodreyes.
The user was not informed that `fleetctl login` was using the PASSWORD
environment variable:
`main`:
```sh
export PASSWORD=wrong
fleetctl login
Log in using the standard Fleet credentials.
Email: a@b.c
Error: Login failed: login received status 401 Authentication failed: Authentication failed
```
And with the changes on this PR:
```
export PASSWORD=wrong
fleetctl login
Log in using the standard Fleet credentials.
Email: a@b.c
Using value of environment variable $PASSWORD as password.
Error: Login failed: login received status 401 Authentication failed: Authentication failed
```
## Addresses
[confidential/2940](https://github.com/fleetdm/confidential/issues/2940)
Patched a potential security issue in UI
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
#11266
PS: I first attempted a serialization trick by introducing a new
`appConfigResponse` and implementing `json.Marshal` to exclude these
fields but it was too hacky and hard to maintain moving forward, so I'm
bitting the bullet now. Happy to hear other ideas.
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
## Addresses #11828
- [x] Add ChromeOS platform filter
- [x] Increase dropdown widths to 180px for screen >1100px of:
- [x] platform dropdown
- [x] Labels dropdown (for consistency)
- [x] Add new null empty cell value “Not supported” for Chromebooks
- [x] Apply to the following columns:
- [x] Disk space available
- [x] MDM status
- [x] MDM server URL
- [x] Last restarted
- [x] Fix a misaligned icon in the labels dropdown, other small fixes
<img width="1184" alt="Screenshot 2023-06-06 at 7 09 26 PM"
src="https://github.com/fleetdm/fleet/assets/61553566/ee6fb72c-f66e-44a4-a321-8eaefff3b70b">
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
for #11257, h/t to @mna for the idea of resetting `token_update_tally`.
this is to cover scenarios where a host might be re-enrolling (eg: the
device has been wiped) but we don't know about it.
since `TokenUpdate` might be called multiple times during the lifecycle
of an MDM enrollment, we add a check on the value of
`nano_enrollments.token_update_tally`. For the scenarios described
above, the tally is still `> 0` even thought the host is enrolling for
the first time.
to mitigate this, we reset its value to 0 when we receive an
`Authenticate` message (which only happens only per enrollment)
I set the value to `0` because it's incremented to `current_value+1` by
nanomdm before calling our handler.
## Addresses #11825
- [x] Add ChromeOS to Dashboard page: <img width="1365" alt="Screenshot
2023-06-02 at 4 01 12 AM"
src="https://github.com/fleetdm/fleet/assets/61553566/e846c4b6-5fcb-4847-af05-67b2237ada39">
- [x] Add to platforms dropdown, confirm order of platform options, add
route
- [x] Hosts summary card
- [x] Add responsiveness for <980px <img width="952" alt="Screenshot
2023-06-02 at 4 02 44 AM"
src="https://github.com/fleetdm/fleet/assets/61553566/93662957-c590-40e0-876d-6ce4adabad2b">
- [x] TODO: Confirm label number of chrome hosts label - ask Juan on
[this issue](https://github.com/fleetdm/fleet/issues/11829) - needed to
call an API to get this id
- [x] Missing hosts card (didn’t need any changes)
- [x] Low disk space hosts card (Not supported)
- [x] Operating systems card
**Note for reviewers:** There is an API call happening from the
HostsSummary component to get the id for the ChromeOS label needed for
the URL to the filtered manage hosts page. This feature working properly
depends on the response from that endpoint, which is WIP. UPDATE 6/5 -
the endpoint is now working and being called correctly, though the id
being returned is WIP (backend). No need to replace anything to test.
## Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
## Addresses #11856
Improve performance of the rendering of live query results by:
- rendering the table on a set interval instead of with each new result
- preventing redundant rerenders of various sorts
Partial run, with memory leak:
<img width="2552" alt="partial run with memory leak, after smaller
optimizations, before debouncing queryResults"
src="https://github.com/fleetdm/fleet/assets/61553566/5288bffb-6940-43da-9083-59adb4a25916">
Full run after debounce, no memory leak (10x improvement of max JS heap
size):
<img width="2559" alt="full run after debounce, no memory leak"
src="https://github.com/fleetdm/fleet/assets/61553566/be056610-e7a5-4289-a433-1070cf016e83">
**NOTE** - there are further optimizations to try on this page, and the
debounce interval can potentially be shortened to improve UX. In
experimenting with that, it's not immediately clear what a good balance
of UX / performance is. Since the customer seems keen to solve this, I
think we should merge as-is and send them a demo build to confirm this
fixes their problem, then iterate once they've confirmed it does.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
Co-authored-by: Sarah Gillespie <sarah@fleetdm.com>
for #10605, this modifies the cron used to ping the list/sync devices
API from ABM to account for the "deleted" and "modified" operation
types.
We know that:
1. Sometimes, Apple sends a "modified" operation type when a device's
MDM server is reassigned in ABM, up until now, we were ignoring these
devices.
2. Devices that are no longer assigned to Fleet in ABM can't be
migrated.
relates to #11238
This implements the Verified status for the profile statute on the macOS
settings pages and the Host Details and My Device pages.
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
- [x] Manual QA for all new/changed functionality
- Add section for viewing recent MDM commands using fleetctl get mdm-commands
- Add example output for fleetctl get mdm-commands and fleetctl get mdm-command-results
This could help future users to detect this issue: #10957
It also adds an error log in Fleet that prints the actual error.
The error is displayed if I kill Redis during a live session or if I set
`client-output-buffer-limit` to something real low like `CONFIG SET
"client-output-buffer-limit" "pubsub 100kb 50kb 60"`:
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- ~[ ] Added/updated tests~
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
For issues #12003 and #12051
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
#10784
The removal of the now deprecated `sso_settings.enable_jit_role_sync`
config will be tackled in: #10688.
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
#10878
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
- Added index on `created_at` which is the sort key used when loading the log activities widget on the dashboard.
- Refactored query used when loading activities to avoid a full table scan.
This was found while working on #10957.
When running a live query, a lot of unused host data is stored in Redis
and sent on every live query result message via websockets. The frontend
and fleetctl just need `id`, `hostname` and `display_name`. (This
becomes worse every time we add new fields to the `Host` struct.)
Sample of one websocket message result when running `SELECT * from
osquery_info;`:
size in `main`: 2234 bytes
```
a["{\"type\":\"result\",\"data\":{\"distributed_query_execution_id\":57,\"host\":
{\"created_at\":\"2023-05-22T12:14:11Z\",\"updated_at\":\"2023-05-23T12:31:51Z\",
\"software_updated_at\":\"0001-01-01T00:00:00Z\",\"id\":106,\"detail_updated_at\":\"2023-05-23T11:50:04Z\",
\"label_updated_at\":\"2023-05-23T11:50:04Z\",\"policy_updated_at\":\"1970-01-02T00:00:00Z\",
\"last_enrolled_at\":\"2023-05-22T12:14:12Z\",
\"seen_time\":\"2023-05-23T09:52:23.876311-03:00\",\"refetch_requested\":false,
\"hostname\":\"lucass-macbook-pro.local\",\"uuid\":\"BD4DFA10-E334-41D9-8136-D2163A8FE588\",\"platform\":\"darwin\",\"osquery_version\":\"5.8.2\",\"os_version\":\"macOS 13.3.1\",\"build\":\"22E261\",\"platform_like\":\"darwin\",\"code_name\":\"\",
\"uptime\":91125000000000,\"memory\":34359738368,\"cpu_type\":\"x86_64h\",\"cpu_subtype\":\"Intel x86-64h Haswell\",\"cpu_brand\":\"Intel(R) Core(TM) i7-1068NG7 CPU @ 2.30GHz\",\"cpu_physical_cores\":4,\"cpu_logical_cores\":8,\"hardware_vendor\":\"Apple Inc.\",\"hardware_model\":\"MacBookPro16,2\",\"hardware_version\":\"1.0\",
\"hardware_serial\":\"0DPQR4HMD1FZ\",
\"computer_name\":\"Lucas’s MacBook Pro\",\"public_ip\":\"\",
\"primary_ip\":\"192.168.0.230\",\"primary_mac\":\"68:2f:67:8e:b6:1f\",
\"distributed_interval\":1,\"config_tls_refresh\":60,\"logger_tls_period\":10,\"team_id\":null,
\"pack_stats\":null,\"team_name\":null,
\"gigs_disk_space_available\":386.23,\"percent_disk_space_available\":40,
\"issues\":{\"total_issues_count\":0,\"failing_policies_count\":0},
\"mdm\":{\"enrollment_status\":null,\"server_url\":null,\"name\":\"\",\"encryption_key_available\":false},
\"status\":\"online\",\"display_text\":\"lucass-macbook-pro.local\",\"display_name\":\"Lucas’s MacBook Pro\"},
\"rows\":[{\"build_distro\":\"10.14\",\"build_platform\":\"darwin\",
\"config_hash\":\"b7ee9363a7c686e76e99ffb122e9c5241a791e69\",\"config_valid\":\"1\",
\"extensions\":\"active\",\"host_display_name\":\"Lucas’s MacBook Pro\",
\"host_hostname\":\"lucass-macbook-pro.local\",\"instance_id\":\"cde5de81-344b-4c76-b1c5-dae964fdd4f2\",\"pid\":\"8370\",\"platform_mask\":\"21\",\"start_time\":\"1684757652\",
\"uuid\":\"BD4DFA10-E334-41D9-8136-D2163A8FE588\",
\"version\":\"5.8.2\",\"watcher\":\"8364\"}],\"error\":null}}"]
```
vs. size of the message result on this branch: 675 bytes
```
a["{\"type\":\"result\",\"data\":{\"distributed_query_execution_id\":59,
\"host\":{\"id\":106,\"hostname\":\"lucass-macbook-pro.local\",
\"display_name\":\"Lucas’s MacBook Pro\"},
\"rows\":[{\"build_distro\":\"10.14\",\"build_platform\":\"darwin\",
\"config_hash\":\"f80dee827635db39077a458243379b3ad63311fd\",
\"config_valid\":\"1\",\"extensions\":\"active\",\"host_display_name\":\"Lucas’s MacBook Pro\",
\"host_hostname\":\"lucass-macbook-pro.local\",
\"instance_id\":\"cde5de81-344b-4c76-b1c5-dae964fdd4f2\",\"pid\":\"8370\",\"platform_mask\":\"21\",
\"start_time\":\"1684757652\",\"uuid\":\"BD4DFA10-E334-41D9-8136-D2163A8FE588\",\"version\":\"5.8.2\",
\"watcher\":\"8364\"}]}}"]
```
Manual tests included running with an old fleetctl running with a new
fleet server, and vice-versa, a new fleetctl running against an old
fleet server.
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
## Addresses #11737
- Write function to enforce Fleet sentence-casing standards
- Apply it to this bug
<img width="642" alt="Screenshot 2023-05-18 at 12 43 20 PM"
src="https://github.com/fleetdm/fleet/assets/61553566/670f4f8d-1c23-4609-bb23-c38038e9bbd8">
*NOTE - this (the host details) endpoint currently returns label names
in Sentence Case – this solution deals with only the UI presentation,
but it might be worth changing the API response in the future:
<img width="369" alt="Screenshot 2023-05-18 at 12 48 58 PM"
src="https://github.com/fleetdm/fleet/assets/61553566/27236524-9c0a-4818-8a74-f445b5765d94">
## Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
## Addresses #11714
- Allow saving policy automations to Ticket integrations without
defining a webhook URL
https://www.loom.com/share/208169b6081846809dc170fcfb43f689
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
relates to #11048
improves the loading UI for the disk encryption page. The aggregate
profile summary no longer pops in and pushed down disk encryption
section.
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
relates to #9996
Added support for wildcard search on host search.
say for example you have the following hosts:
```
+------------------+
|hostname |
+------------------+
|Molly‘s MacbookPro|
|Molly's MacbookPro|
|Molly‘s MacbookPro|
|Molly❛s MacbookPro|
|Molly❜s MacbookPro|
|Alex's MacbookPro |
+------------------+
```
searching for `Molly's` yields just the single host, but searching for
`Molly❜s` will perform a broader wildcard search using the literal `_`
character to match any character _in that position_.
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added/updated tests
## Addresses #11338
- Validate emails on login page
- Fix jumping error state for no email provided ("Email field must be
completed")
- Fix jumping error state for password field
- Fix jumping error state for Forgot password > email field
https://www.loom.com/share/92a238fcd2614d6e8d2655d571aa2757
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
relates to #11450
This will show the profile status aggregate UI at all times when on the
macOS settings page. This is a change from showing it conditionally.
This also cleans up where some of the requests occur to move it closer
to where it is needed and changing the `MdmProfileStatus` enum to a
union.
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
## Addresses #9371
### Adds a suite of UI logic for premium features in the Sandbox
environment
For reviewer: please review the work for the below 3 substasks, which
are the only remaining subtasks encompassed by this PR that have not yet
passed review individually:
- #10822 (9)
- #10823 (10)
- #10824 (11)
## Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
## Addresses #11188
When an _already authenticated_ no-access user tries to access any
authenticated routes:
- Log the user out
- Display the 403 'Forbidden' error page
https://www.loom.com/share/358fd5b534984ab9ab40220986a7d094
The user _can_ still log in – see attached issue.
## Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
#11089
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- [X] Documented any permissions changes
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [x] Added/updated tests
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
## Addresses #11088
Updated solution:
https://www.loom.com/share/e3ca30bebde54df5b8cbd8cac6d26780
- First refresh displays the bug: a white bar at the top of the page as
the login page loads.
- Second refresh displays the login page with the bug gone
- Navigate at the end to an error page to confirm it is unaffected,
since the initial solution affected the background color of those pages
## Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
In #10338 we introduced logic to gate DEP profiles behind Okta auth
using the ROP flow.
We're not going to use that, and instead we're going to gate profiles
behind SSO, which can be used from multiple providers and supports SSO
(the initial motivation behind the ROP flow was to create a local user
account.)
This removes some of the old code, which was never used in
production/documented for the public to use.
At the moment I'm leaving the `mdm_idp_accounts` table and related
methods untouched, as it's unclear yet if we're going to need a similar
auxiliar table, and I would rather deal with the migrations all at once.
#10880
I was not able to reproduce other 500s in `/api/_version_/fleet/hosts`
other than the one fixed in the PR.
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
## Addresses #10913
- Fixes the bug(s) outlined in the issue
- Improves look and feel of the tabbing experience throughout the setup
flow: https://www.loom.com/share/c482317d18314e629b7e5ebab7fd2840
## Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
#10798
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
#8593
This PR adds a new role `gitops` to Fleet.
MDM capabilities for the role coming on a separate PR. We need this
merged ASAP so that we can unblock the UI work for this.
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- [X] Documented any permissions changes
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [x] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
## Addresses #10756https://www.loom.com/share/b5ba605b892c4c789a73f05d160819cc
- Set the teams dropdown's max-height to be 66% of the viewport height
- Made a small script to automate creation of an arbitrary number of
teams
- [x] Changes file added for user-visible changes in `changes/`
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
This should get version numbers for more apps on macOS. Notably,
1Password includes helper apps that were getting vulnerability false
positives because we were not picking up the versions.
Addresses #10702.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
#8593
This PR adds a new role `observer_plus` to Fleet. (The `GitOps` role
will be added on a separate PR.)
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- [X] Documented any permissions changes
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
https://github.com/fleetdm/confidential/issues/1968
It's ready for review but I still need to load test this.
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
relates to #9436
Implementation of the API supporting filtering host by disk encryption
status. This adds this through a `macos_settings_disk_encryption` query
param that can be passed to these endpoints:
`GET /hosts`
`GET /hosts/count`
`GET /lables/:id/hosts`
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
Related to #9459, this adds logic to the cron to add a
`com.fleetdm.fleetd.config` configuration profile to the
`apple_mdm_configuration_profiles` table.
As noted in the comments, this makes some assumptions:
- This profile will be applied to all hosts in the team (or "no team",)
but it will only be used by hosts that have a fleetd installation
without
an enroll secret and fleet URL (mainly DEP enrolled hosts).
- Once the profile is applied to a team (or "no team",) it's not removed
if
AppConfig.MDM.AppleBMDefaultTeam changes, this is to preserve existing
agents using the configuration (mainly ServerURL as EnrollSecret is used
only during enrollment)
## Addresses #9988
* Adjust copy in tooltips to take up less width
* Refactor table headers to take an optional "isLastColumn" property
that is set to true when that header is in the last column.
* Use above property in conjunction with presence of TooltipWrapper as a
value for the header cell to add a class specific to that state.
* Use that class to adjust the location of the tooltip text and the
min-width of the column to avoid the bug.
The 3 states which exhibited this bug, now fixed:
<img width="1496" alt="Screenshot 2023-03-27 at 4 36 01 PM"
src="https://user-images.githubusercontent.com/61553566/228091971-4d5d034d-55c5-4921-955a-4946119f7785.png">
<img width="1496" alt="Screenshot 2023-03-27 at 4 36 12 PM"
src="https://user-images.githubusercontent.com/61553566/228091968-adf90b32-8fd2-45d9-b56d-a64c654151ef.png">
<img width="1496" alt="Screenshot 2023-03-27 at 4 36 24 PM"
src="https://user-images.githubusercontent.com/61553566/228091962-ff626daa-b13d-4093-b34f-de704b820161.png">
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
relates to #10786
This fixes an issue where users would see the incorrect disk encryption
banners on the my device page. This included a change to the ingestion
logic of the `directIngestDiskEncryptionKeyDarwin` method to take into
account if a host was already being encrypted with filevault locally.
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
Related to #10787, this tries to find in the tables with High likelihood
described in the issue.
This successfully accounts for unique keys that contain leading/trailing
whitespace and are using a collation with a pad attribute set to `NO
PAD` (considers whitespace as any other character instead of ignoring
it)
I haven't found a way to successfully detect the same scenario for
special unicode characters, for example:
```
mysql> SELECT TABLE_NAME, TABLE_COLLATION FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 'software';
+------------+--------------------+
| TABLE_NAME | TABLE_COLLATION |
+------------+--------------------+
| software | utf8mb4_general_ci |
+------------+--------------------+
1 row in set (0.01 sec)
mysql> select vendor COLLATE utf8mb4_unicode_ci from software where name = 'zchunk-libs' GROUP BY vendor COLLATE utf8mb4_unicode_ci;
+-----------------------------------+
| vendor COLLATE utf8mb4_unicode_ci |
+-----------------------------------+
| vendor |
| vendor? |
+-----------------------------------+
2 rows in set (0.01 sec)
mysql> ALTER TABLE `software` CONVERT TO CHARACTER SET `utf8mb4` COLLATE `utf8mb4_unicode_ci`;
ERROR 1062 (23000): Duplicate entry 'zchunk-libs-1.2.1-rpm_packages--vendor\2007-x86_64' for key 'unq_name'
```
> **Note** that `?` in "vendor?" is an unicode character
This only applies to Premium users, we want to show the vulnerabilities' published date anywhere vulnerabilities are shown including API endpoints and third party integrations.
#9609
This PR also fixes#10777.
The issue is: We were using `svc.AppConfig` instead of
`svc.ds.AppConfig` to retrieve the SMTP credentials.
`svc.AppConfig` obfuscates credentials, whereas `svc.ds.AppConfig` does
not.
To help prevent this from happening again I've renamed `svc.AppConfig`
to `svc.AppConfigObfuscated`.
I've also added a new test SMTP server
(https://github.com/axllent/mailpit) that supports Basic Authentication
and tests that make use of it to catch these kind of bugs (the tests are
executed when running `go test` with `MAIL_TEST=1`).
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
relates to #9434
implements the `GET /fleet/mdm/apple/filevault/summary` aggregate
endpoint.
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
---------
Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
closes https://github.com/fleetdm/fleet/issues/10778
---------
Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
## Addresses #10257
Removed the 'Issuer URI' field and its associated code. Manually
confirmed that enabling single sign-on still works as expected.
<img width="525" alt="Screenshot 2023-03-24 at 4 32 56 PM"
src="https://user-images.githubusercontent.com/61553566/227661519-c2684a68-8b66-48f9-a6ab-a24f02f07080.png">
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
#8957
To test this feature, build+run Fleet and then visit:
`https://localhost:8080/metrics`.
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [ ] Added/updated tests
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
# Addresses #10038
- Add logic to ensure consistent light-grey coloring of text cells using
DEFAULT_EMPTY_VALUE
<img width="622" alt="Screenshot 2023-03-22 at 4 06 30 PM"
src="https://user-images.githubusercontent.com/61553566/227058308-2c35e0b3-7017-4a0d-9e60-d03d46194f55.png">
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
## Addresses #9834
<img width="1215" alt="added date to vuln table"
src="https://user-images.githubusercontent.com/61553566/226730586-4165f5c9-2a42-4378-b58b-7900838a8707.png">
## Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
related to #10441, inspired by the prior work done in
https://github.com/kolide/fleet/pull/1360, this PR:
1. Adds a migration to use `utf8mb4_general_ci` as the default collation
for the database and all the tables. From [MySQL's documentation][1]:
> To change the table default character set and all character columns
> (CHAR, VARCHAR, TEXT) to a new character set, use a statement like
> this:
>
> ```
> ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name;
> ```
> The statement also changes the collation of all character columns. If
> you specify no COLLATE clause to indicate which collation to use, the
> statement uses default collation for the character set.
2. Changes the connection settings to use `utf8mb4_general_ci` as the
default collation, from the [driver docs][2]:
> Sets the collation used for client-server interaction on
connection. In contrast to charset, collation does not issue additional
queries. If the specified collation is unavailable on the target server,
the connection will fail.
[1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html
[2]: https://github.com/go-sql-driver/mysql
**TODO:** discuss how we can enforce this, is setting the database
default collation enough? should we add some kind of custom lint rule to
all migrations?
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
## Addresses #9406
Premium:
<img width="592" alt="Screenshot 2023-03-09 at 4 26 50 PM"
src="https://user-images.githubusercontent.com/61553566/224191932-bdb5934e-72ff-4ea2-b572-308bce93cecd.png">
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
Fixes broken data collection (e.g. disk space, IP addresses, installed
Python and RPM packages) and "unrecognized platform" error for those
distros.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
#9132
The actual fix for the empty hosts is adding the `--database_path`
argument in the initial `osqueryd -S` invocation when retrieving the
UUID. Osquery attempts to retrieve the UUID from OS files/APIs, when not
possible (which is what happens on some linux distributions), then it
resorts to generating a new random UUID and storing it in the
`osquery.db`. The issue was Orbit's first invocation of `osqueryd -S`
was not using the same `osquery.db` as the main daemon invocation of
`osqueryd`.
I'm also adding a `hostname` + `platform` to the orbit enroll phase so
that if there are any issues in the future we can avoid the "empty" host
and have some information to help us troubleshoot.
## How to reproduce
On Linux, osquery reads `/sys/class/dmi/id/product_uuid` to load the
hardware UUID.
Some Linux distributions running on specific hardware or container
environments do not have such file available.
The way to reproduce on a Linux VM is to do the following:
```sh
$ sudo su
# chmod -r /sys/class/dmi/id/product_uuid
```
which will turn the file inaccessible to root.
## Checklist
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [X] Added/updated tests
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
Related to https://github.com/fleetdm/fleet/issues/10299, this allows to
start the Fleet server with MDM enabled without having to provide ABM
configs.
I have tested this with:
1. Premium account, no ABM config: the server starts normally, but
without ABM features
2. Premium account, invalid ABM config: error starting the server
3. Premium account, valid ABM config: ABM features enabled
4. Free account, no ABM config: the server starts normally
5. Free account, any ABM config: error due to invalid license
#9486
Now cron jobs should recover from a Fleet outage after ~ two hours.
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- ~[ ] Added/updated tests~
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
#9515
Sample output after running `fleetctl trigger --name
cleanups_then_aggregation`:
```sh
./build/fleet serve --dev --dev_license 2>&1 | tee ~/fleet.txt
level=info ts=2023-03-09T19:27:17.324691Z component=redis mode=standalone
level=info ts=2023-03-09T19:27:17.360565Z instanceID="V9mArnX3lPhlIS0enyFau9eWi/dpjUPmOzJ3rwQUkX+l2aU1AMM4lQfdaDFZfeyJSHBwrIt/km1ghmRcyhdWqA=="
level=info ts=2023-03-09T19:27:17.372767Z msg="started cron schedules: automations, cleanups_then_aggregation, integrations, usage_statistics, vulnerabilities"
ts=2023-03-09T19:27:17.391404Z transport=https address=0.0.0.0:8080 msg=listening
level=error ts=2023-03-09T19:27:19.973841Z query=fleet_detail_query_software_macos message="distributed query is denylisted" hostID=58
level=info ts=2023-03-09T19:27:21.262799Z cron=cleanups_then_aggregation schedule=cleanups_then_aggregation instanceID="V9mArnX3lPhlIS0enyFau9eWi/dpjUPmOzJ3rwQUkX+l2aU1AMM4lQfdaDFZfeyJSHBwrIt/km1ghmRcyhdWqA==" status=pending
ts=2023-03-09T19:27:22.218129Z inf="skipping verification of encryption keys as MDM is not fully configured"
level=info ts=2023-03-09T19:27:22.224179Z cron=cleanups_then_aggregation schedule=cleanups_then_aggregation instanceID="V9mArnX3lPhlIS0enyFau9eWi/dpjUPmOzJ3rwQUkX+l2aU1AMM4lQfdaDFZfeyJSHBwrIt/km1ghmRcyhdWqA==" status=completed
```
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- ~[ ] Added/updated tests~
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
#9984
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- [X] Documented any permissions changes: Done by @noahtalerman, see
#10440
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- ~[ ] Manual QA for all new/changed functionality~
- For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
#8129
Apart from fixing the issue in #8129, this change also introduces UUIDs
to Fleet errors. To be able to match a returned error from the API to a
error in the Fleet logs. See
https://fleetdm.slack.com/archives/C019WG4GH0A/p1677780622769939 for
more context.
Samples with the changes in this PR:
```
curl -k -H "Authorization: Bearer $TEST_TOKEN" -H 'Content-Type:application/json' "https://localhost:8080/api/v1/fleet/sso" -d ''
{
"message": "Bad request",
"errors": [
{
"name": "base",
"reason": "Expected JSON Body"
}
],
"uuid": "a01f6e10-354c-4ff0-b96e-1f64adb500b0"
}
```
```
curl -k -H "Authorization: Bearer $TEST_TOKEN" -H 'Content-Type:application/json' "https://localhost:8080/api/v1/fleet/sso" -d 'asd'
{
"message": "Bad request",
"errors": [
{
"name": "base",
"reason": "json decoder error"
}
],
"uuid": "5f716a64-7550-464b-a1dd-e6a505a9f89d"
}
```
```
curl -k -X GET -H "Authorization: Bearer badtoken" "https://localhost:8080/api/latest/fleet/teams"
{
"message": "Authentication required",
"errors": [
{
"name": "base",
"reason": "Authentication required"
}
],
"uuid": "efe45bc0-f956-4bf9-ba4f-aa9020a9aaaf"
}
```
```
curl -k -X PATCH -H "Authorization: Bearer $TEST_TOKEN" "https://localhost:8080/api/latest/fleet/users/14" -d '{"name": "Manuel2", "password": "what", "new_password": "p4ssw0rd.12345"}'
{
"message": "Authorization header required",
"errors": [
{
"name": "base",
"reason": "Authorization header required"
}
],
"uuid": "57f78cd0-4559-464f-9df7-36c9ef7c89b3"
}
```
```
curl -k -X PATCH -H "Authorization: Bearer $TEST_TOKEN" "https://localhost:8080/api/latest/fleet/users/14" -d '{"name": "Manuel2", "password": "what", "new_password": "p4ssw0rd.12345"}'
{
"message": "Permission Denied",
"uuid": "7f0220ad-6de7-4faf-8b6c-8d7ff9d2ca06"
}
```
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [X] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
Fixes both #10378 and
https://github.com/fleetdm/confidential/issues/2133
On `main`:
```sh
curl -v -k -X POST -H "Authorization: Bearer $TEST_TOKEN" \
https://localhost:8080/api/latest/fleet/queries/run \
-d '{ "query": "select \"With automounting enabled anyone with physical access could attach a USB drive or disc and have its contents available in system even if they lacked permissions to mount it themselves.\" as Rationale;" }'
< HTTP/2 500
< content-type: application/json; charset=utf-8
< content-length: 130
< date: Fri, 10 Mar 2023 17:50:40 GMT
<
{
"message": "invalid query's SQL",
"errors": [
{
"name": "base",
"reason": "invalid query's SQL"
}
]
}
```
With changes in this PR:
```sh
curl -v -k -X POST -H "Authorization: Bearer $TEST_TOKEN" \
https://localhost:8080/api/latest/fleet/queries/run \
-d '{ "query": "select \"With automounting enabled anyone with physical access could attach a USB drive or disc and have its contents available in system even if they lacked permissions to mount it themselves.\" as Rationale;", "selected": { "hosts": [57] } }'
< HTTP/2 200
< content-type: application/json; charset=utf-8
< content-length: 325
< date: Fri, 10 Mar 2023 17:49:40 GMT
<
{
"campaign": {
"created_at": "0001-01-01T00:00:00Z",
"updated_at": "0001-01-01T00:00:00Z",
"Metrics": {
"TotalHosts": 1,
"OnlineHosts": 1,
"OfflineHosts": 0,
"MissingInActionHosts": 0,
"NewHosts": 0
},
"id": 87,
"query_id": 85,
"status": 0,
"user_id": 1
}
}
```
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
#8411
PS: I've opened #10209 to solve the issue with Golang Code Coverage CI
checks.
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- ~[] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [x] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
## Summary
This pull request is created by [Secure
Repo](https://app.stepsecurity.io/securerepo) at the request of @zwass.
Please merge the Pull Request to incorporate the requested changes.
Please tag @zwass on your message if you have any questions related to
the PR. You can also engage with the
[StepSecurity](https://github.com/step-security) team by tagging
@step-security-bot.
## Security Fixes
### Secure Dockerfiles
Pin image tags to digests in Dockerfiles. With the Docker v2 API
release, it became possible to use digests in place of tags when pulling
images or to use them in FROM lines in Dockerfiles.
- [The Open Source Security Foundation (OpenSSF) Security
Guide](https://github.com/ossf/scorecard/blob/main/docs/checks.md#pinned-dependencies)
## Feedback
For bug reports, feature requests, and general feedback; please create
an issue in
[step-security/secure-repo](https://github.com/step-security/secure-repo).
To create such PRs, please visit https://app.stepsecurity.io/securerepo.
Signed-off-by: StepSecurity Bot <bot@stepsecurity.io>
---------
Signed-off-by: StepSecurity Bot <bot@stepsecurity.io>
Co-authored-by: Zach Wasserman <zach@fleetdm.com>
These are minor changes needed to support the new ChromeOS extension.
This should have no effect on non-Chrome platforms.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
# Addresses #9595
# Implements
- new Activity types:
- CreatedMacOSProfile
- DeletedMacOSProfile
- EditedMacOSProfile
- Activity message depends on isPremium:
- true: '...macOS hosts with no team' or '...macOS hosts assigned to the
**Team Name** team {?via fleetctl}.'
- false: '...{to | from | for} all macOS hosts.'
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`
- [ ] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
For the CIS benchmark feature, we need a way to import a group of
policies (spec yml) into a team.
This PR adds a flag to `apply -f` to allow setting a team name to a
group of policies.
Sample:
```sh
fleetctl apply --context dogfood --policies-team "📊 CIS Benchmarks" -f ee/cis/macos-13/cis-policy-queries.yml
```
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [ ] Added/updated tests
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
Fixes warnings reported by Suspicious Package about the declared
permissions not matching the actual permissions. This was tested on
macOS and Linux (in the `fleetdm/fleetctl` Docker container) with the
"native tooling" option.
#7852
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
Even though the binaries are "universal" (compatible with both arm64 and
x86_64), this configuration must be provided in the Distribution XML in
order to prevent macOS from installing Rosetta when the user goes to
install the package. Verified with Suspicious Package.
For #9932
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
#9857
The "Public IP address" field is sometimes set to a "Private IP" on the
following types of Fleet deployments:
- Local deployments.
- Deployments where Fleet is on a private network.
- Deployments where an agent connects to Fleet not via the public
internet.
This PR will prevent a private IP to be set on the `host.public_ip`
field.
And this PR also adds documentation on how Fleet deduces the public IPs
of the devices so that a user can make the changes to fix this.
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- ~[ ] Added/updated tests~
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
Issue #9586
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- [ ] Documented any permissions changes
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
#8924
This is reproduced in dogfood for `dogfood-centos-box` and
`dogfood-ubuntu-box` where their "Private IP" is also their "Public IP".
Given that these hosts have their "Primary IP" configured to be their
"Public IP" alongside their "Private IP", the `network_interface_unix`
and `network_interface_windows` queries are now changed to ingest only
private IPs for the "Private IP" field.
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- ~[ ] Added/updated tests~
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
#9535
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
Related to #9571, this adds a new value to both responses which is
calculated when the Fleet server is started, and only set to `true` if
the server is properly configured for MDM.
This helps the UI to determine wether or not we should show certain UI
elements that we only want to show to servers with MDM enabled.
This resolves an issue with adding updates on a macOS 13 machine. It
seems like macOS may have changed the default directory permissions and
these new stricter permissions are compatible with that default.
This is the error that was encountered before these changes:
```
Error: create dst dir for copy: Path staged/targets already exists with
mode 20000000700 instead of the expected 20000000755
```
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
See requirements in #8682.
Two assumptions on the implementation (@zayhanlon please take a look):
- Hosts explicitly selected to run always run the live query (no matter
the values on the selectors).
- When selecting `All hosts`, selecting any other platform or label is
kind of a no-op. We should look into graying out all the selectors if
the user selects `All hosts`.
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- ~[ ] Documented any permissions changes~
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
# Addresses #9365
# Implements
MDM enrollment modal that handles both automatic and manual enrollment
instructions:
- Automatic:
<img width="1181" alt="Screenshot 2023-01-20 at 4 33 50 PM"
src="https://user-images.githubusercontent.com/61553566/213829293-6d4a5053-9a3c-4f52-8cf8-a6607dc8df4e.png">
- Manual:
<img width="1158" alt="Screenshot 2023-01-20 at 4 35 04 PM"
src="https://user-images.githubusercontent.com/61553566/213829369-73ae779d-14a8-4aa7-9c6a-b97d046d0dc1.png">
- Also includes (by mistake, but might as well include them now) some
small bash scripts for use in MDM development
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`
- [x] Updated testing inventory
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
# Addresses #9349
# Implements
https://www.loom.com/share/bbf8d6f97fe74e65a0c9a394f1bda3f1
- New Controls page, only visible to Global|Team Admins|Maintainers
- Header for free users is 'Controls', for premium is a teams filter
dropdown that defaults to 'No teams,' which filters via updating the URL
query param "team_id"
- Includes tabs macUpdates (default) and macSettings
- Cleaned up how site nav items are conditionally included/excluded
based on authorization – see
`frontend/components/top_nav/SiteTopNav/navItems.ts`
- Updated masthead styles: Removed icons from site nav links; updated
colors and spacing; Updated default user avatar TBD in separate PR
(waiting on guidance)
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`
- [x] Updated testing suite inventory
- [x] Manual QA for all new/changed functionality
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
#9119
To test the SSO changes locally you can use:
https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Testing-and-local-development.md#testing-sso
@RachelElysia Please take a look at the UI changes (All I did was
copy/paste and amend the changes for the new activity type.)
IMO we shouldn't display an avatar because there's no "actual user"
involved in these failed login attempts activities (by "actual user" I
mean the user attributed to the activity):
<img width="446" alt="Screenshot 2023-01-19 at 10 41 05"
src="https://user-images.githubusercontent.com/2073526/213524771-b85901ce-eec0-4cf3-919c-73162285e20b.png">
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
I was getting a cryptic error and no logs in fleet when trying to apply
an invalid spec file:
```sh
fleetctl apply -f ./some-queries.yml
Error: applying policies: POST /api/latest/fleet/spec/policies received status 500 forbidden: forbidden
```
With the changes in this PR now I get a more descriptive error (bad
request):
```sh
fleetctl apply -f ./some-queries.yml
Error: applying policies: POST /api/latest/fleet/spec/policies received status 400 Bad request: policy spec payload verification: policy query cannot be empty
```
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
~- [ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [x] Added/updated tests
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
relates to https://github.com/fleetdm/fleet/issues/8928
This adds a new `meta` attribute to the "GET /activities" endpoint that
includes pagination metadata. This can allow clients to know if there
are additional items to request.
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
# Addresses #8872
# Implements
- A modal on the MDM settings tab to change the default Apple Business
Manager team
https://www.loom.com/share/0eb45931d9db4a60843b5d427bdc9e61
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`
- [x] Manual QA for all new/changed functionality
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
# Addresses #9141
# Fixes
- 'Back to all hosts' link on Host Details page was navigating
incorrectly when the user had previously navigated directly with the
browser URL. This prevents this bug by adding a fallback path for that
BackLink to the default Manage Hosts path.
https://www.loom.com/share/579501365f514ac488f86eacadb2fd07
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`
- [x] Manual QA for all new/changed functionality
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
#9012
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- [x] Added/updated tests
Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>
* Fix ingestion of MDM server for macOS and Windows hosts
* Return early if there are no mobile_device_management_solutions to clean up
* Run make dump-test-schema
* Fix TestHosts/DeleteHosts test
* Add test for host unenrolling from MDM
* Fix lint
* Fix typo
* Identify Fleet MDM
* Move logic to deduce MDM name out of Datastore
* Run make dump-test-schema
* Remove printfs from migration
Related to #8167, from RFC5280:
> The serial number MUST be a positive integer assigned by the CA to
> each certificate. It MUST be unique for each certificate issued by a
> given CA (i.e., the issuer name and serial number identify a unique
> certificate).
Without this change, we're issuing two different certificates with the
same serial number (1, one) and issuer name:
1. One during `fleetctl generate mdm-apple`
1. One during the first SCEP enrollment
A customer had a seccomp profile that seemed to be incompatible with the
changes in glibc in ubuntu-22.04. Setting the builder back to 20.04
explicitly to resolve this issue.
* Improve how we are logging failed queries and windows detail queries
* Update failing tests
* Remove unneeded table in this particular select
* Update tests now that we dont rely on release_id
* Update ingestion since we want to ingest failures for policies and live queries
* We also want labels with failures
* Fix test since we are logging errors slightly differently
* Improve logging and error handling
* Revert back test change
* Actually, we need to OR the booleans to maintain logic
This needed to be set specifically for the live query websocket dialer,
since that doesn't use the HTTP client that the rest of the API client
methods use.
Co-authored-by: Tomas Touceda <chiiph@gmail.com>
* add premium feature critical policies
* update documentation
* add test for premium-only field
* update old change-file
* test policies more comprehensively
* also test team policies
* PATCH returns wrong timestamp, updating test
* Update go to 1.19.4
* Comment out failing package test
* Comment out ALL the packaging tests for windows for the moment
* Update go to 1.19.4
* Comment out failing package test
* Comment out ALL the packaging tests for windows for the moment
* Update changelog
* Bump versions
* Update changelog to reflect this being a security release
* Adding custom action to ensure that no fleetdm related processes are running on a product uninstall scenario. This will ensure that no file locks are present during file removal
* Add negation pseuodo-class to button hover to prevent bg color change when disabled
* change file
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
The OVAL analyzer falsely assumes that any vulnerabilities detected on a
host only come from OVAL. However, it is possible that NVD detects
vulnerabilities on these hosts even though it excludes software from
deb_packages and rpm_packages. For example, a python package twisted
v22.20 has a vulnerability CVE-2022-39348 detected by NVD. The OVAL
analyzer would delete this vulnerability, and it would be re-inserted by
the NVD scanner on the next run. This creates a loop.
The fix is to only delete vulnerabilities that are actually detected
using OVAL. We already store this in the source column in the
software_cve table.
* Merge branch 'main' of https://github.com/fleetdm/fleet into chore/testing-docs-on-contribution
add passing columns to inherited policies table
* add updated colors and applied them to icons
* change policy tables to use buildQueryString
Previously, the primary ip was determined using the interface with the most traffic. This had a few issues and required explicitly filtering out certain interfaces that we don't want eg loopback, docker.
Instead, use the default route to determine the primary (private) ip. If there are multiple, use the one with the lowest metric and prefer ipv4 addresses.
Related to https://github.com/fleetdm/fleet/issues/8010 and https://github.com/fleetdm/fleet/issues/8013 this prevents a bug that happens when:
1. A team doesn't have a `config.features` key in the JSON stored in the table or `config` is `NULL`
2. The team is edited from the UI
All `config.features` will default to `false`, which can be a problem if your global settings are `true` for both (which is the default)
* create new components for query side panel
* add reusable icon component that uses svg for icons
* integrate with new osquery_fleet_schema.json data
* update UI to work with osquery_fleet_schema.json
* add remark-gfm to safely support direct urls in markdown
* move fleet ace into markdown component so we can render code with ace editor
* add testing for new query sidebar
* remove incomplete tests for query sidepanel
related to #8031, this adds the following headers to HTML responses:
- Strict-Transport-Security: informs browsers that the site should only
be accessed using HTTPS, and that any future attempts to access it
using HTTP should automatically be converted to HTTPS.
- X-Frames-Options: disallows embedding the UI in other sites via
<frame>, <iframe>, <embed> or <object>, which can prevent attacks like
clickjacking.
- X-Content-Type-Options: prevents browsers from trying to guess the MIME
type which can cause browsers to transform non-executable content into
executable content.
- Referrer-Policy: prevents leaking the origin of the referrer in the
Referer.
additionally, this ensures we set `X-Content-Type-Options` for CSV and
installer responses.
* add awaitng results state to live query search
* make spinner more reusable and add it to query page awaiting results
* create common QueryResultsHeader component to use on query policy flows
Related to #7664, this cleans up all policy memberships for a host when its re-enrolled, afterwards only the relevant policy memberships for the host will be created.
* Support environments with revoked enroll secrets
* Add instructions on how to fix Orbit enroll
* Rename to last_recorded_error
* Add alternative instructions
related to https://github.com/fleetdm/fleet/issues/7199, this adds email validation to the `verifyCreateShared` which is used for user creation in the server.
validation messages come directly from Go's `net/mail` package.
```
~/fleet $ curl 'https://localhost:8080/api/latest/fleet/users/admin' -X POST -H 'Authorization: Bearer $TOKEN' --data-raw '{"email":"asdf","name":"asdf@asd.com","password":"as;lkdfjasdlk;fja3234@","global_role":"observer","teams":[]}'
{
"message": "Validation Failed",
"errors": [
{
"name": "email",
"reason": "mail: missing '@' or angle-addr"
}
]
}
```
Related to #7130, this adds logic to retry native notarization up to three times if it fails for some reason.
Since we're adding retries in various places, I added a new package under pkg for this purpose.
Configuration and fixes for the Fleet server and frontend to add support
for https://github.com/Uptycs/kubequery.
Co-authored-by: Michal Nicpon <michal@fleetdm.com>
- Add a new "Configuration for contributors" doc page. Move settings that are not recommended for production use
- Remove settings modified in the `config` YAML document from the deploying/configuration doc page
- Document all keys in `config` and `teams` YAML documents
- Add comments to several `.go` files and remove unused struct
* Make Change software vendor width mig. more robust
If there are two or more entries in the software table with the same name, version, source, release and arch but different vendors then the migration used for increasing the vendor width will fail.
* Ingest installed Windows updates and store them in the windows_updates table.
* Added config option for enabling/disabling Windows update ingestion and Windows OS vuln. detection.
* Bug 7320: Fixed dulp. vulns. on software details.
The software details page was showing duplicated vulnerabilities if the software was being used by many hosts.
* improve error handling in vulnerabilities cron
* fix tests
* Use errHandler and go mod tidy
* Add dep that got removed by mod tidy
* add dsl to tools
* Add changes file
Co-authored-by: Michal Nicpon <michal@fleetdm.com>
Co-authored-by: Tomas Touceda <chiiph@gmail.com>
Related to #7054 and #6834, this adds the UI changes necessary to support JIT provisioning:
A checkbox in the settings page
A new template for the activity box
This also includes relevant documentation about the feature and how to configure it.
Related to #7036, Tested in:
- [x] macOS via `fleetctl package --type=msi` (non-native)
- [x] Linux native via `docker run -v "$(pwd)" fleetdm/fleetctl package --type=msi` (native)
- [x] Windows via `fleetctl pacakge --type=msi` (thanks to @edwardsb) (non-native)
* create a MainContent and SidePanelContent containers for layout
this creates these two new components for handling layout more cleanly. It also allows us to put in common components into main layout, like sandbox expiration notification
* use MainContent and SidePanelContent in current pages
this brings in the two new components and wraps the page contents in these. This also allowed us to clean up and remove unused/no needed styling code
* add MainContent component to user settings page and clean up user settings component
this cleans up the user settings page to follow the panel convention we have as well as adds the MainContent component to this page
* add MainContent component to team pages
* update Sandbox gate to render optional component when in sandbox mode and add to MainContent
* add call to sandbox api to get expiry time
this adds a conditional call when the user is in sandbox mode to get the expiry time of the instance
* fix sticky elements on settings pages to work with sandbox expiry message
* fix e2e test after MainContent refactor
* Add rate limits for device authed endpoints
* Fix lint
* Add missing test
* Fix test
* Increase the quota for desktop endpoints
* Add comment about quota
Related to #5898, this reports an anonymized summary of errors stored in Redis into the analytics payload.
For each error stored, this includes:
- A `count` attribute with the number of occurrences of the error
- A `loc` attribute with the 3 topmost lines in the stack trace. Note that stack traces only contain package name + line number (example: github.com/fleetdm/fleet/server.go:12
This also includes a minor refactor around error types.
As reported in #6127, the `fleetctl debug` `archive` and `errors` commands were failing on Windows because filenames are not allowed to contain colons `:`.
This changeset removes colina from the filename of the archives generated by both commands.
Feature: Improve our capability to detect vulnerable software on Ubuntu hosts
To improve the capability of detecting vulnerable software on Ubuntu, we are now using OVAL definitions to detect vulnerable software on Ubuntu hosts. If data sync is enabled (disable_data_sync=false) OVAL definitions are automatically kept up to date (they are 'refreshed' once per day) - there's also the option to manually download the OVAL definitions using the 'fleetctl vulnerability-data-stream' command. Downloaded definitions are then parsed into an intermediary format and then used to identify vulnerable software on Ubuntu hosts. Finally, any 'recent' detected vulnerabilities are sent to any third-party integrations.
This removes policy information from `GET /api/_version_/fleet/device/{token}` from non-premium Fleet instances.
Starting the server with `./build/fleet serve --dev --dev_license`
```bash
$ curl -s https://localhost:8080/api/latest/fleet/device/1804e808-171f-4dda-9bec-f695b2f2371a | jq '.host.policies // "not present"'
[
{
"id": 3,
"name": "Antivirus healthy (Linux)",
"query": "SELECT score FROM (SELECT case when COUNT(*) = 2 then 1 ELSE 0 END AS score FROM processes WHERE (name = 'clamd') OR (name = 'freshclam')) WHERE score == 1;",
"description": "Checks that both ClamAV's daemon and its updater service (freshclam) are running.",
"author_id": 1,
"author_name": "Roberto",
"author_email": "test@example.com",
"team_id": null,
"resolution": "Ensure ClamAV and Freshclam are installed and running.",
"platform": "darwin,linux",
"created_at": "2022-05-23T20:53:36Z",
"updated_at": "2022-06-03T13:17:42Z",
"response": ""
}
]
```
Starting the server with `./build/fleet serve --dev`
```bash
$ curl -s https://localhost:8080/api/latest/fleet/device/1804e808-171f-4dda-9bec-f695b2f2371a | jq '.host.policies // "not present"'
"not present"
```
* fix old root dir in orbit
* add changes
* Add automation for orbit shell (with TUF)
* Fix workflow syntax
* Add logging to latest fleetctl preview action
* Add changes to fix workflow
* Use macOS host for TUF server and package generation
* Remove copy/paste if clause
* Fix orbit logs on macOS, Ubuntu
* Simplify TUF and generation of packages
* Set enroll secret instead of getting it
* Increase timeouts
* Add step id
* Fixes to the upload/download of artifacts
* Rearrange steps to not lose the downloads
* Fix copy/paste
* Add fleetctl login step
* Add missing config set
* Fix quotes on Windows
* Increase timeout
* Fix job termination
* Disable FLEET_DESKTOP for now
* Checkout repository on macOS
* Fix logs path
* Enable fleet desktop
* Use cancel, nitpick
Co-authored-by: Michal Nicpon <michal@fleetdm.com>
This adds a new device authenticated endpoint, `/api/_version_/fleet/device/{token}/policies` to retrieve the device policies.
An example request / response looks like:
```bash
curl https://localhost:8080/api/latest/fleet/device/1804e808-171f-4dda-9bec-f695b2f2371a/policies
```
```json
{
"policies": [
{
"id": 3,
"name": "Antivirus healthy (Linux)",
"query": "SELECT score FROM (SELECT case when COUNT(*) = 2 then 1 ELSE 0 END AS score FROM processes WHERE (name = 'clamd') OR (name = 'freshclam')) WHERE score == 1;",
"description": "Checks that both ClamAV's daemon and its updater service (freshclam) are running.",
"author_id": 1,
"author_name": "Admin",
"author_email": "admin@example.com",
"team_id": null,
"resolution": "Ensure ClamAV and Freshclam are installed and running.",
"platform": "darwin,windows,linux",
"created_at": "2022-05-23T20:53:36Z",
"updated_at": "2022-05-23T20:53:36Z",
"response": "fail"
}
]
}
```
Related to [#5685](https://github.com/fleetdm/fleet/issues/5685), in another changeset I will be adding "client" endpoints so we can consume this endpoint from Fleet Desktop
Related to #5776, this ensures that when a host is re-enrolled on a different team we cleanup existing policy memberships in the same way we do when a host is assigned a team through the API.
This adds documentation about our debugging endpoints and a brief excerpt about the fleetctl debug command with instructions to generate the archive so we have a place with instructions to which we can direct people.
It also adds the .prof file extension to profiling files, which hopefully clarifies that they are meant to be used by go tool pprof.
This adds compatibility in `fleetctl preview` to work with docker compose (version 2). Since this version was released this April, we are still keeping backwards compatibility and using docker-compose as a fallback.
As v2 is now the recommended version and v1 is deprecated, this also rewords all prompts and help messages to say "docker compose".
Rel: #5746
this change prevents errors from being automatically cleared once they are read. A new flag `-flush` is introduced to flush errors on read if necessary.
Related to https://github.com/fleetdm/fleet/issues/5504, this change attempts to improve the output of the `fleetctl debug errors` command by:
- Adding a warning message to redact sensitive data
- Adding a `json` extension to the output file
- Allowing to stream the output to stdout via the `-stdout` flag or the `STDOUT` env var
The output after this changes is:
```
~/projects/fleet $ ./build/fleetctl debug errors
################################################################################
# WARNING:
# The generated file may contain sensitive data.
# Please review the file before sharing.
#
# Output written to: fleet-errors-2022-05-05T12:46:42-03:00.json
################################################################################
```
It also modifies the output of `fleetctl debug archive`
```
################################################################################
# WARNING:
# The files in the generated archive may contain sensitive data.
# Please review them before sharing.
#
# Archive written to: fleet-profiles-archive-2022-05-05T12:46:59-03:00.tar.gz
################################################################################
```
* Add (beta) support for Fleet Desktop to linux
* Add dependency for linux desktop
* Amend makefile uname check
* Clarify env vars used for linux in execuser
* Add final set of fixes
* Remove -it from docker run
* Add desktop to the update runner for Linux
* Re-arrange tag.gz and fix upgrade check for linux desktop
Install orbit to /opt instead of /var/lib. When installing to /var/lib,
the default selinux context of var_lib_t gets applied, which results in
an AVC error when running via systemd.
Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>
To add support for #400, we're using the macadmins/osquery-extension to gather Google Chrome profiles from hosts.
Under the hood, the extension looks and parses a json file in which Chrome stores a bunch of data. Given that emails are not required to create Chrome profiles, some of the profiles stored in this file and returned by the query we're using contain empty emails.
The idea after this change is to prevent empty emails from being ingested in the first place instead of filtering them after the fact. I have also included a migration to clean the rows with empty email columns.
Fixes#4780
* Bug 5066: Format config durations
Change duration values returned by 'fleetctl get config --include-server-config' from nanoseconds to a human readable format.
* Fix access control issues with users
* Fix access control issues with packs
* Fix access control issues with software
* Changes suggested by Martin
* All users can access the global schedule
* Restrict access to activities
* Add explicit test for team admin escalation vuln
* All global users should be able to read all software
* Handbook editor pass - Security - GitHub Security (#5108)
* Update security.md
All edits are recorded by line:
395 replaced “open-source” with “open source”
411 replaced “open-source” with “open source”
439 added “the” before “comment”; replaced “repositories,” with “repositories”
445 deleted “being” before “located”
458 added “and” after “PR”
489 replaced “on” with “in”
493 replaced “open-source” with “open source”; Replaced “privileges,” with “privileges”
* Update security.md
line 479
* Update security.md
added (static analysis tools used to identify problems in code) to line 479
* Fix UI
* Fix UI
* revert api v1 to latest in documentation (#5149)
* revert api v1 to latest in documentation
* Update fleetctl doc page
Co-authored-by: Noah Talerman <noahtal@umich.edu>
* Add team admin team policy automation; fix e2e
* Update to company page of the handbook (#5164)
Updated "Why do we use a wireframe-first approach?" section of company.md
* removed extra data on smaller screens (#5154)
* Update for team automations; e2e
* Jira Integration: Cypress e2e tests only (#5055)
* Update company.md (#5170)
This is to update the formatting under "empathy" and to fix the spelling of "help text."
This was done as per @mikermcneil .
This is related to #https://github.com/fleetdm/fleet/pull/4941 and https://github.com/fleetdm/fleet/issues/4902
* fix update updated_at for aggregated_stats (#5112)
Update the updated_at column when using ON DUPLICATE UPDATE so that
the counts_updated_at is up to date
* basic sql formatting in code ie whitespace around operators
* Fix e2e test
* Fix tests in server/authz
Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>
Co-authored-by: Desmi-Dizney <99777687+Desmi-Dizney@users.noreply.github.com>
Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com>
Co-authored-by: Noah Talerman <noahtal@umich.edu>
Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
Co-authored-by: Martavis Parker <47053705+martavis@users.noreply.github.com>
Co-authored-by: RachelElysia <71795832+RachelElysia@users.noreply.github.com>
Update the updated_at column when using ON DUPLICATE UPDATE so that
the counts_updated_at is up to date
* basic sql formatting in code ie whitespace around operators
* Remove required password reset flag when creating new API-only user
* Add test for 'fleetctl user create' command
* Set NewUserFuncInvoked to false in between tests
Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>
* added teams webhook calls
* split global admin and team admin tests; added policy automation
* changes file
* fixed type errors
* fixed e2e test
* fixed typo
* fixed admin test syntax
* fixed test logic for team maintainer
* lint fixes
* more admin e2e test fixes
* fixed team policy test
* removed duplicate test
This optimizes the query in particular for Domain Controllers, where
there are a number of users with no local accounts and don't need to be
searched for software.
See #4261
not set on the INSERT.
- OUT: Only sets the ID on the passed session and returns it. (`CreatedAt`, `AccessedAt`, are not set.)
New version:
```go
func (ds *Datastore) NewSession(ctx context.Context, userID uint, sessionKey string) (*fleet.Session, error) {
sqlStatement := `
INSERT INTO sessions (
user_id,
` + "`key`" + `
)
VALUES(?,?)
`
result, err := ds.writer.ExecContext(ctx, sqlStatement, userID, sessionKey)
if err != nil {
return nil, ctxerr.Wrap(ctx, err, "inserting session")
}
id, _ := result.LastInsertId() // cannot fail with the mysql driver
return ds.sessionByID(ctx, ds.writer, uint(id))
}
```
- IN: Define arguments that are truly used when creating a session.
- OUT: Load and return the fleet.Session struct with all values set (using the `ds.writer` to support read replicas correctly).
PS: The new `NewSession` version mimics what we already do with other entities, like policies (`Datastore.NewGlobalPolicy`).
* Orbit: Add Fleet Desktop support to Windows
* Rename workflow, fix linux build
* Do not compile systray on linux
* nolint on unused
* Fix lint properly
* nolint both checkers
* Fix monitor logic in desktopRunner
* Fix interrupt and execute order
Renaming a temporary file to the final destination does not always work.
Specifically, if the source and destination paths are on different file
systems, you will get the following error
invalid cross-device link
Instead, create temporary file in the destination directory.
- Update CHANGELOG
- Bump versioning
- Tweak documentation
- Default `session_duration` to `5d`
- Add extra `#` to "Team policies" section so it doesn't show up in top level nav for docs
* WIP
* WIP2
* Fix orbit and fleetctl tests
* Amend macos-app default
* Add some fixes
* Use fleetctl updates roots command
* Add more fixes to Updater
* Fixes to app publishing and downloading
* Add more changes to support fleetctl cross generation
* Amend comment
* Add pkg generation to ease testing
* Make more fixes
* Add changes entry
* Add legacy targets (until our TUF system exposes the new app)
* Fix fleetctl preview
* Fix bool flag
* Fix orbit logic for disabled-updates and dev-mode
* Fix TestPreview
* Remove constant and fix zip-slip attack (codeql)
* Return unknown error
* Fix updater's checkExec
* Add support for executable signing in init_tuf.sh
* Try only signing orbit
* Fix init_tuf.sh targets, macos-app only for osqueryd
* Specify GOARCH to support M1s
* Add workflow to generate osqueryd.app.tar.gz
* Use 5.2.2 on init_tuf.sh
* Add unit test for tar.gz target
* Use artifacts instead of releases
* Remove copy paste residue
* Fleet Desktop Packaging WIP
* Ignore gosec warning
* Trigger on PR too
* Install Go in workflow
* Pass url parameter to desktop app
* Fix fleetctl package
* Final set of changes for v1 of Fleet Desktop
* Add changes
* PR fixes
* Fix CI build
* add larger menu bar icon
* Add transparency item
* Delete host_device_auth entry on host deletion
* Add SetTargetChannel
* Update white logo and add desktop to update runner
* Add fleet-desktop monitoring to orbit
* Define fleet-desktop app exec name
* Fix update runner creation
* Add API test before enabling the My device menu item
Co-authored-by: Zach Wasserman <zach@fleetdm.com>
Add new usePlatformSelector custom hook
Add new usePlatformCompatibility custom hook
Add new PlatformSelector global component
Refactor PlatformCompatibility as global component
Refactor sql_tools to TypeScript
Improve type definitions for context/policy
Align PolicyPage and QueryPage with platform compatibility changes
* geoip wip
* return nil if ip is empty string or if ParseIP returns nil
* add ui component to render geolocation if available, address PR feedback
* render public ip if available
* add changes file, document geoip in deployment guide
* update rest-api docs
* WIP
* WIP2
* Fix orbit and fleetctl tests
* Amend macos-app default
* Add some fixes
* Use fleetctl updates roots command
* Add more fixes to Updater
* Fixes to app publishing and downloading
* Add more changes to support fleetctl cross generation
* Amend comment
* Add pkg generation to ease testing
* Make more fixes
* Add changes entry
* Add legacy targets (until our TUF system exposes the new app)
* Fix fleetctl preview
* Fix bool flag
* Fix orbit logic for disabled-updates and dev-mode
* Fix TestPreview
* Remove constant and fix zip-slip attack (codeql)
* Return unknown error
* Fix updater's checkExec
* Add support for executable signing in init_tuf.sh
* Try only signing orbit
* Fix init_tuf.sh targets, macos-app only for osqueryd
* Specify GOARCH to support M1s
* Add workflow to generate osqueryd.app.tar.gz
* Use 5.2.2 on init_tuf.sh
* Add unit test for tar.gz target
* Use artifacts instead of releases
* Remove copy paste residue
* Trigger workflow on PR
* Fixes to ease handling of artifact
* Fix, do not use target name as dir
* Remove workaround
* Error early if email already exists
* Update changes/issue-4361-mail-change-should-error
Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com>
* Fix test
* Lint fixes
* Fix another test
* Also check for invites
* Improve error checks
* Update comment
* Update tests
* Fix test
Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com>
* Add disable-updates flag to fleetctl and orbit
* Fix ruleguard execution error on make lint-go
* Introduce dev-mode for ease of development of orbit
* Add changes file
* Add CentOS parsing and post-processing in fleet
* Add tests and amend SyncCPEDatabase
* Add test for centosPostProcessing
* Changes from PR comments
* Amend software test
* Fix sync test
* Add index to source and vendor
* Use os.MkdirTemp
* Rearrange migrations
* Regenerate test schema
* Add support for testing migrations (#4112)
* Add support for testing migrations
* Rename migration in tests
* Changes suggested in PR
* Go mod tidy
* Fix OSVersion to work properly when distribution does not follow symver
Certain distros such as ClearLinux or ArchLinux do not use symver or any
dotted versioning scheme for their releases. Archlinux uses the static
string "Rolling" and ClearLinux uses a single build number such as 35550
for their versions.
In Fleet console, this shows up as a string like "Archlinux 0.0.0.0"
which makes very little sense to the user. This change makes it so that
if OSQuery cannot generate a dotted version number, we should instead
use the build id as an opaque string.
* Add /changes
* Add tests for os_version
* fixup
* fixup
* Add platform filters for MDM/Munki/Chrome queries
This should help quiet warnings that users/customers have reported when
these queries try to run on platforms without the macadmins extension
tables.
For #4123
* Improve documentation
* add changes file
* revert doc formatting
* Update tests
* Yet another test fix
Co-authored-by: Tomas Touceda <chiiph@gmail.com>
* Add remaining policy templates
* Removeempty resolution field
* Update naming for standard policies
* Add enabled word to SIP policy
* Use full SIP
* Also change in constants
* Update windows disk encryption
* Add changes file
* Tweak windows disk encryption policy
* Address lint errors
* Make requested changes
* Reflect changes in policy tempaltes
* Make sure that standard policies and policy templates are the same
* Edit automatic login disabled description
* Also edit in constants
* Change webhooks db lock duration to one hour
* Reload interval and check time left
* Change interval to one hour to match the db lock
* Add missing continue in case of failure
* Fix lint-go
* Allow to search with target_sw with a period
* Remove tests for a different thign
* Add a test for programs as a source as well
* Use MATCH again and add proper tests
- Add new "Software" tab to host details page
- Add dropdown to filter vulnerable software
- Extend DataTable client-side filtering to accommodate filter inputs that are controlled by parent components
- Refactor host details software into separate component
* Add new software page
* Add hosts counts and last updated to software card on homepage
* Extend InfoCard component to all children to set dynamic title information and hyperlinks
* Add new TeamsDropdownHeader component
* Refactor TableContainer to move server-side pagination buttons to DataTable
* Extend DataTable to include footer element
* Better jitter (#3716) (#3744)
* Better jitter (#3716)
* Better jitter
* Fix lint
* Use milliseconds
* Make duration milliseconds
* Update based on Lucas' suggestion
* Add changes file
* Panic on error
* Fix compilation error
* Use time.after instead of time.tick to not leak
* Include browser extensions in software inventory (#3733) (#3757)
Use appropriate JOINs against users table to include all results.
For #3557
* Build Docker images on push to any branch (#3756) (#3758)
Allows for testing release branches and any others.
* Optimize users detail query (#3754) (#3770)
@Smjert reported instances of Windows Domain Controllers having massive
resource utilization and being killed by the watchdog when running this
query. In his test environment, this new query performs much better.
* Refactor async host processing to avoid redis SCAN keys (for policies) (#3657) (#3755)
* Issue 3707 clean targets on delete (#3739) (#3776)
* wip
* Delete targets when deleting teams, hosts, and labels
* Add changes file
* Fix error message
* Remove unused teamsTable
* Cleanup new pack
* Clean new packs at end of test
* Update operating system policy (#3779)
- Update policy's query to check for operating system versions greater than or equal to
* Improve loading of manage policies page (#3695) (#3781)
Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>
* Fix logo (#3765)
* fixed duplicate query modal not closing (#3787)
* fixed style for connection line on activity feed (#3789)
* Improve loading state for query platform compatibility (#3752) (#3783)
Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>
* Disable pagination for teams page (#3794)
* Shorten "simple" query API period to 25s (#3775) (#3804)
This helps the period stay under the default request timeouts for most
load balancers.
Some default timeouts:
* AWS ALB - 60s
* Nginx - 60s
* GCP LB - 30s
* Clear user checkbox selections after successfully removing selected row from table UI #3798 (#3801)
* Alias fleetctl sandbox to fleetctl preview (#3803) (#3806)
Preparing for some work the product team is planning to rebrand preview
to sandbox.
* Remove flash of spinner to no spinner (#3799)
* Revert "Better jitter (#3716) (#3744)" (#3807)
This reverts commit c857b52d2b.
* Fix teams dropdown sizing (#3759) (#3810)
Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>
* fixed link for expiration message (#3788)
* Fixed empty string for compatible platforms (#3812)
* fixed empty string for compatible platforms
* default checked for platforms on policy query
* Block autocomplete for username and password in SMTP settings (#3732) (#3816)
* No hosts for software/policy renders better message (#3701) (#3821)
Co-authored-by: Martavis Parker <47053705+martavis@users.noreply.github.com>
* Get failing policies webhook data on page load (#3824)
* Fix reset password flow (#3826)
* Prepare for 4.9.0 (#3820)
- Add CHANGELOG
- Bump versioning
- Remove changes files
- Spelling fix for operating system policy's description
* Remove merge artifact
Co-authored-by: Zach Wasserman <zach@fleetdm.com>
Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
Co-authored-by: Luke Heath <luke@fleetdm.com>
Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>
Co-authored-by: RachelElysia <71795832+RachelElysia@users.noreply.github.com>
Co-authored-by: Martavis Parker <47053705+martavis@users.noreply.github.com>
* Add sentry
* Fix gosum
* More gosum fixes
* Add missing def for config
* Enrich sentry scope a bit
* Add changes file
* Add goroutine safe scope to errors
* Encapsulate sentry logic
* Add documentation for new flag
* Add sentry capturing to crons and other background tasks
* Only send to sentry when enabled
This helps the period stay under the default request timeouts for most
load balancers.
Some default timeouts:
* AWS ALB - 60s
* Nginx - 60s
* GCP LB - 30s
* Allow sort by more than one key
* more accurate message for no hosts on filter
* added changes file
* lint fix
* more accurate message for no hosts on filter
* added changes file
* lint fix
* small logical change
Co-authored-by: Tomas Touceda <chiiph@gmail.com>
* wip
* Delete targets when deleting teams, hosts, and labels
* Add changes file
* Fix error message
* Remove unused teamsTable
* Cleanup new pack
* Clean new packs at end of test
@Smjert reported instances of Windows Domain Controllers having massive
resource utilization and being killed by the watchdog when running this
query. In his test environment, this new query performs much better.
* WIP
* Amend tests
* Do not load aggregated stats for packs
* Add option to host lite
* Fix remaining TODOs
* Fix osquery_utils tests
* Fix SQL
* Fix SQL (bis)
* Restore AuthenticateHost to load once
* Code improvements and re-add deferred host save
* More fixes to the PR
* Wrap users table update on tx
* Add caching to ListPacksForHost and ListScheduledQueriesInPack
* Remove SaveHostSoftware (replaced by UpdateHostSoftware)
* Add unit tests for new functionality
* Add changes file
* Fix scheduled queries test
* Remove cleanups and fk for host ids
* Readd missing things to the schema
* Remove unused
* Add changes file and fix some error messages
* Fix test
* Use tx instead of plain writer
* Other fixes
* More not found test fixes
* Go back to getcontext
* Add fixes for running tests with mysql:8
* Add getServer function
* Test github matrix
* Add changes file for the user facing fix
* Remove unused mysql8 docker-compose
* Fix orbit and osqueryd logging on Windows
* Add functionality to test the change and use systemprofile app data
* Add centos syslog to README.md
* Add wait on orbit for osquery extension socket to exist (#3571)
* Wait for osquery extension socket to exist
* Amend changes
* Fix lint
* Restore timeout
* Add webhook to app config
* Add redis failing policies set and webhook
* Add basic webhook test
* Store hostname in redis
* Global policy deletion to remove policy ID from set and config
* Also process new passing policies
* Fix unit test
* Sort hosts
* Add more tests
* Add ListSets to the failing policies interface
* Fix server URL and garbage collect on the triggering side
* Do not use Redis SCAN
* Fix Redis operation order
* Add API changes to doc
* Add comments
* Add more tests
* Fix tests
* Add tests for config update upon deletion of policies
* Run make dump-test-schema
* Ignore policies that failed to run
* Add proper unit tests to trigger logic
* Fix comments
* WIP
* Add tests to service_osquerty_test.go
* Use SSCAN for listing hosts instead of SMEMBERS
* Add failing policies to docs/01-Using-Fleet/configuration-files/README.md
* Remove skip
* Fix PR comments
* Finish first draft of API versions
* wip
* Finalize tests
* Revert change in handler
* Remove made up version
* Update versioning with aliases
* Add changes file
* Address review comments
* Revert overupdated routes
* Expand life time of deprecated APIs
* Fix test
* Comment out problematic part of test
* Revert bad path changes
* Display count is now correct, bug with 0 being falsy
* Empty component renders for client side search even if data table 'disappears'
* Hide display count if display count is 0 includes for client side search
* Hide search and dropdown if no queries have been created
* Remove tx from SaveHost
Also change REPLACE for software for an insert.
* Remove bundle identifier from the get since it's not part of the key
* Revert unneeded change
* Add software count API
* Fix makefile
* Fine no mock generating at this point
* Actually, one last try
* Use go install instead
* Fix go sum/mod
* Improve documentation
* Try setting node to 14
* Do caching of app config per instance instead of across all of them in redis
* Add changes file
* Simplify code based on review comment
* Use go-cache instead of creating our own
* Dont export consts
* Copy app config before returning it
* Fix lint
* Update go sum
* Update go sum
* Add fleetctl command to get missing migrations
* Fix copy paste and lint
* Detect migrations applied out of order
* Add extra bullet to changes
* Trigger creation of migration status tables
* Fix unit tests
* PR fixes
* PR comment fixes
- Orbit automatically loads the flagfile when it exists in the orbit
root.
- Add packaging support to include flagfile with package.
- Fix a panic when osquery fails to start up.
* Add host count to software API
* Update docs
* Update fleetctl tests to account for host counts
* Update docs to mention host_count special case
* Update func comment
* Ignore admin forced password reset if sso is enabled
* Add tests for authenticatedUser
* If SSOEnabled is true, then set AdminForcedPasswordReset to false
* Update tests to cover new password reset and sso flow
* Add changes file
* Serialize hosts writes per instance
* Write hosts asynchronously
* Dont make the save in a goroutine
* Revert "Dont make the save in a goroutine"
This reverts commit 4a890c5271.
* Make all savehosts async
* Address review comments and make this approach configurable
* Address review comments
* Disable bulk seen time marking for a test
* Move host seen times to a new table
* Remove unused
* Add seen_time to list hosts
* Add some jitter to seen time flushing
* Remove unused
* Add timeout to deferred save host
* Add tests for serialSaveHost
* Update hosts in labels and policy executions in a serial way
* Address review comments and remove fk constraints in host software
* Make errCh buffered
* Add changes file
* Readd key
* Make preview work when run from scratch (no orbit running)
* Make vulnerability processing less RAM hungry
* Add changes file
* Only get the cpe list once
* Remove cache
* Try killing osquery as well as orbit and clear their db dir
Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>
- Fix Windows MSI generation by changing permissions (#2655).
- Refactor temp directory initialization.
- Use root user for Wine in WiX Docker container.
- Support .pkg packaging on Linux without dependencies (besides Docker)
