relates to #12986
This is a fix for a case where the `os_versions` endpoint was returning
a 404 when searching for `no teams`.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
related to #12483, we have found out that in distributed scenarios, the
URL of the Puppet server used for the request is appended to the
identifier, and it can be different between `/preassign` and `/match`
calls.
to account for this, we're only grabbing the first 36 characters of the
identifier.
Prior to 4.35.0, some rows in the scheduled_query table might have a
`NULL` value due to a race condition with database replicas and the way
`ds.EnsureGlobalPack` and `ApplyPackSpecs` work together.
This is no longer the case, but some databases are left in weird states,
which were not accounted by this migration.
Chaning the migration in-place because that's the approach we took in
previous migrations with similar problems.
relates to #12907
implement adding an enroll secret when creating a new team with
`fleetctl apply` if none is provided.
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Added/updated tests
- [x] Manual QA for all new/changed functionality
This relates to #12600
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
- [X] Manual QA for all new/changed functionality
## Addresses #12999
<img width="1282" alt="Screenshot 2023-07-27 at 11 59 01 AM"
src="https://github.com/fleetdm/fleet/assets/61553566/b60d3b41-3d7b-4550-ba7c-8615bae085a6">
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
## Addresses #12646
### See issue for list of completed work
### Also see PR #12713 **notes for review** on that PR for help manually
testing this work in lieu of the completed API.
- [x] Changes file added for user-visible changes in `changes/`
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
## Addresses #12636
### See issue for list work done
### Notes for review:
- Because other work is based on this branch, TODOs / fixes are noted
here until the team comes to a strategy for merging all of the work:
- Add missing space in the Performance impact column "Undetermined"
tooltip text
- I'm having trouble confirming that the inherited queries table is
working right with the mock hard-coded data, though I did see it working
correctly previously. There's an issue with the page reverting to "All
teams" when trying to show the inherited table, though it does show the
table before re-rendering.
- This work is organized clearly by commit, so that might be a
manageable way to go through this code.
- Since the updated API for this work is not yet complete, this work can
be manually tested by either:
- Using mock API infrastructure, or
- in `ManageQueriesPage.tsx`, comment out the two `useQuery` calls and
add appropriate mock data. You can then modify any fields of interest to
test their related UI functionality. For example, lines 119 -242 might
read:
```
// const {
// data: curTeamEnhancedQueries,
// error: curTeamQueriesError,
// isFetching: isFetchingCurTeamQueries,
// refetch: refetchCurTeamQueries,
// } = useQuery<IListQueriesResponse, Error, IEnhancedQuery[]>(
// [{ scope: "queries", teamId: teamIdForApi }],
// () => queriesAPI.loadAll(teamIdForApi),
// {
// refetchOnWindowFocus: false,
// enabled: isRouteOk,
// select: (data) => data.queries.map(enhanceQuery),
// }
// );
// // If a team is selected, fetch inherited global queries as well
// const {
// data: globalEnhancedQueries,
// error: globalQueriesError,
// isFetching: isFetchingGlobalQueries,
// refetch: refetchGlobalQueries,
// } = useQuery<IListQueriesResponse, Error, IEnhancedQuery[]>(
// [{ scope: "queries", teamId: -1 }],
// () => queriesAPI.loadAll(),
// {
// refetchOnWindowFocus: false,
// enabled: isRouteOk && isAnyTeamSelected,
// select: (data) => data.queries.map(enhanceQuery),
// }
// );
const [
curTeamEnhancedQueries,
curTeamQueriesError,
isFetchingCurTeamQueries,
refetchCurTeamQueries,
] = useMemo(() => {
return [
[
{
created_at: "2023-06-08T15:31:35Z",
updated_at: "2023-06-08T15:31:35Z",
id: 2,
name: "test",
description: "",
query: "SELECT * FROM osquery_info;",
team_id: 43,
platform: "darwin",
min_osquery_version: "",
automations_enabled: true,
logging: "snapshot",
saved: true,
// interval: 300,
interval: 0,
observer_can_run: false,
author_id: 1,
author_name: "Jacob",
author_email: "jacob@fleetdm.com",
packs: [],
stats: {
// system_time_p50: 1,
// system_time_p95: null,
// user_time_p50: 1,
// user_time_p95: null,
// total_executions: 1,
},
performance: "Undetermined",
platforms: ["darwin"],
},
] as IEnhancedQuery[],
undefined,
false,
() => {
console.log("got the new queries");
},
];
}, []);
const [
globalEnhancedQueries,
globalQueriesError,
isFetchingGlobalQueries,
refetchGlobalQueries,
] = useMemo(() => {
return [
[
{
created_at: "2023-06-08T15:31:35Z",
updated_at: "2023-06-08T15:31:35Z",
id: 200,
name: "test",
description: "",
query: "SELECT * FROM osquery_info;",
team_id: null,
platform: "darwin",
min_osquery_version: "",
automations_enabled: true,
logging: "snapshot",
saved: true,
// interval: 300,
interval: 0,
observer_can_run: false,
author_id: 1,
author_name: "Jacob",
author_email: "jacob@fleetdm.com",
packs: [],
stats: {
// system_time_p50: 1,
// system_time_p95: null,
// user_time_p50: 1,
// user_time_p95: null,
// total_executions: 1,
},
performance: "Undetermined",
platforms: ["darwin"],
},
] as IEnhancedQuery[],
undefined,
false,
() => {
console.log("got the new inherited queries");
},
];
}, []);
```
- [x] Changes file added for user-visible changes in `changes/`
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
Related to #12608, this automatically sets the
`DeferForceAtUserLoginMaxBypassAttempts` property to `1` on the
FileVault profile that's generated by Fleet.
This changeset also includes a migration to modify old FileVault
profiles that already exist in the database, and by virtue of that a
`InstallProfile` command will be issued to hosts that already have FV
enabled. During testing we found:
1. This doesn't affect users with FV already installed, they silently
get the profile updated without any changes.
2. Since the profile needs to be re-delivered, it'll go through the full
"pending" -> "verifying" -> "verified" cycle.
This is related #12701
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
- [X] Manual QA for all new/changed functionality
This relates to #12263
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added/updated tests
---------
Co-authored-by: Roberto Dip <me@roperzh.com>
#10292, #12554
When scanning tens of thousands of files for permissions, using the
`find` command exposed as a fleetd table is more performant than trying
to use the `file` table. This change caused the watchdog to *stop*
killing osquery because of exceeding memory or CPU limit.
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [X] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
relates to #12289
Implements enable and disable Windows mdm activity UI
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
- [x] Manual QA for all new/changed functionality
## Addresses #11355, pt.1 (pt. 2 already fixed)
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
#10292
The query was processing *every* file under `/Applications/`, which
makes it super expensive both in CPU usage and Memory footprint. This
query was the main culprit of triggering worker process kills by the
watchdog.
On some runs it triggered CPU usage alerts:
```
7716:W0623 15:38:05.402959 221732864 watcher.cpp:415] osqueryd worker (72976) stopping:
Maximum sustainable CPU utilization limit 1200ms exceeded for 12 seconds
```
And on other runs it triggered memory usage alerts:
```
4431 W0626 07:28:50.868021 147312640 watcher.cpp:424] osqueryd worker (21453) stopping:
Memory limits exceeded: 214020096 bytes (limit is 200MB)
```
For the above logs I used a custom osqueryd branch to be able to print
more information: https://github.com/osquery/osquery/pull/8070
The metrics for the old query were CPU usage: ~4521 ms
```
435:level=warn ts=2023-06-26T09:58:29.665712Z query=fleet_policy_query_1233 queryTime=4521 memory=12226560 msg="distributed query performance is excessive" hostID=308 platform=darwin
```
With the new query, CPU usage: ~210 ms.
```
23893:level=debug ts=2023-06-26T18:06:08.242456Z query=fleet_policy_query_1233 queryTime=210 msg=stats memory=0 hostID=308 platform=darwin
```
Basically a ~20x improvement.
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- ~[ ] Added/updated tests~
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
## Addresses #11037
### Implement the `privacy_preferences` table for the Fleetd Chrome
extension. Columns correspond to the available properties of
[`chrome.privacy`](https://developer.chrome.com/docs/extensions/reference/privacy/).
Chrome on mac:
<img width="816" alt="Screenshot 2023-06-23 at 11 55 21 AM"
src="https://github.com/fleetdm/fleet/assets/61553566/a4700749-6325-442e-acf2-c14b1c9adf8f">
Chromebook with enterprise access (actual use case):
* Chromebook w/o enterprise access: as you can see, sometimes certain
APIs are not available - this error occurs because the expected API
object that would have a `get` method is actually `undefined` TODO – How
to handle this case given that we want to let errors bubble up to the
level at which Fleet can catch them? Maybe it would be nice to catch
such errors and send them up to the Fleet layer, and still allow the
loop to continue to populate the columns whose APIs _are_ available.
_Decision: catch API errors here to preserve functionality of the
remaining columns_
- [x] Changes file
- [x] Manual QA
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
This PR requires the Windows MDM configuration changes - This will be
updated next week
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- [x] Documented any permissions changes
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
relates to #11932
This improves the UI error messaging for AMB 400 errors
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
Issue reported by @jarodreyes.
The user was not informed that `fleetctl login` was using the PASSWORD
environment variable:
`main`:
```sh
export PASSWORD=wrong
fleetctl login
Log in using the standard Fleet credentials.
Email: a@b.c
Error: Login failed: login received status 401 Authentication failed: Authentication failed
```
And with the changes on this PR:
```
export PASSWORD=wrong
fleetctl login
Log in using the standard Fleet credentials.
Email: a@b.c
Using value of environment variable $PASSWORD as password.
Error: Login failed: login received status 401 Authentication failed: Authentication failed
```
## Addresses
[confidential/2940](https://github.com/fleetdm/confidential/issues/2940)
Patched a potential security issue in UI
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
#11266
PS: I first attempted a serialization trick by introducing a new
`appConfigResponse` and implementing `json.Marshal` to exclude these
fields but it was too hacky and hard to maintain moving forward, so I'm
bitting the bullet now. Happy to hear other ideas.
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
## Addresses #11828
- [x] Add ChromeOS platform filter
- [x] Increase dropdown widths to 180px for screen >1100px of:
- [x] platform dropdown
- [x] Labels dropdown (for consistency)
- [x] Add new null empty cell value “Not supported” for Chromebooks
- [x] Apply to the following columns:
- [x] Disk space available
- [x] MDM status
- [x] MDM server URL
- [x] Last restarted
- [x] Fix a misaligned icon in the labels dropdown, other small fixes
<img width="1184" alt="Screenshot 2023-06-06 at 7 09 26 PM"
src="https://github.com/fleetdm/fleet/assets/61553566/ee6fb72c-f66e-44a4-a321-8eaefff3b70b">
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
for #11257, h/t to @mna for the idea of resetting `token_update_tally`.
this is to cover scenarios where a host might be re-enrolling (eg: the
device has been wiped) but we don't know about it.
since `TokenUpdate` might be called multiple times during the lifecycle
of an MDM enrollment, we add a check on the value of
`nano_enrollments.token_update_tally`. For the scenarios described
above, the tally is still `> 0` even thought the host is enrolling for
the first time.
to mitigate this, we reset its value to 0 when we receive an
`Authenticate` message (which only happens only per enrollment)
I set the value to `0` because it's incremented to `current_value+1` by
nanomdm before calling our handler.
## Addresses #11825
- [x] Add ChromeOS to Dashboard page: <img width="1365" alt="Screenshot
2023-06-02 at 4 01 12 AM"
src="https://github.com/fleetdm/fleet/assets/61553566/e846c4b6-5fcb-4847-af05-67b2237ada39">
- [x] Add to platforms dropdown, confirm order of platform options, add
route
- [x] Hosts summary card
- [x] Add responsiveness for <980px <img width="952" alt="Screenshot
2023-06-02 at 4 02 44 AM"
src="https://github.com/fleetdm/fleet/assets/61553566/93662957-c590-40e0-876d-6ce4adabad2b">
- [x] TODO: Confirm label number of chrome hosts label - ask Juan on
[this issue](https://github.com/fleetdm/fleet/issues/11829) - needed to
call an API to get this id
- [x] Missing hosts card (didn’t need any changes)
- [x] Low disk space hosts card (Not supported)
- [x] Operating systems card
**Note for reviewers:** There is an API call happening from the
HostsSummary component to get the id for the ChromeOS label needed for
the URL to the filtered manage hosts page. This feature working properly
depends on the response from that endpoint, which is WIP. UPDATE 6/5 -
the endpoint is now working and being called correctly, though the id
being returned is WIP (backend). No need to replace anything to test.
## Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
## Addresses #11856
Improve performance of the rendering of live query results by:
- rendering the table on a set interval instead of with each new result
- preventing redundant rerenders of various sorts
Partial run, with memory leak:
<img width="2552" alt="partial run with memory leak, after smaller
optimizations, before debouncing queryResults"
src="https://github.com/fleetdm/fleet/assets/61553566/5288bffb-6940-43da-9083-59adb4a25916">
Full run after debounce, no memory leak (10x improvement of max JS heap
size):
<img width="2559" alt="full run after debounce, no memory leak"
src="https://github.com/fleetdm/fleet/assets/61553566/be056610-e7a5-4289-a433-1070cf016e83">
**NOTE** - there are further optimizations to try on this page, and the
debounce interval can potentially be shortened to improve UX. In
experimenting with that, it's not immediately clear what a good balance
of UX / performance is. Since the customer seems keen to solve this, I
think we should merge as-is and send them a demo build to confirm this
fixes their problem, then iterate once they've confirmed it does.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
Co-authored-by: Sarah Gillespie <sarah@fleetdm.com>
for #10605, this modifies the cron used to ping the list/sync devices
API from ABM to account for the "deleted" and "modified" operation
types.
We know that:
1. Sometimes, Apple sends a "modified" operation type when a device's
MDM server is reassigned in ABM, up until now, we were ignoring these
devices.
2. Devices that are no longer assigned to Fleet in ABM can't be
migrated.
relates to #11238
This implements the Verified status for the profile statute on the macOS
settings pages and the Host Details and My Device pages.
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
- [x] Manual QA for all new/changed functionality
- Add section for viewing recent MDM commands using fleetctl get mdm-commands
- Add example output for fleetctl get mdm-commands and fleetctl get mdm-command-results
This could help future users to detect this issue: #10957
It also adds an error log in Fleet that prints the actual error.
The error is displayed if I kill Redis during a live session or if I set
`client-output-buffer-limit` to something real low like `CONFIG SET
"client-output-buffer-limit" "pubsub 100kb 50kb 60"`:
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- ~[ ] Added/updated tests~
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
For issues #12003 and #12051
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
#10784
The removal of the now deprecated `sso_settings.enable_jit_role_sync`
config will be tackled in: #10688.
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
#10878
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
- Added index on `created_at` which is the sort key used when loading the log activities widget on the dashboard.
- Refactored query used when loading activities to avoid a full table scan.
This was found while working on #10957.
When running a live query, a lot of unused host data is stored in Redis
and sent on every live query result message via websockets. The frontend
and fleetctl just need `id`, `hostname` and `display_name`. (This
becomes worse every time we add new fields to the `Host` struct.)
Sample of one websocket message result when running `SELECT * from
osquery_info;`:
size in `main`: 2234 bytes
```
a["{\"type\":\"result\",\"data\":{\"distributed_query_execution_id\":57,\"host\":
{\"created_at\":\"2023-05-22T12:14:11Z\",\"updated_at\":\"2023-05-23T12:31:51Z\",
\"software_updated_at\":\"0001-01-01T00:00:00Z\",\"id\":106,\"detail_updated_at\":\"2023-05-23T11:50:04Z\",
\"label_updated_at\":\"2023-05-23T11:50:04Z\",\"policy_updated_at\":\"1970-01-02T00:00:00Z\",
\"last_enrolled_at\":\"2023-05-22T12:14:12Z\",
\"seen_time\":\"2023-05-23T09:52:23.876311-03:00\",\"refetch_requested\":false,
\"hostname\":\"lucass-macbook-pro.local\",\"uuid\":\"BD4DFA10-E334-41D9-8136-D2163A8FE588\",\"platform\":\"darwin\",\"osquery_version\":\"5.8.2\",\"os_version\":\"macOS 13.3.1\",\"build\":\"22E261\",\"platform_like\":\"darwin\",\"code_name\":\"\",
\"uptime\":91125000000000,\"memory\":34359738368,\"cpu_type\":\"x86_64h\",\"cpu_subtype\":\"Intel x86-64h Haswell\",\"cpu_brand\":\"Intel(R) Core(TM) i7-1068NG7 CPU @ 2.30GHz\",\"cpu_physical_cores\":4,\"cpu_logical_cores\":8,\"hardware_vendor\":\"Apple Inc.\",\"hardware_model\":\"MacBookPro16,2\",\"hardware_version\":\"1.0\",
\"hardware_serial\":\"0DPQR4HMD1FZ\",
\"computer_name\":\"Lucas’s MacBook Pro\",\"public_ip\":\"\",
\"primary_ip\":\"192.168.0.230\",\"primary_mac\":\"68:2f:67:8e:b6:1f\",
\"distributed_interval\":1,\"config_tls_refresh\":60,\"logger_tls_period\":10,\"team_id\":null,
\"pack_stats\":null,\"team_name\":null,
\"gigs_disk_space_available\":386.23,\"percent_disk_space_available\":40,
\"issues\":{\"total_issues_count\":0,\"failing_policies_count\":0},
\"mdm\":{\"enrollment_status\":null,\"server_url\":null,\"name\":\"\",\"encryption_key_available\":false},
\"status\":\"online\",\"display_text\":\"lucass-macbook-pro.local\",\"display_name\":\"Lucas’s MacBook Pro\"},
\"rows\":[{\"build_distro\":\"10.14\",\"build_platform\":\"darwin\",
\"config_hash\":\"b7ee9363a7c686e76e99ffb122e9c5241a791e69\",\"config_valid\":\"1\",
\"extensions\":\"active\",\"host_display_name\":\"Lucas’s MacBook Pro\",
\"host_hostname\":\"lucass-macbook-pro.local\",\"instance_id\":\"cde5de81-344b-4c76-b1c5-dae964fdd4f2\",\"pid\":\"8370\",\"platform_mask\":\"21\",\"start_time\":\"1684757652\",
\"uuid\":\"BD4DFA10-E334-41D9-8136-D2163A8FE588\",
\"version\":\"5.8.2\",\"watcher\":\"8364\"}],\"error\":null}}"]
```
vs. size of the message result on this branch: 675 bytes
```
a["{\"type\":\"result\",\"data\":{\"distributed_query_execution_id\":59,
\"host\":{\"id\":106,\"hostname\":\"lucass-macbook-pro.local\",
\"display_name\":\"Lucas’s MacBook Pro\"},
\"rows\":[{\"build_distro\":\"10.14\",\"build_platform\":\"darwin\",
\"config_hash\":\"f80dee827635db39077a458243379b3ad63311fd\",
\"config_valid\":\"1\",\"extensions\":\"active\",\"host_display_name\":\"Lucas’s MacBook Pro\",
\"host_hostname\":\"lucass-macbook-pro.local\",
\"instance_id\":\"cde5de81-344b-4c76-b1c5-dae964fdd4f2\",\"pid\":\"8370\",\"platform_mask\":\"21\",
\"start_time\":\"1684757652\",\"uuid\":\"BD4DFA10-E334-41D9-8136-D2163A8FE588\",\"version\":\"5.8.2\",
\"watcher\":\"8364\"}]}}"]
```
Manual tests included running with an old fleetctl running with a new
fleet server, and vice-versa, a new fleetctl running against an old
fleet server.
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
## Addresses #11737
- Write function to enforce Fleet sentence-casing standards
- Apply it to this bug
<img width="642" alt="Screenshot 2023-05-18 at 12 43 20 PM"
src="https://github.com/fleetdm/fleet/assets/61553566/670f4f8d-1c23-4609-bb23-c38038e9bbd8">
*NOTE - this (the host details) endpoint currently returns label names
in Sentence Case – this solution deals with only the UI presentation,
but it might be worth changing the API response in the future:
<img width="369" alt="Screenshot 2023-05-18 at 12 48 58 PM"
src="https://github.com/fleetdm/fleet/assets/61553566/27236524-9c0a-4818-8a74-f445b5765d94">
## Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
## Addresses #11714
- Allow saving policy automations to Ticket integrations without
defining a webhook URL
https://www.loom.com/share/208169b6081846809dc170fcfb43f689
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
relates to #11048
improves the loading UI for the disk encryption page. The aggregate
profile summary no longer pops in and pushed down disk encryption
section.
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
relates to #9996
Added support for wildcard search on host search.
say for example you have the following hosts:
```
+------------------+
|hostname |
+------------------+
|Molly‘s MacbookPro|
|Molly's MacbookPro|
|Molly‘s MacbookPro|
|Molly❛s MacbookPro|
|Molly❜s MacbookPro|
|Alex's MacbookPro |
+------------------+
```
searching for `Molly's` yields just the single host, but searching for
`Molly❜s` will perform a broader wildcard search using the literal `_`
character to match any character _in that position_.
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added/updated tests
## Addresses #11338
- Validate emails on login page
- Fix jumping error state for no email provided ("Email field must be
completed")
- Fix jumping error state for password field
- Fix jumping error state for Forgot password > email field
https://www.loom.com/share/92a238fcd2614d6e8d2655d571aa2757
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
relates to #11450
This will show the profile status aggregate UI at all times when on the
macOS settings page. This is a change from showing it conditionally.
This also cleans up where some of the requests occur to move it closer
to where it is needed and changing the `MdmProfileStatus` enum to a
union.
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
## Addresses #9371
### Adds a suite of UI logic for premium features in the Sandbox
environment
For reviewer: please review the work for the below 3 substasks, which
are the only remaining subtasks encompassed by this PR that have not yet
passed review individually:
- #10822 (9)
- #10823 (10)
- #10824 (11)
## Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
## Addresses #11188
When an _already authenticated_ no-access user tries to access any
authenticated routes:
- Log the user out
- Display the 403 'Forbidden' error page
https://www.loom.com/share/358fd5b534984ab9ab40220986a7d094
The user _can_ still log in – see attached issue.
## Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
#11089
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- [X] Documented any permissions changes
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [x] Added/updated tests
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
## Addresses #11088
Updated solution:
https://www.loom.com/share/e3ca30bebde54df5b8cbd8cac6d26780
- First refresh displays the bug: a white bar at the top of the page as
the login page loads.
- Second refresh displays the login page with the bug gone
- Navigate at the end to an error page to confirm it is unaffected,
since the initial solution affected the background color of those pages
## Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
In #10338 we introduced logic to gate DEP profiles behind Okta auth
using the ROP flow.
We're not going to use that, and instead we're going to gate profiles
behind SSO, which can be used from multiple providers and supports SSO
(the initial motivation behind the ROP flow was to create a local user
account.)
This removes some of the old code, which was never used in
production/documented for the public to use.
At the moment I'm leaving the `mdm_idp_accounts` table and related
methods untouched, as it's unclear yet if we're going to need a similar
auxiliar table, and I would rather deal with the migrations all at once.
#10880
I was not able to reproduce other 500s in `/api/_version_/fleet/hosts`
other than the one fixed in the PR.
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
## Addresses #10913
- Fixes the bug(s) outlined in the issue
- Improves look and feel of the tabbing experience throughout the setup
flow: https://www.loom.com/share/c482317d18314e629b7e5ebab7fd2840
## Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
#10798
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
#8593
This PR adds a new role `gitops` to Fleet.
MDM capabilities for the role coming on a separate PR. We need this
merged ASAP so that we can unblock the UI work for this.
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- [X] Documented any permissions changes
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [x] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
## Addresses #10756https://www.loom.com/share/b5ba605b892c4c789a73f05d160819cc
- Set the teams dropdown's max-height to be 66% of the viewport height
- Made a small script to automate creation of an arbitrary number of
teams
- [x] Changes file added for user-visible changes in `changes/`
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
This should get version numbers for more apps on macOS. Notably,
1Password includes helper apps that were getting vulnerability false
positives because we were not picking up the versions.
Addresses #10702.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
#8593
This PR adds a new role `observer_plus` to Fleet. (The `GitOps` role
will be added on a separate PR.)
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- [X] Documented any permissions changes
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
https://github.com/fleetdm/confidential/issues/1968
It's ready for review but I still need to load test this.
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
relates to #9436
Implementation of the API supporting filtering host by disk encryption
status. This adds this through a `macos_settings_disk_encryption` query
param that can be passed to these endpoints:
`GET /hosts`
`GET /hosts/count`
`GET /lables/:id/hosts`
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
Related to #9459, this adds logic to the cron to add a
`com.fleetdm.fleetd.config` configuration profile to the
`apple_mdm_configuration_profiles` table.
As noted in the comments, this makes some assumptions:
- This profile will be applied to all hosts in the team (or "no team",)
but it will only be used by hosts that have a fleetd installation
without
an enroll secret and fleet URL (mainly DEP enrolled hosts).
- Once the profile is applied to a team (or "no team",) it's not removed
if
AppConfig.MDM.AppleBMDefaultTeam changes, this is to preserve existing
agents using the configuration (mainly ServerURL as EnrollSecret is used
only during enrollment)
## Addresses #9988
* Adjust copy in tooltips to take up less width
* Refactor table headers to take an optional "isLastColumn" property
that is set to true when that header is in the last column.
* Use above property in conjunction with presence of TooltipWrapper as a
value for the header cell to add a class specific to that state.
* Use that class to adjust the location of the tooltip text and the
min-width of the column to avoid the bug.
The 3 states which exhibited this bug, now fixed:
<img width="1496" alt="Screenshot 2023-03-27 at 4 36 01 PM"
src="https://user-images.githubusercontent.com/61553566/228091971-4d5d034d-55c5-4921-955a-4946119f7785.png">
<img width="1496" alt="Screenshot 2023-03-27 at 4 36 12 PM"
src="https://user-images.githubusercontent.com/61553566/228091968-adf90b32-8fd2-45d9-b56d-a64c654151ef.png">
<img width="1496" alt="Screenshot 2023-03-27 at 4 36 24 PM"
src="https://user-images.githubusercontent.com/61553566/228091962-ff626daa-b13d-4093-b34f-de704b820161.png">
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
