Commit graph

1750 commits

Author SHA1 Message Date
Noah Talerman
b7868831c2
Scripts: update host timeout error message (#13626)
- Update copy for erorr message we display when we haven't heard back
from the host in over 1 minute.
2023-08-31 18:01:14 -04:00
gillespi314
72f2f7ac12
Adjust error messages for run scripts API (#13618) 2023-08-31 10:37:51 -05:00
Martin Angers
cbc3f32e9d
Adjust response payload, messages and validations for /scripts/run/* endpoints. (#13607) 2023-08-31 09:08:50 -05:00
Lucas Manuel Rodriguez
9142c5de79
Prevent thundering herd when applying large number of policies on large number of hosts (#13552)
#13527

(Adding @mna to double check the changes in the async implementation of
policy result storage)

This PR also adds the osquery-perf changes needed to define the count of
macOS and Windows hosts.

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes (docs/Using
Fleet/manage-access.md)~
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
  - ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~

Test with 80k hosts: 70k simulated macOS, 10k simulated Windows.
Apply Windows policies first, then apply macOS policies:
```
fleetctl apply -f ee/cis/win-10/cis-policy-queries.yml

# Leave running for some time

fleetctl apply -f ee/cis/macos-13/cis-policy-queries.yml
```

After applying CIS policies previous to these changes:
![Screenshot 2023-08-23 at 11 36
18](https://github.com/fleetdm/fleet/assets/2073526/72c1dc7d-e601-4248-be35-93c85b749f5d)

After applying these changes and applying the same policies:
![Screenshot 2023-08-28 at 15 42
57](https://github.com/fleetdm/fleet/assets/2073526/6b6d76b8-6acb-4893-a913-bf603a68f1a4)
2023-08-31 10:58:50 -03:00
Tim Lee
222b8f9f5c
paginate the policies API (#13459) 2023-08-30 16:30:17 -06:00
Martin Angers
090b142c49
Implement script execution on the fleetd agent (disabled by default) (#13569) 2023-08-30 14:02:44 -04:00
Roberto Dip
b50e1939db
Allow to configure fleetd for script execution (#13564)
Related to #13310 and #13304 this adds two ways to enable script
execution in `fleetd` (the orbit component)

- By building a package with `--enable-scripts`
- By providing a setting via a configuration profile (macOS only)

Due to how the profile assignment works, this change automatically
updates the `com.fleetdm.fleetd.config` for hosts that already have the
profile installed.

> [!NOTE]
> Documentation is in
[#13577](https://github.com/fleetdm/fleet/pull/13577) to decouple
reviews.
2023-08-30 10:18:34 -03:00
Gabriel Hernandez
f810fc31e2
use OrbitNodeKey for windows mdm enrollment authentication instead of HostUUID (#13503)
related to #12847

This changes the authentication method for windows mdm enrollment. We
were using `HostByIndentifier ` method but have changed to
`LoadHostByOrbitNodeKey`.

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2023-08-29 14:50:13 +01:00
Shawn Maddock
5ecd9e5dbe
Add JumpCloud to the list of well-known MDM solutions (#13554)
# Checklist for submitter
2023-08-29 10:41:18 -03:00
Roberto Dip
183e2e56cf
automatically set DEP profile for teams created by Puppet (#13496)
for #13363
2023-08-28 11:36:00 -03:00
Tim Lee
6c7edca368
environment variable to disable orbit enroll logs (#13519) 2023-08-25 15:25:07 -06:00
Roberto Dip
4be557bb57
allow padded strings in mdm/apple/enqueue endpoint (#13502)
for #11384
2023-08-24 15:17:05 -03:00
Roberto Dip
5c7019cfc4
allow clients to report errors back to the server (#13478)
for #13189, #13238 and #13239
2023-08-24 13:04:27 -03:00
Martin Angers
edf4a4d02f
Add script execution simulation to osquery-perf in preparation for load testing (part 3 of ticket) (#13456) 2023-08-23 18:31:47 -04:00
Roberto Dip
d5c7e7eb51
store email used for authentication during MDM SSO (#13480)
related to #13431, this stores the email during SSO auth. Still left to
figure out how to link this email to an specific host.
2023-08-23 18:23:26 -03:00
Martin Angers
4045116fac
Add orbit notification and API endpoints to send/receive scripts to hosts (part 2 of ticket) (#13447) 2023-08-23 16:47:47 -04:00
Tim Lee
29e48f402a
Hotfix: add empty host slice validation in ListHosts (#13483) 2023-08-23 13:57:18 -06:00
Tim Lee
74ccff8161
13433 host query optimization (#13451) 2023-08-23 10:34:55 -06:00
Martin Angers
de32faefdb
Add /scripts/run and scripts/run/sync API endpoints to run scripts (part 1) (#13417) 2023-08-21 14:47:19 -04:00
Tim Lee
3b61adf7a4
Add validation for policy specs (#13294) 2023-08-21 10:22:07 -06:00
gillespi314
e08bb000c9
Update nanomdm dependency (#12721)
Updates include:
- Fix issues where `GetBootstrapToken` returned `500` instead of no data
and no error per Apple MDM
[documentation](https://developer.apple.com/documentation/devicemanagement/get_bootstrap_token)
- Incorporate additional updates from the upstream nanomdm repo
2023-08-21 11:07:57 -03:00
gillespi314
d140a57355
Allow failed profiles to become verified if subsequently reported by osquery (#13343) 2023-08-18 12:34:13 -03:00
Roberto Dip
998e1dfb6b
fix issues when MDM info is empty during migration (#13320)
for #13319
2023-08-14 19:21:06 -03:00
Roberto Dip
902e064d04
fix issues with migration flow (#13297)
For #13094
2023-08-14 09:56:59 -03:00
Roberto Dip
d845720c2d
fix: ensure we assign ABM profiles for modified hosts (#13275)
for #12958 and #13110
2023-08-10 19:51:17 -03:00
Roberto Dip
70bdfe9512
fix panic if concurrent requests write capabilities (#13278) 2023-08-10 19:49:07 -03:00
Martin Angers
b3d0192995
Return 403 instead of 500 when conditions are not met to perform a required pwd reset (#13244) 2023-08-09 15:28:04 -04:00
Gabriel Hernandez
6555d8def4
Feat UI windows automatic enrollment (#12988)
relates to #12606

Implementation of the Windows automatic enrollment Fleet UI pages. This
includes implementation of card for windows automatic enrollment, the
setup page for windows automatic enrollment, and terms and conditions
page for windows (This is currently still being worked on as our current
solution is not working).

**windows mdm auto enrollment card**


![image](https://github.com/fleetdm/fleet/assets/1153709/d4dc1813-dc28-4a63-bacd-cb7e43e18170)

**windows auto enrollment setup page**


![image](https://github.com/fleetdm/fleet/assets/1153709/92da4b05-0d5d-4404-867f-6d315957bdc3)

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Marcos Oviedo <marcos@fleetdm.com>
2023-08-08 15:57:55 +01:00
Roberto Dip
6d186602cd
add flags to configure Windows MDM certs using their contents (#13184)
related to #13033

seems like we're not currently documenting any Windows MDM config flags,
so I left those undocumented too.
2023-08-08 10:54:50 -03:00
Gabriel Hernandez
b4ee1c58b5
update OSversion service method to not search on for no team (#13127)
relates to #12986

This is a fix for a case where the `os_versions` endpoint was returning
a 404 when searching for `no teams`.

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
2023-08-08 13:30:29 +01:00
Martin Angers
37ba43d404
Add fleetctl upgrade-packs command to migrate 2017 packs to queries (#13078) 2023-08-08 08:21:57 -04:00
Martin Angers
554e024f7b
Fix gitops access when using --dry-run with fleetctl apply (#13178) 2023-08-07 13:51:11 -04:00
Roberto Dip
8fda48db8b
use only the UUID part of external_host_identifier for Puppet runs (#13176)
related to #12483, we have found out that in distributed scenarios, the
URL of the Puppet server used for the request is appended to the
identifier, and it can be different between `/preassign` and `/match`
calls.

to account for this, we're only grabbing the first 36 characters of the
identifier.
2023-08-07 12:41:13 -03:00
gillespi314
9ae3aa8036
Update MDM profile verification (#13138) 2023-08-07 09:46:03 -05:00
gillespi314
5b27581fdc
Configure bootstrap package and end user auth for newly created teams in MDM pre-assignment flow (#13089) 2023-08-07 09:43:39 -05:00
Benjamin Edwards
f7296de183
expose env var for orbit enrollment retry interval (#12991) 2023-08-04 14:50:03 -07:00
Roberto Dip
29b9a7fe88
account for NULL values in scheduled_query columns in data migration (#13142)
Prior to 4.35.0, some rows in the scheduled_query table might have a
`NULL` value due to a race condition with database replicas and the way
`ds.EnsureGlobalPack` and `ApplyPackSpecs` work together.

This is no longer the case, but some databases are left in weird states,
which were not accounted by this migration.

Chaning the migration in-place because that's the approach we took in
previous migrations with similar problems.
2023-08-04 09:24:36 -07:00
Gabriel Hernandez
cb89b725ea
implement fleetctl apply new team with secret (#13062)
relates to #12907

implement adding an enroll secret when creating a new team with
`fleetctl apply` if none is provided.

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Added/updated tests
- [x] Manual QA for all new/changed functionality
2023-08-02 17:42:01 +01:00
Marcos Oviedo
7cfea0787e
Windows Installer changes to support MDM Azure flow (#13025)
This relates to #12600 

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
- [X] Manual QA for all new/changed functionality
2023-07-31 12:12:06 -03:00
Jacob Shandling
3bf0344396 Merge branch 'main' into 7765-combined-schedules-and-queries 2023-07-28 13:21:04 -07:00
Jacob Shandling
1cf532dfbb
UI - Fix a synchronicity issue in automations API patches (#13035)
## Addresses #13007 

## Fix a synchronicity issue in automations API patches
<img width="825" alt="Screenshot 2023-07-28 at 12 49 21 PM"
src="https://github.com/fleetdm/fleet/assets/61553566/9729dcd0-4813-452a-9911-6cf60f8352af">

- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
2023-07-28 13:01:50 -07:00
Juan Fernandez
40e8f83829
Always use writer node when listing queries. (#13024)
Always use writer node when listing queries.
2023-07-28 12:41:05 -04:00
Eric
8fb22579ea
Reorganize Fleet documentation (#12871)
Closes: #12611

Changes:
- Added three new documentation sections `/docs/get-started/`,
`/docs/configuration` and `/docs/rest api/`
- Updated folder names: `/docs/Using-Fleet/` » `/docs/Using Fleet` and
`/docs/deploying` » `/docs/deploy/`
- Moved `/docs/using-fleet/process-events.md` to `/articles` and updated
the meta tags to change it into a guide.
- Added support for a new meta tag: `navSection`. This meta tag is used
to organize pages in the sidebar navigation on fleetdm.com/docs
- Moved `docs/using-fleet/application-security.md` and
`docs/using-fleet/security-audits.md` to the security handbook.
- Moved `docs/deploying/load-testing.md` and
`docs/deploying/debugging.md` to the engineering handbook.
- Moved the following files/folders:
- `docs/using-fleet/configuration-files/` »
`docs/configuration/configuration-files/`
- `docs/deploying/configuration.md` »
`docs/configuration/fleet-server-configuration.md`
    -  `docs/using-fleet/rest-api.md` » `docs/rest-api/rest-api.md`
- `docs/using-fleet/monitoring-fleet.md` » `docs/deploy/rest-api.md`
- Updated filenames:
- `docs/using-fleet/permissions.md` »
`docs/using-fleet/manage-access.md`
- `docs/using-fleet/adding-hosts.md` »
`docs/using-fleet/enroll-hosts.md`
    -  `docs/using-fleet/teams.md` » `docs/using-fleet/segment-hosts.md`
- `docs/using-fleet/fleet-ctl-agent-updates.md` »
`docs/using-fleet/update-agents.md`
- `docs/using-fleet/chromeos.md` »
`docs/using-fleet/enroll-chromebooks.md`
- Updated the generated markdown in `server/fleet/gen_activity_doc.go`
and `server/service/osquery_utils/gen_queries_doc.go`
- Updated the navigation sidebar and mobile dropdown links on docs pages
to group pages by their `navSection` meta tag.
- Updated fleetdm.com/docs not to show pages in the `docs/contributing/`
folder in the sidebar navigation
- Added redirects for docs pages that have moved.

.

---------

Co-authored-by: Mike Thomas <mthomas@fleetdm.com>
Co-authored-by: Rachael Shaw <r@rachael.wtf>
2023-07-27 17:40:01 -05:00
gillespi314
09e6bf9807
Update detail query ingestion and UI related to Windows MDM status (#12960) 2023-07-26 17:35:55 -05:00
gillespi314
ac8afb41dc
Mask secret query params when logging url error (#12980) 2023-07-26 16:40:58 -05:00
Martin Angers
6f77911ffe
Fix performance regression found in load testing (#12981) 2023-07-26 17:13:27 -04:00
Juan Fernandez
a7f6686c27
Don't leak Observer permissions between teams for queries. (#12979)
If user is an observer in a team, only show the queries for which observer_can_run is set.
2023-07-26 15:33:21 -04:00
Roberto Dip
442e03b276
Improve the error handling for MDM SSO during DEP enrollment (#12966)
For #12692
2023-07-26 14:20:36 -03:00
Lucas Manuel Rodriguez
2afbd24021
Combine Schedules and Queries: API changes (#12778)
Combining schedules and queries API changes.
2023-07-24 20:17:20 -04:00
Juan Fernandez
a265559ee7
Combined schedules and queries data migration (#12855)
Added data migration for migrating scheduled queries in the global and team packs to the new query structure.
2023-07-24 19:59:34 -04:00