For #11858, I reproduced the issue by running a local server behind
ngrok, _with the exact_ same path as the one in the website:
`https://server-url/images/permanent/mdm-migration-screenshot-768x180@2x.png`
I tried multiple combinations, but at the end, removing the `@` made the
trick. My guess is that's something to do with the markdown parser
library used by swiftDIalog.
I also removed a rogue `\` that was being displayed.
#7970
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
#11065
Since `secret.txt` is written when the installer is built, but
installers using `--use-system-config` don't have an enroll secret at
build time, this file was empty and causing osquery to have trouble
enrolling.
This PR writes the file when the values are read from a configuration
profile.
in #10134 we added a silent mechanism to try to read configuration
values from macOS configuration profiles if --fleet-url and
--enroll-secret weren't present.
while using this logic to test #9459 I have found that there's a race
condition where sometimes `fleetd` is installed before the configuration
profile with the values delivered by Fleet, causing orbit to get stuck
forever.
I added logic to loop every 30 seconds and try to fetch the values again
if none are found, but I didn't felt comfortable adding this logic
without also adding an extra flag to explicitly enable this behavior.
#9132
The actual fix for the empty hosts is adding the `--database_path`
argument in the initial `osqueryd -S` invocation when retrieving the
UUID. Osquery attempts to retrieve the UUID from OS files/APIs, when not
possible (which is what happens on some linux distributions), then it
resorts to generating a new random UUID and storing it in the
`osquery.db`. The issue was Orbit's first invocation of `osqueryd -S`
was not using the same `osquery.db` as the main daemon invocation of
`osqueryd`.
I'm also adding a `hostname` + `platform` to the orbit enroll phase so
that if there are any issues in the future we can avoid the "empty" host
and have some information to help us troubleshoot.
## How to reproduce
On Linux, osquery reads `/sys/class/dmi/id/product_uuid` to load the
hardware UUID.
Some Linux distributions running on specific hardware or container
environments do not have such file available.
The way to reproduce on a Linux VM is to do the following:
```sh
$ sudo su
# chmod -r /sys/class/dmi/id/product_uuid
```
which will turn the file inaccessible to root.
## Checklist
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [X] Added/updated tests
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
This relates to #10300
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [X] Manual QA must be performed in the three main OSs, macOS, Windows,
and Linux.
- [X] Auto-update manual QA from the released version of the component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
---------
Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>
This allows orbit to read enroll-secret and fleet-url from a
configuration profile if both values are not set when the package is
built.
Part of https://github.com/fleetdm/fleet/issues/9459
this accomplishes two things:
1. We're not waiting on Nudge to exit anymore, preventing issues like
https://github.com/fleetdm/fleet/issues/10044
2. Nudge is launched as a local user instead of root, which is the
recommended way to do it.
We have code that builds conditionally depending on the platform (mostly
Orbit code) so we should run `golangci-lint` checks on all OSs.
This adds it to run on macOS, for Windows see:
https://github.com/fleetdm/fleet/issues/9943
#9253
- ~[ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.~
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [X] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
---------
Co-authored-by: Sharon Katz <121527325+sharon-fdm@users.noreply.github.com>
There was an issue when the osqueryd instance ran through `orbit shell` attempted to register the same named pipe name used by the osqueryd instance launched by orbit service
This may be needed for CIS 2.3.2.2 check:
```
Correct date and time settings are required for authentication protocols, file creation,
modification dates and log entries. Ensure that time on the computer is within
acceptable limits. Truly accurate time is measured within milliseconds. For this audit, a
drift under four and a half minutes passes the control check. Since Kerberos is one of
the important features of macOS integration into Directory systems, the guidance here
is to warn you before there could be an impact to operations. From the perspective of
accurate time, this check is not strict, so it may be too great for your organization. Your
organization can adjust to a smaller offset value as needed.
```
#9239
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- ~[ ] Added/updated tests~
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [X] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
the `switch` statement that checks for errors (including license errors) issues a `continue` before we even have the chance to wait for the ticker.
this has the drawback that premium users will have to wait 5 minutes before they see policy info, but the alternative would be to use labels and go-to, at least with the current code structure.
related to https://github.com/fleetdm/fleet/issues/8373
* Adding a new synchronization mechanism between fleet-desktop app and Orbit service. Improved windows service teardown to ensure that fleet-desktop does not get force killed without getting signaled. Improved windows process enumeration to avoid unnecessary delays during windows service start and windows service teardown. Updating windows service to reflect service teardown extra time due to synchronization.
* Support environments with revoked enroll secrets
* Add instructions on how to fix Orbit enroll
* Rename to last_recorded_error
* Add alternative instructions
This adds a new mechanism to allow us to handle compatibility issues between Orbit, Fleet Server and Fleet Desktop.
The general idea is to _always_ send a custom header of the form:
```
fleet-capabilities-header = "X-Fleet-Capabilities:" capabilities
capabilities = capability * (,)
capability = string
```
Both from the server to the clients (Orbit, Fleet Desktop) and vice-versa. For an example, see: 8c0bbdd291
Also, the following applies:
- Backwards compat: if the header is not present, assume that orbit/fleet doesn't have the capability
- The current capabilities endpoint will be removed
### Motivation
This solution is trying to solve the following problems:
- We have three independent processes communicating with each other (Fleet Desktop, Orbit and Fleet Server). Each process can be updated independently, and therefore we need a way for each process to know what features are supported by its peers.
- We originally implemented a dedicated API endpoint in the server that returned a list of the capabilities (or "features") enabled, we found this, and any other server-only solution (like API versioning) to be insufficient because:
- There are cases in which the server also needs to know which features are supported by its clients
- Clients needed to poll for changes to detect if the capabilities supported by the server change, by sending the capabilities on each request we have a much cleaner way to handling different responses.
- We are also introducing an unauthenticated endpoint to get the server features, this gives us flexibility if we need to implement different authentication mechanisms, and was one of the pitfalls of the first implementation.
Related to https://github.com/fleetdm/fleet/issues/7929
As stated by the title, this fixes a bug in token rotation caused by a file having existing read-only permissions. This affects Fleet Desktop after is upgraded if an existing Orbit created a file that's not accessible by non-root users.
I have also included 77a59ac to fix an URL format
* fix old root dir in orbit
* add changes
* Add automation for orbit shell (with TUF)
* Fix workflow syntax
* Add logging to latest fleetctl preview action
* Add changes to fix workflow
* Use macOS host for TUF server and package generation
* Remove copy/paste if clause
* Fix orbit logs on macOS, Ubuntu
* Simplify TUF and generation of packages
* Set enroll secret instead of getting it
* Increase timeouts
* Add step id
* Fixes to the upload/download of artifacts
* Rearrange steps to not lose the downloads
* Fix copy/paste
* Add fleetctl login step
* Add missing config set
* Fix quotes on Windows
* Increase timeout
* Fix job termination
* Disable FLEET_DESKTOP for now
* Checkout repository on macOS
* Fix logs path
* Enable fleet desktop
* Use cancel, nitpick
Co-authored-by: Michal Nicpon <michal@fleetdm.com>
* Add (beta) support for Fleet Desktop to linux
* Add dependency for linux desktop
* Amend makefile uname check
* Clarify env vars used for linux in execuser
* Add final set of fixes
* Remove -it from docker run
* Add desktop to the update runner for Linux
* Re-arrange tag.gz and fix upgrade check for linux desktop
Install orbit to /opt instead of /var/lib. When installing to /var/lib,
the default selinux context of var_lib_t gets applied, which results in
an AVC error when running via systemd.
Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>
Allows identification of which Orbit versions are in use from the update
server.
Refactored the build information into a separate `package build` to
support importing it from multiple places.
* Make Orbit update interval configurable
- Also increase default interval from 10s to 15m
* Add update-interval configuration to fleetctl package (#5050)
Co-authored-by: Lucas Manuel Rodriguez <lucas@fleetdm.com>
* Orbit: Add Fleet Desktop support to Windows
* Rename workflow, fix linux build
* Do not compile systray on linux
* nolint on unused
* Fix lint properly
* nolint both checkers
* Fix monitor logic in desktopRunner
* Fix interrupt and execute order
* Upgrade and replace kolide/osquery-go with osquery/osquery-go
* Upgrade macadmins/osquery-extension to v0.0.7
* Upgrade kolide/launcher to latest
* go mod tidy
* WIP
* WIP2
* Fix orbit and fleetctl tests
* Amend macos-app default
* Add some fixes
* Use fleetctl updates roots command
* Add more fixes to Updater
* Fixes to app publishing and downloading
* Add more changes to support fleetctl cross generation
* Amend comment
* Add pkg generation to ease testing
* Make more fixes
* Add changes entry
* Add legacy targets (until our TUF system exposes the new app)
* Fix fleetctl preview
* Fix bool flag
* Fix orbit logic for disabled-updates and dev-mode
* Fix TestPreview
* Remove constant and fix zip-slip attack (codeql)
* Return unknown error
* Fix updater's checkExec
* Add support for executable signing in init_tuf.sh
* Try only signing orbit
* Fix init_tuf.sh targets, macos-app only for osqueryd
* Specify GOARCH to support M1s
* Add workflow to generate osqueryd.app.tar.gz
* Use 5.2.2 on init_tuf.sh
* Add unit test for tar.gz target
* Use artifacts instead of releases
* Remove copy paste residue
* Fleet Desktop Packaging WIP
* Ignore gosec warning
* Trigger on PR too
* Install Go in workflow
* Pass url parameter to desktop app
* Fix fleetctl package
* Final set of changes for v1 of Fleet Desktop
* Add changes
* PR fixes
* Fix CI build
* add larger menu bar icon
* Add transparency item
* Delete host_device_auth entry on host deletion
* Add SetTargetChannel
* Update white logo and add desktop to update runner
* Add fleet-desktop monitoring to orbit
* Define fleet-desktop app exec name
* Fix update runner creation
* Add API test before enabling the My device menu item
Co-authored-by: Zach Wasserman <zach@fleetdm.com>
* WIP
* WIP2
* Fix orbit and fleetctl tests
* Amend macos-app default
* Add some fixes
* Use fleetctl updates roots command
* Add more fixes to Updater
* Fixes to app publishing and downloading
* Add more changes to support fleetctl cross generation
* Amend comment
* Add pkg generation to ease testing
* Make more fixes
* Add changes entry
* Add legacy targets (until our TUF system exposes the new app)
* Fix fleetctl preview
* Fix bool flag
* Fix orbit logic for disabled-updates and dev-mode
* Fix TestPreview
* Remove constant and fix zip-slip attack (codeql)
* Return unknown error
* Fix updater's checkExec
* Add support for executable signing in init_tuf.sh
* Try only signing orbit
* Fix init_tuf.sh targets, macos-app only for osqueryd
* Specify GOARCH to support M1s
* Add workflow to generate osqueryd.app.tar.gz
* Use 5.2.2 on init_tuf.sh
* Add unit test for tar.gz target
* Use artifacts instead of releases
* Remove copy paste residue
* Trigger workflow on PR
* Fixes to ease handling of artifact
* Fix, do not use target name as dir
* Remove workaround
* Wait for osquery extension manager socket to be ready
* PR review feedback
* Add WIP Github Action to test orbit
* Set fleet address on fleetctl
* Add logging for troubleshooting
* Add prepare db statement
* Use tls-skip-verify on fleetctl
* Move steps around
* Fix addresses
* Fix fleetctl path
* Add certs.pem to orbit root dir
* Add orbit logs
* Increase timeout
* Add proper orbit log paths
* Fix tmp path orbit
* Add get hosts command to troubleshoot
* Fix orbit job termination
* Add comments to workflows
* Wait for server to go down
* Add orbit Windows job
* Use bash on windws-latest
* Fix missing quote
* Run orbit on Ubuntu
* Bump host count
* Increase timeout for extensions and not terminate on err
* Add comment to clarify high timeout value
* Revert change, we do want to exit in case of err
* Add disable-updates flag to fleetctl and orbit
* Fix ruleguard execution error on make lint-go
* Introduce dev-mode for ease of development of orbit
* Add changes file
* Fix orbit and osqueryd logging on Windows
* Add functionality to test the change and use systemprofile app data
* Add centos syslog to README.md
* Add wait on orbit for osquery extension socket to exist (#3571)
* Wait for osquery extension socket to exist
* Amend changes
* Fix lint
* Restore timeout
The database is currently unused and sometimes causing problems in Orbit
deployments due to file corruption on Windows. We may need to look at
something less prone to corruption.
- Orbit automatically loads the flagfile when it exists in the orbit
root.
- Add packaging support to include flagfile with package.
- Fix a panic when osquery fails to start up.
* Make preview work when run from scratch (no orbit running)
* Make vulnerability processing less RAM hungry
* Add changes file
* Only get the cpe list once
* Remove cache
* Try killing osquery as well as orbit and clear their db dir
Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>
* added package command from orbit as fleetctl command
* update deployment docs
* add changes file
* added tests for package command, run go mod tidy & go mod verify
* validate that package files exist
* comment out msi packaging test until we can investigate github runner permission issues
* Add safe mkdirall and open
* Use secure as much as possible and merge gomodules for orbit to fleet
* Improve openfile and mkdirall to check for permissiveness instead of equality
* Don't shift
* Fix links
* Address review comments
