Commit graph

268 commits

Author SHA1 Message Date
Roberto Dip
8d9dcafbbc Merge remote-tracking branch 'origin/main' into feat-software-installers 2024-05-16 17:23:39 -03:00
Lucas Manuel Rodriguez
bb310e685d
Support Ubuntu 24.04 with Xorg (by detecting user's DISPLAY environment variable) (#18996)
#18925 (Should also fix #17660.)

Tests:
- Ubuntu 22.04.2
	- Wayland
		- Works with chrome  
		- Doesn't work with Firefox. 
	- Xorg
		- Works with Chrome.  
		- Works with Firefox. 
- Ubuntu 24.04
	- Wayland
		- Doesn't work with Chrome. 
		- Doesn't work with Firefox. 
- Xorg (when using Xorg it defaults to `DISPLAY=:1`, and with the
changes in this PR it works):
		- Works with Chrome.  
		- Works with Firefox.  
---

How to change between Wayland and Xorg:

- Set `WaylandEnable=false` in `/etc/gdm3/custom.conf` and reboot.

---

How to determine what's running:
```sh
$ loginctl
SESSION  UID USER SEAT  TTY 
      2 1000 luk  seat0 tty2
     c2 1000 luk        

$ loginctl show-session 2 -p Type
# will output
Type=wayland
or
Type=x11
```
---

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [X] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2024-05-16 08:39:30 -03:00
Roberto Dip
1ef19e4ff2 Merge remote-tracking branch 'origin/main' into feat-software-installers 2024-05-15 19:55:02 -03:00
Roberto Dip
ad94dff814
installer report and rollback fixes (#19046)
for https://github.com/fleetdm/fleet/issues/19020

- Fixes the rollback logic to get the right script for the software
being installed
- Fixes the messages displayed in the install results

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-05-15 19:18:35 -03:00
Dante Catalfamo
5fb52f6baf
Install script extension test (#19025)
Fixes a test I accidentally broke in #19012
2024-05-15 12:17:18 -04:00
Dante Catalfamo
6e8e1bd0b4
Install script extension (#19012)
Because windows won't run powershell scripts without it 👎
2024-05-15 10:31:13 -04:00
Dante Catalfamo
71c0026168
Orbit software installer flow (#18797)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).

---------

Co-authored-by: Roberto Dip <rroperzh@gmail.com>
2024-05-14 17:25:35 -03:00
Dante Catalfamo
643fc8314b
Orbit config receiver (#18518)
New interface for adding periodic jobs that rely on notifications/config
changes in Orbit.

Previously if we wanted to have recurring checks in Orbit, we would add
them into a chain of `GetConfig` calls. This call chain would be run
periodically by one of the runners registered with the cli application
framework.

The new method to register `OrbitConfigReceivers` with the
`OrbitClient`, and then register the orbit client itself with the
application framework.

Instead of having giving each fetcher an internal reference to the
previous fetcher that it must call, the receiver is registered with the
client and the new config is passed to the receiver.

This is the old `GetConfig()` interface:

```go
type OrbitConfigFetcher interface {
	GetConfig() (*fleet.OrbitConfig, error)
}
```

This is the new `OrbitConfigReceiver` interface:

```go
type OrbitConfigReceiver interface {
	Run(*OrbitConfig) error
}
```

To register a new receiver, you call the `RegisterConfigReceiver` method
on the client.

```go
orbitClient.RegisterConfigReceiver(extRunner)
```

Downsides of the old method:
- Spaghetti call chain setup
- Cascading failure, of one fails, all after it fail
- Run in series,  one long function call holds up the rest
- Anything that wants to restart orbit is added as a Runner to the
application, meaning there could be several timers calling `GetConfig`
and running the chain

Benefits of the new method:
- Clean `RegisterConfigReceiver` api, no call chaining required
- Config receivers can be added at runtime
- Isolated receivers, one failing call don't effect others
- All calls are run in parallel in goroutines, no calls can hold up the
rest
- No more need for multiple runners, using a context cancel, any
receiver can queue a call to restart orbit
- Single point to handle errors and logging for all receivers
- Panic recovery to stop orbit from crashing
- Easier to test, configs are passed in and do not require a call chain

This branch contains a little bit of code from the installer method I
was working on because I branched it off of that. (oops)

Not all code comments surrounding old `GetConfig()` methods have been
fully updated yet

Possible changes:
- Update the interface to take a context, so we can let receivers know
to exit early. I can imagine two cases for this:
  - The application is about to restart
  - We can set a timeout for how long receivers are allowed to take

Closes #12662

---------

Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2024-05-09 15:22:56 -04:00
Martin Angers
8367166278
Add uninstalled but available software installers to the "List software titles" API (#18842) 2024-05-08 12:41:57 -04:00
Victor Lyuboslavsky
a453979202
Updated macadmins and added new tables. (#18819)
#18808 
Added the new `sofa_security_release_info` and `sofa_unpatched_cves`
tables from `macadmins/osquery-extension` 1.0.1

These tables do not have detailed documentation in macadmins repo, so
not adding documentation at this point.

# Checklist for submitter
<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2024-05-08 08:57:16 -05:00
Dante Catalfamo
2c6e7c71a8
Zsh script support (#18411)
#17321
2024-04-30 14:38:56 -04:00
Dante Catalfamo
48036577eb
Interpret windows exit codes as a signed integer (#18282)
#17695

The windows exit code is a 32-bit unsigned integer, but the command
interpreter treats it like a signed integer. When a process is killed,
it returns 0xFFFFFFFF (interpreted as -1). We convert the integer to an
signed 32-bit integer to flip it to a -1 to match our expectations, and
fit in our db column.

https://en.wikipedia.org/wiki/Exit_status#Windows

FIxed on both the client and server side.
2024-04-16 10:53:50 -04:00
Victor Lyuboslavsky
3367b7e036
Added orbit_version, fleet_desktop_version, and scripts_enabled to host details. (#18123)
#17361
#17148

In GET fleet/hosts/:id response, added the following fields:
- orbit_version
  - `orbit_version == null` means this agent is not an orbit agent
- fleet_desktop_version
- `fleet_desktop_version == null` means this agent is not an orbit agent
or it is an older version which is not collecting the desktop version
- `fleet_desktop_version == ""` means this agent is an orbit agent but
does not have fleet desktop
- scripts_enabled
- `scripts_enabled == null` means this agent is not an orbit agent or it
is an older version which is not collecting scripts_enabled

In orbit_info table, added the following fields:
- desktop_version
- scripts_enabled

Updated docs for orbit_info PR:
https://github.com/fleetdm/fleet/pull/18135

Updated API docs: https://github.com/fleetdm/fleet/pull/17814

MDM lock/unlock/wipe error messages are not part of this PR. They will
be in a separate PR.

# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- [x] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2024-04-09 16:33:44 -05:00
Lucas Manuel Rodriguez
5a02624f27
Add support for pre-release on Windows orbit builder (#18102)
We need to support semantic versioning with pre-releases to comply with
semantic versioning required by goreleaser allowing us to build orbit
pre-releases.
2024-04-05 18:41:46 -03:00
Zach Wasserman
e3deceeb6d
Add parse_json, parse_jsonl, parse_xml, and parse_ini tables to fleetd (#18035)
For #17577

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux. (performed only on macOS)
2024-04-03 09:13:04 -07:00
Noah Talerman
1777b9c5d1
Update macos-install-wine.sh (#17828)
- Add note about running the script in user context
- Add redirect so that we can change the link later
2024-03-27 11:21:19 -04:00
Lucas Manuel Rodriguez
e9a464e0cf
Add exponential backoff to orbit enroll retries (#17368)
#16594

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [X] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [X] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2024-03-13 07:57:00 -03:00
github-actions[bot]
0856c44544
Update Orbit CA certs [automated] (#17556)
Automated change from [GitHub
action](https://github.com/fleetdm/fleet/actions/workflows/update-certs.yml).

Co-authored-by: zwass <zwass@users.noreply.github.com>
2024-03-13 07:55:50 -03:00
github-actions[bot]
6fbee3fc8f
Update Orbit CA certs [automated] (#15576)
Automated change from [GitHub
action](https://github.com/fleetdm/fleet/actions/workflows/update-certs.yml).

Co-authored-by: zwass <zwass@users.noreply.github.com>
2024-02-23 09:59:03 -08:00
Lucas Manuel Rodriguez
58d882b684
Reduce orbit logging when the server is down (#16968)
Orbit changes for #16423. 
Should also fix #16326 (in case of network errors).

Orbit will log the following every 5 minutes:
```
2024-02-20T14:27:40-03:00 INF network error error="Post \"https://localhost:8080/api/fleet/orbit/config\": dial tcp [::1]:8080: connect: connection refused"
```

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2024-02-21 15:36:15 -03:00
Roberto Dip
fa89dff66f
fix race in orbit test (#16589)
The scheduled test run
https://github.com/fleetdm/fleet/actions/runs/7764392848 failed with a
panic because `TestWindowsMDMEnrollmentPrevented` timed out:

```
2024-02-03T05:05:26.3041218Z === RUN   TestWindowsMDMEnrollmentPrevented
2024-02-03T05:05:26.3044251Z === RUN   TestWindowsMDMEnrollmentPrevented/{RenewEnrollmentProfile:false_RotateDiskEncryptionKey:false_NeedsMDMMigration:false_NeedsProgrammaticWindowsMDMEnrollment:true_WindowsMDMDiscoveryEndpoint:http://example.com/_NeedsProgrammaticWindowsMDMUnenrollment:false_PendingScriptExecutionIDs:[]_EnforceBitLockerEncryption:false}
2024-02-03T05:05:26.3047208Z coverage: 2.5% of statements in github.com/fleetdm/fleet/v4/...
2024-02-03T05:05:26.3047963Z panic: test timed out after 1h0m0s
2024-02-03T05:05:26.3048482Z running tests:
2024-02-03T05:05:26.3049005Z 	TestWindowsMDMEnrollmentPrevented (59m52s)
2024-02-03T05:05:26.3052172Z 	TestWindowsMDMEnrollmentPrevented/{RenewEnrollmentProfile:false_RotateDiskEncryptionKey:false_NeedsMDMMigration:false_NeedsProgrammaticWindowsMDMEnrollment:true_WindowsMDMDiscoveryEndpoint:http://example.com/_NeedsProgrammaticWindowsMDMUnenrollment:false_PendingScriptExecutionIDs:[]_EnforceBitLockerEncryption:false} (59m52s)
[...]
2024-02-03T05:05:26.3068624Z goroutine 69 [chan receive]:
2024-02-03T05:05:26.3069997Z github.com/fleetdm/fleet/v4/orbit/pkg/update.TestWindowsMDMEnrollmentPrevented.func2.1({{0xe3ada3, 0x12}, {0x0, 0x0}, {0xe37311, 0xc}})
2024-02-03T05:05:26.3072376Z 	/home/runner/work/fleet/fleet/orbit/pkg/update/notifications_test.go:295 +0x65
2024-02-03T05:05:26.3074514Z github.com/fleetdm/fleet/v4/orbit/pkg/update.(*windowsMDMEnrollmentConfigFetcher).attemptEnrollment(0xc0000f8cf0, {0x0, 0x0, 0x0, 0x1, {0xe3ada3, 0x12}, 0x0, {0x0, 0x0, ...}, ...})
```

I was able to reproduce locally 1/4th of the times, after putting the
following print statements:

```diff
                        if cfg.NeedsProgrammaticWindowsMDMEnrollment {
                                fetcher.execEnrollFn = func(args WindowsMDMEnrollmentArgs) error {
-                                       <-chProceed    // will be unblocked only when allowed
+                                       fmt.Println("fetcher.execEnrollFn A: ", apiCallCount)
+                                       <-chProceed // will be unblocked only when allowed
+                                       fmt.Println("fetcher.execEnrollFn B: ", apiCallCount)
                                        apiCallCount++ // no need for sync, single-threaded call of this func is guaranteed by the fetcher's mutex
                                        return apiErr
                                }
@@ -301,7 +303,9 @@ func TestWindowsMDMEnrollmentPrevented(t *testing.T) {
                                }
                        } else {
                                fetcher.execUnenrollFn = func(args WindowsMDMEnrollmentArgs) error {
-                                       <-chProceed    // will be unblocked only when allowed
+                                       fmt.Println("fetcher.execUnenrollFn A: ", apiCallCount)
+                                       <-chProceed // will be unblocked only when allowed
+                                       fmt.Println("fetcher.execUnenrollFn B: ", apiCallCount)
                                        apiCallCount++ // no need for sync, single-threaded call of this func is guaranteed by the fetcher's mutex
                                        return apiErr
                                }
@@ -317,23 +321,33 @@ func TestWindowsMDMEnrollmentPrevented(t *testing.T) {

                        started := make(chan struct{})
                        go func() {
+                               fmt.Println("before close started")
                                close(started)
+                               fmt.Println("aftre close started")

                                // the first call will block in enroll/unenroll func
+                               fmt.Println("before inner fetchergetconfig")
                                cfg, err := fetcher.GetConfig()
+                               fmt.Println("after inner fetchergetconfig")
                                assertResult(cfg, err)
                        }()

+                       fmt.Println("before started")
                        <-started
+                       fmt.Println("after started")
                        // this call will happen while the first call is blocked in
                        // enroll/unenrollfn, so it won't call the API (won't be able to lock the
                        // mutex). However it will still complete successfully without being
                        // blocked by the other call in progress.
+                       fmt.Println("before first fetchergetconfig")
                        cfg, err := fetcher.GetConfig()
+                       fmt.Println("before first fetchergetconfig")
                        assertResult(cfg, err)

                        // unblock the first call and wait for it to complete
+                       fmt.Println("before close chProceed 1")
                        close(chProceed)
+                       fmt.Println("after close chProceed 2")
                        time.Sleep(100 * time.Millisecond)
```

This is the output I've got every time the test hung:

```
before started
before close started
aftre close started
after started
before first fetchergetconfig
before inner fetchergetconfig
after inner fetchergetconfig
fetcher.execEnrollFn A:  0
```

And this is the output when the tests passed

```
before started
before close started
aftre close started
before inner fetchergetconfig
fetcher.execUnenrollFn A:  0
after started
before first fetchergetconfig
before first fetchergetconfig
before close chProceed 1
after close chProceed 2
fetcher.execUnenrollFn B:  0
after inner fetchergetconfig
fetcher.execUnenrollFn A:  1
fetcher.execUnenrollFn B:  1
```

Note how the deadlock occurs when `GetConfig` is called first outside of
the goroutine. I added some logic to prevent this, but I'm confident
there must be a better way to accomplish the same. cc: @mna you're the
king of concurrency, do you have any ideas?
2024-02-05 12:06:25 -03:00
Lucas Manuel Rodriguez
5360029d67
Allow custom osquery database on fleetd (#16554)
#16014

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
- [x] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2024-02-05 09:41:06 -03:00
Victor Lyuboslavsky
ed7ab1e428
Fixed macOS MSI package -- using local wine and wix (#16307)
New flow for `fleetctl --package --type=msi` on macOS using arm64
processor (M1, M2, etc.)
- wine must be installed locally. See
./orbit/tools/build/install-wine-macos.sh and
https://wiki.winehq.org/MacOS for reference.
- --local-wix-dir can be used to point to a local Wix3 installation
(using this switch requires a current Fleet EE subscription)
#15463 

PR for docs: https://github.com/fleetdm/fleet/pull/16459

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
2024-01-30 11:08:21 -06:00
Lucas Manuel Rodriguez
ea25ce4e9e
Fix Fleet Desktop bugs on Windows (#16402)
#15821

This PR is adding two improvements and fixing two Windows bugs in Fleet
Desktop:

## Improvement
- We are now capturing the stderr of Fleet Desktop. This helped me find
bug (1) below (otherwise the panic output below was hidden from us).
- To reduce complexity I'm removing the "Theme detection" routine
because we made the decision to use the colored icon for both themes...,
see here:
415d1f493b/orbit/cmd/desktop/desktop_windows.go (L21-L27)

## Bug fixes
1. Fleet Desktop icon not showing in the task bar. This was fixed by
updating to use the latest version of `fyne.io/systray`. (See
https://github.com/fyne-io/systray/issues/22#issuecomment-1173157898.)
2. Orbit now properly detects if Fleet Desktop isn't running on Windows.

Bug (1)'s panic output 
```
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x0 pc=0x72b14b]

goroutine 23 [running]:
fyne.io/systray.(*winTray).setTooltip(0x1eb5d40, {0x126923f?, 0x0?})
	/Users/luk/gopath/pkg/mod/fyne.io/systray@v1.10.0/systray_windows.go:260 +0xcb
fyne.io/systray.SetTooltip({0x126923f?, 0x125fc16?})
	/Users/luk/gopath/pkg/mod/fyne.io/systray@v1.10.0/systray_windows.go:961 +0x29
main.main.func1()
	/Users/luk/fleetdm/git/fleet/orbit/cmd/desktop/desktop.go:103 +0xba
fyne.io/systray.Register.func2()
	/Users/luk/gopath/pkg/mod/fyne.io/systray@v1.10.0/systray.go:98 +0x2f
created by fyne.io/systray.Register in goroutine 1
	/Users/luk/gopath/pkg/mod/fyne.io/systray@v1.10.0/systray.go:96 +0xb1
```

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [X] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2024-01-29 18:52:55 -03:00
Lucas Manuel Rodriguez
b65739f035
Orbit to kill pre-existing osqueryd processes during startup (#16343)
This should fix #16006.

On Windows when a process is killed by the Task Manager, it is killed
without any signaling, thus the osqueryd processes are left orphaned.
Executing osqueryd (which we do to get host information) was failing
because the lingering processes had a lock on the database file. The
solution implemented in this PR is to kill any pre-existing osqueryd
processes before running osqueryd.

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
- [X] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [X] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [X] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).

PS: I added a log of the stdout+stderr of osqueryd execution when such
command fails to execute. (This helped me find the root cause.)
```
2024-01-25T11:57:56-08:00 ERR getHostInfo via osquery output= stderr="E0125 11:57:56.744843
7860 shutdown.cpp:79] IO error: Failed to create lock file:
C:\\Program Files\\Orbit\\osquery.db/LOCK: The process cannot access the file because it is
being used by another process.\r\r\n"
```

PPS: I removed some unused exported methods in the `orbit/pkg/platform`
package.
2024-01-29 16:44:50 -03:00
Martin Angers
ca435eb244
Queued scripts feature (#16300)
This is the feature branch for the [queued
scripts](https://github.com/fleetdm/fleet/issues/15529) story.

---------

Co-authored-by: Jahziel Villasana-Espinoza <jahziel@fleetdm.com>
Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com>
Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com>
Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2024-01-29 11:37:54 -03:00
Luke Heath
877cc4d8fd
Windows removal script (#16172) 2024-01-23 10:43:31 -06:00
Victor Lyuboslavsky
e0ece3652e
Fixing tests due to known exec after write Linux issue. (#16243)
Replaces fix in https://github.com/fleetdm/fleet/pull/16241
2024-01-21 10:40:41 -06:00
Jahziel Villasana-Espinoza
d7d55f0e56
fix: don't attempt to launch fleet desktop until the user is logged into GUI (#16090)
> 📜 Related issue: https://github.com/fleetdm/fleet/issues/14698

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2024-01-17 10:00:28 -05:00
Roberto Dip
fedeab6130
attempt to decrypt the disk before performing a BitLocker encryption (#16097)
for #15711, this attempts to decrypt the disk if it was previously
encrypted and Fleet doesn't have the key.
2024-01-16 12:45:23 -03:00
Victor Lyuboslavsky
ab4c505f4b
Enroll secret in macOS keychain and Windows Credential Manager (#16068)
#13832

For macOS hosts, fleetd now stores and retrieves enroll secret from
macOS keychain.
- this feature must use the official signed and notarized version of
fleetd
- for contributors, this feature can disabled with either:
  - fleetctl package flag: --disable-keystore
  - fleetd runtime flag: --disable-keystore

This feature does not cover the MDM usecase where enroll secret is
stored in the MDM profile. This usecase will hopefully be worked on next
sprint with the MDM team.

For Windows hosts, fleetd now stores and retrieves enroll secret from
Windows Credential Manager.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2024-01-16 06:51:37 -06:00
Roberto Dip
50ffdc5d63
make sure we report the correct error during BitLocker encryption (#16096)
for #15916, explanation of the rationale in the description of
`isMisreportedDecryptionError` and in the issue comments.

I refactored the code a little bit, trying to make it easier to follow
even with the added complexity.

This also paves the road for #15711
2024-01-15 12:31:15 -03:00
Victor Lyuboslavsky
672513c5ce
Added warning/info messages when downgrading/upgrading fleetd or osquery. (#15944)
Added warning/info messages when downgrading/upgrading fleetd or
osquery. No other functional changes.
#15890 

Tested with fleetd and osquery on windows, linux, and macOS.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2024-01-10 08:27:10 -06:00
Roberto Dip
3b00b70786
fix unreleased bugs for the increased script timeout (#15897)
for https://github.com/fleetdm/fleet/issues/15196. The main problem was
that we have two timeouts:

1. The timeout used by the host to kill the script execution
2. The timeout used by the server to wait for the script results

Before the changes in https://github.com/fleetdm/fleet/pull/15779, the
server timeout was longer than the host timeout, but we inadvertently
set both values to 5 minutes, which breaks the logic we have to handle
both kinds of timeouts.
2024-01-04 10:26:13 -03:00
Martin Angers
d943fbbf8e
Extend script execution timeout (#15779)
#15196 This is the work of @ghernandez345 except for adding the
`ResponseController` thing in Go to override the server timeout for that
specific sync endpoint so that the calls don't timeout waiting for a
script response (the default HTTP server timeout was 90s for our
server).

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).

---------

Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com>
Co-authored-by: Roberto Dip <me@roperzh.com>
Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2024-01-03 16:39:53 -03:00
Lucas Manuel Rodriguez
d2015d1a36
Remotely configure fleetd update channels (#15848)
#13825

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [X] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [X] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [X] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).

---------

Co-authored-by: Victor Lyuboslavsky <victor.lyuboslavsky@gmail.com>
2024-01-02 17:59:40 -03:00
Sarah Gillespie
583d31b721
Update fleetd for macOS hosts to look for custom end user email field in Fleet MDM enrollment profile (#15761)
Issue #15057 (macOS flow)

Manual QA: 
1. Download a manual enrollment profile for a macOS device from the "My
device" page (click on "Turn on MDM" banner).
2. Open the profile in a text editor and find the following plist entry:
```xml
<key>PayloadIdentifier</key>
<string>com.fleetdm.fleet.mdm.apple.mdm</string>
```
3. Below that entry add a new entry as follows:
```xml
<key>EndUserEmail</key>
<string>user@example.com</string>
```
4. Save the file, then double-click in Finder to activate the MDM
enrollment notification.
5. Run orbit from source:
```sh
go run github.com/fleetdm/fleet/v4/orbit/cmd/orbit \  
    --dev-mode \
    --disable-updates \
    --root-dir /tmp/orbit \
    --fleet-url https://localhost:8080 \
    --insecure \
    --enroll-secret <YOUR ENROLL SECRET GOES HERE> \
    -- --verbose
```
6. Look for the following log entries in the terminal where you are
running orbit:
<img width="679" alt="Screenshot 2023-12-21 at 3 03 03 PM"
src="https://github.com/fleetdm/fleet/assets/73313222/cefc77e3-e209-49b3-a03e-abff0f7f982b">


7. In the UI, navigate to the host details page and check "Used by" in
the "About" section:
<img width="679" alt="Screenshot 2023-12-21 at 3 02 09 PM"
src="https://github.com/fleetdm/fleet/assets/73313222/c58fff3e-cee7-4a94-a53b-f30f5b4bcfa0">


# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).

---------

Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
2024-01-02 17:45:11 -03:00
Victor Lyuboslavsky
ebf1650671
device_token endpoint improvements (#15849)
Fixed badly formatted error messages in /api/fleet/orbit/device_token
endpoint and others.
In /api/fleet/orbit/device_token:
- Added token validation -- empty token not allowed
- Replaced 500 error with 409 when token conflicts with another host

#15832 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA
2023-12-28 14:20:36 -06:00
Martin Angers
2e8da551d0
Custom email device-mapping: implement the CLI (fleetd + fleetctl) changes (#15763)
Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com>
2023-12-21 11:22:59 -06:00
Lucas Manuel Rodriguez
024a20ac11
Allow enrolling fleetd using osquery's instance identifier (#15570)
#14879

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2023-12-15 15:26:32 -03:00
Victor Lyuboslavsky
b011418b71
Added initial randomization to update checker to prevent all agents updating at once. (#15583)
Added initial randomization to update checker to prevent all agents
updating at once.
#15476 

# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Sharon Katz <121527325+sharon-fdm@users.noreply.github.com>
2023-12-14 13:47:30 -06:00
Victor Lyuboslavsky
a99bf1563e
Fixing fleetd to NOT make unnecessary duplicate call to orbit/device_token endpoint. (#15543)
Fixing fleetd to NOT make unnecessary duplicate call to
orbit/device_token endpoint.
#15539 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2023-12-10 17:00:24 -06:00
Roberto Dip
8957078d75
Add backoff functionality for fleetd updates (#15489)
related to #14176, `fleetd` will now retry 3 times and wait 24 hours to
try again for an specific update.
2023-12-08 19:43:56 -03:00
Marcos Oviedo
097114e8f5
Fix windows installer orbit delete pending (#15301)
Relates to #14958 

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [X] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2023-11-23 12:29:40 -03:00
Jahziel Villasana-Espinoza
5131879292
feat: remove file if it exists before creating new one (#15186)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2023-11-22 14:04:38 -05:00
Lucas Manuel Rodriguez
4cf682d78f
Downgrade osquery-go due to panics in Shutdown and add more logging (#15017)
#15022

The issue in the package is being fixed here
https://github.com/osquery/osquery-go/pull/117
But to not block on that we will downgrade the osquery-go version we
use.

- ~[ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.~
- ~[ ] Documented any permissions changes (docs/Using
Fleet/manage-access.md)~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- ~[ ] Added/updated tests~
- [ ] Manual QA for all new/changed functionality
  - ~For Orbit and Fleet Desktop changes:~
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2023-11-13 18:29:45 -03:00
George Karr
51772873bc
Update to latest gon with notarytool until we find a better solution (#14918)
Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2023-11-03 18:56:30 -05:00
Sharon Katz
ab7717009e
Add Kolide osquery tables 2023-11-01 20:11:35 -06:00
Zach Wasserman
f6db734e81
Update default TUF root keys (#14215)
Update to the new root.json for the additional timestamp, snapshot, and
targets keys.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- [ ] Documented any permissions changes (docs/Using
Fleet/manage-access.md)
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [ ] Added/updated tests
- [ ] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2023-10-27 15:35:09 -03:00
guangwu
33858d7301
chore: remove refs to deprecated io/ioutil (#14485)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- [ ] Documented any permissions changes (docs/Using
Fleet/manage-access.md)
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [ ] Added/updated tests
- [ ] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).

Signed-off-by: guoguangwu <guoguangwu@magic-shield.com>
2023-10-27 15:28:54 -03:00