Commit graph

1093 commits

Author SHA1 Message Date
gillespi314
991c7ccadc
Add device_mapping to GET /hosts response (#5383) 2022-05-02 16:34:14 -05:00
gillespi314
1dabf52834
Add Zendesk external service integration for vulnerability automations (#5372) 2022-05-02 15:58:34 -05:00
Martin Angers
5c0031c8f1
Change jira ticket type from Bug to Task (#5426) 2022-05-02 08:56:42 -04:00
Roberto Dip
a4725518ac
filter out Google Chrome profiles without an associated email before ingesting (#5440)
To add support for #400, we're using the macadmins/osquery-extension to gather Google Chrome profiles from hosts.

Under the hood, the extension looks and parses a json file in which Chrome stores a bunch of data. Given that emails are not required to create Chrome profiles, some of the profiles stored in this file and returned by the query we're using contain empty emails.

The idea after this change is to prevent empty emails from being ingested in the first place instead of filtering them after the fact. I have also included a migration to clean the rows with empty email columns.

Fixes #4780
2022-05-02 09:55:40 -03:00
Roberto Dip
2c23cb5f24
fix mysql tests by with a regexp assertion of possible error messages (#5444)
this fixes #5377 by asserting that the error message starts with "tsl" or "x509" as either of the strings represent the same error depending on the mysql version.

we should refactor the code to avoid checking on the error messages and probably use different error types instead.
2022-04-29 09:31:58 -03:00
Martin Angers
0d0c17e0c6
Add osquery.min_software_last_opened_at_diff configuration option (#5394) 2022-04-27 09:47:09 -04:00
Lucas Manuel Rodriguez
407d8b527e
Remove unused Datastore.SaveHost and unused Modified fields (#5245) 2022-04-26 18:33:29 -03:00
Martin Angers
253baee657
Collect last_opened_at for macOS software, and return it in host details payload (#5376) 2022-04-26 14:16:59 -04:00
Michal Nicpon
a64982f96c
Remove use of JSON_ARRAYAGG (#5350)
Not supported in MySQL < 5.7.22
2022-04-25 14:43:08 -06:00
Lucas Manuel Rodriguez
45b9a382a6
fleetctl preview to use v1 setup path, to support previous versions of fleet (#5331) 2022-04-25 10:02:21 -03:00
Martin Angers
26cfa9e358
Migrate client setup api call used by fleetctl to use versionless path (#5266) 2022-04-20 16:30:43 -04:00
Martin Angers
b3fc0cd844
Unversion the /setup endpoint, version the websocket endpoint (#5104) 2022-04-20 15:57:26 -04:00
Michal Nicpon
9f981f9e49
fleetctl query improve error message (#5141) 2022-04-20 11:35:46 -06:00
Martin Angers
fd0cd153ce
Fix SSO paths to always use /v1/ instead of /latest/ (#5246) 2022-04-20 12:46:45 -04:00
Lucas Manuel Rodriguez
42f53b63b2
Remove recursion and return doRetryErr on getOrGenerateSoftwareIdDB (#5239)
* Remove recursion and return doRetryErr on getOrGenerateSoftwareIdDB

* Use GetContext and LIMIT 1
2022-04-20 11:02:35 -03:00
Lucas Manuel Rodriguez
29a902960e
Fix typos and amend PR template (#5227) 2022-04-19 18:09:55 -03:00
Lucas Manuel Rodriguez
f7048b711c
Fix race condition in tests when using global var loginRateLimit (#5197) 2022-04-19 10:35:53 -03:00
Juan Fernandez
2479d58262
Using '@' in target search causes 422 error response (#5148)
* Bug 4852:

When searching for Labels, breakdown query strings containing '@' in multiple search terms to
avoid issues with MySQL FTS.
2022-04-19 09:28:49 -04:00
gillespi314
70a61b86f1
Update os version aggregated stats for all teams (#5083) 2022-04-18 16:19:58 -05:00
Tomas Touceda
7e02bdfa29
hide jira api token from the config API (#5196)
* hide jira api token from the config API

* Update based on reviews and fix tests
2022-04-18 17:55:51 -03:00
Lucas Manuel Rodriguez
da171d3b8d
Merge pull request from GHSA-pr2g-j78h-84cr
* Fix access control issues with users

* Fix access control issues with packs

* Fix access control issues with software

* Changes suggested by Martin

* All users can access the global schedule

* Restrict access to activities

* Add explicit test for team admin escalation vuln

* All global users should be able to read all software

* Handbook editor pass - Security - GitHub Security (#5108)

* Update security.md

All edits are recorded by line:

395 replaced “open-source” with “open source”
411 replaced “open-source” with “open source”
439 added “the” before “comment”; replaced “repositories,” with “repositories”
445 deleted “being” before “located”
458 added “and” after “PR”
489 replaced “on” with “in”
493 replaced “open-source” with “open source”; Replaced “privileges,” with “privileges”

* Update security.md

line 479

* Update security.md

added (static analysis tools used to identify problems in code) to line 479

* Fix UI

* Fix UI

* revert api v1 to latest in documentation (#5149)

* revert api v1 to latest in documentation

* Update fleetctl doc page

Co-authored-by: Noah Talerman <noahtal@umich.edu>

* Add team admin team policy automation; fix e2e

* Update to company page of the handbook (#5164)

Updated "Why do we use a wireframe-first approach?" section of company.md

* removed extra data on smaller screens (#5154)

* Update for team automations; e2e

* Jira Integration: Cypress e2e tests only (#5055)

* Update company.md (#5170)

This is to update the formatting under "empathy" and to fix the spelling of "help text."
This was done as per @mikermcneil .
This is related to #https://github.com/fleetdm/fleet/pull/4941 and https://github.com/fleetdm/fleet/issues/4902

* fix update updated_at for aggregated_stats (#5112)

Update the updated_at column when using ON DUPLICATE UPDATE so that
the counts_updated_at is up to date

* basic sql formatting in code ie whitespace around operators

* Fix e2e test

* Fix tests in server/authz

Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>
Co-authored-by: Desmi-Dizney <99777687+Desmi-Dizney@users.noreply.github.com>
Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com>
Co-authored-by: Noah Talerman <noahtal@umich.edu>
Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
Co-authored-by: Martavis Parker <47053705+martavis@users.noreply.github.com>
Co-authored-by: RachelElysia <71795832+RachelElysia@users.noreply.github.com>
2022-04-18 10:27:30 -07:00
Michal Nicpon
2dbf1d7975
fix update updated_at for aggregated_stats (#5112)
Update the updated_at column when using ON DUPLICATE UPDATE so that
the counts_updated_at is up to date

* basic sql formatting in code ie whitespace around operators
2022-04-15 14:09:47 -06:00
Zach Wasserman
f28d5ab5be
Skip Docker interfaces for host primary IP (#5119)
See #4754
2022-04-13 11:04:38 -07:00
Martin Angers
a4be69d9d1
Return the new recent_vulnerability_max_age config in the REST API config payload (#5107) 2022-04-13 12:05:57 -04:00
Martin Angers
f3926c4677
Introduce forced failures for the Jira client. (#5088) 2022-04-13 09:17:02 -04:00
Martin Angers
36702ede8d
Make recent vulnerabilities max age configurable. (#5081) 2022-04-12 14:48:15 -04:00
Martin Angers
ae85d9d069
Rename jira integration password to api_token, as that is what it is (#5068) 2022-04-12 10:56:05 -04:00
Martin Angers
7187f1adac
Queue jobs for Jira integration when enabled and new vulnerabilities are found. (#4975) 2022-04-11 16:42:16 -04:00
Martin Angers
5483adc26b
Detect the NOPERM error to mean redis cluster is disabled (#5058) 2022-04-11 16:17:30 -04:00
Ricky Grassmuck
eddc2e2bda
Replace uses of ansi quotes (") in SQL statements (#4726)
To ensure the product works properly when using MySQL with `ANSI_QUOTES` mode enabled,
replace all uses of `""` for values inside SQL statements with `''`

Co-authored-by: Ricky Grassmuck <r.grassmuck@cpanel.net>
2022-04-11 11:47:50 -07:00
Lucas Manuel Rodriguez
60b7425bef
Add http basic auth to /metrics (#4974)
* Add http basic auth to /metrics

* Fixes after testing applying of a --config sample.yml

* Add unit test
2022-04-07 09:40:53 -03:00
Tomas Touceda
f2aba83a73
Extend vulnerability age to 30 (#4901) 2022-04-06 11:42:02 -03:00
Martin Angers
193843a97d
Make a test request to Jira when saving AppConfig with an enabled jira integration (#4954) 2022-04-06 07:55:25 -04:00
Zach Wasserman
bc2137e132
Add optimization to Windows software query (#4952)
This optimizes the query in particular for Domain Controllers, where
there are a number of users with no local accounts and don't need to be
searched for software.

See #4261
2022-04-05 10:56:47 -07:00
Martin Angers
d890970844
Implement the Jira API call (#4938) 2022-04-05 12:58:34 -04:00
Martin Angers
f28dc10a51
Merge jira-integration branch to main (#4949) 2022-04-05 12:56:15 -04:00
Martin Angers
90b15071a4
Introduce API version 2022-04, deprecate use of /global in paths (#4731) 2022-04-05 11:35:53 -04:00
Lucas Manuel Rodriguez
57816592ba
Add read replica testing helpers and fix non-sso login bug (#4908)
not set on the INSERT.
- OUT: Only sets the ID on the passed session and returns it. (`CreatedAt`, `AccessedAt`, are not set.)

New version:

```go
func (ds *Datastore) NewSession(ctx context.Context, userID uint, sessionKey string) (*fleet.Session, error) {
	sqlStatement := `
		INSERT INTO sessions (
			user_id,
			` + "`key`" + `
		)
		VALUES(?,?)
	`
	result, err := ds.writer.ExecContext(ctx, sqlStatement, userID, sessionKey)
	if err != nil {
		return nil, ctxerr.Wrap(ctx, err, "inserting session")
	}

	id, _ := result.LastInsertId() // cannot fail with the mysql driver
	return ds.sessionByID(ctx, ds.writer, uint(id))
}
```

- IN: Define arguments that are truly used when creating a session.
- OUT: Load and return the fleet.Session struct with all values set (using the `ds.writer` to support read replicas correctly).

PS: The new `NewSession` version mimics what we already do with other entities, like policies (`Datastore.NewGlobalPolicy`).
2022-04-04 16:52:05 -07:00
Martin Angers
71f2690ada
Add Jira integrations config support (#4863) 2022-03-30 09:10:02 -04:00
Michal Nicpon
779623934c
Fix update invite email conflict (#4751) 2022-03-28 09:38:57 -06:00
Michal Nicpon
a6902cc083
Add os versions endpoint (#4749) 2022-03-28 09:15:45 -06:00
Michal Nicpon
6c2199f2b0
fix failing policies nil pointer dereference (#4802) 2022-03-24 15:56:56 -06:00
Martin Angers
c4946335ff
Add migration to cleanup host-related tables (#4778)
#4448
2022-03-23 15:22:10 -07:00
Lucas Manuel Rodriguez
378b1dd34f
Upgrade osquery-go (#4728)
* Upgrade and replace kolide/osquery-go with osquery/osquery-go

* Upgrade macadmins/osquery-extension to v0.0.7

* Upgrade kolide/launcher to latest

* go mod tidy
2022-03-23 12:15:37 -03:00
Lucas Manuel Rodriguez
8f9ed3ef7b
Fix Datastore.DeleteHosts (#4745) 2022-03-23 12:15:05 -03:00
Michal Nicpon
7b671ac2a3
Add team failing policies webhook (#4633)
* add config to teams
* update api docs
* update tests
2022-03-21 13:16:47 -06:00
Lucas Manuel Rodriguez
ecdfd627b6
Fleet Desktop MVP (#4530)
* WIP

* WIP2

* Fix orbit and fleetctl tests

* Amend macos-app default

* Add some fixes

* Use fleetctl updates roots command

* Add more fixes to Updater

* Fixes to app publishing and downloading

* Add more changes to support fleetctl cross generation

* Amend comment

* Add pkg generation to ease testing

* Make more fixes

* Add changes entry

* Add legacy targets (until our TUF system exposes the new app)

* Fix fleetctl preview

* Fix bool flag

* Fix orbit logic for disabled-updates and dev-mode

* Fix TestPreview

* Remove constant and fix zip-slip attack (codeql)

* Return unknown error

* Fix updater's checkExec

* Add support for executable signing in init_tuf.sh

* Try only signing orbit

* Fix init_tuf.sh targets, macos-app only for osqueryd

* Specify GOARCH to support M1s

* Add workflow to generate osqueryd.app.tar.gz

* Use 5.2.2 on init_tuf.sh

* Add unit test for tar.gz target

* Use artifacts instead of releases

* Remove copy paste residue

* Fleet Desktop Packaging WIP

* Ignore gosec warning

* Trigger on PR too

* Install Go in workflow

* Pass url parameter to desktop app

* Fix fleetctl package

* Final set of changes for v1 of Fleet Desktop

* Add changes

* PR fixes

* Fix CI build

* add larger menu bar icon

* Add transparency item

* Delete host_device_auth entry on host deletion

* Add SetTargetChannel

* Update white logo and add desktop to update runner

* Add fleet-desktop monitoring to orbit

* Define fleet-desktop app exec name

* Fix update runner creation

* Add API test before enabling the My device menu item

Co-authored-by: Zach Wasserman <zach@fleetdm.com>
2022-03-21 14:53:53 -03:00
Benjamin Edwards
74bb559645
Add public ip to hosts & derive geolocation when rendering host (#4652)
* geoip wip
* return nil if ip is empty string or if ParseIP returns nil
* add ui component to render geolocation if available, address PR feedback
* render public ip if available
* add changes file, document geoip in deployment guide
* update rest-api docs
2022-03-21 12:29:52 -04:00
Zach Wasserman
efbc2b92bb
Fix race condition in updates test (#4661)
Copy the DefaultOptions in order to prevent a data race on the Targets
map. This race should only have effected testing.

Race detector output:

```
WARNING: DATA RACE
Read at 0x00c0000908d0 by goroutine 15:
  runtime.mapaccess1_faststr()
      /opt/hostedtoolcache/go/1.18.0/x64/src/runtime/map_faststr.go:13 +0x0
  github.com/fleetdm/fleet/v4/orbit/pkg/update.TestMakeRepoPath.func1()
      /home/runner/work/fleet/fleet/orbit/pkg/update/update_test.go:58 +0xb6
  testing.tRunner()
      /opt/hostedtoolcache/go/1.18.0/x64/src/testing/testing.go:1439 +0x213
  testing.(*T).Run.func1()
      /opt/hostedtoolcache/go/1.18.0/x64/src/testing/testing.go:1486 +0x47

Previous write at 0x00c0000908d0 by goroutine 12:
  runtime.mapassign_faststr()
      /opt/hostedtoolcache/go/1.18.0/x64/src/runtime/map_faststr.go:203 +0x0
  github.com/fleetdm/fleet/v4/orbit/pkg/update.TestMakeRepoPath.func1()
      /home/runner/work/fleet/fleet/orbit/pkg/update/update_test.go:62 +0x1cb
  testing.tRunner()
      /opt/hostedtoolcache/go/1.18.0/x64/src/testing/testing.go:1439 +0x213
  testing.(*T).Run.func1()
      /opt/hostedtoolcache/go/1.18.0/x64/src/testing/testing.go:1486 +0x47

Goroutine 15 (running) created at:
  testing.(*T).Run()
      /opt/hostedtoolcache/go/1.18.0/x64/src/testing/testing.go:1486 +0x724
  github.com/fleetdm/fleet/v4/orbit/pkg/update.TestMakeRepoPath()
      /home/runner/work/fleet/fleet/orbit/pkg/update/update_test.go:53 +0x1a4
  testing.tRunner()
      /opt/hostedtoolcache/go/1.18.0/x64/src/testing/testing.go:1439 +0x213
  testing.(*T).Run.func1()
      /opt/hostedtoolcache/go/1.18.0/x64/src/testing/testing.go:1486 +0x47

Goroutine 12 (running) created at:
  testing.(*T).Run()
      /opt/hostedtoolcache/go/1.18.0/x64/src/testing/testing.go:1486 +0x724
  github.com/fleetdm/fleet/v4/orbit/pkg/update.TestMakeRepoPath()
      /home/runner/work/fleet/fleet/orbit/pkg/update/update_test.go:53 +0x1a4
  testing.tRunner()
      /opt/hostedtoolcache/go/1.18.0/x64/src/testing/testing.go:1439 +0x213
  testing.(*T).Run.func1()
      /opt/hostedtoolcache/go/1.18.0/x64/src/testing/testing.go:1486
      +0x47
 ```
2022-03-18 09:30:45 -07:00
Martin Angers
b2845cd65f
Refactor authz skip for device auth, add org_logo_url for frontend (#4619) 2022-03-16 10:15:25 -04:00