Commit graph

1103 commits

Author SHA1 Message Date
Martin Angers
e6b90ca8b9
Support per-task configuration for async host processing configuration (#5700) 2022-05-16 09:44:50 -04:00
Lucas Manuel Rodriguez
114b678e26
test: Add retry to the correct part that has sync enabled (#5688) 2022-05-11 10:11:51 -03:00
Martin Angers
f05d2be767
Produce hosts' CSV report based on requested columns (#5656) 2022-05-10 14:25:53 -04:00
gillespi314
d172128183
Add label information to host summary response (#5573) 2022-05-10 10:32:55 -05:00
Martin Angers
1fa7bb7a19
Support async saving of hosts' last seen time (#5640) 2022-05-10 11:29:17 -04:00
Martin Angers
421a0ec635
Fix some config mismatches in osquery logging (#5488) 2022-05-10 11:27:32 -04:00
Michal Nicpon
a005b7e7b7
fix schema dump (#5562) 2022-05-10 09:03:04 -06:00
Lucas Manuel Rodriguez
fda79a8770
Run network test serially to prevent timeouts on Github CI (#5557)
* Run network test serially to prevent timeouts on Github CI

* Revert lint changes

* Add simple file lock

* Revert test change

* Clarify error check
2022-05-10 11:52:33 -03:00
Roberto Dip
cba78bdcd1
don't clear errors automatically when reading them from Redis (#5623)
this change prevents errors from being automatically cleared once they are read. A new flag `-flush` is introduced to flush errors on read if necessary.
2022-05-10 11:44:43 -03:00
gillespi314
998471f716
Omit empty device mapping from host responses where optional device_mapping param is missing (#5541) 2022-05-05 13:13:53 -05:00
gillespi314
991c7ccadc
Add device_mapping to GET /hosts response (#5383) 2022-05-02 16:34:14 -05:00
gillespi314
1dabf52834
Add Zendesk external service integration for vulnerability automations (#5372) 2022-05-02 15:58:34 -05:00
Martin Angers
5c0031c8f1
Change jira ticket type from Bug to Task (#5426) 2022-05-02 08:56:42 -04:00
Roberto Dip
a4725518ac
filter out Google Chrome profiles without an associated email before ingesting (#5440)
To add support for #400, we're using the macadmins/osquery-extension to gather Google Chrome profiles from hosts.

Under the hood, the extension looks and parses a json file in which Chrome stores a bunch of data. Given that emails are not required to create Chrome profiles, some of the profiles stored in this file and returned by the query we're using contain empty emails.

The idea after this change is to prevent empty emails from being ingested in the first place instead of filtering them after the fact. I have also included a migration to clean the rows with empty email columns.

Fixes #4780
2022-05-02 09:55:40 -03:00
Roberto Dip
2c23cb5f24
fix mysql tests by with a regexp assertion of possible error messages (#5444)
this fixes #5377 by asserting that the error message starts with "tsl" or "x509" as either of the strings represent the same error depending on the mysql version.

we should refactor the code to avoid checking on the error messages and probably use different error types instead.
2022-04-29 09:31:58 -03:00
Martin Angers
0d0c17e0c6
Add osquery.min_software_last_opened_at_diff configuration option (#5394) 2022-04-27 09:47:09 -04:00
Lucas Manuel Rodriguez
407d8b527e
Remove unused Datastore.SaveHost and unused Modified fields (#5245) 2022-04-26 18:33:29 -03:00
Martin Angers
253baee657
Collect last_opened_at for macOS software, and return it in host details payload (#5376) 2022-04-26 14:16:59 -04:00
Michal Nicpon
a64982f96c
Remove use of JSON_ARRAYAGG (#5350)
Not supported in MySQL < 5.7.22
2022-04-25 14:43:08 -06:00
Lucas Manuel Rodriguez
45b9a382a6
fleetctl preview to use v1 setup path, to support previous versions of fleet (#5331) 2022-04-25 10:02:21 -03:00
Martin Angers
26cfa9e358
Migrate client setup api call used by fleetctl to use versionless path (#5266) 2022-04-20 16:30:43 -04:00
Martin Angers
b3fc0cd844
Unversion the /setup endpoint, version the websocket endpoint (#5104) 2022-04-20 15:57:26 -04:00
Michal Nicpon
9f981f9e49
fleetctl query improve error message (#5141) 2022-04-20 11:35:46 -06:00
Martin Angers
fd0cd153ce
Fix SSO paths to always use /v1/ instead of /latest/ (#5246) 2022-04-20 12:46:45 -04:00
Lucas Manuel Rodriguez
42f53b63b2
Remove recursion and return doRetryErr on getOrGenerateSoftwareIdDB (#5239)
* Remove recursion and return doRetryErr on getOrGenerateSoftwareIdDB

* Use GetContext and LIMIT 1
2022-04-20 11:02:35 -03:00
Lucas Manuel Rodriguez
29a902960e
Fix typos and amend PR template (#5227) 2022-04-19 18:09:55 -03:00
Lucas Manuel Rodriguez
f7048b711c
Fix race condition in tests when using global var loginRateLimit (#5197) 2022-04-19 10:35:53 -03:00
Juan Fernandez
2479d58262
Using '@' in target search causes 422 error response (#5148)
* Bug 4852:

When searching for Labels, breakdown query strings containing '@' in multiple search terms to
avoid issues with MySQL FTS.
2022-04-19 09:28:49 -04:00
gillespi314
70a61b86f1
Update os version aggregated stats for all teams (#5083) 2022-04-18 16:19:58 -05:00
Tomas Touceda
7e02bdfa29
hide jira api token from the config API (#5196)
* hide jira api token from the config API

* Update based on reviews and fix tests
2022-04-18 17:55:51 -03:00
Lucas Manuel Rodriguez
da171d3b8d
Merge pull request from GHSA-pr2g-j78h-84cr
* Fix access control issues with users

* Fix access control issues with packs

* Fix access control issues with software

* Changes suggested by Martin

* All users can access the global schedule

* Restrict access to activities

* Add explicit test for team admin escalation vuln

* All global users should be able to read all software

* Handbook editor pass - Security - GitHub Security (#5108)

* Update security.md

All edits are recorded by line:

395 replaced “open-source” with “open source”
411 replaced “open-source” with “open source”
439 added “the” before “comment”; replaced “repositories,” with “repositories”
445 deleted “being” before “located”
458 added “and” after “PR”
489 replaced “on” with “in”
493 replaced “open-source” with “open source”; Replaced “privileges,” with “privileges”

* Update security.md

line 479

* Update security.md

added (static analysis tools used to identify problems in code) to line 479

* Fix UI

* Fix UI

* revert api v1 to latest in documentation (#5149)

* revert api v1 to latest in documentation

* Update fleetctl doc page

Co-authored-by: Noah Talerman <noahtal@umich.edu>

* Add team admin team policy automation; fix e2e

* Update to company page of the handbook (#5164)

Updated "Why do we use a wireframe-first approach?" section of company.md

* removed extra data on smaller screens (#5154)

* Update for team automations; e2e

* Jira Integration: Cypress e2e tests only (#5055)

* Update company.md (#5170)

This is to update the formatting under "empathy" and to fix the spelling of "help text."
This was done as per @mikermcneil .
This is related to #https://github.com/fleetdm/fleet/pull/4941 and https://github.com/fleetdm/fleet/issues/4902

* fix update updated_at for aggregated_stats (#5112)

Update the updated_at column when using ON DUPLICATE UPDATE so that
the counts_updated_at is up to date

* basic sql formatting in code ie whitespace around operators

* Fix e2e test

* Fix tests in server/authz

Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>
Co-authored-by: Desmi-Dizney <99777687+Desmi-Dizney@users.noreply.github.com>
Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com>
Co-authored-by: Noah Talerman <noahtal@umich.edu>
Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
Co-authored-by: Martavis Parker <47053705+martavis@users.noreply.github.com>
Co-authored-by: RachelElysia <71795832+RachelElysia@users.noreply.github.com>
2022-04-18 10:27:30 -07:00
Michal Nicpon
2dbf1d7975
fix update updated_at for aggregated_stats (#5112)
Update the updated_at column when using ON DUPLICATE UPDATE so that
the counts_updated_at is up to date

* basic sql formatting in code ie whitespace around operators
2022-04-15 14:09:47 -06:00
Zach Wasserman
f28d5ab5be
Skip Docker interfaces for host primary IP (#5119)
See #4754
2022-04-13 11:04:38 -07:00
Martin Angers
a4be69d9d1
Return the new recent_vulnerability_max_age config in the REST API config payload (#5107) 2022-04-13 12:05:57 -04:00
Martin Angers
f3926c4677
Introduce forced failures for the Jira client. (#5088) 2022-04-13 09:17:02 -04:00
Martin Angers
36702ede8d
Make recent vulnerabilities max age configurable. (#5081) 2022-04-12 14:48:15 -04:00
Martin Angers
ae85d9d069
Rename jira integration password to api_token, as that is what it is (#5068) 2022-04-12 10:56:05 -04:00
Martin Angers
7187f1adac
Queue jobs for Jira integration when enabled and new vulnerabilities are found. (#4975) 2022-04-11 16:42:16 -04:00
Martin Angers
5483adc26b
Detect the NOPERM error to mean redis cluster is disabled (#5058) 2022-04-11 16:17:30 -04:00
Ricky Grassmuck
eddc2e2bda
Replace uses of ansi quotes (") in SQL statements (#4726)
To ensure the product works properly when using MySQL with `ANSI_QUOTES` mode enabled,
replace all uses of `""` for values inside SQL statements with `''`

Co-authored-by: Ricky Grassmuck <r.grassmuck@cpanel.net>
2022-04-11 11:47:50 -07:00
Lucas Manuel Rodriguez
60b7425bef
Add http basic auth to /metrics (#4974)
* Add http basic auth to /metrics

* Fixes after testing applying of a --config sample.yml

* Add unit test
2022-04-07 09:40:53 -03:00
Tomas Touceda
f2aba83a73
Extend vulnerability age to 30 (#4901) 2022-04-06 11:42:02 -03:00
Martin Angers
193843a97d
Make a test request to Jira when saving AppConfig with an enabled jira integration (#4954) 2022-04-06 07:55:25 -04:00
Zach Wasserman
bc2137e132
Add optimization to Windows software query (#4952)
This optimizes the query in particular for Domain Controllers, where
there are a number of users with no local accounts and don't need to be
searched for software.

See #4261
2022-04-05 10:56:47 -07:00
Martin Angers
d890970844
Implement the Jira API call (#4938) 2022-04-05 12:58:34 -04:00
Martin Angers
f28dc10a51
Merge jira-integration branch to main (#4949) 2022-04-05 12:56:15 -04:00
Martin Angers
90b15071a4
Introduce API version 2022-04, deprecate use of /global in paths (#4731) 2022-04-05 11:35:53 -04:00
Lucas Manuel Rodriguez
57816592ba
Add read replica testing helpers and fix non-sso login bug (#4908)
not set on the INSERT.
- OUT: Only sets the ID on the passed session and returns it. (`CreatedAt`, `AccessedAt`, are not set.)

New version:

```go
func (ds *Datastore) NewSession(ctx context.Context, userID uint, sessionKey string) (*fleet.Session, error) {
	sqlStatement := `
		INSERT INTO sessions (
			user_id,
			` + "`key`" + `
		)
		VALUES(?,?)
	`
	result, err := ds.writer.ExecContext(ctx, sqlStatement, userID, sessionKey)
	if err != nil {
		return nil, ctxerr.Wrap(ctx, err, "inserting session")
	}

	id, _ := result.LastInsertId() // cannot fail with the mysql driver
	return ds.sessionByID(ctx, ds.writer, uint(id))
}
```

- IN: Define arguments that are truly used when creating a session.
- OUT: Load and return the fleet.Session struct with all values set (using the `ds.writer` to support read replicas correctly).

PS: The new `NewSession` version mimics what we already do with other entities, like policies (`Datastore.NewGlobalPolicy`).
2022-04-04 16:52:05 -07:00
Martin Angers
71f2690ada
Add Jira integrations config support (#4863) 2022-03-30 09:10:02 -04:00
Michal Nicpon
779623934c
Fix update invite email conflict (#4751) 2022-03-28 09:38:57 -06:00