Commit graph

750 commits

Author SHA1 Message Date
Tomas Touceda
d5a5f014ef
Add test to check that two hosts can store stats concurrently (#1929) 2021-09-07 10:39:17 -03:00
Tomas Touceda
cb687f6e2d
Increase timeout to check for smpt availability (#1942) 2021-09-06 14:29:52 -03:00
gillespi314
4a307f1274
Update email template image links (#1938)
* Update email image links; move pngs to fleetdm.com
2021-09-03 17:00:30 -05:00
Tomas Touceda
6d2ae02efd
Rename core->free and basic->premium (#1870)
* Rename core->free and basic->premium

* Fix lint js

* Comment out portion of test that seems to timeout

* Rename tier  to premium if basic is still loaded
2021-09-03 13:05:23 -03:00
Tomas Touceda
db2e04a7c3
Update new sched query stats if there's already some (#1918)
* Update new sched query stats if there's already some

* IGNORE if the sched query is not present

* Make tests less flaky
2021-09-02 17:39:08 -03:00
Tomas Touceda
681e7f18a0
Add default for agent options (#1917) 2021-09-02 15:59:48 -03:00
Tomas Touceda
15ce06263d
Dont fail config if agent opts is nil (#1898)
* Handle agentopts nil better

* Add changes file
2021-09-01 19:17:41 -03:00
Martin Angers
9a0871a2f1
Address issues related to Redis Cluster support (#1885)
Closes #1847 .
2021-09-01 16:32:57 -04:00
Martin Angers
3755a58070
Add support for MySQL read replicas (#1786)
Adds configuration options to use a read-only MySQL replica,
and uses it instead of the primary for reads.
2021-09-01 15:50:52 -04:00
Tomas Touceda
79b5330a43
Do host user inserts one by one to not lock the whole database (#1884) 2021-09-01 11:39:23 -03:00
Benjamin Edwards
047dc5ac62
fix get hosts command to properly output json/yaml (#1774)
* fix get hosts command to properly output json/yaml based on command line flag

* add changes file

* added tests for get hosts when specifiying host

* added additional hosts to be returned in test cases

* go fmt
2021-08-31 11:37:03 -04:00
Tomas Touceda
22267d8e0a
Group concat limit (#1860)
* wiup

* Decouple software select in two

* Add changes file

* Add missing error check
2021-08-30 16:07:24 -03:00
Tomas Touceda
2a17f9fe56
Remove hardcoded limits to the labels (#1857)
* Remove hardcoded limits to the labels

* Add changes file

* Remove test that doens't apply anymore
2021-08-30 14:56:15 -03:00
Tomas Touceda
8b908f6506
Issue 1599 offline webhook (#1777)
* wip

* Add tests and finish implementation

* Add proper default for periodicity, changes file, and documentation

* Fix tests and add defaults also to new installs

* EnableHostUsers should be true if undefined as well

* In some cases, periodicity can be zero because of the migrations

* Apply defaults when migrating appconfig

* Fix lint

* lint

* Address review comments
2021-08-27 11:15:36 -03:00
Tomas Touceda
2fd725c81b
Export fields so the select will work with the struct (#1819) 2021-08-26 11:56:05 -03:00
Tomas Touceda
d6d53acda5
Issue 1600 fleetctl license expiration (#1800)
* Show banner in fleet and fleetctl if license expired

* Ignore if license is nil or tier is core

* Address review comments
2021-08-26 10:28:53 -03:00
Zach Wasserman
5993018214
Fix live query timeout handling (#1759)
Correct cleanup of the connections was verified with the pprof goroutine
tool.

Fixes #1751
2021-08-25 15:28:04 -07:00
Tomas Touceda
1f5094d97e
Make adding routes a bit simpler (#1771)
* Make adding routes a bit simpler

* Remove unused handle

* Lint

* More lint
2021-08-25 10:08:14 -03:00
Tomas Touceda
5fb5995b83
Remove unneeded interfaces (#1779)
* Remove unneeded interfaces

* Remove unused code
2021-08-24 18:49:56 -03:00
Tomas Touceda
a8642493ad
Add global policies (#1750)
* Add global policies

* Update documentation and add extra parameter to config

* Fix failing tests

* Store historic policy records

* Address review comments

And also remove other inmem references I saw by chance

* Add documentation for get by id request

* Add parameter doc

* Move schema generation to a cmd instead of a test

Otherwise it messes up running all tests sometimes depending on how parallel it does

* Remove brain dump for another task

* Make migration tests a separate beast

* Make schema generation idempotent and move dbutils cmd to tools

* Allow all filters and add counts to Policy

* Add test for Policy
2021-08-24 17:24:52 -03:00
Martin Angers
e61cb7e0db
Configure golangci-lint for the whole repository
Add a relatively minimal set of linters that raise safe and 
mostly un-opinionated issues with the code. It runs 
automatically on CI via a github action.
2021-08-24 13:35:03 -04:00
Martin Angers
a3714d2ed9
Add fleetctl debug connection command (#1706)
Adds the `fleetctl debug connection` command to investigate
connection issues to the fleet server.

Closes #1579 .
2021-08-24 08:50:03 -04:00
Tomas Touceda
612fa05dd3
Log errors when osquery endpoints have issues (#1764) 2021-08-23 19:40:00 -03:00
Martin Angers
bfdc014cc1
Fix sql-related missing Close/Err calls (#1769) 2021-08-23 16:33:41 -04:00
Tomas Touceda
a1ceebf267
Make team schedule names more user friendly and hide them from host pack stats (#1737)
* Make team schedule names more user friendly and hide them from host pack stats

* Delete test for a different bug and add migration

* Update name template
2021-08-23 14:23:47 -03:00
Tomas Touceda
66d864cc84
Remove extra logIPs from host auth mw (#1762)
* Remove extra logIPs from host auth mw

* Fix test
2021-08-23 14:23:27 -03:00
Tomas Touceda
26a08096fb
Use GROUP_CONCAT instead of JSON_ARRAYAGG (#1752)
* Use GROUP_CONCAT instead of JSON_ARRAYAGG

* Update server/datastore/mysql/software.go

Co-authored-by: Martin Angers <martin.n.angers@gmail.com>

Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
2021-08-23 14:12:30 -03:00
Tomas Touceda
b7a65e920a
Remove orphan host_software and software before adding fk constriants (#1760) 2021-08-23 13:42:43 -03:00
Tomas Touceda
ad18728ac3
Issue 1601 disk space (#1735)
* wip

* Add tests

* Add migration

* Fix tests and set platform for windows correctly
2021-08-20 14:57:37 -03:00
Tomas Touceda
605970c441
Add cascade deletes for host software (#1739)
* Add cascade deletes for host software

* Add changes file

* The drop doesn't work on certain mysql

* Fix error message
2021-08-20 14:29:00 -03:00
Tomas Touceda
c6c63ab12a
Refactor app config (POC, for now) (#1685) 2021-08-20 12:27:41 -03:00
Tomas Touceda
33791dbee8
Remove fk in scheduled_query_stats table and clean up orphan rows for it (#1720)
* Remove fk in scheduled_query_stats table and clean up orphan rows for it

* Improve test and fix bug with the cleanup
2021-08-18 18:30:48 -03:00
Tomas Touceda
70f728fe84
Make vulnerability processing more configurable (#1718)
* Make vulnerability processing more configurable

* Simplify leader selection
2021-08-18 17:25:14 -03:00
Tomas Touceda
402303bc5d
Add All Linux label (#1582)
* Add All Linux label

* Change name to Linux instead of All Linux to see if e2e likes it better

* Revert "Change name to Linux instead of All Linux to see if e2e likes it better"

This reverts commit 26b79f214e.

* Fix all linux label insert
2021-08-18 15:55:48 -03:00
Tomas Touceda
53e25cf59d
Use tx if within a tx (#1668) 2021-08-13 14:59:31 -03:00
Tomas Touceda
512f5defce
Detect centos as a host platform more accurately (#1664)
* Detect centos as a host platform more accurately

* Add test for centos host labels
2021-08-13 13:22:09 -03:00
Tomas Touceda
5b8a607a81
Add support for MariaDB 10.5.4 and later (#1658)
* Add support for MariaDB 10.5.4 and later

* Fix tests

* Add changes file

* Use require.JSONEq instead of our own version of it
2021-08-13 11:16:19 -03:00
Tomas Touceda
96c0244c04
Fix redis locking receive (#1655)
* Receive redis data with timeout to not hold the connection

* Address review comments
2021-08-13 11:16:11 -03:00
Tomas Touceda
e8def4af00
Allow to refetch hosts if you are an observer/maintainer for the team for that host (#1649) 2021-08-11 19:01:37 -03:00
Zach Wasserman
544652e833
Clarify label responses and API docs (#1633)
- Include platform and description even when empty in API responses.
- Update docs to indicate immutability.
2021-08-11 14:57:05 -07:00
Tomas Touceda
3d8a766ca1
Make receive calls to redis conn thread safe (#1641)
* Make receive calls to redis conn thread safe

Also removes REDIS_TEST env var. Redis is lightweight and fast, no need
to skip these tests.

* No need to increase the wait
2021-08-11 17:34:35 -03:00
Tomas Touceda
ae2bd137b1
Issue 1632 software inventory config (#1636)
* Add config option for software inventory

* Add documentation for the new config
2021-08-11 15:57:53 -03:00
Tomas Touceda
11dff7ca92
Issue 1588 allow disabling host users (#1611)
* Allow users to disable host_users

* Add missing files

* Address review comments

* Fix tests

* Check additional queries for nil, not the whole hostsettings

* Finally fix tests
2021-08-11 14:56:11 -03:00
Tomas Touceda
c781058617
Fix build, add missing tests for cpe translations (#1631)
* Fix build, add missing tests for cpe translations

Also dont fail alltogether if there's one issue translating CPEs, log it and continue

* Make it once every hour again

* Use MATCH but escape strings
2021-08-11 14:52:09 -03:00
Zach Wasserman
740ba7bc25
Remove duplicate registration of results handler (#1605) 2021-08-11 09:39:47 -07:00
Benjamin Edwards
8bdad712d8
add team_id filter to fleetctl & api (#1596)
* add team_id filter to fleetctl via get hosts --team flag & api via api/v1/fleet/hosts and api/v1/fleet/labels/id/hosts

* update tests & add changes file
2021-08-11 10:40:56 -04:00
Tomas Touceda
989e638cc0
Make creating dirs and files more secure by checking permissions (#1566)
* Add safe mkdirall and open

* Use secure as much as possible and merge gomodules for orbit to fleet

* Improve openfile and mkdirall to check for permissiveness instead of equality

* Don't shift

* Fix links

* Address review comments
2021-08-11 11:02:22 -03:00
Tomas Touceda
d51493274d
Issue 1570 stats perf (#1598)
* Dont delete pack stats before inserting new ones to prevent deadlocks

* Remove fk for scheduled_query_stats

* Remove fk removal

* Fix tests

* Remove unneeded comment
2021-08-10 18:17:06 -03:00
Benjamin Edwards
29611d8377
modify packs api to filter non-empty pack_type from response (#1558)
* modify packs api to filter non-empty pack_type from response
* change list packs store method to allow filtering for "system-level" packs, by default the api filters these packs from being returned
* add changes file
* don't allow modifications to global or team packs via apply spec
* refactor to use PackListOptions
2021-08-10 09:43:27 -04:00
Zach Wasserman
a29844120b
Fix live query for observers (#1603)
Set observer can run when performing authz check for live query results.

Final backend fix for #1515
2021-08-09 19:30:17 -07:00