Commit graph

171 commits

Author SHA1 Message Date
Zachary Wasserman
cb23dab4be Clarify label types (#820) 2017-01-09 12:02:21 -08:00
Mike Arpaia
a047ef2211 Quick contexts additions (#739)
* Defining a concrete type for session tokens

* More rightish vc.IsLoggedIn()

* using type conversion instead of a method call

* include sessions in test viewer contexts
2017-01-09 08:10:02 -07:00
Zachary Wasserman
77e4f3d936 Refactor require password reset into separate endpoint (#725)
- Remove require password reset from ModifyUser and
  RequestPasswordReset methods, and UserPayload struct
- Add new RequirePasswordReset method
- Refactor JS for new separate method
2017-01-06 14:38:39 -08:00
Zachary Wasserman
5371647d2c Add builtin labels via migration (#728)
Changing from the existing method of adding built in labels at server startup.
This new method should be friendlier to long term changes, and falls in line
with the new pattern established for osquery options.

Fixes #702
2017-01-06 12:30:43 -08:00
Mike Arpaia
23c86f6c12 Ensure author name and packs are set in NewQuery response (#774) 2017-01-05 18:48:56 -07:00
Zachary Wasserman
35806f1442 Separate tables from data migrations (#745)
This PR separates the table migrations from the data population migrations. Table migrations run before data migrations.

Now, we have the ability to create the database tables without populating them with data. This can be useful for running "unit" tests against a MySQL store that doesn't have any pre-populated data. When performing real migrations, or for more "integration" style testing, the data migrations can also be executed.

Note there are some special cases that must be observed with these migrations, and the README is updated to reflect those.
2017-01-05 09:27:56 -08:00
Mike Arpaia
704ddd424b Host summary endpoint (#742)
* Initial scaffolding of the host summary endpoint

* inmem datastore implementation of GenerateHostStatusStatistics

* HostSummary docstring

* changing the url of the host summary endpoint

* datastore tests for GenerateHostStatusStatistics

* MySQL datastore implementation of GenerateHostStatusStatistics

* <= and >= to catch exact time edge case

* removing clock interface method

* lowercase error wraps

* removin superfluous whitespace

* use updated_at

* adding a seen_at column to the hosts table

* moving the update of seen_time to the caller

* using db.Get instead of db.Select
2017-01-04 14:16:17 -07:00
Victor Vrantchan
046f75295e consolidate delete operations in mysql store (#746)
Adds a helper method which soft deletes entities from the database.
2017-01-04 13:18:21 -05:00
Mike Arpaia
0122f6cb0a Add host_ids and label_ids fields to the packs API (#737)
This PR adds the `host_ids` and `label_ids` field to the packs HTTP API so that one can operate on the hosts/labels which a pack is scheduled to be executed on. This replaces (and deletes) the `/api/v1/kolide/packs/123/labels/456` API in favor of `PATCH /api/v1/packs/123` and specifying the `label_ids` field. This also allows for bulk operations.

Consider the following API examples:

## Creating a pack with a known set of hosts and labels

The key addition is the `host_ids` and `label_ids` field in both the request and the response.

### Request

```
POST /api/v1/kolide/packs
```

```json
{
	"name": "My new pack",
	"description": "The newest of the packs",
	"host_ids": [1, 2, 3],
	"label_ids": [1, 3, 5]
}
```

### Response

```json
{
	"pack": {
		"id": 123,
		"name": "My new pack",
		"description": "The newest of the packs",
		"platform": "",
		"created_by": 1,
		"disabled": false,
		"query_count": 0,
		"total_hosts_count": 5,
		"host_ids": [1, 2, 3],
		"label_ids": [1, 3, 5]
	}
}
```

## Modifying the hosts and/or labels that a pack is scheduled to execute on

### Request

```
PATCH /api/v1/kolide/packs/123
```

```json
{
	"host_ids": [1, 2, 3, 4, 5],
	"label_ids": [1, 3, 5, 7]
}
```

### Response

```json
{
	"pack": {
		"id": 123,
		"name": "My new pack",
		"description": "The newest of the packs",
		"platform": "",
		"created_by": 1,
		"disabled": false,
		"query_count": 0,
		"total_hosts_count": 5,
		"host_ids": [1, 2, 3, 4, 5],
		"label_ids": [1, 3, 5, 7]
	}
}
```

close #633
2017-01-03 10:32:06 -07:00
Mike Arpaia
b03e504a80 MySQL's datastore.Drop should drop all tables (#738)
* MySQL's datastore.Drop should drop all tables

* Use the database name as a parameter

* tx.Exec instead of tx.MustExec
2017-01-03 09:54:24 -07:00
John Murphy
d1ef37b92c Osquery Options - /config/osquery-options #365 (#729) 2016-12-31 11:56:54 -06:00
Mike Arpaia
9a146c5f92 Renaming targets_count to be more clear (#733) 2016-12-30 12:30:34 -07:00
Mike Arpaia
e9aa5bcc8c Return query_count and targets_count in all pack responses (#731) 2016-12-30 12:06:54 -07:00
Victor Vrantchan
154200db8a Add endpoint to retrieve an invite with the invite token. (#719)
Closes #579
2016-12-29 20:58:12 -05:00
Victor Vrantchan
6cb1026d86 always return errors to the client as a map slice (#724)
keep the format for error returns consistent by always returning a
[]map[string]string for json errors. This simplifies the error handling
on the frontend.

Use "name":"base" as the name field for errors which do not have
a specific or known form field.
2016-12-29 19:40:12 -05:00
Victor Vrantchan
a13042e11b Redirect frontend routes to setup if setup is not configured. (#721)
Closes #617
2016-12-29 18:36:36 -05:00
Zachary Wasserman
ca72a5c67a Add check for all platforms in label query retrieval (#705)
Fixes #703
2016-12-29 12:11:26 -08:00
John Murphy
eec835a07a Osquery options 365 (#657) 2016-12-29 12:32:28 -06:00
Victor Vrantchan
52580ff872 Add invite email template and use the new invite mailer pattern (#711)
Closes #693
Closes #581
2016-12-28 11:55:03 -05:00
Zachary Wasserman
39ebd81dc5 Close orphaned distributed query campaign after failed publish attempt (#707)
A distributed query campaign can be "orphaned" (left in the QueryRunning state)
if the Kolide server restarts while it is running, or other weirdness occurs.
When this happens, no subscribers are waiting to read results written by
osqueryd agents, but the agents continue to receive the query. Previously, this
would cause us to error on ingestion.

The new behavior will instead set the campaign to completed when it detects
that it is orphaned. This should prevent sending queries for which there is no
subscriber.

- New NoSubscriber error interface in pubsub
- Detect NoSubscriber errors and close campaigns
- Tests on pubsub and service methods

Fixes #695
2016-12-27 10:35:19 -05:00
Victor Vrantchan
6b41d2b3f0 always print email messages in dev mode (#698) 2016-12-24 10:41:59 -05:00
Victor Vrantchan
f092c614cf add error logging to distributed query campaign handler (#694) 2016-12-22 20:31:45 -05:00
Zachary Wasserman
2c7fc2a6f2 Minor refactoring in HostStatus function (#692)
- Refactor function
- Add test
2016-12-22 13:32:40 -08:00
Mike Arpaia
ef22d81b50 Return query and name after creating a scheduled query (#690) 2016-12-22 14:29:29 -05:00
Victor Vrantchan
3658e4833e add prometheus metrics to every http endpoint in the app (#680)
tracking the following metrics:

http_request_duration_microseconds
http_request_size_bytes
http_response_size_bytes
http_requests_total
2016-12-22 12:39:44 -05:00
Victor Vrantchan
e7e57ddbc0 prevent password reuse when changing passwords (#678)
For #375
Closes #448
2016-12-22 12:08:29 -05:00
Victor Vrantchan
a47179f142 add a /healthz endpoint which checks that the app is in a healthy state (#674)
by pinging the mysql and redis backends.

For #93
2016-12-22 12:07:47 -05:00
Mike Stone
a84c40061a Prepends https protocol to the inmem org logo url (#685) 2016-12-22 10:03:20 -05:00
John Murphy
d653cdf281 Minor App Config API tweaks requested by @mikestone14 (#681)
* Minor App Config API tweaks requested by @mikestone14

* Refactored mail test into separate method, implemented code review changes
2016-12-22 08:12:34 -06:00
Victor Vrantchan
19fc70ed64 change logo_url for devMode (#683)
Closes #682
2016-12-21 16:22:10 -05:00
Victor Vrantchan
8ef11f51bf create user from invite: set admin property from invite. (#675)
Closes #589
2016-12-21 12:34:10 -05:00
Victor Vrantchan
11a5104d2c errors.Wrap mysql User store (#669) 2016-12-20 17:09:49 -05:00
John Murphy
62748bfbb1 App Settings - /admin/settings #363 (#590) 2016-12-20 15:54:30 -06:00
Victor Vrantchan
41610c1f66 catch sql.ErrNoRows in mysql/invites (#667) 2016-12-20 16:31:09 -05:00
Victor Vrantchan
9d49dbc465 change the implementation of ErrNotFound and AlreadyExists to a struct type (#665)
with an exposed interface.

Not checking for a specific sentinel error reduces coupling between packages
and allows adding context like the resource ID and resource type.
2016-12-20 13:35:22 -05:00
Zachary Wasserman
c8e6405220 Use redis for distributed query results when not in dev mode (#653)
- Add appropriate configs for Redis
- Use the Redis pubsub store by default, inmem in dev mode
2016-12-15 16:13:23 -08:00
Zachary Wasserman
d6765377c0 Rename inmem method receivers for consistency (#654)
This makes the inmem method receiver naming consistent with mysql. It also
eliminates potential confusion with the phrase "orm".
2016-12-15 16:12:43 -08:00
Zachary Wasserman
41120ebc00 Improve user endpoint validations (#642)
- Add empty string checks for NewUser
- Create validations for ModifyUser
- Use GravatarURL when creating new user

Fixes #620
2016-12-15 09:28:53 -08:00
Zachary Wasserman
216e1b58fa Fix parentheses in labels search SQL (#644)
Operator precedence was causing incorrect results to be returned. The failing
test was missed because the CI results did not appear in Github before merging.
2016-12-14 19:24:53 -08:00
Zachary Wasserman
b95c1dd6a0 Always return "All Hosts" label in targets unless selected (#637)
Closes #467
2016-12-14 18:27:22 -08:00
Mike Arpaia
1ada011f37 Remove existing bulk functionality (#641) 2016-12-14 17:22:17 -08:00
Mike Arpaia
56b3799457 Add forgotten JSON serialization tag (#640) 2016-12-14 14:29:20 -08:00
Zachary Wasserman
0e783e7887 Add change password endpoint (#628) 2016-12-14 10:11:43 -08:00
Mike Arpaia
bed279b919 Remove "selected" keyword from response (#635) 2016-12-13 15:59:59 -08:00
Mike Arpaia
f109b14f9d Moving query attributes from the query object to the pack-query relationship (#559)
* Moving query attributes from the query object to the pack-query relationship

* some additional tests

* http request parsing test

* QueryOptions in new test_util code

* initial scaffolding of new request structures

* service and datastore

* test outline

* l2 merge conflict scrub

* service tests for scheduled query service

* service and datastore tests

* most endpoints and transports

* order of values are not deterministic with inmem

* transport tests

* rename PackQuery to ScheduledQuery

* removing existing implementation of adding queries to packs

* accounting for the new argument to NewQuery

* fix alignment in sql query

* removing underscore

* add removed to the datastore

* removed differential from the schema
2016-12-13 14:22:05 -08:00
Zachary Wasserman
556cbf43bd Use canPerformActions authZ check appropriately in handlers (#625)
Fixes #282
2016-12-13 08:57:49 -08:00
Zachary Wasserman
0c62bda2f1 Fix session expiration logic with password reset (#619)
- Only expire sessions at reset request time when admin forces reset
- Expire sessions when reset completed

Prior to this, there was a possible DoS vector in which an attacker could
prevent a user from taking actions in the app by constantly requesting password
resets and expiring all the user's active sessions.

Fixes #612
2016-12-12 09:24:51 -08:00
Zachary Wasserman
b362682d90 Add service method for bulk deleting queries (#600)
- New datastore method for bulk deletion
- New service method calling this datastore method
- Endpoint, transport and handler connections for service method

Closes #389
2016-12-09 09:12:45 -08:00
Zachary Wasserman
d9190020fe Record author information with queries (#578) 2016-12-07 12:22:31 -08:00
Victor Vrantchan
6d9ad56e6d use validation error for expired invite tokens. (#587)
Closes #586
2016-12-07 11:25:48 -05:00