- Clean up the top section
- Clean up queries section
- Update naming for the sub-sections in the "Organization settings" section
- Remove `single-file-configuration.yml` and files under `multi-file-configuration/`
- Remove testing section from product handbook
* Add information about osquery release cadence
Added FAQ item about when new versions of osquery are made available.
* Update docs/Using-Fleet/FAQ.md
Co-authored-by: Zach Wasserman <zach@fleetdm.com>
Co-authored-by: Zach Wasserman <zach@fleetdm.com>
- Remove "[CIS x.x.x]" tags from product
- Add "CIS" tag to all policies in the standard query library on fleetdm.com
- Remove duplicate "Guest users"policy
- Update language in policies' names and descriptions to be consistent
- Add a new "Configuration for contributors" doc page. Move settings that are not recommended for production use
- Remove settings modified in the `config` YAML document from the deploying/configuration doc page
- Document all keys in `config` and `teams` YAML documents
- Add comments to several `.go` files and remove unused struct
* cloud.gov deployment
Adds example manifest.yml file and docs covering steps to deploy fleetdm to cloud.gov
* rename manifest
rename manifest and small edits
- Remove note about vulnerability processing being enabled by default. This message is outdated
- Update first sentence to be explicit that "Vulnerability processing" in Fleet detects vulnerable software
- Add sentence about vulnerability date for Fleet Premium (CVSS, EPSS, CISA).
* Adding CIS 1.5 / 1.6 / 2.2.1 to constants.ts
* Adding CIS 1.5 / 1.6 / 2.2.1 to standard query library
* Adding 2.3.1
* Adding 2.3.1 to query library and 2.4.2 to both
* Adding 2.4.10
* Tagging 2.5.1.1
* Tagging 2.5.2.1
* Tagging 2.5.2.2
* Adding 2.5.6
* Adding 2.6.1.4
* Adding 3.6
* Tagging 5.1.2
* Tagging 5.2.2
* Tagging 5.8
The query we have for the screen lock is comprehensive and covers more than one CIS requirement
* Adding 6.1.3 and 6.1.4
* Ingest installed Windows updates and store them in the windows_updates table.
* Added config option for enabling/disabling Windows update ingestion and Windows OS vuln. detection.
* move contributing.md into docs/contributing, update link in readme
* Update CONTRIBUTING.md
Co-authored-by: Chris McGillicuddy <108031970+chris-mcgillicuddy@users.noreply.github.com>
- Update "Learn how to use Fleet" docs page to walk a Fleet Sandbox user through adding their device and running a query
- Add a "Get operating system information" query to standard query library for the "Learn how to use Fleet" walkthrough
- Update Fleet's top level README to point users who want to try Fleet to Fleet Sandbox
- Update "How to install osquery..." (macOS, Windows, Linux) blog posts to point users who want to try Fleet to Fleet Sandbox
- Move `fleetctl preview` questions to "Contributing" FAQ section in docs. This is because `fleetctl preview` is now a testing tool for Fleet contributors
- Update "Deploying" docs to point users who want to try Fleet to Fleet Sandbox
Related to #7054 and #6834, this adds the UI changes necessary to support JIT provisioning:
A checkbox in the settings page
A new template for the activity box
This also includes relevant documentation about the feature and how to configure it.
* Create FAQ.md
I prepped the new FAQ.md, but ran into some snags with how the site renders the space between the FAQ sections. In particular the Contributing section.
* update consolidated FAQ's headings and relative links
* update consolidated FAQ with latest changes
* add missing links
Co-authored-by: Eric <eashaw@sailsjs.com>
* Updating Configuration Files doc table of contents
I moved the headings up a level to include more H2 headings, which would populate the table of contents.
* Update docs/Using-Fleet/configuration-files/README.md
Caught this while I was reviewing. It should be a tip box.
* Update docs/Using-Fleet/configuration-files/README.md
Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
* remove reference to fleet desktop being in beta
* update how to generate a new Fleet Desktop installer
* document --disable-update and --desktop-channel flags
* new file for Fleet Desktop
* fleet desktop custom transparency link explanation
* docs for installing and upgrading Fleet Desktop
* fleet desktop out of beta announcement blog
* images
* copy edits
* convert headers to sentence case
* incorporate copy changes
* Updated images and meta tags
- I updated the images, and renamed to match our image naming conventions
- I added the article meta tags
* Update Fleet-desktop.md
* Specify Fleet Desktop document order
* Update to fleet-desktop-says-hello-world.md
- I added a screencast showing how to remediate a failing policy with Fleet Desktop
- I added inline styles so that the video fills the container width and renders correctly on mobile.
- I re-jigged "self remediation" and "Scope transparency" as H3 tags.
* address some typos and style changes
* update meta tag
* remove single quotes
* Update publishing date fleet-desktop-says-hello-world
Co-authored-by: Mike Thomas <mthomas@fleetdm.com>
Co-authored-by: Andrew Baker <89049099+DrewBakerfdm@users.noreply.github.com>
* Updated testing.md
Updated the name of testing.md to testing-and-local-development.md based off https://github.com/fleetdm/fleet/issues/4706
* add redirect for /docs/contributing/testing
Co-authored-by: Eric <eashaw@sailsjs.com>
* Update LANG markers regex
* update regex to remove extra newlines from codeblocks
* revert regex in build-static-content
* remove nested codeblock
* update whitespace in changed codeblock
* update regex replace to match indentation on LANG tags
* update regex, add error if markdown was compiled with nested codeblocks
* Revert lang marker regex
* remove newlines before codeblocks in lists
* Update error, move LANG tag regex replace
* update comment to clarify what we mean by nested codeblocks
Co-authored-by: Mike McNeil <mikermcneil@users.noreply.github.com>
Related to #6894, this entirely replaces FLEET_DEMO with the server config added in #6597
As part of this, I also implemented a small refactor to the integration test suite to allow setting a custom config when the server is initialized.
Related to #6365, this extends the datastore/s3 package to retrieve installers from S3 according to the conventions listed in the parent issue. This also includes:
- A minor refactor to decouple Carves-related functionality from the core S3 functionality
- Set-up to run tests using minio (only enabled via the FILE_STORAGE_TEST env flag)
* Adding password policy query to the default library
Adding a built-in policy to check the minimum password length on macOS using the recently released password_policy table.
* Addding osquery minimum version + adding query to constants.ts
- Update docs to use `v1` in the osquery endpoints. Ex: /api/v1/osquery/config.
- Update the Fleet UI's (Hosts > Add hosts > Advanced) flagfile.txt to use `v1` in the osquery endpoints.
Related to #5898, this reports an anonymized summary of errors stored in Redis into the analytics payload.
For each error stored, this includes:
- A `count` attribute with the number of occurrences of the error
- A `loc` attribute with the 3 topmost lines in the stack trace. Note that stack traces only contain package name + line number (example: github.com/fleetdm/fleet/server.go:12
This also includes a minor refactor around error types.
* Renaming appsec page + a few updates
Added a few details (name of appsec tools we use) and renamed the page to make it clearer this is about Fleet appsec and now how to secure a Fleet installation
* Apply suggestions from code review
Co-authored-by: Desmi-Dizney <99777687+Desmi-Dizney@users.noreply.github.com>
* Update Application-security.md
Fixed "merging"
Co-authored-by: Desmi-Dizney <99777687+Desmi-Dizney@users.noreply.github.com>
Co-authored-by: Kelvin Oghenerhoro Omereshone <kelvin@fleetdm.com>
* Adding policy query to check firewall on Mac
This commit closes https://github.com/fleetdm/confidential/issues/1410 once merged.
* Adding policies
This commit closes https://github.com/fleetdm/confidential/issues/1412. Right now there is no way to check the screenlock so instead we check if a profile for screenlock is there.
https://github.com/fleetdm/confidential/issues/1410 also closed by this.
* Update constants.ts
Fixed space
* Resolution text fix for new policies
Fixed copy based on @zhumo's comments!
* Screen lock policy
Adding a policy to check if the inactivity timeout is enabled on Windows and set to 1800 seconds or less (30min)
* Update constants.ts
Fix identation
* Update Windows screen lock policy
Changed wording from "administrator" to "IT administrator" in both files.
- Add "Fleet Desktop" section to "Adding hosts" doc page
- Add instructions to add Jira or Zendesk integration to "Vulnerability automations" section in the "Automations" doc page
* update sso image
* clairfy how to find Okta information
* moving comment about user creation since it applies to all IdP configurations
* change url image link to default link
Co-authored-by: Kelvin Oghenerhoro Omereshone <kelvin@fleetdm.com>
This adds documentation about our debugging endpoints and a brief excerpt about the fleetctl debug command with instructions to generate the archive so we have a place with instructions to which we can direct people.
It also adds the .prof file extension to profiling files, which hopefully clarifies that they are meant to be used by go tool pprof.
This adds two small changes to the contributing docs:
1. Instructions to connect to a local Redis REPL
2. Instructions to start `dlv` in headless mode and attach debuggers to it. I modified `.vscode/launch.json` with a new launch debug config and added instructions for vim.
this change prevents errors from being automatically cleared once they are read. A new flag `-flush` is introduced to flush errors on read if necessary.
* feat[WIP]: updating docs for deploying Fleet on Kubernetes
* feat: update spec to include environment variables
* chore: add fleet image version to yml file
* doc: add Kubernetes manifest file
Add Kubernetes manifest file to files to be edited for Fleet version change when releasing.
* docs: link to the docs on deploying Fleet via K8s
* feat: add kubernetes deployment.yml
* feat: update Fleet version
* Website editor pass - Contribution
I made the descriptions complete sentences. Please let me know if these require further revision.
* Update README.md
* Update docs/Contributing/README.md
Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
* Update docs/Contributing/README.md
Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
* Update docs/Contributing/README.md
Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
* Update docs/Contributing/README.md
Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
* Update README.md
Updated with imperative mood.
* Update docs/Contributing/README.md
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
* Move password reset instructions for API-only user
Moved instructions for resetting the password of a new API-only user to FAQ as it is no longer necessary as of `4.13.0`
* Fix link to reset password instructions
Linked to a previous commit with password reset instructions for API-only users (only necessary on legacy versions).
Install orbit to /opt instead of /var/lib. When installing to /var/lib,
the default selinux context of var_lib_t gets applied, which results in
an AVC error when running via systemd.
Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>
* remove outdated config file description
`example_config.json` was removed in f11da7b05b but the documentation
was still there, this removes the description from the README as well.
* add a note about how to use scripts with premium features
* feat: update query example to conform with the rest of the docs
* Update docs/Using-Fleet/REST-API.md
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
* Add API-only user login issues to FAQ
Added question and detailed instructions for API-only user password reset to FAQ
* Update Using Fleet FAQ
Removed instructions for resetting password for new API-only user and added link to commit with instructions. As discussed with @noahtalerman
* Update docs/Using-Fleet/FAQ.md
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
* Editor pass - Revert api v1 to latest in documentation
Editor pass for: https://github.com/fleetdm/fleet/pull/5149/files
Under: docs/Contributing/API-for-contributors.md
Line 457 needs to be carefully read and rewritten for clarity.
* Update API-for-contributors.md
* feat: add link to the docs of retrieving live query result over web socket
* Update REST-API.md
I added 2 small edits. It looks good!
Co-authored-by: Desmi-Dizney <99777687+Desmi-Dizney@users.noreply.github.com>
* Adding full disk access instructions
* Update Adding-hosts.md
As per @ksatter's comments!
* Update Adding-hosts.md
Made the output a code block
* Formatting code snippets
Changed inline code to code blocks in #Grant full disk access to osquery on macOS to follow previous convention.
Co-authored-by: Katheryn Satterlee <me@ksatter.com>
* Reorganized infrastructure, updated for frontend's loadtesting
* Add changes suggested by @chiiph
* Moved files per suggestion by Ben
* Update docs with new links
* Add config for multi account assume role
* App up to date or not installed
Adding "App installed and up to date OR not present" example
* Removed empty last line
* Update standard-query-library.yml
Added right descriptions and resolution for the Docker example, and added a new query to detect unencrypted SSH keys.
* Update docs/01-Using-Fleet/standard-query-library/standard-query-library.yml
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
* Update docs/01-Using-Fleet/standard-query-library/standard-query-library.yml
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
* Update docs/01-Using-Fleet/standard-query-library/standard-query-library.yml
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
* Update standard-query-library.yml
Updated as per @noahtalerman's review
* Update docs/01-Using-Fleet/standard-query-library/standard-query-library.yml
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
* removed global api routes; using 'latest' instead of v1 for api routes
* lint fixes
* updated docs and tests
* lint fixes
* route fix
* fixed routes breaking packs queries
* revert test change
* Weekly community-driven docs update for week ending March 25th
* Update Configuration.md
* Update FAQ.md
All edits are recorded by line:
192 added “-“ to “auto-updates”
194 added “-“ to “auto-updates”; deleted “,” after “(easy)”; replaced “reccomend” with “recommend”
209 added “,” after “work”
211 replaced “am I seeing” with “do I see”
225 replaced “The best way to maintain historical data currently would be to use the [REST API](./REST-API.md) or the [`fleetctl` CLI](./fleetctl-CLI.md) to manually retrieve and save the data you need on your schedule. “ with “Currently, Fleet only stores the current state of your hosts (when they last communicated with Fleet). The best way at the moment to maintain historical data would be to use the [REST API](./REST-API.md) or the [`fleetctl` CLI](./fleetctl-CLI.md) to retrieve it manually. Then save the data you need to your schedule.”
* Update FAQ.md
All edits are recorded by line:
194 deleted ”either” after “disabled”
* Update fleetctl-CLI.md
All edits are recorded by line:
28 deleted “of the” after “many”; replaced “User Interface” with “UI(User Interface)”; replace “, and to” with “. You can even”
43 capitalized “Fleet”
63 replaced “info” with “information”
* Additional context for help menu and add table
Tweaked wording around using the help command and moved available commands in to a table. @DominusKelvin @Desmi-Dizney
* Update Configuration.md
space
* Final grammar revision
Co-authored-by: Desmi-Dizney <99777687+Desmi-Dizney@users.noreply.github.com>
* Add instructions for creating and using an API-only user with fleetctl
* Update fleetctl-CLI.md
* Update example responses for policies and teams
* Update fleetctl-CLI.md
This was a super clean edit!
All edits are recorded by line:
199 replaced “passing” with “picking”; added “,” after “email”
253 replaced “:” with “.”
* Update fleetctl-CLI.md
Line 253 I changed "." to ":"
* Change `Create a new context for your standard user (optional)` > 'Switching users'
* Update fleetctl-CLI.md
All edits are recorded by line:
195 replaced “are” with “is”
207 replaced “will be able to” with “can”; replaced “needs to be” with “is”
249 replaced “has been” with “is”
* Update API-only user creation instructions
Added additional context to the user creation command and fixed some grammar issues.
Co-authored-by: Desmi-Dizney <99777687+Desmi-Dizney@users.noreply.github.com>
* Adding Orbit audit and new .md page
* Update Security-audits.md
Adding content. Link to PDF won't work until the PR with these two files is merged to main.
* Update Security-audits.md
Added pageOrderInSection
* Update Security-audits.md
Put link to the actual commit so the PDF link can be tested before this gets merged to main
* Update Security-audits.md
Added title
* Fixed tables
* a -> an
Co-authored-by: Eric <eashaw@sailsjs.com>
* feat: add FAQ for distinguishing between fleetctl vs the REST API vs Fleet UI
* Update FAQ.md
Edits recorded by line:
189 changed "vs" to "vs."
191 replaced "useful" with "helpful"
193 added "," after "fleetctl"; deleted "make" and "of"
195 replaced "nice to look at" with "visually appealing"; deleted "is" and "meant to"; added "s" to "make"; replaced "wider" with "broader"
Co-authored-by: Desmi-Dizney <99777687+Desmi-Dizney@users.noreply.github.com>
- Update CHANGELOG
- Bump versioning
- Tweak documentation
- Default `session_duration` to `5d`
- Add extra `#` to "Team policies" section so it doesn't show up in top level nav for docs
* Add Host OS compatibility chart for issue 3359
* Refining compatibility for Linux
* Updated wording of Linux note and changed file name to match heading
* Change Windows version to 10+
* Update Supported-host-operating-systems.md
Edits recorded by line:
5 replaced ":" with "."
17 replaced "in" with "for"
22 added "a" before "CPU"
* Update Supported-host-operating-systems.md
@Desmi-Dizney let me know if that helps at all!
Co-authored-by: Desmi-Dizney <99777687+Desmi-Dizney@users.noreply.github.com>
* geoip wip
* return nil if ip is empty string or if ParseIP returns nil
* add ui component to render geolocation if available, address PR feedback
* render public ip if available
* add changes file, document geoip in deployment guide
* update rest-api docs
* docs: add FAQ for migrating from Fleet Free to Fleet Premium
* docs: add note for redeploying Fleet not being necessary
* docs: add note for redeploying Fleet not being necessary
* fix: add question mark to FAQ
* Renaming files and a lot of find and replace
* pageRank meta tags, sorting by page rank
* reranking
* removing numbers
* revert changing links that are locked to a commit
* update metatag name, uncomment github contributers
* Update basic-documentation.page.js
* revert link change
* more explicit errors, change pageOrderInSection numbers, updated sort
* Update build-static-content.js
* update comment
* update handbook link
* handbook entry
* update sort
* update changelog doc links to use fleetdm.com
* move standard query library back to old location, update links/references to location
* revert unintentional link changes
* Update handbook/community.md
Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
Co-authored-by: Mike McNeil <mikermcneil@users.noreply.github.com>
* Improve documentation for how we do vulnerability processing
* Add mermaid diagrams
* Mermaid, next try
* Change style of once an hour node
* Improve collection diagram
* Improve readability of diagrams
* Improve flow charts
* Update broken link
* docs: add new manage-packs.png
* feat: add new team-agent-options.png
* feat: add new global-agent-options.png
* docs: update screenshots and docs context
* chore: delete stale screenshots
* feat: update screenshot to recommended preset size
* chore: remove editor new line
* feat: update new line
On website + constants.ts. Does not support all Linux encryption scenarios, we will add more to this query as we discover the patterns people need.
Closes#4208
* Adding antivirus queries
Adding 3 antivirus queries in the form of an information query as well as in the form of policy queries
* Update standard-query-library.yml
Adding newline at end of file
* Add CentOS parsing and post-processing in fleet
* Add tests and amend SyncCPEDatabase
* Add test for centosPostProcessing
* Changes from PR comments
* Amend software test
* Fix sync test
* Add index to source and vendor
* Use os.MkdirTemp
* Rearrange migrations
* Regenerate test schema
* Add support for testing migrations (#4112)
* Add support for testing migrations
* Rename migration in tests
* Changes suggested in PR
* Go mod tidy
* fix: update headings in configuration files docs
* fix: update heading in testing docs
* fix: update heading in seeding data docs
* fix: update headings in committing changes docs
* fix: update heading from External Contributors to External contributors
* fix: update headings in API for contributors docs.
* fix: update heading in API versioning docs.
* Add platform filters for MDM/Munki/Chrome queries
This should help quiet warnings that users/customers have reported when
these queries try to run on platforms without the macadmins extension
tables.
For #4123
* Improve documentation
* add changes file
* revert doc formatting
* Update tests
* Yet another test fix
Co-authored-by: Tomas Touceda <chiiph@gmail.com>
* Add remaining policy templates
* Removeempty resolution field
* Update naming for standard policies
* Add enabled word to SIP policy
* Use full SIP
* Also change in constants
* Update windows disk encryption
* Add changes file
* Tweak windows disk encryption policy
* Address lint errors
* Make requested changes
* Reflect changes in policy tempaltes
* Make sure that standard policies and policy templates are the same
* Edit automatic login disabled description
* Also edit in constants
* docs: add instructions for enabling SSO for existing users
* fix: update sentence to emphasize admin
Reword sentence for clarity that admins are the ones supposed to carry out the enable SSO action for existing users
* fix: remove extraneous whitespace
* doc: add introduction to fleetctl docs
* Update docs/01-Using-Fleet/02-fleetctl-CLI.md
Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
* handle query tags in build-static-content script, update query readme
* show tags in query library, add ability to filter by tags
* fix lint errors
* update mobile styles
* fix CTA link
* update mobile layout
* remove tag line-height and font size
* Update build-static-content.js
* Style update
* remove margin from selected tag, adjust OS logo placement
* requested changes from code review
Co-authored-by: Mike Thomas <mthomas@fleetdm.com>
* feat: add FAQ for what happens to logs
FAQ for what happens to log if either the Fleet's server or the log destination are offline
* fix: add the default buffered_log_max
Added reference architectures using https://docs.gitlab.com/ee/administration/reference_architectures/ as inspiration.
- updated terraform based on some feedback of usage
- pinned fleet docker version in terraform as to no get unexpected upgrades when applying
- updated some documentation around apply migration tasks
* Add sentry
* Fix gosum
* More gosum fixes
* Add missing def for config
* Enrich sentry scope a bit
* Add changes file
* Add goroutine safe scope to errors
* Encapsulate sentry logic
* Add documentation for new flag
* Add sentry capturing to crons and other background tasks
* Only send to sentry when enabled
This helps the period stay under the default request timeouts for most
load balancers.
Some default timeouts:
* AWS ALB - 60s
* Nginx - 60s
* GCP LB - 30s
* doc: add FAQ for orbit running alongside osquery
* fix: update FAQ to be more clear.
Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
* fix: typo on osquery
Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
* Updating Upgrade docs
* making changes to database section of upgrade docs
* Update docs/02-Deploying/06-Upgrading-Fleet.md
Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
* Update docs/02-Deploying/06-Upgrading-Fleet.md
Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
* Update docs/02-Deploying/06-Upgrading-Fleet.md
Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
* Corrected anchor tags, ready to merge
Co-authored-by: Katheryn Satterlee <ksatter@Kathys-MacBook-Pro.local>
Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
* feat: add query to find running docker containers
* docs: 📝 add query to get Mac and Linux machines with unencrypted primary disks
* feat: add query to find running docker containers
* docs: 📝 add query to get Mac and Linux machines with unencrypted primary disks
* chore: remove queries from develop
* docs: add query to get running docker containers
* docs: add query to get machines with unencrypted primary disks
* fix: remove trailing ---
* fix: remove trailing ---
* chore: remove trailing ---
* docs: add query to get applications hogging memory
* fix: resolve merge conflicts
* chore: update PR
* chore: update PR
* chore: merge previous query
* feat: add query to find servers with root logins within the day
* fix: remove unneeded period
* docs: add instructions for submiting multiple queries
* fix: remove duplicate entry
* fix: remove period from get running docker containers query description
* docs: add instructions for submiting multiple queries
* fix: resolve merge conflicts
* feat: add description for query to fetch failing batteries
* fix: resolve duplicate descriptions
* fix: remove typo in deplying docs
* fix: reword description
* fix: add suggestions to improve description
* feat: add description to query to fet windows machines with unencrypted hard disks
* feat: update description for count apple applications installed query
* chore: add dominuskelvin as maintainer
* docs: 📝 Add query to get apps opened within the last 24 hours
* feat: add link to signing installers
* fix: typo with link to the getting started page
* feat: docs on how to sign an osquery installer
* feat: make signing installer a subsection of osquery installer
* feat: make description for signing installer shorter and compact
* fix: change package to installers
* fix: reword note section
* feat: add query to find running docker containers
* docs: 📝 add query to get Mac and Linux machines with unencrypted primary disks
* feat: add query to find running docker containers
* docs: 📝 add query to get Mac and Linux machines with unencrypted primary disks
* chore: remove queries from develop
* docs: add query to get running docker containers
* docs: add query to get machines with unencrypted primary disks
* fix: remove trailing ---
* fix: remove trailing ---
* chore: remove trailing ---
* docs: add query to get applications hogging memory
* fix: resolve merge conflicts
* chore: update PR
* chore: update PR
* chore: merge previous query
* feat: add query to find servers with root logins within the day
* fix: remove unneeded period
* docs: add instructions for submiting multiple queries
* fix: remove duplicate entry
* fix: remove period from get running docker containers query description
* docs: add instructions for submiting multiple queries
* fix: resolve merge conflicts
* feat: add description for query to fetch failing batteries
* fix: resolve duplicate descriptions
* fix: remove typo in deplying docs
* fix: reword description
* fix: add suggestions to improve description
* feat: add description to query to fet windows machines with unencrypted hard disks
* feat: update description for count apple applications installed query
* docs: 📝 Add query to get apps opened within the last 24 hours
* feat: add query to find apps not in Applications directory
* feat: add query to find subscription based applications that have not been opened for the last 30 days
- Add "Automations" documentation page to document the available automations in Fleet
- Update the "Vulnerability processing" documentation
- Update the "REST API" documentation
* Add webhook to app config
* Add redis failing policies set and webhook
* Add basic webhook test
* Store hostname in redis
* Global policy deletion to remove policy ID from set and config
* Also process new passing policies
* Fix unit test
* Sort hosts
* Add more tests
* Add ListSets to the failing policies interface
* Fix server URL and garbage collect on the triggering side
* Do not use Redis SCAN
* Fix Redis operation order
* Add API changes to doc
* Add comments
* Add more tests
* Fix tests
* Add tests for config update upon deletion of policies
* Run make dump-test-schema
* Ignore policies that failed to run
* Add proper unit tests to trigger logic
* Fix comments
* WIP
* Add tests to service_osquerty_test.go
* Use SSCAN for listing hosts instead of SMEMBERS
* Add failing policies to docs/01-Using-Fleet/configuration-files/README.md
* Remove skip
* Fix PR comments
* Finish first draft of API versions
* wip
* Finalize tests
* Revert change in handler
* Remove made up version
* Update versioning with aliases
* Add changes file
* Address review comments
* Revert overupdated routes
* Expand life time of deprecated APIs
* Fix test
* Comment out problematic part of test
* Revert bad path changes
* updating docs to make things a bit more clear
* fixing broken links
* more broken links
* fixing broken links
* website updates
* PR review changes
* fixing bad links
Queries:
- Get applications hogging memory
- Get Mac and Linux machines with unencrypted primary disks
- Get servers with root login in the last 24 hours
* Add software count API
* Fix makefile
* Fine no mock generating at this point
* Actually, one last try
* Use go install instead
* Fix go sum/mod
* Improve documentation
* Try setting node to 14
- Emphasize generating an "osquery installer" (also referred as "Orbit") using the `fleetctl package` command
- Add instructions for adding multiple hosts and automatically adding hosts to a team
- Remove instructions for generating an enrollment package with the tooling in `tools/mac/`
- Remove the `tools/mac/` directory from the repository because it is no longer used or referenced in any documentation
- Update "Automatically adding hosts to a team" section of "Teams" documentation to point to "Adding hosts" documentation
- Add instructions for migrating from plain osquery to Fleet's osquery installers
* add faq questions
* Apply suggestions from code review
Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
* fix broken links
* Update FAQ.md
Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
- Fix a broken link in the summary of the "Osquery logs" documentation
- Update instructions for linking to a specific section within a page in the Fleet documentation in product handbook
* Add host count to software API
* Update docs
* Update fleetctl tests to account for host counts
* Update docs to mention host_count special case
* Update func comment
- Add top level links to the "Seeding Data" and "API for contributors" doc pages
- Move "Results" section in "Loading testing" closer to the top of document
- Add `07-API-for-contributors.md` documentation page
- Move all API routes used exclusively by the Fleet UI and fleetctl clients into the new documentation page
- Removed create/edit/delete enroll secret permissions from team level users
- Update verbiage to clarify the distinction between users with global access and users with team access.
This PR implements the status/result logger functions necessary interface with a Kafka REST Proxy service.
Specifically, this is compatible with the [Confluent KAFKA Rest Proxy Service ](https://docs.confluent.io/1.0/kafka-rest/docs/intro.html).
* Add 3 policies and update contributor instructions
* Update capitalization
* Add policy kind option
* Add policies spec
* Remove the 'purpose' field
* Add single policy yaml documents. Update Learn how to use Fleet.
* Remove no longer relevant screenshot of live query results
* Revert changes to standard query library
* Revert changes to standard query library
* Update docs/01-Using-Fleet/00-Learn-how-to-use-Fleet.md
typo fix.
Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
Fix syntax for the following queries:
- Get user files matching a specific hash
- Check for artifacts of the Floxif trojan
- Get malicious Python backdoors
* wip
* wip
* wip
* Add performance stats
* Add docs and other self review fixes
* lint
* Update based on review comments
* Add quick cleanup first and then reset to 1hr
* Reduce the load in the test
- Add a summary to the top of the document
- Rename "Baseline Test" section to "Test parameters"
- Rename "Bare minimum setup" section to "1,000 hosts"
- Several smaller edits that call out the number of hosts tested and the results (did Fleet work?)
* Add infra for loadtest
* Move loadtest stuff to a new file and parametrize fleet min/max capacity
* wip
* wip
* wip
* wip
* wip
* wip
* wip
* Update to be ready for review
* Update link and other variables needed
* Address review comments and update links
* Start a fleetctl preview test
* Add tests for fleetctl preview
* Fix setting of fleetctl auth token in test
* Add fleet instance vulnerabilities config to response of GetAppConfig
* Add checks that fleetctl preview enables vulnerability detection
* Adjust doc for get config API response
* Add the include-server-config flag to fleetctl get config
* Update test now that some of the PRs have been merged
Co-authored-by: Tomas Touceda <chiiph@gmail.com>
* Update FAQ.md
* Update FAQ.md
* Update docs/01-Using-Fleet/FAQ.md
Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
* Update docs/01-Using-Fleet/FAQ.md
I couldn't make a decision about the correct science around the mum and baby reference, so I just deleted to be safe 😅
Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
* addred cropped images to replace images in configuration.md, fixed duplicated alt text, added box-shadow to images
* more cropping
* reduce padding on ordered lists, point urls in markdown to where images will be
* Update 02-Configuration.md
* Update build-static-content.js
* remove box-shadow on images
* Update 02-Configuration.md
* Added handbook entry about images
* changed img tags to markdown links to be consistent
* undo small style change
* Update build-static-content.js
* Update handbook/product.md
Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
* handling redirect
* clean up redirect/notFound flow
* dynamic query titles and description
* dynamic docs titles
* Remove locals from queries and docs, updated comment and variable names
* handling FAQ page titles
* update comment, title in meta, and adjust get started title
* remove unneeded meta tag
* handbook titles
* cleaning up conditionals
* update comments
* removed added meta tags, change meta variable names
* passing in meta description if provided, update conditionals and comments
* Update FAQ.md
- Add example `team.yml` configuration file. A file with this format can be used to apply teams using `fleetctl apply`
- Add `spec/teams` API route to API docs
* Add max jitter percent config
* Fix jitter calc
* Remove comment
* Reduce test jitter to make tests less flaky
* Remove jitter entirely
* Document new config
* Fix doc link
* Add team policies
* Add team policy documentation
* Add changes file
* Update titles
* Fix lint
* Rewrite TeamAuthorize for more clarify
* Explicitly use two slices for clarity
* Simplify switch
https://github.com/fleetdm/fleet/pull/2071 (removing hardcoded widths on images) undoes what we previously did for making smaller images look good at <990px breakpoints.
Only current examples of these smaller images are on this page in the docs, although there are a couple of instances in the handbook. So I propose that we only crop images that will work at full container width sizes.
With that in mind I have replaced one of the affected images on this page.
* remove hardcoded width on images
* fix inconsistent image padding on fleet ui docs page
* Broken link fix
Fixed a couple of broken links to help this PR pass the automated tests.
Co-authored-by: Mike Thomas <mthomas@fleetdm.com>
* remove concept of "Detection", for now (in favor of what's coming)
* remove extra --- to make YAML parse properly
* Simplify the check to remove remediation check for now
* Run compile script any time docs or handbook is changed
* Implement fleetctl get software and the underlying API
* Add documentation
* Simplify list software implementation
* Lint fixes
* Make team name unique
* Address review comments
* Fix lint
* Fix tests
* Update standard-query-library.yml
Added new queries to library
* sentence-case capitalization + standardize first word in name
* andrewbare => alphabrevity (so your picture shows up correctly on the website)
Co-authored-by: Mike McNeil <mikermcneil@users.noreply.github.com>
* Skeleton UI
* Rebase to main
* Work towards config API request modification
* Nest and unnest when formatting for server and frontend
* Changelog
* Add validation to UI, fix ? vertical spacing
* Rebase e2e
* 1 of 2 passing unit tests for config
* Update REST-API.md to include webhook_settings
* Destructure / flatten config webhook in unit test
* Merge advance options e2e conflict
* x and y example not x and x
* Fix observer e2e
* Add new data to read only example request
Co-authored-by: Noah Talerman <noahtal@umich.edu>
Co-authored-by: Martavis Parker <martavis@auraticdevelopment.com>
* Add extra debug logging for hosts
* Add changes file
* Ignore if appconfig is nil
* Use slice of uints instead of a string
* Debug response request for enabled hosts
* Add host-id to request/response
* Lint fixes
* Add missing AppConfigFuncs
* added package command from orbit as fleetctl command
* update deployment docs
* add changes file
* added tests for package command, run go mod tidy & go mod verify
* validate that package files exist
* comment out msi packaging test until we can investigate github runner permission issues
* Rename core->free and basic->premium
* Fix lint js
* Comment out portion of test that seems to timeout
* Rename tier to premium if basic is still loaded
* adjusted isUnsupportedBrowser to include all versions of internet explorer and changed unsupported browser message for ie11
* removed ie11 from supported browsers docs page
* Create vuln path if possible
* Make sure we skip creation if static instance is selected
* Document behavior
* Fix return in crons and test without sleeps
- Add item and link to all "Reported bugs" (GitHub issues with the "bug" and ":reproduce" label). This way, the individual on call can navigate to a list of bugs that need reproduction.
- Add a link to the open pull requests. This way, the individual on call can identify any PRs that have been opened by the community.
* wip
* Add tests and finish implementation
* Add proper default for periodicity, changes file, and documentation
* Fix tests and add defaults also to new installs
* EnableHostUsers should be true if undefined as well
* In some cases, periodicity can be zero because of the migrations
* Apply defaults when migrating appconfig
* Fix lint
* lint
* Address review comments
* Add global policies
* Update documentation and add extra parameter to config
* Fix failing tests
* Store historic policy records
* Address review comments
And also remove other inmem references I saw by chance
* Add documentation for get by id request
* Add parameter doc
* Move schema generation to a cmd instead of a test
Otherwise it messes up running all tests sometimes depending on how parallel it does
* Remove brain dump for another task
* Make migration tests a separate beast
* Make schema generation idempotent and move dbutils cmd to tools
* Allow all filters and add counts to Policy
* Add test for Policy
* minor clarifications
* further expand comments and stubs
* absorb custom titles embedded in metadata, plus further comment expansion and a followup fix for something i left hanging in f8cbc14829
* Skip non-markdown files and use real path maths
* Prep for running in parallel (Remove `continue` so this isn't dependent on the `for` loop)
* determine + track unique HTML output paths
* Compile markdown + spit out real HTML (without involving any but the crunchy nougaty dependency from the very center of everything)
* add md metadata parsing
* add timestamp
* Update build-static-content.js
* attach misc metadata as "other"
* how doc images might should work (this also aligns with how the select few images in the sailsjs.com docs work)
* add file extension to generated HTML files
* "options"=>"meta"
* Make "htmlId" useful for alphabetically sorting pages within their bottom-level section
See recent comments on https://github.com/fleetdm/fleet/issues/706 for more information.
* list out the most important, specific build-time transformations
* Omit ordering prefixes like "1-" from expected content page URLs
* add a little zone for consolidating backwards compatible permalinks
* interpret README.md files by mapping their URLs to match their containing folder
* clarify plan for images
* decrease probability of collisions
* Make capitalization smarter using known acronyms, proper nouns, and a smarter numeric word trim
* Resolve app path in case pwd is different in prod
* Delete HTML output from previous runs, if any
* condense the stuff about github emojis
* got rid of "permalink" thing, since id gets automatically attached during markdown compilation anyway
Also "permalink" isn't even a good name for what this is. See https://github.com/fleetdm/fleet/issues/706#issuecomment-884693931
* …and that eliminates the need for the cheerio dep!
* Bring in bubbles+syntax highlighting into build script, and remove sails.helpers.compileMarkdownContent() -- this leaves link munging as a todo though
* trivial (condense comments)
* Remove unused code from toHtml() helper
* Implemented target="_blank" and root-relative-ification
* remove todo about emojis after testing and verifying it works just fine
* trivial: add link to comment in case github emojis matter at some point
* consolidate "what ifs" in comments
* Leave this up to Sarah, for now. (Either bring it back here in the build script or do it all on the frontend)
* Enable /docs and /handbook routes, and add example of a redirect for a legacy/deprecated URL
* implement routing
* Upgrade deps
this takes advantages of the latest work from @eashaw, @rachaelshaw, and the rest of the Sails community
* tweak var names and comments
* make readme pages use their folder names to determine their default (fallback) titles
as discussed in https://github.com/fleetdm/fleet/issues/706#issuecomment-884788002
* first (good enough for now) pass at link rewriting
as discussed in https://github.com/fleetdm/fleet/issues/706#issuecomment-884742072
* Adapt docs pages to build from markdown output
* Continue work on docs pages
* Add landing page
* Remove unused code; minor changes
* Replace regex
* fixes https://github.com/fleetdm/fleet/pull/1380#issuecomment-891429581
* Don't rely on "path" being a global var
* Syle fleetdm doc pages
* Continue work on docs pages
* Fix linting error
* Disable lesshint style warnings
* parasails-has-no-page-script attribute
Added a parasails-has-no-page-script attribute to the docs template, added a check for that attribute in parasails.js and removed the empty page script for 498
* bring in latest parasails dep
* trivial
* Update links to dedupe and not open in new tab unless actually external
* Disable handbook for now til styles are ready
* fix CTA links
* trivial
* make sitemap.xml get served in prod
* hide search boxes for now, remove hard-coded version and make releases open in new tab
* clean out unused files
Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>
Co-authored-by: eashaw <caglc@live.com>
- Remove "How to add Fleet's standard query library" section from the "Ask questions about your devices" walkthrough.
- Add `license`, `vulnerability_settings`, and `logging` properties to the example response for the `GET /api/v1/fleet/config` and `PATCH /api/v1/fleet/config` API routes
* Make receive calls to redis conn thread safe
Also removes REDIS_TEST env var. Redis is lightweight and fast, no need
to skip these tests.
* No need to increase the wait
* Add safe mkdirall and open
* Use secure as much as possible and merge gomodules for orbit to fleet
* Improve openfile and mkdirall to check for permissiveness instead of equality
* Don't shift
* Fix links
* Address review comments
Checked and fixed a couple of typos here and there, and made some edits to some of the sentence structure to improve the flow, and to soften the tone a bit.
- Specify valid time units of `s`, `m`, and `h` for the `session_duration`, `osquery_label_update_interval`, and `osquery_detail_update_interval` configuration options.
This tutorial is the first step at bridging the gaps between trying Fleet, getting to know Fleet in an enterprise setting, and deploying Fleet.
- Add "tutorials" subdirectory
- Add walkthrough
- Remove duplicate documentation from "Fleet UI" section
- Link to walkthrough from top-level README
- Remove changes files that were collected during the 4.1.0 release cycle
- Add removing changes files step to release process documentation
- Update support process docs to reflect the change to discussing the last on-call rotation at 🍽️ Daily standup meeting
- The included query populates the `users` property in the `/api/v1/fleet/hosts/{id}` response object.
- This information also populates the new "Users" table on the Host details page
* update .gitattributes to be explicit about line endings with regards to the test certs
* update building-fleet guide to include python2 dependency on windows
* update configuration to default to OS specific temporary directories
- Remove "Connecting a host" from fleetctl documentation
- Remove queries from REST API examples
- Point to Launcher documentation
- Update community projects
- Move configuration files examples from 2-fleetctl-CLI.md into /configuration-files/README.md.
- Update example configuration files with "agent options" and remove all references to "osquery options"
- Update numbering for documentation files
- Use goreleaser to automate release process.
- Add new dockerfiles for fleet (with fleetctl) and fleetctl (only).
- Add GitHub Action Workflow to run goreleaser on new tag.
- Update NPM to match new archive naming.
- Update enroll secret config file with updated fields for Fleet 4.0.0
- Remove agent options config file. In Fleet 4.0.0, agent options are part of the organization settings configuration
* Remove username from UI code
* Remove username from tests
* Remove username from database
* Modify server endpoints for removing username
* Implement backend aspects of removing username
* Update API docs
* Add name to fleetctl
- Add "Configuring agent options" section to `/1-Using-Fleet/1-Fleet-UI.md`
- Add 2 screenshots: 1. Global agent options form 2. Team agent options form
- Move instructions for connecting to the Mailhog simulated server out of the FAQ
- Merge instructions found in FAQ with existing instructions in `Testing.md`
-Modify build-static-content back-end script to implement GitHub Users API and build contributor profile information into query library pages
-Remove related functionality from client-side page scripts
-Add dropdown menu to select filters
-Refine html and css
- No longer detecting for OS make commands
- Using the same internal commands for every OS
- go.sum auto-updated during build
- Document build process on Windows
- Move team-related service methods to `ee/server/service`.
- Instantiate different service on startup based on license key.
- Refactor service errors into separate package.
- Add support for running E2E tests in both Core and Basic tiers.
- Add `team_id` field to secrets.
- Remove secret `name` and `active` fields (migration deletes inactive secrets).
- Assign hosts to Team based on secret provided.
- Add API for retrieving secrets by Team.
* /sandbox/queries becomes /queries, etc
* Publish fleetdm.com/queries
Expose query library routes the rest of the way, move remediation data sanitization to the point of entry, and update query library to match (pairing w/ @gillespi314)
* Fix accidental commit of sailsrc (again)
- In tests and documentation, replace `@fleetdm.com` with `@example.com`
- In documentation, replace `hello@fleetdm.com` with `fleetdm.com/contact`
- In documentation, replace `security@fleetdm.com` with `fleetdm.com/contact`
- In Dockerfiles, replace `engineering@fleetdm.com` with `hello@fleetdm.com`. These two files are the only remaining files with a `@fleetdm.com` email.
- Add link to "Fleet 3.11.0 released with software inventory" to location in docs where software inventory is described.
- Change "host details" to "host vitals"
- Accept Teams as a searchable target type for the target selection API.
- Accept Teams for targets in running live queries.
- Refactoring to support these changes.
- Update API documentation.
- Move host `additional` into a separate table.
- Join when that data is needed.
- API change: `/api/v1/fleet/hosts` now returns only the requested
`additional` columns, unless `*` is provided as the sole argument.
Background:
A customer reported that MySQL binlogs grew huge and replication lag
went way up when data was stored in the `additional` column. In this
deployment MySQL was running with ROW replication. This would cause the
entire `additional` data to be copied on each update of the host checkin
time. While switching to STATEMENT or MIXED replication would likely
mitigate the issue, this was not an option in their environment.
- Include only hosts that the user has access to in search targets API.
- Add parameter to specify whether `observer` hosts should be included.
- Generate counts based on which hosts user can access.
- Update API doc.
- Add question that addresses upgrading from Kolide Fleet to FleetDM Fleet
- Edit "Automatically add hosts to packs" question and move portion about `targets` field to `fleetctl` docs
Adds the following queries to the Standard query library:
- Get authorized keys for Local Accounts
- Get authorized keys for Domain Joined Accounts
- Get current users with active shell/console on the system
- Get Disk encryption status
- Detect Unencrypted SSH Keys for Local Accounts
- Detect Unencrypted SSH Keys for Domain Joined Accounts
- Line parsed values from system and user cron/tab
- Detect Dynamic Linker Hijacking (MITRE. T1574.006)
- Get etc hosts entries
- Get Network Interfaces
- Get Local User Accounts
- Detect active user accounts on servers
- Detect Nmap Scanner
- Get docker images on a system
- Get docker running containers on a system
- Get docker running process on a system
- Create `/configuration-files/` directory inside of `/1-Using-Fleet` directory. This directory contains example Fleet configuration files in yaml format. Replaces the `/examples` directory.
- Create `/standard-query-library/` directory inside of `/1-Using-Fleet` directory. This directory contains the new `standard-query-library.yml`. This file will act as the source of community contributions to the standard query library.
- Edit references to `/examples` directory
This feature enables a new config option (redis.duplicate_results). When set to true, all Live Query results will be copied to an additional Redis pubsub channel named LQDuplicate
This is useful in a scenario that would involve shipping the Live Query results outside of Fleet, near-realtime.
This allows the host details to be refetched on the next check in,
rather than waiting for the normal interval to go by. Associated UI
changes are in-progress.
- Migration and service methods for requesting refetch.
- Expose refetch over API.
- Change detail query logic to respect this flag.
Add a config setting to allow copying message fields and decorations into Google Pub/Sub attributes, making it possible to use these values for subscription filters.
My changes to the documentation file structure made in #717 resolved all broken documentation links on the `master` branch but not on the `teams` branch. As a result, those developing on the `teams` branch always get a ❌ for the "Markdown link check" test.
- Add relative markdown links in the `teams` branch to resolve failed link test
- Add section on contributing to Fleet documentation. This section covers the use of relative links and best practices for anchor links
- Add 3 questions and their respective answers to the FAQ section in the `1-Using-Fleet` docs
- Add new `2-Orbit-osquery/` directory to the top-level `docs/` directory.
- Rename `2-Deployment/` -> `3-Deployment/` to accommodate new Orbit directory.
- Rename `3-Contribution/` -> `4-Contribution/` to accommodate new Orbit directory.
- Add FAQ section to Orbit documentation.
This PR concludes the Complete API documentation project #43
Add documentation for the following endpoints:
- api/v1/status/live_query
- api/v1/status/result_store
- api/v1/sso/callback
- Maintain software inventory with detail queries.
- Associated database migrations.
- Feature flagged off by default (see documentation for details to turn on).
- Documentation.
- New test helper for slice element comparisons skipping ID.
- Add 2 sections in `CONTRIBUTING.md` these 2 sections correspond to the "Bug report" and "Report a security vulnerability" issue templates
- Add "Is this an issue with the Fleet UI" to "Bug report" section in `CONTRIBUTING.md`. This includes a walkthrough for opening the browser's JS console and network requests
- Fix misspelled file name
This PR contains the initial implementation of the fleetctl updates commands, along with documentation on using this to self-host an agent update server.
Co-authored-by: Noah Talerman <noahtal@umich.edu>
Co-authored-by: Mike McNeil <mikermcneil@users.noreply.github.com>
This PR includes various fixes to anchor links used in the documentation.
There are certain characters GitHub doesn't support for the use of anchor links in markdown files. The general rule I've found is to only use a-z or A-Z characters in anchor links. All other characters should be removed.
For example, consider the section title How do I connect to the Mailhog simulated server?. The valid GitHub anchor link for this section is #how-do-i-connect-to-the-mailhog-simulated-server. Notice no ?.
Closes#494
The api/v1/fleet/queries/run_by_names endpoint was incorrectly documented as discussed in #409. This PR includes the fix.
- Add the query parameter to the queries/run_by_names endpoint and edit the example requests to include this parameter.
This PR includes two documentation fixes.
1. Remove the support property from the osquery queries spec in the fleetctl docs. The support property is not a feature of osquery or Fleet.
2. Edit the api/v1/fleet/queries/run_by_names with accurate examples. This endpoint sends query targets (hosts and labels) by name (hostname and label name). Contrast this with the api/v1/fleet/queries/run which sends query targets by id (host id and label id)
Resolves#409
This PR is part of the Complete documentation for Fleet API project #43.
The endpoint included in these changes:
- POST /api/v1/fleet/reset_password
- GET /api/v1/fleet/sessions/{id}
- DELETE /api/v1/fleet/sessions/{id}
- POST /api/v1/fleet/queries/delete
- GET /api/v1/fleet/email/change/{token}
The enrollment cooldown period was sometimes causing problems when
osquery (probably unintentionally, see
https://github.com/osquery/osquery/issues/6993) tried to enroll more
than once from the same osqueryd process.
We now set this to default to off and make it configurable. With #417
this feature may be unnecessary for most deployments.
Osquery now exposes more information during host enrollment than Fleet
previously handled. We can use this to provide more options to users in
problematic enrollment scenarios.
Users can configure --osquery_host_identifier in Fleet to set which
identifier is used to determine uniqueness of hosts. The
default (provided) replicates existing behavior in Fleet. For many
users, setting this to instance will provide better enrollment
stability.
Closes#373
Add documentation on the websocket endpoints in the Fleet API. These endpoints allow users to retrieve live query results.
Endpoints added include:
- `api/v1/fleet/results`
- `api/v1/fleet/results/websockets`
- Fix api/v1/fleet/queries/run endpoint. Prior the fix, the endpoint was incorrectly documented as api/v1/fleet/spec/queries/run
- Fix api/v1/fleet/queries/run_by_names endpoint. Prior the fix, the endpoint was incorrectly documented as api/v1/fleet/queries/run
The endpoint included in these changes:
- /api/v1/kolide/carves GET
- /api/v1/kolide/carves/{id} GET
- /api/v1/kolide/users/{id} PATCH
- /api/v1/kolide/users/{id}/enable POST
- /api/v1/kolide/users/{id}/admin POST
- /api/v1/kolide/users/{id}/require_password_reset POST
- /api/v1/kolide/users/{id}/sessions GET
- /api/v1/kolide/users/{id}/sessions DELETE
- Add query parameter to the /api/v1/fleet/hosts endpoint and edit the example request and response.
- Add query parameter to the /api/v1/fleet/labels/{id}/hosts endpoint
These changes support the ability to perform a search on the Hosts table by hostname, machine_serial, and ipv4.
This PR is part of the Complete documentation for Fleet API project #43.
The endpoint included in these changes:
- `POST api/v1/fleet/targets`
Additional changes:
- Now order_key and order_direction are included in the parameters table for endpoints that support them. Fixes#326
- Change endpoints to `api/v1/fleet` naming
This PR is part of the Complete documentation for Fleet API project #43.
The endpoints included in these changes:
- /labels POST
- /labels/{id} PATCH
- /labels/{id} GET
- /labels GET
- /labels/{id}/hosts GET
- /labels/{name} DELETE
- /labels/id/{id} DELETE
- /spec/labels POST
- /spec/labels GET
- /spec/labels/{name} GET
- Support both /api/v1/fleet and /api/v1/kolide routes in server.
- Add logging for use of deprecated routes.
- Rename routes in frontend JS.
- Rename routes and add notes in documentation.
In #212 these settings were updated and caused connectivity issues for
users in common environment configurations. The new changes are
aggressive (modern enforces TLS 1.3) and Mozilla indicates that
intermediate is an appropriate default. This will ensure better
compatibility for common deployments while still allowing the option to
use the strictest settings.
Document unintentional mismatched yaml key.
Fixes#269
- Add --dev flag that will set default flag values. This simplifies the
invocation of Fleet in a development environment.
- Change defaults in docker-compose to use `fleet` in place of `kolide`.
- Skip prompt in `prepare db` when `--dev` specified.
- Update developer documentation.
Updates to MySQL configuration in docker-compose.yml may require
existing development containers and volumes to be deleted (this will
delete data in MySQL):
```shell
docker-compose rm -sf
docker volume rm fleet_mysql-persistent-volume
```
Closes#170
Reformat the anchor links for specific endpoints by moving these links to their respective sections.
The endpoints included in these changes:
- /api/v1/kolide/packs POST
- /api/v1/kolide/packs/{id} PATCH "modify_pack"
- /api/v1/kolide/packs/{id} GET "get_pack"
- /api/v1/kolide/packs GET "list_packs"
- /api/v1/kolide/packs/{name} DELETE "delete_pack"
- /api/v1/kolide/packs/id/{id} DELETE "delete_pack_by_id"
- /api/v1/kolide/packs/{id}/scheduled GET "get_scheduled_queries_in_pack"
- /api/v1/kolide/schedule POST "schedule_query"
- /api/v1/kolide/schedule/{id} GET "get_scheduled_query"
- /api/v1/kolide/schedule/{id} PATCH "modify_scheduled_query"
- /api/v1/kolide/schedule/{id} DELETE "delete_scheduled_query"
- /api/v1/kolide/spec/packs POST "apply_pack_specs"
- /api/v1/kolide/spec/packs GET "get_pack_specs"
- /api/v1/kolide/spec/packs/{name} GET "get_pack_spec"
The endpoints included in these changes:
- /api/v1/kolide/queries/{id} GET
- /api/v1/kolide/queries/{id} PATCH
- /api/v1/kolide/queries/{name} DELETE
- /api/v1/kolide/queries GET
- /api/v1/kolide/queries POST
- /api/v1/kolide/queries/id/{id} DELETE
- /api/v1/kolide/spec/queries POST
- /api/v1/kolide/spec/queries GET
- /api/v1/kolide/spec/queries/{name} GET
- /api/v1/kolide/queries/run POST
- /api/v1/kolide/queries/run_by_names POST
The endpoints included in these changes:
- /api/v1/kolide/host_summary GET
- /api/v1/kolide/hosts/{id} GET
- /api/v1/kolide/hosts/identifier/{identifier} GET
- /api/v1/kolide/hosts/{id} DELETE
- /api/v1/kolide/spec/osquery_options POST
- /api/v1/kolide/spec/osquery_options GET
Mozilla's recommended settings have changed since this was last updated.
We now link directly to the relevant revision in the Mozilla wiki to
avoid confusion if this is updated.
This PR is part of the Complete documentation for Fleet API project #43.
The endpoints included in these changes:
- /api/v1/kolide/config/certificate GET
- /api/v1/kolide/config GET
- /api/v1/kolide/config PATCH
- /api/v1/kolide/spec/enroll_secret POST
- /api/v1/kolide/spec/enroll_secret GET
- /api/v1/kolide/invites POST
- /api/v1/kolide/invites GET
- /api/v1/kolide/invites/{id} DELETE
- /api/v1/kolide/invites/{token} GET
- Quick attempt to alleviate potential confusion for how the osquery configuration overrides option works
- Leave in the commented explanation in the sample yaml file
- Split the array of objects to separate objects for each pattern in ignorePatterns. Rename the key in each object to "pattern." The documentation for the example config file is poor.
- Add hello@fleetdm.com and /server/datastore/mysql/migrations/ to ignorePatterns
- Add 999 to aliveStatucCodes (Seems to be a LinkedIn edge case)
- Fix all broken markdown links
The current implementation of FleetDM doesn't support Docker secrets for supplying the MySQL password and JWT key. This PR provides the ability for a file path to read in secrets. The goal of this PR is to avoid storing secrets in a static config or in an environment variable.
Example config for Docker:
```yaml
mysql:
address: mysql:3306
database: fleet
username: fleet
password_path: /run/secrets/mysql-fleetdm-password
redis:
address: redis:6379
server:
address: 0.0.0.0:8080
cert: /run/secrets/fleetdm-tls-cert
key: /run/secrets/fleetdm-tls-key
auth:
jwt_key_path: /run/secrets/fleetdm-jwt-key
filesystem:
status_log_file: /var/log/osquery/status.log
result_log_file: /var/log/osquery/result.log
enable_log_rotation: true
logging:
json: true
```
This PR includes the Using Fleet section of the documentation restructure #144.
It shouldn't be merged until changes are approved for the entire restructuring (part 1, part 2, and part 3).
Update the naming convention for the files to number prefixes.
This PR includes the Deployment section of the documentation restructure #144.
- Changes include the addition of 3 overarching folders/sections named Deployment, Using fleet, and Contribution guide. Each folder includes new files for the subsections.
- Move the copy from the appropriate files into the new files in the Deployment folder. Removed old files. A detailed outline of the exact old files can be found in the linked issue above.
- Add navigation to the top of each file via links to anchors in the markdown.
The naming convention for each folder and file is up for discussion. I chose to use number prefixes (1, 2, 3) to order the folders and files. This way we have control over the order when viewing on GitHub.
- Add help text within dropdown in smaller font size underneath "Require password reset" saying "This will revoke all active Fleet API tokens for this user."
- Update API docs to use "API token" parlance instead of "Auth token"
This adds the option to set up an S3 bucket as the storage backend for file carving (partially solving #111).
It works by using the multipart upload capabilities of S3 to maintain compatibility with the "upload in blocks" protocol that osquery uses. It does this basically replacing the carve_blocks table while still maintaining the metadata in the original place (it would probably be possible to rely completely on S3 by using object tagging at the cost of listing performance). To make this pluggable, I created a new field in the service struct dedicated to the CarveStore which, if no configuration for S3 is set up will be just a reference to the standard datastore, otherwise it will point to the S3 one (effectively this separation will allow in the future to add more backends).
@ahmedmusaad added documentation for the following API endpoints:
/api/v1/kolide/users GET
/api/v1/kolide/users POST
/api/v1/kolide/users/admin POST
/api/v1/kolide/users/{id} GET
Endpoints that have been added:
- /api/v1/kolide/login POST
- /api/v1/kolide/logout POST
- /api/v1/kolide/forgot_password POST
- /api/v1/kolide/me GET
- /api/v1/kolide/change_password POST
- /api/v1/kolide/perform_required_password_reset POST
- /api/v1/kolide/sso POST
- /api/v1/kolide/sso GET
Added walkthrough for users attempting to connect to MailHog server.
Connecting to the simulated mail server allows contributors to interact with features in Fleet UI that require email configuration.
Added link to development-infrastructure docs at the end of building-the-code docs to walk contributors to the next step of serving Fleet locally.
Added --auth_jwt_key="insecure" flag to fleet serve command in development-infrastructure docs.
$ character is now removed from all shell commands in markdown documentation.
When docs are eventually compiling to we can add $ automatically to code blocks.
My best attempt at steps to solve the common database connection error users receive when installing or updating Fleet.
Inserted 'Why am I receiving a database connection error when attempting to "prepare" the database?'.
Also simplified the answer to "Is Fleet available as a SaaS product?".
- Add endpoints for osquery to register and continue a carve.
- Implement client functionality for retrieving carve details and contents in fleetctl.
- Add documentation on using file carving with Fleet.
Addresses kolide/fleet#1714
* Perform migration to delete any entries with `deleted` set, and
subsequently drop columns `deleted` and `deleted_at`.
* Remove `deleted` and `deleted_at` references.
Closes#2146
The example in the README won't follow the github redirect without `-L`, so the example as is will download an html file instead of the actual `fleet.zip`. This fixes that.
"Manual" labels can be specified by hostname, allowing users to specify
the membership of a label without having to use a dynamic query. See the
included documentation.
Additional information is collected when host details are updated using
the queries specified in the Fleet configuration. This additional
information is then available in the host API responses.
- Add the server_url_prefix flag for configuring this functionality
- Add prefix handling to the server routes
- Refactor JS to use appropriate paths from modules
- Use JS template to get URL prefix into JS environment
- Update webpack config to support prefixing
Thanks to securityonion.net for sponsoring the development of this feature.
Closes#1661
Adds Google Cloud PubSub logging for status and results.
This also changes the Write interface for logging modules to add a context.Context (only used by pubsub currently).
- Refactor configuration for logging to use separate plugins
- Move existing filesystem logging to filesystem plugin
- Create new AWS firehose plugin
- Update documentation around logging
Almost two years ago, we began referring to the project as Fleet, but there are
many occurences of the term "Kolide" throughout the UI and documentation. This
PR attempts to clear up those uses where it is easily achievable.
The term "Kolide" is used throughout the code as well, but modifying this would
be more likely to introduce bugs.
Useful for SAML login users who cannot log in with `fleetctl login`. Instead
they can pull their session token from the UI and configure the fleetctl client
to use it.
Closes#1865
Individuals unaccustomed to building Go projects from source may encounter difficulties cloning into the repo if they choose an arbitrary directory not in ~/go/src. Attempting to build the repo elsewhere will result in the `make deps` command failing. This change adds workspace prep instructions.
-Updated cp to reflect addition of fleetctl to archive
-Updated MySQL setup to reflect setting root password when installed from .rpm and taking into account password validation requirement defaults.
-Updated echo command with sudo tee due to permission denied
- Updat binary names to reflect binaries in latest archive
- Add line to 'sudo cp fleet/linux/fleetctl /usr/bin/fleetctl
- Change echo string for enroll_secret to use sudo tee due to permission denied
- Revert erroneous changes in 6442736c.
- Remove Osquery prefix from entity kinds.
- Define Query first in combination with Label and Decorator definitions.
Add a gRPC server that will interact with osquery through Launcher. This endpoint will expose the osquery configuration suitable for use via the Launcher plugin, and collect log and query results.
- Add SSH configuration to allow checkout of Kolide private repos in CI
- Add kolide/agent-api repo to glide.yaml
- Update testify version to fix broken test build
Closes#1545
Closes issue #1456 This PR adds a single sign on option to the login form, exposes single sign on to the end user, and allows an admin user to set single sign on configuration options.
This PR partially addresses #1456, providing SSO SAML support. The flow of the code is as follows.
A Kolide user attempts to access a protected resource and is directed to log in.
If SSO identity providers (IDP) have been configured by an admin, the user is presented with SSO log in.
The user selects SSO, which invokes a call the InitiateSSO passing the URL of the protected resource that the user was originally trying access. Kolide server loads the IDP metadata and caches it along with the URL. We then build an auth request URL for the IDP which is returned to the front end.
The IDP calls the server, invoking CallbackSSO with the auth response.
We extract the original request id from the response and use it to fetch the cached metadata and the URL. We check the signature of the response, and validate the timestamps. If everything passes we get the user id from the IDP response and use it to create a login session. We then build a page which executes some javascript that will write the token to web local storage, and redirect to the original URL.
I've created a test web page in tools/app/authtest.html that can be used to test and debug new IDP's which also illustrates how a front end would interact with the IDP and the server. This page can be loaded by starting Kolide with the environment variable KOLIDE_TEST_PAGE_PATH to the full path of the page and then accessed at https://localhost:8080/test
This PR contains a fix for a bug that turned up when I was testing configimporter. If the platform field is not specified, its supposed to default to all per the osquery configuration spec. The default was not properly implemented, and if the platform value was missing from the imported configuration it failed. The PR also added instructions to the api documentation describing how to import an osquery configuration.
If server is started without a JWT key, a message like the following is printed:
```
################################################################################
# ERROR:
# A value must be supplied for --auth_jwt_key. This value is used to create
# session tokens for users.
#
# Consider using the following randomly generated key:
# om3w95gMA2drT5xAdLd2Q5oE8fLw+Miz
################################################################################
```
Closes#1480.
Made log rotation for osquery results and status logs optional. This required writing the logwriter package which is a drop in replacement for lumberjack. We still use lumberjack if the log rotation flag --osquery_enable_log_rotation flag is set. Note that the performance of the default is quite a bit better than lumberjack.
BenchmarkLogger-8 2000000 747 ns/op
BenchmarkLumberjack-8 1000000 1965 ns/op
PASS
BenchmarkLogger-8 2000000 731 ns/op
BenchmarkLumberjack-8 1000000 2040 ns/op
PASS
BenchmarkLogger-8 2000000 741 ns/op
BenchmarkLumberjack-8 1000000 1970 ns/op
PASS
BenchmarkLogger-8 2000000 737 ns/op
BenchmarkLumberjack-8 1000000 1930 ns/op
PASS
When `kolide serve --debug` is used, additional handlers will be started to
provide access to profiling tools. These endpoints are authenticated with a
randomly generated token that is printed to the Kolide logs at startup. The
profiling tools are not intended for general use, but they may be useful when
providing performance-related bug reports to the Kolide developers.
Use the [SockJS Protocol](https://github.com/sockjs/sockjs-protocol) to handle
bidirectional communication instead of plain websockets. This allows
distributed queries to function in situations in which they previously failed
(Load balancers not supporting websockets, issues with Safari and self-signed
certs, etc.).
Also includes fixes to the JS message handling logic where slightly different
message delivery semantics (when using XHR) were exposing bugs.
Fixes#1241, #1327.
I think these were useful when none of us knew how to use glide except
@groob and needed copy-pasteable commands to run, but this seems like
it's much less useful now.
Adds a `make lint-license` command that will crawl through the Golang and
Javascript dependencies, trying to automatically determine the license of each
dependency. If any dependencies have incompatible licensing (or if the
automatic detection fails), the tool will print an error and exit with a
non-zero status code. After a successful run, the
docs/third-party/dependencies.md file will be updated with attribution
information as determined by the crawl.
The configuration file has been bootstrapped with the manual analysis needed for
all of the existing dependencies.
* add graceful server shutdown
Also refactors server to use `http.Server` instead of the global
http.ListenAndServe method. In the future it allows us to set
ratelimits and timeouts for http connections.
