Elgato_dark/fleet
1
0
Fork 0
mirror of https://github.com/fleetdm/fleet synced 2026-05-23 17:08:53 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 83

Update handbook for engineering/product process (#5355)

Browse source 
* Update handbook for engineering meetings

* Add development groups, boads, slack channels

* Update language for API versioning

* Add process updates and timeframes

* revert fleetctl change

* Whitespace tweak

* Capitalization

* Update company.md

* Update handbook/product.md

Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>

* Update board name

Co-authored-by: Mike McNeil <mikermcneil@users.noreply.github.com>
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
This commit is contained in:
Zach Wasserman 2022-05-05 09:01:21 -07:00 committed by GitHub
parent 32a093eb92
commit 4a1929ce6f
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
3 changed files with 293 additions and 13 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

126
handbook/company.md
View file

@ -262,6 +262,132 @@ Groups are organized around goals. Connecting people with the same goals helps t
Every group at Fleet maintains specific Slack channels, which all group members join and keep unmuted. Everyone else at Fleet is encouraged to mute these channels, using them only as needed. Each channel has a directly responsible individual responsible for keeping up with all new messages, even if they aren't explicitly mentioned (`@`).
## Development groups
Development groups are organized by their goals, and include members from Design, Engineering, and Product.
Goals:
_progress (+) guarantee_
- **Interface** - more, successfully adopted features faster
- (+) keep UI & API simple, minimalist, consistent, and bug free
- **Platform** - improve productivity of Interface team through patterns and infrastructure for implementing new features, reduce REST API latency, increase max load test size, make upgrading seamless for users, improve accuracy and reliability of data
- (+) maintain quick time-til-merge timeframe for PRs reviewed, maintain clean, empathetic interfaces that allow contributors in other groups to execute quickly and without the need to wait for review or approvals
- **Agent**: grow # open source, osquery-based agents by making Fleet’s agents better, faster, and broader in capabilities
- (+) every table works intuitively with user-friendly docs and empathetic caveats, warnings, and error messages
At Fleet, groups define the relevant sections of the engineering org chart. Each group is represented by a product manager (PM) who reports to the Product department (or a founder serving as an
interim product manager):
- Interface PM: Noah Talerman
- Platform PM: Zach Wasserman
- Agent PM: Mo Zhu
Each group is associated with an engineering manager (EM), who, with their group of engineers, form the engineering members of the group.
Each group's PM works very closely with engineers within their group:
- The PM **prioritizes** work and defines **what** to iteratively build and release next, within their group's domain, to best serve the group's goals and the goals of the company as a whole. The PM communicates **why** this work is prioritized and works with engineering to come up with the best possible **how**.
- The EM (along with engineers) defines **how** to implement that definition within the surface area of code, processes, and rituals owned by their group, while serving their group’s goals and the goals of the company as a whole.
### Interface group
#### Responsibilities
- Everything related to Fleet's graphical user interface (besides for the desktop application portion of Fleet Desktop)
- `fleetctl` (the Fleet command-line interface) and the associated yaml documents (almost everything in fleetctl besides the `fleetctl debug` subcommands)
- The REST API that serves these
- The UX/developer experience, flow, steps, and associated UI and API interfaces for how integrations that require any user interaction or configuration (e.g. GeoIP, Zendesk, Jira), including which 3rd party integrations are supported and which API styles and versions are chosen
- End to end testing of the application (eg. Cypress)
- The REST API documentation
- Future officially-supported wrapper SDKs, such as the Postman collection, or e.g. a Python SDK
- Fleet's configuration surface, including
- The config settings that exist for Fleet deployments, and how they're configured
- How feature flags are used
- Their default values, supported data types, and error messages
- Associated documentation on fleetdm.com
- The UX of upgrading and downgrading and sidegrading Fleet tiers, and managing license keys
#### Consumers
- A human using Fleet's graphical user interface
- A human who is writing code that integrates Fleet's REST API
- A human reading Fleet's REST API docs
- A human using fleetctl, Fleet's Postman collection, or Fleet's other future SDK wrappers
These humans might be working within the "Interface" group itself, insofar as they consume the Fleet REST API. Or they might be a contributor in the Fleet community. Or one of Fleet's core users or customers, usually in an SRE, IT, or security role in an organization.
#### Goals
- Bring value to Fleet users through delivery of new features and iteration on existing features.
- Increase adoption and stickiness of features.
- Keep the graphical user interface, REST API, fleetctl, and SDKs like Postman reliable, minimal, consistent, and easy to use.
- Promote stability of the API, introducing breaking changes only through the documented [API versioning](https://fleetdm.com/docs/contributing/api-versioning#what-kind-of-versioning-will-we-use-for-the-api) strategy, or at major version releases.
- Ensure observance of semantic versioning for the Fleet API and config between releases, so that only major versions include breaking changes.
- Delight users of Fleet's API, UI, SDKs, and documentation with a simple, secure, widely-adopted user and developer experience.
- **Improve Fleet’s feature-value and ease of use.**
### Platform group
#### Responsibilities
- The implementation of Fleet Agent API: i.e.
- The API used by agents on enrolled hosts to communicate with Fleet
- The API used by agents for doing autoupdates and future installation/upgrade of custom extensions (eg. TUF)
- Everything that is related to providing a stable, simple-to-build-on platform for Fleet contributors to use when implementing changes to the REST API
- APIs for storing, retrieving, and modifying device data
- APIs for running asynchronous and scheduled tasks
- APIs for communicating with external services (eg. HTTP, SMTP)
- The challenges of scale
- Sometimes taking over development/improvement of features from the “Interface” group when these features have unexpected backend complexity or scaling challenges.
- Production infrastructure, including
- Fleet Cloud demo
- Fleet Cloud prod
- The registry (TUF) used for autoupdates and (in the future) extensions
- whatever backend is needed to generate installers in self-managed and hosted Fleet deployments
- The future monitoring and 24/7/365 enhancement to the on-call rotation necessary for Fleet Cloud
- behind-the-scenes integrations
- i.e. integrations that make Fleet "just work" and don't involve configuration from users
- example: the code that fetches and manages CVE data from NVD, and other behind-the-scenes infrastructure that enables vulnerability management to exist in Fleet without requiring any interactions or configuration from users
- The CI/CD pipeline
- `loadtest.fleetdm.com`
- `dogfood.fleetdm.com`
#### Consumers
- A contributor from inside or outside the company.
- A host enrolled in Fleet running an osquery-based agent.
- The person who deploys, upgrades, and operates Fleet.
- A person who uses Fleet and expects it to be fast, reliable, and joyful.
#### Goals
- Reduce REST API latency
- Increase max load test size
- Reduce infrastructure costs for Fleet deployments.
- Make upgrading Fleet versions seamless for users.
- Improve integrity of data (both collected directly from agents, or derived from that data eg. vulnerabilities).
- Maintain quick time-til-merge timeframe for PRs reviewed.
- Maintain clean, empathetic interfaces that allow contributors in other groups (or from outside the company) to execute quickly and without the need to wait for review or approvals
- **Make Fleet as easy as possible to operate and contribute to**
### Agent group
#### Responsibilities
- osquery core, including the plugin interfaces and its config surface.
- Orbit
- Fleet Desktop
- Any future agent-based software built by Fleet.
- Fleet Agent API: the API interface and contributor docs used by osquery-enrolled agents for communicating with Fleet (how to implement the internals is up to the "Platform" group).
- The extensions/tables bundled in Orbit/Fleet Desktop, such as macadmins.
#### Consumers
- An engineer working on a custom solution (usually built in-house) on top of osquery.
- An SRE, IT operations, or DevOps professional using osquery-based agents in their default AMIs or container images, who deploys and manages osquery-based agents on their laptops, production servers/containers, and other corporate infrastructure.
- An end user running an osquery-based agent (Fleet Desktop, orbit, or osquery) on their work laptop, who wants their laptop to be stable, performant, and as private as possible.
- An enterprise app owner (software engineer) running osquery-based agents on her app's servers.
- A contributor working on vanilla osquery in osquery/osquery.
- A contributor working on Fleet Desktop or orbit in fleetdm/fleet.
- Fleet itself, consuming the data generated by osquery and any other agent software.
#### Goals
- Grow mind/market share of open source, osquery-based agents by making Fleet’s agents better, faster, and broader in capabilities
- Every table works intuitively with user-friendly docs and empathetic caveats, warnings, and error messages
- Every table is documented within the Fleet UI and in fleetdm.com/docs, with gotchas, deprecation notices, and the version when the table was added
- **Make Fleet’s agents easy to operate and contribute to**
## History
### 2014: Origins of osquery

120
handbook/engineering.md
View file

@ -1,5 +1,109 @@
# Engineering
## Meetings
### Goals
* Stay in alignment across the whole organization.
* Build teams, not groups of people.
* Provide substantial time for engineers to work on "focused work".
### Principles
* Keep meetings to a minimum. Sometimes that will be very very few meetings, and sometimes the minimum will be quite a few of them. But always try to reduce meetings, just like we do with process.
* Each individual must have a weekly sync 1:1 meeting with their manager. This is key to making sure each individual has a voice within the organization.
* Each team should have a fixed weekly sync check in. This helps reinforce team bonds and alignment.
* Favor async communication when possible. This is very important to make sure every stakeholder on a project can have a clear understanding of what’s happening, or what was decided, without needing to attend every meeting (i.e. if a person is sick or on vacation or just life happened.)
* If an async conversation is not proving to be effective, never hesitate to hop on a call. Always document the decisions made in a ticket, document, or whatever makes sense for the conversation.
The following is the subset of proposed engineering meetings. Each group is free to treat these as a subset of the expected meetings, and add any other meetings as they see fit.
### Eng Together (Weekly ~ 1 hour)
Promote cohesion across groups in the engineering team. Disseminate engineering-wide announcements.
#### Participants
All of engineering
#### Sample agenda
- Announcements
- “Show and tell”
- Each engineer gets 2 minutes to explain (showing, if desired) what they are working on, and why it’s important to the business and/or engineering team.
- Deeper dive
- One or a few engineers go deeper on a topic relevant to all of engineering.
- Social
- Structured and/or unstructured social activities
### Release Retro (Each release ~ 30 minutes)
Gather feedback from all participants in each release. Used to improve communication and processes.
This meeting will likely need to be split in the future as the number of attendees increases.
#### Participants
Members of each group (+ quality)
#### Sample agenda
For each attendee:
- What went well this release cycle?
- What could have gone better this release cycle?
- What should we remember next time?
### Group Weeklies (Weekly ~ 30 minutes - 1 hour)
A chance for deeper, synchronous discussion on topics relevant to that group.
eg. “Interface Weekly” - “Platform Weekly” - “Agent Weekly”
In some groups, this may be split into smaller discussions related to the differing focuses of members within the group.
#### Participants
Members of each group
#### Sample agenda (Platform)
- Announcements
- Anything at risk for the release?
- Bug assignment
- Retries in the datastore
- Platform scale gotchas doc
- MarshalJSON to hide passwords and API tokens. Thoughts?
#### Sample Agenda (Interface)
- What’s good?
- Anything at risk for the release?
- Bug assignment
- Confirm response payload matches spec
- Discuss completion of Redux removal
### Standup (Optional, varies by group)
Provide status reports, discover blockers, and keep the group in sync.
Each group can implement daily (or some other cadence) standups if desired. Ultimately, it’s up to the Engineering Manager to ensure that the team is communicating appropriately to deliver results.
#### Participants
Members of the group
### Engineering Leadership Weekly (Weekly ~ 1 hour)
Engineering leaders discuss topics of importance that week.
#### Participants
CTO + Engineering managers
#### Sample agenda
- Fullstack engineer hiring
- Engineering process discussion
- Review Q2 OKRs
### Product/Eng Weekly (Weekly - 30 minutes)
Engineering and Product sync on priorities for the upcoming release, surface and address any inter-group dependencies.
#### Participants
CTO + Engineering managers + PMs
#### Sample agenda
- Plan for what's going into next release
- Identify inter-group dependencies
- Ensure items are moving through architect/estimation
## Release process
This section outlines the release process at Fleet.
@ -34,11 +138,8 @@ Our primary quality objectives are *customer service* and *defect reduction*. We
- Customer response time.
- The number of bugs resolved per release cycle.
- Stay abreast of what our community wants and the problems they're having.
- Make people feel heard and understood.
- Celebrate contributions.
- Triage bugs, identify community feature requests, community pull requests, and community questions.
### How?
@ -81,9 +182,6 @@ It is often a good idea to let the original poster (OP) close their issue themse
Keep in mind that this can feel jarring to the OP. The effect is worse if issues are closed automatically by a bot (See [balderashy/sails#3423](https://github.com/balderdashy/sails/issues/3423#issuecomment-169751072) and [balderdashy/sails#4057](https://github.com/balderdashy/sails/issues/4057) for examples of this).
To provide another way of tracking status without closing issues altogether, consider using the
green labels that begin with "+". To explore them, type `+` from GitHub's label picker.
### Version support
In order to provide the most accurate and efficient support, Fleet will only target fixes based on the latest released version. Fixes in current versions will not be backported to older releases.
@ -154,6 +252,12 @@ The following rituals are engaged in by the directly responsible individual (DR
| Release ritual | Every three weeks | Go through the process of releasing the next iteration of Fleet. | Zach Wasserman |
## Project boards
[🚀 Release](https://github.com/orgs/fleetdm/projects/40) - The current release (daily go-to board) for engineers.
[⚗️ Roadmap](https://github.com/orgs/fleetdm/projects/41) - Planning for the next release (shared with product).
## Slack channels
The following [Slack channels are maintained](https://fleetdm.com/handbook/company#group-slack-channels) by this group:
@ -161,7 +265,9 @@ The following [Slack channels are maintained](https://fleetdm.com/handbook/compa
| Slack channel | [DRI](https://fleetdm.com/handbook/company#group-slack-channels) |
|:------------------------------------|:--------------------------------------------------------------------|
| `#help-engineering` | Zach Wasserman
| `#help-oncall` | Zach Wasserman
| `#g-platform` | Tomás Touceda
| `#g-interface` | Luke Heath
| `#g-agent` | Zach Wasserman
| `#_pov-environments` | Ben Edwards

60
handbook/product.md
View file

@ -152,19 +152,67 @@ What happens during priority drafting?
3. UI changes are reviewed, and the UI changes are brought back to the engineering team to continue
the development task.
## Planning
- The intake process for a given group (how new issues are received from a given requestor and estimated within the group's timeframe) is up to each group's PM. For example, the Interface group's intake process consists of attending Interface PM's office hours and making a case, at which time a decision about whether to draft and estimate will be made on the spot.
- New unestimated issues are created in the Planning board, which is shared by each group.
- The estimation process to use is up to the EM of each group (with buy-in from the PM), with the goal of delivering estimated issues within the group's timeframe, which is set for each group by the Head of Product. No matter the group, only work that is slated to be released into the hands of users within ≤6 weeks will be estimated. Estimation is run by each group's EM, and occurs in the Planning board. Some groups may choose to use "timeboxes" rather than estimates.
- Prioritization will now occur at the point of intake, by the PM of the group. Besides the 20% "engineering initiatives", only issues prioritized by the group PM will be estimated or worked on. On the first day of each release, all estimated issues are moved into the relevant section of the new "Release" board, which has a kanban view per-group.
- Work that does not "fit" into the scheduled release (due to lack of capacity or otherwise) remains in the "Estimated" column of the product board, and is removed from that board if it is not prioritized in the following release.
### Process
1. **Intake:** Each group has a "time til estimated" timeframe, which measures the time from when an idea is first received until it is written up as an estimated issue and the requestor notified exactly which aspects are scheduled for release. How intake works, and the estimation timeframe, vary per group, but every group has an estimation timeframe.
2. **Estimation:** The estimation process varies per-group. In the Interface group, it consists of drafting, API design, and either planning poker or a quick timebox decided by the group EM. When the Interface group relies on the Platform group for part of an issue, only the Interface group's work is estimated. It is up to the Interface PM to obtain estimated Platform issues for any needed work, and thus ensure it is scheduled in the appropriate release. It is up to the Platform PM to get those specced (in consultation with Engineering), then up to the Engineering to estimate and communicate promptly if issues arise. We avoid having more estimated issues than capacity in the next release. If the team is fully allocated, no more issues will be estimated, or the PM will decide whether to swap anything out. Once estimated, an issue is scheduled for release.
3. **Development:** Development starts on the first day of the new release. Only estimated issues are scheduled for release.
4. **Quality assurance (QA):** Everyone in each group is responsible for quality: engineers, PM, and the EM. The QA process varies per group, and is set by the group's PM. For example, in the Interface group, every issue is QA'd (i.e. a per-change basis), as well as a holistic "smoke test" during the last few days of each release.
5. **Release:** Release dates are time-based and happen even if all features are not complete (± a day or two sometimes, if there's an emergency. Either way, the next release cycle starts on time). If anything is not finished, or can only be finished with changes, the PM finds out immediately, and notifies the requestor right away.
### Timeframes
These are effectively internal SLAs. We moved away from the term "SLA" to avoid potential confusion with future, contractual Service Level Agreements Fleet might sign with its customers.
#### Prioritization
≤5 business days from when the initial request is weighed by PM, requestor has heard back from the group PM whether the request will be prioritized.
#### Release
≤6 weeks from when initial request is weighed by PM, this is released into the hands of the Fleet community, generally available (no feature flags or limitations except as originally specced or as adjusted if necessary).
Work that is prioritized by the group PM should be released in the 6 week timeframe (2 releases). Work that is too large for this timeframe should be split up.
#### Estimation
≤5 business days from initial request, an issue is created with a summary of the purpose, the goal, and the plan to achieve it. The level of detail in that plan is up to the PM of the product group. The issue also has an estimation, expressed in story points, which is either determined through planning poker or a "timebox".
For the Interface group "estimated" means UI wireframes and API design are completed and the work to implement them has been estimated.
#### Adjustment
≤1 business day from discovering some blocker or change necessary to already prioritized and estimated work. The group PM decides how the usage/UI will be changed and notifies original requestor of changes to spec.
## Product quality
Fleet uses a human-oriented quality assurance (QA) process to ensure the product meets the standards of users and organizations.
To try Fleet locally for QA purposes, run `fleetctl preview`, which defaults to running the latest stable release.
To target a different version of Fleet, use the `--tag` flag to target any tag in [Docker Hub](https://hub.docker.com/r/fleetdm/fleet/tags?page=1&ordering=last_updated), including any git commit hash or branch name. For example, to QA the latest code on the `main` branch of fleetdm/fleet, you can run: `fleetctl preview --tag=main`
To target a different version of Fleet, use the `--tag` flag to target any tag in [Docker Hub](https://hub.docker.com/r/fleetdm/fleet/tags?page=1&ordering=last_updated), including any git commit hash or branch name. For example, to QA the latest code on the `main` branch of fleetdm/fleet, you can run: `fleetctl preview --tag=main`
To start preview without starting the simulated hosts, use the `--no-hosts` flag (e.g. `fleetctl preview --no-hosts`).
### Why human-oriented QA?
Automated tests are important, but they can't catch everything. Many issues are hard to notice until a human looks empathetically at the user experience, whether in the user interface, the REST API, or the command line.
Automated tests are important, but they can't catch everything. Many issues are hard to notice until a human looks empathetically at the user experience, whether in the user interface, the REST API, or the command line.
The goal of quality assurance is to catch unexpected behavior before release:
- Bugs
@ -182,7 +230,7 @@ The goal of quality assurance is to catch unexpected behavior before release:
### Collecting bugs
All QA steps should be possible using `fleetctl preview`. Please refer to [docs/Contributing/Testing.md](https://fleetdm.com/docs/contributing/testing) for flows that cannot be completed using `fleetctl preview`.
All QA steps should be possible using `fleetctl preview`. Please refer to [docs/Contributing/Testing.md](https://fleetdm.com/docs/contributing/testing) for flows that cannot be completed using `fleetctl preview`.
Please start the manual QA process by creating a blank GitHub issue. As you complete each of the
flows, record a list of the bugs you encounter in this new issue. Each item in this list should
@ -306,13 +354,13 @@ Logout of your current user and log in with the newly created user.
### Communicating design changes to the engineering team.
Something NEW that has been added to [Figma Fleet EE (current, dev-ready)](https://www.figma.com/file/qpdty1e2n22uZntKUZKEJl/?node-id=0%3A1):
1. Create a new [GitHub issue](https://github.com/fleetdm/fleet/issues/new)
2. Detail the required changes (including page links to the relevant layouts), then assign the issue to the __“Initiatives”__ project.
2. Detail the required changes (including page links to the relevant layouts), then assign the issue to the __"Initiatives"__ project.
<img src="https://user-images.githubusercontent.com/78363703/129840932-67d55b5b-8e0e-4fb9-9300-5d458e1b91e4.png" alt="Assign to Initiatives project"/>
> ___NOTE:___ Artwork and layouts in Figma Fleet EE (current) are final assets, ready for implementation. Therefore, it’s important NOT to use the “idea” label, as designs in this document are more than ideas - they are something that WILL be implemented.
> ___NOTE:___ Artwork and layouts in Figma Fleet EE (current) are final assets, ready for implementation. Therefore, it’s important NOT to use the "idea" label, as designs in this document are more than ideas - they are something that WILL be implemented.
3. Navigate to the [Initiatives project](https://github.com/orgs/fleetdm/projects/8), hit “+ Add cards,” pick the new issue, and drag it into the “🤩Inspire me” column.
3. Navigate to the [Initiatives project](https://github.com/orgs/fleetdm/projects/8), hit "+ Add cards," pick the new issue, and drag it into the "🤩Inspire me" column.
<img src="https://user-images.githubusercontent.com/78363703/129840496-54ea4301-be20-46c2-9138-b70bff7198d0.png" alt="Add cards"/>