Elgato_dark/fleet
1
0
Fork 0
mirror of https://github.com/fleetdm/fleet synced 2026-05-24 09:28:54 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 30
23998 commits 3355 branches 577 tags 2.4 GiB
Commit graph

3 commits

Author SHA1 Message Date
Dante Catalfamo
ecc7d2ce02
Add macOS 26 Tahoe CIS benchmark v1.0.0 (#44090) 
**Related issue:** Resolves #35173

# macOS 26 Tahoe CIS benchmark v1.0.0 (new benchmark)

Adds a brand-new policy set covering the **CIS Apple macOS 26 Tahoe
Benchmark, v1.0.0** under `ee/cis/macos-26/`. Follows the same layout as
`macos-13`/`-14`/`-15` (`cis-policy-queries.yml`, `README.md`,
`test/scripts/`, `test/profiles/`).

## Coverage

| Section | Title | Status |
|---|---|---|
| 1 | Install Updates, Patches and Additional Security Software |
complete (6/6 automated) |
| 2 | System Settings | complete (all automated across §2.1–§2.18) |
| 3 | Logging and Auditing | complete (5/5 automated) |
| 4 | Network Configurations | complete (3/3 automated) |
| 5 | System Access, Authentication and Authorization | complete (19/19
automated) |
| 6 | Applications | complete (7/7 automated) |
| 7 | Supplemental | skipped (per Fleet convention) |

Total automated policies shipped: **89**. Manual-assessment
recommendations are documented in `ee/cis/macos-26/README.md` under
**Limitations**.

## Notable query/format choices

- **Combined-key profiles per CIS instructions.** §2.2.1+§2.2.2
(Firewall + Stealth Mode) are shipped as a single
`2.2.1-and-2.2.2.mobileconfig` because CIS explicitly requires both keys
in the same profile. §2.6.5 (Gatekeeper) and §2.11.2 (screensaver
wake-password + delay) follow the same pattern.
- **§2.5.2.1 (Siri)** uses the new `allowAssistant=false` key on
`com.apple.applicationaccess`, replacing the deprecated
`com.apple.ironwood.support` payload from earlier benchmarks.
- **§2.6.3.2** uses the spaced literal key `Siri Data Sharing Opt-In
Status` (integer 2) on `com.apple.assistant.support` — the v1.0.0
PayloadType move from `com.apple.applicationaccess`.
- **§5.1.6, §5.1.7, §3.1, §5.7** use fleetd-only osquery tables
(`find_cmd`, `authdb`, `pwd_policy`, `dscl`, etc.) and are flagged
`(Fleetd Required)` in the policy descriptions.
- **§2.10.1.2** (Apple Silicon sleep ≤15 min) default-passes on Intel
hosts via a `system_info.cpu_type` check.

## Test artifacts added

| Type | Count | Location |
|---|---|---|
| Pass scripts | 48 | `ee/cis/macos-26/test/scripts/CIS_*_pass.sh` |
| Fail scripts | 46 | `ee/cis/macos-26/test/scripts/CIS_*_fail.sh` |
| Pass-only scripts | 2 | `CIS_1.1.sh`, `CIS_5.1.6.sh` |
| MDM profiles | 37 | `ee/cis/macos-26/test/profiles/*.mobileconfig` |

Profile-only recommendations (§2.3.1.x AirDrop/AirPlay, §2.5.x Apple
Intelligence, §2.6.3.x Analytics, §6.x Safari/Terminal) ship with a
`.mobileconfig` only and no script counterpart, since CIS marks them as
configurable solely via profile.

## Documentation updates

| File | Change |
|---|---|
| `ee/cis/macos-26/README.md` | New file — coverage table, limitations,
per-section notes (query patterns, fleetd dependencies, FDA
requirements). |
| `ee/cis/CIS-BENCHMARKS.md` | Added `macos-26/` to the directory
layout; updated **Query patterns** doc to include the `EXISTS`/`NOT
EXISTS` user-vs-system-scope guidance and `username = ''` notes. |
| `ee/cis/prompt.md` | Refreshed authoring prompts with macOS-26
conventions (combined-key profiles, fleetd-table flagging). |
| `tools/cis/cis-test-runner.py` | Minor adjustments to support the new
benchmark directory. |
| `changes/35173-cis-macos-26-v1` | User-visible change note. |


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added macOS 26 CIS Benchmark v1.0.0 with comprehensive configuration
profiles to enforce recommended system and app settings (updates,
firewall/stealth, privacy, backups, FileVault, Safari, Terminal, etc.).

* **Tests**
* Added extensive pass/fail remediation and validation scripts for CIS
controls across macOS subsystems; test runner updated to include macOS
26 support and mark an SSH-related control as manual.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-04-29 17:15:25 -04:00
Dante Catalfamo
9ff63eb52e
macOS 14 CIS benchmark v3.0.0 update (#43797) 
**Related issue:** Resolves #35172

Updates the macOS 14 (Sonoma) CIS policy set to benchmark v3.0.0, adds a
`cis_id` field to every policy, fixes several broken test scripts,
introduces an automated test runner, and ships `CIS-BENCHMARKS.md` as a
central guide for authoring and maintaining CIS benchmarks.

## Summary of changes

- `ee/cis/macos-14/cis-policy-queries.yml`: v3.0.0 updates + `cis_id`
added to every entry
- `ee/cis/CIS-BENCHMARKS.md`: new authoring/testing/automation guide for
all macOS CIS benchmarks (and the pattern other OS dirs follow)
- `tools/cis/cis-test-runner.py`: new 2150-line Python runner that
drives end-to-end validation against a real tart VM + Fleet server
- `ee/cis/macos-14/test/scripts/`: 10 new pass/fail script pairs, 8
existing scripts fixed (several were silently broken)

# How the automated testing works

The runner (`tools/cis/cis-test-runner.py`) exercises the full policy
lifecycle against a real macOS VM:

**Phases**

1. **Parse** `cis-policy-queries.yml` and filter by `--all`,
`--cis-ids`, `--match`, and type flags (`--only-scripts`, `--only-mdm`,
`--only-manual`).
2. **Classify** each policy into a test type based on available
artifacts:

   | Priority | Type | Artifacts | Behavior |
   |---|---|---|---|
| 1 | `PASS_FAIL` | `CIS_{id}_pass.sh` + `_fail.sh` | Run fail → verify
query fails → run pass → verify passes |
   | 2 | `PASS_ONLY` | `CIS_{id}.sh` | Run script → verify passes |
| 3 | `PROFILE` | `.mobileconfig` only | Verify query fails before
profile → push profile → verify passes |
| 4 | `ORG_DECISION` | paired `-enable`/`-disable` profiles | Toggle
between variants |
| 5 | `MANUAL` | none | Prompt operator, or skip with `--skip-manual` |

3. **Provision**: create a fresh Fleet team with a unique enroll secret,
build a fleetd pkg bound to it, create+boot a tart VM, install the
agent, and enroll.
4. **MDM**: prompt the operator for MDM enrollment if any tests need it.
Clear team profiles, baseline the VM, push all required profiles in one
batch, wait for delivery.
5. **Execute**: for each plan, SCP the script, run it over SSH, then run
the policy via `fleetctl query --hosts <hostname>`. A query that returns
rows = pass.
6. **Report** summary with PASS/FAIL/SKIP/ERROR counts.
7. **Cleanup** (with `--cleanup`) deletes the team, host record, and VM.

**Special-case handling** (keyed by OS version because CIS IDs aren't
stable across releases):

- `SSH_BREAKING_CIS_IDS`: tests that disable sshd (2.3.3.4, 2.3.3.5) are
forced to `MANUAL` so the runner doesn't lock itself out.
- `PASSWORD_POLICY_CIS_IDS`: 5.2.x profiles invalidate the VM's
`admin`/`admin` login — forced to `MANUAL`.
- `NON_AUTOMATABLE_CIS_IDS`: tests that can't run reliably in a VM
(Location Services, Touch ID, shared Siri profile state) forced to
`MANUAL` with a per-entry reason.
- `--keep-vm`: reuses the VM across runs, skipping agent
install/enrollment if the host is already in Fleet. Falls back to fresh
creation if SSH is unreachable.

**Credential resolution order**: CLI flag →
`FLEET_URL`/`FLEET_API_TOKEN` env → `~/.fleet/config` (from `fleetctl
login`).

## How to use `CIS-BENCHMARKS.md` going forward

The doc is the single reference for authoring and maintaining CIS
benchmark policies across all macOS (and Windows) versions. For each new
benchmark release, the workflow is:

1. **Read "Updating benchmarks when a new CIS version is released"** —
directs you to the PDF's *Appendix: Change History* to enumerate
Added/Modified/Removed recommendations.
2. **Use the field reference and query patterns** to write or update
policies: direct table check, `managed_policies` EXISTS/NOT EXISTS, or
plist negation check. Name qualifiers `(MDM Required)` / `(Fleetd
Required)` / `(FDA Required)` are documented.
3. **Create matching test artifacts** — pass/fail scripts for togglable
settings, `.mobileconfig` profiles for MDM-only settings. Script
conventions (full paths, sudo pattern, `not_always_working_` prefix) are
standardized.
4. **Update the per-OS README** with limitations, org-decision policies,
and optional policies.
5. **Run the test runner** to validate.

The doc also contains an **end-to-end AI agent prompt** (section at the
bottom) designed to be handed a new CIS PDF plus the previous version's
PDF, to automatically generate the diff, write policies, produce test
artifacts, update docs, and run validation. This lets future benchmark
updates start from a consistent, repeatable baseline rather than being
hand-authored from scratch.

## Query changes

All entries in `cis-policy-queries.yml` received a `cis_id` field so the
runner (and humans) can map policies → scripts → profiles → the
benchmark document without parsing the display name.

| CIS ID | Change |
|---|---|
| 1.1 | Renamed "Ensure All Apple-provided Software Is Current" →
"Ensure Apple-provided Software Updates Are Installed"; added terminal
remediation to `resolution` |
| 1.6 | Expanded description with v3.0.0 language about rapid security
response updates |
| 1.x (deferment) | **Removed** — "Ensure Software Update Deferment Is
Less Than or Equal to 30 Days" dropped from v3.0.0 |
| 2.3.1.1 | Renamed "Ensure AirDrop Is Disabled" → "Ensure AirDrop Is
Disabled When Not Actively Transferring Files"; expanded description |
| 2.3.3.1 (DVD/CD Sharing) | **Removed** — dropped from v3.0.0 |
| 2.3.3.4 (Remote Login) | Query now checks BOTH `disabled.plist` AND
that `com.openssh.sshd` is not in the `launchd` table; resolution
updated to terminal method |
| 5.1.7 | Query rewritten: sticky-bit dirs now properly excluded
(first-char-of-mode check instead of bit-AND on full mode string);
SIP-protected dirs excluded via `com.apple.rootless` xattr check |
| 6.3.1, 6.3.2, 6.3.3, 6.3.4, 6.3.7 | Dropped `username = ''` filter —
Safari profiles deliver at user scope, so the system-scope filter
guaranteed zero rows |
| 6.3.3 | Fixed `NOT EXISTS` domain typo: `com.apple.loginwindow` →
`com.apple.Safari` (the check was previously meaningless) |
| Wi-Fi/Bluetooth menu bar | **Removed** — "Show Wi-Fi status in Menu
Bar" and "Show Bluetooth status in Menu Bar" dropped from v3.0.0 |
| Show All Filename Extensions | **Removed** — dropped from v3.0.0 |

## Script changes

### New scripts

| Script | Purpose |
|---|---|
| `CIS_1.1_pass.sh` / `_fail.sh` | Install updates (pass); clear
`LastFullSuccessfulDate` (fail — caveat: only works when real updates
are pending) |
| `CIS_1.6_pass.sh` / `_fail.sh` | Open/remove the `1.6.mobileconfig`
profile |
| `CIS_2.3.1.1_pass.sh` / `_fail.sh` | Open/remove the AirDrop profile |
| `CIS_2.3.2.2_pass.sh` / `_fail.sh` | `launchctl load/unload -w` of
`com.apple.timed.plist` |
| `CIS_2.3.3.4_pass.sh` / `_fail.sh` | `systemsetup -setremotelogin
off/on` (runs as MANUAL via the runner's SSH-breaking safeguard) |

### Existing scripts fixed

| Script | Bug | Fix |
|---|---|---|
| `CIS_2.3.3.1.sh` | Disabled `com.apple.ODSAgent` (DVD/CD sharing), not
Screen Sharing | Now disables `com.apple.screensharing` |
| `CIS_2.9.2.sh` | `pmset -a womp 0` sets Wake-on-Network, not Power Nap
| Now `pmset -a/-b/-c powernap 0` |
| `CIS_3.2.sh` | `sed` pipeline into root-owned file via user
redirection silently failed; did nothing if the flags line was missing |
`awk` with `tee`/`mv`; appends a flags line when absent; enforces 0400
root:wheel |
| `CIS_3.3.sh` | Only stripped `all_max=`; never added `ttl=365` when
missing, so the query could never pass from a fresh system | `awk` now
both strips `all_max=` and inserts/updates `ttl=365` on the `* file`
line |
| `CIS_3.4.sh` | `sudo sed … > /etc/security/audit_control` — redirect
runs as caller, not root; write silently failed | Rewrites via
`tee`/`mv` with proper perms; appends when line is absent |
| `CIS_3.5.sh` | `chmod -R o-rw` doesn't produce the exact `0400`/`0440`
modes the query requires | Explicit `chmod 0400` on `audit_control`,
`find … -exec chmod 0440 {}` under `/var/audit/` |
| `CIS_5.1.7.sh` | `sudo IFS=$'\n'` runs IFS in a subshell that exits
immediately; searched `/System/Volumes/Data/Library` but the query looks
at `/Library/%` | IFS set in parent shell; searches `/Library`; skips
SIP-protected dirs via xattr |
| `CIS_5.7.sh` | Wrote `use-login-window-ui` which the query doesn't
accept | Writes `authenticate-session-owner` |
| `CIS_6.3.6.sh` | Contained literal `<username>` placeholders that were
never substituted | Iterates non-system users (`UniqueID >= 500`) and
runs `defaults write` as each |


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* macOS 14 (Sonoma) CIS policies updated to v3.0.0 with refreshed policy
names and CIS IDs.
  * New CLI test runner to automate CIS validation against macOS VMs.

* **Bug Fixes / Improvements**
* Updated remediations and audit/query logic; safer, atomic config
updates; several policies revised or removed.

* **Tests**
* Many new and improved pass/fail helper scripts for validating CIS
checks and profiles.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-04-23 12:56:38 -04:00
Sharon Katz
d40555e7cd
Script for comparing two CIS PDF files (#15307) 2023-12-06 09:21:12 -05:00