# Agent Test Spec: analytics-engineer

## Agent Summary
- **Domain**: Telemetry architecture and event schema design, A/B test framework design, player behavior analysis methodology, analytics dashboard specification, event naming conventions, data pipeline design (schema → ingestion → dashboard)
- **Does NOT own**: Game implementation of event tracking (appropriate programmer), economy design decisions informed by analytics (economy-designer), live ops event design (live-ops-designer)
- **Model tier**: Sonnet
- **Gate IDs**: None; produces schemas and test designs; defers implementation to programmers

---

## Static Assertions (Structural)

- [ ] `description:` field is present and domain-specific (references telemetry, A/B testing, event tracking, analytics)
- [ ] `allowed-tools:` list matches the agent's role (Read/Write for design/analytics/ and documentation; no game source or CI tools)
- [ ] Model tier is Sonnet (default for operations specialists)
- [ ] Agent definition does not claim authority over game implementation, economy design, or live ops scheduling

---

## Test Cases

### Case 1: In-domain request — tutorial event tracking design
**Input**: "Design the analytics event tracking for our tutorial. We want to know where players drop off and which steps they complete."
**Expected behavior**:
- Produces a structured event schema for each tutorial step: at minimum, `event_name`, `properties` (step_id, step_name, player_id, session_id, timestamp), and `trigger_condition` (when exactly the event fires — on step start, on step complete, on step skip)
- Includes a funnel-completion event and a drop-off event (e.g., `tutorial_step_abandoned` if the player exits during a step)
- Specifies the event naming convention: snake_case, prefixed by domain (e.g., `tutorial_step_started`, `tutorial_step_completed`, `tutorial_abandoned`)
- Does NOT produce implementation code — marks implementation as [TO BE IMPLEMENTED BY PROGRAMMER]
- Output is a schema table or structured list, not a narrative description

### Case 2: Out-of-domain request — implement the event tracking in code
**Input**: "Now that the event schema is designed, write the GDScript code to fire these events in our Godot tutorial scene."
**Expected behavior**:
- Does not produce GDScript or any implementation code
- States clearly: "Telemetry implementation in game code is handled by the appropriate programmer (gameplay-programmer or systems-programmer); I provide the event schema and integration requirements"
- Optionally produces an integration spec: what the programmer needs to know to implement correctly (event name, properties, when to fire, what analytics SDK or endpoint to use)

### Case 3: Domain boundary — A/B test design for a UI change
**Input**: "We want to A/B test two versions of our HUD: the current version and a minimal version with only a health bar. Design the test."
**Expected behavior**:
- Produces a complete A/B test design document:
  - **Hypothesis**: The minimal HUD will increase player engagement (measured by session length) by reducing UI cognitive load
  - **Primary metric**: Average session length per player
  - **Secondary metrics**: Tutorial completion rate, Day 1 retention
  - **Sample size**: Calculated estimate based on expected effect size (or notes that exact calculation requires baseline data) — does NOT skip this field
  - **Duration**: Minimum duration (e.g., "at least 2 weeks to capture weekly player behavior patterns")
  - **Randomization unit**: Player ID (not session ID, to prevent players seeing both versions)
- Output is structured as a formal test design, not a bullet list of ideas

### Case 4: Conflict — overlapping A/B test player segments
**Input**: "We have two A/B tests running simultaneously: Test A (HUD variants) affects all players, and Test B (tutorial variants) also affects all players."
**Expected behavior**:
- Flags the overlap as a mutual exclusion violation: if both tests affect the same player, their results are confounded — neither test produces clean data
- Identifies the problem precisely: players in both tests will have HUD and tutorial variants interacting, making it impossible to attribute outcome differences to either variable alone
- Proposes resolution options: (a) run tests sequentially, (b) split the player population into exclusive segments (50% in Test A, 50% in Test B, 0% in both), or (c) run a factorial design if the interaction effect is also of interest (more complex, requires larger sample)
- Does NOT recommend continuing both tests on overlapping populations

### Case 5: Context pass — new events consistent with existing schema
**Input context**: Existing event schema uses the naming convention: `[domain]_[object]_[action]` in snake_case. Example events: `combat_enemy_killed`, `inventory_item_equipped`, `tutorial_step_completed`.
**Input**: "Design event tracking for our new crafting system: players gather materials, open the crafting menu, and craft items."
**Expected behavior**:
- Produces events following the exact naming convention from the provided schema: `crafting_material_gathered`, `crafting_menu_opened`, `crafting_item_crafted`
- Does NOT invent a different naming pattern (e.g., `gatherMaterial`, `craftingOpened`) even if it might seem natural
- Properties follow the same structure as existing events: `player_id`, `session_id`, `timestamp` as standard fields; domain-specific fields (material_type, item_id, crafting_time_seconds) as additional properties
- Output explicitly references the provided naming convention as the standard being followed

---

## Protocol Compliance

- [ ] Stays within declared domain (event schema design, A/B test design, analytics methodology)
- [ ] Redirects implementation requests to appropriate programmers with an integration spec, not code
- [ ] Produces complete A/B test designs (hypothesis, metric, sample size, duration, randomization unit) — never partial
- [ ] Flags mutual exclusion violations in overlapping A/B tests as data quality blockers
- [ ] Follows provided naming conventions exactly; does not invent alternative conventions

---

## Coverage Notes
- Case 3 (A/B test design completeness) is a quality gate — an incomplete test design wastes experiment budget
- Case 4 (mutual exclusion) is a data integrity test — overlapping tests produce unusable results; this must be caught
- Case 5 is the most important context-awareness test; naming convention drift across schemas causes dashboard breakage
- No automated runner; review manually or via `/skill-test`