DataDesigner/docs/devnotes
Andre Manoel 0e90ea644b
docs: add async engine dev note (#490)
* fix: address review feedback on async engine dev note

- Fix wall-clock claim: 41% -> 22% to match benchmark table
- Fix dual-model speedup rounding: 1.7x -> 1.6x (10.0/6.1 = 1.64)
- Fix run_config API: use dd.set_run_config() instead of passing to create()

* docs: add async engine dev note

Add "Async All the Way Down" dev note covering the async task-queue
scheduler built across PRs #356, #378, #404, #429, #456. Includes
benchmark results, architecture diagrams, and DAG shape illustrations.

* feat: add docs preview workflow for PRs

Build MkDocs site on PRs that touch docs and deploy to Cloudflare
Pages. Each PR gets a browseable preview URL posted as a comment.
Notebook tutorials use placeholder stubs since they require API
keys to execute.

Requires CLOUDFLARE_API_TOKEN and CLOUDFLARE_ACCOUNT_ID repo secrets.

* fix: update speedup chart alt text from 1.7x to 1.6x

* docs: improve timeline figure context and labeling

Add DAG subtitle to sync-vs-async timeline figure and bridge the
surrounding text to explain which workload shape is being shown.

* edits+additions to async-all-the-way-down dev notes

* clarify two semaphore dance

* remove dead link

* replace hero image

* docs: update scale figures with nginx-accurate data and adjust sizing

Regenerate scale-model-timeline and scale-boxplot from nginx access
logs (column_progress.csv, sync/summary.json) instead of buffered
execution logs. Optimize both PNGs to palette mode. Adjust figure
widths and update model timeline commentary.

* add link from owning-the-model-stack to async-dev-node

* docs: address review feedback on async blog post

- Tighten intro to a concise abstract, move pipeline narrative into
  "The Bottleneck Was Structural" section
- Remove multi-column generators / seed readers paragraph (TMI)
- Clarify sync engine ran columns sequentially within each batch

---------

Co-authored-by: Nabin Mulepati <nmulepati@nvidia.com>
2026-04-08 15:51:04 -03:00
..
posts docs: add async engine dev note (#490) 2026-04-08 15:51:04 -03:00
.authors.yml docs: add async engine dev note (#490) 2026-04-08 15:51:04 -03:00
index.md Updated url (#325) 2026-02-11 14:43:38 -08:00