Currently when `yarn benchmarks run-compare` is used, the local
repository could be converted to a shallow clone. This is not
a problem but might result in confusion. We avoid this by
not performing a shallow fetch when a comparison is initiated.
The performance gains seem neglegible.
PR Close#50851
We already collect and report benchmark results for comparison runs, but
for normal benchmark runs we should do the same to make the results
more discoverable (and not in the middle of Bazel output).
PR Close#50816
`rev-parse` returns the original ref name in stdout, even if it wasn't
able to resolve the reference. This was preventing the upstream
resolution from never happening- causing `main` to be never resolved
to the actual SHA.
When we fetch e.g. `main`, the branch name will not be available
locally, and `rev-parse` will fail later. We can make the logic
more safe by just using `FETCH_HEAD` then.
PR Close#50767
If we try to resolve the benchmark compare ref (which may be just
`main`), we are just looking inside the PR branch- but that may
not include the `main` SHA. i.e. it's possible to run a comparison
where the PR is slightly behind of the `main` branch, or a comparison
commit from a different branch is used.
We fix this/ and simplify the logic by resolving the SHAs directly in
the TypeScript code, instead of relying on the rather brittle Bash.
Note that current solution still works, but we sometimes may not be able
to resolve to an actual SHA- causing issues as in 864bd72cb2
PR Close#50764
This commit creates a new script that solves the following use-cases:
- Running benchmarks. It's not trivial to figure out the benchmark
target names, and it's also easy to mess up the right Bazel flags.
- Performing comparisons. When e.g. working on a runtime senstive
change, it should be trivial to run benchmarks between the current
working stage, and a base revision (e.g. `main`).
The script takes care of both these use-cases and comes with a
prompt-based command line tool experience.
The script will also be used by a future GitHub action that can run
comparisons triggered via GitHub PR comment (by trusted team members).
PR Close#50745
