For quite some time now, since we started to use Bazel for integration tests, we
relied on some size tracking logic that did not actually fully work under Bazel.
It was thought that all the necessary CI push/PR information is available to the
Bazel test, but that was not the case. This was now fixed with the recent Rules NodeJS
v5 update where I made sure the `env.sh` variables are actually available before we
write them to the temporary file for the Bazel-access.
This now will unveil an issue because payload size goldens would start being based
on their branch name. e.g. the golden key in `13.3.x` should not be `master` but
`13.3.x`. This makes more sense than `master` as key, but makes things more
cumbersome and ideally we would not store the branch name at all (this is a larger
change though -- not worth now since we might refactor this anyway). For now we will
update the size tracking logic to always use `master` as golden key (like it worked
in the past year(s))
With the environment fix we now (again) start uploading payload size results to Firebase.
This did not work by accident either. The uploading logic is reliant on the CircleCI
commit range which is not working/reliable in upstream branches. This commit
removes this reliance on `COMMIT_RANGE` since it's not strictly necessary and
currently breaking renovate PRs. We can re-enable this when we have a solution with
CircleCI, or a workaround/resolution logic provided in e.g. `ng-dev ci determine-commit-range`.
PR Close#45444
Previously, if a commit message contained a carriage return in its
header, the `payload-size.sh` script would fail to upload the payload
size data to Firebase, because the JSON payload would be messed up when
trying to concatenate the commit message headers. See an example
[here][1].
This commit avoids this problem by replacing carriage returns before
concatenating the commit message headers with the JSON payload string.
[1]: https://app.circleci.com/pipelines/github/angular/angular/37437/workflows/d0fa4adf-43bb-464e-a2fd-d87da15226dd
PR Close#43569
* integration tests target definitions in integration/BUILD.bazel updated to use a single dict
* payload tracking for integration tests updated to work under Bazel
* legacy integration_test CI job removed
* integration/run_tests.sh script no longer used in CI so it has been updated for running integration tests locally in the legacy way
PR Close#35985
The change type was only recorded for `aio/` and was not correct anyway.
For example:
- It considered `package.json` changes as `application` (even if only
`package.json` and `yarn.lock` had changed).
- It failed to account for changes in `@angular/*` dependencies, when
using the locally built Angular packages (instead reporting them as
`other`).
- It only looked at the last commit, so it failed to provide accurate
information for multi-commit builds (which are rare, but possible).
For the above reasons (and because there is no straight-forward way of
fixing it), this commit removes the change type from the uploaded data.
If necessary, it is still possible to find the type of changes from the
uploaded info (e.g. extract the associated commits and look at their
changes using git).
PR Close#33987
As part of the `payload-size` npm script in `aio/package.json` (which is
run on CI), the sizes of the angular.io app bundles are checked to
ensure they do not exceed certain limits and are also uploaded to
Firebase to be available for later analysis. The uploaded data include
the type of the changes (dependencies only, application only, or both).
The type of changes is inferred by looking at the files that have
changed inside the `aio/` directory.
When the `payload-size.sh` script was first introduced, the only files
that could affect bundle sizes were inside the `aio/` directory.
Therefore, the script would skip uploading the data and checking the
sizes if no changes were detected inside the `aio/` directory.
However, this assumption stopped being valid over time. For example:
- We started tracking/checking bundle sizes when building the angular.io
app with the locally built Angular packages (which live outside the
`aio/` directory.
- Due to CircleCI limitations, the `CI_COMMIT_RANGE` environment
variable (which is used for determining what files have been affected)
stopped reflecting the whole commit range of the build and only
included the last commit instead.
Based on the above, there were many cases were size data would not be
uploaded to Firebase, even when they may have been affected (because the
affecting changes were outside `aio/` - e.g. in framework packages).
This makes it harder to analyze size regressions, because important
data-points are missing.
Even worse, in these cases, the sizes were not even checked against the
specified limits, thus making it possible for size regressions to go
unnoticed (unless caught by other similar tests).
This commit fixes the `scripts/ci/payload-size.sh` script to always
track and check payload sizes for angular.io bundles.
NOTE: This change will result in more data being recorded (i.e.
recording data when it is not possible for the bundle sizes to
have been affected by the changes). This is still preferable to
failing to record and/or check when sizes could have been
affected.
PR Close#33987
The `payload-size.sh` script is mainly used on CI to calculate, check
and potentially save (on non-PR builds) the sizes of the bundles for
various apps (including angular.io). If everything goes well (i.e. the
checks pass, meaning that the sizes did not increase above the specified
threshold) nothing is shown in the CI logs.
In some cases, it is useful to be able to see what the sizes were in a
specific build; e.g. for debugging purposes or when investigating a
gradual increase that happened over time. (Some of this info is
available on https://size.angular.io/, but not all.)
Previously, the only way to find out what the sizes were for a specific
build was to checkout the corresponding commit locally and build the
target app, which in turn requires building all Angular packages and can
take some time. Given that the sizes are already calculated on CI, this
was a waste.
This commit makes it easy to find out the bundle sizes for a specific
build/commit by always printing out the calculated sizes (thus making
them show up in the CI logs).
PR Close#33099
The commit range that is associated with a CI build is used for a couple
of things (mostly related to payload-size tracking):
- Determine whether a size change was caused by application code or
dependencies (or both).
- Add the messages of the commits associated with the build (and thus
the payload-size change).
NOTE: The commit range is only used on push builds.
Previously, the commit range was computed based on the
`CIRCLE_COMPARE_URL` environment variable. With [CircleCI Pipelines][1]
enabled, `CIRCLE_COMPARE_URL` is no longer available and the commit
range cannot be reliably detected.
This commit switches `CI_COMMIT_RANGE` to only include the last commit.
This can be less accurate in some rare cases, but is true in the
majority of cases (on push builds). Additionally, it stores the CircleCI
build URL in the database along with the payload data, so the relevant
info can be retrieved when needed.
[1]: https://circleci.com/docs/2.0/build-processing
PR Close#32537
Payload size tracking was temporarily disabled in #31057, due to
`CIRCLE_COMPARE_URL` stopping being available. It turned out this was
related to turning on the new [Pipelines][1] feature, which was required
for testing Windows on CircleCI.
Since then, we have turned `Pipelines` off and got `CIRCLE_COMPARE_URL`
back (e.g. see [build 362971][2]). According to CircleCI, failing to
populate `CIRCLE_COMPARE_URL` with `Pipelines` on is a bug and they are
working on fixing it.
[1]: https://circleci.com/docs/2.0/build-processing/
[2]: https://circleci.com/gh/angular/angular/362971Fixes#31121
PR Close#31138
The `CIRCLE_COMPARE_URL` is not available in builds any more since we
enabled `Pipelines` on CircleCI. We have contected CircleCI, but until
this is solved we cannot get the commit range and thus disabling
uploading of payload size data to avoid broken builds.
PR Close#31057
These files are not needed once the size has been calculated and there
is no point in keeping them around.
Deleting them prevents, for example, uploading unnecessary files from
`aio/dist/` to Firebase (because `deploy-to-firebase.sh` runs the
payload size checks right before deploying).
PR Close#26746
The scripts were accidentally broken in #20524. More specifically, when a limit
was exceeded the script would break while trying to log an error message due to
a missing `commit` variable.
This commit fixes it and also does some minor clean-up (improve docs, use more
descriptive variable names, remove dead code, etc).
PR Close#20683
