mirror of
https://github.com/angular/angular
synced 2026-05-24 09:28:37 +00:00
84 lines
3.9 KiB
Markdown
84 lines
3.9 KiB
Markdown
# Saucelabs testing with Bazel
|
|
|
|
## Local testing
|
|
|
|
1. Set up your `SAUCE_USERNAME`, `SAUCE_ACCESS_KEY` & `SAUCE_TUNNEL_IDENTIFIER` environment variables.
|
|
These are required. You can find the values for `SAUCE_USERNAME` and `SAUCE_ACCESS_KEY` in `/.circleci/env.sh`. `SAUCE_TUNNEL_IDENTIFIER` can be set to any unique value.
|
|
|
|
If you are having trouble running Saucelabs tests locally you can contact [Joey Perrott](https://github.com/josephperrott) or [Paul Gschwendtner](https://github.com/devversion) for support.
|
|
|
|
2. Once you have your environment variables set up, run the setup task in the root of the repo:
|
|
|
|
``` bash
|
|
yarn bazel run //tools/saucelabs-daemon/background-service -- <number_of_browsers>
|
|
```
|
|
|
|
This will start a daemon process that will connect to Saucelabs and provision browsers
|
|
once you start running your first test target.
|
|
|
|
3. In another terminal, you can run a particular test target through SauceLabs by suffixing the target name with "_saucelabs".
|
|
|
|
For example, `packages/core/test:test_web` becomes `packages/core/test:test_web_saucelabs`.
|
|
|
|
```
|
|
yarn bazel test //packages/core/test:test_web_saucelabs
|
|
```
|
|
|
|
## Additional test features
|
|
|
|
To see the test output while the tests are running (as these are long tests), add the `--test_output=streamed` option.
|
|
Note, this option will also prevent bazel from using the test cache and will force the test to run.
|
|
|
|
For running all Saucelabs tests in the project, `bazel query` is used to gather up all karma Saucelabs test labels because they are otherwise hidden by the `manual` tag.
|
|
|
|
Running all karma tests in Saucelabs:
|
|
|
|
``` bash
|
|
./scripts/test/run-saucelabs-tests.sh
|
|
```
|
|
|
|
to override the default number of parallel browsers to acquire on Saucelabs you
|
|
pass an optional configuration parameters. For example,
|
|
|
|
``` bash
|
|
./scripts/test/run-saucelabs-tests.sh 5
|
|
```
|
|
|
|
## Under the hood
|
|
|
|
The `//tools/saucelabs-daemon/background-service` target does not start the Sauce Connect proxy at start-up, but instead listens for the start signal from the saucelabs karma launcher.
|
|
This signal is sent by saucelabs-daemon custom karma launcher `tools/saucelabs-daemon/launcher/launcher.ts`.
|
|
This is necessary as the Sauce Connect Proxy process must be started outside of `bazel test` as Bazel will automatically kill any processes spawned during a test when that tests completes, which would prevent the tunnel & provisioned browsers from being shared by multiple tests.
|
|
|
|
The karma_web_test rule for saucelabs must have a few important tags:
|
|
|
|
* `no-remote-exec` as they cannot be executed remotely since tests need to communicate with the daemon.
|
|
* `manual` so they are not automatically tested with `//...`
|
|
* `saucelabs` so that they can be easily gathered up for testing in a `bazel query`
|
|
|
|
These are added automatically the by `karma_web_test_suite` macro in `tools/defaults.bzl`.
|
|
|
|
## Debugging
|
|
|
|
**Q: How do I get the tests to run on IE? I only see Chromium.**
|
|
|
|
If you see something like this at the end of your test output, it means you're not actually running SauceLabs:
|
|
|
|
```
|
|
INFO: Build completed successfully, 43 total actions
|
|
/packages/core/test:test_web_chromium
|
|
```
|
|
|
|
This is a common error caused by forgetting to suffix your test target with "_saucelabs".
|
|
|
|
For example, `/packages/core/test:test_web` becomes `/packages/core/test:test_web_saucelabs`.
|
|
|
|
**Q: How can I tell that the SauceLabs connection was successfully made?**
|
|
|
|
There is a dashboard at saucelabs.com where you can see active tunnel connections (Angular has an account).
|
|
As soon as you actually run the test target (not after the setup task), you should see an active tunnel connection under the SAUCE_TUNNEL_IDENTIFICATION_KEY you entered.
|
|
If a tunnel connection is not there, you are not actually connecting with SauceLabs.
|
|
|
|
Note: It may *look* like the tests are running because of the Bazel output.
|
|
The progress Bazel is showing does not mean that SauceLabs is connected.
|
|
If the tests are actually running, you should see the "..." test report for passing tests.
|