The repository currently has two globbing packages. To minimize the number of packages in
the framework repository, the uses of the `glob` package are being converted
to `fast-glob` which is used by the tooling repository. The change is mostly mechanical
and in this change the build and test scripts are converted.
PR Close#53397
Currently internally Angular has some customized tsconfig files, because we don't align with the tsconfig of the rest of g3. These changes enable `noImplicitReturns` and `noPropertyAccessFromIndexSignature` to align better with the internal config.
PR Close#51728
Before this commit, building everything to run `@angular/core` tests:
```
INFO: Elapsed time: 76.496s, Critical Path: 72.92s
INFO: 225 processes: 125 internal, 5 linux-sandbox, 2 local, 93 worker.
INFO: Build completed successfully, 225 total actions
```
After:
```
Use --sandbox_debug to see verbose messages from the sandbox
INFO: Elapsed time: 15.952s, Critical Path: 10.75s
INFO: 200 processes: 128 internal, 4 linux-sandbox, 2 local, 66 worker.
```
This being on a specialist Cloudtop.
PR Close#50426
This is basically a pre-step for combining devmode and prodmode into a
single compilation. We are already achieving this now, and can claim
with confidence that we reduced possible actions by half. This is
especially important now that prodmode is used more often, but rules
potentially still using the devmode ESM sources. We can avoid double
compilations (which existed before the whole ESM migration too!).
We will measure this more when we have more concrete documentation
of the changes & a better planning document.
Changes:
* ts_library will no longer generate devmode `d.ts`. Definitions are
generated as part of prodmode. That way only prodmode can be exposed
via providers.
* applied the same to `ng_module`.
* updates migrations to bundle because *everything* using `ts_library`
is now ESM. This is actually also useful in the future if
schematics rely on e.g. the compiler.
* updates schematics for localize to also bundle. similar reason as
above.
PR Close#48521
The `generate-locales-tool` now needs to run as ESM because we changed
the `ts_library` rule. This commit accounts for the ESM syntax but
removing CJS code parts like `require.main === module`.
Also imports using `require` need to be changed to their ESM
equivalents.
PR Close#48521
This commit cleans-up/removes a check we added before we supported CLDR
39. This check was necessary due to a incomplete/invalid list of locales
provided as part of the JSON data.
PR Close#46606
The recent update to CLDR 41 highlighted (internally) that some time
formats have changed to rely on locale-specific day periods. In
particular the `zh_TW` (or canonical: `zh_Hant`) has changed some
time formats/patterns from the universal `AM/PM` symbols (`a`) to `B`.
The `b`/`B` symbols rely on locale-specific day period rules. This data
is only extracted from CLDR into the so-called extra locale data that
Angular provides.
To fix this, and to be able to reverse the internal workaround that
doesn't allow us to use the actual CLDR 41 data here, we always provide
the locale extra data when conditonally loading CLDR data based on
`goog.LOCALE`.
This will result in additional payload cost that locale-specific JS
bundles have to pay, but the only alternative would be to break
Angular's `b`/`B` symbol support in favor of falling back to `AM/PM`
rules (similar to how Closure Libray itself does it).
For our external users (something I want to note here), they will be
able to load the extra data easily (like it was possible before).
Angular has some good erroring if the extra data is required. Most of
our users will always use the global locale files anyway (the Angular
CLI always uses them). These come with the extra data by default.
Resources:
- 0d538327d1
- 0d538327d1/common/main/zh_Hant.xml
- 6a4353cb40/packages/common/src/i18n/format_date.ts (L609-L639)
- https://cldr.unicode.org/translation/date-time/date-time-symbols
- 4be7cdce82/packages/angular_devkit/build_angular/src/utils/i18n-options.ts (L23)
PR Close#46167
The NodeJS Bazel linker does not work well on Windows because there
is no sandboxing and linker processes from different tests will attempt
to modify the same `node_modules`, causing concurrency race conditions
and resulting in flakiness.
PR Close#45872
Update CLDR, which is used to generate the locales files, to version 41.
Also, make necessary code changes to account for changes in the CLDR
data.
Fixes#43301
PR Close#45714
While generating locales, two sets of data (both derived from the
[CLDR project][1]) are used:
- JSON data with the [`cldrjs` package][2], which is used for most
operations.
- XML data with the [`cldr` package][3], which is used for generating
plural-related data.
The JSON data is brought in from the [unicode-org/cldr-json][4]
repository. Since we control the version of the repository that we use,
we can control the CLDR version that these correspond to.
Previously, however, we used the XML data that were bundled with the
[`cldr` package[3]. As a result, the two sets of data could correspond
to different CLDR versions, resulting in incorrect/inconsistent locales
files.
This commit addresses the problem by utilizing the `load()` method of
the [`cldr` package][3], which allows passing in a custom path to the
CLDR XML data (instead of using the bundled data. This way, we can
ensure that the data used for all operations correspond to the same CLDR
version.
Related discussion: #43301
[1]: https://github.com/unicode-org/cldr
[2]: https://www.npmjs.com/package/cldrjs
[3]: https://www.npmjs.com/package/cldr
[4]: https://github.com/unicode-org/cldr-json
PR Close#45714
Previously, some functions/targets related to the
generation/verification of the `closure-locale.ts` file included
`closure_locales` instead of `closure_locale` in their name, which was
inconsistent and confusing.
Rename all tooling to use `closure_locale` (which more closely matches
the generated file's name).
PR Close#45714
Speeds up the dev-turnaround by only bundling types when packaging. Currently
bundling occurs for all the `ng_module` targets in devmode.
This has various positive benefits:
* Avoidance of this rather slower operation in development
* Makes APF-built packages also handle types for `ts_library` targets consistently.
* Allows us to ensure APF entry-points have `d.ts` _always_ bundled (working with ESM
module resolution in TypeScript -- currently experimental)
* Allows us to remove the secondary `package.json` files from APF (maybe APF v14? - seems
low-impact). This would clean-up the APF even more and fix resolution issues (like in Vite)
PR Close#45405
As outlined in the previous commit which enabled the `esModuleInterop`
TypeScript compiler option, we need to update all namespace imports
for `typescript` to default imports. This is needed to allow for
TypeScript to be imported at runtime from an ES module.
Similar changes are needed for modules like `semver` where the types incorrectly
suggest named exports that will not exist at runtime when imported from ESM.
This commit refactors all imports to match with the lint rule we have
configured in the previous commit. See the previous commit for more
details on why certain imports have been changed.
A special case are the imports to `@babel/core` and `@babel/types`. For
these a special interop is needed as both default imports, or named
imports break the other module format. e.g default imports would work
well for ESM, but it breaks for CJS. For CJS, the named imports would
only work, but in ESM, only the default export exist. We work around
this for now until the devmode is using ESM as well (which would be
consistent with prodmode and gives us more valuable test results). More
details on the interop can be found in the `babel_core.ts` files (two
interops are needed for both localize/or the compiler-cli).
PR Close#43431
Similar to other code that is shipped as part of `@angular/common`
(with APF v13), we should ship the generated locale files as ESM
files as well. This is necessary/reasonable because we explicitly
set `type: "module"` for the common package, so it makes sense to
have the same apply for the locale sub-directory.
Note: The global locale scripts remain having the `.js` extension
and will continue to be unmodified. They are CJS/ESM compatible either
way, but refer to browser globals.
PR Close#43431
With the refactoring from a Gulp task to a Bazel too, we tried switching
away from the hard-coded list of locales and aliases for the Closure
Locale file generation. After multiple attempts of landing this, it
turned out that Closure Compiler/Closure Library relies on locale
identifiers CLDR does not capture within it's `availableLocales.json`
or `aliases.json` data.
Closure Library does not use any unknown locale identifiers here. The
locale identifiers can be resolved within CLDR using the bundle lookup
algorithm that is specified as part of CLDR; instead the problem is that
the locale identifiers do not follow any reasonable pattern and
therefore it's extremely difficult to generate them automatically (it's
almost like we'd need to build up _all_ possible combinations). Instead
of doing that, we just use the hard-coded locales and aliases from the
old Closure Locale generation script.
PR Close#42230
Within Google, closure compiler is used for dealing with translations.
We generate a closure-compatible locale file that allows for
registration within Angular, so that Closure i18n works well together
with Angular applications. Closure compiler does not limit its
locales to BCP47-canonical locale identifiers. This commit updates
the generation logic so that we also support deprecated (but aliased)
locale identifiers, or other aliases which are likely used within
Closure. We use CLDR's alias supplemental data for this. It instructs
us to alias `iw` to `he` for example. `iw` is still supported in Closure.
Note that we do not manually extract all locales supported in Closure;
instead we only support the CLDR canonical locales (as done before) +
common aliases that CLDR provides data for. We are not aware of other
locale aliases within Closure that wouldn't be part of the CLDR aliases.
If there would be, then Angular/Closure would fail accordingly.
PR Close#42230
In the past, the closure file has been generated so that all individual
locale files were imported individually. This resulted in a huge
slow-down in g3 due to the large amount of imports.
With 90bd984ff7 this changed so that we
inline the locale data for the g3 closure locale file. Also the file
only contained data for locales being supported by Closure. For this a
list of locales has been extracted from Closure Compiler, as well as a
list of locale aliases.
This logic is prone to CLDR version updates, and also broke as part of
the Gulp -> Bazel migration where this logic has been slightly modified
but caused issues in G3. e.g. a locale `zh-Hant` was requested in g3,
but the locale data had the name of the alias locale that provided the
data at index zero (which represents the locale name). Note that the
locale names at index zero always could differentiate from the requested
`goog.LOCALE` due to the aliasing logic. This just didn't come up before.
We simplify this logic by generating a `goog.LOCALE` case for all
locales CLDR provides data for. We don't need to bother about aliasing
because with the refactorings to the CLDR generation tool, all locales
are built (which also captures the aliases), and we can generate the locale
file on the fly (which has not been done before).
PR Close#42230
The CLDR extraction tool has been reworked to run as part of Bazel.
This adds a initial readme explaining what the tool generates. It's
far from a detailed description but it can serve as foundation for more
detailed explanations.
PR Close#42230
Introduces a few Starlark macros for running the new Bazel
CLDR generation tool. Wires up the new tool so that locales
are generated properly. Also updates the existing
`closure-locale` file to match the new output generated by the Bazel tool.
This commit also re-adds a few locale files that aren't
generated by CLDR 37, but have been accidentally left in
the repository as the Gulp script never removed old locales
from previous CLDR versions. This problem is solved with the
Bazel generation of locale files, but for now we re-add these
old CLDR 33 locale files to not break developers relying on these
(even though the locale data indicies are incorrect; but there might
be users accessing the data directly)
PR Close#42230
Converts the CLDR locale extraction script to a Bazel tool.
This allows us to generate locale files within Bazel, so that
locales don't need to live as sources within the repo. Also
it allows us to get rid of the legacy Gulp tooling.
The migration of the Gulp script to a Bazel tool involved the
following things:
1. Basic conversion of the `extract.js` script to TypeScript.
This mostly was about adding explicit types. e.g. adding `locale:
string` or `localeData: CldrStatic`.
2. Split-up into separate files. Instead of keeping the large
`extract.js` file, the tool has been split into separate files.
The logic remains the same, just that code is more readable and
maintainable.
3. Introduction of a new `index.ts` file that is the entry-point
for the Bazel tool. Previously the Gulp tool just generated
all locale files, the default locale and base currency files
at once. The new entry-point accepts a mode to be passed as
first process argument. based on that argument, either locales
are generated into a specified directory, or the default locale,
base currencies or closure file is generated.
This allows us to generate files with a Bazel genrule where
we simply run the tool and specify the outputs. Note: It's
necessary to have multiple modes because files live in separate
locations. e.g. the default locale in `@angular/core`, but the
rest in `@angular/common`.
4. Removal of the `cldr-data-downloader` and custom CLDR resolution
logic. Within Bazel we cannot run a downloader using network.
We switch this to something more Bazel idiomatic with better
caching. For this a new repository rule is introduced that
downloads the CLDR JSON repository and extracts it. Within
that rule we determine the supported locales so that they
can be used to pre-declare outputs (for the locales) within
Bazel analysis phase. This allows us to add the generated locale
files to a `ts_library` (which we want to have for better testing,
and consistent JS transpilation).
Note that the removal of `cldr-data-downloader` also requires us to
add logic for detecting locales without data. The CLDR data
downloader overwrote the `availableLocales.json` file with a file
that only lists locales that CLDR provides data for. We use the
official `availableLocales` file CLDR provides, but filter out
locales for which no data is available. This is needed until we
update to CLDR 39 where data is available for all such locales
listed in `availableLocales.json`.
PR Close#42230
This is a pre-refactor commit allowing us to move
the CLDR locale generation to Bazel where files would
no longer be checked-in, except for the `closure-locale`
file that is synced into Google3.
PR Close#42230
Within Google, closure compiler is used for dealing with translations.
We generate a closure-compatible locale file that allows for
registration within Angular, so that Closure i18n works well together
with Angular applications. Closure compiler does not limit its
locales to BCP47-canonical locale identifiers. This commit updates
the generation logic so that we also support deprecated (but aliased)
locale identifiers, or other aliases which are likely used within
Closure. We use CLDR's alias supplemental data for this. It instructs
us to alias `iw` to `he` for example. `iw` is still supported in Closure.
Note that we do not manually extract all locales supported in Closure;
instead we only support the CLDR canonical locales (as done before) +
common aliases that CLDR provides data for. We are not aware of other
locale aliases within Closure that wouldn't be part of the CLDR aliases.
If there would be, then Angular/Closure would fail accordingly.
PR Close#42230
In the past, the closure file has been generated so that all individual
locale files were imported individually. This resulted in a huge
slow-down in g3 due to the large amount of imports.
With 90bd984ff7 this changed so that we
inline the locale data for the g3 closure locale file. Also the file
only contained data for locales being supported by Closure. For this a
list of locales has been extracted from Closure Compiler, as well as a
list of locale aliases.
This logic is prone to CLDR version updates, and also broke as part of
the Gulp -> Bazel migration where this logic has been slightly modified
but caused issues in G3. e.g. a locale `zh-Hant` was requested in g3,
but the locale data had the name of the alias locale that provided the
data at index zero (which represents the locale name). Note that the
locale names at index zero always could differentiate from the requested
`goog.LOCALE` due to the aliasing logic. This just didn't come up before.
We simplify this logic by generating a `goog.LOCALE` case for all
locales CLDR provides data for. We don't need to bother about aliasing
because with the refactorings to the CLDR generation tool, all locales
are built (which also captures the aliases), and we can generate the locale
file on the fly (which has not been done before).
PR Close#42230
The CLDR extraction tool has been reworked to run as part of Bazel.
This adds a initial readme explaining what the tool generates. It's
far from a detailed description but it can serve as foundation for more
detailed explanations.
PR Close#42230
Introduces a few Starlark macros for running the new Bazel
CLDR generation tool. Wires up the new tool so that locales
are generated properly. Also updates the existing
`closure-locale` file to match the new output generated by the Bazel tool.
This commit also re-adds a few locale files that aren't
generated by CLDR 37, but have been accidentally left in
the repository as the Gulp script never removed old locales
from previous CLDR versions. This problem is solved with the
Bazel generation of locale files, but for now we re-add these
old CLDR 33 locale files to not break developers relying on these
(even though the locale data indicies are incorrect; but there might
be users accessing the data directly)
PR Close#42230
Converts the CLDR locale extraction script to a Bazel tool.
This allows us to generate locale files within Bazel, so that
locales don't need to live as sources within the repo. Also
it allows us to get rid of the legacy Gulp tooling.
The migration of the Gulp script to a Bazel tool involved the
following things:
1. Basic conversion of the `extract.js` script to TypeScript.
This mostly was about adding explicit types. e.g. adding `locale:
string` or `localeData: CldrStatic`.
2. Split-up into separate files. Instead of keeping the large
`extract.js` file, the tool has been split into separate files.
The logic remains the same, just that code is more readable and
maintainable.
3. Introduction of a new `index.ts` file that is the entry-point
for the Bazel tool. Previously the Gulp tool just generated
all locale files, the default locale and base currency files
at once. The new entry-point accepts a mode to be passed as
first process argument. based on that argument, either locales
are generated into a specified directory, or the default locale,
base currencies or closure file is generated.
This allows us to generate files with a Bazel genrule where
we simply run the tool and specify the outputs. Note: It's
necessary to have multiple modes because files live in separate
locations. e.g. the default locale in `@angular/core`, but the
rest in `@angular/common`.
4. Removal of the `cldr-data-downloader` and custom CLDR resolution
logic. Within Bazel we cannot run a downloader using network.
We switch this to something more Bazel idiomatic with better
caching. For this a new repository rule is introduced that
downloads the CLDR JSON repository and extracts it. Within
that rule we determine the supported locales so that they
can be used to pre-declare outputs (for the locales) within
Bazel analysis phase. This allows us to add the generated locale
files to a `ts_library` (which we want to have for better testing,
and consistent JS transpilation).
Note that the removal of `cldr-data-downloader` also requires us to
add logic for detecting locales without data. The CLDR data
downloader overwrote the `availableLocales.json` file with a file
that only lists locales that CLDR provides data for. We use the
official `availableLocales` file CLDR provides, but filter out
locales for which no data is available. This is needed until we
update to CLDR 39 where data is available for all such locales
listed in `availableLocales.json`.
PR Close#42230
This is a pre-refactor commit allowing us to move
the CLDR locale generation to Bazel where files would
no longer be checked-in, except for the `closure-locale`
file that is synced into Google3.
PR Close#42230
In the past, the closure file has been generated so that all individual
locale files were imported individually. This resulted in a huge
slow-down in g3 due to the large amount of imports.
With 90bd984ff7 this changed so that we
inline the locale data for the g3 closure locale file. Also the file
only contained data for locales being supported by Closure. For this a
list of locales has been extracted from Closure Compiler, as well as a
list of locale aliases.
This logic is prone to CLDR version updates, and also broke as part of
the Gulp -> Bazel migration where this logic has been slightly modified
but caused issues in G3. e.g. a locale `zh-Hant` was requested in g3,
but the locale data had the name of the alias locale that provided the
data at index zero (which represents the locale name). Note that the
locale names at index zero always could differentiate from the requested
`goog.LOCALE` due to the aliasing logic. This just didn't come up before.
We simplify this logic by generating a `goog.LOCALE` case for all
locales CLDR provides data for. We don't need to bother about aliasing
because with the refactorings to the CLDR generation tool, all locales
are built (which also captures the aliases), and we can generate the locale
file on the fly (which has not been done before).
PR Close#42230
The CLDR extraction tool has been reworked to run as part of Bazel.
This adds a initial readme explaining what the tool generates. It's
far from a detailed description but it can serve as foundation for more
detailed explanations.
PR Close#42230
Introduces a few Starlark macros for running the new Bazel
CLDR generation tool. Wires up the new tool so that locales
are generated properly. Also updates the existing
`closure-locale` file to match the new output generated by the Bazel tool.
This commit also re-adds a few locale files that aren't
generated by CLDR 37, but have been accidentally left in
the repository as the Gulp script never removed old locales
from previous CLDR versions. This problem is solved with the
Bazel generation of locale files, but for now we re-add these
old CLDR 33 locale files to not break developers relying on these
(even though the locale data indicies are incorrect; but there might
be users accessing the data directly)
PR Close#42230
Converts the CLDR locale extraction script to a Bazel tool.
This allows us to generate locale files within Bazel, so that
locales don't need to live as sources within the repo. Also
it allows us to get rid of the legacy Gulp tooling.
The migration of the Gulp script to a Bazel tool involved the
following things:
1. Basic conversion of the `extract.js` script to TypeScript.
This mostly was about adding explicit types. e.g. adding `locale:
string` or `localeData: CldrStatic`.
2. Split-up into separate files. Instead of keeping the large
`extract.js` file, the tool has been split into separate files.
The logic remains the same, just that code is more readable and
maintainable.
3. Introduction of a new `index.ts` file that is the entry-point
for the Bazel tool. Previously the Gulp tool just generated
all locale files, the default locale and base currency files
at once. The new entry-point accepts a mode to be passed as
first process argument. based on that argument, either locales
are generated into a specified directory, or the default locale,
base currencies or closure file is generated.
This allows us to generate files with a Bazel genrule where
we simply run the tool and specify the outputs. Note: It's
necessary to have multiple modes because files live in separate
locations. e.g. the default locale in `@angular/core`, but the
rest in `@angular/common`.
4. Removal of the `cldr-data-downloader` and custom CLDR resolution
logic. Within Bazel we cannot run a downloader using network.
We switch this to something more Bazel idiomatic with better
caching. For this a new repository rule is introduced that
downloads the CLDR JSON repository and extracts it. Within
that rule we determine the supported locales so that they
can be used to pre-declare outputs (for the locales) within
Bazel analysis phase. This allows us to add the generated locale
files to a `ts_library` (which we want to have for better testing,
and consistent JS transpilation).
Note that the removal of `cldr-data-downloader` also requires us to
add logic for detecting locales without data. The CLDR data
downloader overwrote the `availableLocales.json` file with a file
that only lists locales that CLDR provides data for. We use the
official `availableLocales` file CLDR provides, but filter out
locales for which no data is available. This is needed until we
update to CLDR 39 where data is available for all such locales
listed in `availableLocales.json`.
PR Close#42230
