In PR #45405, the Angular Package Format (APF) was updated so that
secondary entry-points (such as `@angular/common/http`) do not have
their own `package.json` file, as they used to. Instead, the paths to
their various formats and types are exposed via the primary
`package.json` file's `exports` property. As an example, see the v13
[@angular/common/http/package.json][1] and compare it with the v14
[@angular/common/package.json > exports][2].
Previously, `ngcc` was not able to analyze such v14+ entry-points and
would instead error as it considered such entry-points missing.
This commit addresses the issue by detecting this situation and
synthesizing a `package.json` file for the secondary entry-points based
on the `exports` property of the primary `package.json` file. This data
is only used by `ngcc` in order to determine that the entry-point does
not need further processing, since it is already in Ivy format.
[1]: https://unpkg.com/browse/@angular/common@13.3.5/http/package.json
[2]: https://unpkg.com/browse/@angular/common@14.0.0-next.15/package.json
PR Close#45833
Adds support for TypeScript 4.7. Changes include:
* Bumping the TS version as well as some Bazel dependencies to include https://github.com/bazelbuild/rules_nodejs/pull/3420.
* Adding a backwards-compatibility layer for calls to `updateTypeParameterDeclaration`.
* Making `LView` generic in order to make it easier to type the context based on the usage. Currently the context can be 4 different types which coupled with stricter type checking would required a lot of extra casting all over `core`.
* Fixing a bunch of miscellaneous type errors.
* Removing assertions of `ReferenceEntry.isDefinition` in a few of the language service tests. The field isn't returned by TS anymore and we weren't using it for anything.
* Resolving in error in the language service that was caused by TS attempting to parse HTML files when we try to open them. Previous TS was silently setting them as `ScriptKind.Unknown` and ignoring the errors, but now it throws. I've worked around it by setting them as `ScriptKind.JSX`.
PR Close#45749
Bundlers like Rollup may use an element access expression for an export
declaration, which causes ngcc to ignore those export declarations possibly
resulting in incomplete processing of packages.
Element access syntax may be used when the declared name is not considered
as valid JS identifier, but bundlers may be conservative in determining whether
an identifier can be used (to emit a property access) and opt for a string
literal in an element access instead.
The element access syntax introduces a problem for ngcc, where it wouldn't
consider such export as class declaration, causing them to be skipped. The
ngtsc compiler is implemented with the assumption that all class declarations
use a `ts.Identifier` as name, whereas the element access is using a string
literal for the declared name. This makes it troublesome for ngcc to support
this syntax form in UMD bundles.
To work around the problem, this function transforms these access expressions
into regular property accesses. The source text is parsed to an AST to allow
finding the element accesses in a robust way, after which the affected text
ranges are replaced with property accesses in the original source text.
Closes#44037
PR Close#44669
Previously, ngcc could only handle UMD modules whose wrapper function
was implemented as a `ts.ConditionalExpression` (i.e. using a ternary
operator). This is the format emitted by popular bundlers, such as
Rollup. However, this failed to account for a different format, using
`if/else` statements, such as the one [emitted by Webpack][1].
This commit prepares ngcc for supporting different UMD wrapper function
formats by decoupling the operation of parsing the wrapper function body
to capture the various factory function calls and that of operating on
the factory function calls (for example, to read or update their
arguments). In a subsequent commit, this will be used to add support for
the Webpack format.
[1]: https://webpack.js.org/configuration/output/#type-umd
PR Close#44245
This commit adds additional information to encourage developers to contact
the author of View Engine libraries and ask them to update to partial Ivy.
Fixes#42308
PR Close#43996
Recently rollup, used by ng-packagr, changed the position of parentheses
around its generated UMD wrapper functions.
This commit ensures that ngcc can handle both.
Fixes#43870
PR Close#43879
As outlined in the previous commit which enabled the `esModuleInterop`
TypeScript compiler option, we need to update all namespace imports
for `typescript` to default imports. This is needed to allow for
TypeScript to be imported at runtime from an ES module.
Similar changes are needed for modules like `semver` where the types incorrectly
suggest named exports that will not exist at runtime when imported from ESM.
This commit refactors all imports to match with the lint rule we have
configured in the previous commit. See the previous commit for more
details on why certain imports have been changed.
A special case are the imports to `@babel/core` and `@babel/types`. For
these a special interop is needed as both default imports, or named
imports break the other module format. e.g default imports would work
well for ESM, but it breaks for CJS. For CJS, the named imports would
only work, but in ESM, only the default export exist. We work around
this for now until the devmode is using ESM as well (which would be
consistent with prodmode and gives us more valuable test results). More
details on the interop can be found in the `babel_core.ts` files (two
interops are needed for both localize/or the compiler-cli).
PR Close#43431
Prior to this commit ngcc stored its package configuration in JavaScript
objects, which caused the builtin `Object` members to be found as
package configuration. This would subsequently crash as their shape was
not as expected.
This commit moves away from using raw JavaScript objects in favor of a
Map. To code was refactored such that `PartiallyProcessedConfig` is
now a class.
Fixes#43570
PR Close#43589
The previous default algorithm was `md5`, which is not compliant with FIPS.
The default is now set to `sha256`, which is compliant.
Fixes#42577
PR Close#42582
The ngcc configuration gets hashed to be used when caching
but it was hardcoded to use the `md5` algorithm, which is
not FIPS compliant.
Now the hash algorithm can be configured in the ngcc.config.js
file at the project level.
PR Close#42582
Some tools (such as Language Server and ng-packagr) only care about
the processed typings generated by ngcc. Forcing these tools to process
the JavaScript files as well has two disadvantages:
First, unnecessary work is being done, which is time consuming.
But more importantly, it is not always possible to know how the final bundling
tools will want the processed JavaScript to be configured. For example,
the CLI would prefer the `--create-ivy-entry-points` option but this would
break non-CLI build tooling.
This commit adds a new option (`--typings-only` on the command line, and
`typingsOnly` via programmatic API) that instructs ngcc to only render changes
to the typings files for the entry-points that it finds, and not to write any
JavaScript files.
In order to process the typings, a JavaScript format will need to be analysed, but
it will not be rendered to disk. When using this option, it is best to offer ngcc a
wide range of possible JavaScript formats to choose from, and it will use the
first format that it finds. Ideally you would configure it to try the `ES2015` FESM
format first, since this will be the most performant.
Fixes#40969
PR Close#40976
Now that `ReadonlyFileSystem` and `PathManipulation` interfaces are
available, this commit updates the compiler-cli to use these more
focussed interfaces.
PR Close#40281
ngtsc has a robust suite of testing utilities, designed for in-memory
testing of a TypeScript compiler. Previously these utilities lived in the
`test` directory for the compiler-cli package.
This commit moves those utilities to an `ngtsc/testing` package, enabling
them to be depended on separately and opening the door for using them from
the upcoming language server testing infrastructure.
As part of this refactoring, the `fake_core` package (a lightweight API
replacement for @angular/core) is expanded to include functionality needed
for Language Service test use cases.
PR Close#39594
ngcc creates typically two `ts.Program` instances for each entry-point,
one for processing sources and another one for processing the typings.
The creation of these programs is somewhat expensive, as it concerns
module resolution and parsing of source files.
This commit implements several layers of caching to optimize the
creation of programs:
1. A shared module resolution cache across all entry-points within a
single invocation of ngcc. Both the sources and typings program
benefit from this cache.
2. Sharing the parsed `ts.SourceFile` for a single entry-point between
the sources and typings program.
3. Sharing parsed `ts.SourceFile`s of TypeScript's default libraries
across all entry-points within a single invocation. Some of these
default library typings are large and therefore expensive to parse,
so sharing the parsed source files across all entry-points offers
a significant performance improvement.
Using a bare CLI app created using `ng new` + `ng add @angular/material`,
the above changes offer a 3-4x improvement in ngcc's processing time
when running synchronously and ~2x improvement for asynchronous runs.
PR Close#38840
When ngcc creates an entry-point program, the `allowJs` option is enabled
in order to operate on the JavaScript source files of the entry-point.
A side-effect of this approach is that external modules that don't ship
declaration files will also have their JavaScript source files loaded
into the program, as the `allowJs` flag allows for them to be imported.
This may pose an issue in certain edge cases, where ngcc would inadvertently
operate on these external modules. This can introduce all sorts of undesirable
behavior and incompatibilities, e.g. the reflection host that is selected for
the entry-point's format could be incompatible with that of the external
module's JavaScript bundles.
To avoid these kinds of issues, module resolution that would resolve to
a JavaScript file located outside of the package will instead be rejected,
as if the file would not exist. This would have been the behavior when
`allowJs` is set to false, which is the case in typical Angular compilations.
Fixes#37508
PR Close#37596
The `Logger` interface and its related classes are general purpose
and could be used by other tooling. Moving it into ngtsc is a more
suitable place from which to share it - similar to the FileSystem stuff.
PR Close#37114
Previously, ngcc would only be able to match an ngcc configuration to
packages that were located inside the project's top-level
`node_modules/`. However, if there are multiple versions of a package in
a project (e.g. as a transitive dependency of other packages), multiple
copies of a package (at different versions) may exist in nested
`node_modules/` directories. For example, one at
`<project-root>/node_modules/some-package/` and one at
`<project-root>/node_modules/other-package/node_modules/some-package/`.
In such cases, ngcc was only able to detect the config for the first
copy but not for the second.
This commit fixes this by returning a new instance of
`ProcessedNgccPackageConfig` for each different package path (even if
they refer to the same package name). In these
`ProcessedNgccPackageConfig`, the `entryPoints` paths have been
processed to take the package path into account.
PR Close#37040
This commit adds a `packageName` property to the `EntryPoint` interface.
In a subsequent commit this will be used to retrieve the correct ngcc
configuration for each package, regardless of its path.
PR Close#37040
In order to retrieve the ngcc configuration (if any) for an entry-point,
ngcc has to detect the containing package's version.
Previously, ngcc would try to read the version from the entry-point's
`package.json` file, which was different than the package's top-level
`package.json` for secondary entry-points. For example, it would try to
read it from `node_modules/@angular/common/http/package.json` for
entry-point `@angular/common/http`. However, the `package.json` files
for secondary entry-points are not guaranteed to include a `version`
property.
This commit fixes this by first trying to read the version from the
_package's_ `package.json` (falling back to the entry-point's
`package.json`). For example, it will first try to read it from
`@angular/common/package.json` for entry-point `@angular/common/http`.
PR Close#37040
Rename the `package` property to `packagePath` on the `EntryPoint`
interface. This makes it more clear that the `packagePath` property
holds the absolute path to the containing package (similar to how `path`
holds the path to the entry-point). This will also align with the
`packageName` property that will be added in a subsequent commit.
This commit also re-orders the `EntryPoint` properties to group related
properties together and to match the order of properties on instances
with that on the interface.
PR Close#37040
Previously, when an entry-point was ignored via an ngcc config, ngcc
would scan sub-directories for sub-entry-points, but would not use the
correct `packagePath`. For example, if `@angular/common` was ignored, it
would look at `@angular/common/http` but incorrectly use
`.../@angular/common/http` as the `packagePath` (instead of
`.../@angular/common`). As a result, it would not retrieve the correct
ngcc config for the actual package.
This commit fixes it by ensuring the correct `packagePath` is used, even
if the primary entry-point corresponding to that path is ignored. In
order to do this, a new return value for `getEntryPointInfo()` is added:
`IGNORED_ENTRY_POINT`. This is used to differentiate between directories
that correspond to no or an incompatible entry-point and those that
correspond to an entry-point that could otherwise be valid but is
explicitly ignored. Consumers of `getEntryPointInfo()` can then use this
info to discard ignored entry-points, but still use the correct
`packagePath` when scanning their sub-directories for secondary
entry-points.
PR Close#37040
As of version 10, libraries following the APF will no longer contain
ESM5 output. Hence, tests in ngcc need to be updated as they currently
rely on the release output of `@angular/core`.
Additionally, we'd need to support in ngcc that the `module`
property of entry-points no longer necessarily refers to
`esm5` output, but instead can also target `esm2015`.
We currently achieve this by checking the path the `module`
property points to. We can do this because as per APF, the
folder name is known for the esm2015 output. Long-term for
more coverage, we want to sniff the format by looking for
known ES2015 constructs in the file `module` refers to.
PR Close#36944
The commit adds support to the ngcc.config.js file for setting the
`retryAttempts` and `retryDelay` options for the `AsyncLocker`.
An integration test adds a new check for a timeout and actually uses the
ngcc.config.js to reduce the timeout time to prevent the test from taking
too long to complete.
PR Close#36838
Strictly this method only returns config for packages. So this commit
renames it to `getPackageConfig()`, which frees us up to add other
"getXxxxConfig()` methods later.
PR Close#36838
This test is basically duplicated (and slightly enhanced) in the
following test. So it is superfluous. (I suspect it was the result
of a broken rebase.)
PR Close#36838
The base path for package and entry-points is known so there is
no need to store these in the file. Also this commit avoids storing
empty arrays unnecessarily.
PR Close#36486
Previously, even if an entry-point did not need to be processed,
ngcc would always parse the files of the entry-point to compute
its dependencies. This can take a lot of time for large node_modules.
Now these dependencies are cached in the entry-point manifest,
and read from there rather than computing them every time.
See https://github.com/angular/angular/issues/36414\#issuecomment-608401834
FW-2047
PR Close#36486
1. update jasmine to 3.5
2. update @types/jasmine to 3.5
3. update @types/jasminewd2 to 2.0.8
Also fix several cases, the new jasmine 3 will help to create test cases correctly,
such as in the `jasmine 2.x` version, the following case will pass
```
expect(1 == 2);
```
But in jsamine 3, the case will need to be
```
expect(1 == 2).toBeTrue();
```
PR Close#34625
Recently we added support for ignoring specified deep-import
warnings by providing sets of regular expressions within the
`ngcc.config.js` file. But this was only working for the project
level configuration.
This commit fixes ngcc so that it will also read these regular
expressions from package level configuration too.
Fixes#35750
PR Close#36423
The `browser` package.json property is now supported to the same
level as `main` - i.e. it is sniffed for UMD, ESM5 and CommonJS.
The `browser` property can also contain an object with file overrides
but this is not supported by ngcc.
Fixes#36062
PR Close#36396
Previously, `main` was only checked for `umd` or `commonjs`
formats. Now if there are `import` or `export` statements in the
source file it will be deemed to be in `esm5` format.
Fixes#35788
PR Close#36396
Ngcc supports providing a project-level configuration to affect how
certain dependencies are processed and also has a built-in fallback
configuration for some unmaintained packages. Each entry in these
configurations could be scoped to specific versions of a package by
providing a version range. If no version range is provided for a
package, it defaults to `*` (with the intention of matching any
version).
Previously, the installed version of a package was tested against the
version range using the [semver][1] package's `satisfies()` function
with the default options. By default, `satisfies()` does not match
pre-releases (see [here][2] for more details on reasoning). While this
makes sense when determining what version of a dependency to install
(trying to avoid unexpected breaking changes), it is not desired in the
case of ngcc.
This commit fixes it by explicitly specifying that pre-release versions
should be matched normally.
[1]: https://www.npmjs.com/package/semver
[2]: https://github.com/npm/node-semver#prerelease-tags
PR Close#36370
The new `EntryPointManifest` class can read and write a
manifest file that contains all the paths to the entry-points
that have been found in a node_modules folder.
This can be used to speed up finding entry-points in
subsequent runs.
The manifest file stores the ngcc version and hashes of
the package lock-file and project config, since if these
change the manifest will need to be recomputed.
PR Close#35931
This reduces the time that `findEntryPoints` takes from 9701.143ms to 4177.278ms, by reducing the file operations done.
Reference: #35717
PR Close#35756
Now `hasBeenProcessed()` will no longer throw if there
is an entry-point that has been built with an outdated
version of ngcc.
Instead it just returns `false`, which will include it in this
processing run.
This is a precursor to adding functionality that will
automatically revert outdate build artifacts.
PR Close#35079
Previously the `rootDir` was set to the entry-point path but
this is incorrect if the source files are stored in a directory outside
the entry-point path. This is the case in the latest versions of the
Angular CDK.
Instead the `rootDir` should be the containing package path, which is
guaranteed to include all the source for the entry-point.
---
A symptom of this is an error when ngcc is trying to process the source of
an entry-point format after the entry-point's typings have already been
processed by a previous processing run.
During processing the `_toR3Reference()` function gets called which in turn
makes a call to `ReflectionHost.getDtsDeclaration()`. If the typings files
are also being processed this returns the node from the dts typings files.
But if we have already processed the typings files and are now processing
only an entry-point format without typings, the call to
`ReflectionHost.getDtsDeclaration()` returns `null`.
When this value is `null`, a JS `valueRef` is passed through as the DTS
`typeRef` to the `ReferenceEmitter`. In this case, the `ReferenceEmitter`
fails during `emit()` because no `ReferenceEmitStrategy` is able to provide
an emission:
1) The `LocalIdentifierStrategy` is not able help because in this case
`ImportMode` is `ForceNewImport`.
2) The `LogicalProjectStrategy` cannot find the JS file below the `rootDir`.
The second strategy failure is fixed by this PR.
Fixes https://github.com/angular/ngcc-validation/issues/495
PR Close#34212
In a package.json file, the "typings" or "types" field could be an array
of typings files. ngcc would previously crash unexpectedly for such
packages, as it assumed that the typings field would be a string. This
commit lets ngcc skip over such packages, as having multiple typing
entry-points is not supported for Angular packages so it is safe to
ignore them.
Fixes#33646
PR Close#33973
Some declaration files may not be referenced from an entry-point's
main typings file, as it may declare types that are only used internally.
ngcc has logic to include declaration files based on all source files,
to ensure internal declaration files are available.
For packages following APF layout, however, this logic was insufficient.
Consider an entry-point with base path of `/esm2015/testing` and typings
residing in `/testing`, the file
`/esm2015/testing/src/nested/internal.js` has its typings file at
`/testing/src/nested/internal.d.ts`. Previously, the declaration was
assumed to be located at `/esm2015/testing/testing/internal.d.ts` (by
means of `/esm2015/testing/src/nested/../../testing/internal.d.ts`)
which is not where the declaration file can be found. This commit
resolves the issue by looking in the correct directory.
PR Close#33875
This commit adapts the private NgModule re-export system (using aliasing) to
ngcc. Not all ngcc compilations are compatible with these re-exports, as
they assume a 1:1 correspondence between .js and .d.ts files. The primary
concern here is supporting them for commonjs-only packages.
PR Close#33177
Normally, when ngcc encounters a package with missing dependencies while
attempting to determine a compilation ordering, it will ignore that package.
This commit adds a configuration for a flag to tell ngcc to compile the
package anyway, regardless of any missing dependencies.
FW-1931 #resolve
PR Close#33192
It is now possible to include a set of default ngcc configurations
that ship with ngcc out of the box. This allows ngcc to handle a
set of common packages, which are unlikely to be fixed, without
requiring the application developer to write their own configuration
for them.
Any packages that are configured at the package or project level
will override these default configurations. This allows a reasonable
level of control at the package and user level.
PR Close#33008
