ESM resolution and loading is now always synchronous from a
non-loader-hook thread. If no asynchrnous loader hooks are
registered, the resolution/loading is entirely synchronous.
If asynchronous loader hooks are registered, these would be
synchronous on the non-loader-hook thread, and asynchronous
on the loader hook thread.
This avoids several races caused by async/sync loading sharing
the same cache. In particular, asynchronous loader hooks
now works with `require(esm)` - previously it tends to break
due to races.
In addition, when an asynchronous loader hook
returns a promise that never settles, the main thread no longer
silently exits with exit code 13, leaving the code below
any module loading calls silently ignored without being executed.
Instead, it now throws ERR_ASYNC_LOADER_REQUEST_NEVER_SETTLED
which can be caught and handled by the main thread. If the module
request comes from `import()`, the never-settling promise is
now relayed to the result returned by `import()`.
Drive-by: when annotating the error about importing undetectable
named exports from CommonJS, it now no longer reload the source
code of the CommonJS module, and instead reuses format information
cached when the module was loaded for linking.
PR-URL: https://github.com/nodejs/node/pull/60380
Fixes: https://github.com/nodejs/node/issues/59666
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Jacob Smith <jacob@frende.me>
- This updates the comments that assume loader hooks must be async
- Differentiate the sync/async loader hook paths in naming
`#customizations` is now `#asyncLoaderHooks` to make it clear
it's from the async APIs.
- Differentiate the paths running on the loader hook thread
(affects the loading of async other loader hooks and are async)
v.s. paths on the main thread calling out to code on the loader
hook thread (do not handle loading of other async loader hooks, and
can be sync by blocking).
- `Hooks` is now `AsyncLoaderHooksOnLoaderHookWorker`
- `CustomizedModuleLoader` is now
`AsyncLoaderHooksProxiedToLoaderHookWorker` and moved into
`lib/internal/modules/esm/hooks.js` as it implements the same
interface as `AsyncLoaderHooksOnLoaderHookWorker`
- `HooksProxy` is now `AsyncLoaderHookWorker`
- Adjust the JSDoc accordingly
- Clarify the "loader worker" as the "async loader hook worker"
i.e. when there's no _async_ loader hook registered, there won't
be this worker, to avoid the misconception that this worker
is spawned unconditionally.
- The code run on the loader hook worker to process
`--experimental-loader` is moved into
`lib/internal/modules/esm/worker.js` for clarity.
- The initialization configuration `forceDefaultLoader` is split
into `shouldSpawnLoaderHookWorker` and `shouldPreloadModules`
as those can be separate.
- `--experimental-vm-modules` is now processed during pre-execution
and no longer part of the initialization of the built-in ESM
loader, as it only exposes the vm APIs of ESM, and is unrelated
to built-in ESM loading.
PR-URL: https://github.com/nodejs/node/pull/60278
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
The option name should be `directory` to be consistent with the
returned result. It should also allow environment variable
overrides.
This also adds documentation for the new options and improves it.
PR-URL: https://github.com/nodejs/node/pull/59931
Reviewed-By: Aditi Singh <aditisingh1400@gmail.com>
Adds an option (NODE_COMPILE_CACHE_PORTABLE) for
the built-in compile cache to encode the hashes with
relative file paths. On enabling the option,
the source directory along with cache directory can be
bundled and moved, and the cache continues to work.
When enabled, paths encoded in hash are relative to
compile cache directory.
PR-URL: https://github.com/nodejs/node/pull/58797
Fixes: https://github.com/nodejs/node/issues/58755
Refs: https://github.com/nodejs/node/issues/52696
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
`v8::Module::IsGraphAsync()` traverses the dependencies to find if they
contain TLA each time. `ModuleWrap.hasAsyncGraph` caches the result
and exposes the property to JS land so that the presence of the property
`module.hasAsyncGraph` can be consistent.
This also allows C++ access of cached `hasAsyncGraph`.
This merges the `intantiateSync`/`instantiate` and
`getNamespaceSync`/`getNamespace` as they are always sync.
PR-URL: https://github.com/nodejs/node/pull/59703
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Otherwise if the ESM happens to be cached separately by the ESM loader
before it gets loaded with `require(esm)` from within an imported
CJS file (which uses a re-invented require() with a couple of quirks,
including a separate cache), it won't be able to load the esm properly
from the cache.
PR-URL: https://github.com/nodejs/node/pull/59679
Refs: https://github.com/nodejs/node/issues/59666
Refs: https://github.com/nodejs/node/issues/52697
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Clarify the names - hasAsyncGraph means either the module or its
dependencies contains top-level await; hasTopLevelAwait means the
module itself contains top-level await. Theoratically the former
can be inferred from iterating over the dependencies but for the
built-in loader it's currently not fully reliable until we eliminate
async linking.
Also remove the hasTopLevelAwait method - we can simply put it
on the module wrap for source text modules right after compilation.
PR-URL: https://github.com/nodejs/node/pull/59675
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
This completes the TODO to compile WASM synchronously and thus
making translation (i.e. compilation + instantiation) synchronous.
PR-URL: https://github.com/nodejs/node/pull/59453
Refs: https://github.com/nodejs/node/issues/55782
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
1. Make sure that the conditions are converted into arrays when
being passed into user hooks.
2. Pass the conditions from user hooks into the ESM resolution
so that it takes effect.
PR-URL: https://github.com/nodejs/node/pull/59011
Fixes: https://github.com/nodejs/node/issues/59003
Reviewed-By: Zeyu "Alex" Yang <himself65@outlook.com>
Reviewed-By: Jacob Smith <jacob@frende.me>
PR-URL: https://github.com/nodejs/node/pull/58521
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: LiviaMedeiros <livia@cirno.name>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Re-evaluating an errored ESM should lead to rejecting
the rejection again - this is also the case when importing
it twice. In the case of retrying with
require after import, just throw the cached error.
Drive-by: add some debug logs.
PR-URL: https://github.com/nodejs/node/pull/58957
Fixes: https://github.com/nodejs/node/issues/58945
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
This commit refactors the CommonJS loader to remove TypeScript-specific
extensions from the require.extensions object for compatibility with
libraries that depended on it to initialize extenal TypeScript loaders.
PR-URL: https://github.com/nodejs/node/pull/58657
Refs: https://github.com/nodejs/typescript/issues/37
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Pietro Marchini <pietro.marchini94@gmail.com>
Reviewed-By: Xuguang Mei <meixuguang@gmail.com>
When --import is used, the ESM loader is used to handle even pure
CJS entry points, and it can run into CJS module facades in the
evaluating state when the parent CJS module is being evaluated.
In this case it should be allowed, since the ESM <-> CJS cycles
that are meant to be disallowed (for the time being) should
already be detected before evaluation and wouldn't get here,
and CJS <-> CJS cycles are fine.
PR-URL: https://github.com/nodejs/node/pull/58598
Fixes: https://github.com/nodejs/node/issues/58515
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Ethan Arrowood <ethan@arrowood.dev>
Obtain sourceURL magic comments via V8 API to avoid a second round of
extraction of magic comments.
Updates source map snapshot normalization to allow testing full path
names inside the test outputs.
PR-URL: https://github.com/nodejs/node/pull/58389
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Boolean value to check if an ES Module is the entrypoint of the
current process.
Implements: #57226
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/57804
Fixes: https://github.com/nodejs/node/issues/57226
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
`EXTENSIONLESS_FORMAT_JAVASCRIPT` and `EXTENSIONLESS_FORMAT_WASM` are
only used internally through binding `getFormatOfExtensionlessFile`.
They should not be exposed publicly.
PR-URL: https://github.com/nodejs/node/pull/58327
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: LiviaMedeiros <livia@cirno.name>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Juan José Arboleda <soyjuanarbol@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/56491
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Jacob Smith <jacob@frende.me>
PR-URL: https://github.com/nodejs/node/pull/57596
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/57704
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
