This uses the same mechanism as [large
strings](https://github.com/facebook/react/pull/26932) to encode chunks
of length based binary data in the RSC payload behind a flag.
I introduce a new BinaryChunk type that's specific to each stream and
ways to convert into it. That's because we sometimes need all chunks to
be Uint8Array for the output, even if the source is another array buffer
view, and sometimes we need to clone it before transferring.
Each type of typed array is its own row tag. This lets us ensure that
the instance is directly in the right format in the cached entry instead
of creating a wrapper at each reference. Ideally this is also how
Map/Set should work but those are lazy which complicates that approach a
bit.
We assume both server and client use little-endian for now. If we want
to support other modes, we'd convert it to/from little-endian so that
the transfer protocol is always little-endian. That way the common
clients can be the fastest possible.
So far this only implements Server to Client. Still need to implement
Client to Server for parity.
NOTE: This is the first time we make RSC effectively a binary format.
This is not compatible with existing SSR techniques which serialize the
stream as unicode in the HTML. To be compatible, those implementations
would have to use base64 or something like that. Which is what we'll do
when we move this technique to be built-in to Fizz.
We already support these in the sense that they're Iterable so they just
get serialized as arrays. However, these are part of the Structured
Clone algorithm [and should be
supported](https://github.com/facebook/react/issues/25687).
The encoding is simply the same form as the Iterable, which is
conveniently the same as the constructor argument. The difference is
that now there's a separate reference to it.
It's a bit awkward because for multiple reference to the same value,
it'd be a new Map/Set instance for each reference. So to encode sharing,
it needs one level of indirection with its own ID. That's not really a
big deal for other types since they're inline anyway - but since this
needs to be outlined it creates possibly two ids where there only needs
to be one or zero.
One variant would be to encode this in the row type. Another variant
would be something like what we do for React Elements where they're
arrays but tagged with a symbol. For simplicity I stick with the simple
outlining for now.
This introduces a Text row (T) which is essentially a string blob and
refactors the parsing to now happen at the binary level.
```
RowID + ":" + "T" + ByteLengthInHex + "," + Text
```
Today, we encode all row data in JSON, which conveniently never has
newline characters and so we use newline as the line terminator. We
can't do that if we pass arbitrary unicode without escaping it. Instead,
we pass the byte length (in hexadecimal) in the leading header for this
row tag followed by a comma.
We could be clever and use fixed or variable-length binary integers for
the row id and length but it's not worth the more difficult
debuggability so we keep these human readable in text.
Before this PR, we used to decode the binary stream into UTF-8 strings
before parsing them. This is inefficient because sometimes the slices
end up having to be copied so it's better to decode it directly into the
format. The follow up to this is also to add support for binary data and
then we can't assume the entire payload is UTF-8 anyway. So this
refactors the parser to parse the rows in binary and then decode the
result into UTF-8. It does add some overhead to decoding on a per row
basis though.
Since we do this, we need to encode the byte length that we want decode
- not the string length. Therefore, this requires clients to receive
binary data and why I had to delete the string option.
It also means that I had to add a way to get the byteLength from a chunk
since they're not always binary. For Web streams it's easy since they're
always typed arrays. For Node streams it's trickier so we use the
byteLength helper which may not be very efficient. Might be worth
eagerly encoding them to UTF8 - perhaps only for this case.
The bindings upstream in Relay has been removed so we don't need these
builds anymore. The idea is to revisit an FB integration of Flight but
it wouldn't use the Relay specific bindings. It's a bit unclear how it
would look but likely more like the OSS version so not worth keeping
these around.
The `dom-relay` name also included the FB specific Fizz implementation
of the streaming config so I renamed that to `dom-fb`. There's no Fizz
implementation for Native yet so I just removed `native-relay`.
We created a configurable fork for how to encode the output of Flight
and the Relay implementation encoded it as JSON objects instead of
strings/streams. The new implementation would likely be more stream-like
and just encode it directly as string/binary chunks. So I removed those
indirections so that this can just be declared inline in
ReactFlightServer/Client.
Just a small upgrade to keep us current and remove unused suppressions
(probably fixed by some upgrade since).
- `*` is no longer allowed and has been an alias for `any` for a while
now.
Stacked on #26557
Supporting Float methods such as ReactDOM.preload() are challenging for
flight because it does not have an easy means to convey direct
executions in other environments. Because the flight wire format is a
JSON-like serialization that is expected to be rendered it currently
only describes renderable elements. We need a way to convey a function
invocation that gets run in the context of the client environment
whether that is Fizz or Fiber.
Fiber is somewhat straightforward because the HostDispatcher is always
active and we can just have the FlightClient dispatch the serialized
directive.
Fizz is much more challenging becaue the dispatcher is always scoped but
the specific request the dispatch belongs to is not readily available.
Environments that support AsyncLocalStorage (or in the future
AsyncContext) we will use this to be able to resolve directives in Fizz
to the appropriate Request. For other environments directives will be
elided. Right now this is pragmatic and non-breaking because all
directives are opportunistic and non-critical. If this changes in the
future we will need to reconsider how widespread support for async
context tracking is.
For Flight, if AsyncLocalStorage is available Float methods can be
called before and after await points and be expected to work. If
AsyncLocalStorage is not available float methods called in the sync
phase of a component render will be captured but anything after an await
point will be a noop. If a float call is dropped in this manner a DEV
warning should help you realize your code may need to be modified.
This PR also introduces a way for resources (Fizz) and hints (Flight) to
flush even if there is not active task being worked on. This will help
when Float methods are called in between async points within a function
execution but the task is blocked on the entire function finishing.
This PR also introduces deduping of Hints in Flight using the same
resource keys used in Fizz. This will help shrink payload sizes when the
same hint is attempted to emit over and over again
Added an explicit type to all $FlowFixMe suppressions to reduce
over-suppressions of new errors that might be caused on the same lines.
Also removes suppressions that aren't used (e.g. in a `@noflow` file as
they're purely misleading)
Test Plan:
yarn flow-ci
This adds `encodeReply` to the Flight Client and `decodeReply` to the
Flight Server.
Basically, it's a reverse Flight. It serializes values passed from the
client to the server. I call this a "Reply". The tradeoffs and
implementation details are a bit different so it requires its own
implementation but is basically a clone of the Flight Server/Client but
in reverse. Either through callServer or ServerContext.
The goal of this project is to provide the equivalent serialization as
passing props through RSC to client. Except React Elements and
Components and such. So that you can pass a value to the client and back
and it should have the same serialization constraints so when we add
features in one direction we should mostly add it in the other.
Browser support for streaming request bodies are currently very limited
in that only Chrome supports it. So this doesn't produce a
ReadableStream. Instead `encodeReply` produces either a JSON string or
FormData. It uses a JSON string if it's a simple enough payload. For
advanced features it uses FormData. This will also let the browser
stream things like File objects (even though they're not yet supported
since it follows the same rules as the other Flight).
On the server side, you can either consume this by blocking on
generating a FormData object or you can stream in the
`multipart/form-data`. Even if the client isn't streaming data, the
network does. On Node.js busboy seems to be the canonical library for
this, so I exposed a `decodeReplyFromBusboy` in the Node build. However,
if there's ever a web-standard way to stream form data, or if a library
wins in that space we can support it. We can also just build a multipart
parser that takes a ReadableStream built-in.
On the server, server references passed as arguments are loaded from
Node or Webpack just like the client or SSR does. This means that you
can create higher order functions on the client or server. This can be
tokenized when done from a server components but this is a security
implication as it might be tempting to think that these are not fungible
but you can swap one function for another on the client. So you have to
basically treat an incoming argument as insecure, even if it's a
function.
I'm not too happy with the naming parity:
Encode `server.renderToReadableStream` Decode: `client.createFromFetch`
Decode `client.encodeReply` Decode: `server.decodeReply`
This is mainly an implementation details of frameworks but it's annoying
nonetheless. This comes from that `renderToReadableStream` does do some
"rendering" by unwrapping server components etc. The `create` part comes
from the parity with Fizz/Fiber where you `render` on the server and
`create` a root on the client.
Open to bike-shedding this some more.
---------
Co-authored-by: Josh Story <josh.c.story@gmail.com>
## Summary
Adds support for returning `undefined` from Server Components.
Also fixes a bug where rendering an empty fragment would throw the same
error as returning undefined.
## How did you test this change?
- [x] test failed with same error message I got when returning undefined
from Server Components in a Next.js app
- [x] test passes after adding encoding for `undefined`
This is just moving some stuff around and renaming things.
This tuple is opaque to the Flight implementation and we should probably
encode it separately as a single string instead of a model object.
The term "Metadata" isn't the same as when used for ClientReferences so
it's not really the right term anyway.
I also made it optional since a bound function with no arguments bound
is technically different than a raw instance of that function (it's a
clone).
I also renamed the type ReactModel to ReactClientValue. This is the
generic serializable type for something that can pass through the
serializable boundary from server to client. There will be another one
for client to server.
I also filled in missing classes and ensure the serializable sub-types
are explicit. E.g. Array and Thenable.
We support any super type of anything that we can serialize. Meaning
that as long as the Type that's passed through is less precise, it means
that we can encoded it as any subtype and therefore the incoming type
doesn't have to be the subtype in that case. Basically, as long as
you're only passing through an `Iterable<T>` in TypeScript, then you can
pass any `Iterable<T>` and we'll treat it as an array.
For example we support Promises *and* Thenables but both are encoded as
Promises.
We support Arrays and since Arrays are also Iterables, we can support
Iterables.
For @wongmjane
Previously when a called server reference function was rejected, the
emitted error chunk was not flushed, and the request was not properly
closed.
Co-authored-by: Sebastian Markbage <sebastian@calyptus.eu>
We always look up these references in a map so it doesn't matter what
their value is. It could be a hash for example.
The loaders now encode a single $$id instead of filepath + name.
This changes the react-client-manifest to have a single level. The value
inside the map is still split into module id + export name because
that's what gets looked up in webpack.
The react-ssr-manifest is still two levels because that's a reverse
lookup.
This is the first of a series of PRs, that let you pass functions, by
reference, to the client and back. E.g. through Server Context. It's
like client references but they're opaque on the client and resolved on
the server.
To do this, for security, you must opt-in to exposing these functions to
the client using the `"use server"` directive. The `"use client"`
directive lets you enter the client from the server. The `"use server"`
directive lets you enter the server from the client.
This works by tagging those functions as Server References. We could
potentially expand this to other non-serializable or stateful objects
too like classes.
This only implements server->server CJS imports and server->server ESM
imports. We really should add a loader to the webpack plug-in for
client->server imports too. I'll leave closures as an exercise for
integrators.
You can't "call" a client reference on the server, however, you can
"call" a server reference on the client. This invokes a callback on the
Flight client options called `callServer`. This lets a router implement
calling back to the server. Effectively creating an RPC. This is using
JSON for serializing those arguments but more utils coming from
client->server serialization.
This lets you pass Promises from server components to client components
and `use()` them there.
We still don't support Promises as children on the client, so we need to
support both. This will be a lot simpler when we remove the need to
encode children as lazy since we don't need the lazy encoding anymore
then.
I noticed that this test failed because we don't synchronously resolve
instrumented Promises if they're lazy. The second fix calls `.then()`
early to ensure that this lazy initialization can happen eagerly. ~It
felt silly to do this with an empty function or something, so I just did
the attachment of ping listeners early here. It's also a little silly
since they will ping the currently running render for no reason if it's
synchronously available.~ EDIT: That didn't work because a ping might
interrupt the current render. Probably need a bigger refactor.
We could add another extension but we've already taken a lot of
liberties with the Promise protocol. At least this is one that doesn't
need extension of the protocol as much. Any sub-class of promises could
do this.
This is just shifting around some encoding strategies for Flight in
preparation for more types.
```
S1:"react.suspense"
J2:["$", "$1", {children: "@3"}]
J3:"Hello"
```
```
1:"$Sreact.suspense"
2:["$", "$1", {children: "$L3"}]
3:"Hello"
```
The old version of prettier we were using didn't support the Flow syntax
to access properties in a type using `SomeType['prop']`. This updates
`prettier` and `rollup-plugin-prettier` to the latest versions.
I added the prettier config `arrowParens: "avoid"` to reduce the diff
size as the default has changed in Prettier 2.0. The largest amount of
changes comes from function expressions now having a space. This doesn't
have an option to preserve the old behavior, so we have to update this.
This renames Module References to Client References, since they are in
the server->client direction.
I also changed the Proxies exposed from the `node-register` loader to
provide better error messages. Ideally, some of this should be
replicated in the ESM loader too but neither are the source of truth.
We'll replicate this in the static form in the Next.js loaders. cc
@huozhi @shuding
- All references are now functions so that when you call them on the
server, we can yield a better error message.
- References that are themselves already referring to an export name are
now proxies that error when you dot into them.
- `use(...)` can now be used on a client reference to unwrap it server
side and then pass a reference to the awaited value.
These suppressions are no longer required.
Generated using:
```sh
flow/tool update-suppressions .
```
followed by adding back 1 or 2 suppressions that were only triggered in
some configurations.
This enables the "exact_empty_objects" setting for Flow which makes
empty objects exact instead of building up the type as properties are
added in code below. This is in preparation to Flow 191 which makes this
the default and removes the config.
More about the change in the Flow blog
[here](https://medium.com/flow-type/improved-handling-of-the-empty-object-in-flow-ead91887e40c).
This setting is an incremental path to the next Flow version enforcing
type annotations on most functions (except some inline callbacks).
Used
```
node_modules/.bin/flow codemod annotate-functions-and-classes --write .
```
to add a majority of the types with some hand cleanup when for large
inferred objects that should just be `Fiber` or weird constructs
including `any`.
Suppressed the remaining issues.
Builds on #25918
The old (unstable) mechanism for suspending was to throw a promise. The
purpose of throwing is to interrupt the component's execution, and also
to signal to React that the interruption was caused by Suspense as
opposed to some other error.
A flaw is that throwing is meant to be an implementation detail — if
code in userspace catches the promise, it can lead to unexpected
behavior.
With `use`, userspace code does not throw promises directly, but `use`
itself still needs to throw something to interrupt the component and
unwind the stack.
The solution is to throw an internal error. In development, we can
detect whether the error was caught by a userspace try/catch block and
log a warning — though it's not foolproof, since a clever user could
catch the object and rethrow it later.
The error message includes advice to move `use` outside of the try/catch
block.
I did not yet implement the warning in Flight.
Same as #25537 but for Flight.
I was going to wait to do this later because the temporary
implementation of async components uses some of the same code that
non-used wakables do, but it's not so bad. I just had to inline one bit
of code, which we'll remove when we unify the implementation with `use`.
This extends the scope of the cache and fetch instrumentation using
AsyncLocalStorage for microtasks. This is an intermediate step. It sets
up the dispatcher only once. This is unique to RSC because it uses the
react.shared-subset module for its shared state.
Ideally we should support multiple renderers. We should also have this
take over from an outer SSR's instrumented fetch. We should also be able
to have a fallback to global state per request where AsyncLocalStorage
doesn't exist and then the whole client-side solutions. I'm still
figuring out the right wiring for that so this is a temporary hack.
To derisk the rollout of `use`, and simplify the implementation, this
reverts the yield-to-microtasks behavior for promises that are thrown
directly (as opposed to being unwrapped by `use`).
We may add this back later. However, the plan is to deprecate throwing a
promise directly and migrate all existing Suspense code to `use`, so the
extra code probably isn't worth it.
* Facebook -> Meta in copyright
rg --files | xargs sed -i 's#Copyright (c) Facebook, Inc. and its affiliates.#Copyright (c) Meta Platforms, Inc. and affiliates.#g'
* Manual tweaks
* Print built-in specific error message for toJSON
This is a better message for Date.
Also, format the message to highlight the affected prop.
* Describe error messages using JSX elements in DEV
We don't have access to the grand parent objects on the stack so we stash
them on weakmaps so we can access them while printing error messages.
Might be a bit slow.
* Capitalize Server/Client Component
* Special case errror messages for children of host components
These are likely meant to be text content if they're not a supported object.
* Update error messages
This is a temporary step until we allow Promises everywhere.
Currently this serializes to a Lazy which can then be consumed in this same
slot by the client.
* Missing Hooks
* Remove www forks. These can use __SECRET... instead.
* Move cache to separate dispatcher
These will be available in more contexts than just render.
- method unbinding is no longer supported in Flow for soundness, this added a bunch of suppressions
- Flow now prevents objects to be supertypes of interfaces/classes
ghstack-source-id: d7749cbad8
Pull Request resolved: https://github.com/facebook/react/pull/25412
Similar to Fizz, Flight now supports a return value from the user provided onError option. If a value is returned from onError it will be serialized and provided to the client.
The digest is stashed on the constructed Error on the client as .digest
Follow up to #25084. Implements experimental_use(promise) API in
the Server Components runtime (Flight).
The implementation is much simpler than in Fiber because there is no
state. Even the "state" added in this PR — to track the result of each
promise across attempts — is reset as soon as a component
successfully renders without suspending.
There are also fewer caveats around neglecting to cache a promise
because the state of the promises is preserved even if we switch to a
different task.
Server Components is the primary runtime where this API is intended to
be used.
The last runtime where we need to implement this is the server renderer
(Fizz).
* Fix error handling when the Flight client itself errors
* Serialize references to errors in the error priority queue
It doesn't make sense to emit references to future values at higher pri
than the value that they're referencing.
This ensures that we don't emit hard forward references to values that
don't yet exist.
Add aborting to the Flight Server. This encodes the reason as an "error"
row that gets thrown client side. These are still exposed in prod which
is a follow up we'll still have to do to encode them as digests instead.
The error is encoded once and then referenced by each row that needs to
be updated.
In Fizz this got split into Task and Segment. We don't have a concept of
Segment in Flight yet because we don't inline multiple segments into one
"Row". We just emit one "Row" for each Segment if something suspends.
This makes Flight non-deterministic atm but that's something we'll want to
address.
Regardless, right now, this is more like a Task than a Segment.
* Implements useId hook for Flight server.
The approach for ids for Flight is different from Fizz/Client where there is a need for determinancy. Flight rendered elements will not be rendered on the client and as such the ids generated in a request only need to be unique. However since FLight does support refetching subtrees it is possible a client will need to patch up a part of the tree rather than replacing the entire thing so it is not safe to use a simple incrementing counter. To solve for this we allow the caller to specify a prefix. On an initial fetch it is likely this will be empty but on refetches or subtrees we expect to have a client `useId` provide the prefix since it will guaranteed be unique for that subtree and thus for the entire tree. It is also possible that we will automatically provide prefixes based on a client/Fizz useId on refetches
in addition to the core change I also modified the structure of options for renderToReadableStream where `onError`, `context`, and the new `identifierPrefix` are properties of an Options object argument to avoid the clumsiness of a growing list of optional function arguments.
* defend against useId call outside of rendering
* switch to S from F for Server Component ids
* default to empty string identifier prefix
* Add a test demonstrating that there is no warning when double rendering on the client a server component that used useId
* lints and gates
* [Flight] add support for Lazy components in Flight server
Lazy components suspend until resolved just like in Fizz. Add tests to confirm Lazy works with Shared Components and Client Component references.
* Support Lazy elements
React.Lazy can now return an element instead of a Component. This commit implements support for Lazy elements when server rendering.
* add lazy initialization to resolveModelToJson
adding lazying initialization toResolveModelToJson means we use attemptResolveElement's full logic on whatever the resolved type ends up being. This better aligns handling of misued Lazy types like a lazy element being used as a Component or a lazy Component being used as an element.
* Flight side of server context
* 1 more test
* rm unused function
* flow+prettier
* flow again =)
* duplicate ReactServerContext across packages
* store default value when lazily initializing server context
* .
* better comment
* derp... missing import
* rm optional chaining
* missed feature flag
* React.__SECRET_INTERNALS_DO_NOT_USE_OR_YOU_WILL_BE_FIRED ??
* add warning if non ServerContext passed into useServerContext
* pass context in as array of arrays
* make importServerContext nott pollute the global context state
* merge main
* remove useServerContext
* dont rely on object getters in ReactServerContext and disallow JSX
* add symbols to devtools + rename globalServerContextRegistry to just ContextRegistry
* gate test case as experimental
* feedback
* remove unions
* Lint
* fix oopsies (tests/lint/mismatching arguments/signatures
* lint again
* replace-fork
* remove extraneous change
* rebase
* 1 more test
* rm unused function
* flow+prettier
* flow again =)
* duplicate ReactServerContext across packages
* store default value when lazily initializing server context
* .
* better comment
* derp... missing import
* rm optional chaining
* missed feature flag
* React.__SECRET_INTERNALS_DO_NOT_USE_OR_YOU_WILL_BE_FIRED ??
* add warning if non ServerContext passed into useServerContext
* pass context in as array of arrays
* make importServerContext nott pollute the global context state
* merge main
* remove useServerContext
* dont rely on object getters in ReactServerContext and disallow JSX
* add symbols to devtools + rename globalServerContextRegistry to just ContextRegistry
* gate test case as experimental
* feedback
* remove unions
* Lint
* fix oopsies (tests/lint/mismatching arguments/signatures
* lint again
* replace-fork
* remove extraneous change
* rebase
* reinline
* rebase
* add back changes lost due to rebase being hard
* emit chunk for provider
* remove case for React provider type
* update type for SomeChunk
* enable flag with experimental
* add missing types
* fix flow type
* missing type
* t: any
* revert extraneous type change
* better type
* better type
* feedback
* change import to type import
* test?
* test?
* remove react-dom
* remove react-native-renderer from react-server-native-relay/package.json
* gate change in FiberNewContext, getComponentNameFromType, use switch statement in FlightServer
* getComponentNameFromTpe: server context type gated and use displayName if available
* fallthrough
* lint....
* POP
* lint
This function was modeled after Node streams where write returns a boolean
whether to keep writing or not. I think we should probably switch this
up and read desired size explicitly in appropriate places.
However, in the meantime, we don't have to return a value where we're
not going to use it. So I split this so that we call writeChunkAndReturn
if we're going to return the boolean.
This should help with the compilation so that they can be inlined.
* tests: add failing test to demonstrate bug in ReadableStream implementation
* Re-add reentrancy avoidance
I removed the equivalency of this in #22446. However, I didn't fully
understand the intended semantics of the spec but I understand this better
now.
The spec is not actually recursive. It won't call pull again inside of a
pull. It might not call it inside startWork neither which the original
issue avoided. However, it will call pull if you enqueue to the controller
without filling up the desired size outside any call.
We could avoid that by returning a Promise from pull that we wait to
resolve until we've performed all our pending tasks. That would be the
more idiomatic solution. That's a bit more involved but since we know
understand it, we can readd the reentrancy hack since we have an easy place
to detect it. If anything, it should probably throw or log here otherwise.
I believe this fixes#22772.
This includes the test from #22889 but should ideally have one for Fizz.
Co-authored-by: Josh Larson <josh.larson@shopify.com>
* [RFC] Add onHydrationError option to hydrateRoot
This is not the final API but I'm pushing it for discussion purposes.
When an error is thrown during hydration, we fallback to client
rendering, without triggering an error boundary. This is good because,
in many cases, the UI will recover and the user won't even notice that
something has gone wrong behind the scenes.
However, we shouldn't recover from these errors silently, because the
underlying cause might be pretty serious. Server-client mismatches are
not supposed to happen, even if UI doesn't break from the users
perspective. Ignoring them could lead to worse problems later. De-opting
from server to client rendering could also be a significant performance
regression, depending on the scope of the UI it affects.
So we need a way to log when hydration errors occur.
This adds a new option for `hydrateRoot` called `onHydrationError`. It's
symmetrical to the server renderer's `onError` option, and serves the
same purpose.
When no option is provided, the default behavior is to schedule a
browser task and rethrow the error. This will trigger the normal browser
behavior for errors, including dispatching an error event. If the app
already has error monitoring, this likely will just work as expected
without additional configuration.
However, we can also expose additional metadata about these errors, like
which Suspense boundaries were affected by the de-opt to client
rendering. (I have not exposed any metadata in this commit; API needs
more design work.)
There are other situations besides hydration where we recover from an
error without surfacing it to the user, or notifying an error boundary.
For example, if an error occurs during a concurrent render, it could be
due to a data race, so we try again synchronously in case that fixes it.
We should probably expose a way to log these types of errors, too. (Also
not implemented in this commit.)
* Log all recoverable errors
This expands the scope of onHydrationError to include all errors that
are not surfaced to the UI (an error boundary). In addition to errors
that occur during hydration, this also includes errors that recoverable
by de-opting to synchronous rendering. Typically (or really, by
definition) these errors are the result of a concurrent data race;
blocking the main thread fixes them by prevents subsequent races.
The logic for de-opting to synchronous rendering already existed. The
only thing that has changed is that we now log the errors instead of
silently proceeding.
The logging API has been renamed from onHydrationError
to onRecoverableError.
* Don't log recoverable errors until commit phase
If the render is interrupted and restarts, we don't want to log the
errors multiple times.
This change only affects errors that are recovered by de-opting to
synchronous rendering; we'll have to do something else for errors
during hydration, since they use a different recovery path.
* Only log hydration error if client render succeeds
Similar to previous step.
When an error occurs during hydration, we only want to log it if falling
back to client rendering _succeeds_. If client rendering fails,
the error will get reported to the nearest error boundary, so there's
no need for a duplicate log.
To implement this, I added a list of errors to the hydration context.
If the Suspense boundary successfully completes, they are added to
the main recoverable errors queue (the one I added in the
previous step.)
* Log error with queueMicrotask instead of Scheduler
If onRecoverableError is not provided, we default to rethrowing the
error in a separate task. Originally, I scheduled the task with
idle priority, but @sebmarkbage made the good point that if there are
multiple errors logs, we want to preserve the original order. So I've
switched it to a microtask. The priority can be lowered in userspace
by scheduling an additional task inside onRecoverableError.
* Only use host config method for default behavior
Redefines the contract of the host config's logRecoverableError method
to be a default implementation for onRecoverableError if a user-provided
one is not provided when the root is created.
* Log with reportError instead of rethrowing
In modern browsers, reportError will dispatch an error event, emulating
an uncaught JavaScript error. We can do this instead of rethrowing
recoverable errors in a microtask, which is nice because it avoids any
subtle ordering issues.
In older browsers and test environments, we'll fall back
to console.error.
* Naming nits
queueRecoverableHydrationErrors -> upgradeHydrationErrorsToRecoverable
* Add useId to dispatcher
* Initial useId implementation
Ids are base 32 strings whose binary representation corresponds to the
position of a node in a tree.
Every time the tree forks into multiple children, we add additional bits
to the left of the sequence that represent the position of the child
within the current level of children.
00101 00010001011010101
╰─┬─╯ ╰───────┬───────╯
Fork 5 of 20 Parent id
The leading 0s are important. In the above example, you only need 3 bits
to represent slot 5. However, you need 5 bits to represent all the forks
at the current level, so we must account for the empty bits at the end.
For this same reason, slots are 1-indexed instead of 0-indexed.
Otherwise, the zeroth id at a level would be indistinguishable from
its parent.
If a node has only one child, and does not materialize an id (i.e. does
not contain a useId hook), then we don't need to allocate any space in
the sequence. It's treated as a transparent indirection. For example,
these two trees produce the same ids:
<> <>
<Indirection> <A />
<A /> <B />
</Indirection> </>
<B />
</>
However, we cannot skip any materializes an id. Otherwise, a parent id
that does not fork would be indistinguishable from its child id. For
example, this tree does not fork, but the parent and child must have
different ids.
<Parent>
<Child />
</Parent>
To handle this scenario, every time we materialize an id, we allocate a
new level with a single slot. You can think of this as a fork with only
one prong, or an array of children with length 1.
It's possible for the the size of the sequence to exceed 32 bits, the
max size for bitwise operations. When this happens, we make more room by
converting the right part of the id to a string and storing it in an
overflow variable. We use a base 32 string representation, because 32 is
the largest power of 2 that is supported by toString(). We want the base
to be large so that the resulting ids are compact, and we want the base
to be a power of 2 because every log2(base) bits corresponds to a single
character, i.e. every log2(32) = 5 bits. That means we can lop bits off
the end 5 at a time without affecting the final result.
* Incremental hydration
Stores the tree context on the dehydrated Suspense boundary's state
object so it resume where it left off.
* Add useId to react-debug-tools
* Add selective hydration test
Demonstrates that selective hydration works and ids are preserved even
after subsequent client updates.
* Hoist error codes import to module scope
When this code was written, the error codes map (`codes.json`) was
created on-the-fly, so we had to lazily require from inside the visitor.
Because `codes.json` is now checked into source, we can import it a
single time in module scope.
* Minify error constructors in production
We use a script to minify our error messages in production. Each message
is assigned an error code, defined in `scripts/error-codes/codes.json`.
Then our build script replaces the messages with a link to our
error decoder page, e.g. https://reactjs.org/docs/error-decoder.html/?invariant=92
This enables us to write helpful error messages without increasing the
bundle size.
Right now, the script only works for `invariant` calls. It does not work
if you throw an Error object. This is an old Facebookism that we don't
really need, other than the fact that our error minification script
relies on it.
So, I've updated the script to minify error constructors, too:
Input:
Error(`A ${adj} message that contains ${noun}`);
Output:
Error(formatProdErrorMessage(ERR_CODE, adj, noun));
It only works for constructors that are literally named Error, though we
could add support for other names, too.
As a next step, I will add a lint rule to enforce that errors written
this way must have a corresponding error code.
* Minify "no fallback UI specified" error in prod
This error message wasn't being minified because it doesn't use
invariant. The reason it didn't use invariant is because this particular
error is created without begin thrown — it doesn't need to be thrown
because it's located inside the error handling part of the runtime.
Now that the error minification script supports Error constructors, we
can minify it by assigning it a production error code in
`scripts/error-codes/codes.json`.
To support the use of Error constructors more generally, I will add a
lint rule that enforces each message has a corresponding error code.
* Lint rule to detect unminified errors
Adds a lint rule that detects when an Error constructor is used without
a corresponding production error code.
We already have this for `invariant`, but not for regular errors, i.e.
`throw new Error(msg)`. There's also nothing that enforces the use of
`invariant` besides convention.
There are some packages where we don't care to minify errors. These are
packages that run in environments where bundle size is not a concern,
like react-pg. I added an override in the ESLint config to ignore these.
* Temporarily add invariant codemod script
I'm adding this codemod to the repo temporarily, but I'll revert it
in the same PR. That way we don't have to check it in but it's still
accessible (via the PR) if we need it later.
* [Automated] Codemod invariant -> Error
This commit contains only automated changes:
npx jscodeshift -t scripts/codemod-invariant.js packages --ignore-pattern="node_modules/**/*"
yarn linc --fix
yarn prettier
I will do any manual touch ups in separate commits so they're easier
to review.
* Remove temporary codemod script
This reverts the codemod script and ESLint config I added temporarily
in order to perform the invariant codemod.
* Manual touch ups
A few manual changes I made after the codemod ran.
* Enable error code transform per package
Currently we're not consistent about which packages should have their
errors minified in production and which ones should.
This adds a field to the bundle configuration to control whether to
apply the transform. We should decide what the criteria is going
forward. I think it's probably a good idea to minify any package that
gets sent over the network. So yes to modules that run in the browser,
and no to modules that run on the server and during development only.
* Pass in Destination lazily in startFlowing instead of createRequest
* Delay fatal errors until we have a destination to forward them to
* Flow can now be inferred by whether there's a destination set
We can drop the destination when we're not flowing since there's nothing to
write to.
Fatal errors now close once flowing starts back up again.
* Defer fatal errors in Flight too
* Remove reentrant check from Fizz/Flight
* Make startFlowing explicit in Flight
This is already an explicit call in Fizz. This moves flowing to be explicit.
That way we can avoid calling it in start() for web streams and therefore
avoid the reentrant call.
* Add regression test
This test doesn't actually error due to the streams polyfill not behaving
like Chrome but rather according to spec.
* Update the Web Streams polyfill
Not that we need this but just in case there are differences that are fixed.
* Revise ESLint rules for string coercion
Currently, react uses `'' + value` to coerce mixed values to strings.
This code will throw for Temporal objects or symbols.
To make string-coercion safer and to improve user-facing error messages,
This commit adds a new ESLint rule called `safe-string-coercion`.
This rule has two modes: a production mode and a non-production mode.
* If the `isProductionUserAppCode` option is true, then `'' + value`
coercions are allowed (because they're faster, although they may
throw) and `String(value)` coercions are disallowed. Exception:
when building error messages or running DEV-only code in prod
files, `String()` should be used because it won't throw.
* If the `isProductionUserAppCode` option is false, then `'' + value`
coercions are disallowed (because they may throw, and in non-prod
code it's not worth the risk) and `String(value)` are allowed.
Production mode is used for all files which will be bundled with
developers' userland apps. Non-prod mode is used for all other React
code: tests, DEV blocks, devtools extension, etc.
In production mode, in addiiton to flagging `String(value)` calls,
the rule will also flag `'' + value` or `value + ''` coercions that may
throw. The rule is smart enough to silence itself in the following
"will never throw" cases:
* When the coercion is wrapped in a `typeof` test that restricts to safe
(non-symbol, non-object) types. Example:
if (typeof value === 'string' || typeof value === 'number') {
thisWontReport('' + value);
}
* When what's being coerced is a unary function result, because unary
functions never return an object or a symbol.
* When the coerced value is a commonly-used numeric identifier:
`i`, `idx`, or `lineNumber`.
* When the statement immeidately before the coercion is a DEV-only
call to a function from shared/CheckStringCoercion.js. This call is a
no-op in production, but in DEV it will show a console error
explaining the problem, then will throw right after a long explanatory
code comment so that debugger users will have an idea what's going on.
The check function call must be in the following format:
if (__DEV__) {
checkXxxxxStringCoercion(value);
};
Manually disabling the rule is usually not necessary because almost all
prod use of the `'' + value` pattern falls into one of the categories
above. But in the rare cases where the rule isn't smart enough to detect
safe usage (e.g. when a coercion is inside a nested ternary operator),
manually disabling the rule will be needed.
The rule should also be manually disabled in prod error handling code
where `String(value)` should be used for coercions, because it'd be
bad to throw while building an error message or stack trace!
The prod and non-prod modes have differentiated error messages to
explain how to do a proper coercion in that mode.
If a production check call is needed but is missing or incorrect
(e.g. not in a DEV block or not immediately before the coercion), then
a context-sensitive error message will be reported so that developers
can figure out what's wrong and how to fix the problem.
Because string coercions are now handled by the `safe-string-coercion`
rule, the `no-primitive-constructor` rule no longer flags `String()`
usage. It still flags `new String(value)` because that usage is almost
always a bug.
* Add DEV-only string coercion check functions
This commit adds DEV-only functions to check whether coercing
values to strings using the `'' + value` pattern will throw. If it will
throw, these functions will:
1. Display a console error with a friendly error message describing
the problem and the developer can fix it.
2. Perform the coercion, which will throw. Right before the line where
the throwing happens, there's a long code comment that will help
debugger users (or others looking at the exception call stack) figure
out what happened and how to fix the problem.
One of these check functions should be called before all string coercion
of user-provided values, except when the the coercion is guaranteed not
to throw, e.g.
* if inside a typeof check like `if (typeof value === 'string')`
* if coercing the result of a unary function like `+value` or `value++`
* if coercing a variable named in a whitelist of numeric identifiers:
`i`, `idx`, or `lineNumber`.
The new `safe-string-coercion` internal ESLint rule enforces that
these check functions are called when they are required.
Only use these check functions in production code that will be bundled
with user apps. For non-prod code (and for production error-handling
code), use `String(value)` instead which may be a little slower but will
never throw.
* Add failing tests for string coercion
Added failing tests to verify:
* That input, select, and textarea elements with value and defaultValue
set to Temporal-like objects which will throw when coerced to string
using the `'' + value` pattern.
* That text elements will throw for Temporal-like objects
* That dangerouslySetInnerHTML will *not* throw for Temporal-like
objects because this value is not cast to a string before passing to
the DOM.
* That keys that are Temporal-like objects will throw
All tests above validate the friendly error messages thrown.
* Use `String(value)` for coercion in non-prod files
This commit switches non-production code from `'' + value` (which
throws for Temporal objects and symbols) to instead use `String(value)`
which won't throw for these or other future plus-phobic types.
"Non-produciton code" includes anything not bundled into user apps:
* Tests and test utilities. Note that I didn't change legacy React
test fixtures because I assumed it was good for those files to
act just like old React, including coercion behavior.
* Build scripts
* Dev tools package - In addition to switching to `String`, I also
removed special-case code for coercing symbols which is now
unnecessary.
* Add DEV-only string coercion checks to prod files
This commit adds DEV-only function calls to to check if string coercion
using `'' + value` will throw, which it will if the value is a Temporal
object or a symbol because those types can't be added with `+`.
If it will throw, then in DEV these checks will show a console error
to help the user undertsand what went wrong and how to fix the
problem. After emitting the console error, the check functions will
retry the coercion which will throw with a call stack that's easy (or
at least easier!) to troubleshoot because the exception happens right
after a long comment explaining the issue. So whether the user is in
a debugger, looking at the browser console, or viewing the in-browser
DEV call stack, it should be easy to understand and fix the problem.
In most cases, the safe-string-coercion ESLint rule is smart enough to
detect when a coercion is safe. But in rare cases (e.g. when a coercion
is inside a ternary) this rule will have to be manually disabled.
This commit also switches error-handling code to use `String(value)`
for coercion, because it's bad to crash when you're trying to build
an error message or a call stack! Because `String()` is usually
disallowed by the `safe-string-coercion` ESLint rule in production
code, the rule must be disabled when `String()` is used.
Recoil uses useMutableSource behind a flag. I thought this was fine
because Recoil isn't used in any concurrent roots, so the behavior
would be the same, but it turns out that it is used by concurrent
roots in a few places.
I'm not expecting it to be hard to migrate to useSyncExternalStore, but
to de-risk the change I'm going to roll it out gradually with a flag. In
the meantime, I've added back the useMutableSource API.
* Remove redundant initial of isArray (#21163)
* Reapply prettier
* Type the isArray function with refinement support
This ensures that an argument gets refined just like it does if isArray is
used directly.
I'm not sure how to express with just a direct reference so I added a
function wrapper and confirmed that this does get inlined properly by
closure compiler.
* A few more
* Rename unit test to internal
This is not testing a bundle.
Co-authored-by: Behnam Mohammadi <itten@live.com>
* Add onError option to Flight Server
The callback is called any time an error is generated in a server component.
This allows it to be logged on a server if needed. It'll still be rethrown
on the client so it can be logged there too but in case it never reaches
the client, here's a way to make sure it doesn't get lost.
* Add fatal error handling
* Remove react/unstable_cache
We're probably going to make it available via the dispatcher. Let's remove this for now.
* Add readContext() to the dispatcher
On the server, it will be per-request.
On the client, there will be some way to shadow it.
For now, I provide it on the server, and throw on the client.
* Use readContext() from react-fetch
This makes it work on the server (but not on the client until we implement it there.)
Updated the test to use Server Components. Now it passes.
* Fixture: Add fetch from a Server Component
* readCache -> getCacheForType<T>
* Add React.unstable_getCacheForType
* Add a feature flag
* Fix Flow
* Add react-suspense-test-utils and port tests
* Remove extra Map lookup
* Unroll async/await because build system
* Add some error coverage and retry
* Add unstable_getCacheForType to Flight entry
* Encode Symbols as special rows that can be referenced by models
If a symbol was extracted from Symbol.for(...) then we can reliably
recreate the same symbol on the client.
S123:"react.suspense"
M456:{mySymbol: '$123'}
This doesn't suffer from the XSS problem because you have to write actual
code to create one of these symbols. That problem is only a problem because
values pass through common other usages of JSON which are not secure.
Since React encodes its built-ins as symbols, we can now use them as long
as its props are serializable. Like Suspense.
* Refactor resolution to avoid memo hack
Going through createElement isn't quite equivalent for ref and key in props.
* Reuse symbol ids that have already been written earlier in the stream
This ensures that if this server component was the child of a client
component that has an error boundary, it doesn't trigger the error until
this gets rendered so it happens as deep as possible.
* Remove Blocks
* Remove Flight Server Runtime
There's no need for this now that the JSResource is part of the bundler
protocol. Might need something for Webpack plugin specifically later.
* Devtools
This now means that if a server component suspends, its value becomes a
React.lazy object. I.e. the element that rendered the server component
gets replaced with a lazy node.
As of #19033 lazy objects can be rendered in the node position. This allows
us to suspend at the location of the server component while we're waiting
on its content.
Now server components has the same capabilities as Blocks to progressively
reveal its content.
These references are currently transformed into React.lazy values. We can use these in
React positions like element type or node position.
This could be expanded to a more general concept like Suspensey Promises, asset references or JSResourceReferences.
For now it's only used in React Element type position.
The purpose of these is to let you suspend deeper in the tree.
* Refactor Flight to require a module reference to be brand checked
This exposes a host environment (bundler) specific hook to check if an
object is a module reference. This will be used so that they can be passed
directly into Flight without needing additional wrapper objects.
* Emit module references as a special type of value
We already have JSON and errors as special types of "rows". This encodes
module references as a special type of row value. This was always the
intention because it allows those values to be emitted first in the stream
so that as a large models stream down, we can start preloading as early
as possible.
We preload the module when they resolve but we lazily require them as they
are referenced.
* Emit module references where ever they occur
This emits module references where ever they occur. In blocks or even
directly in elements.
* Don't special case the root row
I originally did this so that a simple stream is also just plain JSON.
However, since we might want to emit things like modules before the root
module in the stream, this gets unnecessarily complicated. We could add
this back as a special case if it's the first byte written but meh.
* Update the protocol
* Add test for using a module reference as a client component
* Relax element type check
Since Flight now accepts a module reference as returned by any bundler
system, depending on the renderer running. We need to drastically relax
the check to include all of them. We can add more as we discover them.
* Move flow annotation
Seems like our compiler is not happy with stripping this.
* Some bookkeeping bug
* Can't use the private field to check
* Improve error message by expanding the object in question
* Don't warn for key/ref getters
* Error if refs are passed in server components or to client components
* Lazily initialize models as they're read intead of eagerly when received
This ensures that we don't spend CPU cycles processing models that we're
not going to end up rendering.
This model will also allow us to suspend during this initialization if
data is not yet available to satisfy the model.
* Refactoring carefully to ensure bundles still compile to something optimal
* Remove generic from Response
The root model needs to be cast at one point or another same as othe
chunks. So we can parameterize the read instead of the whole Response.
* Read roots from the 0 key of the map
The special case to read the root isn't worth the field and code.
* Store response on each Chunk
Instead of storing it on the data tuple which is kind of dynamic, we store
it on each Chunk. This uses more memory. Especially compared to just making
initializeBlock a closure, but overall is simpler.
* Rename private fields to underscores
Response objects are exposed.
* Encode server components as delayed references
This allows us to stream in server components one after another over the
wire. It also allows parallelizing their fetches and resuming only the
server component instead of the whole parent block.
This doesn't yet allow us to suspend deeper while waiting on this content
because we don't have "lazy elements".
* Enable prefer-const rule
Stylistically I don't like this but Closure Compiler takes advantage of
this information.
* Auto-fix lints
* Manually fix the remaining callsites
This is equivalent to the jsx-runtime in that this is what the compiled
output on the server is supposed to target.
It's really just the same code for all the different Flights, but they
have different types in their arguments so each one gets their own entry
point. We might use this to add runtime warnings per entry point.
Unlike the client-side React.block call this doesn't provide the factory
function that curries the load function. The compiler is expected to wrap
this call in the currying factory.
* Resolve Server-side Blocks instead of Components
React elements should no longer be used to extract arbitrary data but only
for prerendering trees.
Blocks are used to create asynchronous behavior.
* Resolve Blocks in the Client
* Tests
* Bug fix relay JSON traversal
It's supposed to pass the original object and not the new one.
* Lint
* Move Noop Module Test Helpers to top level entry points
This module has shared state. It needs to be external from builds.
This lets us test the built versions of the Noop renderer.
This replaces the HTML renderer with instead resolving host elements into
arrays tagged with the react.element symbol. These turn into proper
React Elements on the client.
The symbol is encoded as the magical value "$". This has security implications
so this special value needs to remain escaped for other strings.
We could just encode the element as {$$typeof: "$", key: key props: props}
but that's a lot more bytes. So instead I encode it as:
["$", key, props] and then convert it back.
It would be nicer if React's reconciler could just accept these tuples.
* Add ReactFlightServerConfig intermediate
This just forwards to the stream version of Flight which is itself forked
between Node and W3C streams.
The dom-relay goes directly to the Relay config though which allows it to
avoid the stream part of Flight.
* Separate streaming protocol into the Stream config
* Split streaming parts into the ReactFlightServerConfigStream
This decouples it so that the Relay implementation doesn't have to encode
the JSON to strings. Instead it can be fed the values as JSON objects and
do its own encoding.
* Split FlightClient into a basic part and a stream part
Same split as the server.
* Expose lower level async hooks to Relay
This requires an external helper file that we'll wire up internally.
* Rename to clarify that it's client-only
* Rename FizzStreamer to FizzServer for consistency
* Rename react-flight to react-client/flight
For consistency with react-server. Currently this just includes flight
but it could be expanded to include the whole reconciler.
* Add Relay Flight Build
* Rename ReactServerHostConfig to ReactServerStreamConfig
This will be the config specifically for streaming purposes.
There will be other configs for other purposes.
* Update Flow to 0.84
* Fix violations
* Use inexact object syntax in files from fbsource
* Fix warning extraction to use a modern parser
* Codemod inexact objects to new syntax
* Tighten types that can be exact
* Revert unintentional formatting changes from codemod
* Return whether to keep flowing in Host config
* Emit basic chunk based streaming in the Flight server
When something suspends a new chunk is created.
* Add reentrancy check
The WHATWG API is designed to be pulled recursively.
We should refactor to favor that approach.
* Basic streaming Suspense support on the client
* Add basic suspense in example
* Add comment describing the protocol that the server generates
* Rename ReactFlightStreamer -> ReactFlightServer
* Unify Browser/Node stream tests into one file and use the client reader
* Defer to the actual ReactDOM for HTML rendering for now
This will need to use a variant of Fizz to do inline SSR in Flight.
However, I don't want to build the whole impl right now but also don't
want to exclude the use case yet. So I outsource it to the existing
renderer. Ofc, this doesn't work with Suspense atm.
