Streaming

Send the first bytes of a page before all of it is ready: loading.tsx and Suspense, cached PPR shells, streamed route handler bodies and Server-Sent Events.

Streaming lets the browser start painting while the slow parts of a response are still being produced. GioJS streams in four places, all through the same Rust server, which forwards each chunk the moment the Node worker writes it:

Streaming pages

A page renders with React's streaming renderer. When a component suspends - it reads a pending promise with use(), or is a React.lazy component still loading - React sends everything around it at once with the nearest Suspense fallback in its place, and streams the real content into the same response when it is ready. The browser shows the fallback, then swaps the content in without a request of its own.

Whether a page response streams depends on whether it can be shared:

• Pages without revalidate (and personalized pages, which are never cached) stream: chunked, Cache-Control: private, no-cache, X-Gio-Cache: bypass.
• Pages with revalidate render completely before anything is sent, so the whole page can be stored in the page cache and served from it. Suspense still works there; it just resolves on the server first. To stream a cached page, use partial prerendering.
• HEAD requests, static export, and a server with a Node plugin that has an onResponse hook (it needs the whole body) render completely too.

Whole-page loading UI with loading.tsx

A loading.tsx wraps everything below its folder in a Suspense boundary. While the page suspends, the layouts above the folder and the loading UI are sent at once:

export default function Loading() {
  return <p>Loading dashboard…</p>;
}

import React, { use } from 'react';
import { cached, loadStats } from '../../lib/stats';

export default function Dashboard(): React.JSX.Element {
  // Suspends the whole page: dashboard/loading.tsx shows until it resolves.
  const stats = use(cached('stats', loadStats));
  return (
    <main>
      <h1>Dashboard</h1>
      <p>{stats.orders} orders today</p>
    </main>
  );
}

export interface Stats { orders: number }

export async function loadStats(): Promise<Stats> {
  const response = await fetch('https://stats.example.com/today');
  return (await response.json()) as Stats;
}

// One promise per key for a few seconds. When a component suspends in the
// browser, React renders again from its Suspense boundary and must get the
// same promise back - a new one every render would suspend forever.
const pending = new Map<string, Promise<unknown>>();
export function cached<T>(key: string, load: () => Promise<T>, ttlMs = 5000): Promise<T> {
  let promise = pending.get(key) as Promise<T> | undefined;
  if (promise === undefined) {
    promise = load();
    pending.set(key, promise);
    setTimeout(() => pending.delete(key), ttlMs);
  }
  return promise;
}

getServerSideProps runs before rendering starts, so the loading UI does not cover it: if your slow work is there, the page waits for it before the first byte. Move slow, non-essential reads into the component tree to stream them.

Streaming one part of a page

Wrap only the slow part in your own <Suspense> and the rest of the page goes out with the first chunk. Start the work in the page and read it in a child inside the boundary - the page itself never suspends, so the promise is created once:

import React, { Suspense, use } from 'react';
import { loadActivity } from '../../lib/activity';

export default function Orders(): React.JSX.Element {
  const activity = loadActivity(); // started now, read below
  return (
    <main>
      <h1>Orders</h1>
      <Suspense fallback={<p>Loading activity…</p>}>
        <Activity items={activity} />
      </Suspense>
    </main>
  );
}

function Activity({ items }: { items: Promise<string[]> }): React.JSX.Element {
  const list = use(items);
  return <ul>{list.map((item) => <li key={item}>{item}</li>)}</ul>;
}

Sibling boundaries stream independently, each as soon as its data arrives; nested boundaries reveal detail step by step. Every boundary the server could not finish before the response ended is finished by React in the browser.

Data in the browser

GioJS has no React Server Components: every page is server-rendered and then hydrated, so a component that suspends runs again in the browser during hydration. The data function it calls must work there too - a fetch to a public API or to one of your own route handlers, not a database client. Only GIO_PUBLIC_* variables exist in the browser, so build such URLs from one of those. Data that must stay on the server belongs in getServerSideProps, which runs only there.

Status codes and errors

The status code and headers go out with the first chunk, so they are decided by what happened before the first suspension:

• A page that throws, or calls notFound(), before it suspends still answers 500 or 404 with the nearest error.tsx or not-found.tsx.
• After it has suspended the 200 is on its way. A later error is handled like in any Suspense boundary: React renders that segment in the browser, and if it fails there too, the nearest error.tsx boundary shows it. Production responses carry only a digest (data-dgst), never the message.
• A redirect() from getServerSideProps runs before rendering, so it is a real redirect - except on a cached PPR shell, which was sent first (see below).

See Error Handling for the details.

Partial prerendering

A cached page is fast but the same for everyone; a streamed page is personal but renders on every request. Partial prerendering (PPR) combines them: everything before the first pending Suspense boundary - the shell - is stored in the page cache and sent instantly, and the Suspense content (the holes) renders per request and streams in behind it. Opt in with shell = 'cache' next to revalidate:

import React, { Suspense, use } from 'react';
import type { GsspContext } from '@gio.js/core';
import { cartFor, type CartItem } from '../../lib/cart';

export const revalidate = 60;
export const shell = 'cache';

export async function getServerSideProps(ctx: GsspContext) {
  // Runs for every request, with that visitor's cookies.
  return { props: { who: ctx.cookies['who'] ?? 'guest' } };
}

export default function Storefront({ who }: { who: string }): React.JSX.Element {
  const cart = cartFor(who);
  return (
    <main>
      <h1>Storefront</h1>{/* shell: cached, the same for everyone */}
      <Suspense fallback={<p>Loading your cart…</p>}>
        <Cart cart={cart} />{/* hole: rendered per request */}
      </Suspense>
    </main>
  );
}

function Cart({ cart }: { cart: Promise<CartItem[]> }): React.JSX.Element {
  const items = use(cart);
  return <p>{items.length} items in your cart</p>;
}

The shell must render the same bytes for every visitor; only content inside a boundary that actually suspends may depend on the visitor. A loading.tsx is a shell edge too. Responses say what happened in X-Gio-Cache: ppr; shell=stored, ppr; shell=hit or ppr; shell=stale; .... The contract, what is checked before a shell is stored, and how a per-visitor redirect reaches a visitor after the shell was sent are in Caching Layers.

Streaming route handlers

A route.ts handler that returns a Response whose body is a ReadableStream streams it chunk by chunk: model output token by token, a large export, a generated file. Produce chunks in pull() so a slow client slows the producer down, and stop in cancel():

// A CSV export written row by row: the client starts receiving it at once.
async function* rows(): AsyncGenerator<string> {
  yield 'id,total\n';
  for (let id = 1; id <= 5; id++) {
    await new Promise((resolve) => setTimeout(resolve, 300)); // a database page
    yield `${id},${id * 10}\n`;
  }
}

export function GET(): Response {
  const source = rows();
  const encoder = new TextEncoder();
  const body = new ReadableStream<Uint8Array>({
    async pull(controller) {
      const { done, value } = await source.next();
      if (done) controller.close();
      else controller.enqueue(encoder.encode(value));
    },
    async cancel() {
      await source.return(undefined); // the client went away: stop reading
    },
  });
  return new Response(body, {
    headers: {
      'content-type': 'text/csv; charset=utf-8',
      'content-disposition': 'attachment; filename="report.csv"',
    },
  });
}

$ curl -N http://localhost:3000/api/report     # rows arrive 300 ms apart
id,total
1,10
2,20
...

• Status, headers and every Set-Cookie are sent before the first chunk, so they must be final when you return the Response.
• A body that is already complete and at most 1 MiB crosses from the worker in one piece; anything longer, or still being produced, is forwarded chunk by chunk.
• When the client stops reading, the server stops pulling once about 1 MiB is waiting for it, so a large download never piles up in memory.
• A streamed route body has no idle limit: it ends when you close the stream or the client leaves. The handler must still return its Response within [server] render_timeout_secs (30 seconds).
• A text/html body is passed through as written: GioJS injects its head scripts into page renders only.

More in Route Handlers.

Server-Sent Events

GioEventStream from @gio.js/core is the shortest way to push events to a browser. Its callback gets a stream with send(data, event?, id?) and close(), and returns a cleanup function that runs when the client disconnects:

import { GioEventStream } from '@gio.js/core';

export function GET(): GioEventStream {
  return new GioEventStream((stream) => {
    let n = 0;
    const timer = setInterval(() => {
      n += 1;
      stream.send({ n, at: Date.now() }, 'tick', String(n));
    }, 1000);
    return () => clearInterval(timer); // the client disconnected
  });
}

$ curl -N http://localhost:3000/api/ticker
id: 1
event: tick
data: {"n":1,"at":1791392165346}

id: 2
...

import React, { useEffect, useState } from 'react';

export function Ticker(): React.JSX.Element {
  const [n, setN] = useState(0);
  useEffect(() => {
    const source = new EventSource('/api/ticker');
    source.addEventListener('tick', (event) => setN(JSON.parse(event.data).n));
    return () => source.close();
  }, []);
  return <p>{n} ticks</p>;
}

• data is sent as JSON on one data: line; line breaks in event and id are stripped, so neither can inject fields.
• Event streams are never compressed (that would hold events back), get Cache-Control: no-cache, and are not bounded by render_timeout_secs.
• send() does not wait for the client. While more than 8 MiB is waiting to reach the server, further events are dropped with a warning, so a stalled client cannot exhaust the worker's memory. Use a ReadableStream body with a text/event-stream content type when every event must arrive.
• An open stream counts toward [server] max_connections and keeps its worker busy for load balancing until it ends.

For two-way messages, use WebSockets.

What happens at shutdown

On SIGTERM or Ctrl+C the server stops accepting connections, closes idle keep-alive connections, and gives what is in flight up to 8 seconds to finish:

• Event streams (GioEventStream and any route handler answering text/event-stream) are ended cleanly the moment shutdown starts, and their producers are cancelled in the worker. Browsers' EventSource reconnects on its own - to the new process once it is up.
• Page renders and other streamed bodies (downloads, exports) are left to finish. One still running after the drain has its connection reset, so the client sees a failed transfer instead of a short file that looks complete.
• WebSockets are closed with 1001.

Give your process manager at least the drain time plus the workers' shutdown grace before it kills the server - the Deploying recipes do.

When chunks are held back

• Reverse proxies that buffer responses hold every chunk until the end. With nginx, set proxy_buffering off for the app (see the nginx recipe).
• CDNs differ: some pass chunks through, some buffer. Cached pages are not streamed anyway; check streamed ones through the CDN.
• Compression (gzip or Brotli) applies to streamed pages and route bodies as they stream - their length is unknown, so [compression] min_size_bytes does not hold them back. Event streams are never compressed.
• Checking it: curl -N prints chunks as they arrive. A streamed page answers with X-Gio-Cache: bypass (or ppr; ...), and its Suspense fallback appears in the HTML before the content. Over HTTP/1.1, a response rendered or buffered whole carries a Content-Length unless it is compressed, so a transfer-encoding: chunked response without Content-Encoding streamed. A compressed response (curl --compressed, every browser) is chunked whether it streamed or not: look at X-Gio-Cache and where the fallback sits in the HTML instead.

Related

• loading.tsx and Loading UI
• Partial prerendering and the shell export
• Route Handlers: streaming responses
• GioEventStream
• Error Handling: streaming
• [server] render_timeout_secs

Version history