Server Streaming

Context

Server Streaming is a public demo of real-time data streaming from a .NET 8 Web API to a Next.js 15 client. It puts two server techniques beside the same browser pattern so their responsibilities are easy to compare.

One endpoint returns IAsyncEnumerable<DataItem> and uses the framework’s built-in JSON data streaming behavior. The other writes JSON manually with Response.WriteAsync(). On the client, Fetch exposes the response body as a ReadableStream that can be decoded and processed while the request remains open.

Problem

Large or gradually produced datasets do not always fit a wait-for-everything response model. Holding the full result delays the first useful item and can create avoidable buffering.

Incremental delivery changes the problem rather than removing it. The server must notice cancellation, the browser must handle chunks that do not align with logical records, and the interface must avoid replacing network buffering with an ever-growing list of rendered rows.

Role and responsibilities

The project creator designed and implemented the server endpoints, browser stream reader, controls, interruption handling, live statistics, and recent-row display. The project also compares framework-managed and manually framed streamed responses.

Constraints

• JSON values can be split across network chunks, so a chunk cannot be treated as a complete record by default.
• Cancellation must travel from the browser interaction to the request and into the backend producer.
• Rendering every received item indefinitely would make the interface progressively heavier.
• The demonstration needs to make pacing observable without tying the architecture to one fixed workload.

Approach

The .NET 8 API exposes both endpoint styles for the same goal. IAsyncEnumerable<DataItem> keeps code focused on producing one item at a time while the framework handles JSON serialization. The manual endpoint uses Response.WriteAsync() to make response framing and flushing explicit.

The Next.js 15 client calls an endpoint with Fetch, reads the response body through ReadableStream, decodes incoming bytes, and extracts complete values. Start, stop, and clear controls make the request lifecycle visible. Item-count and delay inputs show stream duration and pacing without changing source code.

The interface reports live statistics but keeps only a bounded set of recent rows in the table. That separates the total amount processed from the amount retained for immediate display.

Important decisions

Show both server abstractions

The IAsyncEnumerable endpoint demonstrates the concise path when framework serialization matches the response contract. Manual JSON data streaming demonstrates the additional control, along with the additional correctness burden, of writing the response directly.

Treat parsing as an incremental operation

The browser retains incomplete text between reads and emits only complete values. This avoids assuming that transport chunks align with JSON boundaries.

Connect stop to cancellation

Stopping is not only a visual state change. The client interrupts the active fetch, and backend cancellation tokens allow production work to stop when the request is no longer useful.

Bound the rendered history

Recent-row rendering is intentionally bounded. The demonstration can count and process a longer stream while keeping DOM growth controlled and the latest data visible.

Outcome

The project provides a runnable comparison of two .NET 8 streaming styles and a browser client that consumes output incrementally. It demonstrates pacing, live progress, interruption, error handling, and bounded presentation without buffering the complete dataset first.

Lessons learned

Streaming is an end-to-end contract. An incremental server producer does not deliver the intended benefit if an intermediary buffers the response or the client waits for completion before parsing it.

Cancellation and memory behavior also need clear design. Propagating interruption prevents abandoned work, while a bounded UI prevents unbounded rendering. The best endpoint style depends on how much control the response format needs. Framework-managed streaming reduces code, while manual streaming needs careful framing and failure handling.