javascript / expert
Snippet
Manual Stream Backpressure Management
When writing data to a stream faster than the destination can consume it, memory buffers swell. This pattern manually monitors the return value of write() and pauses execution until the 'drain' event is emitted, ensuring stable memory usage.
snippet.js
javascript
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
const { once } = require('events');const fs = require('fs');async function writeLargeData(stream, dataGen) {for (const chunk of dataGen) {const canWrite = stream.write(chunk);if (!canWrite) {await once(stream, 'drain');}}stream.end();}const writer = fs.createWriteStream('output.log');writeLargeData(writer, ['large', 'data', 'chunks']);
nodejs
Breakdown
1
const canWrite = stream.write(chunk);
Returns false if the internal buffer exceeds the highWaterMark threshold.
2
await once(stream, 'drain');
Uses the events.once helper to pause the async loop until the buffer is cleared.