# Reading NDJSON stream in the browser

**URL:** https://forum.kirupa.com/t/reading-ndjson-stream-in-the-browser/646715
**Category:** programming
**Created:** [March 21, 2021, 2:28am UTC](https://forum.kirupa.com/t/reading-ndjson-stream-in-the-browser/646715 "2021-03-21T02:28:03Z")
**Posts on this page:** 1
**Page:** 1

<div class="post-metadata">

### Author: ![steve.mills](https://yyz1.discourse-cdn.com/flex011/user_avatar/forum.kirupa.com/steve.mills/32/14961_2.png) [@steve.mills](https://forum.kirupa.com/u/steve.mills)
#### Post date: [March 21, 2021, 2:28am UTC](https://forum.kirupa.com/t/reading-ndjson-stream-in-the-browser/646715/1 "2021-03-21T02:28:03Z")

</div>

I recently was trying to stream some NDJSON to a service worker and also into a comment feed.  
It was a bit of an issue finding just a NDJSON reader as anything I could find was a reader → new readableStream pipe. (returns a new readable stream).  
There was Jake Archibald’s XMLHTTP → NDJSON but I wanted to use fetch, so I re-did it with fetch and an async generator (I like generators).

> [streaming-html/xhr-ndjson.js at master · jakearchibald/streaming-html · GitHub](https://github.com/jakearchibald/streaming-html/blob/master/xhr-ndjson.js)

The reader seems to work fine,  
It takes 13-\> 21ms (performance.now()) for the function to run(after response) and stream → encode → json parse → template → and insert 1000 basic separate JSON objects (each line is a new JSON object).

```
{"id":1,"name":"Alice"}

{"id":2,"name":"Bob"}

{"id":3,"name":"Carol"}

```

The browser then takes 100 - 300ms to paint/ layout ect of the 1000 basic objects.

If anyone is in the same boat and needs to read an NDJSON stream and insert into the DOM this will work.

```
const readNDJSONStream = function(url, element) {
    return new Promise(async function(resolve, reject) {
    const decoder = new TextDecoder();
    let el = document.querySelector(element);
    let buff =[];
    let pos = 1; 
    const response = await fetch(url);
    const reader = response.body.getReader();
    
const pump = async function*(){
    while (true) {
        const { done, value } = await reader.read();
        yield value;
    if (done) { pos = 0; return yield value}
    }
  }
const pipe = async function() {
  for await (const i of pump()) {
    try { 
        buff += decoder.decode(i, {stream: true});
        buff = buff.split("\n"); 
        let chunk = buff.splice(0, (buff.length -pos))
      for(let i of chunk){
          let obj = JSON.parse(i);
          let temp = `<h3>id: ${obj.id}, name: ${obj.name}</h3>`;
          el.insertAdjacentHTML('beforeend', temp)
        }
    }
    catch (error) {
        reject(Error(`threw at ${decoder.decode(i)}`)); return 
        }
    }
  }();
resolve();
 })
}

```

Obviously this is all inspired by Jake and Thomas Steiner.

> [स्ट्रीम—खास गाइड &nbsp;|&nbsp; Articles &nbsp;|&nbsp; web.dev](https://web.dev/streams/)

If anyone has any suggestions on how to make this better whilst still performant that would be much appreciated. 🙂
