Fast treatment of data from server

chrisdone · June 23, 2021, 3:40pm

Hi all,

In an app I’m working on, I’ve noticed that if I parse a large JSON blob, e.g. 19MB that parsing into PureScript is slow. The data is basically a table of data of about 10k rows.

It takes 100ms for the browser to parse this JSON blob into a JS object, which is fine. A JavaScript app would be ready to work with this data promptly.

It takes about 5s for foreign-generic to consume that into a PureScript value. Too slow.

Now, I intend to make my data schema leaner to substantially reduce the redundancy in the JSON output. I also will be paginating data from the server, so I won’t necessarily be sending more than N thousand rows at a time. However, the baseline performance is very important too.

However, there are additional considerations, such as:

I will only be presenting about 10-30 rows at a time. With Halogen, I can lazily scroll through the table on-demand, only showing 10-30 rows at once and only rendering that much. (That’s how Google Sheets works. It generates a view of only the rows currently scrolled in view.) But that assumes I’ve already loaded the data.
Unlike Haskell, I can’t lazily load the data on-demand like with aeson, which would lex the document (JSON.parse equivalent) and then only allocate objects when I ask for it.

I’ve considered an API like:

foreign import data OutputDocument :: Type

foreign import outputDocumentCells :: OutputDocument -> Array OutputCell

foreign import data OutputCell :: Type

foreign import outputCellUuid :: OutputCell -> UUID

foreign import outputCellResult :: OutputCell -> Result

foreign import data Result :: Type

foreign import caseResult :: forall r. Result -> { resultError :: CellError -> r, resultOk :: Cell -> r } -> r

...

which would simply access the objects as necessary rather than constructing PureScript objects for them.

Sort of like Argonaut, but well-typed. You are able to walk the spine without filling in the contents.

I’ve taken this approach in Haskell with ViewPatterns on a binary format.

Is there any work in the PureScript ecosystem to generate something like this? Essentially, we want to retain the original format sent over the wire and walk it in a type-safe way.

Another way might be like:

-- normal types
data Doc = Doc (Array Cell)
data Cell = ...

-- generated
foreign import data View a :: Type
foreign import docView :: Json -> View Doc -- assumes well-structured JSON
foreign import docCells :: View Doc -> Array (View Cell)
foreign import docMaterialise :: View Doc -> Doc
foreign import cellMaterialise :: View Cell -> Cell
foreign import cellUUID :: View Cell -> String

That would let you choose between a shallow spine-only walk, and when convenient, materialise a view into a proper PureScript object.

A codegen tool (template-haskell, in my case) could consume the normal types and produce the “view” code.

I considered a binary format like flatpack/protobuf/cbor/etc. but I think JSON.parse is plenty efficient, it’s rather the materialization into PS objects that is slow as far as I can tell.

Aside:

I’ve been avoiding writing my own “bridge” between Haskell and PureScript, but

foreign-generic had a bug where a sum type with nullary constructors was incompatible between aeson and it, which disappointed me. See also my underwhelming exploration of other serialization libraries.
At this point I think I need full control over performance characteristics.

natefaubion · June 23, 2021, 4:19pm

At Awake, we have an internal codec library that uses unsafe exceptions instead of binding through Either. The monadic overhead is just way too high. I’ve found this makes performance acceptable for all of our use cases so far. I’ve measured it decoding a 7MB JSON string to PS objects in about 60ms.

chrisdone · June 23, 2021, 9:49pm

Good tip to be aware of, thanks.

Is that decoding from a JS object (I.e. leaning on JSON.parse) or parsing the string directly?

natefaubion · June 23, 2021, 9:55pm

It uses JSON.parse under the hood, so it’s materializing the whole response.

garyb · June 24, 2021, 11:08am

I’m not sure if this is entirely relevant, but we have a similar-ish scenario where we’re receiving large amounts of JSON via a websocket.

My solution was to make the handling of it somewhat lazy - there’s a series of buffers ending in a coroutine that is used to populate a scrolling view.

The coroutine can pause on either end - on the producer side it’s because we ran out of socket messages or we’re waiting for things to be processed, on the consumer side it’s because we’ve finished populating the viewable area.

The first buffer is the raw socket strings.
The second buffer pulls from that, runs JSON.parse and Data.Argonaut.Core.toArray (the socket messages are arrays of things to process) and stores the result of that.
A looping process pulls from that second buffer, parsing entries the rest of the way, and uses them to build up a cache data structure that lives in a Ref while also emitting via an Event whenever a change is made to the cache (some of the items being processed have no effect here). The loop is interrupted every 30ms to keep the UI responsive.
The scrolling view in the UI is first populated with entries by reading from the cache Ref, and after that it is updated with the coroutine when the user attempts to scroll past the already available items. The producer for the coroutine here gets its entries from the Event mentioned in 3.

It’s a little convoluted because we don’t have a 1:1 socket message:UI entry, if that were the case we could probably skip the separate looping process and Event and deal with things more directly in the coroutine… but anyway, maybe there’s something of use in here?

chrisdone · June 25, 2021, 1:26pm

Right, there are a few attacks here:

Make the single-threaded synchronous consumption step fast (e.g. JSON.parse + something). This lets the client process more data with less latency if the connection is fast.
Stream the data from a socket and parse incrementally. This lets you avoid loading the whole data set in memory to perform the parse. Higher latency, but can be easier on a smaller device.
Paginate the data from a socket/rest request. This lets the browser control when it wants to receive more data e.g. by user prompting. This can give the server a break if the data wasn’t needed anyway.
Paginate the data in the UI. This lets you avoid overloading the browser by allocating too many DOM elements, even if you already have all items in memory.

I plan to do a combination of (1) and (3) and (4).

I’ve already done a test of (4) with 5k rows of a table of e.g. 12 fields. I did this test on Jan 15th. It’s a dump of daily covid deaths (sorry for the macabre dataset). Anyway, the NEXT button simply changes an Int in the Halogen renderer to say which part of the array to render. I’m able to press and hold down that button and it scrolls through them all smoothly. What google spreadsheets does is have a DOM element whose inner height matches what you expect, and then they pick up scroll events. When you scroll that dom element, and scroll bubbles up, it does the equivalent of mashing the NEXT button. It’s a very clever trick (a friend who worked on another spreadsheet library told me the trick). Even Halogen whose performance isn’t anything special is capable of doing it. So that’ll be my approach for tables.

As it happens, my data is nested like JSON. So it could be a table of a tree of a table, so pagination (3) won’t always save me. That’s why I want to make (1) perform well.

I’ve already started an implementation of the View approach. I’ll update the thread when I’ve got it working with an experience report. But that might be a while away.

chrisdone · June 25, 2021, 1:36pm

In case I get randomized to another task, here’s my WIP module: Frisson · GitHub

garyb · June 25, 2021, 2:24pm

Yeah, the catching-the-scroll trick is pretty much what we do too although not quite as smoothly as in google sheets, as we can also be held up on processing data before it’s ready to display.

In our case the socket connection isn’t just because we’re trying to prevent blowing up the client, it’s that the backend is scanning an external dataset and producing information about the schema of the dataset incrementally - that process is also not fast enough to return a single response to us, given that in theory it would need to see the entire dataset to fully determine the schema, and there’s no explicit limit set on how large the external dataset might be.

We have similar problems in a way, only you’re showing the actual data where we’re compressing that somewhat by working with its structure and a sampling of scalar values within it.

chrisdone · June 25, 2021, 3:54pm

I see, that’s another good point. In my case the data is available and just ready to be dumped (for now, later I will do streaming). In your case the server is processing it slowly so you’re showing whatever arrives to the client immediately. In that case I understand why you have the websocket!

chrisdone · June 30, 2021, 2:40pm

The latest version of Frisson is now working for sending data from the server to the client (Haskell to PureScript).

I’m using it like this:

$(Frisson.deriveAll
  "../inflex-client/src/Inflex/Frisson.purs"
  "../inflex-client/src/Inflex/Frisson.js"
  "module Inflex.Frisson where\n\
   \\n\
   \import Inflex.Schema\n\
   \import Data.UUID (UUID)\n\
   \import Data.Argonaut.Core (Json)\n\
   \"
  [''UUID,''Version1,''Version2,''RefreshDocument,''UpdateDocument,''UpdateSandbox,''UpdateResult, ...])

I have some other infrastructure which basically uses an affjax REST call to produce a Json (argonaut’s) and then calls unsafeView json which produces View a for the given output type. The RPC call infra I have ensures that the type returned is statically matching the RPC method.

So decoding the 14.6MB of JS with JSON.parse takes 100ms and then I can immediately access the data in a “lazy” way. Properties use accessors, and sum types are accessed similar to argonaut.

Inflex.Frisson.purs looks like this:

import Inflex.Schema
import Data.UUID (UUID)
import Data.Argonaut.Core (Json)
foreign import data View :: Type -> Type

foreign import unsafeView :: forall a. Json -> View a

foreign import unviewRenameCell :: (View RenameCell) -> Json

foreign import renameCellUuid :: (View RenameCell) -> (View UUID)

foreign import renameCellNewname :: (View RenameCell) -> String

foreign import outputDocumentCells :: (View OutputDocument) -> (Array (View OutputCell))
...

foreign import unviewUpdateResult :: (View UpdateResult) -> Json

foreign import caseUpdateResult :: forall r. {
  "UpdatedDocument" :: (View OutputDocument) -> r,
  "NestedError" :: (View NestedCellError) -> r
  } -> (View UpdateResult) -> r

With the equivalent Inflex/Frisson.js:

exports.updateSandboxDocument = function(a){return a[1]};

exports.caseUpdateResult = function(k) {
return function(a) {
switch (a[0]) {
case 0: return k["UpdatedDocument"](a[1]);
case 1: return k["NestedError"](a[1]);
default: throw Exception('BUG: case accessor failed');
}
}
}
;

foreign import outputCellResult :: (View OutputCell) -> (View Result)

foreign import outputCellOrder :: (View OutputCell) -> Int

This supports server->client, I don’t currently need client->server as argonaut works well for that (the client doesn’t send much data to the server).

But I might implement that later. It could be done by deriving dematerialization functions: dematerializeFoo :: Foo -> View Foo, and completely avoid argonaut.

Once could also implement generation of materialization functions, materializeFoo :: View Foo -> Foo, for when performance doesn’t matter as much as convenience.

chrisdone · July 2, 2021, 9:35am

I’ve ported my Halogen components to use the View X instead of X types and here are my latest stats:

For a small doc (about 216KB of JSON):

The server takes 1863ms to generate the data (that’s much easier to optimise as it’s just Haskell).

Here JSON.parse takes 2ms and my Halogen component takes <1ms for each cell in the doc.

For a large doc (about 14.6MB of JSON):

The server takes 3800ms to generate the data.

Here JSON.parse takes 155ms and my Halogen component takes <1ms to render the cell (a table).

As there is no parse step after JSON.parse the data is ready immediately for use in PS.

This solves my client-side problem.