Latest and greatest Haskell<->PureScript serialization

Help
#1

Hi all,

I’m writing a Haskell web service and PureScript frontend. I’m researching techniques that can serialize to/from Haskell and PureScript data types. So far I’m struggling to find anything that isn’t worrying.

Here’s my survey so far. Hopefully it will be of used to others to avoid spending the time I’ve spent trawling for answers.

  • :skull_and_crossbones: purescript-bridge - this package generates
    PureScript code, but it states that “For compatible JSON
    representations you should be using aeson’s generic
    encoding/decoding with default options and encodeJson and
    decodeJson from “Data.Argonaut.Generic.Aeson” in purescript-argonaut-generic-codecs.” However, that package hasn’t been touched since 2017. It has funny parse issues like this, and apparently is unmaintained, and doesn’t use Generics.Rep: “I am no longer a PureScript user and did not know about this. But I will definitely merge PRs, making this library fit for future PureScript releases!” purescript-bridge is maintained by the same author.
  • :skull_and_crossbones: purescript-argonaut-aeson-generic supports
    the new generics: “It is using Data.Generic.Rep hence it will
    work with purescirpt-0.12 , unlike
    purescript-argonaut-generic-codec which at the moment is based on
    Data.Generic for which generic deriving has been removed from the
    purescript compiler in this commit.” But this is also apparently buggy, and has an issue that has been reported on this, and not responded to since 2018. Has been fixed 11th Oct 2020, see comment below.
  • purescript-foreign-generic - is seemingly maintained, as it was last touched in 2019, although it has three open pull requests. There’s some discussion on aeson compatibility, but it’s left unclear and is from last year. So the compatibility with aeson is at best a big question-mark.
  • purescript-interop is a 5 years old untouched attempt at Haskell-PS interop, but seems dead.
  • purescript-iso is an undocumented but possible Haskell-PS bridge
    last touched in 2019.

(I originally put links to all the things I was referring to, but
Discord rejected my comment saying I can only put 2 links in a post
because I’m “new”. So you’ll have to just find the things yourself.)

There’s a pattern here. There are lots of alternatives, and they are all either out of date or unmaintained or buggy, or all three.

Ideally, I would just use aeson’s encoding and PureScript’s aeson-compatible encoding to ferry types between Haskell and PureScript, but my faith is wavering in any libraries providing this in a stable non-buggy way.

My options are:

  • I pick up and maintain one of these in my project.
  • I find another one that someone else out there is using for real work and doesn’t have blatant outstanding encoding errors.
  • I write my own thing.

I’ve been wanting to version my API’s schema in a tightly-controlled way (hashing all types), so I might end up writing template Haskell anyway and generate Data.Argonaut.Core to decode on the PureScript side. :man_shrugging:

9 Likes
Fast treatment of data from server
#2

We use purescript-foreign-generic extensively in production at Lumi, together with a Haskell backend using aeson, and it works fine.

7 Likes
#3

I’m going to help by providing you with some context and connection to theory, which will give more ways to approach this problem. If you people starting the thread think this is too generic or far away from practical solution, a distraction, let me know and I’ll move it up to my blog off your way without becoming offended. Theory-wise, we are talking about isomorphisms when we talk about this kind of serialization bridges.

To illustrate what an isomorphism is, I give you few examples of isomorphisms, later below we’ll prove the first one to show how it works:

(Tuple a b) ≅ (Tuple b a)
(Either Unit a) ≅ a
(Pair a b → c) ≅ a → b → c

Isomorphism between two types, X ≅ Y means that there are two functions.

f :: X → Y
g :: Y → X

These functions are satisfying the following equations, these equations mean that going between the two representations keeps the value equal:

(f >>> g) = (\x -> x)
(g >>> f) = (\y -> y)

Isomorphic mappings like these are very important for several reasons.

  1. They allow to automate bridging of languages. This is important because it is a very hard roadbump to throw away or reconnect everything every time when we find out more to improve in programming language development. Theory can connect this together. I’d wish the work on Haskell, Purescript, Even C and Javascript would all remain accessible as we keep getting better functional programming languages.
  2. The allow to bridge things inside the language together. Those different representations for different things neatly connect together and become compatible.

Ok, now to explaining why they’re isomorphisms that I gave as an example. This is very similar to proving they’re isomorphisms. For each one we find two functions and see that they satisfy the equations.

(Tuple a b) ≅ (Tuple b a)
f = swap
g = swap
Refl :: (f >>> g) = id
Refl :: (g >>> f) = id

Swap flips the parameters around. Flipping the parameters around twice just does nothing. The equations are proved by reflection, as the both sides reduce to same expression.

There’s a thing I’m not sure about and it should be mentioned here. The order of evaluation does affect isomorphic mappings. To illustrate think about mapping an infinite list between the bridges. This connects to well-formedness of recursion. It’s the data/codata -thing, or catamorphisms/anamorphisms -thing.

I believe some of you already know these subject, and possibly much better than me. I will gladly listen you. I’ll try to make these more accessible for everybody as we keep going. Also, if you got more questions, I’ll answer those too. I’ll be available to discuss this.

2 Likes
#4

OK, if you’ve been using it in production for a while then I’ll give it a try. Thanks.

2 Likes
#5

It seems like you’re looking mostly at stringy typeclass-based serialization, but if you want to do binary non-typeclass serialization between Haskell and Purescript, then this combination of libraries works pretty well.

https://pursuit.purescript.org/packages/purescript-parsing-dataview

https://pursuit.purescript.org/packages/purescript-arraybuffer-builder

3 Likes
#6

It actually seems like purescript-bridge is a lot more modern than it seems; the hackage version is just really old. In v0.14 (just a few months old)

  • support for Generics.Rep
  • support for foreign-generic

I’ll probably start updating the rest of purescript-bridge for modern purescript. Looks like it could become a really nice tool.

EDIT: To use generics-rep and foreign-generic, use these settings:

import Language.PureScript.Bridge.CodeGenSwitches (useGenRep, genForeign)

 writePSTypesWith (useGenRep <> genForeign (ForeignOptions False))
6 Likes
#7

As of 2020-09-18, there is now a Protocol Buffers implementation for Purescript https://github.com/xc-jp/purescript-protobuf

I wrote it specifically for serializing data between Purescript and Haskell.

7 Likes
#8

Update:

I ended up taking this approach to making my Haskell/PureScript endpoints.

I also tested that aeson/foreign’s encodings are reliably roundtripping by using property tests to generate random values of variously shaped types (sum, record, product, maybes, vector) as a sanity check. qjs is the QuickJS binary, which is very fast at loading and running JS code.

One could also apply the latter roundtrip test to the former set of API endpoints. I’ve considered doing that as an extra layer of confidence, especially in the versioning and migrations.

3 Likes
#9

I think I understand the approach of using the code for the DTOs that’s generic to both languages and embed it in a template that has the specific parts of each language.

But I’m a complete noob in Template Haskell and I don’t understand your gists specially when you use $types and $calls. Are they just placeholders to replace the content of the DTO definitions? Is there a way using TH to generate all the deriving instance sentences from the data definitions?

Could you expand on the problem or just point to some reading resources? Thanks

#10

News for argonaut-aeson-generic!

I patched the encoding issue mentioned in the opening post. I also found and fixed some other encoding issues. The test suite is also considerably extended, covering all combinations of supported encoding options and data shapes. The library can now be used out of the box with default Aeson options on the Haskell side. A tag 0.2.0 is assigned to the new version.

The library is currently being integrated into a private project I am working on, so there is a good chance that I shall be keeping an eye on it and helping out with new issues as they appear.

I hope other people find this library useful. Together we can do a better job improving it further.

@chrisdone Chris, possibly you would consider editing your opening post to account for this update?

7 Likes
#11

I’ve updated the post.

1 Like
#12

There is a mistake — I am not a maintainer. Although I may note that the maintainer was very responsive and merged my patches in no time!

#13

Did I understand correctly that for this approach, you still have to define data type manually in both Purescript and Haskell?

2 Likes
#14

No, we have a bit of code which generates PureScript data types from our Haskell data types. It’s not open source unfortunately; while I think we’d like to open source it eventually, we’re unlikely to find the time for it any time soon.

4 Likes
#15

Thanks @csicar , this helped me. This example project uses your suggestion: https://github.com/eskimor/purescript-bridge/tree/master/example

I will look into updating it for Purescript 0.14.

2 Likes
#16

Hi there, just a heads up that

there is a fork argonaut-aeson-generic updated to work with Purescript v0.15:

Hope this would help anyone~

6 Likes
#18

Do you use browser plugins for decoding request bodies while debugging?

#19

You mean like this? pbkit-devtools - Chrome Web Store

No I haven’t used a browser plugin like that, but that’s a good idea.

1 Like