Leveraging a blazing-fast runtime: a (new) Go backend for PureScript

[0x000000000000000000]

Hello everyone!

A few weeks ago, I introduced phpurs (here), an alternate backend to compile PureScript into modern PHP.

Today, I’d like to announce that I’m taking all these hard-won lessons and applying them to a new project: gopurs, a modern PureScript-to-Go compiler. Still an experimental WIP, but the first 50% official tests are already starting to show green and positive results.

Some of you might immediately (and legitimately) think: “Wait, doesn’t purescript-native (here) already do this?”. And yes, it does!

However, reading through the discussions and challenges raised by users in its thread (like initialization orders, performance limits of interface{} or module qualifications), I realized that the ecosystem has evolved drastically over the last few years. This unlocked new architectural paradigms that make (IMHO) building a completely new Go backend highly relevant today, specifically to address these exact limitations:

1. The optimizer & bootstrapping
   While purescript-native was written in Haskell and parsed raw CoreFn, gopurs is written 100% in PureScript. It plugs directly into the purescript-backend-optimizer (which I also used for phpurs). This means we do not reinvent the wheel for classical optimizations. The gopurs compiler can then strictly focus on translating this highly-optimized AST into idiomatic, performant Go code, adding further optimizations when it can, for Go (specifically). And it remains fully accessible to anyone in the PureScript ecosystem (installable via spago and npm).

2. Heap vs. Stack: a new memory layout for Go
   Dynamic typing in statically typed languages like Go often relies heavily on interface{} (or any). However, assigning primitive values to interfaces forces them to escape to the heap (Boxing), generating massive Garbage Collector pressure. For gopurs, I ran extensive benchmarks and decided to completely ditch any. Instead, the runtime uses a universal flat Value struct (a tagged union), inspired by V8 or LuaJIT. This ensures that dynamic operations stay mostly on the stack, when possible.
   To give you an idea, on a 1 billion operations benchmark, native static Go took ~250ms, a dynamic any approach took ~9 seconds, and the Value struct solution completed in ~240ms! The performance difference is staggering, completely bypassing the GC overhead in heavy iterative loops.

3. Up-to-date with modern PureScript
   purescript-native hasn’t seen major activity in a while. gopurs aims to be fully aligned with the current v0.15+ ecosystem (and v0.16+ soon). I am currently mirroring the standard libraries (gopurs-prelude, gopurs-effect, etc.) to provide native Go FFIs.

4. Up-to-date with modern Go
   Go has evolved significantly since the early days of purescript-native. By targeting modern Go (1.18+), gopurs takes full advantage of recent advancements: from the much-improved Garbage Collector and Goroutine scheduler (making Aff mappings practically free), to the introduction of Generics that open new doors for safe, typed, and performant FFI definitions.

I want to deeply acknowledge the fantastic work done by Andy on purescript-native. It really paved the way. It is always much easier to come second and learn from the limits encountered by the pioneers. This project is a response, an extension of his work, an apology, a love letter. It simply couldn’t exist without it. All my gratitude for his initial effort.

As we all know, Go offers exceptional concurrency (goroutines fit perfectly with Aff), amazing speed, and rock-solid native binaries.

Being able to bring a mesomorphic approach to this ecosystem is a great opportunity: the solid and safe part of PureScript, with the liquid and battled-test part of Go’s runtime (and FFIs). I often use this metaphor when talking about PureScript: like our modern screens (i.e. liquid crystals), it’s a language that allows us to elevate the discussions and rise above the dialectical processes, hybridizing the strengths rather than canceling them out.

The repo may be ready for production soon, this summer (it will still be pretty messy until then)

Love

Edit: 100% of the official tests are now green.

[druchan]

Would be nice to add a “how to use” section in the readme?

Excited to see how this pans out for some of my Purescript projects that are (unfortunately) built for Node/npm ecosystem.

[0x000000000000000000]

Absolutely! I plan to add that today.

[0x000000000000000000]

News #1:

Milestone yesterday: 100% of the official PureScript tests are now passing on gopurs!
The compiler is formally complete and generates reliable Go code.

However, after pushing the current architecture to its limits (the flat Value struct does wonders for the GC) and multiple micro-optimizations, I’ve hit a ceiling. Performance is good, but it’s not yet the raw, blazing speed Go is natively capable of. My stress-tests actually show better results with V8 right now.

Why the ceiling?
gopurs relies on a highly optimized AST that is very close to CoreFn. Unfortunately, CoreFn strips away crucial static typing information. It’s made for JIT runtimes, less for AOT ones. This forces the Go backend to rely on generic, dynamic structures instead of native, strictly typed Go constructs (e.g., using a universal Value struct instead of raw Go ints or unboxed structs). Even if Value gives better results than any, and is as fast as idiomatic Go for simple algorithms, that still is insufficient for complex ones.

The next step: The TAST fork
To break this ceiling and reach a new standard of performance, the next logical move is to explore a TAST (Typed Abstract Syntax Tree) fork. By partially intercepting the fully typed AST before it loses information in the CoreFn conversion, gopurs will be able to generate highly idiomatic, statically typed Go code end-to-end, eliminating residual boxing.

The fork builds successfully, and I’m already working with enriched corefn.json files right now. By simply keeping primitives (int, float, bool, etc.) as they are instead of boxing them into a Value, I’ve already seen a 40% performance gain. And new opportunities are expected to emerge for further optimization.

Stay tuned.

Note: as suggested by @druchan, a complete “How to use” section will be added to the README.md of gopurs, once mature.

[0x000000000000000000]

News #2:

Victory!

That’s fantastic.

The TAST approach has more than paid off: that’s a big x10 in perfs. So much so that I’ll soon be submitting a PR to add tcorefn (= corefn JSON + dataDecls array at the root + ann.type on each node) to the available options for purs compile --codegen (among corefn, docs, js…). ADTs, Records and functions have been added to the whitelist, and they’ve enabled numerous improvements (struct, eager uncurrying…).

For now, I’m working with a fork of Purescript, but I think this option will greatly benefit the official and durable advent of AOT backends like Go or WASM, complementing the JIT backends (JS, PHP…) that have been in place so far.

In my stress tests, the gopurs compiler now produces code that performs better than the official JS compiler on V8. AOT is not longer a problem. And all of this on a single thread! Here is the benchmark. I’ll enrich it with new tests, but that’s a good starting point.

I’m not even mentioning the benefits of multithreading for speed, RAM usage that’s much lower than V8’s, the instant startup of a very lightweight binary, and so on. It started out as a dream, given the real constraints that existed (and that we’d already faced in the past), but it’s becoming a reality.

I hope to share this with our lovely PureScript community soon, as a production-ready backend.

The Arista compiler perf is still a goal to be achieved. But the optimizations aren’t finished yet (e.g. Constraint Dictionaries). And I fully intend to reach the 80 ms they managed to get (with V8 too, in my benchmark). That should be closer to the usual x1.5 factor we can see online, in exhaustive Go/V8 computational benchmarks. If I can go even lower than that, I’ll obviously do it!

That’s the next step, alongside a full-scale test on a real-world project (Aff, logging, FFI Postgres, S3, RabbitMQ…).

Cheers

[0x000000000000000000]

News #3:

Great improvements for FFI: no need to worry about what the compiler is doing. You can write anything, including using any! Everything will be converted on the fly by the compiler to ensure the best possible performance. You can write var foo = function or function foo. No package header needed.

[0x000000000000000000]

Note on news #2:

To give everyone a concrete idea of why this custom tcorefn (Typed CoreFn) makes building AOT backends drastically easier, I thought I’d share what the JSON output actually looks like under the hood.

It is important to note that tcorefn.json simply pushes further the idea of a desugared JSON that represents the PureScript code, by whitelisting additional info (not needed for JIT backends like JS).

Instead of just dumping source spans, the compiler now hands over the full memory layout (via dataDecls) and the deep type information at every AST node (via annotation.type).

Here are only 3 quick side-by-side examples of PureScript code and the resulting condensed JSON:

1. Data Declarations (no more guessing around ADTs)

In standard CoreFn, constructors appear as you go, forcing the backend to gather them. With tcorefn, the AOT backend knows the exact memory structure as soon as it reads the file.

PureScript:

data Maybe a = Nothing | Just a

tcorefn.json (at the root of the module):

"dataDecls": [
  {
    "typeName": "Maybe",
    "constructors": [
      { "constructorName": "Nothing", "fieldTypes": [] },
      { "constructorName": "Just", "fieldTypes": [ { "TypeVar": { "name": "a" } } ] }
    ]
  }
],
"decls": ...

The backend immediately sees the ADT structure. No need to crawl the AST to infer constructor arities or memory layouts.

---

2. Fully Typed AST Nodes (no more “Shadow Typing”)

In classic CoreFn, an AST node only holds a sourceSpan. With this format, every node knows exactly what type it handles.

PureScript:

-- implemented internally as: isJust = maybe false (const true)
isJust :: forall a. Maybe a -> Boolean

tcorefn.json (excerpt showing the root expression for isJust):

"expression": {
  "type": "App",
  "annotation": {
    "sourceSpan": { "start": [279, 10], "end": [279, 34] },
    // HERE: The full function signature is retained on the node
    "type": {
      "Func": {
        "args": [ { "ADT": { "path": ["Data", "Maybe", "Maybe"], "args": [ { "TypeVar": { "name": "a" } } ] } } ],
        "ret": "Boolean"
      }
    }
  },
  "abstraction": {
    "type": "App",
    "abstraction": { "type": "Var", "value": { "identifier": "maybe" } },
    "argument": { "type": "Literal", "value": { "value": false } }
  },
  "argument": {
    "type": "App",
    "abstraction": { "type": "Var", "value": { "identifier": "const" } },
    "argument": {
      "type": "Literal",
      "value": { "literalType": "BooleanLiteral", "value": true },
      "annotation": {
        // Primitive types are also explicitly attached to literals deep down
        "type": "Boolean" 
      }
    }
  }
}

Every single AST node in decls has its exact type attached. You don’t need an external JIT-like type-checker or shadow structures in the backend to know if you’re dealing with a Boolean, an Int32, or a specific ADT.

---

3. Records and FFI (structural typing preserved)

One of the biggest challenges for AOT is knowing when you are manipulating a raw record/structure. This JSON makes it explicit.

PureScript:

foreign import identity :: forall a. { identity :: a } -> a

tcorefn.json (excerpt):

"abstraction": {
  "type": "Var",
  "value": { "identifier": "identity", "moduleName": ["Control", "Category"] },
  "annotation": {
    "meta": { "metaType": "IsForeign" },
    "type": {
      "Func": {
        // The exact structure of the Record is preserved!
        "args": [ { "Record": { "identity": { "TypeVar": { "name": "a" } } } } ],
        "ret": { "TypeVar": { "name": "a" } }
      }
    }
  }
}

When calling FFI or handling PureScript records, the backend receives the exact shape of the expected object. This makes generating C/Go structs (or anything similar, in other languages) drastically easier. And of course, all of this is recursive (e.g. "args": [ { "Record": { "identity": { "Record": ... } } } ])

[0x000000000000000000]

News #4:

Final stretch: It’s getting harder and harder to make gains.

But perfs have greatly improved: 125 ms has become 89 ms. Full benchmark here.

I’m currently tackling what will likely be the final piece of the puzzle: monomorphization. It’s a relatively complex topic, even if it’s simple on paper. If it works out, we can hopefully reach Arista’s 80 ms milestone.

But we’re already very close. For an AOT compiling flow, this is truly satisfying, because it required moving into the compiler what modern JITs have been constantly striving to improve for years.

Stay tuned.

[0x000000000000000000]

News #5:

Monomorphization is done on the simplest parts (i.e. primitives). Arista’s miletone has been reached: both Arista ES and Go take ~ 80 ms to complete the tests.

Optimization efforts are still ongoing (i.e. monomorphization around ADTs, Records…).

[0x000000000000000000]

@harryprayiv

Just so you know, I’m currently moving optimizations shared by phpurs and gopurs into a TASTed branch of purescript-backend-optimizer, locally. In my opinion, the gopurs project isn’t mature enough yet to be ported to Nix (still a mess, dependencies are still local, etc.). I’ll let you know when it’s ready!

And phpurs is going through huge improvements by the way. Even if PHP is not massively typed, it partially is, and the compiled code will be faster thanks to TASTs.

[0x000000000000000000]

News #6:

gopurs performance is starting to take off, with 69 ms. We’re getting closer and closer to native performance levels.

Online benchmarks often show a 1.5x to 2.5x speed advantage for Go over JS (V8). Our benchmark aligns perfectly with this, showing a ~ 2x speedup between the official compiler and gopurs .

[0x000000000000000000]

News #7:

A new approach, inspired by Koka/Lean, is currently being tested: FBIP/Sticky Sharing. This could represent a huge leap forward in terms of performance, bringing us closer to native results. But it completely changes the game (i.e., the entire codebase).

TASTs play a crucial role in this (@harryprayiv I’ve edited my last message, in the WASM thread, regarding this subject: a POC has been made!).

[harryprayiv]

(post deleted by author)

[harryprayiv]

Read through the gopurs README. The flat Value struct instead of any is a good call, that boxing cost is exactly what made the older Go backend slow, and building on backend-optimizer rather than parsing CoreFn yourself seems right.

One thing I keep getting stuck on in the benchmark: Value comes out at 240ms against 250ms for static Go. A tagged union with a tag check shouldn’t be able to beat monomorphic Go on the same work, so I’d want to rule out the loop being partly optimized away before leaning on that number. And if it does hold, doesn’t it undercut the TAST case? If Value is already at static-Go parity there’s nothing left for monomorphization to win.

Separately on FBIP: Perceus works because everything is a refcounted heap pointer with dup/drop at every use. A Value passed by value gets copied constantly with no ownership event on the pointer arm. Adding reuse seems to mean putting refcount discipline back onto the representation whose whole benefit is being cheap to copy. Is that what “changes the entire codebase” means? Because that reads like trading the thing you’ve measured for the thing you haven’t.

[0x000000000000000000]

For FBIP:

I tried various workarounds.

One of them was to use a reference counter in Value. We’d increment it without worrying about decrementing it (that resulted in -50% for RBTree perfs). But that required reworking the pattern matching to copy the values into temporary variables before using them later, and so on. Ultimately, it completely undermined both PureScript’s design and Go’s strengths (its GC). Given my priorities, I scrapped this experimental feature. It might be worth revisiting for another backend in the future, but for Go, I don’t think it’s the right path to take right now.

The FFI feature of gopurs allows us to use fully native types in .go files. We can still use them to achieve good performance in critical hot spots.

I’m going to refocus on cleaning up the compiler, adding support for Aff, and the official release.

[0x000000000000000000]

For Value & TASTs:

I’ll be traveling until the end of the week. The questions raised are very relevant, and I’ll be happy to answer them upon my return.

[harryprayiv]

Scrapping it on a measurement is the right call, and publishing the negative result is more useful than most positive ones. -50% on RBTree is worth recording somewhere permanent, since tree insertion is Koka’s showcase case for reuse and it’s the first thing anyone else will reach for.

One narrow caveat for the archive, not an argument to resume: a counter that increments and never decrements isn’t Perceus, it’s a monotone “everything is shared” signal, so reuse would never fire in the first place. The result is probably about that shortcut plus the by-value Value representation rather than about FBIP on Go in general. Someone will come back to this in a year and that distinction will save them a week.

On benchmarks, and given you clearly do measure, the issue is legibility rather than rigour. Across the thread the reference point moves (static Go, then V8, then Arista’s 80ms, then ~2x over the official compiler) and the headline figures (40%, x10, then 125 to 89 to 80 to 69) don’t compose into one series anyone else can read. A single frozen suite, every configuration against the same baseline on the same machine, would make it citable. Right now I can’t point people at it, which is a shame given the work behind it. Related: the 240ms Value figure is still unqualified in the README, and you’ve since said Value is only at parity for simple algorithms.

The piece I think matters most is tcorefn, and a cleanup phase is the right time to write the format spec before the PR. The diagnosis is right and CoreFn’s type erasure has been the known ceiling for AOT backends for years. From the JSON you posted, some things that will come up in review:

• Rows. “Record”: { “identity”: … } uses labels as object keys. PureScript rows are ordered, admit duplicate labels, and can be open with a tail variable. None of that survives a flat map.
• Quantifiers. forall a. is gone from the isJust example. Monomorphization needs the binding structure, not just occurrences of TypeVar a.
• Constraints and kinds. I don’t see either represented.
• Encoding consistency. “ret”: “Boolean” is a bare string while other types are tagged objects. That needs to be uniform before anyone writes a decoder against it.
• Size. corefn.json is already a build-time complaint, and a type on every node is a large multiplier. Someone will ask for the numbers.

The structural one: this serializes post-elaboration types, which commits the compiler to the stability of an internal representation, skolems and unknowns included. That’s why this has stalled before, and it’s a spec question rather than an implementation one. I’d like to see it land, and I’m happy to review a draft if that’s useful.

One thing on Aff while it’s still ahead of you rather than behind you. I’d be interested in the goroutine mapping semantically rather than in performance terms. Aff is a cooperative single-threaded scheduler with specific guarantees around cancellation, forking and supervision, and Ref has no synchronisation. Mapping it onto goroutines buys real parallelism, which changes the semantics and not just the speed. Worth deciding deliberately whether gopurs Aff is the same Aff or a different one behind the same API, because library authors will assume the former.

Cleanup, Aff, release sounds like the right order to me.

[0x000000000000000000]

Thanks for the detailed feedback, @harryprayiv

Just to clarify regarding the FBIP experiment: the “-50% on RBTree” actually meant a 50% gain in performance (execution time). However, properly implementing the reference decrementing was extremely tedious and complex to get right, which was one of the main factors in abandoning this approach for now.

For all the other points (benchmarks legibility, the tcorefn format specifications, and the Aff semantics), we are completely on the same page. It’s all in my notes.

I’ll focus on the cleanup phase as planned, and I’ll gladly take you up on your offer to review the tcorefn spec draft when it’s ready.

Cheers!