Leveraging a blazing-fast runtime: a (new) Go backend for PureScript

Would be nice to add a “how to use” section in the readme?

Excited to see how this pans out for some of my Purescript projects that are (unfortunately) built for Node/npm ecosystem.

1 Like

Absolutely! I plan to add that today.

News #1:

Milestone yesterday: 100% of the official PureScript tests are now passing on gopurs! :green_circle:
The compiler is formally complete and generates reliable Go code.

However, after pushing the current architecture to its limits (the flat Value struct does wonders for the GC) and multiple micro-optimizations, I’ve hit a ceiling. Performance is good, but it’s not yet the raw, blazing speed Go is natively capable of. My stress-tests actually show better results with V8 right now.

Why the ceiling?
gopurs relies on a highly optimized AST that is very close to CoreFn. Unfortunately, CoreFn strips away crucial static typing information. It’s made for JIT runtimes, less for AOT ones. This forces the Go backend to rely on generic, dynamic structures instead of native, strictly typed Go constructs (e.g., using a universal Value struct instead of raw Go ints or unboxed structs). Even if Value gives better results than any, and is as fast as idiomatic Go for simple algorithms, that still is insufficient for complex ones.

The next step: The TAST fork
To break this ceiling and reach a new standard of performance, the next logical move is to explore a TAST (Typed Abstract Syntax Tree) fork. By partially intercepting the fully typed AST before it loses information in the CoreFn conversion, gopurs will be able to generate highly idiomatic, statically typed Go code end-to-end, eliminating residual boxing.

The fork builds successfully, and I’m already working with enriched corefn.json files right now. By simply keeping primitives (int, float, bool, etc.) as they are instead of boxing them into a Value, I’ve already seen a 40% performance gain. And new opportunities are expected to emerge for further optimization.

Stay tuned.

Note: as suggested by @druchan, a complete “How to use” section will be added to the README.md of gopurs, once mature.

News #2:

Victory! :v:

That’s fantastic.

The TAST approach has more than paid off: that’s a big x10 in perfs. So much so that I’ll soon be submitting a PR to add tcorefn (= corefn JSON + dataDecls array at the root + ann.type on each node) to the available options for purs compile --codegen (among corefn, docs, js…). ADTs, Records and functions have been added to the whitelist, and they’ve enabled numerous improvements (struct, eager uncurrying…).

For now, I’m working with a fork of Purescript, but I think this option will greatly benefit the official and durable advent of AOT backends like Go or WASM, complementing the JIT backends (JS, PHP…) that have been in place so far.

In my stress tests, the gopurs compiler now produces code that performs as well as the official JS compiler on V8. AOT is not longer a problem. And all of this on a single thread! Here is the benchmark. I’ll enrich it with new tests, but that’s a good starting point.

I’m not even mentioning the benefits of multithreading for speed, RAM usage that’s much lower than V8’s, the instant startup of a very lightweight binary, and so on. It started out as a dream, given the real constraints that existed (and that we’d already faced in the past), but it’s becoming a reality.

I hope to share this with our lovely PureScript community soon, as a production-ready backend.

The Arista compiler perf is still a goal to be achieved. But the optimizations aren’t finished yet (e.g. Constraint Dictionaries). And I fully intend to reach the 80 ms they managed to get (with V8 too, in my benchmark). That should be closer to the usual x1.5 factor we can see online, in exhaustive Go/V8 computational benchmarks. If I can go even lower than that, I’ll obviously do it!

That’s the next step, alongside a full-scale test on a real-world project (Aff, logging, FFI Postgres, S3, RabbitMQ…).

Cheers :wave:

1 Like

News #3:

Great improvements for FFI: no need to worry about what the compiler is doing. You can write anything, including using any! Everything will be converted on the fly by the compiler to ensure the best possible performance. You can write var foo = function or function foo. No package header needed.

Note on news #2:

To give everyone a concrete idea of why this custom tcorefn (Typed CoreFn) makes building AOT backends drastically easier, I thought I’d share what the JSON output actually looks like under the hood.

It is important to note that tcorefn.json simply pushes further the idea of a desugared JSON that represents the PureScript code, by whitelisting additional info (not needed for JIT backends like JS).

Instead of just dumping source spans, the compiler now hands over the full memory layout (via dataDecls) and the deep type information at every AST node (via annotation.type).

Here are only 3 quick side-by-side examples of PureScript code and the resulting condensed JSON:

1. Data Declarations (no more guessing around ADTs)

In standard CoreFn, constructors appear as you go, forcing the backend to gather them. With tcorefn, the AOT backend knows the exact memory structure as soon as it reads the file.

PureScript:

data Maybe a = Nothing | Just a

tcorefn.json (at the root of the module):

"dataDecls": [
  {
    "typeName": "Maybe",
    "constructors": [
      { "constructorName": "Nothing", "fieldTypes": [] },
      { "constructorName": "Just", "fieldTypes": [ { "TypeVar": { "name": "a" } } ] }
    ]
  }
],
"decls": ...

The backend immediately sees the ADT structure. No need to crawl the AST to infer constructor arities or memory layouts.


2. Fully Typed AST Nodes (no more “Shadow Typing”)

In classic CoreFn, an AST node only holds a sourceSpan. With this format, every node knows exactly what type it handles.

PureScript:

-- implemented internally as: isJust = maybe false (const true)
isJust :: forall a. Maybe a -> Boolean

tcorefn.json (excerpt showing the root expression for isJust):

"expression": {
  "type": "App",
  "annotation": {
    "sourceSpan": { "start": [279, 10], "end": [279, 34] },
    // HERE: The full function signature is retained on the node
    "type": {
      "Func": {
        "args": [ { "ADT": { "path": ["Data", "Maybe", "Maybe"], "args": [ { "TypeVar": { "name": "a" } } ] } } ],
        "ret": "Boolean"
      }
    }
  },
  "abstraction": {
    "type": "App",
    "abstraction": { "type": "Var", "value": { "identifier": "maybe" } },
    "argument": { "type": "Literal", "value": { "value": false } }
  },
  "argument": {
    "type": "App",
    "abstraction": { "type": "Var", "value": { "identifier": "const" } },
    "argument": {
      "type": "Literal",
      "value": { "literalType": "BooleanLiteral", "value": true },
      "annotation": {
        // Primitive types are also explicitly attached to literals deep down
        "type": "Boolean" 
      }
    }
  }
}

Every single AST node in decls has its exact type attached. You don’t need an external JIT-like type-checker or shadow structures in the backend to know if you’re dealing with a Boolean, an Int32, or a specific ADT.


3. Records and FFI (structural typing preserved)

One of the biggest challenges for AOT is knowing when you are manipulating a raw record/structure. This JSON makes it explicit.

PureScript:

foreign import identity :: forall a. { identity :: a } -> a

tcorefn.json (excerpt):

"abstraction": {
  "type": "Var",
  "value": { "identifier": "identity", "moduleName": ["Control", "Category"] },
  "annotation": {
    "meta": { "metaType": "IsForeign" },
    "type": {
      "Func": {
        // The exact structure of the Record is preserved!
        "args": [ { "Record": { "identity": { "TypeVar": { "name": "a" } } } } ],
        "ret": { "TypeVar": { "name": "a" } }
      }
    }
  }
}

When calling FFI or handling PureScript records, the backend receives the exact shape of the expected object. This makes generating C/Go structs (or anything similar, in other languages) drastically easier. And of course, all of this is recursive (e.g. "args": [ { "Record": { "identity": { "Record": ... } } } ])

1 Like

News #4:

Final stretch: It’s getting harder and harder to make gains.

But perfs have greatly improved: 125 ms has become 89 ms. Full benchmark here.

I’m currently tackling what will likely be the final piece of the puzzle: monomorphization. It’s a relatively complex topic, even if it’s simple on paper. If it works out, we can hopefully reach Arista’s 80 ms milestone.

But we’re already very close. For an AOT compiling flow, this is truly satisfying, because it required moving into the compiler what modern JITs have been constantly striving to improve for years.

Stay tuned.

1 Like

News #5:

Monomorphization is done on the simplest parts (i.e. primitives). Arista’s miletone has been reached: both Arista ES and Go take ~ 80 ms to complete the tests.

Optimization efforts are still ongoing (i.e. monomorphization around ADTs, Records…).

1 Like

@harryprayiv

Just so you know, I’m currently moving optimizations shared by phpurs and gopurs into a TASTed branch of purescript-backend-optimizer, locally. In my opinion, the gopurs project isn’t mature enough yet to be ported to Nix (still a mess, dependencies are still local, etc.). I’ll let you know when it’s ready!

And phpurs is going through huge improvements by the way. Even if PHP is not massively typed, it partially is, and the compiled code will be faster thanks to TASTs.

1 Like

News #6:

gopurs performance is starting to take off, with 69 ms. We’re getting closer and closer to native performance levels.

Online benchmarks often show a 1.5x to 2.5x speed advantage for Go over JS (V8). Our benchmark aligns perfectly with this, showing a ~ 2x speedup between the official compiler and gopurs .

1 Like

News #7:

A new approach, inspired by Koka/Lean, is currently being tested: FBIP/Sticky Sharing. This could represent a huge leap forward in terms of performance, bringing us closer to native results. But it completely changes the game (i.e., the entire codebase).

TASTs play a crucial role in this (@harryprayiv I’ve edited my last message, in the WASM thread, regarding this subject: a POC has been made!).

1 Like

Read through the gopurs README. The flat Value struct instead of any is a good call, that boxing cost is exactly what made the older Go backend slow, and building on backend-optimizer rather than parsing CoreFn yourself seems right.

One thing I keep getting stuck on in the benchmark: Value comes out at 240ms against 250ms for static Go. A tagged union with a tag check shouldn’t be able to beat monomorphic Go on the same work, so I’d want to rule out the loop being partly optimized away before leaning on that number. And if it does hold, doesn’t it undercut the TAST case? If Value is already at static-Go parity there’s nothing left for monomorphization to win.

Separately on FBIP: Perceus works because everything is a refcounted heap pointer with dup/drop at every use. A Value passed by value gets copied constantly with no ownership event on the pointer arm. Adding reuse seems to mean putting refcount discipline back onto the representation whose whole benefit is being cheap to copy. Is that what “changes the entire codebase” means? Because that reads like trading the thing you’ve measured for the thing you haven’t.

1 Like

For FBIP:

I tried various workarounds.

One of them was to use a reference counter in Value. We’d increment it without worrying about decrementing it (that resulted in -50% for RBTree perfs). But that required reworking the pattern matching to copy the values into temporary variables before using them later, and so on. Ultimately, it completely undermined both PureScript’s design and Go’s strengths (its GC). Given my priorities, I scrapped this experimental feature. It might be worth revisiting for another backend in the future, but for Go, I don’t think it’s the right path to take right now.

The FFI feature of gopurs allows us to use fully native types in .go files. We can still use them to achieve good performance in critical hot spots.

I’m going to refocus on cleaning up the compiler, adding support for Aff, and the official release.

1 Like

For Value & TASTs:

I’ll be traveling until the end of the week. The questions raised are very relevant, and I’ll be happy to answer them upon my return.

1 Like

Scrapping it on a measurement is the right call, and publishing the negative result is more useful than most positive ones. -50% on RBTree is worth recording somewhere permanent, since tree insertion is Koka’s showcase case for reuse and it’s the first thing anyone else will reach for.

One narrow caveat for the archive, not an argument to resume: a counter that increments and never decrements isn’t Perceus, it’s a monotone “everything is shared” signal, so reuse would never fire in the first place. The result is probably about that shortcut plus the by-value Value representation rather than about FBIP on Go in general. Someone will come back to this in a year and that distinction will save them a week.

On benchmarks, and given you clearly do measure, the issue is legibility rather than rigour. Across the thread the reference point moves (static Go, then V8, then Arista’s 80ms, then ~2x over the official compiler) and the headline figures (40%, x10, then 125 to 89 to 80 to 69) don’t compose into one series anyone else can read. A single frozen suite, every configuration against the same baseline on the same machine, would make it citable. Right now I can’t point people at it, which is a shame given the work behind it. Related: the 240ms Value figure is still unqualified in the README, and you’ve since said Value is only at parity for simple algorithms.

The piece I think matters most is tcorefn, and a cleanup phase is the right time to write the format spec before the PR. The diagnosis is right and CoreFn’s type erasure has been the known ceiling for AOT backends for years. From the JSON you posted, some things that will come up in review:

  • Rows. “Record”: { “identity”: … } uses labels as object keys. PureScript rows are ordered, admit duplicate labels, and can be open with a tail variable. None of that survives a flat map.
  • Quantifiers. forall a. is gone from the isJust example. Monomorphization needs the binding structure, not just occurrences of TypeVar a.
  • Constraints and kinds. I don’t see either represented.
  • Encoding consistency. “ret”: “Boolean” is a bare string while other types are tagged objects. That needs to be uniform before anyone writes a decoder against it.
  • Size. corefn.json is already a build-time complaint, and a type on every node is a large multiplier. Someone will ask for the numbers.

The structural one: this serializes post-elaboration types, which commits the compiler to the stability of an internal representation, skolems and unknowns included. That’s why this has stalled before, and it’s a spec question rather than an implementation one. I’d like to see it land, and I’m happy to review a draft if that’s useful.

One thing on Aff while it’s still ahead of you rather than behind you. I’d be interested in the goroutine mapping semantically rather than in performance terms. Aff is a cooperative single-threaded scheduler with specific guarantees around cancellation, forking and supervision, and Ref has no synchronisation. Mapping it onto goroutines buys real parallelism, which changes the semantics and not just the speed. Worth deciding deliberately whether gopurs Aff is the same Aff or a different one behind the same API, because library authors will assume the former.

Cleanup, Aff, release sounds like the right order to me. :blush:

1 Like

Thanks for the detailed feedback, @harryprayiv

Just to clarify regarding the FBIP experiment: the “-50% on RBTree” actually meant a 50% gain in performance (execution time). However, properly implementing the reference decrementing was extremely tedious and complex to get right (all other tests were either slower or broken), which was one of the main factors in abandoning this approach for now.

For all the other points (benchmarks legibility, the tcorefn format specifications, and the Aff semantics), we are completely on the same page. It’s all in my notes. :wink:

I’ll focus on the cleanup phase as planned, and I’ll gladly take you up on your offer to review the tcorefn spec draft when it’s ready.

Cheers!

1 Like

I love your enthusiasm and I’m happy to help!

1 Like

News #8:

Aff is now fully supported! :tada:

To achieve this, we’re diving headfirst into goroutines. Of course, primitives like AVar and Ref now include thread-safeguards that classic JS Aff didn’t need to implement. This isn’t an actor model like purerl, nor is it a hack to mimic a single-threaded Event Loop: Go introduces true, shared-memory asynchronous parallelism, rather than just single-threaded concurrency.

But from a DX perspective? It’s still the straightforward Aff you know and love. The underlying parallelism simply boosts performance behind the scenes.

Another major milestone has just been reached for gopurs: the unit tests for a real, full-scale project are now passing (259/259 green tests :white_check_mark:). Integration tests will be my next step, but this validates a significant portion of what has been implemented so far, especially since the tests themselves are formidable Aff nesting dolls (combining Spec, Aff, AVar, and complex FFI). You can see how it looks here: I swap from JS to PHP, then from PHP to Go.

The project is still a mess, but I’m staying focused on the results. I’ll make it maintainable down the line, when we’re ready to get this project running at a steady pace. I’ll keep you posted (e.g. Aff benchmarks).

:v:

Note: @harryprayiv I haven’t forgotten you, and I’m working on a reply for you!

Hi @harryprayiv

Regarding your question about the Value vs TAST paradox and the 240ms benchmark: your intuition was absolutely right.

In that specific micro-benchmark, the Go compiler was able to heavily optimize the tight loop (likely through aggressive inlining and branch prediction), which artificially brought the Value (tagged union) approach up to par with static Go.

However, as soon as we step out of these simple algorithms and run real-world, complex code (with deep ADTs, nested Records, and higher-order functions), the cumulative overhead of runtime tag checks and the limitations it imposes on the Go compiler become a real bottleneck too, even it’s still far better than using any everywhere. This is the first “ceiling” I hit. Value is a fantastic way to avoid boxing and GC pressure, but it cannot truly match native performance at scale.

This is exactly why TAST (and partial monomorphization) were the real game changers here. They aren’t meant to optimize simple loops (where Value already shines), but to break that ceiling on complex code.

To avoid any confusion for future readers, I have edited my original post regarding the 240/250ms benchmark.

I’ve also updated the benchmark links across my previous posts to ensure they accurately reflect the historical state of those experiments (for instance, pointing to the exact snapshots where gopurs was hitting >100ms rather than the current ~67ms). Before TASTs, the best I could get was 1000ms. Yes… more than 1 second… At that point, I’d had 2 or 3 very very short nights in a row, and I had even almost given up on the project and on the hope of creating an effective compilation…

Once the project is mature, all the information will be included in the README.md file.

Thanks again for your sharp and constructive feedback. :pray:

(and as for the TAST improvements, I’m just getting started on them)

1 Like

News #9:

FFI & DX: a Wasm AST Parser :mage:

One of the pain points with FFI in alternative backends is the boilerplate (manual boxing/unboxing, runtime dependencies). With gopurs, I wanted FFI to feel 100% native for Go developers.

To achieve this, the compiler uses a Go AST parser compiled to WebAssembly (ffi_gen.wasm) that analyzes your .go FFI files on the fly. It reads your Go signatures and automatically generates the perfect, highly-optimized bridging code.

1. Uncurrying & idiomatic native types

I also wanted to avoid the hassle of currying. In modern JS (ES6+), writing a => b => a + b is highly ergonomic. But in Go, manual currying quickly becomes a nightmare of nested func(interface{}) interface{} closures.

By analyzing your signatures, the Wasm parser allows you to write perfectly flat and strongly typed Go functions:

Before (Typical FFI boilerplate with manual currying & boxing):

func DoMath(arg0 gopurs_runtime.Value) gopurs_runtime.Value {
    return gopurs_runtime.Func(func(arg1 gopurs_runtime.Value) gopurs_runtime.Value {
        a := gopurs_runtime.Unbox[int64](arg0)
        b := gopurs_runtime.Unbox[int64](arg1)
        return gopurs_runtime.Box(a + b)
    })
}

After (gopurs FFI):

func DoMath(a int64, b int64) interface{} {
    return a + b
}

You just write pure Go. The generated bridge takes care of all the uncurrying and type conversions under the hood.

2. Flexible Effect semantics

In PureScript, Effect delays execution. Thanks to the Wasm parser, you can choose exactly how you want your native Go effects to behave, and the compiler will adapt the generated wrapper automatically:

Option A: Flattened (N+1 argument)
Perfect for simple, direct side-effects.

func Log(message string, _ interface{}) interface{} {
    fmt.Println(message)
    return nil
}

Option B: Curried (returning a closure)
Perfect for caching expensive pure work before the effect runs.

func MatchRegex(pattern string) func(interface{}) interface{} {
    compiled := regexp.MustCompile(pattern) // Evaluated ONCE!
    
    return func(_ interface{}) interface{} {
        return compiled.MatchString("...")  // Evaluated on Effect run
    }
}

The Wasm parser detects your Go function’s return type (func(...)) and automatically adapts the backend optimizer to preserve your partial application caching, without sacrificing the overall uncurrying performance.

Best of both worlds.