Leveraging a blazing-fast runtime: a (new) Go backend for PureScript

[0x000000000000000000]

News #2:

Victory!

That’s fantastic.

The TAST approach has more than paid off: that’s a big x10 in perfs. So much so that I’ll soon be submitting a PR to add tcorefn (= corefn JSON + dataDecls array at the root + ann.type on each node) to the available options for purs compile --codegen (among corefn, docs, js…). ADTs, Records and functions have been added to the whitelist, and they’ve enabled numerous improvements (struct, eager uncurrying…).

For now, I’m working with a fork of Purescript, but I think this option will greatly benefit the official and durable advent of AOT backends like Go or WASM, complementing the JIT backends (JS, PHP…) that have been in place so far.

In my stress tests, the gopurs compiler now produces code that performs as well as the official JS compiler on V8. AOT is not longer a problem. And all of this on a single thread! Here is the benchmark. I’ll enrich it with new tests, but that’s a good starting point.

I’m not even mentioning the benefits of multithreading for speed, RAM usage that’s much lower than V8’s, the instant startup of a very lightweight binary, and so on. It started out as a dream, given the real constraints that existed (and that we’d already faced in the past), but it’s becoming a reality.

I hope to share this with our lovely PureScript community soon, as a production-ready backend.

The Arista compiler perf is still a goal to be achieved. But the optimizations aren’t finished yet (e.g. Constraint Dictionaries). And I fully intend to reach the 80 ms they managed to get (with V8 too, in my benchmark). That should be closer to the usual x1.5 factor we can see online, in exhaustive Go/V8 computational benchmarks. If I can go even lower than that, I’ll obviously do it!

That’s the next step, alongside a full-scale test on a real-world project (Aff, logging, FFI Postgres, S3, RabbitMQ…).

Cheers

[0x000000000000000000]

News #3:

Great improvements for FFI: no need to worry about what the compiler is doing. You can write anything, including using any! Everything will be converted on the fly by the compiler to ensure the best possible performance. You can write var foo = function or function foo. No package header needed.

[0x000000000000000000]

Note on news #2:

To give everyone a concrete idea of why this custom tcorefn (Typed CoreFn) makes building AOT backends drastically easier, I thought I’d share what the JSON output actually looks like under the hood.

It is important to note that tcorefn.json simply pushes further the idea of a desugared JSON that represents the PureScript code, by whitelisting additional info (not needed for JIT backends like JS).

Instead of just dumping source spans, the compiler now hands over the full memory layout (via dataDecls) and the deep type information at every AST node (via annotation.type).

Here are only 3 quick side-by-side examples of PureScript code and the resulting condensed JSON:

1. Data Declarations (no more guessing around ADTs)

In standard CoreFn, constructors appear as you go, forcing the backend to gather them. With tcorefn, the AOT backend knows the exact memory structure as soon as it reads the file.

PureScript:

data Maybe a = Nothing | Just a

tcorefn.json (at the root of the module):

"dataDecls": [
  {
    "typeName": "Maybe",
    "constructors": [
      { "constructorName": "Nothing", "fieldTypes": [] },
      { "constructorName": "Just", "fieldTypes": [ { "TypeVar": { "name": "a" } } ] }
    ]
  }
],
"decls": ...

The backend immediately sees the ADT structure. No need to crawl the AST to infer constructor arities or memory layouts.

---

2. Fully Typed AST Nodes (no more “Shadow Typing”)

In classic CoreFn, an AST node only holds a sourceSpan. With this format, every node knows exactly what type it handles.

PureScript:

-- implemented internally as: isJust = maybe false (const true)
isJust :: forall a. Maybe a -> Boolean

tcorefn.json (excerpt showing the root expression for isJust):

"expression": {
  "type": "App",
  "annotation": {
    "sourceSpan": { "start": [279, 10], "end": [279, 34] },
    // HERE: The full function signature is retained on the node
    "type": {
      "Func": {
        "args": [ { "ADT": { "path": ["Data", "Maybe", "Maybe"], "args": [ { "TypeVar": { "name": "a" } } ] } } ],
        "ret": "Boolean"
      }
    }
  },
  "abstraction": {
    "type": "App",
    "abstraction": { "type": "Var", "value": { "identifier": "maybe" } },
    "argument": { "type": "Literal", "value": { "value": false } }
  },
  "argument": {
    "type": "App",
    "abstraction": { "type": "Var", "value": { "identifier": "const" } },
    "argument": {
      "type": "Literal",
      "value": { "literalType": "BooleanLiteral", "value": true },
      "annotation": {
        // Primitive types are also explicitly attached to literals deep down
        "type": "Boolean" 
      }
    }
  }
}

Every single AST node in decls has its exact type attached. You don’t need an external JIT-like type-checker or shadow structures in the backend to know if you’re dealing with a Boolean, an Int32, or a specific ADT.

---

3. Records and FFI (structural typing preserved)

One of the biggest challenges for AOT is knowing when you are manipulating a raw record/structure. This JSON makes it explicit.

PureScript:

foreign import identity :: forall a. { identity :: a } -> a

tcorefn.json (excerpt):

"abstraction": {
  "type": "Var",
  "value": { "identifier": "identity", "moduleName": ["Control", "Category"] },
  "annotation": {
    "meta": { "metaType": "IsForeign" },
    "type": {
      "Func": {
        // The exact structure of the Record is preserved!
        "args": [ { "Record": { "identity": { "TypeVar": { "name": "a" } } } } ],
        "ret": { "TypeVar": { "name": "a" } }
      }
    }
  }
}

When calling FFI or handling PureScript records, the backend receives the exact shape of the expected object. This makes generating C/Go structs (or anything similar, in other languages) drastically easier. And of course, all of this is recursive (e.g. "args": [ { "Record": { "identity": { "Record": ... } } } ])

[0x000000000000000000]

News #4:

Final stretch: It’s getting harder and harder to make gains.

But perfs have greatly improved: 125 ms has become 89 ms. Full benchmark here.

I’m currently tackling what will likely be the final piece of the puzzle: monomorphization. It’s a relatively complex topic, even if it’s simple on paper. If it works out, we can hopefully reach Arista’s 80 ms milestone.

But we’re already very close. For an AOT compiling flow, this is truly satisfying, because it required moving into the compiler what modern JITs have been constantly striving to improve for years.

Stay tuned.

[0x000000000000000000]

News #5:

Monomorphization is done on the simplest parts (i.e. primitives). Arista’s miletone has been reached: both Arista ES and Go take ~ 80 ms to complete the tests.

Optimization efforts are still ongoing (i.e. monomorphization around ADTs, Records…).

[0x000000000000000000]

@harryprayiv

Just so you know, I’m currently moving optimizations shared by phpurs and gopurs into a TASTed branch of purescript-backend-optimizer, locally. In my opinion, the gopurs project isn’t mature enough yet to be ported to Nix (still a mess, dependencies are still local, etc.). I’ll let you know when it’s ready!

And phpurs is going through huge improvements by the way. Even if PHP is not massively typed, it partially is, and the compiled code will be faster thanks to TASTs.

[0x000000000000000000]

News #6:

gopurs performance is starting to take off, with 69 ms. We’re getting closer and closer to native performance levels.

Online benchmarks often show a 1.5x to 2.5x speed advantage for Go over JS (V8). Our benchmark aligns perfectly with this, showing a ~ 2x speedup between the official compiler and gopurs .

[0x000000000000000000]

News #7:

A new approach, inspired by Koka/Lean, is currently being tested: FBIP/Sticky Sharing. This could represent a huge leap forward in terms of performance, bringing us closer to native results. But it completely changes the game (i.e., the entire codebase).

TASTs play a crucial role in this (@harryprayiv I’ve edited my last message, in the WASM thread, regarding this subject: a POC has been made!).

[harryprayiv]

Read through the gopurs README. The flat Value struct instead of any is a good call, that boxing cost is exactly what made the older Go backend slow, and building on backend-optimizer rather than parsing CoreFn yourself seems right.

One thing I keep getting stuck on in the benchmark: Value comes out at 240ms against 250ms for static Go. A tagged union with a tag check shouldn’t be able to beat monomorphic Go on the same work, so I’d want to rule out the loop being partly optimized away before leaning on that number. And if it does hold, doesn’t it undercut the TAST case? If Value is already at static-Go parity there’s nothing left for monomorphization to win.

Separately on FBIP: Perceus works because everything is a refcounted heap pointer with dup/drop at every use. A Value passed by value gets copied constantly with no ownership event on the pointer arm. Adding reuse seems to mean putting refcount discipline back onto the representation whose whole benefit is being cheap to copy. Is that what “changes the entire codebase” means? Because that reads like trading the thing you’ve measured for the thing you haven’t.

[0x000000000000000000]

For FBIP:

I tried various workarounds.

One of them was to use a reference counter in Value. We’d increment it without worrying about decrementing it (that resulted in -50% for RBTree perfs). But that required reworking the pattern matching to copy the values into temporary variables before using them later, and so on. Ultimately, it completely undermined both PureScript’s design and Go’s strengths (its GC). Given my priorities, I scrapped this experimental feature. It might be worth revisiting for another backend in the future, but for Go, I don’t think it’s the right path to take right now.

The FFI feature of gopurs allows us to use fully native types in .go files. We can still use them to achieve good performance in critical hot spots.

I’m going to refocus on cleaning up the compiler, adding support for Aff, and the official release.

[0x000000000000000000]

For Value & TASTs:

I’ll be traveling until the end of the week. The questions raised are very relevant, and I’ll be happy to answer them upon my return.

[harryprayiv]

Scrapping it on a measurement is the right call, and publishing the negative result is more useful than most positive ones. -50% on RBTree is worth recording somewhere permanent, since tree insertion is Koka’s showcase case for reuse and it’s the first thing anyone else will reach for.

One narrow caveat for the archive, not an argument to resume: a counter that increments and never decrements isn’t Perceus, it’s a monotone “everything is shared” signal, so reuse would never fire in the first place. The result is probably about that shortcut plus the by-value Value representation rather than about FBIP on Go in general. Someone will come back to this in a year and that distinction will save them a week.

On benchmarks, and given you clearly do measure, the issue is legibility rather than rigour. Across the thread the reference point moves (static Go, then V8, then Arista’s 80ms, then ~2x over the official compiler) and the headline figures (40%, x10, then 125 to 89 to 80 to 69) don’t compose into one series anyone else can read. A single frozen suite, every configuration against the same baseline on the same machine, would make it citable. Right now I can’t point people at it, which is a shame given the work behind it. Related: the 240ms Value figure is still unqualified in the README, and you’ve since said Value is only at parity for simple algorithms.

The piece I think matters most is tcorefn, and a cleanup phase is the right time to write the format spec before the PR. The diagnosis is right and CoreFn’s type erasure has been the known ceiling for AOT backends for years. From the JSON you posted, some things that will come up in review:

• Rows. “Record”: { “identity”: … } uses labels as object keys. PureScript rows are ordered, admit duplicate labels, and can be open with a tail variable. None of that survives a flat map.
• Quantifiers. forall a. is gone from the isJust example. Monomorphization needs the binding structure, not just occurrences of TypeVar a.
• Constraints and kinds. I don’t see either represented.
• Encoding consistency. “ret”: “Boolean” is a bare string while other types are tagged objects. That needs to be uniform before anyone writes a decoder against it.
• Size. corefn.json is already a build-time complaint, and a type on every node is a large multiplier. Someone will ask for the numbers.

The structural one: this serializes post-elaboration types, which commits the compiler to the stability of an internal representation, skolems and unknowns included. That’s why this has stalled before, and it’s a spec question rather than an implementation one. I’d like to see it land, and I’m happy to review a draft if that’s useful.

One thing on Aff while it’s still ahead of you rather than behind you. I’d be interested in the goroutine mapping semantically rather than in performance terms. Aff is a cooperative single-threaded scheduler with specific guarantees around cancellation, forking and supervision, and Ref has no synchronisation. Mapping it onto goroutines buys real parallelism, which changes the semantics and not just the speed. Worth deciding deliberately whether gopurs Aff is the same Aff or a different one behind the same API, because library authors will assume the former.

Cleanup, Aff, release sounds like the right order to me.

[0x000000000000000000]

Thanks for the detailed feedback, @harryprayiv

Just to clarify regarding the FBIP experiment: the “-50% on RBTree” actually meant a 50% gain in performance (execution time). However, properly implementing the reference decrementing was extremely tedious and complex to get right (all other tests were either slower or broken), which was one of the main factors in abandoning this approach for now.

For all the other points (benchmarks legibility, the tcorefn format specifications, and the Aff semantics), we are completely on the same page. It’s all in my notes.

I’ll focus on the cleanup phase as planned, and I’ll gladly take you up on your offer to review the tcorefn spec draft when it’s ready.

Cheers!

[harryprayiv]

@0x000000000000000000

I’ll gladly take you up on your offer to review the tcorefn spec draft when it’s ready.

I love your enthusiasm and I’m happy to help!

[0x000000000000000000]

News #8:

Aff is now fully supported!

To achieve this, we’re diving headfirst into goroutines. Of course, primitives like AVar and Ref now include thread-safeguards that classic JS Aff didn’t need to implement. This isn’t an actor model like purerl, nor is it a hack to mimic a single-threaded Event Loop: Go introduces true, shared-memory asynchronous parallelism, rather than just single-threaded concurrency.

But from a DX perspective? It’s still the straightforward Aff you know and love. The underlying parallelism simply boosts performance behind the scenes.

Another major milestone has just been reached for gopurs: the unit tests for a real, full-scale project are now passing (259/259 green tests ). Integration tests will be my next step, but this validates a significant portion of what has been implemented so far, especially since the tests themselves are formidable Aff nesting dolls (combining Spec, Aff, AVar, and complex FFI). You can see how it looks here: I swap from JS to PHP, then from PHP to Go.

The project is still a mess, but I’m staying focused on the results. I’ll make it maintainable down the line, when we’re ready to get this project running at a steady pace. I’ll keep you posted (e.g. Aff benchmarks).

Note: @harryprayiv I haven’t forgotten you, and I’m working on a reply for you!

[0x000000000000000000]

Hi @harryprayiv

Regarding your question about the Value vs TAST paradox and the 240ms benchmark: your intuition was absolutely right.

In that specific micro-benchmark, the Go compiler was able to heavily optimize the tight loop (likely through aggressive inlining and branch prediction), which artificially brought the Value (tagged union) approach up to par with static Go.

However, as soon as we step out of these simple algorithms and run real-world, complex code (with deep ADTs, nested Records, and higher-order functions), the cumulative overhead of runtime tag checks and the limitations it imposes on the Go compiler become a real bottleneck too, even it’s still far better than using any everywhere. This is the first “ceiling” I hit. Value is a fantastic way to avoid boxing and GC pressure, but it cannot truly match native performance at scale.

This is exactly why TAST (and partial monomorphization) were the real game changers here. They aren’t meant to optimize simple loops (where Value already shines), but to break that ceiling on complex code.

To avoid any confusion for future readers, I have edited my original post regarding the 240/250ms benchmark.

I’ve also updated the benchmark links across my previous posts to ensure they accurately reflect the historical state of those experiments (for instance, pointing to the exact snapshots where gopurs was hitting >100ms rather than the current ~67ms). Before TASTs, the best I could get was 1000ms. Yes… more than 1 second… At that point, I’d had 2 or 3 very very short nights in a row, and I had even almost given up on the project and on the hope of creating an effective compilation…

Once the project is mature, all the information will be included in the README.md file.

Thanks again for your sharp and constructive feedback.

(and as for the TAST improvements, I’m just getting started on them)

[0x000000000000000000]

News #9:

FFI & DX: a Wasm AST Parser

One of the pain points with FFI in alternative backends is the boilerplate (manual boxing/unboxing, runtime dependencies). With gopurs, I wanted FFI to feel 100% native for Go developers.

To achieve this, the compiler uses a Go AST parser compiled to WebAssembly (ffi_gen.wasm) that analyzes your .go FFI files on the fly. It reads your Go signatures and automatically generates the perfect, highly-optimized bridging code.

1. Uncurrying & idiomatic native types

I also wanted to avoid the hassle of currying. In modern JS (ES6+), writing a => b => a + b is highly ergonomic. But in Go, manual currying quickly becomes a nightmare of nested func(interface{}) interface{} closures.

By analyzing your signatures, the Wasm parser allows you to write perfectly flat and strongly typed Go functions:

Before (Typical FFI boilerplate with manual currying & boxing):

func DoMath(arg0 gopurs_runtime.Value) gopurs_runtime.Value {
    return gopurs_runtime.Func(func(arg1 gopurs_runtime.Value) gopurs_runtime.Value {
        a := gopurs_runtime.Unbox[int64](arg0)
        b := gopurs_runtime.Unbox[int64](arg1)
        return gopurs_runtime.Box(a + b)
    })
}

After (gopurs FFI):

func DoMath(a int64, b int64) interface{} {
    return a + b
}

You just write pure Go. The generated bridge takes care of all the uncurrying and type conversions under the hood.

2. Flexible Effect semantics

In PureScript, Effect delays execution. Thanks to the Wasm parser, you can choose exactly how you want your native Go effects to behave, and the compiler will adapt the generated wrapper automatically:

Option A: Flattened (N+1 argument)
Perfect for simple, direct side-effects.

func Log(message string, _ interface{}) interface{} {
    fmt.Println(message)
    return nil
}

Option B: Curried (returning a closure)
Perfect for caching expensive pure work before the effect runs.

func MatchRegex(pattern string) func(interface{}) interface{} {
    compiled := regexp.MustCompile(pattern) // Evaluated ONCE!
    
    return func(_ interface{}) interface{} {
        return compiled.MatchString("...")  // Evaluated on Effect run
    }
}

The Wasm parser detects your Go function’s return type (func(...)) and automatically adapts the backend optimizer to preserve your partial application caching, without sacrificing the overall uncurrying performance.

Best of both worlds.

[0x000000000000000000]

News #10:

Here we go!

gopurs can now emulate the behavior of the event loop with respect to “fibers”: the main thread will patiently wait for child threads to finish before automatically terminating itself. Just like our ol’ Node Event Loop pal, when it detects relevant child handles. No abrupt exits when your main() function returns while Aff tasks are still running in the background.

This was a major milestone to unlock the full potential of Aff in Go. And speaking of potential… I’ve just updated the extended benchmarks to include asynchronous operations, File I/O, and most importantly: parallelism. You can find the test here.

Since Go natively maps goroutines to multiple OS threads, gopurs implicitly transforms forkAff into true multi-core execution. The results on a heavily CPU-bound parallel workload, running 4 concurrent fib 42 computations (intentionally written in a naive, unoptimized recursive way to maximize CPU stress), speak for themselves:

Benchmark	JS (V8)	Arista ES (V8)	Go (gopurs)
File I/O (10k writes/reads)	~ 401 ms	~ 442 ms	~ 421 ms
Aff Ops (Async Delays)	~ 11 ms	~ 10 ms	~ 11 ms
Parallelism (4x Fib 42)	~ 5863 ms	~ 6028 ms	~ 1181 ms
Total Execution Time	~ 6403 ms	~ 6560 ms	~ 1690 ms

JS and Arista run Fib 42 sequentially on a single thread blocking the event loop, while Go successfully distributes the 4 heavy tasks across 4 different CPU cores, making it 4x faster for this specific workload).

Of course, the more cores you have, the wider the gap becomes. For example, testing the limits on a local machine with 10 cores (10 concurrent tasks), at home:

Benchmark	JS (V8)	Arista ES (V8)	Go (gopurs)
Parallelism (10x Fib 42)	~ 15113 ms	~ 14690 ms	~ 1255 ms

JS and Arista scale linearly and predictably choke, while Go devours the 10 tasks in roughly the same time it took for 4, making it ~12x faster. As you can see, Go’s performance has remained almost unchanged! There may only be a slight additional overhead due to orchestration (more children to manage).

---

It’s incredibly satisfying to write standard, pure PureScript code with Aff and forkAff, and get true, unboxed multi-threading entirely for free on the backend. The perf gain is awesome. This is quite similar to what TS devs have seen in their comptime with v7 (their recent game-changing major release). But here, that’s pure runtime progress.

A summary is available here.

Next step: put gopurs through its paces with integration tests (Postgres, S3, RabbitMQ…), complete the core lib FFIs (e.g. parAff), etc.

[0x000000000000000000]

News #11:

100% green on the integration tests of b8x, but more importantly on the unit tests of Aff (gopurs-aff, mirror of purescript-aff)

Aff is one of the most complex and critical core libraries in the PureScript ecosystem. It handles asynchronous fibers, AVars, intricate error handling (bracket), cancellation, parallel execution (parAff), etc.

Getting all these tests to pass proves that the Go implementation (which relies natively on true multi-threading via goroutines and shared memory) is not just blazing fast, but semantically sound and strictly correct. We’ve successfully mapped the subtle edge cases of JS cooperative concurrency to Go’s parallel execution model without breaking the API or the expected behaviors.

To be even more precise: I found that a few original tests were actually somewhat “tainted” by the JS Event Loop conceptual model. They implicitly relied on synchronous execution happening before encountering the first blocking handler (which leaked in the assertions). In a true parallel environment like Go, where forkAff spawns an actual concurrent goroutine instantly, these hidden race conditions were immediately exposed. We had to slightly adjust those tests to be truly async-safe, which further proves the strictness and robustness of this new parallel foundation. Example: before and after.

Aff is now a pure abstraction, decoupled from the historical JS runtime.

Next steps: gopurs-arrays, gopurs-assert, etc.

[0x000000000000000000]

News #12:

Finally done with the existing official tests for each module. They are all green now.

E.g. tests are passing for gopurs-arrays (Go-FFIed mirror of purescript-arrays). Of course, the PureScript code does not need to change, in your project(s). The signatures of all functions has been kept intact. The single thing that changes is the lib source in spago.yaml (e.g. Data.Array will target gopurs-arrays, not purescript-arrays).

I’m going to go back through them now and improve their coverage, by adding non-official tests. But the bulk of this looong work is behind me.

List of validated modules

1. gopurs-argonaut-core

2. gopurs-arrays

3. gopurs-assert

4. gopurs-avar

5. gopurs-catenable-lists

6. gopurs-console

7. gopurs-datetime

8. gopurs-effect

9. gopurs-enums

10. gopurs-exceptions

11. gopurs-foldable-traversable

12. gopurs-foreign

13. gopurs-foreign-object

14. gopurs-free

15. gopurs-functions

16. gopurs-integers

17. gopurs-js-bigints

18. gopurs-js-date

19. gopurs-js-promise

20. gopurs-js-promise-aff

21. gopurs-js-uri

22. gopurs-lazy

23. gopurs-node-buffer

24. gopurs-node-event-emitter

25. gopurs-node-fs

26. gopurs-node-http

27. gopurs-node-net

28. gopurs-node-path

29. gopurs-node-process

30. gopurs-node-streams

31. gopurs-now

32. gopurs-nullable

33. gopurs-numbers

34. gopurs-ordered-collections

35. gopurs-partial

36. gopurs-prelude

37. gopurs-quickcheck

38. gopurs-random

39. gopurs-record

40. gopurs-refs

41. gopurs-run

42. gopurs-simple-json

43. gopurs-spec

44. gopurs-st

45. gopurs-strings

46. gopurs-strings-extra

47. gopurs-unfoldable

48. gopurs-unsafe-coerce

49. gopurs-uuid

50. gopurs-variant

51. gopurs-yoga-json

Of course, other modules will be supported in the future (feel free to open a PR if needed).