Leveraging a blazing-fast runtime: a (new) Go backend for PureScript

[0x000000000000000000]

News #11:

100% green on the integration tests of b8x, but more importantly on the unit tests of Aff (gopurs-aff, mirror of purescript-aff)

Aff is one of the most complex and critical core libraries in the PureScript ecosystem. It handles asynchronous fibers, AVars, intricate error handling (bracket), cancellation, parallel execution (parAff), etc.

Getting all these tests to pass proves that the Go implementation (which relies natively on true multi-threading via goroutines and shared memory) is not just blazing fast, but semantically sound and strictly correct. We’ve successfully mapped the subtle edge cases of JS cooperative concurrency to Go’s parallel execution model without breaking the API or the expected behaviors.

To be even more precise: I found that a few original tests were actually somewhat “tainted” by the JS Event Loop conceptual model. They implicitly relied on synchronous execution happening before encountering the first blocking handler (which leaked in the assertions). In a true parallel environment like Go, where forkAff spawns an actual concurrent goroutine instantly, these hidden race conditions were immediately exposed. We had to slightly adjust those tests to be truly async-safe, which further proves the strictness and robustness of this new parallel foundation. Example: before and after.

Aff is now a pure abstraction, decoupled from the historical JS runtime.

Next steps: gopurs-arrays, gopurs-assert, etc.

[0x000000000000000000]

News #12:

Finally done with the existing official tests for each module. They are all green now.

E.g. tests are passing for gopurs-arrays (Go-FFIed mirror of purescript-arrays). Of course, the PureScript code does not need to change, in your project(s). The signatures of all functions has been kept intact. The single thing that changes is the lib source in spago.yaml (e.g. Data.Array will target gopurs-arrays, not purescript-arrays).

I’m going to go back through them now and improve their coverage, by adding non-official tests. But the bulk of this looong work is behind me.

List of validated modules

1. gopurs-argonaut-core

2. gopurs-arrays

3. gopurs-assert

4. gopurs-avar

5. gopurs-catenable-lists

6. gopurs-console

7. gopurs-datetime

8. gopurs-effect

9. gopurs-enums

10. gopurs-exceptions

11. gopurs-foldable-traversable

12. gopurs-foreign

13. gopurs-foreign-object

14. gopurs-free

15. gopurs-functions

16. gopurs-integers

17. gopurs-js-bigints

18. gopurs-js-date

19. gopurs-js-promise

20. gopurs-js-promise-aff

21. gopurs-js-uri

22. gopurs-lazy

23. gopurs-node-buffer

24. gopurs-node-event-emitter

25. gopurs-node-fs

26. gopurs-node-http

27. gopurs-node-net

28. gopurs-node-path

29. gopurs-node-process

30. gopurs-node-streams

31. gopurs-now

32. gopurs-nullable

33. gopurs-numbers

34. gopurs-ordered-collections

35. gopurs-partial

36. gopurs-prelude

37. gopurs-quickcheck

38. gopurs-random

39. gopurs-record

40. gopurs-refs

41. gopurs-run

42. gopurs-simple-json

43. gopurs-spec

44. gopurs-st

45. gopurs-strings

46. gopurs-strings-extra

47. gopurs-unfoldable

48. gopurs-unsafe-coerce

49. gopurs-uuid

50. gopurs-variant

51. gopurs-yoga-json

Of course, other modules will be supported in the future (feel free to open a PR if needed).

[0x000000000000000000]

@harryprayiv Opened a dedicated thread for TASTs

[0x000000000000000000]

News #13:

I’ve recently added new comparisons to the stress tests, measuring the performance of code generated by our backends against fully hand-optimized, imperative native equivalents (using the Native FFI).

We are consistently observing a ~2x performance factor between the compiled PureScript code and the raw native implementations. This is quite an interesting threshold.

When you consider the Arista ES backend (which features advanced optimizations for JS) and our own gopurs backend, both seem to plateau around this exact same overhead ratio. Having spent significant time trying to squeeze out a few more milliseconds from the Go compiler myself, and facing increasing difficulty doing so, it strongly suggests we might be reaching the hard limits of optimization here. There is an inherent, irreducible cost to maintaining high-level functional abstractions (like deep closures, persistent immutability, and recursion) on targets that aren’t purely designed for them.

But this is actually great news, because the factor is fine, and it brings us to a fundamental philosophy about how we should write and build applications:

These stress tests are deliberately unoptimized. They are designed as boundary cases to push the compilers to their absolute limits. In a real-world project, 99% of the codebase will perform remarkably close to native speeds.

This performance is more than enough for the vast majority of our work, and it grants us the immense privilege of focusing entirely on our high-level domain concepts, safely ignoring the low-level machinery details. For the remaining 1% (those critical algorithmic hot paths) we have three options: accept a 2x slowdown, use safe mutability abstractions like the ST monad (which compile down to highly efficient imperative loops), or seamlessly drop down into FFI to write raw, imperative code as close to the metal as needed.

At this stage, optimization gains are often on the scale of mere microseconds. While further improvement is always possible, chasing these micro-optimizations is no longer a priority for me, right now.

Just one exception: Go 1.27 has been released . Huge improvements, especially around generics and polymorphism. That could unlock further optimizations for gopurs. I’ll try that in the coming hours/days.

[0x000000000000000000]

News #14:

Good news. Got -10ms thanks to Go v1.27 (64ms → 54ms). Still things to do, but that reactivated the optimization step, which I thought was over.

[0x000000000000000000]

News #15:

When I started to optimize gopurs, this was a goal that seemed difficult to achieve, but it has now become the new standard: we’ve almost matched Scheme’s performance on the benchmark!

Go gives less stable metrics than Scheme; the standard deviation is higher around the mean. But it’s still a real cause for satisfaction. Scheme is one of the go-to compilers for FP, one of the most powerful, with decades of optimizations under its belt. So this is really great news.

[0x000000000000000000]

I recently noticed that @i-am-the-slime has been actively maintaining and rescuing the purescript-native and go-ffi efforts (kudos for the incredible work and dedication by the way, it’s amazing to see fellow enthusiasts keeping this alive! I also saw that you managed to implement support for Aff on your side too, which is awesome ).

Since I’ve been working on my own experimental Go backend, I thought it would be interesting to run a side-by-side benchmark to see if my alternative approach still makes sense. I added the maintained psgo (the fork of purescript-native Go backend) to my stress-test benchmark suite to compare the two.

At first glance, the results seem to validate the new architectural choices made in gopurs (TAST, backend-optimizer, intensive use of goroutines, Go 1.27 features, etc.). By preserving deep type information, gopurs can generate statically-typed Go structs and avoid interface{} boxing/dynamic type assertions, which seems to yield a significant performance boost (especially on heavy polymorphic lookups and closures). The tests are 35x faster.

However, I really want to remain as objective and fair as possible. Feel free to tell me if you spot any issues with the benchmark setup for psgo in my repository (especially @i-am-the-slime or anyone familiar with the psgo internals). I want to make sure the comparison is perfectly balanced, and that I haven’t missed any optimization flags or introduced any unfair bottlenecks on the psgo side.

My goal is a collective progress, above all

You can find the benchmark repository and the results here. The runner script for psgo is located here.

Cheers

[0x000000000000000000]

News #16:

Go exhibited some of the highest variance during cold starts (initial runs could take 69 ms, dropping and stabilizing to 50 ms when repeated). I stabilized the benchmarks by introducing a warm-up phase before taking measurements. The results are now highly accurate and consistent.

I added further optimizations (I am currently working on a Worker/Wrapper pattern, to shave off the last few microseconds).

It’s completely trivial now, but still… I have to say: what follows really made me happy…

Go has overtaken Scheme for the first time, since I started working on gopurs a month ago.

It now ranks first among the backends tested in the benchmark. (In reality, fluctuations after the decimal point make the match very close on my local machine. But hey, I’m still really happy about it.)

---

Go:

Scheme:

[0x000000000000000000]

News #17:

An article has been written about gopurs. Hope it will find readers, for PureScript’s sake.

[0x000000000000000000]

News #18:

Alright, so after pushing down the performance numbers on the benchmark (in a good way! ~44ms), there is one significant caveat left: the compiler is now working harder to achieve them. This is a classic trade-off, but when re-testing all of this on a real-world project, I ran into a few little surprises (e.g., RAM saturation).

Now, it’s time to focus on improving compile-time performance, an area I hadn’t spent much time on until now. It might be a less time-consuming task, but it’s just as essential.

To give you an example of how I’m tackling this: I’m now drawing heavy inspiration from things like Rust’s .rmeta files (we now have a .purmeta equivalent). The goal is to relieve both RAM and CPU. Instead of keeping the entire AST in memory, the RAM now only temporarily loads the specific chunks it needs from the disk, which acts as the main receptacle for memoization.

In short, fairly simple, but fundamental architectural changes.

[0x000000000000000000]

News #19:

The project is now officially entering its cleanup and maintainability phase.

However, as I started diving deep into the compiler’s performance and architecture, I’ve been maturing a rather radical choice: rewriting the compiler backend in Rust (a temporary repo will evolve here). I’ve managed to completely avoid RAM overloads using a few tricks (.purmeta, etc.), but the further I go, the more I realize that there are things more important than that missing (see the list below).

Here is why I’m considering this pivot:

• True parallelism: I want to fully leverage multi-core processing to parse and transform TAST files, etc. While we could theoretically bootstrap gopurs to compile itself and use Go’s concurrency, debugging a self-hosted AOT compiler that relies on its own generated code is a chicken-and-egg challenge I don’t really have the time to deal with, these days. Using a dedicated systems language like Rust gives us a direct and fearless parallelism out of the box.
• Raw speed: I want the compilation times to be as fast as possible, even during single-threaded execution. E.g., Rust’s performance profile and insanely fast JSON deserialization are perfectly suited for crunching the tcorefn payloads.
• Memory reliability: TAST never erases types, the AST graph we hold is really big. In GC-based environments, allocating millions of tiny AST nodes puts great pressure on the Garbage Collector. Rust allows us to bypass this entirely (e.g., using Arena Allocators), giving us a perfectly predictable, highly reliable, and performant memory model.
• Natural translation: you might wonder if porting from PureScript to Rust means losing our expressive FP abstractions. Thankfully, the core of this compiler isn’t heavily reliant on advanced type-level programming. It is essentially a pure pipeline transforming massive ADTs. This maps flawlessly to Rust: it won’t require translating into complex traits or fighting the lack of HKTs. It just means writing a lot of enums and exhaustive match statements. I’m oversimplifying a bit, but it guarantees a smooth architectural transition.

I want to be transparent: this decision might delay the official release by at least a month. But I believe the long-term benefits for our ecosystem are well worth the wait.

Perhaps purust will reach a level of maturity where we can leverage it directly one day to write compilers with native parallelism. But as it stands, writing the tooling directly in Rust seems to be the most reasonable path to ensure the compiler is as performant as possible.

Note: a Purescript developer could certainly use Node.js for local development (no AOT compiling penalty), and Go or Rust for production, but my initial goal was a bit more ambitious than that: I’d like to enable a Purescript developer (who doesn’t work on frontend projects) to completely forget about Node.js and npm. Not out of any particular animosity toward this technology, but for the sake of convenience (only one runtime target, no prod surprise, etc.).

Note #2: purbo has been created, and it is a direct Rust port of @natefaubion’s brilliant purescript-backend-optimizer. All credit goes to him for the semantic architecture, this will just be a translation to leverage Rust’s parallelism and memory management. That is something gopurs will rely on, like other backends in the future.

---

Edit: A slightly more nuanced compromise was made. More details here.

[0x000000000000000000]

News #20:

Oh… I was starting to doubt it, but in the end, the Wrapper/Worker pattern paid off in gopurs: we’re now down to 42ms… It really is a very strange feeling, like a never-ending logarithmic slide. But that’s a good thing. Benchmark updated here.

[0x000000000000000000]

News #21:

I am considering that the best approach moving forward might be to have a coexistence of PureScript and Rust code within gopurs.

The idea would be to use Rust to compile the official, highly-performant release binary, while keeping a PureScript version to compile a dev binary and easily understand what the compiler does. This could ensure that the community can easily experiment, create proof of concepts, and submit PRs using PureScript. In this context, more time to compile is absolutely not a problem.

Nowadays, AI makes it more easy to “transpile” diffs if a contributor wants to patch the Rust mirror, without learning it in depth. (Of course, I could handle the Rust porting myself. Or purust could do this automatically in the future.)

Ultimately, gopurs is a tool for the PureScript community, and my goal is for it to remain maintainable by the community, not just by me. Rust would be there strictly for performance reasons (serde for JSON decoding, parallelism…), not to serve as the documentation medium for compilation methods. PureScript remains much more refined, concise, and its expressivity (which is one of its greatest qualities) makes it the perfect language for documenting and designing the compiler’s logic.

Also: currently testing the Scrutinee Fusion method (inspired by id3as and their purescript-backend-erl compiler).

[0x000000000000000000]

News #22:

Latest work completed on Records, making the most of the power of Go struct-ures.

Deep Record Updates: 150 μs → 5 μs.

[0x000000000000000000]

News #23:

A final optimization push has been made.

The gopurs-compiled code (~ 40.5 ms), which doesn’t hesitate to use every means at its disposal (e.g., deep monomorphization, stack-allocated structs, goto…), finally reached the symbolic milestone of being twice as fast as the Arista-compiled version (~ 81 ms), on single-thread tasks.

Just in case, I’m still going to try using gopurs.js soon to compile gopurs.go so I can use it as the official binary (and thus take advantage of its parallelism). Of course, Rust is cool and useful, but I’m still a little unsure about whether it really makes sense to use it right now. The debate is still in my mind…

[0x000000000000000000]

News #24:

Another major milestone achieved: the total execution time on the core stress tests just dropped from ~40.5 ms to ~28 ms!

This massive 30% gain comes from a single, targeted compiler pass I just implemented, which I’d call “Thunk Fusion” (added a dedicated test for it in the official passing/ folder).

By deeply analyzing the TAST, gopurs can now detect when a Lazy thunk encapsulates pure, total integer arithmetic and is immediately forced by its consumer. When this precise pattern is matched, the compiler safely bypasses the dynamic closure allocation entirely and generates a strict, unboxed, tight loop in Go. It essentially synthesizes the “cheatcode” performance automatically.

Crucially, this does not break the Lazy contract or semantics:

• It only applies to total operations (no side-effects, no risk of evaluating diverging effects prematurely).
• It strictly requires an immediate consumer. If the Lazy value is passed around or stored, it gracefully falls back to a standard deferred closure.
• Recursive LetRec blocks are explicitly excluded to maintain safe initialization semantics.
• etc.

This perfectly embodies the AOT philosophy I was aiming for with this backend: letting the compiler cancel out the overhead of high-level functional abstractions (like abusing Lazy in a hot path) without forcing the developer to manually drop down to FFI or rewrite their code imperatively.

In doing so, the compiler becomes more and more forgiving of upfront design mistakes. A PureScript developer can write naive, unoptimized code (abusing closures or deferring execution unnecessarily) and the compiler will silently catch and clean up those architectural flaws behind the scenes, ensuring top-tier performance anyway.

I’m remaining cautious, of course. With optimizations this aggressive, there is always a risk of hitting unexpected edge cases in the wild. But the strict semantic guards I’ve put in place, along with the test suite passing flawlessly, give me more confidence.

We are now incredibly close to imperative and native Go (~27 ms).

Today is a good day.

[0x000000000000000000]

News #25:

Once we crossed that threshold, it didn’t take long to achieve performance levels (25ms) that got better than native performance, for the first time! At this point, we’re not even that far off from native Scheme code (20ms) or native Rust code (16ms).

That’s the magic of compiled code: it doesn’t care about being as readable as hand-optimized code. It can do whatever it takes to run as fast as possible.

What’s great is that any big compilation achievement unlocks new ground for optimizations that were already primed to act, on code still trapped in a shell waiting to be dissolved.

[erikagain]

Whoa, pretty soon you’ll be whizzing past zero into ahead-of-time execution. Very exciting stuff, hats off to you!

[0x000000000000000000]

Thank you, Erik. That’s a beautiful message that I’ll hold close to my heart when other challenges arise.

[0x000000000000000000]

News #26:

Quick note to say that I had to run through all the test suites (modules, official ones, etc.) again after the recent performance optimizations. A lot of things were broken. Everything is green again.

The cleanup is still underway.