Announcing Spago Next: a PureScript rewrite, Registry support, and more

Announcements
#1

what is this?

The next question to consider after launching the alpha of the new Registry one year ago was “how are we going to support this in package managers?”
We briefly attempted to bolt some Registry support onto the Spago codebase, with poor results. So, after running a few small experiments to test the viability of a new implementation from scratch, we decided to sink the old Haskell codebase (now living here), and start anew with a PureScript implementation of Spago.

This is what I’m proud to announce today: Spago Next is a full rewrite of Spago, rebuilt from the ground up, in PureScript, to integrate with the new Registry from day one. :tada:

Today’s release is an “alpha”: it works well enough to be usable for day-to-day needs (we use it at work since months already) but it still has a few rough edges, and some things may change dramatically before a release that we can call “stable”.

Nonetheless, it is already available for Linux, macOS and Windows, and you can install it as follows:

npm install -g spago@next

Documentation can be found in the usual place

what’s new?

Quite a lot of things - let’s go through the main ones:

  1. It’s implemented in PureScript, which will make it easier for you all to contribute!
    Personally speaking I always had trouble with Haskell tooling even though I spent a lot of time with it.
    PureScript’s tooling is easier to grasp, and I find myself being able to patch Spago much more quickly than it was possible with the Haskell codebase. I hope this will make it more accessible for new contributors and users alike, since any failure modes will be the ones that we are used to deal with rather than some arcane Haskell error.

  2. Dhall has been replaced by YAML.
    This is a complex issue and I’m aware that some folks will be disappointed about this.
    We found two main issues with using Dhall as a config language for Spago:

    • it’s an unfamiliar language, which meant that folks that were approaching PureScript for the first time - especially coming from JS - would have to learn an additional language just to make sense of the build, which is somewhat suboptimal if the goal is to have an onboarding process that is as smooth as possible
    • it’s too powerful: while new users would struggle with the unfamiliar language, power users - the ones comfortable with the language - would be unable to use it to its fullest, because its power meant that Spago was supposed to deal with very complex AST-manipulation operations if we were to support even just simple “programmatic editing of your configuration”. E.g. this is how the patch for “spago install can work with all the Dhall files out there” looked like - it’s a lot of code because it needs to support a lot of edge cases, and we deemed this complexity unacceptable.

    As of “why not TOML”: I experimented with TOML for a few months, and ended up with a terrible feeling about it. The specification is underspecified, the language is fairly verbose in the end, handling inline records is not pleasant as they all have to be on a single line, and so on. I just couldn’t come up with a schema that produced a config that was nice to look at.
    So YAML it is - it’s widely used in the industry, it allows comments, and the config files are actually quite pleasant to look at! I’m aware that it’s a complicated beast, but we are parsing it so that only JSON-like things come through, so no weird things. If you are able to break out of the sandbox please open an issue or a pull request :slight_smile:

  3. Proper multipackage/monorepo/polyrepo support: handling multiple packages in a single repository has traditionally been a pain in PureScript, and the old Spago doesn’t really help with it.
    This one should make it considerably easier - head over to the monorepo docs for more details.
    Once the Registry gets out of its alpha state then it will be possible to even publish packages from this monorepo setup!

  4.  spago publish

    That’s right, this time Spago should be able to just publish your package - right from the command line, with no other tools required :rocket:
    This is all thanks to the fact that there is a lot of shared code between the Registry pipeline and Spago, so your package will run through the same checks locally before Spago hands it over to the Registry.

  5. Solver support: the old Spago only supported a package-sets-based workflow. Again thanks to the deep Registry integration Spago is now able to offer a solver-based build plan, where you don’t have to specify a package set, and instead define version bounds for the solver to build a plan based on the packages in the Registry, truly replacing the last reason for using Bower. See the docs for more info

can you help?

Yes, absolutely!

Any help is welcome:

  • we’ll need folks to test this and to report any issues or weird things
  • and if you feel like contributing with code that’s even more lovely - there are still a few things missing so any additional pair of hands is a blessing. We keep a list of tickets that are up for grabs for new contributors. Don’t hesitate to ask for clarifications if you’d like to work on one and it’s unclear and/or poorly specified :smile:

how long will this alpha state be?

For a while, a few months at least, or until we work through the known bugs for the alpha release - whichever is longer :slight_smile:

Enjoy!

30 Likes
#2

Congrats, and thank you for all your hard work!

3 Likes
#3

Amazing job! Looking forward to migrating my spago.dhall files :slight_smile:

3 Likes
#4

Really cool! I’m starting testing it right away. Thanks a lot for all the work!

1 Like
#5

Wow! Amazing job and write up. I was wondering why dhall was dropped when I heard about it a while ago, but I can totally see the reasoning now.

The work done here looks very promising! Having the codebase in PureScript now is awesome. I can’t wait to have a proper look at the source code to learn.

Congrats on the alpha release!

2 Likes
#6

A quick update on the above: migrating your spago.dhall files to spago.yaml files is now easier - I just published the spago-legacy package to NPM, so you can have the spago-legacy and spago commands available in the same shell.

The “migration” section of the docs is also up to date, and you can find it here.

Enjoy!

7 Likes
#7

I have a question on monorepo support.

If I have several libraries in a monorepo, and I use the shiny new Spago to publish them to the registry, can my consumers use them while running the rusty old Spago? Or do they have to also be on the new one?

#8

I’m pretty sure they need to be on the new spago.

Maybe there’s a hacky way to make things work on the old Spago? But, the new spago is likely what you’d want.

1 Like
#9

Got it. So this means I shouldn’t migrate libraries to monorepos yet.

#10

@fsoikin Unfortunately you won’t be able to put libraries in monorepos for a little while; even though both the new Spago and the registry both have implemented support for monorepos, this support is disabled in the registry. That’s because Pursuit and older package managers don’t know how to deal with them.

Once Pursuit has been updated to handle libraries from monorepos we’ll open them up in the registry, too. Those libraries won’t be usable by legacy package managers but at that point spago@next will have been out of alpha for a long time and users will simply need to switch to it (or another monorepo-supporting package manager).

2 Likes
#11

Implemented in doesn’t specify the PureScript back-end. Could the community expect a future version in the Chez Scheme or other native back-end? It would be awesome to see the tool, like the compiler & old Spago, be available without the need for Microsoft’s npm or requiring Node (while making sure a native back-end frequently dogfooded within the community).

#12

Yes, the main idea behind the work on the Chez backend is that it would be fast enough to allow us to port the compiler and the rest of the tooling to it.
That’s still not ready though, so for now we have to rely on Node and NPM.

2 Likes