Strategy for Updating the Ecosystem to the Upcoming v0.14.0 Release

Announcements
#1

In case you weren’t aware, the core team made a release candidate for PureScript v0.14.0 some time ago: v0.14.0-rc2. I’d like to summarize how we plan to proceed from here. In other words, I’ll explain the context behind what has occurred before, what we’ll be doing now, why, and how. At this time, I do not have concrete actions on how you can help. I wrote instructions on how to help but came across a few issues with that approach. When I update those instructions to account for a few issues, I’ll post a new thread here.

Please note that this post was written by me and is largely my understanding. Other core team members may add their thoughts that fill in gaps missing here or even counter what I am discussing below. In short, this is a nuanced situation and we’re still figuring things out.

Context

When the v0.12.0 PureScript release was made, most people were using bower for their dependency manager. psc-package (Spago’s predecessor) was being used by some people but wasn’t recommended as the default dependency manager to use. Since Bower’s solver wasn’t the best, many developers and new learners (including myself) experienced a lot of dependency hell when trying to get anything to compile. This is a situation we would like to avoid again. While there were breaking changes in v0.13.0, the breakage wasn’t nearly as bad a v0.12.0 IIRC.

Without explaining everything entirely, the upcoming v0.14.0 release has a few things that make updating the ecosystem harder:

  • Whenever a new breaking-changes PureScript release is made, we will also make breaking changes to the core purescript organization’s repos. Some of these changes can be done quickly. Others still require further thought and work to finish. Unlike non-core repos, these changes will need the feedback and oversight of the core team. Until these libraries are updated, most of the ecosystem cannot be updated. The more fixes we make here now, the less often we’ll have to do this in the future.
  • The new Coercible feature will make some libraries usage of unsafeCoerce unneeded anymore as we now have a safe coerce that is verified by the compiler. However, to support that, we will need to add role annotations to some libraries. Fortunately, @kl0tl has already done most of this work in the core purescript organization’s repos. This usually isn’t hard to do, but it does require some thought.
  • Things that were outputted via stderr have been switched to stdout. Thus, some tooling (e.g. purescript-psa) needed to be updated to handle this fact. Fortunately, this tool has been updated in v0.8.0 to handle both v0.13.x and v0.14.x PureScript compiler releases.
  • The new PolyKinds feature is probably what will cause the most work when updating the ecosystem for the following reasons:
    • Compiler warnings about missing kind signatures for declarations with inferred polymorphic kinds. This warning serves to notify that it would be better to add a kind signature here than not to. When I attempted to compile as much of the package set as I could, I got about 510 compiler warnings. A large percentage of those warnings (maybe 80%?) were due to this specific warning.
    • New syntax for explicit kind signatures. We can now more clearly annotate a type definition (e.g. data, newtype, type, and class keywords) with its kind signature. People who want to help might have difficulty understanding how to do this, especially if they aren’t familiar with type-level programming in the first place.
    • Previously, we needed a Proxy type that was specific to each kind (e.g. SProxy, RProxy, RLProxy, etc.). These are no longer needed and will be deprecated. To reduce breakage, we will update the code so that usages of the kind-specific proxy types will still compile, giving developers time to migrate to the sole Proxy type.
  • Some libraries may/will be heavily refactored due to the PolyKinds feature, the Coercible feature, and the lessening of the “type synonyms cannot be used in type class instances” restriction.

Strategy

To prevent the painful v0.12.0 situation that occurred previously, it’s my understanding that we will not be making an official v0.14.0 release until most of the ecosystem can compile on the new release. What counts as “most” is still being determined.

Below are the stages I believe we will go through:

  1. purescript organization and purescript-contrib organization’s repos are updated to compile on the latest v0.14.0 release candidate. Library authors should not be concerned about updating their libraries, but they should be watching their depenedencies’ updates (e.g. PRs, issues) for major breaking changes. This is where we are currently at. See purescript/purescript#3942 for more context.
  2. The community is notified via Discourse that the above goal has been accomplished and that people can start updating their libraries. When any are updated, one should submit a PR to the prepare-0.14 branch of the package-sets repo.
  3. At some point “most” of the ecosystem will be updated to v0.14.0 and tooling will be updated. It is my understanding that we will make an official v0.14.0 release, merge the prepare-0.14 branch into the master branch, make a v0.14.0 package set, and drop packages from the set that are not updated by commenting them out. This will help others know which packages were originally in the set but which haven’t been updated yet. We will uncomment them with their updated version once it works again.

To get this done as fast as possible, we need multiple people working to update libraries in parallel, starting with the ones that have the smallest number of dependencies first.
Via a tool I wrote for this purpose, I created the libDeps.txt file that shows how many dependencies (direct and transitive) each package had in the then-master branch of the package-sets repo. I included transitive dependencies in the list as it helps you know whether any breaking changes we make in one place could affect packages indirectly.
At some point, I will update that small tool to remove dependencies that have been updated from a package’s list of dependencies in the outputted file, so that one can know which dependencies are currently blocking a given package from being updated.

Despite the general approach described above, there are still a few unanswered questions:

  • When does the Book get updated in this process?
  • How long will it take to update the unofficial purescript-cookbook repo? (Keep in mind that many of the recipes are built on Halogen, which depends on most of the ecosystem)
  • Will we block a v0.14.0 release until either or both of these are updated completely?

Again, I will post updated instructions in another thread once I’ve gotten them updated and figured out a few other issues.

18 Likes
How to Help Update the Ecosystem to v0.14.0
#2

Since it’s been a while, here’s the latest updates on things:

  • All core repos now compile on v0.14.0 when using their master branch.
  • There are still some changes we will be making in the core repos. I’m not sure how many of these are breaking and non-breaking, but I believe most if not all are non-breaking.
  • Due to Travis CI cancelling their free tier, core repos will be switching over to GitHub Actions. I’m not sure whether this will be done before/after v0.14.0 is released.
  • The scripts I created to update these repos worked really well. They currently only work when using bower. When we start updating the purescript-contrib repos, I will update those scripts to use spago, so that it’ll work for whichever tool a repo uses. I hope these tools will be used by the community when updating the rest of the ecosystem.
  • the prepare-0.14 package-set branch’s src/groups/purescript.dhall has been updated to use v0.14.0-compatible core repos.
  • See the libDeps.txt file for the entire ecosystem based on the then-current package set. The first value is the number of dependencies that need to be updated before that package can be updated. Second value is the repo. Third value is the not-yet-updated direct and transitive packages on which that repo depends.

Thank you again for all who helped out in this effort!

Lastly, I’m going to take a week off from OSS contributions to work on a personal project.

13 Likes
#3

We’ll now be working on updating the purescript-contrib libraries. We will not be making any new releases in those libraries from this point forward until v0.14.0 has been released (or some milestone like that).

7 Likes
#4

Latest update. We’ve updated almost all of the -contrib libraries. However, a few of them depend on purescript-web libraries. So, we’re in the process of updating those libraries and the purescript-node libraries. As a result, web and node libraries will not be making any new releases until v0.14.0 comes out.

Here’s a non-exhausitve summary of the work we have left to do:

  • switch -web libraries back to Bower (I had CI problems with Bower, so we switched those libraries to Spago. Then, we figured out what the issue was and now need to switch them back)
  • Update -web libraries to v0.14.0-rc3
  • Update -node libraries to v0.14.0-rc3 and switch to GitHub Actions
  • Update Pursuit so it can handle v0.14.0 docs.
  • Migrate packages within the prepare-0.14 package set to their correct file (e.g. some repos were transferred from Slamdata to purescript-contrib, but their files appear in slamdata.dhall, not purescript-contrib.dhall).
  • Do a final check to verify we’ve properly updated all libraries, reviewed all current PRs and other issues for any breaking changes we can merge, and have finished everything else we need
  • Make new releases for the core, contrib, web, and node libraries and revert dependencies back to non-master ones
  • Update the prepare-0.14 package set to refer to these new releases
4 Likes
#5

Another quick update for everyone: if you’d like to help us complete the library updates for 0.14, then please see this link for open issues and pull requests that we’d like to fix before the release:

https://github.com/issues?page=1&q=is%3Aopen+archived%3Afalse+user%3Apurescript-contrib+user%3Apurescript-web+user%3Apurescript-node+label%3A"fix+before+0.14"

#6

I wanted to post another update here. We’re almost done writing the release notes for core, contrib, node, and web libraries. You can thank @thomashoneyman and @milesfrain for all their work on this.

Here are the packages in the package set that currently do and do not compile on v0.14.0-rc5. If you do not see your package in either list, that’s because you added your package after the prepare-0.14 branch branched off the master branch.

Also, I want to revise what I said previously:

  1. The community is notified via Discourse that the above goal has been accomplished and that people can start updating their libraries. When any are updated, one should submit a PR to the prepare-0.14 branch of the package-sets repo.
  2. At some point “most” of the ecosystem will be updated to v0.14.0 and tooling will be updated. It is my understanding that we will make an official v0.14.0 release, merge the prepare-0.14 branch into the master branch, make a v0.14.0 package set, and drop packages from the set that are not updated by commenting them out. This will help others know which packages were originally in the set but which haven’t been updated yet. We will uncomment them with their updated version once it works again.

We will not be waiting until “most” of the ecosystem has been updated to v0.14.0 before we make the compiler release and before we make a v0.14.0 package set. AFAIK, we’ll be making the compile release as soon as we can and will then release a v0.14.0 package set soon thereafter. I made the above statements because I was naive about how long it would take to do this work and the complexities that can arise.

We’re getting close.

11 Likes
#7

Does anyone have a shell.nix for purs v0.14.0-rc5? Could you drop a link here? Or just paste it?

#8

As an extra note, the initial package set includes all libraries from the purescript, purescript-contrib, purescript-web, and purescript-node organizations. And, as Jordan said, there are also many other packages that compile and will be included. Some other major libraries like Halogen will be included soon afterwards.

2 Likes
#9

I used @cvlad version of easy-purescript-nix for 0.14.0-rc3 which you can find here: https://github.com/vladciobanu/easy-purescript-nix So it is probably just a matter of changing revision and hash.

2 Likes
#10

I have my own fork that I updated to -rc5

let
  pkgs = import (builtins.fetchTarball {
    url = "https://github.com/NixOS/nixpkgs/archive/20.09.tar.gz";
  }) {};

  pursPkgs = import (pkgs.fetchFromGitHub {
    owner = "ptrfrncsmrph";
    repo = "easy-purescript-nix";
    rev = "d53b10391c3ec289f8afdc664f743824115bbe70";
    sha256 = "0rmm141dhy8aivs066dq998g69h4y5amxsixbzd3nc5ngfzk3y26";
  }) { inherit pkgs; };

in pkgs.stdenv.mkDerivation {
  name = "sandbox";
  buildInputs = with pursPkgs; [
    pursPkgs.purs-0_14_0-rc5
    pursPkgs.spago
    # pursPkgs.zephyr
    pkgs.nodejs-14_x
  ];
}

Disclaimer: I really don’t know Nix, but forking Justin’s project and guessing my way through this configuration seemed like the easiest way to play around with the latest rc :sweat_smile: If there’s an easier way I would like to know.

2 Likes