The kind of issues we’re encountering here are also why I’m reluctant to make a 1.0 release in the near future, because we still do have a lot of ecosystem stuff to solve - I don’t believe for a second most people (esp. newcomers) will see a distinction between the PureScript language/compiler 1.0 and the “PureScript product” as a whole. But that’s probably another discussion.
3 Likes
From @paf31 on Slack:
For the other major release as we used to have a checklist of all of the stuff that came with the compiler
Where would be a good place to keep a more comprehensive checklist? The compiler repo doesn’t seem appropriate as we want to cover docs, libs, and third-party tooling as well.
It is unrealistic to expect that we wait for everything to update before we release a compiler version. If we did that we’d probably never get a release. For example, many newcomers are using the PureScript book, but we obviously can’t wait for Phil to update everything in the book before a release. The book is currently problematic in how it recommends users get started and start installing libraries, which results in incompatibilities.
The number one issue is that there is not any easy or reliable way to deal with compiler compatibility as long as we are recommending bower in our introductory materials. We absolutely need to be able to do a release, and let the ecosystem gradually catch up, because we just don’t have the community resources do it all at once. Right now, the only viable option is psc-package. If all our introductory resources specified exact compiler versions (Just recommending npm install purescript is extremely problematic) along with an appropriate package set, we would not be having these problems. The only problem with psc-package is that it is currently lacking features/tooling that make it convenient to use as a general purpose dependency manager. I know Christoph is experimenting with ways to make it more extensible via Dhall. I know for sure we will adopt it at Awake if that happens.
2 Likes
Absolutely, but I think we can add a line like “things are in flux, expect things not to work” to the docs repo at least.
Yes, we should absolutely update docs and install instructions as a minimum. Anything that recommends a blanket install without pointing out the available versions will result in problems.
I was wondering whether it might be possible to have a purescript release badge that we could stick in library READMEs so at least you’d know in advance with Pursuit whether the library was compatible.
I migrated halogen-select to 0.12 yesterday, and encountered a few problems that I’d like to add into the mix here just for awareness’ sake:
- The compiler version change was not the only change in the ecosystem around that time.
purescript-domandpurescript-dom-classyseem to have been superceded by a collection ofpurescript-web*libraries likeweb-html,web-dom,web-events, andweb-uievents.purescript-domandpurescript-dom-classywere moved to thepurescript-deprecated/organization with readmes noting they are deprecated, but those changes never made it to Pursuit. So if I just update versions in Bower and install, I’ll get all sorts of version conflicts to resolve, and if I look at Pursuit, I have no indication as to why – as far as I could tell at first I just needed to wait for them to be updated. It wasn’t until I asked in Slack and @natefaubion and @garyb chimed in that I realized about the re-organization. To this day the readme on Pursuit is this:
- The
purescript-foreignlibrary had several updates and changed namespaces.AVarwas removed frompurescript-affand moved to its own package. I like all of these changes, but I wish there were some sort of notice summarizing the changes (to these and to thedom*libraries). For example, theAffrelease notes for 5.0.0 simply say “Update for version 0.12 of the PureScript compiler.” I had to track down the new location ofAVaron my own.
Once I was aware of all these changes that happened at the same time as the compiler bump I could quite easily update the project, but without having access to Slack and the ability to bug y’all I would have wasted a ton of time trying to figure out what had changed and where to look for new information.
I understand these are libraries, not the compiler, but if you’re building anything for the web with PureScript you’re inevitably going to reach for these fundamental libraries. It seems worth the effort to provide an official notice of core library changes that accompany a new compiler version.
I would love to help – I’m not shy about writing documentation or putting time into these updates – but without being a core contributor I have no idea what these plans even are. As a consumer I largely find myself digging around readmes and docs that may or may not have been updated trying to guess at what ecosystem shifts have just happened. This is a little disconcerting as a heavily-invested commercial user of PureScript.
I’m aware that I’ve just mentioned a number of problems without noting what I like about these changes. I don’t mind the reorganization and in most cases I quite like it. I’m glad the compiler has seen a new release and I’m grateful for the work everyone has put into making the release happen. I’m aware there will always be rough parts in a transition. But, like Gabe, I would like to discuss how to make these particular rough parts better, and I am more than willing to invest my own time where it would be helpful.
4 Likes
I would find it useful to be able to use Pursuit in a way that allows me to select a version of the compiler and then only see results compatible with that version. Obviously that involves other things and requires packages to declare that as well.
3 Likes
I think making it more obvious which version of things are running on try.purescript.org might be helpful to newcomers too.
I was also planning on making a PR to the purescript-maps README to make it obvious where to go instead but it looks like @garyb beat me to it 
I have some more thoughts on this!
When the release candidate of the compiler came out, several core libraries opened compiler/0.12 branches, and by the time the official release occurred the majority of core libraries were already ready to go (like purescript-web*, purescript-ordered-collections, purescript-prelude, etc.). However, unless I missed something major, there wasn’t much of a push to notify the rest of library authors about the release candidate, nor did the release candidate last very long.
To date, there are still pull requests trickling in to libraries like purescript-rationals, purescript-remotedata, and so on which are relied upon by at least meaningful bits of the ecosystem.
This seems unavoidable: we can’t force anyone to update, and if they aren’t tuned in on Slack or Discourse or Reddit then they’ll simply miss the new release, and won’t update until someone opens an issue or a pull request on their repository.
However, perhaps we could do something more. We could at least make a good faith effort to notify people who maintain repositories available on Pursuit if their code is going to break in the new release. Perhaps we could do something like this:
- Aggregate the packages on Pursuit, and their GitHub URLs
- For each, make a simple check for breaking changes in the new compiler (still using
idoveridentity, still importingAfffromControl.Monad.Affinstead ofEffect.Aff, etc.) - For packages that will break, delegate to a small team to make the final decision as to whether the library author should be notified (perhaps there are some circumstances where our simple check will be incorrect) – I volunteer for this team, for starters.
- For each package that will break, that team can open an issue on the repository stating “There is a new version of the compiler coming out that will break your code. Here is the release candidate. Here is a summary of changes in the compiler. In 3 weeks, the compiler will come out and the ecosystem will need to upgrade.”
With this process ^^ it still makes sense to update core libraries first and anything that core contributors have access to in order to minimize the number of blockers preventing library authors from updating (from their further dependencies). Perhaps we could follow a similar process for that:
- Aggregate from Pursuit all packages + their dependencies
- Rank libraries in order of the number of dependent packages on Pursuit (as we don’t have access to application code, and can’t judge dependents that way)
- Ensure some threshold of core libraries are updated before the next process starts (release candidate + notify authors with GitHub issues)
Finally, there could be a “documentation team” which I also volunteer for:
- For core libraries, we might need to update documentation when things have changed
- If there is an accompanying ecosystem change, like the
purescript-web*andpurescript-avarchanges, then there ought to be an accompanying official blog post that summarizes the changes, why, and where to find the new packages - I think the team did a great job moving libraries to
purescript-deprecatedand updating READMEs. It’s unfortunate these did not get pushed to Pursuit and that still ought to be fixed. It’s the right idea. - There should also be a ‘migration guide’ that details a) What processes are being done to upgrade everything; and b) some tips as a library author for migrating.
I think we can learn from this compiler update and put together processes to be better next time. And the same processes can work if there’s a major library update, in which the compiler remains the same but we realize we need a re-organization in things like the purescript-web* libraries.
As I noted in a few places: I believe it’s crucial that new users are able to come to PureScript within 30 days of a compiler release and have things working, and equally that commercial users are able to feel comfortable that new releases will resolve themselves reasonably quickly. I understand there are challenges to this, PureScript is a relatively new and niche language, and anyone using it should expect breaking changes and to have to figure things out from time to time.
But these challenges don’t seem too formidable, and that gives me confidence we can resolve them and provide a better experience for new and existing users during a transition.
9 Likes
Definitely a lot of good stuff to work on listed already (and it’s a little late for 0.12) but I just wanted to point out that releasing a new compiler version doesn’t need to mean it’s becomes the default. For example, most docs and beginner tutorials say to install Pulp and pulp init. If Pulp defaulted to psc-package, used purescript@latest from npm, and then found a package set for that compiler version the new user experience would be so much smoother. And we’re nearly there… just need to make sure the new npm packages published aren’t tagged as latest until a pulp init on that version just works (i.e. 0.11 and 0.12 can both exist on npm with latest still resolving to 0.11). Psc-package isn’t perfect yet, but at least it defers having to learn which libraries are compatible with which compiler until the user has an edge-case need or is writing their own libraries. It might be good to have compiler version flags as well for tutorials and books to use, like pulp init [optional compiler version].
2 Likes
Maybe the Pulp project should even be the gatekeeper for releases to npm specifically.
As a newbie to Purescript one of the biggest hurdles has been dealing with out of date documentation. I understand that keeping the book up to date would be challenging.
I think the per package documentation could be improved. In particular having up-to-date working examples that people could copy/paste and modify.
I haven’t run into this yet, but I wonder if there’s a way to specify which versions of the compiler a library works with. Maybe we could add a “compiler” field to the psc-package.json.
See @chexxor and I’s work about this in https://github.com/chexxor/purescript-documentation-discussion
I think others have already thought of the idea of a “compiler version” field of some sort. However, I think the issue is that it may require a breaking change in the compiler.
Reading this thread made me wonder if we should:
- associate a compiler version with a Spacchetti package-set (or with a Spago version)
- bundle the compiler in Spago (so you don’t depend on whatever purs version is installed in the system)
It would not be a big deal, and would only require some changes to editor integrations.
1 Like
I’ll respond to that in the issue to which you linked here
Coming back to this now – as far as I know, Spago now solves this with #101, and I’ve been using it in blog posts to ensure that if users download the right PureScript and Spago versions and run spago init they’re good to go with all dependencies in the article.
I’m leaning more and more towards PureScript documentation recommending Spago + a package set to minimize how frequently dependency issues occur for new users and how quickly recommendations go out of date (like “just run pulp init!”)
2 Likes
I agree; at some point in the not too distant future I would also like to have the getting started guide on purescript.org use spago. However I think it’s worth pointing out that "Run pulp init" is actually the correct recommendation for getting started for people who are going to use pulp, and it does work the vast majority of the time; the only time it doesn’t work is the short period directly after a breaking release where the ecosystem is still catching up. (Of course, the buzz surrounding a breaking release often gets lots of people to try purescript for the first time at this point, which is more or less the worst possible time to do so, so I’m definitely not advocating staying with bower.)