RFC: documenting PureScript's governance/structures/policies/processes

Why am I posting this

The context for this post is that some time ago I voiced a proposal to move Spago under the PureScript org on GitHub (the various reasons for why this is a decent idea are in the linked post).
People seemed to like it, so this finally happened some days ago! :tada: However you might have noticed that I just moved the repo back to the Spacchetti org.
The reason for that is that I found myself unable to perform administrative tasks in the new location, and the reason for that is that Iā€™d need to be member of the PureScript org to have access to the aforementioned things.
However, it was decided that I didnā€™t have enough contributions to the org repos to deserve a spot in there (I could quote the exact words, but then I might be misquoting and/or leaving context out, etc etc, so note that what youā€™re reading here is my perception of the story, how I experienced it, how I felt, and so on)

This said, the point of this post is to bring awareness to the larger community on how the language governance is handled, and hopefully spark some change (which some of you might read as ā€œmaking a fuss about things publiclyā€, but you know, thatā€™s kind of the point of democracy as well, so I guess itā€™s a good practice to have? You know, ā€œcall your representativesā€, etc)

Whatā€™s my problem here

In the span of a couple of days both me (in Spago) and @justinw (in psc-package, see this thread) lacked access to CI settings for performing administrative tasks that are normal as maintainers of these pieces of software, and I think this is terrible: I feel itā€™s frustrating to be a maintainer of something but still not have access to all the things related to the lifecycle of that something.

(Note: of course I could continue just fine to develop stuff under my profile or the Spacchetti org - and thatā€™s what Iā€™ll do anyways - but that wouldnā€™t solve the problem that we were trying to solve in the first place)
(Note: I tagged Justin but heā€™s not involved in all these words, I just noticed the pattern happening)

Whatā€™s my wish for today: empowering people that want to do things

So I just wish that weā€™d liberally give out membership of the PureScript org (on GitHub). In particular, the policy that makes most sense to me is ā€œif someoneā€™s a maintainer of a repo in the org, they should be a member of the orgā€.
Note: member of the org is not the same as admin of the org.

But isnā€™t this dangerous?

No: org members cannot interact in any special way with repos they donā€™t have access to in the org. They can only create repos new repos under the org, which admins can quickly revert if anything suspicious happens.

My main source of inspiration for how this empowerment can happen beautifully in the context of Open Source projects is dhall-langā€™s ā€œcontributingā€ document (shoutout to Gabriel for handling all of this so gracefully). In there - very early in the history of the language - were defined things like:

  • how to propose changes
  • how they get approved and who can approve them
  • timeouts for decisions
  • how to get the commit bit to things

Iā€™ve been involved with Dhall for some time now and these policies have worked out wonderfully and very effectively.
I also adapted that document for Spago, and as a matter of fact I give out the commit permissions very liberally (one needs very few non-trivial commits usually), which seems to work out very nicely as well :blush:


Whatā€™s my wish for this topic in general: documenting processes for PureScript

Currently it is not possible to find publicly any info about the following things in PureScript land:

  • if thereā€™s a ā€œcore teamā€
  • whoā€™s in the ā€œcore teamā€ exactly
  • what are the rights and obligations of the members of the ā€œcore teamā€
  • how this ā€œcore teamā€ overlaps with the purescript org on GitHub
  • what are the rights and obligations of the org members
  • whatā€™s the process to get in and out of both the ā€œcore teamā€ and the org
  • whatā€™s the contribution process to get the commit bit to the repos owned by the org
  • etc

What I would like to see happening in the longer term is that we create a place to write all of this down (maybe a purescript/governance repo? It doesnā€™t really matter how, it just needs to be a public container)

Note: something that I am not expressing here is any kind of moral judgement on what this ā€œgovernance modelā€ is. For what I care we can have open membership, or a BDFL, etc. Whatever thing is fine by me, as long as procedures are documented in the public: this is so that expectations are clear and openly stated, and this avoids mismatches and implicit behaviours from happening.

Why do I wish for this?

Subtitle: why is transparency important?

Now, the last thing I want to do is to sit here and give a lecture about why transparency in organizational processes is important, but I guess the need is present after all.
Here are some reasons why a public organization should make their processes as transparent as possible:

  • accessibility: if processes are documented publicly, everyone is aware of them, and can engage freely into these processes if they want to, and thereā€™s a ā€œcorrectā€ way to do that
  • ā€¦which enables diversity: if the rules of engagement are clear, this levels the field for everyone to contribute, since thereā€™s a standard measure that everyone is being hold to, and implicit power structures are less effective
  • bus factor is also a very important one: publicly-documented procedures empower enthusiastic contributors to contribute to their maximum potential. If procedures are unclear they might get discouraged and shy away. If knowlegeable contributors get tired/busy/sick/etc and policies are not written down this might lead to lost knowledge, etc.

All of these factors are major contributors to project health, also known as ā€œthe probability that this language is still relevant in 5 yearsā€.
I really want this probability to be as high as possible, and I really like this community, and Iā€™d like all of us to succeed, so letā€™s try our best at this :slightly_smiling_face:

Peace,
Fabrizio

11 Likes

Thanks for creating this thread. In general, Iā€™m in favor of overall making member access more open. Iā€™d like to address the issue of the scope of your inclusion, which may be in part due to miscommunication or misunderstanding.

This is just based on my understanding of the conversation, and I havenā€™t gone back to verify anything, so Iā€™m happy to be corrected. When it was originally brought up, my understanding was there was a request to be an org owner. I expressed hesitation on the grounds that (and this is just my opinion):

  • The PureScript org is the language as we know (being implementation defined), which is the compiler and core libraries.
  • Org owners have the responsibility to maintain the language.
  • Org ownership should be predicated on significant and long-term contributions to both.

I was not under the impression that was what you wanted to do, and I didnā€™t think it would be necessary for maintaining the Spago tooling. Iā€™ll admit, I donā€™t know the details of what tooling you use, what access you need, and I donā€™t really know what all the details of what membership vs ownership entails as far as GitHub permissions for those things. I am supportive of giving you and Justin everything you need to make your contributions successful. If you need membership to be successful, then you should have it. I apologize for that not being the case, and for it being frustrating.

As far as governance goes, itā€™s ad hoc, and thatā€™s really all there is to it. It can be very frustrating, but weā€™ve generally taken the stance that we donā€™t want any given maintainer to feel pressured to do things that their life and schedules donā€™t permit. Unfortunately, we are all pretty strapped for OSS time right now, and so most of that goes to existing issues and features, and not into structure, governance, and outreach. I personally hope I can be supportive of anyone willing to take on these things and work it out if they have a vision to do so. I initially started work on a document outlining the org but I have not had time to get back to it, and I didnā€™t receive much feedback at the time from the other org owners (it happens). But I agree with you that it would be good to take this further and document it in a repo.

7 Likes

Thanks for writing this up @f-f - I agree that this conversation is worth having publicly, and and project health is also something that I have been thinking more about recently.

That gist @natefaubion shared looks good to me so far, although I guess @f-f youā€™d be happier if it were a little more detailed?

The following sets out my personal understanding of the kind of governance procedures I think you are interested in. I donā€™t want to speak for the rest of the core team here, so what follows isnā€™t necessarily the official stance. Also, Iā€™m happy to receive criticism on this.

On the topic of commit bits, I do think our current policy of only very rarely handing them out is preferable, at least for the compiler and core libraries. I have two main reasons for this: firstly, I donā€™t want the compiler to move too quickly. Decisions will often have unintended knock-on effects, and can sometimes be very difficult to walk back from once made, and handing out lots of commit bits can lead to well-meaning people taking the initiative on issues and moving decisions along faster than they really ought to be moved. In my experience, having been involved with the compiler for a while is usually necessary for developing the context to be able to foresee potential problems with suggested features.

On the topic of how decisions are made, the way I would describe it is that itā€™s not a democracy - community input is a large factor and of course we do want to make sure the community has a voice in the development of the language, but the core team has the final word. There isnā€™t really a BDFL situation either, i.e. thereā€™s no (explicit) hierarchy among the core team; when there has been disagreement amongst the core team, we either find a compromise solution or defer making a decision on the issue. (If we were going to have a BDFL it probably would have been Phil, but he has stepped down and isnā€™t really involved with the development of the compiler or core libraries at all these days).

For the specific problem that you draw attention to regarding CI settings, do you know if being an org member (not admin) on GitHub is sufficient to be able to view and/or modify CI settings for repos in that org? I think different CI services might have different policies around this. In particular I think AppVeyor doesnā€™t handle this use case very well, I donā€™t think it really supports ā€œorg-scopedā€ CI projects at all.

I literally just asked what was the policy for (quoting myself here) ā€œpulling people in the orgā€, so the one you mention was definitely not my request.
I apologise for not having that conversation in the open, as I feel that would have avoided the miscommunication here. Iā€™ll stick to public-posting only from now on.
(small note: if anyone is curious I can post the whole email thread here, as I assume all my emails to be public communication anyways)

Thank you! Iā€™d be happy to move back the Spago repo if pulled in as a member of the org

That was exactly my impression (and itā€™s a very common situation in OSS to be strapped on time) and in fact my intention behind asking for clearer policies in general is to help involving more people in the process - so that some tasks can be lifted off the shoulders of someone that might not have enough time to dedicate to them - thus alleviating this whole problem. You could frame it as an investment for the language in the long term that has greater leverage than working on features.
Point in case the CI settings stuff: instead of having to sync with Harry to access them weā€™d save everyoneā€™s time if we could just access them, and so on.

This already looks like a good start! :clap: I do have some a vision on this (as outlined in my first post, which is just a summary of how I see it, happy to expand further), so if you push this document to a purescript/governance repo I can start opening issues/PRs there so we can discuss every point further.

I think itā€™s important to differentiate the groups of people that can ā€œapprove changesā€ with the people that have the ā€œcommit bitā€

The ā€œcompiler movementā€ should be limited by the first group that approves/rejects things - which would coincide with the core team - so nothing would really change if weā€™d change the ā€œcommit bitā€ policy.

Having an additional ā€œcommit bitā€ group would mean that these people could do things like:

  • merge approved PRs
  • manage the issue tracker (label, close, tidy up issues)
  • cut releases if thereā€™s the green light to do so
  • etc etc

As an example, in Spago a bunch of people have the commit bit, but they usually only merge once I approve things. And even in the case a change gets merged without approval, it is trivial to roll back.
Again, I invite you to read through Dhallā€™s contribution document to see what I mean.

I donā€™t have much to say on this, other than Iā€™d like to move these discussions to a place where all of this can be documented and called an ā€œofficial policyā€ (note: this includes the fact that thereā€™s a clear process to change this official policy, etc)

I think just being a member is enough. E.g. in Spago @klntsky is member of the org and he can manage CI settings of projects just fine. And in the case of services where itā€™s not enough I can write to their support and enact a manual change (the important point is that I can do so as a representative of the org, which means probably having a chance to get the fix through)

Sure, but I donā€™t think any of these things are a bottleneck. For example, merging a PR once it has been approved is also trivial - the hard bit is getting it to a state where itā€™s mergeable. Iā€™ve read Dhallā€™s contribution document by the way - as it happens I remember it being written, as I followed the evolution of Dhall from back when Gabriel first started blogging about it - and my reaction to it then, which is the same as it is now, was that Iā€™m impressed that heā€™s committed himself to all of those obligations and procedures, but thereā€™s absolutely no way I can ever see myself running an OSS project in a remotely similar way. In particular I think itā€™s more effective, and less stressful, to wait until we are confident that any given PR is good to merge before merging it.

(note: this includes the fact that thereā€™s a clear process to change this official policy, etc)

While Iā€™m happy to move these things to a GitHub repo, I am not keen on having all of them being permanently up for debate with anyone who wants to debate them. Of course Iā€™m prepared to explain and defend governance decisions to an extent but I donā€™t see the value of having an official process to change them - I donā€™t think we should be changing them particularly often, and anyway, any change to it is up to the core team, which hasnā€™t really suffered from not having set out explicit frameworks for making these sorts of decisions in the past.

I do agree with @natefaubion in that you having org membership on GitHub would be a good thing to do if it enables you to do the things you need to do to maintain things like package-sets and Spago.

@f-f I agree completely with what youā€™ve said.

A few months ago, I tried to start documenting things with the thought that we could make that documentation public. I got frustrated again pretty quick, and gave up the ghost. Sorry for not following through. Iā€™ll be real honest; Iā€™m probably not going to pick that back up.

Hereā€™s where I stand:

  • I donā€™t think the current model/process is good.

    Decisions are made by one person, basically, assuming they have the support of other people. Who else needs to support it, who knows? Conflicts are resolved by someone backing down, or someone giving up. This doesnā€™t foster growth in any way. People are discouraged from trying something different because it involves so much effort.

    What can someone on the core team do? Who knows what the limits are of one person? Who is going to stop someone that does something others donā€™t agree with? How would we even do that? I have no clue what Iā€™m allowed to do, or what Iā€™m not allowed to do.

  • The lack of open membership is a bigger problem than anybody is willing to talk about.

    Thereā€™s no rhyme or reason to membership. Why am I part of the core team? I make like one package every three months, which I rarely contribute to. I donā€™t participate in conversations with the core teamā€“primarily because theyā€™re all in private. I donā€™t participate in conversations in the general. I donā€™t respond to PRs in a timely manner. Yet, here I sit on the core team.

    Why isnā€™t Jordan part of the core team? Heā€™s documenting so much, and in such detail. Why isnā€™t Alex part of the core team? Heā€™s done a ton in community events. Why arenā€™t you part of the core team? Youā€™re fixing so many rough edges around tooling. How does anybody else even become part of the core team? None of this stuff is clear, and itā€™s not clear why not.

    Keeping the core team as minimal as it has been hasnā€™t been great for the ecosystem as a whole. People get burned out and leave. And thereā€™s always been this talk about how the core team is understaffed, but we donā€™t bring in more people, so it stays understaffed. I donā€™t know what other outcome could be expected. It takes nothing to staff more contributors to the core team except relinquishing control that we were given by the luck of the draw.

    As you mentioned, Dhall took a different approach. Dhall hasnā€™t collapsed under its own weight, and itā€™s probably not going to any time soon. Itā€™s a thriving community because thereā€™s no gate keeping (well, much less). You can also look at things other than languages, like k8s. k8s is a huge project with tons of people contributing to it daily, yet itā€™s also not collapsed under this model.

The long and short of all this is that I support what youā€™re saying, so I wanted to say that explicitly. Iā€™m probably not going to fight for change because I donā€™t have the energy to try and change things anymore.

4 Likes

Ok, yes, I suppose I have to concede that you are right that the understaffing situation is not going to improve with the current model. I think there probably are some people who are deserving of the commit bit to the compiler, as well as some autonomy in directing its development.

If it helps explain my defensiveness, I think (having been burned in the past by allowing myself to be exploited by poorly behaved users) my default response to these kinds of suggestions has been to think ā€œok, but how are the worst people going to exploit this.ā€ But I am starting to see that this approach is probably not healthy for the community (or, indeed, me).

Decisions are made by one person, basically, assuming they have the support of other people

Do you mean ā€œdecisions are made by one person who, in the process of making the decision, makes the assumption that they have the support of other peopleā€, or ā€œdecisions are made by one person provided that others support themā€?

Thereā€™s no rhyme or reason to core membership.

Alex, Jordan, and @f-f are all doing really fantastic work, but as @natefaubion said, core team membership is predicated on compiler contributions; my impression (happy to be corrected here) is that their work hasnā€™t really involved much contribution to the compiler, and therefore not being on the core team has not really gotten in the way of what it is they have been working on, until just now, with the situation @f-f described.

In your case @joneshf, I donā€™t know the circumstances of you being added, as (if I remember correctly) it happened before I became involved with PureScript. The reason you are still on it is because currently the only way someone leaves the core team is by them deciding to do so (as Phil did a while ago).

I am not keen on ā€œresponding to PRs in a timely mannerā€ being a prerequisite for core team membership, by the way; that would probably disqualify all of us based on past performance, and it would definitely stress me out to the extent that I would probably end up stepping down voluntarily anyway.

1 Like

Under a less-than-careful (less-than-charitable?) reading of this, I worry that there is political implication here (strong-arming, stonewalling, dictatorial, etc.). I donā€™t think that was your intent. I would just like to clarify that it has not been the case in my personal experience as a maintainer so far. Actions are taken on by those with the motivation to do so, and I am not aware of a situation where something was done despite reservations just because someone ā€œgave upā€. I understand the frustration here, but I believe everyone has been genuinely open to the concerns and voices of other maintainers.

@joneshf Iā€™d like to express my gratitude for letting all of this out publicly, I really see you.

The first post of this thread was written down and sitting in my browser for three days before I could send it.
I was scared that Iā€™d be the troublemaker, an unnecessary source of drama, and I was very close to give up on this as Iā€™ve already seen some people doing.

In the end I allowed myself to post it, and seeing that it allowed you to let your feelings out - which I imagine would otherwise bitterly sit inside - makes me feel like I did something useful for the community at large in sparking this discussion.

I want all of us to talk about this hard stuff, to reflect as a community, to be transparent about things, because this is how collaboration is best fostered: by communicating as much as possible.

You do not need to put all the energy to change things all by yourself (as I do not need to do that either), we can move things forward all together :slightly_smiling_face:

3 Likes

If a change to the process doesnā€™t change anything other than making people feel more empowered, thatā€™s a very good outcome and should be enacted.

Iā€™ll note again that Iā€™m not arguing for a specific process here - the process can look like something from ā€œwe approve changes that have X thumbs up on githubā€ to ā€œchanges need a majority vote of the core teamā€ to ā€œany changes will be decided by our BDFLā€. Anything is fine really, as long as itā€™s written down somewhere.

If changes are ā€œup to the core teamā€, thatā€™s totally fine! Letā€™s write that down.

The key point Iā€™m trying to convey here is: the moment you go from ā€œad-hocā€ processes (which are fluid by definition) to ā€œwell-defined, documented processesā€ there will be the need to change these processes sometimes.
If we accept the above sentence, then we cannot avoid having a ā€œchange management processā€ too.

As Hardy clearly noted above, at least some part of the core team has suffered from the current approach.

I see you and I just want to let you know that youā€™re not alone in this, as weā€™re supposed to help each other move forward as a community :slightly_smiling_face:

Not putting much trust in drive-by-contributors is fairly reasonable, but there should also be a strive to empower and trust regular ones so that they can eventually step up and help out with all of these worries!

It looks to me that the whole discussion here is hinting at the need for a separation between:

  • a ā€œcompiler teamā€: people that commit to the compiler, guide language evolution, and for which membership would be predicated by compiler contributions
  • and a ā€œcore teamā€: people involved with core libraries, packaging infrastructure, package managers, websites, documentation, etc, and for which membership would be predicated on merits for the things listed (i.e. ā€œeverything else communityā€)

I have the feeling that adopting such an arrangement could have a bunch of nice side effects:

  • separation of concerns: the two teams could have different processes and structure
  • the compiler would move at the speed that the ā€œcompiler teamā€ would consider appropriate, and the ā€œcore teamā€ would not be involved in this
  • there could be different processes to staff the two teams, making it easier to distribute responsibilities, etc.
  • ā€¦and last but not least: getting the ecosystem ready for a 1.0 version
3 Likes

Iā€™ve been following this thread and having been thinking about what I could say in this situation. Iā€™m still thinking about it, so for now, I thought Iā€™d just respond to @joneshfā€™s comment

Thereā€™s a few reasons for that:

  1. No oneā€™s asked me to be a part of the core team
  2. I havenā€™t asked anyone whether I could be a part of the core team
  3. As @hdgarrood pointed out above, I donā€™t need to be a part of the ā€œcore teamā€ (or maybe ā€œcompiler teamā€ ā€¦?) to still document things. My only contribution to the language was adding the docs for the Prim modules (or something like that).

Related to joneshfā€™s comment, I also donā€™t feel like Iā€™m qualified to approve changes on PRs, much less merge things. Thereā€™s a lot about compilers, languages, and general type theory that I likely do not know. In short, I understand Harryā€™s concern about not just letting anyone have write access to the language repo.

Thereā€™s a lot to discuss in this thread, so apologies for skipping over most of it with this post, just wanted to chime in on two things quickly:

It looks to me that the whole discussion here is hinting at the need for a separation between: ā€¦

I think thereā€™s something to be said for having more possible roles under purescript, but splitting the core library and compiler teams is something Iā€™d have reservations about. Despite there being no implicit prelude or anything, the core libraries are quite closely linked to the language in my mind at least. We went through several releases where we did a poor job of making a coordinated release of libraries along with the compiler, and it was pretty disastrous.

ā€¦and last but not least: getting the ecosystem ready for a 1.0 version

Iā€™d like to ask everyone that we donā€™t discuss this any further in this thread. Thereā€™s plenty to talk about, but it could completely hijack the main discussion in here, so if anyone wants to talk more about it now, letā€™s start a new thread for it.

Sorry @joneshf, I didnā€™t realise how frustrated with it you were. I appreciated your efforts on starting discussions about many of these things, even though they all stalled.

I think one of the reasons these efforts went nowhere is that in the current core team none of us are leader types, weā€™re basically all just in here as we were early enthusiasts of the language, and governance/process stuff requires someone to kinda take charge briefly even if itā€™s just to codify the ad-hoc way weā€™ve been doing things. So when you posed questions about how we do/should do things you never got clear answers as weā€™re mostly just, like, whatever maaan.

Maybe Iā€™m speaking out of turn for the other core members here, so if any of you disagree, apologies. Iā€™m definitely this way at least. And not from indifference, more that I donā€™t feel qualified to provide authoritative answers or take a lead on any of this as I have little experience in being in this kind of position.

Iā€™m also fine with having more of the core team chat public. Thereā€™s almost nothing said in the private channel that needs to be kept private - itā€™s predominantly rubber ducking/discussing specific compiler issues or coordinating releases, just things that might get disrupted in the flow of the normal discussion channels. Maybe we could move more of the technical stuff back to GH issues, and for things where we donā€™t want interruption, use issues that are locked to members, as at least people can read them then.

3 Likes

Sorry for not making this clearer, but I imagined weā€™d have an overlap between the two sets, exactly to avoid having the concern youā€™re pointing out here

It seems like we are all basically in agreement, at least in terms of what the next steps should be:

  • Create a purescript/governance repo, where we can start opening issues and PRs to flesh out all of these things in more detail
  • Move the spago repo back to the purescript org, and add @f-f as an org member
  • Within the core team: start thinking about inviting more people to the core team (while we want most of these governance discussions to be in the open, I think the ā€˜should we invite this personā€™ discussions should probably stay private, at least for now)

Does that sound reasonable to everyone? Have I missed anything?

3 Likes

Iā€™ve created the governance repo with the initial draft I had written. https://github.com/purescript/governance

5 Likes

Any news on this? I wanted to cut a new release, but moving the repo back and forth somehow really messed up the CI setup - i.e. I donā€™t think Iā€™d be able to release without spending more time on fixing CI - so Iā€™d like to have the repo in its final location before trying to fix CI again

@f-f ah yes sorry, how about this: Iā€™ll add you as an org admin again, you move the repo across and get everything set up, and then change your status to a regular org member?

1 Like

Sounds great, letā€™s go with this :+1: