Threads & Async in Haskell vs PureScript - Aff & AVar vs IO & MVar/TVar

aff
#1

joneshf [6:19 PM]
Can you bracket an AVar _ to have atomicity in the callback if that’s the only way the AVar _ is used?
Or does that still not work?
“the only way the AVar _ is used” I assume would be done with a newtype and hiding the constructor behind a module.

natefaubion [6:35 PM]
@joneshf please clarify
“atomicity in the callback”
Are we in Aff, Effect

joneshf [6:43 PM]
Let’s say you do something like:

module Atomic (Atom, new, with) where

import Effect.Aff as Effect.Aff
import Effect.Aff.AVar as Effect.Aff.AVar

newtype Atom a
  = Atom (Effect.Aff.AVar.AVar a)

new :: forall a. a -> Effect.Aff.Aff (Atom  a)
new value = ado
  ref <- Effect.Aff.AVar.new value
  in Atom ref

with :: forall a. Atom a -> (a -> Effect.Aff.Aff Unit) -> Effect.Aff.Aff Unit
with (Atom ref) = Effect.Aff.generalBracket acquire release
  where
  acquire :: Effect.Aff.Aff a
  acquire = Effect.Aff.AVar.take ref
  release :: Effect.Aff.BracketConditions a Unit
  release =
    { completed: \value _ -> Effect.Aff.AVar.put value ref
    , failed: \_ value -> Effect.Aff.AVar.put value ref
    , killed: \_ value -> Effect.Aff.AVar.put value ref
    }

Probably doesn’t type, but that’s the general idea.
If you do

Atomic.with ref callback

Can you atomically modify the value within callback?

natefaubion [6:46 PM]
should your callback be (a -> Aff a)?
right now you always put the original value back
Any sort of hard guarantees would require https://github.com/slamdata/purescript-aff/pull/169 unfortunately :confused:
which requires fixing purescript-bus
but “in general”, yeah it would work

joneshf [6:50 PM]
Yeah, probably should be a -> Aff a :sweat_smile:
Interesting.
I do wonder how we’ve gone this long without STM. What are people using instead?

natefaubion [6:51 PM]
STM? in Aff?

joneshf [6:51 PM]
In whatever.

natefaubion [6:52 PM]
I guess it all depends on what you want to do in your Aff
Since STM should be pure

joneshf [6:52 PM]
When you say “in Aff,” what do you mean?

natefaubion [6:52 PM]
Your callback

joneshf [6:52 PM]
Oh.
Nah.
I mean there’s no STM in PS, near as I can tell.

natefaubion [6:53 PM]
Sure, that’s because in JS there’s no such thing as sync contention

joneshf [6:53 PM]
Sorry, what is that?

natefaubion [6:53 PM]
multiple threads trying to use the same resource at the same time
Since there’s only one thread, there’s always a distinct ordering

joneshf [6:54 PM]
Oh.
How does that work?
Like, if you have one AVar _ and attempt to use it in two places, how do you know what the ordering is?

natefaubion [6:58 PM]
the ordering doesn’t really “matter”, just whether or not you can get an inconsistency. In a JS environment a Ref is enough.
it depends on how you are using the AVar

joneshf [6:58 PM]
I’m confused.

natefaubion [6:58 PM]
Are you using an AVar as a mutable reference, or as an async coordination primitive

joneshf [6:59 PM]
The latter.

natefaubion [6:59 PM]
That’s distinct from STM
which is all about consistency in the presence of multiple threads gunning for the same resource
So when you say you want an atomic AVar
the question is, do you want to atomically modify it, in which case it’s a mutable reference
If you take an avar, there’s nothing that prevents some other fiber from putting to it

joneshf [7:01 PM]
I guess I don’t understand the distinction between a mutable reference and an async coordination primitive.

natefaubion [7:02 PM]
So like, AVar works as a queue of producers vs consumers. All it cares about is whether it can provide a value to consumers or take a value from producers.
If it has a producer, and a consumer takes it, there’s nothing preventing more producers from producing values after that value is taken
there’s not atomicity in that regard
it doesn’t let you say “only I can hold this resource and no one else” (edited)
Unless you maintain that invariant somehow
by only allowing takes or something
Your newtype is fine for most general cases, as long as no one else can arbitrarily put to the AVar
What I’m saying is, AVar can be used to implement a lock, however AVar itself cannot enforce the invariants of a lock.

joneshf [7:07 PM]
That much makes sense.

natefaubion [7:07 PM]
When it comes to STM, STM only deals with transactions among other STM entities, not effects in general Aff.

joneshf [7:07 PM]
Right.

natefaubion [7:07 PM]
So in that sense, a callback a -> Aff a is too general

joneshf [7:07 PM]
But you wouldn’t need that.

natefaubion [7:07 PM]
since it allows non-Atomic effects

joneshf [7:08 PM]
Because you’d have tvars.
Right?

natefaubion [7:08 PM]
So when you made a comment about STM, my question is about what it is you are wanting to do with your Aff lock (edited)
are you only wanting to transact over other Atomic vars
or do you want to make an API request or something

joneshf [7:09 PM]
I’m curious how people get along with only AVar _ in PS. Like, in Haskell using MVar _ a bunch is really error prone, vs. using TVar _.

natefaubion [7:10 PM]
I think that’s because Ref serves just fine as a mutable reference. In a single threaded environment, it’s already atomic and does not experience contention in the way a TVar would in Haskell.
So when you just want concurrent mutable state where two fibers can’t write at the same time, Ref is fine, since the fact that it’s single threaded enforces it.

joneshf [7:11 PM]
Hmm.
How does that work with multiple Ref _s? It seems like the atomicity doesn’t compose since they run in Effect _.

jkachmar [7:29 PM]
I was always under the impression that IORef was safe in Haskell as long as you used it in a single-threaded context; does that sort of logic not apply in PureScript with Ref?

joneshf [6:54 AM]
I don’t really know about single-threaded vs. multi-threaded stuff. I just know that in practice, MVar _ leads to more bugs than TVar _ in Haskell. And that the few things I’ve tried to do with AVar _ so far have been similar to MVar _ (read, buggy).

natefaubion [1 day ago]
This is why I asked about mutable reference vs async coordination. MVar and TVar aren’t really comparable.

natefaubion [1 day ago]
Avar is not a good mutable reference

natefaubion [1 day ago]
Avar is good for “do this after this and that become available”, and not “I need consistent global state”

joneshf [6:56 AM]
Since I haven’t really tried to do a whole lot of async stuff in PS–but many other people have built a bunch of stuff, I have to assume I’m using the wrong stuff.

joneshf [7:03 AM]
I don’t particularly need STM, it’s just what I’m used to from Haskell. But, I don’t know how you make an async bounded queue, make a pool of resources, or solve the “money transfer” problem with AVar _/Ref _ without being a brain genius.

hdgarrood [7:56 AM]
Whether you’re working in Effect or Aff is quite important
If you’re working in Effect and using Ref, the kind of problem STM solves can’t happen

joneshf [7:57 AM]
Can you solve any of those problems in Effect _?

hdgarrood [7:57 AM]
if you’re working in Aff then you need to be a bit more careful because things could get interleaved in a more complicated way
i mean the problem where you have one thread reading some state and then writing it again a little later assuming it hasn’t changed in the meantime
I don’t think TVar makes solving the problems you refer to easy though
if you’re interleaving effectful actions with modifying TVars you still need to be careful

hdgarrood [8:05 AM]
like TVar can’t help you if you want to transactionally read a value, perform an API call, and then write that value again based on the response

joneshf [1 day ago]
I think STM _ would prevent that, right?

hdgarrood [1 day ago]
it wouldn’t because it doesn’t let you perform any effects inside a transaction

hdgarrood [1 day ago]
this is crucial because it’s the absence of effects which makes transactions safe to retry if the first attempt to commit fails

hdgarrood [1 day ago]
~I don’t think TVar makes a huge difference over MVar unless you’re performing complex pure computations on mutable state concurrently from multiple threads~ (edited)

hdgarrood [1 day ago]
sorry, scratch that

hdgarrood [1 day ago]
TVar is great in that it lets you be sure that any pure computation involving reading/writing mutable references is happening atomically

hdgarrood [1 day ago]
in Haskell, that is

hdgarrood [1 day ago]
but if our PureScript code is running in JS, Effect already gives us that guarantee (edited)

joneshf [1 day ago]
Sorry, I think we said the same thing. STM _ would prevent that whole thing from happening because it’s STM _, not IO _.

joneshf [1 day ago]
Prevent at compile-time.

joneshf [1 day ago]
Also, I don’t think that Effect _ gives us the same guarantees as STM _.

joneshf [1 day ago]
If you have

transfer :: Money -> Ref Account -> Ref Account -> Effect Unit
transfer amount old new = do
  withdraw amount old
  deposit amount new

and something happens to interrupt transfer between the withdraw and deposit lines, someone is going to be upset in real life. (edited)

joneshf [1 day ago]
That something could be the new account being “closed,” so the deposit would fail. The money would be gone from the old account.

natefaubion [1 day ago]
If it’s in effect how would it be interrupted?

natefaubion [1 day ago]
I guess it depends on what you’re guarding against. Killing a process? I don’t think anything in PS would help in a case like that.

hdgarrood [1 day ago]

Sorry, I think we said the same thing. STM _ would prevent that whole thing from happening because it’s STM _, not IO _.

ah right of course, yes

hdgarrood [1 day ago]
I guess if you’re writing stuff in Effect you might be tempted to make use of something which can throw

hdgarrood [1 day ago]
Then again I guess it’s just as important to avoid the temptation to wrap stuff in atomically prematurely when you’re using STM

hdgarrood [8:06 AM]
or at least not without a bit of extra effort on your part to set up a lock or something

joneshf [1 day ago]
Right, TVar _ is the building block upon which more useful abstractions are built. But since it’s in STM _, it seems to make things easier for people to write those abstractions correctly than MVar _ does.

natefaubion [1 day ago]
The big thing is that STM is pure except for other STM stuff, exactly like ST.

natefaubion [1 day ago]
It’s purpose is just data consistency in the presence of multithreading

natefaubion [1 day ago]

liftEffect do
  Ref.write a foo
  Ref.write b bar

Is equivalent to atomically with TVars in a JS runtime

natefaubion [1 day ago]
since it’s single threaded, everything is “atomic”

#2

Related: How would one get “rollback” functionality with Ref? check in STM is nice for this and combined with all the other combinators like orElse it provides a pretty nice “Do stuff and check whether or not it made sense or else just move on to the next case” kind of behavior in my experience.

#3

I don’t believe there is a need for retrying since there is no contention in a single-threaded runtime.

#4

Ugh, right. It’s hard to stop thinking in these patterns.