I would like to improve the REPL experience in two ways:
Currently it reevaluates all bindings in your session on every expression submitted. That is, it bundles up a new module with everything in it and runs a fresh node process. This makes it problematic if you want to support effectful bindings. Currently you cannot run effectful code and use the result in subsequent expressions. You can use unsafePerformEffect, but that will run your effects again and again. I would like to run a persistent node process for the entire repl session rather than a process to evaluate each expression.
There’s no way to run async code. Once we allow effectful code to run, it would be nice to support running Aff, where the repl will block until a result is received.
For the first case, I’ve written some JavaScript to handle a persistent REPL session, where you can submit bindings or an expression to evaluate. It correctly handles recursive binding groups, which is a little tricky. It would need a basic socket server wrapper, but I don’t see any issues with that. The real work is updating the purs repl code to manage this node process, and likely changing how it invokes the compiler, so that it can grab and submit the individual bindings to the evaluator instead of a module with everything in it. Additionally it would have to know how to extract the type of an evaluated expression and retain that in the type environment.
For the second case, I would like to change the signature of the Eval code to return an Aff-like signature instead of Effect Unit, and also return the evaluated type. Something like
eval :: forall a b.
Eval a b =>
-- The value to evaluate
(Unit -> a) ->
-- The callback to invoke if an exception occurs
(Error -> Effect Unit) ->
-- The callback to invoke with evaluated value `b`
(b -> Effect Unit) ->
-- Returns a canceller, which will cleanup whatever is running when invoked
Effect (Effect Unit)
I’ve prototyped something like this in my evaluator, and doesn’t seem to be an issue.
Would anyone like to collaborate on this? Do other PureScript maintainers or users have any opinions on these changes?
I’m definitely keen for this, although I don’t think I can set much time aside to collaborate on it unfortunately. I’d love to be able to support <- in the repl, for instance.
There’s no way to run async code. Once we allow effectful code to run, it would be nice to support running Aff , where the repl will block until a result is received.
Well, that explains the error I was unable to get past this evening. lol
It appears that both Aff code and <- work in the repl in a :paste block. But in this case, all three lines are printed after the delay. Maybe this is really just Effect code with the launchAff_. Is the goal of these enhancements to get this working without the :paste block, and have the print timing work as expected?
It appears that both Aff code and <- work in the repl in a :paste block. But in this case, all three lines are printed after the delay. Maybe this is really just Effect code with the launchAff_ . Is the goal of these enhancements to get this working without the :paste block, and have the print timing work as expected?
I mean <- as a top-level repl binding, so you can do:
> a <- readFile "file.txt"
And a would be in scope for future repl calls. The repl currently does not let you evaluate effectful code, and maintain a binding to its result.
The real work is updating the purs repl code to manage this node process, and likely changing how it invokes the compiler so that it can grab and submit the individual bindings to the evaluator instead of a module with everything in it
What part of of the code we would need to change? Using your solution the evalExpr function should be passed to the handlExpression as the evaluator?
Can you give me a bit more detail about how you’d expect your solution to fit with the current code?
Right now the node repl backend has noops for everything but eval, however it supports other stateful hooks. We should use the other hooks to manage a basic socket server which wraps the session evaluator. This server should support things like adding imports, evaluating declarations/binding groups, evaluating expressions, and resetting state.
The current workflow is too coarse to support this however. handleExpression for example builds a new module with the entire history baked into it as let bindings and writes that module out. The node backend will then run a new process invoking that whole module. This means every binding in your history is reevaluated on every submitted expression. handleExpression and handleDecls would need to return only the JS source for those expression that have been submitted, so they could be sent to the repl evaluator. A non-trivial part of the work will be teasing this out.
I had a look at the version that @kritzcreek was working on (thanks for mentioning it @paulyoung ), and it seems like he was trying to use node’s vm object. It seems like vm have all the functionalities that we would need, is that not a possible solution? (There is also a repl module)
Also, wouldn’t it be better to just rip the REPL into a separate project? What do we gain from having it within the compiler?
It was not clear to me what advantages you get by using vm as opposed to stock eval. You still have to be careful about scope and shadowing since you still want PureScript’s let semantics. That is you don’t want to mutate a context object when you add bindings. I did not look too heavily into all that the vm module offers. What I wanted to achieve was possible with normal eval, so I didn’t see a need to involve it. If there’s a clear advantage to vm, then it should probably be used instead.
You want your compiler and repl to be in sync. A separate project just introduces ways for this to drift, so the question is what advantage do you get by having it as a separate project?
I don’t think that there are any clear advantages except that it seems like it has everything that allows us to control the context/scope in which a code can be evaluated. I am working on implementing small POC.
You want your compiler and repl to be in sync. A separate project just introduces ways for this to drift, so the question is what advantage do you get by having it as a separate project?
I see it as an app/tool of its own, so I just thought that it would just be easier to manage/document if it was separate.
As of now, we cannot use do notation in the REPL — what part of the code do we need to change so that we can allow top-level binding?
Last question, it seems like quite a lot would not need to change so would this improvement end up requiring pretty much re-writing the core of the REPL module and the handleCommand of the Language.Purescript.Interactive module?
I’m interested in this as well, I had a crack a long time ago at a POC of a persistant repl env and have long thought it should be done properly.
I’m resurrecting the repl for the purerl backend at the moment, which is rather involving both making use of the compiler library and also duplicating most of the repl code in there - which is not something that should influence the design here as such, just to say that I’m in the headspace
So I finally had a closer look at this, here are some notes:
it looks like the main problem is that for every new expression evaluated at the repl, we generate a new module, and run it on a new node process. We want to move to a model where there is a long-running node process (but actually this should be more general, something like a general “repl server backend”, so that e.g. we could just “send PureScript expression” to the repl, and get “generated JS” back), and each new expression is evaluated separately, but in the same context. In these terms, the backend would be just a port that accepts a protocol (such as the browser repl right now) and the default repl just starts a default server as a node process.
To do this we’d need to define a richer protocol - the current one only defines setup, eval, refresh and shutdown, but we might want to differentiate e.g. between “non-effectful”, “effectful”, and “async effectful” evaluation/binding. However, I’m not entirely clear on what’s needed here, @natefaubion might have a clearer picture of how this should look like
I’m still not sure if introducing the a <- effectful syntax would be a challenge or very straightforward
Example implementations from our pair-programming session, for reference:
class Eval a b | a -> b where
eval
-- The value to evaluate
:: (Unit -> a)
-- The callback to invoke if an exception occurs
-> (Error -> Effect Unit)
-- The callback to invoke with evaluated value `b`
-> (b -> Effect Unit)
-- Returns a canceller, which will cleanup whatever is running when invoked
-> Effect (Effect Unit)
instance Eval (Effect a) a where
eval exp errCb evalCb = do
val :: Either Error a <- ?catchEither (exp unit)
case val of
Right a ->
evalCb a
Left err ->
errCb err
pure mempty
else instance Eval (Aff a) a where
eval exp errCb evalCb = do
fiber <- runAff
case _ of
Right a ->
evalCb a
Left err ->
evalErr err
(exp unit)
pure (launchAff_ (killFiber (error "Cancel PSCi") fiber))
else instance Show a => Eval a a where
eval exp errCb evalCb = do
evalCb (exp unit)
pure mempty
-- Effectful binding
>>> a <- readFile "passwords.txt"
-- Wrap in eval for compilation
(eval \_ -> readFile "passwords.txt")
-- Wire Protocol
data PSCIProtocol
= Reload
| EvalBindingGroup (Array ({ name :: String, expr :: String }))
| EvalExpr { name :: String, expr :: String }