Handling events in callbacks in Deku

Help
,
#1

I am trying to get my head around Deku and using it for an application that handles webcam video. At this point all I want to do is capture video, display on a canvas and render boxes (detected faces, etc) on the Canvas when a button is pressed.

I’ve wired up the video source as follows:

videoSource :: MediaStream -> AppMonad
videoSource mstream = Deku.do
  { imgHandler, detectionState } <- ask
  D.video
    [ D.Controls !:= "false"
    , D.Preload !:= "none"
    , D.Autoplay !:= "true"
    , D.Style !:= "display: none"
    , D.Self !:= \video -> for_ (V.fromElement video) \ve -> do
        playMStream mstream ve
        imgCancel <- subscribe animationFrame \_ -> handleAnnimationFrame ve imgHandler
          
        void $ subscribe detectionState \dt -> case dt of
          Abort -> do
            imgCancel
            shutdown mstream
          _ -> pure unit
    ]
    [
    ]

I’m able to cancel and close the video camera by subscribing to detectionState (which changes with a button at the moment). Now I want to do different things in the animationFrame handler, depending on the value of the detectionState, not just handleAnnimationFrame .
For reference:

type Env = {
  detectionState :: Event DetectionState
  , imgHandler :: EventIO VideoEvent
}

type AppMonad = NutWith Env

What is the best way to get access to the detection state in the other callback? Or is this approach wrong and should I be structuring this differently?

Thanks

Sumit

#2

Potential way forward with ‘Use ref’ in more hooks in the Deku docs.
Env becomes

type Env = {
  detectionState :: Event DetectionState
  , detectionStateRef :: Effect DetectionState
  , imgHandler :: EventIO VideoEvent
}

On app creation is do the following:

runInBody $ Deku.do
    setDetectionState /\ detectionState <- useState Unchallenged
    detectionStateRef <- useRef Unchallenged detectionState
    app mstream { mediaDevices, imgHandler, imgDataHandler, setDetectionState, detectionState, detectionStateRef }

Now I can ask, bind and use in functions on as required.

`videoSource mstream = Deku.do
  { imgHandler, detectionState, detectionStateRef } <- ask
  D.video
    [ D.Controls !:= "false"
     .
     .
    , D.Self !:= \video -> for_ (V.fromElement video) \ve -> do
        .
        .
        imgCancel <- subscribe animationFrame \_ -> do
          ds <- detectionStateRef
          handleAnnimationFrame ve imgHandler ds
        .
        .

Have to use in anger but seems to solve it.

1 Like
#3

Sorry for the delayed reply, this looks great, well done!