How to FFI a non-ECMAScript js?

Help
#1

Turns out, FFI only works with ECMAScript js, which most js in the wild is not.

I’m trying to create binding to makeSortable function of sorttable small library, which is distributed as a single JS file. My naive implementation of just appending an export const makeSortable = … to the end of the file fails with various … cannot be used in an ECMAScript module errors.

My second approach was creating an ECMAScript FFI.js and importing from it the plain js. You can see it below in “steps-to-reproduce” and it results in not seeing the file. Presumably I could change path to point to output/, which would probably be confusing to other programmers and lsp-servers.

I feel though I’m doing something completely wrong. How do you implement that?

Steps to reproduce

  1. Create new project with spago init
  2. Download the lib: mkdir -p src/3rdparty && wget -P src/3rdparty https://www.kryogenix.org/code/browser/sorttable/sorttable.js
  3. Fix encoding of ./src/3rdparty/sorttable.js (it’s a known bug I reported today to the author: there are letters that ISO-8859 doesn’t handle)
  4. Create 3 files:

    • src/Main.purs:

      module Main where

import Prelude

import Effect (Effect)
import FFI (makeSortable)
import Undefined (undefined)

main :: Effect Unit
main = makeSortable undefined

    • src/FFI.purs:

      module FFI where

import Prelude
import Effect (Effect)
import Web.DOM.Node (Node)

foreign import makeSortable :: Node -> Effect Unit

    • src/FFI.js:

      import "./3rdparty/sorttable.js";

export const makeSortable = function(table) {
    return function () {
        sorttable.makeSortable(table);
    };
};
  5. Build the project as spago bundle-app

Expected

It succeeds

Actual

Error:

✘ [ERROR] Could not resolve "./3rdparty/sorttable.js"

    output/FFI/foreign.js:1:7:
      1 │ import "./3rdparty/sorttable.js";
        ╵        ~~~~~~~~~~~~~~~~~~~~~~~~~

1 error
[error] Bundle failed.
#2

The bundler will bundle files from the output dir, so your FFI modules refs should be relative to output’s location. I would propose to use some path mappings for such deps to get rid of relative paths.

#3

I see, so one solution is replacing import "./3rdparty/sorttable.js"; with import "../../src/3rdparty/sorttable.js";. Seems to work for spago bundle-app.

However, if I execute spago bundle-module --main Main --to /dev/null I for some reason get:

✘ [ERROR] Delete of a bare identifier cannot be used with the "esm" output format due to strict mode

    src/3rdparty/sorttable.js:74:13:
      74 │       delete sortbottomrows;
         ╵              ~~~~~~~~~~~~~~

✘ [ERROR] Delete of a bare identifier cannot be used with the "esm" output format due to strict mode

    src/3rdparty/sorttable.js:160:16:
      160 │           delete row_array;
          ╵                  ~~~~~~~~~

✘ [ERROR] Delete of a bare identifier cannot be used with the "esm" output format due to strict mode

    src/3rdparty/sorttable.js:255:11:
      255 │     delete newrows;
          ╵            ~~~~~~~
#4

I’m not familiar with front-end stuff. I tried doing it and found that it’s possible, even though it doesn’t look very elegant.
I didnot. test it, but it “spago bundle-app” succeed.

const rootDir = process.cwd();
const Sorttable = import(`${rootDir}/src/Sorttable.js`);

export function makeSortable(table) {
  Sorttable.makeSortable(table);
}
#5

It seems to complete successfully due to a bug in PureScript compiler…

You see, I just experimented, and this way you can actually write anything you like as a .js file, and it will always complete successfully. purs just doesn’t parse the file.

If you open the index.js file that bundle-app or bundle-module produced this way, you’ll find the .js code you were referring to is missing, so yeah, if you’d try actually running an app assembled this way it wouldn’t work.

#6

yes, the compiler should parse the foreign.js and copy recursively all js it reference.

#7

It parses JS just to to check if declared foreign imports are present (as exports), purs is not a JS compiler.

The bundler is responsible for this.

#8

Do you know how to reproduce it manually? I’m trying to do that to potentially report a bug, but having no success.

I am using bpftrace to see what processes and with which params are being launched by spago build and I see two of them:

$ purs graph .spago/aff/v8.0.0/src/**/*.purs .spago/aff-promise/v4.0.0/src/**/*.purs .spago/arrays/v7.3.0/src/**/*.purs .spago/bifunctors/v6.0.0/src/**/*.purs .spago/console/v6.1.0/src/**/*.purs .spago/const/v6.0.0/src/**/*.purs .spago/contravariant/v6.0.0/src/**/*.purs .spago/control/v6.0.0/src/**/*.purs .spago/datetime/v6.1.0/src/**/*.purs .spago/distributive/v6.0.0/src/**/*.purs .spago/effect/v4.0.0/src/**/*.purs .spago/either/v6.1.0/src/**/*.purs .spago/enums/v6.0.1/src/**/*.purs .spago/exceptions/v6.1.0/src/**/*.purs
$ /usr/bin/esbuild --platform=browser --format=esm --bundle --outfile=/dev/null output/Main/index.js

Now, the bundler is clearly the second one. But for what purpose purs is launched? purs --help doesn’t make it any clear what “graph” does, but knowing how compilers usually work I’d presume this creates some file that shows which files depend upon what, and then this file would be parsed by esbuild. And then, the bug may be in the file, in which case it is a purs’ problem.

The problem though, when I try to run the purs command manually (with double quotes put around the paths of course, but also tried without), I get 236 errors! So I don’t really understand how come the same command works fine from spago :man_shrugging:

#9

Don’t know what the error you are fighting, but I believe it is not a bug in the compiler or bundler.

purs graph returns modules with dependencies, and spago uses it to check modules’ usage, it is not used for bundling etc, esbuild does the job.

#10

The 236 errors are of style Module Prelude was not found.

Yeah, so basically as I said. This implies the bug is in purs not parsing the .js file as a dependency in Jintao’s code above, so esbuild later does not receive it.

#11

Purs embedded js backend is not supposed to parse/transform JS in the manner you are asking. Your imports in JS foreign modules should be ready for bundling.

#12

The compiler isn’t saying that files from imports can’t be found, instead it simply produces the wrong graph output.

So there is a bug, it’s just, whether it is in the inability to evaluate import correctly or not bailing out upon finding such import — it’s something for purs devs to decide.

#13

Do you mean import in JS file? Purs doesn’t and should not care.

You are getting Module Prelude was not found because your purs graph command is not complete, it should contain all the globs for all packages, your purs graph .spago/aff/v8.0.0/src/**/*.purs... posted here does not contain all globs probably due to some output limitations.

#14

Okay, so, wait, I’m confused… Given a command spago bundle-app --no-build, two commands get run: purs graph and esbuild. Later one I understand what does: the bundling. But why purs graph runs?

So, just to clarify how I see it and why I’m confused: spago is a build system and all it does is passing files to either purs or esbuild. I re-read your comments, and you’re saying purs graph has nothing to do with esbuild, it doesn’t create deps chain to be used by esbuild, esbuild instead handles everything on its own. Okay, but then for what purpose purs graph gets run in this command?

#15

I guess it checks if the main module is present, you can look in the code of old spago.

You see that esbuild just takes js entry path, and it builds its own js deps chain.

1 Like
#16

Okay, so, back to the original question… I just got a crazy workaround idea: how about bundling the library only with bundle-app but not with bundle-module? The way I see this, the “bundle-app” is the initial code that always loads disregarding what page a user opens. So the JS library may be bundled there; and then when a page/module gets loaded, it will use the JS code that’s already loaded from bundle-app!

Is it possible to implement with Spago? Or even more so, is it even a viable idea (because “bundle-module” uses ESM format, and I read this documentation bit as that potentially global variables may be invisible to a module, and Idk how JS/PS works on the lower-level here)?

#17

TBH, I don’t get your workflow and what you are trying to accomplish with bundle-app/bundle-module.

  1. Third party js modules mentioned in foreign modules (that use relative paths) should be relative to output.
  2. If you are bundling for browser and want dynamic imports loading, it is better to to use some specialized bundler like vite, webpack, etc.
#18

Yeah, that part I get, it was your first comment in the thread and I tested it in the next one :blush:

Why? What I’m currently doing works fine for me, barring the FFI question (unless you’re implying that migrating to such bundler would somehow resolve the FFI problem…?)

Oh, it’s simple. It’s described in more detail in this SO question, but in short: I don’t want to have a humongous SPA (because all good websites/apps serve exactly the page a user asked for and not everything), and additionally I want to implement server-side rendering at some point. IOW, I need to serve a user the page it’s asked for; and people on SO say it has to be done with spago bundle-module.

So that’s what I’m doing: I use bundle-module to generate each page, and I use bundle-app (might probably use bundle-module too :thinking:) to generate the “navigation panel”, which uses a <script> to load the actual page the user wanted to fetch.