Using unpublished JS dependencies for FFI

Help
#1

If a JS dependency is available on NPM, I’d just add it to my package.json, but what’s the workflow to include a dependency that exists as one or more .js files?

For example:

// dependency.js
"use strict";
const thing = 42

// Foo.js
"use strict";

// Neither of these work:
//const thing = require("./dependency.js").thing;
import { thing } from './dependency.js';

exports.doThing = function (input) {
  return input + thing;
}

-- Foo.purs
module Foo where

foreign import doThing :: Int -> Int

With the above three files in my src directory, when building with spago I get a parsing error for the import statement. I assume this is because ES6 module syntax is not yet supported.

One workaround is to copy the contents of dependencies.js into Foo.js, but this is tedious if there are lots of dependencies files.

#2

We use webpack module aliases. https://webpack.js.org/configuration/resolve/#resolvealias. I suspect that something similar likely exists for other bundlers. This does not work if you want to ship this as a library though. In those cases I recommend putting FFI in a monolithic Internal module.

#3

I use absolute paths (relative to the root of the project)

1 Like
#4

If it’s in different directory - use https://docs.npmjs.com/cli/link or make a symlink manually rm -dRf ~/projects/yourproj/node_modules/dep && ln -s ~/projects/dep ~/projects/yourproj/node_modules

N.B. in your example you don’t export thing

#5

Would it be envisageable for the compiler to rewrite ./dependency.js to …/…/src/dependency.js?

That is to rewrite relative imports in foreign modules to point to the file that would have been imported from the foreign module orignal location instead of its location in the output directory?

1 Like
#6

I think that can be problematic because it means output artifacts are no longer portable. There are no assumptions in compiler output that prevent you from relocating, caching, etc these artifacts. If the compiler were to rewrite relative imports, this would no longer be the case. I also means that the output artifacts can’t exist independently on their own. They must be paired with the source tree.

1 Like
#7

Is the output directory really portable when foreign modules can import arbitrary files? In this case nothing prevents the foreign module to import ../../src/dependency.js for instance.

I can understand wanting to keep the compilation artifacts self-contained in the output directory for libraries and PureScript only applications but when embedding PureScript modules in a JavaScript codebase the PureScript output directory is necessarily not self-sufficient anyway.

#8

The difference is that the compiler is not doing anything to support or prevent this workflow. If we bake in support for path rewriting, libraries will start depending on it as well since the compiler has no distinction. We treat foreign modules as opaque blobs, except to validate that exported bindings line up with the PureScript module. I think this is a good thing and kind of the point. If you are embedding in a JavaScript codebase then you very likely already have a bundler setup which can handle things like import aliases.

1 Like
#9

The difference is that the compiler is not doing anything to support or prevent this workflow.

While not preventing anything, I’ll argue that the compiler is forcing you to reach for a loader sooner than you might have otherwise by not rewriting imports in foreign modules.

If we bake in support for path rewriting, libraries will start depending on it as well since the compiler has no distinction.

@hdgarrood suggested in “Restrictions on ES6 Syntax” to warn when parsing ES6+ syntax in foreign modules and turning that into an error in our future registry CI pipeline. I thought we could do the same for relative imports in foreign modules.

1 Like