A quandry with using another library's test code to test your library's code

Feedback
#1

I’m going to bed, but I thought I’d post this to gather some thoughts. The corresponding issue is here: https://github.com/thomashoneyman/purescript-halogen-hooks/issues/24

So, purescript-halogen-hooks defines code in the repo’s src folder. To test it, Thomas wrote code in the repo’s test folder. Here’s the problem. If I want to use the test code in the halogen-hooks library to test other hooks in purescript-halogen-hooks-extra, I run into a quandry.

  1. Since the halogen-hooks test code is in the test folder, Spago doesn’t pick it up by default when I depend upon the halogen-hooks library (i.e. the source globs that Spago adds by default is .spago/library/version/src/**/*.purs. I can get around this by adding a .spago/halogen-hooks/*/test/**/*.purs source glob to my test.dhall file, but that doesn’t seem ideal because the Test.Main module name is already taken by the halogen hooks library. Thus, spago test will rerun halogen-hooks’ tests, not the halogen-hooks-extra tests. I can get around that by using spago test --main Test.Hooks.Extra.Main, but it seems like there should be a better way.
  2. If the test code is moved into its own repo, the next problem is a circular dependency. The hooks library would define the main code, the test repo would define the main test code, and the hooks library would use the main test code to test itself. I guess this could actually work. If Spago supported the concept of targets (https://github.com/purescript/spago/issues/416#issuecomment-613634984), then this would likely be bearable, too. However, we wouldn’t want to do this because there is one function in the test code that “just so happens” to be very similar to the real source code. If the source code’s implementation of that function changes, so would the tests code’s implementation. That is easier to track if they are within the same repo.
  3. If the test code is moved into the src folder, which solves both of the above problems, the main code is now depending upon testing libraries (e.g. purescript-spec), which seems weird. Consider this situation. purescript-spec makes a breaking change. Since halogen-hooks would depend upon spec in this approach, halogen-hooks would account for the breaking changes. Then, in the next halogen-hooks release, it would need to make a breaking change, even though the main code that everyone uses in their projects never actually changed. Wuuuuuuut?
#2

At the moment, the ecosystem and tools have a built-in assumption that test code is not public. I think that’s probably the right approach: 99% of the time your tests are not part of your public API, and as you note, you should be able to change them in any way whatsoever without having to reflect that in your version bumps. In the case where you do want to make some package up some test code so that it can be reused by other projects, I think that should be a separate package. I think the issue that the test code should usually change together with the sources can be addressed by the registry proposal, which removes the need to have one package per repository.

2 Likes
#3

I’m glad to hear that. I was hoping to release the test code for Halogen Hooks as a separate package but to keep the source in a monorepo with the main Hooks package.

1 Like
#4

… which removes the need to have one package per repository.

Having multiple packages in one repository would be a nice feature to have. I imagine one could consolidate packages together if they form some ecosystem and the consolidation would speed up feedback loops.