See a typo? Have a suggestion? Edit this page on Github
This blog post is about a potential policy decision affecting the maintenance of the Stack code base itself. It will affect contributors to the project, and those building Stack for other purposes (such as maintainers of Linux distro packages). It will only indirectly affect end users, as hopefully is made clear in the discussion below.
Github issue for official discussion
Until now, every version of Stack that has been released (or even
master, unless I'm mistaken) has exclusively used versions
of dependencies available on Hackage. It has not used the
archive or Git repo feature, or submodules to include alternative
versions of source code. This means that, for the most part, you get
the same Stack whether you get an official download, run
inside the source tree, use
stack build using a Stackage snapshot,
cabal install stack.
Now, as it happens, this isn't completely true either. The official
Stack binaries pin the dependencies to exact versions which have been
tested together, via the
stack.yaml file. This means that the latter
two approaches of getting Stack binaries may have different behavior,
due to the snapshot or the dependency solver choosing different
versions. Some distros have
already run into bugs
because of this.
To pull all of that back in: the official way to get Stack today will guarantee a specific set of dependencies which have gone through the full Stack integration test suite. Some alternative methods may not provide the same level of guarantees. But with a bit of effort, you can force Stack or cabal-install to build exactly the same thing the official binaries provide.
The new problem
One issue that pops up is: what do we do in a situation where an upstream package has a bug, and either cannot (within the timeframe desired) or will not release a new version with a fix? The concrete example that pops up is hackage-security pull request #203 (addressing issue #187), though the specific details aren't too important for the discussion here. The discussion here is about the general rule: what should Stack do in this case?
Others may be more creative than me, but I can see four different options to respond in a situation like this:
- Continue using the officially released upstream version of hackage-security, bugs and all
- Fork hackage-security on Hackage, and depend on the fork
- Inline the code from hackage-security into Stack itself, and drop the explicit dependency on hackage-security
- Include hackage-security via an
extra-deppointing at a Git commit. Our official builds will use the patched version of hackage-security, and anyone building from Hackage will end up with the unpatched version
Option (1) is the status quo: we cannot fix this bug until upstream fixes it. This is a disappointing outcome for users, as we know how to fix the bug, and can imminently do so, but users will continue to suffer regardless. However, it makes maintenance of Stack relatively easy, and has no impact on packagers.
Options (2) and (3) are relatively similar: you end up with a forked
version of the codebase feeding into Stack, but all of the code
necessary is still available from Hackage. Packagers, and people
building with a command like
cabal install stack, will still be able
to get the right version of the executable, assuming they pin their
dependencies the same way we do (as mentioned above).
Option (4) is a more radical departure. It means that
cabal install stack, without quite a bit of extra work, will not result in the
same executable. You can argue that, given the assumed lack of pinning
of dependency versions, this isn't too terribly different from the
status quo. And with the patch I've written for hackage-security now,
that's basically true. However, it's theoretically possible that, in
the future, we could have a patch that changes the API, and makes it
impossible to build Stack against the Hackage version of a package. So
let's break up option 4 into two subchoices:
- Option 4a: we can use an extra-dep, but we must ensure that the Stack codebase continues to build against the Hackage version of the package, even if it's missing a bug fix, performance enhancement, or whatever else we wrote
- Option 4b: free-for-all: use whatever extra-deps we want, and state that there is no support for building from Hackage alone.
I lean towards option 4a. I don't want to upload forks to Hackage (option (2)); it's a confusing situation for users, and may be seen as an aggressive move (which is certainly not the intent here). Option (3) could work, but makes it more painful than it should be to work on the Stack codebase. I'd rather not subject contributors (or myself!) to that.
Option 4b is IMO a step too far: we'd be uploading something to Hackage which we know for a fact could never be built there. At that point, there's not really any reason for uploading to Hackage. And option 1 (we cannot fix bugs) is just too limiting.
The biggest impact I can see is how others will end up packaging Stack. But frankly, this is already a situation that deserves an official discussion. There have certainly been plenty of cases in the past where users tripped on bugs that didn't exist in the official Stack releases, and the Stack team needed to spend inordinate time tracing this back to a bad build. So if nothing else, hopefully this post will spawn some discussion of correct packaging behavior.
As mentioned above, I've created a Github issue for an official discussion of this topic: issue #3866. Other discussions (Disqus below, mailing list, etc) are welcome, but may not receive the full attention of the Stack team.