* Make sure hoogle db is marked as available, when there is a copy on S3
* Create db even with `--do-no-upload` flag (useful for testing)
* Make sure home page uses latest lts with hoogle db available
* Fix atomic durable writing, since issue in RIO was fixed
* Log information about falling back onto the core-cabal-files repo
* Convert conduit pipe to Maybe fishes.
* Make sure module names, package name and version are added for fallback
cabal files
Problem
====
Too slow response from stackage.org/feed/.
So slow that my favorite RSS client (Slack's RSS integration) doesn't work due to timeout.
See https://github.com/fpco/stackage-server/issues/277 for details.
How?
====
Delete the content of the feed if stackage.org/feed is given `withDiff=False` as its query parameter.
Why?
====
I can't confirm it's the true cause of the slowdown (because the server is too hard to run on my machine!).
But anyway I think the html content of the feed is too much:
I just want to know the new LTS Haskell is released by the feed.
I'll click the link if I do want to see the detailed updates.
In addition, there's a reason generating the content causes the slowdown:
Other pages using `getSnapshots` (e.g. https://www.stackage.org/snapshots, https://www.stackage.org/)
are not as slow as https://www.stackage.org/feed/.
So the `getSnapshots` query dosen't seem to be the biggest cause.
And the left possible cause is `mkFeed`.
NOTE
====
I've tested nothing because it's too hard to run this app
without configuring my AWS account.
* Moved all extensions into modules that are using them, rather than globally,
since they mess up ghci session and introduce conflicts among
packages. Removed those from `.ghci` file as well
* Redesigned the schema to use Pantry and moved it into it's own module
* Switched all of the db and cron related stuff to RIO. Yesod part is
still on classy-prelude
* Got pantry to update stackage-server database from hackage
* Got import of stackage-snapshots implemented
* Moved some logic from all-cabal-tool
* Switched everything to `PackageNameP`, `VersionP`, etc. from a la Text.
* Fixed haddock, so it now does proper redirects and pipes the docs
correctly. Also implemented piping of json files from S3 bucket,
so index-doc.json is also served by stackage-server thus making
Ctrl+S feature work properly on haddock. Fix for commercialhaskell/stackage#4301
* Import of modules is done through cabal file parsing, which slows
down the initial import process drastically, but incremental update
is not a problem.
* Just as with modules, dependencies are also imported from cabal file.
* In general improved type safety by introducing a few data types:
eg. `ModuleNameP`, `HackageCabalInfo`, and many more.
* Implemented pulling of deprecation map from hackages and storing it in db
* Implementation of forward/backward dependencies within a snapshot only.
* Drastically improved performance of cron import job, by checking which
snapshots are not up to date
* Implemented pulling haddock list from S3 bucket. Modules that have
documentation are marked from the availability of actual haddock. This
process happens concurrently with snapshots loading.
* Rearranged modules a bit:
* github related functions went into it's own module
* cron related functions where moved from Database to Cron module
* Split up some functions to reduce individual complexity
* Parallelized package loading in cron job
* Implemented parsed cabal file caching.
* All queries where reqritten with esqueleto
* Syntactic improvements:
* Added stylish-haskell config
* Formatted all imports and extensions with stylish-haskell.
* Fixed inconsistent indentation across all modules
* Many improvements to the package page as well as few others.
* Reimplemented hoogledb creation.
* Dropped dependency on tar in favor of tar-conduit
* Added cli for stackage-server-cron
* Add cabal sha and size to the package page
* Fixed links in hoogle searches. Improved type safety for a hoogle handler
* satckage-server-cron is customizable with cli arguments
Final adjustments for the new stackage server release:
* Upgrade to lts-13.16.
* Stackage server related code has been merged to pantry. Made the code
compatible with the newer version pantry
* Added cli '--snapshots-repo'
* Add readme to package page
* Adjust snapshots expected format:
* Added `publish-time`
* Removed name `field`
* `compiler` field is now in the `resolver` field with fallback to
the root
