* Make sure hoogle db is marked as available, when there is a copy on S3
* Create db even with `--do-no-upload` flag (useful for testing)
* Make sure home page uses latest lts with hoogle db available
* Fix atomic durable writing, since issue in RIO was fixed
* Log information about falling back onto the core-cabal-files repo
* Convert conduit pipe to Maybe fishes.
* Make sure module names, package name and version are added for fallback
cabal files
Problem
====
Too slow response from stackage.org/feed/.
So slow that my favorite RSS client (Slack's RSS integration) doesn't work due to timeout.
See https://github.com/fpco/stackage-server/issues/277 for details.
How?
====
Delete the content of the feed if stackage.org/feed is given `withDiff=False` as its query parameter.
Why?
====
I can't confirm it's the true cause of the slowdown (because the server is too hard to run on my machine!).
But anyway I think the html content of the feed is too much:
I just want to know the new LTS Haskell is released by the feed.
I'll click the link if I do want to see the detailed updates.
In addition, there's a reason generating the content causes the slowdown:
Other pages using `getSnapshots` (e.g. https://www.stackage.org/snapshots, https://www.stackage.org/)
are not as slow as https://www.stackage.org/feed/.
So the `getSnapshots` query dosen't seem to be the biggest cause.
And the left possible cause is `mkFeed`.
NOTE
====
I've tested nothing because it's too hard to run this app
without configuring my AWS account.
This is needed to we can easily migrate to the new db with the
recent pantry changes. If something goes wrong we can simply revert to
the old secret (and hence the old db).
Once the upgrade is done (and verified) we can update the old secret and
update the name in this commit (back to it's original name)
* Moved all extensions into modules that are using them, rather than globally,
since they mess up ghci session and introduce conflicts among
packages. Removed those from `.ghci` file as well
* Redesigned the schema to use Pantry and moved it into it's own module
* Switched all of the db and cron related stuff to RIO. Yesod part is
still on classy-prelude
* Got pantry to update stackage-server database from hackage
* Got import of stackage-snapshots implemented
* Moved some logic from all-cabal-tool
* Switched everything to `PackageNameP`, `VersionP`, etc. from a la Text.
* Fixed haddock, so it now does proper redirects and pipes the docs
correctly. Also implemented piping of json files from S3 bucket,
so index-doc.json is also served by stackage-server thus making
Ctrl+S feature work properly on haddock. Fix for commercialhaskell/stackage#4301
* Import of modules is done through cabal file parsing, which slows
down the initial import process drastically, but incremental update
is not a problem.
* Just as with modules, dependencies are also imported from cabal file.
* In general improved type safety by introducing a few data types:
eg. `ModuleNameP`, `HackageCabalInfo`, and many more.
* Implemented pulling of deprecation map from hackages and storing it in db
* Implementation of forward/backward dependencies within a snapshot only.
* Drastically improved performance of cron import job, by checking which
snapshots are not up to date
* Implemented pulling haddock list from S3 bucket. Modules that have
documentation are marked from the availability of actual haddock. This
process happens concurrently with snapshots loading.
* Rearranged modules a bit:
* github related functions went into it's own module
* cron related functions where moved from Database to Cron module
* Split up some functions to reduce individual complexity
* Parallelized package loading in cron job
* Implemented parsed cabal file caching.
* All queries where reqritten with esqueleto
* Syntactic improvements:
* Added stylish-haskell config
* Formatted all imports and extensions with stylish-haskell.
* Fixed inconsistent indentation across all modules
* Many improvements to the package page as well as few others.
* Reimplemented hoogledb creation.
* Dropped dependency on tar in favor of tar-conduit
* Added cli for stackage-server-cron
* Add cabal sha and size to the package page
* Fixed links in hoogle searches. Improved type safety for a hoogle handler
* satckage-server-cron is customizable with cli arguments
Final adjustments for the new stackage server release:
* Upgrade to lts-13.16.
* Stackage server related code has been merged to pantry. Made the code
compatible with the newer version pantry
* Added cli '--snapshots-repo'
* Add readme to package page
* Adjust snapshots expected format:
* Added `publish-time`
* Removed name `field`
* `compiler` field is now in the `resolver` field with fallback to
the root
