-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Go faster, Travis CI! #359
Draft
jashank
wants to merge
3
commits into
master
Choose a base branch
from
jashank/2020/02/27/zoom-zoom-ci
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,196 +1,166 @@ | ||
# This file is modified by Zilin Chen, based on the template generated by https://github.com/hvr/multi-ghc-travis | ||
|
||
language: python | ||
python: | ||
- "3.7" | ||
sudo: required | ||
dist: xenial | ||
|
||
group: edge | ||
|
||
git: | ||
submodules: false | ||
|
||
# branches: | ||
# except: | ||
# - branch-name | ||
|
||
# addons: | ||
# apt: | ||
# packages: | ||
# - "python3" | ||
# - "python3-pip" | ||
|
||
# Travis-CI configuration for Cogent. | ||
# | ||
# Resources to spelunk to understand how this all works: | ||
# * <https://docs.travis-ci.com/user/languages/haskell/> | ||
# * <https://docs.haskellstack.org/en/stable/travis_ci/> | ||
|
||
group: edge # <dev|edge|stable> | ||
os: linux # <linux|osx|windows|linux-ppc64le> | ||
dist: bionic # <bionic|xenial|trusty|precise> | ||
language: shell # no explicit toolchain, please | ||
|
||
# Cache dependency artefacts. | ||
cache: | ||
directories: | ||
- $HOME/.cabsnap | ||
- $HOME/.ghc | ||
- $HOME/cabal/lib | ||
- $HOME/.cabal/share | ||
- $HOME/.cabal/packages | ||
- $HOME/.cabal/bin | ||
# - $REPO/cogent/$SANDBOX | ||
- $HOME/z3 | ||
- $HOME/.cabal | ||
- $HOME/.stack | ||
- $HOME/objects | ||
- $REPO/isa-parser/.stack-work | ||
- $REPO/cogent/.stack-work | ||
- $REPO/minigent/.stack-work | ||
|
||
before_cache: | ||
- rm -fv $HOME/.cabal/packages/hackage.haskell.org/build-reports.log | ||
- rm -fv $HOME/.cabal/packages/hackage.haskell.org/00-index.tar | ||
git: | ||
submodules: false | ||
|
||
env: | ||
global: | ||
- SANDBOX=".cabal-sandbox" | ||
- REPO=$TRAVIS_BUILD_DIR | ||
- TRAVIS_BUILD_APT_RETRIES_PERCENTAGE=100 | ||
- OBJECT_DIR=$HOME/objects | ||
- AUTOCORRES_VERSION=1.6 | ||
|
||
addons: | ||
apt: | ||
packages: | ||
- z3 # for arrays | ||
- libgmp-dev # for GHC | ||
- python3-ruamel.yaml # for test scripts | ||
- python3-termcolor # for test scripts | ||
- libfile-which-perl # for man-gen | ||
- linux-headers-$(uname -r) # for ext2fs/bilbyfs | ||
|
||
matrix: | ||
include: | ||
- env: CABALVER=2.4 GHCVER=8.6.5 | ||
compiler: "ghc-8.6.5" | ||
addons: {apt: {packages: [cabal-install-2.4,ghc-8.6.5,alex-3.1.7,happy-1.19.5], sources: [hvr-ghc], config: {retries: true}}} | ||
- env: CABALVER=2.4 GHCVER=8.6.4 | ||
compiler: "ghc-8.6.4" | ||
addons: {apt: {packages: [cabal-install-2.4,ghc-8.6.4,alex-3.1.7,happy-1.19.5], sources: [hvr-ghc], config: {retries: true}}} | ||
- env: CABALVER=2.4 GHCVER=8.6.3 | ||
compiler: "ghc-8.6.3" | ||
addons: {apt: {packages: [cabal-install-2.4,ghc-8.6.3,alex-3.1.7,happy-1.19.5], sources: [hvr-ghc], config: {retries: true}}} | ||
- env: CABALVER=2.2 GHCVER=8.6.2 | ||
compiler: "ghc-8.6.2" | ||
addons: {apt: {packages: [cabal-install-2.2,ghc-8.6.2,alex-3.1.7,happy-1.19.5], sources: [hvr-ghc], config: {retries: true}}} | ||
- env: CABALVER=2.2 GHCVER=8.6.1 | ||
compiler: "ghc-8.6.1" | ||
addons: {apt: {packages: [cabal-install-2.2,ghc-8.6.1,alex-3.1.7,happy-1.19.5], sources: [hvr-ghc], config: {retries: true}}} | ||
- env: CABALVER=2.2 GHCVER=8.4.4 | ||
compiler: "ghc-8.4.4" | ||
addons: {apt: {packages: [cabal-install-2.2,ghc-8.4.4,alex-3.1.7,happy-1.19.5], sources: [hvr-ghc], config: {retries: true}}} | ||
- env: CABALVER=2.2 GHCVER=8.4.3 | ||
compiler: "ghc-8.4.3" | ||
addons: {apt: {packages: [cabal-install-2.2,ghc-8.4.3,alex-3.1.7,happy-1.19.5], sources: [hvr-ghc], config: {retries: true}}} | ||
- env: CABALVER=2.2 GHCVER=8.4.2 | ||
compiler: "ghc-8.4.2" | ||
addons: {apt: {packages: [cabal-install-2.2,ghc-8.4.2,alex-3.1.7,happy-1.19.5], sources: [hvr-ghc], config: {retries: true}}} | ||
- env: CABALVER=2.2 GHCVER=8.4.1 | ||
compiler: "ghc-8.4.1" | ||
addons: {apt: {packages: [cabal-install-2.2,ghc-8.4.1,alex-3.1.7,happy-1.19.5], sources: [hvr-ghc], config: {retries: true}}} | ||
- env: CABALVER=2.2 GHCVER=8.2.2 | ||
compiler: "ghc-8.2.2" | ||
addons: {apt: {packages: [cabal-install-2.2,ghc-8.2.2,alex-3.1.7,happy-1.19.5], sources: [hvr-ghc], config: {retries: true}}} | ||
- env: CABALVER=2.2 GHCVER=8.2.1 | ||
compiler: "ghc-8.2.1" | ||
addons: {apt: {packages: [cabal-install-2.2,ghc-8.2.1,alex-3.1.7,happy-1.19.5], sources: [hvr-ghc], config: {retries: true}}} | ||
before_install: | ||
- >- | ||
[ -d $OBJECT_DIR ] || | ||
mkdir -p $OBJECT_DIR | ||
|
||
# Download and unpack the stack executable | ||
- >- | ||
mkdir -p ~/.local/bin | ||
- >- | ||
export PATH=$HOME/.local/bin:$PATH | ||
- >- | ||
travis_retry | ||
curl -L https://get.haskellstack.org/stable/linux-x86_64.tar.gz | | ||
tar -x -z -f - --wildcards --strip-components=1 -C ~/.local/bin '*/stack' | ||
|
||
# Get and unroll AutoCorres | ||
- >- | ||
[ -e $OBJECT_DIR/autocorres-${AUTOCORRES_VERSION}.tar.gz ] || | ||
travis_retry | ||
curl -o $OBJECT_DIR/autocorres-${AUTOCORRES_VERSION}.tar.gz | ||
http://ts.data61.csiro.au/projects/TS/autocorres/autocorres-${AUTOCORRES_VERSION}.tar.gz | ||
- >- | ||
tar xf $OBJECT_DIR/autocorres-${AUTOCORRES_VERSION}.tar.gz | ||
- >- | ||
mv autocorres-${AUTOCORRES_VERSION} autocorres | ||
|
||
# Work out what flags to set. | ||
- >- | ||
STACK_FLAGS='' | ||
- >- | ||
[ -z ${WITH_DOCGENT} ] || | ||
STACK_FLAGS="${STACK_FLAGS} --flag cogent:docgent" | ||
- >- | ||
[ -z ${WITH_HASKELL_BACKEND} ] || | ||
STACK_FLAGS="${STACK_FLAGS} --flag cogent:haskell-backend" | ||
- >- | ||
[ -z ${WITH_BUILTIN_ARRAYS} ] || | ||
STACK_FLAGS="${STACK_FLAGS} --flag cogent:builtin-arrays" | ||
|
||
# Pandoc causes Stack to choke; so, with Docgent enabled, no fish. | ||
# - <https://github.com/commercialhaskell/stack/issues/3989> | ||
# - <https://github.com/commercialhaskell/stack/issues/4428> | ||
# | ||
# `haskell-src-exts' causes Haddock to choke (by segfaulting?!) | ||
# - <https://travis-ci.org/NICTA/cogent/jobs/656011492> | ||
- >- | ||
if [ -z ${WITH_DOCGENT} ] && [ -z ${WITH_HASKELL_BACKEND} ]; | ||
then | ||
STACK_FLAGS="${STACK_FLAGS} --haddock --haddock-deps"; | ||
fi | ||
|
||
- >- | ||
export STACK_FLAGS; | ||
echo $STACK_FLAGS | ||
|
||
allow_failures: | ||
install: | ||
# Build dependencies | ||
- cd $REPO/cogent | ||
- stack --no-terminal --install-ghc test --only-dependencies $STACK_FLAGS | ||
|
||
jobs: | ||
fast_finish: true | ||
include: | ||
- env: STACK_YAML=stack-8.2.yaml | ||
compiler: "GHC 8.2.2 + LTS 11.22" | ||
- env: STACK_YAML=stack-8.2.yaml WITH_DOCGENT=1 | ||
compiler: "GHC 8.2.2 + LTS 11.22" | ||
- env: STACK_YAML=stack-8.2.yaml WITH_HASKELL_BACKEND=1 | ||
compiler: "GHC 8.2.2 + LTS 11.22" | ||
- env: STACK_YAML=stack-8.2.yaml WITH_BUILTIN_ARRAYS=1 | ||
compiler: "GHC 8.2.2 + LTS 11.22" | ||
|
||
- env: STACK_YAML=stack-8.4.yaml | ||
compiler: "GHC 8.4.4 + LTS 12.26" | ||
- env: STACK_YAML=stack-8.4.yaml WITH_DOCGENT=1 | ||
compiler: "GHC 8.4.4 + LTS 12.26" | ||
- env: STACK_YAML=stack-8.4.yaml WITH_HASKELL_BACKEND=1 | ||
compiler: "GHC 8.4.4 + LTS 12.26" | ||
- env: STACK_YAML=stack-8.4.yaml WITH_BUILTIN_ARRAYS=1 | ||
compiler: "GHC 8.4.4 + LTS 12.26" | ||
|
||
- env: STACK_YAML=stack-8.6.yaml | ||
compiler: "GHC 8.6.5 + LTS 14.22" | ||
- env: STACK_YAML=stack-8.6.yaml WITH_DOCGENT=1 | ||
compiler: "GHC 8.6.5 + LTS 14.22" | ||
- env: STACK_YAML=stack-8.6.yaml WITH_HASKELL_BACKEND=1 | ||
compiler: "GHC 8.6.5 + LTS 14.22" | ||
- env: STACK_YAML=stack-8.6.yaml WITH_BUILTIN_ARRAYS=1 | ||
compiler: "GHC 8.6.5 + LTS 14.22" | ||
|
||
# TODO: add mixtures of feature flags to test combos | ||
|
||
before_install: | ||
- unset CC | ||
- export ALEXVER=3.1.7 | ||
- export HAPPYVER=1.19.5 | ||
- export PATH=~/.cabal/bin:/opt/ghc/$GHCVER/bin:/opt/cabal/$CABALVER/bin:/opt/happy/$HAPPYVER/bin:/opt/alex/$ALEXVER/bin:$REPO/cogent/$SANDBOX/bin:$PATH | ||
# - pip --version | ||
# - sudo pip --version | ||
- pip install termcolor ruamel.yaml | ||
allow_failures: | ||
# `builtin-arrays' on `master' and derived branches is broken. | ||
# When Dargent support lands, we can re-enable these. | ||
- env: STACK_YAML=stack-8.2.yaml WITH_BUILTIN_ARRAYS=1 | ||
compiler: "GHC 8.2.2 + LTS 11.22" | ||
- env: STACK_YAML=stack-8.4.yaml WITH_BUILTIN_ARRAYS=1 | ||
compiler: "GHC 8.4.4 + LTS 12.26" | ||
- env: STACK_YAML=stack-8.6.yaml WITH_BUILTIN_ARRAYS=1 | ||
compiler: "GHC 8.6.5 + LTS 14.22" | ||
|
||
install: | ||
# Get AutoCorres | ||
- wget http://ts.data61.csiro.au/projects/TS/autocorres/autocorres-1.6.tar.gz | ||
- tar -xf autocorres-1.6.tar.gz | ||
- mv autocorres-1.6 autocorres | ||
# setup Cabal | ||
- cabal --version | ||
- echo "$(ghc --version) [$(ghc --print-project-git-commit-id 2> /dev/null || echo '?')]" | ||
- if [ -f "$HOME/.cabal/packages/hackage.haskell.org/00-index.tar.gz" ]; | ||
then | ||
echo "==== Hackage index file found ===="; | ||
zcat $HOME/.cabal/packages/hackage.haskell.org/00-index.tar.gz > | ||
$HOME/.cabal/packages/hackage.haskell.org/00-index.tar; | ||
fi | ||
- travis_retry cabal update # -v | ||
- cd cogent | ||
- cabal sandbox init | ||
- cabal sandbox add-source ../isa-parser | ||
- "sed -i 's/^jobs:.*$/jobs: 2/' $HOME/.cabal/config" | ||
# - sed -i 's/^jobs:/ jobs:/' ${HOME}/.cabal/config | ||
# Use fixed build plans | ||
- cp misc/cabal.config.d/cabal.config-$GHCVER cabal.config | ||
- | | ||
# In the future, the nix-style thingy might be helpful | ||
if cabal install --only-dependencies --force-reinstalls --enable-tests --dry -v --flags="haskell-backend docgent" > installplan.txt; | ||
then | ||
sed -i -e '1,/^Resolving /d' installplan.txt; | ||
echo "==== Succeeded to generate an install plan. ===="; | ||
cat installplan.txt; # Debugging | ||
else | ||
echo "==== Failed to generate an install plan. ===="; | ||
cat installplan.txt; # Display errors | ||
fi | ||
|
||
# check whether current requested install-plan matches cached package-db snapshot | ||
- | | ||
if diff -u $HOME/.cabsnap/installplan.txt installplan.txt; | ||
then | ||
echo "==== cabal build-cache HIT ===="; # so copy cached sandbox | ||
rm -rf $SANDBOX | ||
cp -a $HOME/.cabsnap/$SANDBOX $REPO/cogent/ | ||
else | ||
echo "==== cabal build-cache MISS (or error) ===="; | ||
rm -rf $HOME/.cabsnap; | ||
# mkdir -p $HOME/.ghc $HOME/.cabal/lib $HOME/.cabal/share $HOME/.cabal/bin; | ||
cabal install --only-dependencies --force-reinstalls --flags="haskell-backend docgent"; # --enable-tests; | ||
fi | ||
|
||
# snapshot package-db on cache miss | ||
- | | ||
if [ ! -d $HOME/.cabsnap ]; | ||
then | ||
echo "==== snapshotting package-db to build-cache ===="; | ||
mkdir $HOME/.cabsnap; | ||
cp installplan.txt $HOME/.cabsnap/. | ||
cp -a $SANDBOX $HOME/.cabsnap/ | ||
fi | ||
|
||
# Install z3 | ||
- cd $REPO | ||
- git submodule update --init -- z3 | ||
- cd z3 | ||
- mkdir -p "$HOME"/z3 | ||
- export PATH="$HOME/z3/bin:$PATH" | ||
- | | ||
if ! which z3; | ||
then # install z3 | ||
python scripts/mk_make.py --prefix="$HOME/z3" | ||
cd build | ||
make | ||
make install | ||
fi | ||
- which z3 || travis_terminate 1 | ||
|
||
# Here starts the actual work to be performed for the package under test; | ||
# any command which exits with a non-zero exit code causes the build to fail. | ||
script: | ||
- cd $REPO/cogent | ||
# - echo 'Configuring Cogent' | ||
- cabal configure --flags="haskell-backend docgent" # --enable-tests # or -v2 to provide useful information for debugging | ||
# - echo 'Installing Cogent' | ||
- cabal install --force-reinstalls --flags="haskell-backend docgent" # --enable-tests | ||
# - cabal build # this builds all libraries and executables (including tests/benchmarks) | ||
# - cabal test # this rarely changes, so disabled for now to save some time / zilinc | ||
# - cabal check | ||
- cabal haddock | ||
# Check if Cogent has been correctly installed | ||
- which cogent || travis_terminate 1 | ||
# - echo 'Running tests' | ||
- make test-compiler | ||
# - echo 'Typechecking libgum' | ||
- make test-libgum | ||
# - echo 'Building Cogent examples' | ||
- make examples | ||
# - echo 'Generating ext2fs C code from Cogent' | ||
- cd $REPO/impl/fs/ext2/cogent && travis_wait make .c-gen | ||
# - echo 'Generating BilbyFs C code from Cogent' | ||
- cd $REPO/impl/fs/bilby/cogent && travis_wait make .c-gen | ||
# Build the package, its tests, and run the tests | ||
- cd $REPO/cogent | ||
- stack --no-terminal test $STACK_FLAGS | ||
- cogent_bin=$(stack --no-terminal exec -- which cogent) | ||
- install -m 755 ${cogent_bin} ~/.local/bin | ||
|
||
- which cogent || travis_terminate 1 | ||
|
||
- make test-compiler | ||
- make test-libgum | ||
|
||
- make examples | ||
|
||
- cd $REPO/impl/fs/ext2/cogent && make linux | ||
- cd $REPO/impl/fs/bilby/cogent && make .c-gen | ||
# cd $REPO/impl/fs/bilby/cogent && make linux | ||
|
||
notifications: | ||
slack: | ||
rooms: | ||
- cogent-club:dUKOfdIU3BGsvalWStE0BCGG#ci | ||
# EOF | ||
secure: HFQu8x7qm0lsh5PtVP4DQki2dpBfDaXL0Hp4ocCITFrNtIXX/6Oba3qNAomHfGtUcSiIMl+gPiRl9aFtKGwJT/8znePL6WndTT0vyaS2T/OvYr4joHA6XHRlNjtL4jg/CT1Yqu+Kb9+Njs5xG88MdENQupNkCOKWJfGY5pggUtAdnE5HMJ6jO0u6KRM2rOY8uf2NLABR+2habfYwHgF2RR8fplAzcPv32xikA08Ne70SBmhSYg/ALiSjrmMy1k5gxm46Qeckde12TqsP3xijr06kOiAOo57PZYSmZCwk5La3kRmOf6L+IZc8fSzg7HaDP8Hpk8xnkqvx3Wcq0nZK4LMi4/4OpKLHOHd/MeD3GCy7oyCFMw/+z+LvlbWB/W4QOkCqNsLnSCLnzt+GbV3CZsdVuUtHi34JFssLkbRqdQiYtSUrh0q5mqRyFuOruB8v9s6wiL1Bla7ZMX0qYn6tJYsxurCOdDDgMHV0QitralcFsynhbXcGUjHSinNolBFtUxoyOErlpC6IaFVGS2ch0szFoPCqilYifNl48/8XjBHA52JDKD2tsl+K5PTKyLcLDWwgQePzN2NIThYO3UDs/Gyz68VHgihpMw7XYaRw+T7d/2KQ3FM+/2JeohmDKIJMkupGHZ8m+185ixu8Px6U4GKCh+892c6Lek2fd7VAlHE= |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
resolver: lts-11.22 # ghc-8.2.2; 2018-08-18 | ||
packages: | ||
- '.' | ||
- '../isa-parser' | ||
|
||
flags: | ||
language-c-quote: | ||
full-haskell-antiquotes: false | ||
|
||
extra-package-dbs: [] | ||
extra-deps: | ||
- sbv-7.12 # for the `array' branches | ||
- crackNum-2.3 # for `sbv' on old GHC | ||
- microlens-0.4.9.1 # we use (<>~), new in 0.4.9 | ||
- microlens-ghc-0.4.9.1 | ||
- haskell-exp-parser-0.1.1 | ||
- pretty-show-ansi-wl-1.9.2.1 |
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
stack-8.2.2.yaml |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
resolver: lts-12.14 # ghc-8.4.3; 2018-10-21 | ||
packages: | ||
- '.' | ||
- '../isa-parser' | ||
|
||
flags: | ||
language-c-quote: | ||
full-haskell-antiquotes: false | ||
|
||
extra-package-dbs: [] | ||
extra-deps: | ||
- sbv-7.12 # for the `array' branches | ||
- haskell-exp-parser-0.1.1 | ||
- pretty-show-ansi-wl-1.9.2.1 |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A question: on my branch, I use
sbv > 8
. Which version ofsbv
can I have viaextra-deps
?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is only to get the pins right given the current builtin-arrays code. In theory, you can have whatever version you like. I believe sbv-8.x works fine, although I vaguely recall running into issues with crackNum, a sbv dependency, on GHC 8.2; I'd need to test that again to check, and I'd want to make sure stuff still compiles happily in general anyway.