Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Speed up database construction pipeline #43

Merged
merged 51 commits into from
Mar 14, 2024
Merged

Speed up database construction pipeline #43

merged 51 commits into from
Mar 14, 2024

Commits on Jan 31, 2024

  1. Merge pull request #8 from stijndcl/feature/rust-chunks 

    Feature/Rust chunks
    @stijndcl
    stijndcl authored Jan 31, 2024
    Configuration menu
    Copy the full SHA
    c0d73e3 View commit details
    Browse the repository at this point in the history

  2. Merge pull request #9 from stijndcl/feature/rust-dat-parser 

    Feature/Rust DAT parser
    @stijndcl
    stijndcl authored Jan 31, 2024
    Configuration menu
    Copy the full SHA
    df55342 View commit details
    Browse the repository at this point in the history

  3. Try using rust binaries

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    3f8e340 View commit details
    Browse the repository at this point in the history

  4. Actually download dependencies

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    4551b6d View commit details
    Browse the repository at this point in the history

  5. Don't pull files from gh

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    6389a08 View commit details
    Browse the repository at this point in the history

  6. Update comments

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    d9c00ec View commit details
    Browse the repository at this point in the history

  7. Add path to xml file

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    e09fed1 View commit details
    Browse the repository at this point in the history

  8. Use local xml file

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    c5d33ec View commit details
    Browse the repository at this point in the history

  9. Fix arguments

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    7f5417c View commit details
    Browse the repository at this point in the history

  10. List root

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    07a7b9a View commit details
    Browse the repository at this point in the history

  11. Remove ls

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    a0091c9 View commit details
    Browse the repository at this point in the history

  12. Add java version of downloadless

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    b4750c4 View commit details
    Browse the repository at this point in the history

  13. Don't remove xml file yet

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    4791ac0 View commit details
    Browse the repository at this point in the history

  14. Use rust tools

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    da7a00f View commit details
    Browse the repository at this point in the history

  15. Use pigz instead of gzip

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    215c30a View commit details
    Browse the repository at this point in the history

  16. Fix zcat

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    4adda03 View commit details
    Browse the repository at this point in the history

  17. Usel z4

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    72646b5 View commit details
    Browse the repository at this point in the history

  18. Fix typo

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    be583a8 View commit details
    Browse the repository at this point in the history

  19. Decompress multiple filesg

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    e1766ca View commit details
    Browse the repository at this point in the history

  20. Try something

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    dddf00d View commit details
    Browse the repository at this point in the history

  21. Fix find command

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    2d2e0c1 View commit details
    Browse the repository at this point in the history

  22. Fix missing lz4 extensions

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    ddd6316 View commit details
    Browse the repository at this point in the history

  23. Add debug prints

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    4906468 View commit details
    Browse the repository at this point in the history

  24. Fix typo

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    942b03d View commit details
    Browse the repository at this point in the history

  25. Print taxon id

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    b7da44c View commit details
    Browse the repository at this point in the history

  26. Stash

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    3b2ca13 View commit details
    Browse the repository at this point in the history

  27. Remove all sorts

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    83bce27 View commit details
    Browse the repository at this point in the history

  28. use two files

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    a0fb218 View commit details
    Browse the repository at this point in the history

  29. Fix typo

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    7b6de76 View commit details
    Browse the repository at this point in the history

  30. Fix typo

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    f099c11 View commit details
    Browse the repository at this point in the history

  31. Fix typo

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    9c41264 View commit details
    Browse the repository at this point in the history

  32. Fix compression

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    66e3f1a View commit details
    Browse the repository at this point in the history

  33. Remove debugging line

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    bc7c07d View commit details
    Browse the repository at this point in the history

  34. Parallellize

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    820e638 View commit details
    Browse the repository at this point in the history

  35. Remove awk

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    38a43ac View commit details
    Browse the repository at this point in the history

  36. Add parallellization back

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    98755e8 View commit details
    Browse the repository at this point in the history

  37. Produce actual valid output

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    6e41f80 View commit details
    Browse the repository at this point in the history

  38. Optimise number-sequences and remove todo

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    87e65a7 View commit details
    Browse the repository at this point in the history

  39. Remove binaries

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    1f5bdd9 View commit details
    Browse the repository at this point in the history

  40. Remove all non-rust code from the pipeline

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    fec706f View commit details
    Browse the repository at this point in the history

  41. Add lz4 to db scripts

    @stijndcl
    stijndcl committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    9fd58af View commit details
    Browse the repository at this point in the history

Commits on Feb 6, 2024

  1. Create script to compile and move binaries, ignore them

    @stijndcl
    stijndcl committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    8c854b0 View commit details
    Browse the repository at this point in the history

  2. Remove hardcoded list

    @stijndcl
    stijndcl committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    a378d2c View commit details
    Browse the repository at this point in the history

Commits on Mar 6, 2024

  1. Merge pull request #40 from stijndcl/feature/pure-rust 

    Remove all Java and JS tools, use Lz4
    @pverscha
    pverscha authored Mar 6, 2024
    Configuration menu
    Copy the full SHA
    9005cf6 View commit details
    Browse the repository at this point in the history

Commits on Mar 8, 2024

  1. Fix changes from Pieter

    @stijndcl
    stijndcl committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    e90e7cc View commit details
    Browse the repository at this point in the history

  2. Fix ifs quoting

    @stijndcl
    stijndcl committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    d963063 View commit details
    Browse the repository at this point in the history

Commits on Mar 14, 2024

  1. Merge pull request #42 from stijndcl/fix/rust-changes 

    Fix requested changes from Pieter for Rust-only version
    @pverscha
    pverscha authored Mar 14, 2024
    Configuration menu
    Copy the full SHA
    c8886bb View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into feature/stijn-changes

    @pverscha
    pverscha committed Mar 14, 2024
    Configuration menu
    Copy the full SHA
    25013e9 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'feature/stijn-changes' of https://github.com/unipept/ma… 

    …ke-database into feature/stijn-changes
    @pverscha
    pverscha committed Mar 14, 2024
    Configuration menu
    Copy the full SHA
    f8b6b74 View commit details
    Browse the repository at this point in the history

  4. UniProtType no longer required

    @pverscha
    pverscha committed Mar 14, 2024
    Configuration menu
    Copy the full SHA
    409f0d5 View commit details
    Browse the repository at this point in the history

  5. Fix formatting errors

    @pverscha
    pverscha committed Mar 14, 2024
    Configuration menu
    Copy the full SHA
    9ea99e5 View commit details
    Browse the repository at this point in the history