Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/k8s support #1069

Merged
merged 68 commits into from
Oct 1, 2024
Merged

Commits on May 6, 2024

  1. Configuration menu
    Copy the full SHA
    7147bfe View commit details
    Browse the repository at this point in the history

Commits on May 7, 2024

  1. Update Exacloud support

    lbeckman314 committed May 7, 2024
    Configuration menu
    Copy the full SHA
    af00d03 View commit details
    Browse the repository at this point in the history
  2. Clean up console debug output

    [no ci]
    lbeckman314 committed May 7, 2024
    Configuration menu
    Copy the full SHA
    fc3d277 View commit details
    Browse the repository at this point in the history

Commits on May 8, 2024

  1. Add default Worker engine ('docker')

    [no ci]
    lbeckman314 committed May 8, 2024
    Configuration menu
    Copy the full SHA
    9dbef33 View commit details
    Browse the repository at this point in the history

Commits on May 9, 2024

  1. Configuration menu
    Copy the full SHA
    a3a3445 View commit details
    Browse the repository at this point in the history

Commits on May 10, 2024

  1. Configuration menu
    Copy the full SHA
    8ec817b View commit details
    Browse the repository at this point in the history

Commits on May 15, 2024

  1. Configuration menu
    Copy the full SHA
    afc5825 View commit details
    Browse the repository at this point in the history

Commits on May 16, 2024

  1. Update ContainerEngine interface

    [no ci]
    lbeckman314 committed May 16, 2024
    Configuration menu
    Copy the full SHA
    28905c0 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    703cf44 View commit details
    Browse the repository at this point in the history

Commits on May 17, 2024

  1. Add initial Exadocker unit test

    [no ci]
    lbeckman314 committed May 17, 2024
    Configuration menu
    Copy the full SHA
    83d6e1c View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    39584ad View commit details
    Browse the repository at this point in the history
  3. Add small fix in HPC Template config

    [no ci]
    lbeckman314 committed May 17, 2024
    Configuration menu
    Copy the full SHA
    1d93375 View commit details
    Browse the repository at this point in the history
  4. Update worker config path in HPC Template

    [no ci]
    lbeckman314 committed May 17, 2024
    Configuration menu
    Copy the full SHA
    1c4221d View commit details
    Browse the repository at this point in the history

Commits on May 20, 2024

  1. Configuration menu
    Copy the full SHA
    53bb0cb View commit details
    Browse the repository at this point in the history

Commits on May 24, 2024

  1. Update input/output working example (md5sum)

    - Test command:
    ```sh
    funnel task create examples/md5sum.json
    ```
    
    [no ci]
    lbeckman314 committed May 24, 2024
    Configuration menu
    Copy the full SHA
    a422b82 View commit details
    Browse the repository at this point in the history

Commits on May 30, 2024

  1. Update docker.go to implement ContainerEngine interface

    - Can be used in HPC environment with unique docker setups (e.g. exadocker) as well as future command templating efforts.
    
    [no ci]
    lbeckman314 committed May 30, 2024
    Configuration menu
    Copy the full SHA
    021eeef View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    8e2474c View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    f5acbab View commit details
    Browse the repository at this point in the history

Commits on May 31, 2024

  1. Update docker and kin-openapi to compatible versions

    - github.com/docker/docker versions above v24.0.9 resulted in `panic: Can't connect docker client`:
    
    ```
    ➜ go test tests/auth/auth_test.go
    --- FAIL: TestBasicAuthFail (0.01s)
    panic: Can't connect docker client [recovered]
            panic: Can't connect docker client
    
    FAIL    command-line-arguments  0.673s
    FAIL
    ```
    
    - This error appears to be originating from [util/dockerutil/docker.go](https://github.com/ohsu-comp-bio/funnel/blob/f5acbabe85d28f45f2f86db959c4d7bc929334a4/util/dockerutil/docker.go#L28).
    
    - github.com/getkin/kin-openapi versions above v0.112.0 resulted in type errors:
    
    v0.113.0:
    ```
    ➜ go test util/openapi2proto/main.go
    
    util/openapi2proto/main.go:54:38: invalid operation: p.Value.AdditionalProperties != nil (mismatched types openapi3.AdditionalProperties and untyped nil)
    util/openapi2proto/main.go:55:24: cannot use p.Value.AdditionalProperties (variable of type openapi3.AdditionalProperties) as *openapi3.SchemaRef value in argument to getType
    FAIL    command-line-arguments [build failed]
    FAIL
    ```
    
    v0.123.0 (latest):
    ```
    ➜ go test util/openapi2proto/main.go
    
    util/openapi2proto/main.go:43:7: cannot convert "integer" (untyped string constant) to type *openapi3.Types
    util/openapi2proto/main.go:45:7: cannot convert "boolean" (untyped string constant) to type *openapi3.Types
    util/openapi2proto/main.go:47:7: cannot convert "number" (untyped string constant) to type *openapi3.Types
    util/openapi2proto/main.go:49:7: cannot convert "object" (untyped string constant) to type *openapi3.Types
    util/openapi2proto/main.go:54:38: invalid operation: p.Value.AdditionalProperties != nil (mismatched types openapi3.AdditionalProperties and untyped nil)
    util/openapi2proto/main.go:55:24: cannot use p.Value.AdditionalProperties (variable of type openapi3.AdditionalProperties) as *openapi3.SchemaRef value in argument to getType
    util/openapi2proto/main.go:60:7: cannot convert "array" (untyped string constant) to type *openapi3.Types
    util/openapi2proto/main.go:72:17: cannot use p.Value.Type (variable of type *openapi3.Types) as string value in return statement
    util/openapi2proto/main.go:84:32: invalid operation: param.Schema.Value.Type != "" (mismatched types *openapi3.Types and untyped string)
    util/openapi2proto/main.go:175:17: resp.Get undefined (type *openapi3.Responses has no field or method Get)
    util/openapi2proto/main.go:175:17: too many errors
    FAIL    command-line-arguments [build failed]
    FAIL
    ```
    lbeckman314 committed May 31, 2024
    Configuration menu
    Copy the full SHA
    5450366 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    2269fd1 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    8648580 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    75647ce View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    997d3c2 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    56e74c2 View commit details
    Browse the repository at this point in the history

Commits on Jun 1, 2024

  1. Fix minor linting errors

    lbeckman314 committed Jun 1, 2024
    Configuration menu
    Copy the full SHA
    a9411bb View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    ee25bd5 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    449e144 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    f5df70f View commit details
    Browse the repository at this point in the history

Commits on Jun 18, 2024

  1. Configuration menu
    Copy the full SHA
    6374d2f View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    f27f302 View commit details
    Browse the repository at this point in the history

Commits on Jun 20, 2024

  1. Configuration menu
    Copy the full SHA
    0f36f11 View commit details
    Browse the repository at this point in the history
  2. Update Nextflow tests

    lbeckman314 committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    c0cc2a8 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    69bddbe View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    2af9c4d View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    0f14338 View commit details
    Browse the repository at this point in the history
  6. Update Base64Encode test to match Github Actions test

    - TODO: investigate why this is necessary...
    lbeckman314 committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    85ce5de View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    56ec776 View commit details
    Browse the repository at this point in the history

Commits on Jun 21, 2024

  1. Revert glob output change based on TesOutput spec

    - Wildcards in outputs will be uploaded to a directory, with no special handling for the filename.
    
    ```yaml
    tesOutput:
      properties:
        url:
         description: |-
            URL at which the TES server makes the output accessible after the task is complete. When tesOutput.path contains wildcards, it must be a directory; see `tesOutput.path_prefix` for details on how output URLs are constructed in this case.
    ```
    lbeckman314 committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    4bbec3f View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    ece1266 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    c81d0a7 View commit details
    Browse the repository at this point in the history

Commits on Jul 23, 2024

  1. Configuration menu
    Copy the full SHA
    f7905e5 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    edc1600 View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2024

  1. Configuration menu
    Copy the full SHA
    9caa800 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    615b46a View commit details
    Browse the repository at this point in the history
  3. Run go mod tidy

    lbeckman314 committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    b3e6066 View commit details
    Browse the repository at this point in the history
  4. Update Github Actions

    lbeckman314 committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    20c5238 View commit details
    Browse the repository at this point in the history

Commits on Jul 25, 2024

  1. Configuration menu
    Copy the full SHA
    80bf9a7 View commit details
    Browse the repository at this point in the history

Commits on Jul 26, 2024

  1. Configuration menu
    Copy the full SHA
    0660edc View commit details
    Browse the repository at this point in the history
  2. Update test suite

    lbeckman314 committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    9e71330 View commit details
    Browse the repository at this point in the history
  3. Nextflow test debug

    lbeckman314 committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    bd1e79b View commit details
    Browse the repository at this point in the history
  4. Nextflow test debug

    lbeckman314 committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    59fce8b View commit details
    Browse the repository at this point in the history
  5. Nextflow test debug

    lbeckman314 committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    d0f6824 View commit details
    Browse the repository at this point in the history
  6. Nextflow test debug

    lbeckman314 committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    90b69f4 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    d9fb95f View commit details
    Browse the repository at this point in the history
  8. Update S3 tests

    lbeckman314 committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    60de248 View commit details
    Browse the repository at this point in the history
  9. Update S3 tests

    lbeckman314 committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    582b834 View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    f405231 View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    d891fc3 View commit details
    Browse the repository at this point in the history

Commits on Jul 27, 2024

  1. Configuration menu
    Copy the full SHA
    0bdd604 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    eac39ef View commit details
    Browse the repository at this point in the history

Commits on Jul 31, 2024

  1. Configuration menu
    Copy the full SHA
    d7272d4 View commit details
    Browse the repository at this point in the history

Commits on Sep 10, 2024

  1. Update K8s support and documentation

    - Update docker images to use quay.io/ohsu-comp-bio/funnel
    lbeckman314 committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    87fbe0b View commit details
    Browse the repository at this point in the history
  2. Update README.md

    lbeckman314 authored Sep 10, 2024
    Configuration menu
    Copy the full SHA
    976aee0 View commit details
    Browse the repository at this point in the history

Commits on Sep 17, 2024

  1. Configuration menu
    Copy the full SHA
    f5ed646 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    42fe597 View commit details
    Browse the repository at this point in the history

Commits on Sep 18, 2024

  1. Configuration menu
    Copy the full SHA
    a570bf3 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    119b554 View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2024

  1. Configuration menu
    Copy the full SHA
    f8db3a5 View commit details
    Browse the repository at this point in the history