Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Evaluators can access all columns #1606

Merged
merged 76 commits into from
May 31, 2024
Merged

Evaluators can access all columns #1606

merged 76 commits into from
May 31, 2024

Commits on May 2, 2024

  1. access to all columns

    @aakrem
    aakrem committed May 2, 2024
    Configuration menu
    Copy the full SHA
    7145533 View commit details
    Browse the repository at this point in the history

  2. fix correct answer in ai critique

    @aakrem
    aakrem committed May 2, 2024
    Configuration menu
    Copy the full SHA
    d3d3412 View commit details
    Browse the repository at this point in the history

  3. fix tests

    @aakrem
    aakrem committed May 2, 2024
    Configuration menu
    Copy the full SHA
    e4a481f View commit details
    Browse the repository at this point in the history

Commits on May 3, 2024

  1. add correct answer to evaluators

    @aakrem
    aakrem committed May 3, 2024
    Configuration menu
    Copy the full SHA
    6c09e6e View commit details
    Browse the repository at this point in the history

  2. get correct answer from evaluator instead of evaluation payload

    @aakrem
    aakrem committed May 3, 2024
    Configuration menu
    Copy the full SHA
    2a62d29 View commit details
    Browse the repository at this point in the history

  3. access correct answer directly in the evaluators and handle passing m… 

    …ultiple correct answers
    @aakrem
    aakrem committed May 3, 2024
    Configuration menu
    Copy the full SHA
    ae58b0e View commit details
    Browse the repository at this point in the history

  4. add default value for correct_answer and small renaming

    @aakrem
    aakrem committed May 3, 2024
    Configuration menu
    Copy the full SHA
    d54f20d View commit details
    Browse the repository at this point in the history

Commits on May 9, 2024

  1. adjust schema

    @aakrem
    aakrem committed May 9, 2024
    Configuration menu
    Copy the full SHA
    c9c212a View commit details
    Browse the repository at this point in the history

  2. add as much correct answers columns there are in an ES

    @aakrem
    aakrem committed May 9, 2024
    Configuration menu
    Copy the full SHA
    be1ee39 View commit details
    Browse the repository at this point in the history

  3. ajust correct answer type in frontend

    @aakrem
    aakrem committed May 9, 2024
    Configuration menu
    Copy the full SHA
    e18c3a4 View commit details
    Browse the repository at this point in the history

  4. small build fix

    @aakrem
    aakrem committed May 9, 2024
    Configuration menu
    Copy the full SHA
    ed2d413 View commit details
    Browse the repository at this point in the history

  5. fix build

    @aakrem
    aakrem committed May 9, 2024
    Configuration menu
    Copy the full SHA
    2b131d2 View commit details
    Browse the repository at this point in the history

  6. hsndle multiple correct answers

    @aakrem
    aakrem committed May 9, 2024
    Configuration menu
    Copy the full SHA
    519e99d View commit details
    Browse the repository at this point in the history

  7. revert to single ground truth

    @aakrem
    aakrem committed May 9, 2024
    Configuration menu
    Copy the full SHA
    68f626e View commit details
    Browse the repository at this point in the history

  8. toggle correct answer input visibility

    @bekossy
    bekossy committed May 9, 2024
    Configuration menu
    Copy the full SHA
    ac6bf19 View commit details
    Browse the repository at this point in the history

  9. rename correct_answer to value

    @aakrem
    aakrem committed May 9, 2024
    Configuration menu
    Copy the full SHA
    d2e1a36 View commit details
    Browse the repository at this point in the history

  10. migration script

    @aakrem
    aakrem committed May 9, 2024
    Configuration menu
    Copy the full SHA
    07e0e86 View commit details
    Browse the repository at this point in the history

  11. added antd collapse to toggle correct_answer input

    @bekossy
    bekossy committed May 9, 2024
    Configuration menu
    Copy the full SHA
    fc30f52 View commit details
    Browse the repository at this point in the history

  12. fix evaluators tests

    @aakrem
    aakrem committed May 9, 2024
    Configuration menu
    Copy the full SHA
    7e24866 View commit details
    Browse the repository at this point in the history

  13. select ground truth to apply diff in eval scenario view

    @bekossy
    bekossy committed May 9, 2024
    Configuration menu
    Copy the full SHA
    bab7582 View commit details
    Browse the repository at this point in the history

  14. display only unique correct_answers

    @aakrem
    aakrem committed May 9, 2024
    Configuration menu
    Copy the full SHA
    0f67bd3 View commit details
    Browse the repository at this point in the history

Commits on May 10, 2024

  1. filtered out duplicate keys from correctAnswer array

    @bekossy
    bekossy committed May 10, 2024
    Configuration menu
    Copy the full SHA
    c01f6b2 View commit details
    Browse the repository at this point in the history

  2. added filtercolumns component and improve table headername display

    @bekossy
    bekossy committed May 10, 2024
    Configuration menu
    Copy the full SHA
    cb3b08f View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into access-to-all-columns

    @bekossy
    bekossy authored May 10, 2024
    Configuration menu
    Copy the full SHA
    725cf8d View commit details
    Browse the repository at this point in the history

  4. bug fix

    @bekossy
    bekossy committed May 10, 2024
    Configuration menu
    Copy the full SHA
    00529d0 View commit details
    Browse the repository at this point in the history

  5. bug fix

    @bekossy
    bekossy committed May 10, 2024
    Configuration menu
    Copy the full SHA
    398dfe8 View commit details
    Browse the repository at this point in the history

Commits on May 11, 2024

  1. added dropdown diff and cleanup

    @bekossy
    bekossy committed May 11, 2024
    Configuration menu
    Copy the full SHA
    89faad9 View commit details
    Browse the repository at this point in the history

Commits on May 12, 2024

  1. made static onClick prop dynamic and improve diff feature

    @bekossy
    bekossy committed May 12, 2024
    Configuration menu
    Copy the full SHA
    14e5eb4 View commit details
    Browse the repository at this point in the history

Commits on May 13, 2024

  1. added ground truth column to comparison view and improved diff feature

    @bekossy
    bekossy committed May 13, 2024
    Configuration menu
    Copy the full SHA
    e152c51 View commit details
    Browse the repository at this point in the history

  2. fixed correct answer output

    @bekossy
    bekossy committed May 13, 2024
    Configuration menu
    Copy the full SHA
    0495376 View commit details
    Browse the repository at this point in the history

Commits on May 14, 2024

  1. added helper to remove correctAnswer prefix and improved dropdown def… 

    …ault text
    @bekossy
    bekossy committed May 14, 2024
    Configuration menu
    Copy the full SHA
    d879572 View commit details
    Browse the repository at this point in the history

  2. rename variable

    @aakrem
    aakrem committed May 14, 2024
    Configuration menu
    Copy the full SHA
    3ebabb3 View commit details
    Browse the repository at this point in the history

  3. Merge pull request #1645 from Agenta-AI/sub-issue/-improve-eval-compa… 

    …rison-view-to-show-all-ground-truth-columns

[Sub issue]: Improve Evaluation comparison view to show all ground truth columns
    @aakrem
    aakrem authored May 14, 2024
    Configuration menu
    Copy the full SHA
    975b01f View commit details
    Browse the repository at this point in the history

Commits on May 15, 2024

  1. improved diff button text

    @bekossy
    bekossy committed May 15, 2024
    Configuration menu
    Copy the full SHA
    fc8ae62 View commit details
    Browse the repository at this point in the history

  2. access to all columns

    @aakrem
    aakrem committed May 15, 2024
    Configuration menu
    Copy the full SHA
    90f7380 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into access-to-all-columns

    @aakrem
    aakrem committed May 15, 2024
    Configuration menu
    Copy the full SHA
    34f2436 View commit details
    Browse the repository at this point in the history

  4. small refactor for correct answers logic

    @aakrem
    aakrem committed May 15, 2024
    Configuration menu
    Copy the full SHA
    14c8ee2 View commit details
    Browse the repository at this point in the history

  5. fix errors type

    @aakrem
    aakrem committed May 15, 2024
    Configuration menu
    Copy the full SHA
    7424be3 View commit details
    Browse the repository at this point in the history

Commits on May 16, 2024

  1. convert correct_answer_keys to list

    @aakrem
    aakrem committed May 16, 2024
    Configuration menu
    Copy the full SHA
    95b82ef View commit details
    Browse the repository at this point in the history

  2. improve type

    @aakrem
    aakrem committed May 16, 2024
    Configuration menu
    Copy the full SHA
    1d3d9d8 View commit details
    Browse the repository at this point in the history

Commits on May 17, 2024

  1. access to all columns

    @aakrem
    aakrem committed May 17, 2024
    Configuration menu
    Copy the full SHA
    87c1d81 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into access-to-all-columns

    @aakrem
    aakrem committed May 17, 2024
    Configuration menu
    Copy the full SHA
    a706a69 View commit details
    Browse the repository at this point in the history

  3. add default correct answer in case its not provided

    @aakrem
    aakrem committed May 17, 2024
    Configuration menu
    Copy the full SHA
    b5300df View commit details
    Browse the repository at this point in the history

  4. advanced settings in a separate component

    @aakrem
    aakrem committed May 17, 2024
    Configuration menu
    Copy the full SHA
    1002198 View commit details
    Browse the repository at this point in the history

Commits on May 18, 2024

  1. bug fix

    @bekossy
    bekossy committed May 18, 2024
    Configuration menu
    Copy the full SHA
    7624354 View commit details
    Browse the repository at this point in the history

Commits on May 19, 2024

  1. Merge pull request #1665 from Agenta-AI/access-to-all-columns-advance… 

    …d-settings

advanced settings in a separate component
    @aakrem
    aakrem authored May 19, 2024
    Configuration menu
    Copy the full SHA
    f4e2d47 View commit details
    Browse the repository at this point in the history

  2. fix backend tests

    @aakrem
    aakrem committed May 19, 2024
    Configuration menu
    Copy the full SHA
    f610819 View commit details
    Browse the repository at this point in the history

  3. create direct_use evaluators with default correct answers

    @aakrem
    aakrem committed May 19, 2024
    Configuration menu
    Copy the full SHA
    0aa62e1 View commit details
    Browse the repository at this point in the history

  4. remove not needed code

    @aakrem
    aakrem committed May 19, 2024
    Configuration menu
    Copy the full SHA
    27f8a0a View commit details
    Browse the repository at this point in the history

  5. Add condition to evaluator card to show action buttons when direct_us… 

    …e is false and settings_template has correct_answer_keys
    @bekossy
    bekossy committed May 19, 2024
    Configuration menu
    Copy the full SHA
    dc8ed51 View commit details
    Browse the repository at this point in the history

Commits on May 20, 2024

  1. filtered out evaluators when direct_use is true or settings_template … 

    …without correct_answer_keys from evaluators list
    @bekossy
    bekossy committed May 20, 2024
    Configuration menu
    Copy the full SHA
    ffb3c2d View commit details
    Browse the repository at this point in the history

Commits on May 28, 2024

  1. Modify the evaluator definition for correct answer key

    @mmabrouk
    mmabrouk committed May 28, 2024
    Configuration menu
    Copy the full SHA
    5811770 View commit details
    Browse the repository at this point in the history

  2. Refactored the evaluator service to use specific correct_answers

    @mmabrouk
    mmabrouk committed May 28, 2024
    Configuration menu
    Copy the full SHA
    ca309b5 View commit details
    Browse the repository at this point in the history

  3. Show the advanced settings undere a hidden collapse

    @mmabrouk
    mmabrouk committed May 28, 2024
    Configuration menu
    Copy the full SHA
    1f14f0e View commit details
    Browse the repository at this point in the history

  4. Made the code more secure by removing the global().get which would al… 

    …low running any function
    @mmabrouk
    mmabrouk committed May 28, 2024
    Configuration menu
    Copy the full SHA
    7e11adc View commit details
    Browse the repository at this point in the history

  5. Improved the logic to use a correct_answer as a ground truth column i… 

    …f it is specified as such in the evalutor definition file
    @mmabrouk
    mmabrouk committed May 28, 2024
    Configuration menu
    Copy the full SHA
    81f9b12 View commit details
    Browse the repository at this point in the history

  6. rewrote logic for creating ready to use evaluators

    @mmabrouk
    mmabrouk committed May 28, 2024
    Configuration menu
    Copy the full SHA
    20d54ab View commit details
    Browse the repository at this point in the history

  7. Allow editing ready to use evalutors

    @mmabrouk
    mmabrouk committed May 28, 2024
    Configuration menu
    Copy the full SHA
    e6dcfd3 View commit details
    Browse the repository at this point in the history

  8. allow the addition of ready to use evaluators

    @mmabrouk
    mmabrouk committed May 28, 2024
    Configuration menu
    Copy the full SHA
    9faa400 View commit details
    Browse the repository at this point in the history

  9. Fixed evaluators definition

    @mmabrouk
    mmabrouk committed May 28, 2024
    Configuration menu
    Copy the full SHA
    df16e88 View commit details
    Browse the repository at this point in the history

  10. minor fix

    @mmabrouk
    mmabrouk committed May 28, 2024
    Configuration menu
    Copy the full SHA
    87b0c2f View commit details
    Browse the repository at this point in the history

  11. updated pyproject

    @mmabrouk
    mmabrouk committed May 28, 2024
    Configuration menu
    Copy the full SHA
    a0d0420 View commit details
    Browse the repository at this point in the history

  12. Added auto similarity 

    Removed langchain and use openai directly
    @mmabrouk
    mmabrouk committed May 28, 2024
    Configuration menu
    Copy the full SHA
    1a6e4ce View commit details
    Browse the repository at this point in the history

  13. formatting

    @mmabrouk
    mmabrouk committed May 28, 2024
    Configuration menu
    Copy the full SHA
    6b44f5c View commit details
    Browse the repository at this point in the history

  14. updated docker

    @mmabrouk
    mmabrouk committed May 28, 2024
    Configuration menu
    Copy the full SHA
    3fba85d View commit details
    Browse the repository at this point in the history

  15. fix-lenshtein test

    @mmabrouk
    mmabrouk committed May 28, 2024
    Configuration menu
    Copy the full SHA
    111c2e5 View commit details
    Browse the repository at this point in the history

  16. t

    @mmabrouk
    mmabrouk committed May 28, 2024
    Configuration menu
    Copy the full SHA
    7b69bec View commit details
    Browse the repository at this point in the history

  17. fix the test

    @mmabrouk
    mmabrouk committed May 28, 2024
    Configuration menu
    Copy the full SHA
    31d7c88 View commit details
    Browse the repository at this point in the history

  18. improved tests

    @mmabrouk
    mmabrouk committed May 28, 2024
    Configuration menu
    Copy the full SHA
    ea6919b View commit details
    Browse the repository at this point in the history

Commits on May 29, 2024

  1. remove comment

    @mmabrouk
    mmabrouk committed May 29, 2024
    Configuration menu
    Copy the full SHA
    f5ca784 View commit details
    Browse the repository at this point in the history

  2. improved label

    @mmabrouk
    mmabrouk committed May 29, 2024
    Configuration menu
    Copy the full SHA
    c78856a View commit details
    Browse the repository at this point in the history

  3. fixed correct_answer_key payload

    @bekossy
    bekossy committed May 29, 2024
    Configuration menu
    Copy the full SHA
    7bffe15 View commit details
    Browse the repository at this point in the history

  4. cleanup

    @bekossy
    bekossy committed May 29, 2024
    Configuration menu
    Copy the full SHA
    d285754 View commit details
    Browse the repository at this point in the history

Commits on May 30, 2024

  1. Merge pull request #1711 from Agenta-AI/fix-all-columns 

    Fix evaluation PR
    @mmabrouk
    mmabrouk authored May 30, 2024
    Configuration menu
    Copy the full SHA
    2080680 View commit details
    Browse the repository at this point in the history

Commits on May 31, 2024

  1. Merge branch 'main' into access-to-all-columns

    @mmabrouk
    mmabrouk committed May 31, 2024
    Configuration menu
    Copy the full SHA
    d6132bc View commit details
    Browse the repository at this point in the history

  2. update lock

    @mmabrouk
    mmabrouk committed May 31, 2024
    Configuration menu
    Copy the full SHA
    b9dde52 View commit details
    Browse the repository at this point in the history