Instructions on PIE Evaluation

[mcao516]

Hi,

Thank you for your fantastic work!

It seems like the instructions for conducting PIE evaluation are absent. Would you be able to provide instructions on how to use the pie_eval.py script? I'm particularly uncertain about the process of obtaining the .report file. Thanks!

[madaan]

Hi,

Thanks for your question. The instructions for evaluating the runtime are located here: https://github.com/madaan/pie-perf#evaluating-your-method.

[mcao516]

Hello @madaan,

Thank you for the quick reply.

Could you please provide instructions on how to convert the output file generated by pie/run.py into the jsonl file mentioned in step 1? Thank you!