Question about the generate_one_completion

[xupeng1910]

Hey, I wanna ask one question about this 'generate_one_completion' in your usage part in the README file. What is that? Is that any Python function that has complete logic?

{"task_id": "Corresponding HumanEval task ID", "completion": "Completion only without the prompt"}
We provide example_problem.jsonl and example_solutions.jsonl under data to illustrate the format and help with debugging.

Here is a nearly functional example code (you just have to provide generate_one_completion to make it work) that saves generated completions to samples.jsonl.

[abtExp]

@xupeng1910 the generate_one_completion function returns the model output or the completed code without the input prompt to the model. As you can see in the example_solutions.jsonl and example_problem.jsonl, the input "prompt" in example_problem.jsonl is "def return1():\n" and the "completion"s are in example_solutions.jsonl, these will be the predictions from your model which will then be appended with the prompt and will be tested by executing them. So the generate_one_completion takes in one prompt, calls the model and returns the model_prediction