Unaccurate response regarding a prompt and context

[sancelot]

Hi,

I am trying to generate questions from a given context.
Unfortunately, the answer is not accurate and sometimes out of context .

I don't understand if this is a prompt problem or something else.

here is the code :

from llama_cpp import Llama

# define n_ctx manually to permit larger contexts
LLM = Llama(model_path="../../../../llama-2-7b.Q5_K_M.gguf", n_ctx=512)

# create a text prompt
context = """
Nous sommes dans les Yvelines
Il fait beau
Nous sommes a quelques km d'un joli lac
le boulanger est sympatique
Il y a également une boucherie
"""

prompt = """
You are a Teacher/ Professor. Your task is to setup
4 questions for an upcoming
    quiz/examination. The questions should be diverse in nature 
   across the document. Restrict the questions to the 
    context  provided.
 Context is provided below :                                                                                                                                                                                                                                                                                                                                             
   %s                                                                                                                                                                                                                                                                                                                  
""" % context


print(prompt)
# set max_tokens to 0 to remove the response size limit
output = LLM(prompt, max_tokens=0)

# display the response
print(output["choices"][0]["text"])

print("---------------")
print(output)`

[sancelot]

Answer : I had to use an instruct model and not a "base" model
And GIVE the right prompt needed for the model : [INST] .... [/INST]