Score function

[DataGomes]

I am building an evaluation using Langfuse. However, for full reliability I need to know how the scores function works, I was not able to find it in the open source component. How does it extract the data?

In my case I am using a scale from 1 to 5, and want to use the score with it. Since the evaluation is fundamental for my company I need to know how does it extract the score. Does anyone know where can I find this information?


[dosubot[bot]]

Olá @DataGomes! Eu sou um bot que pode te ajudar a resolver bugs, responder perguntas e se tornar um contribuidor. Estou aqui para te ajudar enquanto aguardamos um mantenedor humano.

The score function in Langfuse is designed to store evaluation metrics as Scores, which can be linked to a trace and optionally reference a specific observation within that trace. These scores can be numeric, categorical, or boolean, and you can add comments for additional context. They can be associated with a score configuration to ensure they adhere to a specific schema. Scores are used for displaying on traces, segmenting execution traces, and detailed analytics reporting.

Langfuse offers a flexible scoring system to consolidate all your evaluations in one place and make them actionable. You can add scores using the Python or JS SDK, and the system supports various evaluation methods, including model-based evaluations, manual annotations, user feedback, and custom evaluations via SDKs or API.

For detailed information about the score function, you can refer to the Langfuse documentation at https://langfuse.com/docs/scores/overview and https://langfuse.com/docs/scores/data-model ^[1][2].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[marcklingen]

No idea why the initial sentence is spanish here, the overall response looks good though! let me know if you have any follow up questions

[DataGomes]

Thank you for the answer , can I change the value of the scores?

My objective is to have 5 discrete values that are explained to the LLM, with each value having one clear expectation.

For example, 1: incorrect answer, 2: some components of a correct answer, 3: partially correct answer, 4: mostly correct answer, 5: correct answer.

I want the scores to always follow these guidelines.

[marcklingen]

You can use a categorical score (setup a score config) and then add these via the api. Alternatively, you can annotate this via the UI.