Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jq #965

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

Jq #965

wants to merge 7 commits into from

Conversation

pelikhan
Copy link
Member

@pelikhan pelikhan commented Dec 22, 2024


Here are the key enhancements introduced by the changes:

  • Introduction of JQ Support for Data Querying:

    • 🔄 Added a query option to the DefDataOptions interface, allowing users to specify a jq query to filter data before rendering. This enables more flexible and powerful data manipulation within prompts.
  • Enhanced Public API Surface:

    • 🚀 Updated the public API in "packages/core/src/prompt_template.d.ts" and "packages/core/src/prompt_type.ts". The defData function now supports awaiting data, making it more versatile to handle asynchronous data sources.
    • 📈 Added support for specifying output formats in CSV format directly when calling defData.
  • Improved Prompt Rendering:

    • 😄 Updated the rendering logic for data in the renderDefDataNode function. This change now dynamically handles different data structures and formats, ensuring that prompts are tailored accurately based on the provided data.
  • Internal Logic Improvements:

    • 🔄 Refactored the logic handling data resolution within resolvePromptNode. Now, the system can more efficiently manage asynchronous data fetching and rendering.
    • 💡 Integrated JQ querying into the data resolution process, enhancing the flexibility and power of data handling in prompts.
  • Documentation Update:

    • Updated documentation to reflect the new features, providing clear instructions on how users can leverage these enhancements to manipulate data effectively within their prompts.

AI-generated content by pr-describe may be incorrect

@@ -257,6 +257,7 @@ The `defData` function also supports functions to slice the input rows and colum
- `sliceTail`, number of rows to include from the end
- `sliceSample`, number of rows to pick at random
- `distinct`, list of column names to deduplicate the data based on
- `query`, a [jq](https://jqlang.github.io/jq/) query to filter the data

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The link to the jq documentation is missing a closing parenthesis. It should be https://jqlang.github.io/jq/.

AI-generated content by pr-docs-review-commit missing_jq_link may be incorrect

Copy link

LGTM 🚀

The changes in the GIT_DIFF look good functional-wise. The introduction of PromptDefDataNode provides an additional way to handle complex data structures within prompts, which can be useful for various applications.

Specifically:

  • Adding "defData" functionality with optional format and jq query options enhances the flexibility of handling and displaying data.
  • The implementation to render and resolve these nodes is solid, ensuring data correctness and performance considerations.

Therefore, I approve of this PR. 😊

AI-generated content by pr-review may be incorrect

## jq

Apply a [jq](https://jqlang.github.io/jq/) query to a JSON object.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The jq section is added but lacks an explanation of how to use it with the parsers. Consider providing more details on how to apply a jq query to a JSON object using the parsers.

AI-generated content by pr-docs-review-commit missing_jq_section may be incorrect


```js
const d = parsers.jq(rows, "map({ a })")
```

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The jq example is provided but lacks an explanation of what the code does. Consider adding a brief description or comment to explain the purpose of the example.

AI-generated content by pr-docs-review-commit missing_jq_example may be incorrect

The [def](/genaiscript/reference/scripts/context) also supports setting a `detectPromptInjection` flag to apply the detection to each file.
The [def](/genaiscript/reference/scripts/context)
and [defData](/genaiscript/reference/scripts/context)
functions supports setting a `detectPromptInjection` flag to apply the detection to each file.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The links to def and defData should be formatted as inline code for clarity.

AI-generated content by pr-docs-review-commit missing_link may be incorrect

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant