Help Needed: Multiple PDF Summarizer based on Metadata

[ealss101]

I have a lot of pdfs chunked, embedded in astraDB and i have now figured out a way to have metadata associated with them which is a breakthrough for me. But I need now to be able to summarize them.

Let's say the file names in astraDB are

1. epic_file_101.pdf
2. awesome_paper123.pdf
3. crazy_langfow_documentation.pdf

Typical query:
Summarize the epic file and combine insights from the awesome paper to generate a nice report of them both?

What would be the best tool or set of tools or components to make something like this happen? As you can see, i want the agent to identify that i mean use files 1) and 2) due to similarities in the file names and only extract text from those 2 files, not from any other files.

Thanks!

[dosubot[bot]]

To achieve the summarization of specific PDF files based on metadata or file name patterns in Langflow, you can use a combination of components to filter, extract, and summarize the content. Here's a suggested approach:

1. File Retrieval and Filtering:

   • Use the Astra DB Search component to query your database for files that match specific patterns in their names or metadata. You can construct a query that identifies files with names containing "epic" and "awesome" to retrieve only those files.

2. Text Extraction:

   • Once you have the relevant files, use the File Component to load these PDFs.
   • Use the Parse Data component to extract text from the loaded documents.

3. Text Summarization:

   • Use the Prompt Component to create a structured query that combines the extracted text from both documents. You can design a prompt that instructs the model to summarize the content and combine insights from both documents.
   • Use the OpenAI Model component to process the prompt and generate a summary.

4. Report Generation:

   • The output from the OpenAI Model can be formatted into a cohesive report using additional components or custom logic to structure the final output.

This setup allows you to dynamically filter and process specific documents based on their metadata or file names, and generate a combined summary report. While Langflow does not have built-in support for querying and filtering files based on metadata or file names, you can implement this logic using the Astra DB Search component and custom queries ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}