feat: search for built-ins inside functions

[morgante]

Allow us to statically analyze a pattern for llm_chat usage.

The goal is to replace the (very) hacky isAi function we currently use.

Summary by CodeRabbit

• New Features

  • Introduced AI analysis capabilities with the addition of functions to check for AI usage in patterns.
  • Added support for StaticDefinitions to improve pattern matching performance and clarity.
  • Enhanced PatternInfo structure to include a field indicating AI utilization.

• Bug Fixes

  • Improved handling of uncommitted changes warnings during command execution.

• Chores

  • Cleaned up code by removing commented-out sections and streamlining logic for better readability.

Greptile Summary

This is an auto-generated summary

Enhanced static analysis for detecting built-in functions within the codebase, including updates to track AI usage in parsing processes.

• Updated crates/grit-pattern-matcher/src/pattern/call_built_in.rs to add a name field to CallBuiltIn struct, improving identification and handling of built-in function calls.
• Modified snapshot files in crates/cli_bin/tests/snapshots/ to include a new usesAi field set to false, aiding in tracking AI usage in parsing.
• Ensured that GritCall and PatternName implementations for CallBuiltIn utilize the new name field for enhanced pattern matching capabilities.

[coderabbitai]

Actionable comments posted: 0

Outside diff range, codebase verification and nitpick comments (1)

crates/core/src/built_in_functions.rs (1)

Issue Found: Incorrect parameter passed to call_from_args

The new parameter name is not being correctly passed to the call_from_args function. Instead, kind is being passed.

• File: crates/core/src/pattern_compiler/call_compiler.rs
• Line: The call to BuiltIns::call_from_args needs to pass the name parameter instead of kind.

Please update the function call to correctly pass the name parameter.

Analysis chain

Line range hint 88-98: LGTM! But verify the usage of the new parameter name.

The code changes are approved.

However, ensure that all calls to call_from_args correctly pass the new name parameter.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify all calls to `call_from_args` correctly pass the new `name` parameter.

# Test: Search for the function usage. Expect: Only occurrences of the new signature.
rg --type rust -A 5 $'call_from_args'

Length of output: 976

---

Script:

#!/bin/bash
# Description: Extract lines around the call to `call_from_args` in `crates/core/src/pattern_compiler/call_compiler.rs` to verify the correct usage of the `name` parameter.

# Extract lines around the call to `call_from_args`.
rg --type rust -A 10 -B 5 $'BuiltIns::call_from_args' crates/core/src/pattern_compiler/call_compiler.rs

Length of output: 809

[coderabbitai]

Walkthrough

Walkthrough

The changes introduce several enhancements to the pattern matching and analysis capabilities across multiple modules. Key improvements include the introduction of the StaticDefinitions struct for streamlined access to pattern definitions, the addition of new functions to check for specific AI-related functions, and modifications to existing structures like PatternInfo to include AI usage information. These updates aim to improve clarity, performance, and flexibility in handling patterns and their corresponding definitions.

Changes

File(s)	Change Summary
crates/cli/src/.../apply_pattern.rs	Modified access to pattern definitions in run_apply_pattern, ensuring user warnings remain intact.
crates/core/src/.../analysis.rs	Added uses_named_function and uses_ai functions for enhanced pattern analysis and updated tests.
crates/core/src/.../api.rs	Introduced uses_ai field in PatternInfo struct to indicate AI usage in patterns.
crates/core/src/.../ast_node.rs	Changed children method parameter to use StaticDefinitions, reflecting a refactor in definition handling.
crates/core/src/.../built_in_functions.rs	Updated call_builtin method to include a new name parameter for better identification of built-ins.
crates/core/src/.../pattern_compiler/...	Various updates to CallBuiltIn and PatternBuilder to enhance function call identification.
crates/core/src/.../problem.rs	Added definitions method to Problem struct to retrieve associated static definitions.
crates/grit-pattern-matcher/src/...	Introduced StaticDefinitions struct for static analysis and modified functions to utilize this structure.
crates/grit-pattern-matcher/src/pattern/...	Adjusted pattern-related structures and iterators to incorporate StaticDefinitions, enhancing flexibility.
crates/wasm-bindings/src/.../match_pattern.rs	Updated parse_input_files_internal to integrate AI usage checks into pattern analysis.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant CLI
    participant Analysis
    participant ProblemManager
    participant PatternInfo
    
    User->>CLI: run_apply_pattern()
    CLI->>Analysis: check for uncommitted changes
    Analysis->>ProblemManager: retrieve problem definitions
    ProblemManager->>Analysis: provide StaticDefinitions
    Analysis->>PatternInfo: create with uses_ai info
    PatternInfo-->>CLI: return pattern info
    CLI-->>User: display results

Loading

---

Recent review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 5923f64 and 4632151.

Files ignored due to path filters (4)

• crates/cli_bin/tests/snapshots/parse__correct_variable_ranges_in_snippet_with_multiple_contexts.snap is excluded by !**/*.snap
• crates/cli_bin/tests/snapshots/parse__correct_variable_ranges_multiple_snippets_multiple_contexts.snap is excluded by !**/*.snap
• crates/cli_bin/tests/snapshots/parse__parses_foreign_function.snap is excluded by !**/*.snap
• crates/cli_bin/tests/snapshots/parse__parses_grit_file.snap is excluded by !**/*.snap

Files selected for processing (16)

• crates/cli/src/commands/apply_pattern.rs (1 hunks)
• crates/core/src/analysis.rs (7 hunks)
• crates/core/src/api.rs (2 hunks)
• crates/core/src/ast_node.rs (2 hunks)
• crates/core/src/built_in_functions.rs (2 hunks)
• crates/core/src/pattern_compiler/builder.rs (1 hunks)
• crates/core/src/pattern_compiler/call_compiler.rs (1 hunks)
• crates/core/src/pattern_compiler/not_compiler.rs (3 hunks)
• crates/core/src/problem.rs (2 hunks)
• crates/grit-pattern-matcher/src/analysis.rs (2 hunks)
• crates/grit-pattern-matcher/src/context.rs (1 hunks)
• crates/grit-pattern-matcher/src/pattern/ast_node_pattern.rs (2 hunks)
• crates/grit-pattern-matcher/src/pattern/call.rs (1 hunks)
• crates/grit-pattern-matcher/src/pattern/call_built_in.rs (1 hunks)
• crates/grit-pattern-matcher/src/pattern/iter_pattern.rs (13 hunks)
• crates/wasm-bindings/src/match_pattern.rs (1 hunks)

Additional context used

Learnings (1)

Common learnings

Learnt from: morgante
URL: https://github.com/getgrit/gritql/pull/231

Timestamp: 2024-04-15T05:47:47.743Z
Learning: User morgante prefers not to be asked for documentation.

Additional comments not posted (36)

crates/grit-pattern-matcher/src/analysis.rs (2)

2-3: Update import statements to include StaticDefinitions.

The import statements have been updated to include StaticDefinitions from the context module. This change is necessary for the updated parameter type in the has_rewrite function.

---

11-11: Update parameter type to StaticDefinitions<Q>.

The definitions parameter type has been updated to StaticDefinitions<Q>, which likely improves the structure and access of pattern definitions. Ensure that this change is compatible with the rest of the codebase.

crates/grit-pattern-matcher/src/pattern/ast_node_pattern.rs (2)

5-5: Update import statements to include StaticDefinitions.

The import statements have been updated to include StaticDefinitions from the context module. This change is necessary for the updated parameter type in the children method.

---

17-17: Update parameter type to StaticDefinitions<Q>.

The definitions parameter type in the children method has been updated to StaticDefinitions<Q>, which likely improves the structure and access of pattern definitions. Ensure that this change is compatible with the rest of the codebase.

crates/grit-pattern-matcher/src/pattern/call_built_in.rs (2)

16-18: Add name field to CallBuiltIn struct.

The CallBuiltIn struct has been updated to include a name field, which likely represents the name of the built-in function being called. Ensure that this change is compatible with the rest of the codebase.

---

21-26: Update constructor method to accept name parameter.

The constructor method for CallBuiltIn has been updated to accept a name parameter, which is converted to a String and assigned to the name field. Ensure that this change is compatible with the rest of the codebase.

crates/grit-pattern-matcher/src/pattern/call.rs (1)

48-48: Visibility change approved.

The change to pub(crate) for the index field in the PrCall struct improves encapsulation while allowing greater flexibility within the crate.

crates/core/src/pattern_compiler/not_compiler.rs (2)

7-10: Enhanced pattern iteration approved.

The change to use StaticDefinitions::default() for pattern iteration in NotCompiler improves the accuracy or efficiency of the pattern matching process.

Also applies to: 29-29

---

65-65: Enhanced pattern iteration approved.

The change to use StaticDefinitions::default() for pattern iteration in PrNotCompiler improves the accuracy or efficiency of the pattern matching process.

crates/grit-pattern-matcher/src/context.rs (2)

71-78: New structure StaticDefinitions approved.

The introduction of the StaticDefinitions structure enhances static analysis capabilities and provides a clear structure for managing definitions related to patterns and functions.

---

80-114: Implementation of StaticDefinitions approved.

The constructor method and accessor methods provide necessary functionality for initializing and retrieving definitions. The Default implementation ensures the structure can be initialized with empty slices.

crates/core/src/ast_node.rs (1)

35-37: Verify the usage of the new type StaticDefinitions.

The change from a slice of PatternDefinition to StaticDefinitions is noted. Ensure that the new type is correctly used throughout the codebase.

Verification successful

---

Verify the usage of the new type StaticDefinitions.

The change from a slice of PatternDefinition to StaticDefinitions is noted. The new type StaticDefinitions is used consistently across multiple files in the codebase. Ensure that the new type is correctly used in all relevant places.

• crates/core/src/problem.rs
• crates/core/src/ast_node.rs
• crates/core/src/analysis.rs
• crates/core/src/pattern_compiler/not_compiler.rs
• crates/grit-pattern-matcher/src/context.rs
• crates/grit-pattern-matcher/src/pattern/ast_node_pattern.rs
• crates/grit-pattern-matcher/src/pattern/iter_pattern.rs

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the usage of `StaticDefinitions` in the codebase.

# Test: Search for the usage of `StaticDefinitions` in the codebase. Expect: Correct usage in all relevant places.
rg --type rust 'StaticDefinitions'

Length of output: 3494

crates/core/src/pattern_compiler/call_compiler.rs (1)

103-103: Verify the usage of the new parameter kind.

The addition of the kind parameter is noted. Ensure that the new parameter is correctly used throughout the codebase.

crates/core/src/pattern_compiler/builder.rs (1)

292-294: Verify the usage of the new parameter in CallBuiltIn.

The addition of the formatted string parameter is noted. Ensure that the new parameter is correctly used throughout the codebase.

Verification successful

---

The new parameter in CallBuiltIn is correctly used.

The formatted string parameter is correctly passed in the CallBuiltIn::new instantiation within the builder.rs file.

• crates/core/src/pattern_compiler/builder.rs: Correct usage of the formatted string parameter.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the usage of the new parameter in `CallBuiltIn` in the codebase.

# Test: Search for the usage of the new parameter in `CallBuiltIn` in the codebase. Expect: Correct usage in all relevant places.
rg --type rust 'CallBuiltIn'

Length of output: 4150

---

Script:

#!/bin/bash
# Description: Inspect the instantiation of `CallBuiltIn` and include surrounding lines for context.

# Search for instantiation of `CallBuiltIn` and include 5 lines of context before and after each match.
rg --type rust 'CallBuiltIn::new' -A 5 -B 5

Length of output: 1884

crates/grit-pattern-matcher/src/pattern/iter_pattern.rs (13)

6-6: Import of StaticDefinitions is appropriate.

The import is necessary for the changes introduced in this file.

---

16-16: Change of definitions field type in PatternOrPredicateIterator is appropriate.

The change from &'a [PatternDefinition<Q>] to &'a StaticDefinitions<'a, Q> is consistent with the introduction of the StaticDefinitions struct.

---

33-35: Change of definitions parameter type in from_pattern function is appropriate.

The change from &'a [PatternDefinition<Q>] to &'a StaticDefinitions<Q> is consistent with the introduction of the StaticDefinitions struct.

---

39-41: Change of definitions parameter type in from_predicate function is appropriate.

The change from &'a [PatternDefinition<Q>] to &'a StaticDefinitions<Q> is consistent with the introduction of the StaticDefinitions struct.

---

52-52: Addition of DynamicPattern variant to PatternOrPredicate enum is appropriate.

This change enhances the flexibility of the pattern matching system by enabling it to handle dynamic patterns alongside static ones.

---

56-60: Change of definitions parameter type in children function is appropriate.

The change from &'a [PatternDefinition<Q>] to &'a StaticDefinitions<Q> is consistent with the introduction of the StaticDefinitions struct.

---

68-70: Change of definitions parameter type in iter function is appropriate.

The change from &'a [PatternDefinition<Q>] to &'a StaticDefinitions<Q> is consistent with the introduction of the StaticDefinitions struct.

---

75-85: Change of definitions parameter type in children function is appropriate.

The change from &'a [PatternDefinition<Q>] to &'a StaticDefinitions<Q> is consistent with the introduction of the StaticDefinitions struct.

---

129-142: Change of definitions parameter type in children function is appropriate.

The change from &'a [PatternDefinition<Q>] to &'a StaticDefinitions<Q> is consistent with the introduction of the StaticDefinitions struct.

---

Line range hint 150-154: Change of definitions parameter type in children function is appropriate.

The change from &'a [PatternDefinition<Q>] to &'a StaticDefinitions<Q> is consistent with the introduction of the StaticDefinitions struct.

---

Line range hint 166-178: Change of definitions parameter type in children function is appropriate.

The change from &'a [PatternDefinition<Q>] to &'a StaticDefinitions<Q> is consistent with the introduction of the StaticDefinitions struct.

---

Line range hint 194-205: Change of definitions parameter type in children function is appropriate.

The change from &'a [PatternDefinition<Q>] to &'a StaticDefinitions<Q> is consistent with the introduction of the StaticDefinitions struct.

---

Line range hint 213-297: Change of definitions parameter types in args_children, patterns_children, predicates_children, and children functions is appropriate.

The changes from &'a [PatternDefinition<Q>] to &'a StaticDefinitions<Q> are consistent with the introduction of the StaticDefinitions struct.

crates/core/src/analysis.rs (3)

9-13: Import of StaticDefinitions is appropriate.

The import is necessary for the changes introduced in this file.

---

192-213: Addition of uses_named_function function is appropriate.

This function enhances the ability to analyze patterns by allowing the code to determine the presence of specific function calls.

---

215-220: Addition of uses_ai function is appropriate.

This function adds specialized functionality for analyzing AI-related functions within patterns.

crates/wasm-bindings/src/match_pattern.rs (2)

124-125: Invocation of uses_ai function is appropriate.

This invocation provides additional context regarding AI usage in the pattern analysis process.

---

132-132: Addition of uses_ai field to PatternInfo structure is appropriate.

This field enhances the PatternInfo structure by including information about AI usage.

crates/cli/src/commands/apply_pattern.rs (1)

534-534: LGTM! But verify the correctness of the new method definitions.

The code changes are approved.

However, ensure that the new method definitions correctly retrieves the necessary pattern definitions.

Verification successful

---

The new method definitions correctly retrieves the necessary pattern definitions.

The definitions method in crates/core/src/problem.rs aggregates pattern_definitions, predicate_definitions, and function_definitions into a StaticDefinitions object, ensuring the proper retrieval of pattern definitions.

• crates/core/src/problem.rs:

  pub fn definitions(&self) -> StaticDefinitions<'_, MarzanoQueryContext> {
      StaticDefinitions::new(
          &self.pattern_definitions,
          &self.predicate_definitions,
          &self.function_definitions,
      )
  }

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the correctness of the new method `definitions`.

# Test: Search for the method implementation. Expect: Correct retrieval of pattern definitions.
rg --type rust -A 5 $'fn definitions'

Length of output: 440

crates/core/src/problem.rs (1)

70-76: LGTM! But verify the usage of the new method definitions.

The code changes are approved.

However, ensure that the new method definitions is used correctly throughout the codebase.

Verification successful

---

The new method definitions is used correctly throughout the codebase.

The search results indicate that the method is invoked in various contexts and files without any issues.

• crates/wasm-bindings/src/match_pattern.rs
• crates/core/src/analysis.rs
• crates/core/src/api.rs
• crates/cli/src/commands/apply_pattern.rs

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the usage of the new method `definitions`.

# Test: Search for the method usage. Expect: Correct usage of the new method.
rg --type rust -A 5 $'definitions()'

Length of output: 78169

crates/core/src/api.rs (2)

301-301: Addition of uses_ai field looks good.

The new field uses_ai of type bool is added to the PatternInfo struct to indicate AI usage.

---

315-316: Initialization of uses_ai field looks good.

The uses_ai field is initialized by calling crate::analysis::uses_ai with compiled.pattern and compiled.definitions().

Ensure that the uses_ai function is reliable and performs as expected.

Verification successful

---

Initialization of uses_ai field looks good.

The uses_ai field is initialized by calling crate::analysis::uses_ai with compiled.pattern and compiled.definitions(). The uses_ai function is well-defined and relies on the uses_named_function to check for the presence of the "llm_chat" function. Test cases validate the behavior of uses_named_function, indirectly ensuring the correctness of uses_ai.

• crates/core/src/analysis.rs: Implementation of uses_ai and uses_named_function
• Test cases for uses_named_function found in crates/core/src/analysis.rs

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the correctness of the `uses_ai` function.

# Test: Search for the `uses_ai` function definition and its usage. Expect: The function should be defined and used correctly.
rg --type rust 'fn uses_ai'

Length of output: 73

---

Script:

#!/bin/bash
# Description: Inspect the implementation of the `uses_ai` function.

# Extract the implementation of the `uses_ai` function.
ast-grep --lang rust --pattern 'fn uses_ai($_, $_) { $$$ }'

# Search for test cases related to the `uses_ai` function.
rg --type rust 'uses_ai' --context 5

Length of output: 3585

---

Script:

#!/bin/bash
# Description: Inspect the implementation of the `uses_named_function` function and search for related test cases.

# Extract the implementation of the `uses_named_function` function.
ast-grep --lang rust --pattern 'fn uses_named_function($_, $_, $_) { $$$ }'

# Search for test cases related to the `uses_ai` or `uses_named_function` functions.
rg --type rust 'uses_ai|uses_named_function' --context 5

Length of output: 5528

---

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

• X
• Mastodon
• Reddit
• LinkedIn

Tips

Chat

There are 3 ways to chat with CodeRabbit:

• Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  • I pushed a fix in commit <commit_id>.
  • Generate unit testing code for this file.
  • Open a follow-up GitHub issue for this discussion.
• Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
  • @coderabbitai generate unit testing code for this file.
  • @coderabbitai modularize this function.
• PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  • @coderabbitai generate interesting stats about this repository and render them as a table.
  • @coderabbitai show all the console.log statements in this repository.
  • @coderabbitai read src/utils.ts and generate unit testing code.
  • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
  • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

• @coderabbitai pause to pause the reviews on a PR.
• @coderabbitai resume to resume the paused reviews.
• @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
• @coderabbitai full review to do a full review from scratch and review all the files again.
• @coderabbitai summary to regenerate the summary of the PR.
• @coderabbitai resolve resolve all the CodeRabbit review comments.
• @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
• @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configuration File (.coderabbit.yaml)

• You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
• Please see the configuration documentation for more information.
• If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

• Visit our Documentation for detailed information on how to use CodeRabbit.
• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[greptile-apps]

LGTM

_{5 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings}