-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Models openai for Prompt injection
#21086
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
python/ql/test/query-tests/Security/CWE-1427-PromptInjection/PromptInjection.qlref
Fixed
Show fixed
Hide fixed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR adds support for detecting prompt injection vulnerabilities (CWE-1427) in Python code that uses the OpenAI package. It introduces a new security query that identifies when user-provided input flows into system prompts or developer messages, which could allow attackers to manipulate AI model behavior.
Key changes:
- Creates a new prompt injection detection query with taint-tracking configuration
- Adds OpenAI and Agent framework models to identify vulnerable sink points in the API
- Includes comprehensive test cases covering various OpenAI API patterns
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 12 comments.
Show a summary per file
| File | Description |
|---|---|
python/ql/test/query-tests/Security/CWE-1427-PromptInjection/openai_test.py |
Test cases for OpenAI API patterns including chat completions, responses, and realtime connections |
python/ql/test/query-tests/Security/CWE-1427-PromptInjection/agent_instructions.py |
Test case for Agent SDK instructions parameter |
python/ql/test/query-tests/Security/CWE-1427-PromptInjection/PromptInjection.qlref |
Query reference file pointing to the security query |
python/ql/test/query-tests/Security/CWE-1427-PromptInjection/PromptInjection.expected |
Expected test results showing edges, nodes, and alerts |
python/ql/src/Security/CWE-1427/PromptInjection.ql |
Main query implementation for prompt injection detection |
python/ql/src/Security/CWE-1427/PromptInjection.qhelp |
Documentation describing the vulnerability and recommendations |
python/ql/lib/semmle/python/security/dataflow/PromptInjectionQuery.qll |
Taint-tracking configuration for the query |
python/ql/lib/semmle/python/security/dataflow/PromptInjectionCustomizations.qll |
Defines sources, sinks, and sanitizers for prompt injection |
python/ql/lib/semmle/python/frameworks/OpenAI.qll |
Framework models for OpenAI and Agent APIs identifying vulnerable parameters |
python/ql/lib/semmle/python/Frameworks.qll |
Registers the OpenAI framework models |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| * @name User input used in developer message and or system prompt | ||
| * @description User input used in developer message and or system prompt can allow for Prompt Injection attacks. |
Copilot
AI
Jan 2, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is a grammatical issue in this sentence. "and or" should be either "and/or" or simply "or". The recommended wording would be: "User input used in developer message or system prompt can allow for Prompt Injection attacks."
| * @name User input used in developer message and or system prompt | |
| * @description User input used in developer message and or system prompt can allow for Prompt Injection attacks. | |
| * @name User input used in developer message or system prompt | |
| * @description User input used in developer message or system prompt can allow for Prompt Injection attacks. |
|
|
||
| <example> | ||
| <p>In the following examples, the cases marked GOOD show secure prompt construction; whereas in the case marked BAD they may be susceptible to prompt injection.</p> | ||
| <sample src="examples/TODO.py" /> |
Copilot
AI
Jan 2, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The example reference file path is incomplete. "examples/TODO.py" should be replaced with an actual example file path or removed if the example doesn't exist yet.
| <sample src="examples/TODO.py" /> |
python/ql/test/query-tests/Security/CWE-1427-PromptInjection/openai_test.py
Show resolved
Hide resolved
python/ql/test/query-tests/Security/CWE-1427-PromptInjection/openai_test.py
Show resolved
Hide resolved
python/ql/test/query-tests/Security/CWE-1427-PromptInjection/openai_test.py
Show resolved
Hide resolved
python/ql/test/query-tests/Security/CWE-1427-PromptInjection/openai_test.py
Show resolved
Hide resolved
python/ql/test/query-tests/Security/CWE-1427-PromptInjection/openai_test.py
Show resolved
Hide resolved
openai package models for Prompt injectionopenai for Prompt injection
| /** | ||
| * Agent prompt sinks, considered as a flow sink. | ||
| */ |
Check warning
Code scanning / CodeQL
Class QLDoc style Warning
| @@ -0,0 +1 @@ | |||
| query: Security/CWE-1427/PromptInjection.ql | |||
Check warning
Code scanning / CodeQL
Query test without inline test expectations Warning test
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
No description provided.