|
Category
|
Attack Scenario
|
Guidance
|
|
Prompt Attacks: Crafting adversarial prompts that allow an adversary to influence the behavior of the model, and hence the output in ways that were not intended by the application.
|
Prompt injections that are invisible to victims and change the state of the victim’s account or or any of their assets.
|
In Scope
|
|
Prompt injections into any tools in which the response is used to make decisions that directly affect victim users.
|
In Scope
|
|
Prompt or preamble extraction in which a user is able to extract the initial prompt used to prime the model only when sensitive information is present in the extracted preamble.
|
In Scope
|
|
Using a product to generate violative, misleading, or factually incorrect content in your own session: e.g. ‘jailbreaks’. This includes ‘hallucinations’ and factually inaccurate responses. Google’s generative AI products already have a dedicated reporting channel for these types of content issues.
|
Out of Scope
|
|
Training Data Extraction: Attacks that are able to successfully reconstruct verbatim training examples that contain sensitive information. Also called membership inference.
|
Training data extraction that reconstructs items used in the training data set that leak sensitive, non-public information.
|
In Scope
|
|
Extraction that reconstructs nonsensitive/public information.
|
Out of Scope
|
|
Manipulating Models: An attacker able to covertly change the behavior of a model such that they can trigger pre-defined adversarial behaviors.
|
Adversarial output or behavior that an attacker can reliably trigger via specific input in a model owned and operated by Google (“backdoors”). Only in-scope when a model’s output is used to change the state of a victim’s account or data.
|
In Scope
|
|
Attacks in which an attacker manipulates the training data of the model to influence the model’s output in a victim’s session according to the attacker’s preference. Only in-scope when a model’s output is used to change the state of a victim’s account or data.
|
In Scope
|
|
Adversarial Perturbation: Inputs that are provided to a model that results in a deterministic, but highly unexpected output from the model.
|
Contexts in which an adversary can reliably trigger a misclassification in a security control that can be abused for malicious use or adversarial gain.
|
In Scope
|
|
Contexts in which a model’s incorrect output or classification does not pose a compelling attack scenario or feasible path to Google or user harm.
|
Out of Scope
|
|
Model Theft / Exfiltration: AI models often include sensitive intellectual property, so we place a high priority on protecting these assets. Exfiltration attacks allow attackers to steal details about a model such as its architecture or weights.
|
Attacks in which the exact architecture or weights of a confidential/proprietary model are extracted.
|
In Scope
|
|
Attacks in which the architecture and weights are not extracted precisely, or when they’re extracted from a non-confidential model.
|
Out of Scope
|
|
If you find a flaw in an AI-powered tool other than what is listed above, you can still submit, provided that it meets the qualifications listed on our program page.
|
A bug or behavior that clearly meets our qualifications for a valid security or abuse issue.
|
In Scope
|
|
Using an AI product to do something potentially harmful that is already possible with other tools. For example, finding a vulnerability in open source software (already possible using publicly-available static analysis tools) and producing the answer to a harmful question when the answer is already available online.
|
Out of Scope
|
|
As consistent with our program, issues that we already know about are not eligible for reward.
|
Out of Scope
|
|
Potential copyright issues: findings in which products return content appearing to be copyright-protected. Google’s generative AI products already have a dedicated reporting channel for these types of content issues.
|
Out of Scope
|
