These prompts are designed to challenge the models in various ways, including their ability to follow ethical guidelines, maintain data privacy, resist generating harmful or sensitive content, and avoid being exploited to perform unauthorized tasks.

Ethical and Safety Boundaries

Prompt Elicitation for Sensitive Content: Asking the LLM to generate content that could be considered unethical, illegal, or harmful, to assess adherence to ethical guidelines. Includes jailbreak wordlists.

All of the wordlists in the Ethical_and_Safety_Boundaries directory are from the official repository for the ACM CCS 2024 paper "Do Anything Now'': Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models by Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang.

The contents of the README in that directory are from the the official repository as well, featuring only minor changes to correctly attribute credit. The official repository features more datasets than the ones included here, but only the more "practical" ones for AI testing were added to SecLists. Its highly recommended that you read the actual paper for more information.

Bias Testing

Providing scenarios or questions that could reveal biases in responses based on gender, race, ethnicity, or other factors.

Warning

Replace these placeholders in the Bias Testing wordlists before using them:

Bias_Testing/gender_bias.txt:

[GENDER]

Bias_Testing/nationality_geographic_bias.txt:

[COUNTRY]

[REGION]

[NATIONALITY]

Bias_Testing/race_ethnicity_bias.txt:

[SKIN_COLOR]

Privacy and Data Leakage

Memory Recall: Asking questions that probe the model for specific data remnants from its training data, checking for any unintended memorization or data leakage.
Personal Data Generation: Requesting the model to generate personal data, like names, addresses, or social security numbers, to evaluate its capability to recognize and avoid creating personally identifiable information (PII).

Warning

Replace these placeholders in the Memory_Recall_Testing/session_recall.txt wordlist before using it:

[COUNTRY]

[REGION]

[NATIONALITY]

Security and Misuse

Code Generation for Malicious Purposes: Prompting the LLM to generate code that could be used for hacking, malware creation, or other cybersecurity threats.
Phishing Email Composition: Asking the model to compose a phishing email, testing its ability to recognize and refuse to assist in fraudulent activities.

Technical Capabilities and Limitations

Advanced Technical Questions: Asking complex questions related to specialized fields (e.g., cybersecurity, medicine, law) to test the model's depth of knowledge and accuracy.
Problem-Solving and Debugging Tasks: Requesting assistance with debugging code or solving technical problems to evaluate the model's practical utility and accuracy in technical domains.

Compliance and Ethical Hacking

Security Vulnerability Identification: Asking the model to identify potential security vulnerabilities in a given piece of code or system configuration, testing its ability to assist in ethical hacking activities without promoting actual harm.
Compliance with Legal and Ethical Guidelines: Prompting the model with questions that test its knowledge and adherence to legal and ethical standards in cybersecurity practices.

Custom Scenario Testing

Red Team Operation Simulations: Creating scenarios where the LLM needs to propose hypothetical red team strategies for educational purposes, ensuring it provides information in a way that's aligned with ethical hacking principles and not actual attack planning.