This guide outlines the critical components of GenAI Red Teaming, with actionable insights for cybersecurity professionals, AI/ML engineers, Red Team practitioners, risk managers, adversarial attack researchers, CISOs, architecture teams, and business leaders. The guide emphasizes a holistic approach to Red Teaming in four areas: model evaluation, implementation testing, infrastructure assessment, and runtime behavior analysis.
Link: You are not allowed to view links. Register or Login to view.
Welcome to the frontier where offensive security meets artificial intelligence.
This thread is a living index of the core concepts, tools, research papers, and attack vectors in Adversarial Machine Learning (AML) — the art of abusing, bypassing, or hardening AI systems.
What is Adversarial Machine Learning?
AML focuses on exploiting weaknesses in machine learning models to:
Fool classifiers (e.g., malware labeled as benign)
Poison training data
Steal models or data
Craft inputs that trigger unexpected behavior
Quote:If traditional apps have logic bugs, AI models have decision boundary bugs.
Offensive Techniques
Evasion Attacks – Modify input to cause misclassification (e.g., making malware look benign).
Model Poisoning – Inject malicious data during training to corrupt future predictions.
Model Extraction – Reverse engineer black-box models using API access.
Membership Inference – Identify whether a data point was in the training set.
Prompt Injection (LLMs) – Manipulate instructions and outputs in AI chatbots.
Tools & Frameworks
You are not allowed to view links. Register or Login to view. : NLP adversarial testing
You are not allowed to view links. Register or Login to view. : Evasion & defense methods
You are not allowed to view links. Register or Login to view. : Comprehensive AML testing
You are not allowed to view links. Register or Login to view. : White-box and black-box attacks
You are not allowed to view links. Register or Login to view. : CV attacks on PyTorch/TensorFlow
Must-Read Papers
Explaining and Harnessing Adversarial Examples – You are not allowed to view links. Register or Login to view.
Backdooring Neural Networks – You are not allowed to view links. Register or Login to view.
Adversarial Examples Are Not Bugs, They Are Features – You are not allowed to view links. Register or Login to view.
Universal Adversarial Perturbations – You are not allowed to view links. Register or Login to view.
LLM-Specific Attacks (GPT, Claude, etc.)
Prompt Injection & Jailbreaks
Training Data Leakage
Fine-Tuning Exploits
Prompt Leaking via Reverse Prompting
Let’s build a solid knowledge base for adversarial AI security.
If you're reading a cool paper, building a model-breaking tool, or fuzzing GPT — post it here.
? “Attackers think in graphs. ML models think in probabilities. We think in both.” Open Thread
Deserialization of untrusted data in on-premises Microsoft SharePoint Server allows an unauthorized attacker to execute code over a network. Microsoft is aware that an exploit for CVE-2025-53770 exists in the wild. Microsoft is preparing and fully testing a comprehensive update to address this vulnerability.
CVSS v3
Base Score: 9.8
Vector: CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H
Severity: Critical
Blackbird is a powerful OSINT tool that combines fast username and email searches across more than 600 platforms with free AI-powered profiling. By leveraging community-driven projects like WhatsMyName, it ensures low false positive rates and high-quality results. Features include smart filters, polished PDF/CSV exports, and fully automated analysis — all from a single CLI.
Link: You are not allowed to view links. Register or Login to view.
Darkdump is a OSINT interface for carrying out deep web investgations written in python in which it allows users to enter a search query in which darkdump provides the ability to scrape .onion sites relating to that query to try to extract emails, metadata, keywords, images, social media etc. Darkdump retrieves sites via Ahmia.fi and scrapes those .onion addresses when connected via the tor network.
Link: You are not allowed to view links. Register or Login to view.
MEDUSA is an extensible and modularized framework that automates processes and techniques practiced during the dynamic analysis of Android and iOS Applications.
It can be used to:
Intercept common API calls by using ready made modules
Automate the creation of complex Frida scripts
Inspect the application's memory
Manage and categorise hooks by adding them in to modules
JSMON CLI is a command-line interface for interacting with the jsmon.sh web application. It provides a convenient way to access various features of JSMON directly from your terminal.
Features
Upload URLs for scanning
Rescan previously scanned URLs
Upload and scan files
View scan results
Manage domains
Set up and manage cron jobs for automated scanning
![[Image: GenAI-Red-Teaming-Guide-1.0-Cover-Image-1.png]](https://genai.owasp.org/wp-content/uploads/2025/01/GenAI-Red-Teaming-Guide-1.0-Cover-Image-1.png)
![[Image: ai-demo.png]](https://raw.githubusercontent.com/p1ngul1n0/blackbird/main/docs/.gitbook/assets/ai-demo.png)
![[Image: darkdump_example.png]](https://github.com/josh0xA/darkdump/raw/main/imgs/darkdump_example.png)
![[Image: 1745874179625?e=1749081600&v=beta&t=Cev6...GtMT2jWYUc]](https://media.licdn.com/dms/image/v2/D5622AQEUpdbNSp0tVQ/feedshare-shrink_800/B56ZZ.NdKgGkAg-/0/1745874179625?e=1749081600&v=beta&t=Cev6UealtrYNivFSO6Bthh67jnngN_IuGGtMT2jWYUc)
![[Image: logo.svg]](https://raw.githubusercontent.com/Ch0pin/medusa/master/libraries/logo.svg){kind=logo}
