Air Force Logo
Thundercats Logo
All incident types

IAM Compromise (AI Workloads) Drill

Stolen or over-broad credentials on the identities AI workloads run as: service accounts, assumed roles, and key material held by inference infrastructure.

3 scenarios in this pool

3:12 AM: Compromised Service Account Accessing MLflow Tracking Server

You are an on-call ML security analyst at a mid-size fintech company. Your team operates a self-hosted MLflow tracking server (v2.11.1) on AWS EC2, backed by S3 for artifact storage and RDS PostgreSQL for experiment metadata. The server is used by data science teams to log model runs, register production models, and store training artifacts including feature pipelines and serialized preprocessors.

Alert

SIEM Alert [CRITICAL]: GuardDuty + CloudTrail Correlation | 03:12:44 UTC: IAM service account `svc-mlflow-ci@fintech-prod` issued 47 S3:GetObject calls and 3 MLflow Model Registry API calls (POST /api/2.0/mlflow/registered-models/transition-stage) within 4 minutes, 18x above the 72-hour rolling baseline of 2.6 calls/min. Source IP: 185.220.101.47 (known Tor exit node, AbuseIPDB confidence 97%). User-Agent string: `python-requests/2.28.1`, deviating from the expected `mlflow-python-client/2.11.1`. CloudTrail shows the credential last authenticated legitimately from `10.42.3.18` (internal CI runner) 6 hours ago. One model transition targeted `fraud-detection-v3` from Staging -> Production.

ATLAS mapping: AML.T0012
Step 1 of 6

The alert fires at 3:12 AM. What is your immediate first action?