Training Data Poisoning Drill
Corrupted examples introduced into a training or fine-tuning set, so the resulting model carries the attacker’s behavior as a property of its weights.
4 scenarios in this pool
3:12 AM: Anomalous Embedding Drift Detected in Third-Party Fine-Tuning Corpus
You are an ML Security Engineer on-call for a fintech company running a customer-facing RAG chatbot powered by a quarterly fine-tuned LLM. The model is fine-tuned on a corpus assembled from three third-party data vendors via an automated S3-based ingestion pipeline. MLflow tracks all training runs and dataset versions. Tonight's fine-tuning job (run ID: ft-20240914-03) completed at 02:58 AM and is staged for promotion to production in 4 hours.
SIEM ALERT [HIGH], 03:12:07 UTC | Rule: ML_CORPUS_DRIFT_ANOMALY MLflow Model Registry: Dataset fingerprint mismatch on fine-tuning run ft-20240914-03. Vendor feed "MarketSentimentPro-v2" ingested 14,882 new records at 01:34 UTC (expected delta: ~2,100-2,400 records/night). Embedding cosine similarity vs. baseline corpus: 0.61 (threshold: >= 0.85). SHA-256 hash of vendor bundle s3://corpus-ingest/msp-v2/20240914.tar.gz does not match vendor-provided manifest hash (recorded: a3f9d1c..., expected: 7b02ee4...). Training loss curve shows anomalous dip at step 1,840/3,200 (loss: 0.0041, 6.2 sigma below rolling mean).
Your first action at 3:12 AM: the fine-tuned model is staged for production promotion in 4 hours. What do you do immediately?