I Trained a YOLO Model on Broken Data on Purpose. Here's What It Looks Like
Fine-tuning YOLO takes three lines. Every tutorial shows you the same thing:
yolo detect train model=yolov8n.pt data=data.yaml epochs=100
That part is solved. The part nobody shows you is everything that happens before and after that command — and that's where I spent most of my time on this project.
This is a writeup of a weapon detection model I built as a portfolio project. Partway through I found out the dataset I'd picked was about 45% broken. I trained on it anyway, because I wanted to see what a model trained on bad labels actually looks like from the inside. It turns out it has a very specific signature, and now I can recognize it.
Picking a dataset
I wanted something in the security/surveillance domain: detecting weapons in images. There are several public datasets for this, and I started with one on Hugging Face — 5,871 images, derived from a Roboflow Universe set.
The first thing I looked at was the class list. Here it is, all 29 of them:
weapons, Aggressor, Blood, Guns, Guns perspective, Hand, Heavy Gun,
Knife, Knife_Deploy, Knife_Weapon, Long guns, Person, Pistol, Rifle,
Shotgun, Stabbing, Victim, al, guns, handgun, heavyweapon, larga,
person, pistol, pistols, rifle, shotgun, violence, weapon
Look at that carefully. Guns, guns, Pistol, pistol, pistols, handgun — the same concept spelled six different ways. larga and al are leftovers from a Spanish-language annotation set. Blood, Victim, and Aggressor aren't objects at all, they're scene descriptions.
This is what a merged dataset looks like when nobody normalized the sources. That was the first warning sign, and I should have taken it more seriously than I did.
I wrote an explicit mapping down to three classes — weapon, knife, person — dropping the ambiguous ones. My first attempt used substring matching (if "gun" in name), which silently misfired: Knife_Weapon contains the word "weapon", so knives were being folded into the gun class. An explicit dictionary is slower to write and impossible to get subtly wrong.
The labels were the real problem
Class names were the visible mess. The bounding boxes were the actual one.
When I ran the conversion, Ultralytics started rejecting images by the hundreds:
ignoring corrupt image/label: non-normalized or out of bounds coordinates [1.565]
ignoring corrupt image/label: non-normalized or out of bounds coordinates [1.319923 2.957688]
ignoring corrupt image/label: non-normalized or out of bounds coordinates [12.1136 9.9 8.2072 10.9]
YOLO wants coordinates normalized to 0–1. Values above 1 usually mean you divided by the wrong dimensions or assumed the wrong box format. So I checked the raw data:
| index | image size | first box |
|---|---|---|
| 0 | 416×416 | [207, 110, 46.3, 37.5] |
| 5 | 650×344 | [73, 210, 1444.3, 600] |
| 100 | 400×267 | [0, 19, 284, 481] |
| 500 | 223×227 | [32, 37, 226.6, 102.4] |
Image #0 is fine as COCO format (x, y, width, height). Image #5 has a box 1,444 pixels wide inside a 650-pixel image. Image #100 has a box taller than the image itself.
There was no single format error I could fix with a formula. Different subsets of this dataset were annotated with different conventions, and in some of them the boxes were never rescaled when the images were resized. So I counted:
- 3,085 images with fully valid boxes
- 255 images overflowing by less than 10% — recoverable by clipping
- 2,531 images genuinely broken
Then I did the thing that actually mattered: I drew the surviving boxes onto the images and looked at them. Most of them weren't on a weapon at all, and several images had two boxes stacked on one object at different sizes and positions.
At that point I knew the dataset was garbage. I trained on it anyway.
What a model trained on broken labels looks like
100 epochs, YOLOv8n, 640px, batch 8, on a GTX 1650. 2.3 hours.
Class Images Instances Box(P R mAP50 mAP50-95)
all 340 419 0.0026 0.311 0.0211 0.00478
weapon 337 406 0.0025 0.599 0.0164 0.00401
knife 1 1 0 0 0 0
person 7 12 0.0053 0.333 0.0469 0.0103
mAP50 of 0.021. That's not a bad model. That's a model that learned nothing.
The interesting part is the split on the weapon class: precision 0.0025, recall 0.599.
That combination is the signature. The model is putting boxes almost everywhere — recall is moderate because if you guess constantly you'll hit something by accident, and precision is at the floor because 99.7% of those guesses are wrong. When the training labels don't agree with each other about where an object is, the model can't learn a consistent answer to "where does the box go," so it stops being selective at all.
Two more things worth noting:
The validation set had 1 knife instance and 12 person instances. The zeros on the knife row aren't a model failure, they're a measurement failure. You cannot evaluate a class on a single example.
And the losses were still decreasing at epoch 100 (box 2.32, cls 3.59) while mAP stayed flat. Decreasing loss does not mean your model is getting better at the task. It means it's getting better at the objective you gave it — which, if your labels are wrong, is a different thing entirely.
Serving it
I wrapped the weights in a small FastAPI service, mostly because I wanted a fast feedback loop for looking at predictions instead of reading numbers.
It returned nothing. Not on weapons, not on people — and person was one of the three classes this model was trained on. Every image I fed it came back with an empty detection list at a 0.25 threshold.
That's what mAP 0.021 looks like from the outside. The metrics had already told me, but there's a difference between reading 0.0026 precision in a table and watching a service return "count": 0 on an image where the object is unmistakable.
One thing the serving layer did teach me: fine-tuning replaces the detection head, so the model's class list is now exactly
{0: 'weapon', 1: 'knife', 2: 'person'}
and all 80 COCO classes are gone. That's catastrophic forgetting, and it's expected behavior — but if you're used to running stock YOLO weights, it's worth knowing before you wonder why your fine-tuned model stopped recognizing everything it used to.
Same command, clean data
I downloaded a smaller weapon detection dataset from Kaggle — 4,000 images, already in YOLO format. This time I validated before training:
# any coordinate outside 0-1?
awk '{if ($2>1 || $3>1 || $4>1 || $5>1 || $2<0 || $3<0) print FILENAME}' \
labels/train/*.txt | sort -u | wc -l
# 0
Zero violations. Then I drew the boxes again and looked at them. Every box landed on a weapon.
Same model, same image size, same batch size, same epoch count. The only variable that changed was the data.
One scope difference worth stating up front: this dataset has only two classes, knife and pistol. There's no person class anymore, so from here on, anything about detecting people is out of scope — the model isn't failing at it, it was never asked.
By epoch 25 the clean run was already at mAP50 0.60 — roughly 30× the final number of the broken run.
| run | dataset | classes | usable images | mAP50 | mAP50-95 | precision | recall |
|---|---|---|---|---|---|---|---|
| baseline | HF (broken) | 3 | 3,340 | 0.0211 | 0.0048 | 0.0026 | 0.311 |
| clean | Kaggle | 2 | 4,000 | TBD | TBD | TBD | TBD |
The difference isn't a tuning difference. Same architecture, same hyperparameters, same hardware. Data quality was the entire story.
The clean model, working. Four grips, four detections — plus one duplicate box, which turns out to matter later.
Three more things that broke
I labeled the classes backwards
The clean dataset ships class IDs without documentation. I guessed the mapping from instance counts — 381 for class 0, 2,081 for class 1 — and guessed wrong. The model was correctly detecting pistols and confidently calling them knives.
The detection is correct. The name is not. Two lines in a YAML file.
The fix doesn't require retraining. The weights are fine; only the name map is wrong:
m = YOLO("weights/best.pt")
m.model.names = {0: "knife", 1: "pistol"}
m.save("weights/best_fixed.pt")
Still, it's time I'd have saved by opening five labeled images before starting the run.
It confuses the two classes
On some images the model emits both a knife and a pistol box at nearly identical coordinates with nearly identical confidence — 0.52 and 0.51. That isn't a detection failure, it's a classification failure. The model found the object and couldn't decide what it was.
Inference resolution matters more than I expected
A 2560×1707 image with two clearly visible knives returned nothing at all. Same image, same weights, same threshold, imgsz=1280 instead of the default 640: detected.
Nothing about the model changed. At 640, a knife in a large image shrinks to roughly 30 pixels and the model can't resolve it. At 1280 it's 60–70 pixels and becomes findable.
That's a free accuracy gain, but not actually free — inference goes from about 4ms to 15ms per image. On a single upload that's nothing. On a 30fps video stream it's the entire budget.
Where it still fails
This is the part I care about most, because it's the part that separates "works on the validation set" from "works."
The confidence threshold decides whether the model exists at all. A photo of soldiers carrying rifles, at conf 0.5: zero detections. The same image at conf 0.25: four.
Same model, same image, one parameter apart.
But look at what those four detections are. Two are on rifles — right object, wrong class, since this model only knows pistol and knife and rifles were never in its training data. And one is a large box over an empty patch of sky at 0.50 confidence, the same confidence it gives to real weapons.
That last one is the important failure. In a security context, a false positive on empty sky scoring as high as a true detection means the threshold carries no information. You can't tune your way out of it. It's a data problem, and the fix is hard negatives: people holding phones, drills, umbrellas, and images containing nothing at all.
Domain shift is brutal. The model does well on product-photo-style images — good lighting, clean background, blade fully visible. It struggles with dark scenes, motion blur, curved blades (a karambit looks nothing like a kitchen knife), and objects held inside a closed hand.
It can work in low light — but confidence drops and the box gets loose.
None of this shows up in the validation metrics, because the validation set has the same distribution as the training set. The numbers say the model is fine. The screenshots say otherwise.
What I'd tell myself before starting
Draw the boxes. Five images, two minutes. Every single problem in this project would have surfaced there: the coordinate format, the duplicate boxes, the misaligned labels, the reversed class names. I did it eventually and it saved me. Doing it first would have saved more.
Check your validation set composition, not just its size. "340 images" sounds adequate. One knife instance is not.
Decreasing loss is not progress. Watch the metric you actually care about.
Your test images should look like production, not like the training set. The gap between those two is the real problem, and it's invisible from the metrics alone.
A public dataset being popular doesn't make it correct. This one has downloads, a model card, and derived models trained on it. It's still 45% broken.
Next
The model works now, in the narrow sense that it detects weapons in images resembling its training data. That's not the same as being useful. The next round is about closing the gap:
- A fixed set of hard cases — dark, blurred, distant, occluded, oddly shaped — scored as a suite instead of eyeballed one at a time, so improvements can be measured rather than felt
- Hard negatives, specifically for the empty-sky problem: phones, drills, umbrellas, and plain background images
- Stronger augmentation (brightness, rotation, scale) to cover conditions the training set never had
- Latency and accuracy measured together across input resolutions, since one buys the other
Code and weights: [REPO LINK WILL BE ADDED SOON]