Fine-tuning SAM 3 to read maps
SAM 3 selected the correct region on 40% of our land-use maps. Fine-tuned on a small set of annotated maps, it selected 80%.
The task
A client was digitizing a large library of land-use maps. Each legend entry had to be traced as a clean region for downstream use, and on a busy map that took a person most of a working day. The goal was to automate most of each selection and leave a person to review and correct it, so the number that mattered was time saved, not a perfect score.
Why SAM 3
SAM 3 segments every instance of a concept from a short text or visual prompt, across a large open vocabulary, which made it the obvious first attempt.
It is trained on natural images, where objects differ in color, texture and depth. Land-use maps are flat line drawings. A region is whatever a thin border encloses, and many lines cross a plot without bounding it.
Baseline: 40%
Out of the box, SAM 3 selected the correct region 40% of the time. The errors were consistent: partial regions, leaks across thin borders into the neighboring area, and two legend areas merged into one.
Changing the prompts didn't fix it, and neither did recoloring the maps from RGB to high-contrast cyan or brightening them. The model had no prior for how these maps encode a region.
Fine-tuning
We fine-tuned SAM 3 on our maps. Two behaviors mattered. The model had to return the full region as one clean shape. And it had to select a plot as a single region even when lines marking planned land use cross it and split it into smaller shapes on paper.
The training set was small: a limited number of maps with hand-made selections, well below the size of a typical research dataset.
Result: 80%
| SAM 3 as it ships | 40% |
|---|---|
| Fine-tuned SAM 3 | 80% |
Fine-tuned, SAM 3 selected the correct region 80% of the time, twice the baseline. A reviewer could check each selection and fix the occasional miss, which cut a full day of tracing down to a couple of hours of review.
Limits
The main limit was data. The project was time-boxed, and a larger, more varied training set is the first thing we would add. For this task, 80% with human review was enough, because the goal was less manual work, not full automation.
Sources
- SAM 3facebookresearch/sam3