0°N 0°E · Test fixtures for map software

Break your map before your users do.

Star on GitHub
working…
generating…

Output file

Features

Positions

Working with AI? Paste this context:

Everything wrong with this file, in one block.

generating…

Preview

Package

A run of files in one download — one per format, each broken differently, with a README.md and manifest.json describing every one of them. Hand the folder to an agent and it has the context without opening a file.

no package yet

Problems

3/42

Reference

How to use this23 data types · 9 formats · 72 problems · 47 search quirks

1. Say what the file is

Pick a format9 of them, from GeoJSON to a real shapefile bundle — and a data type for what it holds. A data type swaps the generic columns for a real schema: ADS-B gives you icao24 and baro_altitude, a parcel export gives you APN and LAND_VAL, and the geometry follows suit, so tracks are tracks and parcels tile a block. That is where the bugs live, because that is where your assumptions live.

2. Prove the happy path first

Press Clean and nothing is wrong with the file: a valid export wearing a real schema. That is the control case, and it is worth settling before anything else — a reader that mangles a clean shapefile will fail every broken fixture too, for a reason that has nothing to do with the fixture. Every clean file is checked before you get it, and the panel lists what was measured: coordinates inside the WGS84 domain, rings closed, one schema throughout, no BOM. If this one does not load, the bug is on the reading side.

3. Say what is wrong with it

Tick problems in the grid below, or press Typical to load what that kind of feed actually arrives with. 42 problems apply to anything; 30 more exist only inside a particular data type and appear when you choose it. Chaos sets how much of the file each one touches — low is a needle in a haystack, high is a file that fails immediately.

4. Watch it happen

The plot redraws as you go, so a problem is something you see rather than something you read about: coincident points collapse to one weighted dot, swapped lat/lon flips the cloud into the wrong hemisphere, projected metres throw everything off-world. Toggle Fit and World to tell “wrong place” from “wrong shape”.

5. Add a boundary to get an answer, not a picture

Turn on a boundary and you get a second GeoJSON — the area you would upload to filter by — plus the number your filter should return. Both numbers, in fact: contains and intersects disagree the moment a line straddles the edge, and every feature is tagged with the expected answer. The counts are measured from the finished file, so they stay true even when a problem has dragged features across the world.

6. Take the file, and the context

Download or Copy the fixture. The dark panel holds a written account of what it is — everything wrong with it and what the format silently dropped, or, for a clean file, the checks it passed. Paste that into an agent or an issue so whoever reads the file knows what it is meant to contain.

7. Make it reproducible

Every file comes from its seed: the same seed and settings always produce byte-identical output. Share copies a link carrying the whole configuration, so a bug report can be a URL. Put the seed in your test suite and the fixture rebuilds itself on every run — nothing to commit, nothing to keep in sync.

8. Or take a package

A package is 5, 9 or 18 fixtures in one zip — every format, a spread of data types, each broken differently — with a README.md describing all of them and a manifest.json beside it. Drop the folder into an agent’s working directory and it can test against the notes without opening a file. One seed rebuilds the whole package; every entry links back to the settings for its own file. Switch it to Clean for the same sweep with nothing wrong with any of it — the set to run before the broken one, where every file should load and no feature should go missing.

9. Then test the box above the map

A file is half of it. The other half is what somebody types into the search box — devices in Tokyo and Kyoto, devices in new zealand, devices that were in the Estádio da Luz in Lisbon last week. Switch to the Search terms tab and the catalogue is dealt out 47 at a time, one problem apiece: a misspelled venue, a name that means two different cities, a window that reads two ways. Each one carries the parse it should have been given — the places resolved to real coordinates, the window resolved to instants — so you can assert on the answer rather than squint at it. Clean gives you the control set, same as everywhere else.

Handle the output with care

Generated CSVs contain real formula-injection payloads, not defanged lookalikes — that is the point, but do not double-click one into a spreadsheet. Open it in a text editor, or import it with formulas disabled. The same category includes XSS-shaped and SQL-shaped strings: inert as data, dangerous only if something renders or executes them, which is exactly the property you are testing for. Nothing you generate is uploaded anywhere, and none of it is real data about real places.

Generating a lot of themThe command line — same generator, same bytes, in CI

One file at a time is what the page above is for. A fixture for every data type, or a matrix of every data type in every format, is a loop — so there is a command line, running the same generator from the same seeds. A file you build by clicking and a file you build in CI are the same bytes.

First, once

Not on npm yet, so the commands below run from a clone — the workspace links nullisland for you once it is built. Node 20 or newer, and jq for the loops that read the catalogue.

git clone https://github.com/Between-Collective/nullisland.git
cd nullisland && npm install && npm run build

One clean file

The control case, in whichever container you want to test. It prints the checks it ran before writing, and exits non-zero if any of them failed.

npx nullisland --clean --type cadastral-parcels --format shapefile --count 500

One for every data type — 24 files

Asks the generator what data types exist, then builds a clean fixture of each. The per-type seed matters: filenames are nullisland-<count>-<seed> and carry no data type, so one shared seed would write all 24 to the same filename.

for t in $(npx nullisland --list types --json | jq -r '.dataTypes[].id'); do npx nullisland --clean --type "$t" --count 200 --seed "clean-$t" --out fixtures/clean; done

Every data type, in every format

24 data types across all nine containers, sorted into a directory per format. Around a minute, and roughly 20 MB.

for t in $(npx nullisland --list types --json | jq -r '.dataTypes[].id'); do for f in geojson ndjson csv kml kmz gpx wkt topojson shapefile; do npx nullisland --clean --type "$t" --format "$f" --count 100 --seed "clean-$t" --out "fixtures/clean/$f"; done; done

Or one archive to hand someone

Nine clean fixtures sweeping formats and data types, with the README.md and manifest.json that describe them. Fewer files than the loops above, and it arrives explaining itself.

npx nullisland --package 9 --clean --extract --out fixtures/clean

Search terms — 47 of them, one problem each

The queries your users type, with the parse each one should receive. JSONL, one term per line, so a test can read it straight off disk. Swap --term-format md for a report to hand a reviewer, or --clean for the control set.

npx nullisland --terms 46 --type mobile-location-pings --out fixtures/search

Queries across the feeds you ship

Name the kinds and every query spans exactly those — devices and aircraft and vessels in one sentence. The answer is their union, which is the case where an empty result looks like a correct one.

npx nullisland --terms 40 --types mobile-location-pings,flight-adsb,maritime-ais --quirks many-subjects

Search terms pinned to one place

Everything in and around a place from the gazetteer — --list places has the ids. A quirk needing a name nowhere near it will still reach further, and the expected parse always names the place it actually used.

npx nullisland --terms 40 --near tokyo --term-format jsonl --seed harbor-lantern-drift --stdout | jq -r '.query'

npx nullisland --help lists every option, and --list types|formats|problems|quirks|places --json hands the whole catalogue back as data — every id, the formats and data types each problem applies to, and for the gazetteer every alias and every other place that answers to the same name — so a script can choose without guessing at what will be skipped. Add --json to any run to get the counts, the bounds and the checks instead of prose.

ContributingA way to break a map that isn't here yet

Know a way to break a map that isn’t here?

72 problems is not all of them. If a real file broke your viewer in a way this can’t reproduce yet, open an issue and describe it — or send a pull request. A new problem is one catalogue entry and one transform function.