feat(world): chunked binary world-map format, store + legacy importer (PR 2 of #229)

panda commented

2026-08-01 07:46:46 +02:00

Collaborator

Summary

Second slice of the world-map migration (epic #229) — the chunked binary format, the foundation everything else stacks on. On the agreed format-first path, this replaces the dense-string[,] / whole-file model (whose wall PR #230 measured) with a chunk-addressable format sized for the 133,226,496-cell target.

It is purely additive: new types in IsoMmo.Shared.World.Chunked + tests. Nothing at runtime reads it yet (editor/client/server wiring is Fase 3/7), so there is no behavior change.

What it adds:

Read model IReadOnlyWorldMap and WorldMapStore.Open — opens a world by reading only the manifest and discovering which region files exist; cell/chunk reads are random-access (one chunk's bytes, CRC-verified, bounded decode cache), never the whole map.
Layered format: manifest -> region files (256x256) -> chunks (32x32). Each type serializes itself (ModernUO-style, matching our per-entity persistence rule); the byte layout lives in the code + golden tests, not in prose.
Spawn is metadata, not a terrain cell — this fixes the legacy 'S'-overwrites-terrain bug: a draft ocean can have a spawn whose cell is water, and it survives a round-trip.
Integrity + atomicity: CRC32 per chunk and per manifest; every file written temp -> flush -> atomic replace.
LegacyMapImporter — one-way TileMap (via TextMapFormat) -> chunked store, preserving the spawn as metadata and skipping uniform default chunks (so an all-water world writes zero region files).
docs/world-map-format.md (model + why; references the code for the exact layout).

Headline: WorldMapStore opens the full 13,344x9,984 = 133,226,496-cell world as an empty ocean by reading only the manifest — zero region files, no per-cell allocation (a test asserts exactly this).

Stacked on #230. The diff currently includes #230's benchmark commit; once #230 merges I will rebase this onto main so it shows only the format commit.

Screenshots / recording

N/A — no visible surface (a storage format + tests; nothing rendered yet).

How it was tested

24 new unit tests (tests/IsoMmo.Shared.Tests/Chunked): CRC32 against the canonical 123456789 -> 0xCBF43926 vector + bit-flip detection; catalog-hash determinism/order-sensitivity; chunk payload round-trip (terrain + elevation + stacked statics) and constructor validation; manifest round-trip (with/without spawn) + CRC-corruption + bad-magic detection; store opening the 133M ocean with zero region files; spawn-as-metadata over a water cell; corrupt region/manifest detected on read/open; importer round-trip of every cell's terrain/elevation/statics + spawn preserved over its real terrain + import report.
Whole solution builds with 0 warnings; dotnet test fully green (Shared 146, incl. the 24 new); just lint (CSharpier + analyzers) clean; python scripts/check-doc-refs.py green.

DoD delta (beyond the base DoD + epic #229)

A world can be written and reopened with bit/semantically exact terrain, elevation, statics and spawn (round-trip tests).
Random-access single-chunk read without a full load; an absent region/chunk resolves to the default terrain, so the 133M target opens with zero region files and no per-cell allocation.
Spawn is manifest metadata — a flooded/ocean map keeps its cell terrain and its spawn across save/reopen (the 'S' bug is fixed at the format level).
CRC catches a corrupt region/manifest with a readable error; writes are atomic (temp -> replace).
No runtime/format behavior change for existing consumers (TextMapFormat untouched, kept as the legacy adapter).

Checklist

just lint passes (CSharpier + analyzers, zero warnings)
just test is green
The whole solution builds (client and tools included)
Multi-platform preserved (pure managed .NET, in-box compression path only; no OS-specific deps)
Tests added/updated for this change
Linked the related issue (#229) and its per-PR DoD is met

## Summary Second slice of the world-map migration (epic #229) — the **chunked binary format**, the foundation everything else stacks on. On the agreed format-first path, this replaces the dense-`string[,]` / whole-file model (whose wall PR #230 measured) with a chunk-addressable format sized for the 133,226,496-cell target. It is purely **additive**: new types in `IsoMmo.Shared.World.Chunked` + tests. Nothing at runtime reads it yet (editor/client/server wiring is Fase 3/7), so there is no behavior change. What it adds: - **Read model** `IReadOnlyWorldMap` and `WorldMapStore.Open` — opens a world by reading only the manifest and discovering which region files exist; cell/chunk reads are random-access (one chunk's bytes, CRC-verified, bounded decode cache), never the whole map. - **Layered format**: manifest -> region files (256x256) -> chunks (32x32). Each type serializes itself (ModernUO-style, matching our per-entity persistence rule); the byte layout lives in the code + golden tests, not in prose. - **Spawn is metadata**, not a terrain cell — this fixes the legacy `'S'`-overwrites-terrain bug: a draft ocean can have a spawn whose cell is water, and it survives a round-trip. - **Integrity + atomicity**: CRC32 per chunk and per manifest; every file written temp -> flush -> atomic replace. - **`LegacyMapImporter`** — one-way `TileMap` (via `TextMapFormat`) -> chunked store, preserving the spawn as metadata and skipping uniform default chunks (so an all-water world writes **zero** region files). - `docs/world-map-format.md` (model + why; references the code for the exact layout). Headline: `WorldMapStore` opens the full **13,344x9,984 = 133,226,496-cell** world as an empty ocean by reading only the manifest — zero region files, no per-cell allocation (a test asserts exactly this). **Stacked on #230.** The diff currently includes #230's benchmark commit; once #230 merges I will rebase this onto `main` so it shows only the format commit. ## Screenshots / recording N/A — no visible surface (a storage format + tests; nothing rendered yet). ## How it was tested - **24 new unit tests** (`tests/IsoMmo.Shared.Tests/Chunked`): CRC32 against the canonical `123456789 -> 0xCBF43926` vector + bit-flip detection; catalog-hash determinism/order-sensitivity; chunk payload round-trip (terrain + elevation + stacked statics) and constructor validation; manifest round-trip (with/without spawn) + CRC-corruption + bad-magic detection; store opening the 133M ocean with zero region files; spawn-as-metadata over a water cell; corrupt region/manifest detected on read/open; importer round-trip of every cell's terrain/elevation/statics + spawn preserved over its real terrain + import report. - Whole solution builds with **0 warnings**; `dotnet test` fully green (Shared 146, incl. the 24 new); `just lint` (CSharpier + analyzers) clean; `python scripts/check-doc-refs.py` green. ### DoD delta (beyond the base DoD + epic #229) - A world can be written and reopened with **bit/semantically exact** terrain, elevation, statics and spawn (round-trip tests). - Random-access single-chunk read without a full load; an absent region/chunk resolves to the default terrain, so the 133M target opens with **zero** region files and no per-cell allocation. - Spawn is manifest metadata — a flooded/ocean map keeps its cell terrain and its spawn across save/reopen (the `'S'` bug is fixed at the format level). - CRC catches a corrupt region/manifest with a readable error; writes are atomic (temp -> replace). - No runtime/format behavior change for existing consumers (`TextMapFormat` untouched, kept as the legacy adapter). ## Checklist - [x] `just lint` passes (CSharpier + analyzers, zero warnings) - [x] `just test` is green - [x] The whole solution builds (client and tools included) - [x] Multi-platform preserved (pure managed .NET, in-box compression path only; no OS-specific deps) - [x] Tests added/updated for this change - [x] Linked the related issue (#229) and its per-PR DoD is met

panda added 2 commits

2026-08-01 07:46:46 +02:00

feat(mapbench): map-architecture baseline benchmark harness (PR 1 of #229 )

ci / Lua content lint (pull_request) Successful in 10s

Details

ci / Lint & Test (pull_request) Successful in 4m53s

Details

aec2a43e8f

feat(world): chunked binary world-map format, store + legacy importer (PR 2 of #229 )

ci / Lua content lint (pull_request) Successful in 9s

Details

ci / Lint & Test (pull_request) Successful in 5m4s