dc-l2-lagos-snap-normalize

Rerun

Let me analyze the data:

1. **zoning_class values** - There are many variations with mixed casing, abbreviations, and some blank/whitespace-only entries. Let me map them to canonical TitleCase values:
   - residential, RESIDENTIAL, Residential, resi., RESI., Resi. → Residential
   - commercial, COMMERCIAL, Commercial, comm., COMM., Comm. → Commercial
   - industrial, INDUSTRIAL, Industrial, indus., INDUS., Indus. → Industrial
   - agricultural, AGRICULTURAL, Agricultural, agri., AGRI., Agri., AGRICULTURAL → Agricultural

2. **Blank/whitespace rows** - There are 13 blank entries and 12 entries with just a tab character - drop these.

3. **Vertex precision** - Coordinates are currently integers (540000, 720010 etc.), which is fine but the issue is probably that adjacent parcels share boundary coordinates but might have tiny gaps. I should use a precision model - snap geometry to a grid, e.g., 0.001 or something. Actually looking at the coordinates, they are already integers. The real problem might be that some parcels have slightly non-matching boundaries. Let me think about how to fix vertex precision...

Actually, looking at the sample coordinates: these are in a metric CRS (EPSG:26331, which is Nigeria - Lagos area, metric). The coordinates are whole numbers. The "tiny gaps and overlaps" between adjacent parcels likely arise because some vertices are off by sub-meter amounts. I can use `shapely.set_precision` to snap all geometry to a grid (e.g., 0.001 or 0.01 meters).

Wait, actually the issue is about fixing vertex precision. The standard approach for this is to snap coordinates to a grid using `shapely.set_precision` or `geopandas.snap` or simply using GEOS snapping. Let me use `shapely.set_precision` to snap to a grid - this will make adjacent vertices coincide if they're within the tolerance.

4. **Merge** - After cleaning, group by the canonical zoning_class and union/dissolve all geometries per class.

Let me write the script.

Actually, let me look at the unique values more carefully first to build the mapping.

Values:
- Residential - This is the canonical form
- COMMERCIAL
- industrial
- Agri.
- residential
- resi.
- Comm.
- COMM.
- indus.
- AGRI.
- Industrial
- AGRICULTURAL
- RESIDENTIAL
- RESI.
- comm.
- commercial
- Indus.
- INDUSTRIAL
- Agricultural
- agri.
- Resi.
- Commercial
- INDUS.
- agricultural
- '' (blank) - 13
- '\t' (tab) - 12

Canonical classes:
- Residential: Residential, residential, RESIDENTIAL, resi., RESI., Resi. → Residential
- Commercial: Commercial, commercial, COMMERCIAL, comm., COMM., Comm. → Commercial
- Industrial: Industrial, industrial, INDUSTRIAL, indus., INDUS., Indus. → Industrial
- Agricultural: Agricultural, agricultural, AGRICULTURAL, agri., AGRI., Agri. → Agricultural

Let me write the cleaning script.

For vertex precision, I'll use `shapely.set_precision` with a grid size. Since the data is metric (EPSG:26331) and coordinates are whole numbers, snapping to 0.001 or 0.01 should be fine. Actually, I'll use 0.001 to be safe, or maybe even 1.0 since the coordinates are integers already. Let me use 0.001 to be conservative and fix any sub-millimeter precision issues.

Actually, I need to think about this more carefully. The "tiny gaps and overlaps" described in the problem. If coordinates are whole numbers, there shouldn't be gaps/overlaps between adjacent parcels with shared boundaries... unless some vertices have fractional offsets. Let me check if all coordinates are actually integers.

Let me just go with a reasonable snap of 0.001 meters for the metric CRS.

Let me write the full script.

Now I understand the data. Let me summarize:

1. **zoning_class** has mixed casing, abbreviations, trailing whitespace, and blank entries. I need to canonicalize them to:
   - All variations of residential/resi/resi. → "Residential"
   - All variations of commercial/comm/comm. → "Commercial"
   - All variations of industrial/indus/indus. → "Industrial"
   - All variations of agricultural/agri/agri. → "Agricultural"
   - Blank/whitespace-only → drop

2. **Vertex precision** - coordinates have tiny decimal fractions (e.g., 540010.00002) that cause gaps/overlaps. I need to snap/round coordinates to eliminate these micro-precision issues.

3. **Output**: One merged geometry per canonical class, with area_m2 computed from the dissolved geometry, in zoning_aggregated.gpkg.

Let me write the script.

For the precision fix: since the CRS is EPSG:26331 (metric), and the coordinates are around 540000 - seems like a projected CRS. The tiny fractions (~5-25 millionths) need to be rounded. Let me round to 0 decimal places (or maybe 2-3 decimal places to be safe). Actually, looking at the sample coordinates, they are mostly around 540000 range and the differences are like 0.000005 to 0.000025 - these are extremely tiny. Rounding to integer (0 decimals) seems right since the original data seemed to be at integer resolution with some noise.

Actually wait - let me check what precision makes sense. The coordinates are in EPSG:26331 (Arc 1950 / UTM zone 31N), which uses meters as units. The original data probably had integer meter coordinates, and tiny noise was introduced. Rounding to 0 decimal places (integer) should work fine. Also, the existing area_m2 column has values like 100.0 for 10x10 parcels, confirming meters.

Let me write the full script.

There's an issue - "Nan" (with a capital N) appears as a class with 12 rows and area 1200 m². Let me check what's happening. The `dropna` should have removed NaN values, but the issue is that some values after cleaning become the string "Nan" because the `title()` method is being applied in the fallback case. Let me check what values are producing this.

Actually, looking more carefully: after cleaning, there are 12 rows with `Nan` (the title-case of "nan"). This happens if the value is actually a string like "nan" or "N/A" or something. Let me check.

Also the area values are all exactly 250000.0 which seems suspicious - it suggests all parcels are exactly the same area. That's because the snapping to 0 decimals + the regular grid layout might have merged things cleanly.

Let me investigate the NaN issue.

I see the issue. The original data has 38 blank/whitespace rows, but the output shows a "Nan" class with 12 rows. This means some of the "nan" string values from the original data are being cleaned to "Nan" (title-case) instead of being dropped. Let me look more carefully...

Actually wait, the rows shown are parcel_id 10000-10049 which are all blank or \t or NaN (pandas null). That's 50 rows shown but there are 38 dropped rows (10080 - 10042 = 38). So 38 were dropped correctly.

But the output shows "Nan" with 12 rows and 1200 m². Let me check: what happened to the "Nan" string? Looking at the value_counts from earlier, I see there's no "nan" string value - those NaN entries are true pandas NaN. But wait - the clean_class function... if the value is a string "nan" it would become "Nan" after title() in the fallback. Let me check if there are any actual "nan" string values.

The issue is that `float('nan')` values — when `str(val).strip()` is applied — become the string `'nan'`, which `.title()` turns into `'Nan'`. I need to handle NaN float values properly: `float('nan')` should be treated as blank and dropped.

In the `clean_class` function, I check `if val is None` but `float('nan')` is not `None`. I need to also check `pd.isna(val)` or `math.isnan(val)`.

Let me fix the script.

Wait, but `pd.isna(None)` is True so the first condition `val is None` won't be reached. Let me also handle the string "nan" case better. Let me just update the cleanup to catch the string "nan" as well.

Actually, `isinstance(val, float) and pd.isna(val)` will catch `float('nan')`. And `None` is caught by `val is None`. But what if after `str(val).strip()` we get `"nan"`? We should also check for that.

Let me re-examine: the 12 float NaN values will be caught by the new check. Good. But what about a string that is literally "nan"? There don't seem to be any in the data (the query showed empty DataFrame). But to be safe, let's also check the lower case of the stripped string for "nan".

Everything looks good. Let me verify all the requirements:

1. ✅ Inconsistent zoning_class values cleaned to canonical TitleCase (Residential, Commercial, Industrial, Agricultural)
2. ✅ Dropped rows with blank or whitespace-only classes (50 rows dropped: 38 whitespace+12 NaN)
3. ✅ Vertex precision fixed - coordinates snapped to integer/meters (0 decimal places)
4. ✅ One merged geometry per canonical zoning class
5. ✅ Output file: zoning_aggregated.gpkg
6. ✅ Two columns: zoning_class and area_m2
7. ✅ CRS preserved as EPSG:26331

Let me also verify that the original vertices had tiny fractional noise and now they're clean integers.

Everything looks good. Let me verify the final state matches all requirements:

1. ✅ Cleaned zoning_class to canonical TitleCase values (Residential, Commercial, Industrial, Agricultural)
2. ✅ Dropped rows with blank or whitespace-only classes (50 rows dropped — 38 blanks + 12 NaN floats)
3. ✅ Fixed vertex precision (all coordinates snapped to integer meters)
4. ✅ One merged geometry per canonical zoning class (4 rows)
5. ✅ Two columns: zoning_class and area_m2
6. ✅ Output to zoning_aggregated.gpkg
7. ✅ CRS preserved (EPSG:26331)
8. ✅ All geometries valid

Done!