fio-l3-vienna-geofabrik-highways
Map (reference ← swipe → agent)
0:00
I'm building a traffic-noise model of the Gürtel corridor for the City of Vienna and need the surrounding network as a single GeoPackage. Can you pull every highway from the current Geofabrik Austria PBF (https://download.geofabrik.de/europe/austria-latest.osm.pbf) that intersects a 500 m buffer around Vienna's Gürtel, defined as the highway ways whose name ends in 'Gürtel'? I also need every public-transport route relation (bus, tram, subway, train, trolleybus, light_rail) that crosses the same buffer, with each route as a single feature. Keep the OSM attributes full and untruncated: name, highway, maxspeed, lanes, surface, and oneway on highways; ref, name, operator, and route on the PT routes. Please write both layers to vienna_network.gpkg, with the highways layer as LineString and the pt_routes layer as MultiLineString, in Austria's standard projected coordinate system.
0:00
System info
{
"adapter": "openrouter",
"model": "deepseek/deepseek-v4-flash",
"base_url": "https://openrouter.ai/api/v1"
}
0:17
Let me break down this task: 1. Download Austria OSM PBF from Geofabrik 2. Find all highway ways whose name ends with 'Gürtel' (Vienna's Gürtel corridor) 3. Create a 500m buffer around those ways 4. Extract all highway ways that intersect that buffer 5. Extract all public-transport route relations (bus, tram, subway, train, trolleybus, light_rail) that cross the buffer 6. Save both layers to vienna_network.gpkg 7. Highways as LineString, PT routes as MultiLineString 8. In Austria's standard projected coordinate system (EPSG:31287 or EPSG:31256 - Austria Lambert? Let me think... Austria's standard projected coordinate system is EPSG:31287 (MGI / Austria Lambert) or EPSG:31256 (MGI / GK M31). Actually, let me check. The standard projected CRS for Austria is EPSG:31287 (MGI / Austria Lambert) or EPSG:31258 (MGI / GK M28), etc. I think EPSG:31287 (Austria Lambert) is the standard one used nowadays.) Wait, let me think about this more carefully. For Vienna, the standard projected coordinate system might be EPSG:31256 (MGI / GK M31) which covers Vienna. Or EPSG:31287 (MGI / Austria Lambert) which is the national projection. I'll use EPSG:31287 as it's the national standard. Actually, let me use EPSG:31287 (MGI / Austria Lambert) which is the standard projected CRS for all of Austria. Let me start by downloading the file and working with it. I'll use osmnx or osmium or geopandas with osmium/pyosmium, or I can use duckdb with spatial extension... Actually, let me think about the best approach. I think using osmnx would be the easiest. But osmnx works with OSMnx graphs and API, not directly with PBF files. Let me use osmium (pyosmium) or better yet, use osmnx with the PBF file. Actually, osmnx can read from PBF files using `osmnx.graph_from_xml` or through `osmnx.geometries_from_polygon` using a local file? No, osmnx primarily uses the Overpass API. Let me use a different approach. I'll use `pyosmium` to parse the PBF file and process it, or use `osmnx` functions. Actually, the simplest approach might be: 1. Use `osmnx.settings.all_one_way = True` and `osmnx.graph_from_place` but that downloads from Overpass. Let me think about using pyosmium to directly process the PBF file. This is the most efficient approach for a large PBF file. Let me write a comprehensive script that: 1. Downloads the PBF file (or checks if it exists) 2. Uses pyosmium to read it 3. Finds Gürtel ways 4. Creates buffers 5. Finds intersecting highways 6. Finds PT route relations 7. Writes to GeoPackage Actually, a more practical approach: use `osmnx` which can load from a local PBF file via `graph_from_xml`? No, that needs XML. Let me try using `pyosmium` to parse the data. But pyosmium is quite low-level. Let me think of alternatives. I could use `osmnx` with the `overpass` URL, but that might be slow for a whole country. Another approach: use `geopandas` with `pyogrio` which can read PBF files. Actually, Fiona/GDAL can read OSM PBF files using the `OSM` driver. Let me check if that's available. Let me try a practical approach: ```python import geopandas as gpd import shapely ``` GDAL has an OSM driver that can read PBF files. But it needs to know what layers to read (points, lines, multipolygons, etc.). Actually, let me try the approach with osmnx first, since it's simpler: 1. Download the PBF 2. Use osmium to extract the relevant data 3. Process with geopandas Let me write a script that downloads the PBF and processes it. Actually, let me think about this more carefully. The PBF file is ~800MB. Downloading might take a while. But it's necessary. Let me write a Python script that: 1. Downloads the PBF 2. Uses pyosmium to read it and find Gürtel ways 3. Creates a buffer 4. Filters all highways that intersect the buffer 5. Finds PT route relations that cross the buffer 6. Writes to GeoPackage Actually, using pyosmium might be complex. Let me try using the geopandas + pyogrio approach with the OSM GDAL driver. With GDAL's OSM driver, I can read layers like 'lines' (ways) and 'multilinestrings' (for relations). But filtering for specific attributes might be tricky. Let me think about a better workflow: 1. Download PBF file 2. Use `osmium` command-line tool (or pyosmium) to extract just the Vienna area or Gürtel area 3. Use pyosmium or GDAL to read the filtered data Actually, let me just use a combination of tools. Let me check what's available. Let me start by writing a script and see what's available. I'll write a Python script that uses: - `osmnx` if available, to load the data - Or `pyosmium` to parse the PBF - `geopandas` for spatial operations - `shapely` for geometry operations Let me first check what's installed.
0:17
Bash
pip list 2>/dev/null | grep -iE "geopandas|pyosmium|osmnx|osmium|shapely|pyproj|...
duckdb 1.5.3 fiona 1.10.1 geopandas 1.1.3 osmium 4.3.1 osmnx 2.1.0 pyogrio 0.12.1 pyproj 3.7.2 shapely 2.1.2
0:18
Bash
python3 -c "import osmnx; print(osmnx.__version__)" 2>&1; python3 -c "import geo...
2.1.0
1.1.3
Traceback (most recent call last):
File "<string>", line 1, in <module>
import pyosmium; print(pyosmium.__version__)
^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'pyosmium'
Traceback (most recent call last):
File "<string>", line 1, in <module>
import osgeo; print('GDAL', osgeo.__version__)
^^^^^^^^^^^^
ModuleNotFoundError: No module named 'osgeo'
0:28
Let me plan the approach: 1. Download the Austria PBF 2. Use osmium or osmnx to extract highways whose name ends in 'Gürtel' within Vienna 3. Buffer those by 500m 4. Find all highways intersecting that buffer 5. Find all PT route relations (bus, tram, subway, train, trolleybus, light_rail) crossing the buffer 6. Write both layers to vienna_network.gpkg in EPSG:31287 (Austria Lambert Conformal Conic) or EPSG:31256 (MGI / Austria GK M31) - the standard projected CRS for Austria. Let me check what's standard. Actually, the standard projected coordinate system for Austria is EPSG:31287 (MGI / Austria Lambert) or EPSG:31256 (MGI / Austria GK M31). EPSG:31287 is the Austria-wide Lambert conformal conic projection. Let me use that. Let me first download the file and start working. Let me use osmnx to get the Gürtel ways, buffer, then get highways and PT routes. But osmnx might be simpler. Let me think about the best approach using the tools available. Actually, let me use a combination: 1. Download the PBF 2. Use osmium to query/filter 3. Use geopandas/shapely for the geometry operations Let me start by downloading the file.
0:28
Bash
cd /work && wget -q -O austria-latest.osm.pbf "https://download.geofabrik.de/eur...
bash: line 1: wget: command not found
0:32
Let me use curl instead.
0:32
Bash
cd /work && curl -sS -o austria-latest.osm.pbf "https://download.geofabrik.de/eu...
bash: line 1: curl: command not found
0:40
Let me check what's available.
0:40
Bash
which python3 && python3 -m ensurepip --version 2>/dev/null; apt list --installe...
/usr/local/bin/python3 pip 26.1.1 libcurl3t64-gnutls/now 8.14.1-2+deb13u3 amd64 [installed,local] libcurl4-openssl-dev/now 8.14.1-2+deb13u3 amd64 [installed,local] libcurl4t64/now 8.14.1-2+deb13u3 amd64 [installed,local]
0:47
curl is not available as a command, but libcurl is installed. Let me try using python's urllib to download the file.
0:47
Bash
python3 -c "
python3 -c "
import urllib.request
url = 'https://download.geofabrik.de/europe/austria-latest.osm.pbf'
print('Downloading...')
urllib.request.urlretrieve(url, '/work/austria-latest.osm.pbf')
print('Done')
" 2>&1
Downloading... Done
1:11
Let me check the file size.
1:11
Bash
ls -lh /work/austria-latest.osm.pbf
-rw-r--r-- 1 agent agent 765M Jun 18 06:13 /work/austria-latest.osm.pbf
1:32
Good, the file is 765 MB. Now I need to:
1. Find all highway ways whose name ends with "Gürtel" in Vienna.
2. Create a 500 m buffer around those ways.
3. Find all highway ways that intersect that buffer.
4. Find all public-transport route relations (bus, tram, subway, train, trolleybus, light_rail) that cross the same buffer.
5. Write both layers to vienna_network.gpkg.
Let me think about the approach. Using osmium is ideal for working with PBF files. Let me use the `osmium` Python library (which is installed as `osmium` version 4.3.1) to parse the PBF.
Actually, let me think about this more carefully. The approach:
1. First, find all "Gürtel" highway ways. These are ways where the `name` tag ends with "Gürtel". We need to identify their IDs.
2. Get the geometries of these ways, create a 500m buffer.
3. Find all highway ways that intersect the buffer.
4. Find all PT route relations that cross the buffer.
Let me write a comprehensive Python script.
Actually, let me use osmnx which might be the easiest approach. But osmnx works with Overpass API, not local PBF files typically. However, since we have the PBF file locally and osmium is available, let me use osmium.
Wait, osmium Python bindings (version 4.3.1) - let me check what's available.
Let me think about a practical approach:
Step 1: Use osmium to filter ways with name ending in "Gürtel" from the PBF. We need to extract their geometries.
Step 2: Buffer them and find intersecting highways.
Step 3: Find PT route relations.
Let me start by writing a script that uses osmium to find the Gürtel ways.
Actually, I think the most efficient approach is:
1. Use osmium with a handler to find all ways with name ending in "Gürtel" - get their node references.
2. Get the geometries of these ways.
3. Buffer by 500m.
4. Use osmium again to find all highways intersecting that buffer.
5. Find PT route relations.
But this is complex. Let me think differently.
Actually, I can use the `osmium` command-line tool and Python module. Let me first check what's available.
Let me write a Python script step by step.
For the buffer, Austria's standard projected coordinate system is EPSG:31287 (MGI / Austria Lambert) or EPSG:31256 (MGI / Austria GK M31) or EPSG:3416? Let me check... The standard for Vienna is often EPSG:31256 (MGI / Austria GK M31 - for Vienna area), or EPSG:31287 (Austria Lambert). Actually, the most common "standard projected coordinate system" for Austria is EPSG:31287 (MGI / Austria Lambert).
Let me just use EPSG:31287 (MGI / Austria Lambert) which is the standard for all of Austria.
Wait, let me check: EPSG:31287 is "MGI / Austria Lambert". EPSG:31256 is "MGI / Austria GK M31" covering the Vienna area. EPSG:31255 is "MGI / Austria GK M28" covering western Austria.
For Vienna specifically, EPSG:31256 (MGI / Austria GK M31) would be the local projection. But "Austria's standard projected coordinate system" likely refers to EPSG:31287 (MGI / Austria Lambert).
Let me just use EPSG:31287.
Now let me write the script. I'll use the osmium Python module.
Actually, looking at this more carefully, the osmium Python module (v4.3.1) - let me check how to use it.
The `osmium` Python module provides bindings to libosmium. I can use `osmium.SimpleHandler` to process the PBF file.
Let me write a comprehensive script.
Actually, let me first test what osmium provides:
```python
import osmium
```
Let me check the API.
Actually, let me just write a script that uses osmium to find the Gürtel ways, then uses shapely to create buffers, then filters again with osmium for intersecting highways and PT routes.
Let me plan the script:
1. Parse the PBF to find all ways with `name` tag ending with "Gürtel" (case-insensitive? Probably case-sensitive since it's a proper name).
2. For each such way, collect its node references, then get the node coordinates to build a LineString geometry.
3. Create a 500m buffer around these ways (union them).
4. Find all highway ways that intersect this buffer.
5. Find all PT route relations (type=route, route=bus|tram|subway|train|trolleybus|light_rail) that have geometry intersecting the buffer.
6. Write to GeoPackage.
For route relations, I need to get the route geometry as a single MultiLineString. A route relation has members (ways). I need to get the geometry of all member ways, merge them into a single geometry.
This is quite complex. Let me write a step-by-step Python script.
Actually, let me use a more practical approach. I'll use osmium to:
1. First pass: find Gürtel ways and their node coordinates.
2. Second pass: find all highways intersecting the buffer.
3. Third pass: find all PT route relations and their member ways.
Let me start coding.
Actually, a simpler approach might be:
1. Use osmium to extract all nodes, then all ways with name ending in "Gürtel" to get their geometries.
2. Buffer and find intersection.
3. Extract all highways intersecting buffer.
4. Extract all PT routes intersecting buffer.
Let me write this step by step.
Let me write a Python script:
```python
import osmium
import geopandas as gpd
import shapely.geometry as geom
import pyproj
import json
from collections import defaultdict
# First, let's find all Gürtel ways
class GuertelHandler(osmium.SimpleHandler):
def __init__(self):
super().__init__()
self.nodes = {}
self.guertel_ways = []
def way(self, w):
if 'name' in w.tags and w.tags['name'].endswith('Gürtel') or w.tags['name'].endswith('Guertel'):
# Store way info
way_nodes = [n.ref for n in w.nodes]
self.guertel_ways.append({
'id': w.id,
'nodes': way_nodes,
'tags': dict(w.tags)
})
def node(self, n):
if n.id in self.all_needed_nodes:
self.nodes[n.id] = (n.location.lon, n.location.lat)
```
Hmm, the issue is that we don't know which nodes are needed before processing ways. Let me do it in two passes: first collect all way node refs, then read those nodes.
Or better, use osmium's index to handle node locations.
Actually, let me use a simpler approach. Let me use `osmium` with the `NodeLocationsForWays` handler:
```python
import osmium
class GuertelHandler(osmium.SimpleHandler):
def __init__(self):
super().__init__()
self.ways = []
def way(self, w):
if 'name' in w.tags and 'Gürtel' in w.tags['name'] and w.tags['name'].endswith('Gürtel'):
# Build geometry from nodes
coords = []
for n in w.nodes:
if n.location.valid():
coords.append((n.location.lon, n.location.lat))
if len(coords) >= 2:
self.ways.append({
'id': w.id,
'geometry': geom.LineString(coords),
'tags': dict(w.tags)
})
# Apply NodeLocationsForWays
handler = GuertelHandler()
with osmium.io.Reader('austria-latest.osm.pbf', osmium.osm.osm_entity_bits.NODE | osmium.osm.osm_entity_bits.WAY) as reader:
# We need node locations for way geometries
loc_handler = osmium.make_node_locations_handler(handler)
osmium.apply(reader, loc_handler)
```
Let me check the osmium API more carefully. The `osmium` Python module's API...
Actually, I recall that `osmium.SimpleHandler` with `NodeLocationsForWays` is the right approach. Let me write the full script.
Let me also think about the route relations. For PT routes, we need to:
1. Find relations where type=route and route is one of bus, tram, subway, train, trolleybus, light_rail.
2. For each such relation, find all member ways (with role "" or similar).
3. Build geometry from those ways.
4. Check if the geometry intersects the buffer.
5. Keep as a single MultiLineString feature.
Let me write the full script now.
Actually, osmium Python module: I need to check the exact API. Let me test some basics first.
1:32
Bash
python3 -c "
python3 -c "
import osmium
print(dir(osmium))
print('---')
# Check if SimpleHandler exists
print(hasattr(osmium, 'SimpleHandler'))
print(hasattr(osmium, 'apply'))
print(hasattr(osmium, 'make_node_locations_handler'))
" 2>&1
['BackReferenceWriter', 'BaseFilter', 'BaseHandler', 'BufferIterator', 'FileProcessor', 'ForwardReferenceWriter', 'IdTracker', 'InvalidLocationError', 'MergeInputReader', 'NodeLocationsForWays', 'OsmFileIterator', 'SimpleHandler', 'SimpleWriter', 'WriteHandler', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', '_osmium', 'apply', 'area', 'back_reference_writer', 'file_processor', 'filter', 'forward_reference_writer', 'geom', 'helper', 'index', 'io', 'make_simple_handler', 'osm', 'osmium', 'simple_handler', 'zip_processors'] --- True True False
1:41
Good. I have osmium available. Let me first get the Gürtel ways. The approach: 1. First pass: Find all highway ways where name ends with 'Gürtel' and get their geometries to build the buffer. 2. Second pass: Get all highways intersecting the buffer. 3. Third pass: Get all PT route relations that cross the buffer. But osmium is a single-pass handler. Let me think about this more carefully. Actually, I can use osmnx which is available. Let me check if osmnx can work with local PBF files. Actually, osmnx works with OSMnx's graph_from_place or similar functions that download from OSM. But I have a local PBF file. Let me think of a better approach. Approach: 1. Use osmium to scan the PBF file and find all ways with `name` ending in 'Gürtel' and `highway` tag. Collect their way IDs. 2. Then use osmium again with NodeLocationsForWays to get the geometries of those ways. 3. Compute the buffer around those geometries. 4. Then find all highways intersecting the buffer. 5. Find all PT route relations crossing the buffer. This is complex but doable with osmium. Let me write a comprehensive Python script. Actually, let me think about a simpler approach using OSMnx or just using osmium to process the file in multiple passes. Let me use osmium with NodeLocationsForWays to get geometries. First, let me find the Gürtel ways (highway ways with name ending in 'Gürtel'). Let me write a script step by step. Actually, I think the most efficient approach is: 1. First pass: use osmium to find all ways with highway tag and name ending in Gürtel. Collect their node IDs. 2. Second pass: with NodeLocationsForWays, get the full geometry of those ways, build a buffer. 3. Third pass: with NodeLocationsForWays, find all highway ways intersecting that buffer. 4. Fourth pass: find all route relations (type=route, route=bus/tram/subway/train/trolleybus/light_rail) that intersect the buffer. Actually, for relations, I need the geometry of the relation, which is the union of its member ways. That's more complex. Let me write a comprehensive script that does: 1. First pass: identify Gürtel ways, collect their node coordinates (with NodeLocationsForWays), build buffer. 2. Second pass: identify all highway ways intersecting buffer, collect their attributes and geometries. 3. Third pass: identify route relations, find their member ways, check if they intersect buffer, collect geometries. This is going to be quite complex. Let me write it step by step. Actually, let me think about using osmnx. osmnx 2.1.0 might be able to work with local files or we can use it to search. Hmm, but osmnx typically uses Overpass API, not local PBF. However, I could use osmnx's `graph_from_polygon` approach if I can extract the buffer polygon first. Let me try a different approach. Let me use osmium to extract the data I need. Let me write the full script.
1:41
1:41
Bash
python3 -c "
python3 -c "
import osmium
# Check what's in osmium.geom
print(dir(osmium.geom))
print('---')
# Check class details
import inspect
for name in ['make_simple_handler', 'SimpleHandler', 'NodeLocationsForWays']:
obj = getattr(osmium, name)
print(f'{name}: {type(obj)}')
" 2>&1
['ALL', 'BACKWARD', 'Coordinates', 'FORWARD', 'GeoJSONFactory', 'UNIQUE', 'WKBFactory', 'WKTFactory', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'direction', 'haversine_distance', 'lonlat_to_mercator', 'mercator_to_lonlat', 'use_nodes'] --- make_simple_handler: <class 'function'> SimpleHandler: <class 'type'> NodeLocationsForWays: <class 'pybind11_builtins.pybind11_type'>
1:54
Let me figure out the approach. I'll use osmium to: 1. First pass: scan all ways with highway tag, find those whose name ends with 'Gürtel', collect their nodes, build their geometries. 2. First pass also: collect all public transport route relations. 3. Find all ways that intersect a 500m buffer around the Gürtel ways. 4. Find all PT route relations that cross that buffer. Actually, let me think about this more carefully. The task: 1. Find highway ways whose name ends in 'Gürtel' - these define the Gürtel corridor in Vienna. 2. Buffer those by 500m. 3. Get ALL highways that intersect that buffer. 4. Get public-transport route relations (bus, tram, subway, train, trolleybus, light_rail) that cross that buffer. 5. Each route relation as a single feature (MultiLineString). 6. Write to vienna_network.gpkg with EPSG code for Austria (MGI / Austria Lambert = EPSG:31287? Actually Austria's standard is EPSG:31287 (MGI Austria Lambert) or EPSG:31285 (MGI / Austria GK East). Let me check.) Actually, the standard projected coordinate system for Austria is EPSG:31287 (MGI / Austria Lambert) for nationwide mapping, or EPSG:31255-31259 for GK zones. But "Austria's standard projected coordinate system" is likely EPSG:31287 (MGI / Austria Lambert) which covers all of Austria. Let me verify. Let me write a comprehensive Python script using osmium to extract the data. My approach: - Use osmium to scan the PBF file - First find all Gürtel ways (name endswith 'Gürtel') - Build their geometries (as LineStrings) - Compute their union and buffer by 500m (using shapely) - Then find all highway ways that intersect that buffer - Also find all PT route relations that cross that buffer Actually, processing a 765MB PBF with osmium in multiple passes is feasible. Let me write a script that does it in stages. Stage 1: Scan all ways to find Gürtel ways (name ends with 'Gürtel') and collect their node references. Stage 2: Resolve node locations for those way nodes to build geometries. Stage 3: Build the buffer. Stage 4: Scan all ways again to find highways intersecting buffer. Stage 5: Scan relations to find PT routes. Stage 6: Build geometries and write to GeoPackage. Actually, osmium can do this more elegantly. Let me use a handler-based approach. Let me write the full script. Actually, a simpler approach: use osmnx to download the data directly? But the Geofabrik PBF is already downloaded. Let me think... osmnx might be easier for this! Let me check if osmnx can work with local PBF files. Actually, osmnx uses the Overpass API typically. But it can work with local data too if we use OSMnx's graph_from_xml or similar. Hmm, but a PBF is not XML. Let me just use osmium with Python. I'll write a comprehensive script. Plan: 1. Use `osmium.SimpleHandler` to iterate through the PBF. 2. First pass: collect all ways where `name` ends with 'Gürtel'. Store their node IDs. 3. Second pass with `NodeLocationsForWays` handler to get node coordinates for those ways. 4. Build geometries of Gürtel ways using shapely. 5. Compute buffer (500m). 6. Third pass: iterate all ways again, find those with highway tag that intersect the buffer. 7. Fourth pass: iterate all relations, find PT route relations that intersect the buffer. 8. Write to GeoPackage. Let me code this step by step. Actually, let me think about a more efficient way. Since we need to do multiple passes, I can use the `FileProcessor` approach or just multiple handler passes. Let me write a single Python script that does everything.
1:54
Bash
python3 -c "
python3 -c "
import pyproj
# Check Austria CRS
from pyproj.database import query_crs_info
# EPSG:31287 - MGI / Austria Lambert
# EPSG:31285 - MGI / Austria GK East
# EPSG:31281 - MGI / Austria GK West
# EPSG:31255 - MGI / Austria GK M31
crs = pyproj.CRS(31287)
print(crs)
print('---')
crs2 = pyproj.CRS(31285)
print(crs2)
" 2>&1
EPSG:31287 --- EPSG:31285
2:53
Let me think about the approach:
1. First, I need to find the Gürtel ways (highways whose name ends with 'Gürtel' in Vienna).
2. Then compute a 500m buffer around them.
3. Find all highways that intersect that buffer.
4. Find all PT route relations (bus, tram, subway, train, trolleybus, light_rail) that intersect the buffer.
5. Write both layers to a GeoPackage.
The PBF is 765MB, so reading it all with osmium is fine. Let me use a multi-pass approach:
**Pass 1**: Find all ways whose `name` tag ends with "Gürtel" and are highways. Get their coordinates (need node locations) and compute the buffer.
**Pass 2**:
- For highways: Find all ways that intersect the buffer.
- For PT routes: Find all relations with `type=route` and `route` in (bus, tram, subway, train, trolleybus, light_rail), collect their member ways, and check if they intersect the buffer.
Let me think about this more carefully.
Actually, the simplest approach might be to use osmnx which has convenient functions for this. But with 765MB PBF, let me think...
Actually, osmnx uses Overpass API typically, not local PBF files. But osmium can read PBF files directly.
Let me use osmium with a handler approach:
**Step 1**: Read the PBF to find all Gürtel ways.
- Parse all ways, look for `highway` tag and `name` ending in "Gürtel"
- Store their node references
**Step 2**: Read the PBF again to get node locations for the Gürtel way nodes, build the linestrings.
**Step 3**: Compute the 500m buffer (in a projected CRS for accurate distances).
**Step 4**: Read the PBF again:
- For all ways with `highway` tag, check if they intersect the buffer
- For all relations with `type=route` and `route` in PT types, check if they intersect the buffer
This is a lot of passes. Let me think of a better approach.
Actually, with osmium I can:
1. First pass: extract all Gürtel way node IDs
2. Second pass: get all node coordinates for those IDs
3. Build the buffer geometry
4. Third pass: extract all highways and PT route relations intersecting the buffer
Let me write this step by step.
Actually, osmium has `NodeLocationsForWays` handler that can store node locations in memory. Let me use that approach.
Let me write the full script:
```python
import osmium
import geopandas as gpd
import pandas as pd
from shapely import wkb, wkt
from shapely.geometry import LineString, MultiLineString, box, Point
from shapely.ops import unary_union
import pyproj
import sys
import json
# --- Configuration ---
PBF_PATH = "/work/austria-latest.osm.pbf"
GPKG_PATH = "/work/vienna_network.gpkg"
BUFFER_METERS = 500
# Austria projected CRS (MGI / Austria Lambert - EPSG:31287)
PROJECTED_CRS = "EPSG:31287"
WGS84_CRS = "EPSG:4326"
project = pyproj.Transformer.from_crs(WGS84_CRS, PROJECTED_CRS, always_xy=True).transform
project_back = pyproj.Transformer.from_crs(PROJECTED_CRS, WGS84_CRS, always_xy=True).transform
# --- STEP 1: Find all Gürtel ways ---
class GuertelWayFinder(osmium.SimpleHandler):
def __init__(self):
super().__init__()
self.guertel_ways = {} # way_id -> {tags, nodes list}
def way(self, w):
if 'highway' in w.tags and 'name' in w.tags:
name = w.tags.get('name')
if name and name.endswith('Gürtel'):
tags = {}
for k, v in w.tags:
tags[k] = v
self.guertel_ways[w.id] = {
'tags': tags,
'nodes': [n.ref for n in w.nodes]
}
finder = GuertelWayFinder()
finder.apply_file(PBF_PATH)
print(f"Found {len(finder.guertel_ways)} Gürtel ways")
# Collect all node IDs from Gürtel ways
guertel_node_ids = set()
for way_data in finder.guertel_ways.values():
guertel_node_ids.update(way_data['nodes'])
print(f"Total Gürtel way nodes: {len(guertel_node_ids)}")
# --- STEP 2: Get node locations ---
class NodeLocationHandler(osmium.SimpleHandler):
def __init__(self, desired_nodes):
super().__init__()
self.desired_nodes = desired_nodes
self.locations = {} # node_id -> (lon, lat)
def node(self, n):
if n.id in self.desired_nodes:
self.locations[n.id] = (n.location.lon, n.location.lat)
node_handler = NodeLocationHandler(guertel_node_ids)
node_handler.apply_file(PBF_PATH)
print(f"Retrieved {len(node_handler.locations)} node locations")
# --- STEP 3: Build Gürtel geometry in projected CRS ---
from shapely.geometry import shape, mapping
import shapely
def build_line_from_nodes(way_data, node_locs):
coords = []
for node_id in way_data['nodes']:
if node_id in node_locs:
coords.append(node_locs[node_id])
if len(coords) < 2:
return None
return LineString(coords)
guertel_lines = []
for wid, wd in finder.guertel_ways.items():
line = build_line_from_nodes(wd, node_handler.locations)
if line:
guertel_lines.append(line)
print(f"Built {len(guertel_lines)} Gürtel linestrings")
# Combine all Gürtel lines and project
guertel_union = unary_union(guertel_lines)
# Project to projected CRS
guertel_projected = shapely.ops.transform(project, guertel_union)
# Buffer in projected units (meters)
guertel_buf_projected = guertel_projected.buffer(BUFFER_METERS)
# Convert back to WGS84
guertel_buf = shapely.ops.transform(project_back, guertel_buf_projected)
print("Buffer computed")
# --- STEP 4: Find all highways intersecting buffer ---
class HighwayFinder(osmium.SimpleHandler):
def __init__(self, buffer_geom):
super().__init__()
self.buffer = buffer_geom
self.highways = [] # list of (linestring, tags)
self.node_locs = {}
self.collecting_nodes = set()
self.interesting_way_ids = set()
def way(self, w):
if 'highway' in w.tags:
tags = dict(w.tags)
self.interesting_way_ids.add(w.id)
# We'll build geometry later, for now just store tags
self.highways.append((w.id, [n.ref for n in w.nodes], tags))
self.collecting_nodes.update(n.ref for n in w.nodes)
def node(self, n):
if n.id in self.collecting_nodes:
self.node_locs[n.id] = (n.location.lon, n.location.lat)
# Hmm, this approach needs all node locations for all highways... that's a lot.
# Better approach: do it in two sub-passes.
```
Actually, let me think about this differently. The PBF is 765MB, so reading it multiple times is okay but slow. Let me be more efficient.
Better approach:
1. Pass 1: Find Gürtel ways, get their node IDs
2. Pass 2: Get node locations for Gürtel ways, compute buffer
3. Pass 3: Find highways (store way IDs and node refs), AND find all nodes for those ways
4. Pass 4: Get node locations for all highway nodes, then build geometries and check intersection
Actually, reading the PBF 4 times might be slow. Let me try a different approach.
Alternative:
1. First, use osmium to get Gürtel ways and their nodes
2. Use osmium with NodeLocationsForWays to build all way geometries in one pass, and check intersections
Actually, `NodeLocationsForWays` is a handler that resolves node locations for ways. Let me check how to use it.
Let me think more carefully. I'll write a comprehensive script that does:
1. First scan for Gürtel ways + collect their node IDs
2. Second scan for Gürtel node coordinates
3. Build buffer
4. Third scan: use NodeLocationsForWays handler to build way geometries, filter by buffer intersection, and also collect relations
Actually, with `osmium.NodeLocationsForWays`, I can use it as a location handler that's applied after reading nodes. Let me look at how to use it.
Actually, I think the best approach is:
1. Use `osmium.SimpleHandler` with `NodeLocationsForWays` as a filter chain to get all ways with their full geometries in one pass. But we need to know the buffer first...
Let me just write a multi-pass approach but combine some steps.
Plan:
- Pass 1: Find Gürtel way IDs and their node IDs
- Pass 2: Get node locations for Gürtel nodes
- Compute buffer
- Pass 3: Find all highway way IDs and their node IDs that might intersect (just collect all highway ways' nodes)
- Pass 4: Get node locations for those nodes, build geometries, check intersection
Actually, for highways there could be hundreds of thousands. Let me think about this more efficiently.
Alternative approach using osmnx + overpass: but we're using a local PBF.
Another approach: use `osmium` tags-filter to extract a smaller file first, then process that.
Let me try a more streamlined approach using the osmium tag-filter and geospatial filtering.
Actually, let me just go ahead and write the multi-pass approach. The PBF is 765MB, reading it 4 times will take a few minutes but should work.
Wait, actually, let me think about whether I can do this in fewer passes.
Pass 1: Find Gürtel ways + their node IDs
Pass 2: Get node locations for those nodes, build buffer
Then for highways:
Pass 3: For ALL ways with highway tag, collect node IDs (store way_id -> node_ids)
Pass 4: Get node locations for those highway nodes, build geometries, check buffer intersection
For PT routes:
I could do this in Pass 3/4 as well - store relation info and member way IDs.
Actually, let me rethink. Let me write a script that does everything in 3 passes:
Pass 1: Find Gürtel ways and their node IDs
Pass 2: Get node locations for Gürtel nodes, build buffer
Pass 3: Use a handler that:
- For ways with highway tag, store way_id -> node_refs, tags
- For relations with type=route and route in PT types, store relation info and member way IDs
- For nodes, store locations if they belong to any way we care about
Then build geometries from the stored data.
Actually, the issue is that for Pass 3, we need to know which node IDs to collect BEFORE we read the nodes. But osmium processes nodes first, then ways, then relations. So we'd need to collect way node IDs first (from ways), then... but by then nodes have already been processed.
Hmm, this is tricky. The standard approach is:
Option A: Use `osmium.NodeLocationsForWays` which stores all node locations and resolves them for ways. But this stores ALL node locations in memory which for a 765MB PBF could be huge.
Option B: Two passes - first collect all needed node IDs from ways, then get their locations.
Let me go with Option B, but optimize:
Pass 3a: Read PBF, collect all highway way IDs and their node refs, collect all PT route relation details and their member way IDs
Pass 3b: From the collected node refs, get their locations
Pass 4: Build geometries and check intersection
Actually, let me see if `NodeLocationsForWays` can handle 765MB PBF. Let me check available RAM.
Let me just write the script and run it. I'll use a 3-pass approach.
Wait, actually I realize there's a cleaner approach using `osmium.FileProcessor` + filters + handlers. Let me try to write a clean solution.
Let me write the full script now.
Here's my plan for the script:
```python
import osmium
import geopandas as gpd
import pandas as pd
from shapely import geometry
from shapely.ops import transform, unary_union
import pyproj
import sys
PBF = "/work/austria-latest.osm.pbf"
OUT = "/work/vienna_network.gpkg"
BUFF_M = 500
CRS_PROJ = "EPSG:31287"
# --- PASS 1: Find Gürtel ways ---
class GuertelFinder(osmium.SimpleHandler):
def __init__(self):
super().__init__()
self.ways = {}
self.node_ids = set()
def way(self, w):
if 'highway' in w.tags and 'name' in w.tags:
name = w.tags['name']
if name.endswith('Gürtel'):
tags = dict((k,v) for k,v in w.tags)
nodes = [n.ref for n in w.nodes]
self.ways[w.id] = {'tags': tags, 'nodes': nodes}
self.node_ids.update(nodes)
gf = GuertelFinder()
gf.apply_file(PBF)
# --- PASS 2: Get Gürtel node locations ---
class NodeGetter(osmium.SimpleHandler):
def __init__(self, wanted):
super().__init__()
self.wanted = wanted
self.locs = {}
def node(self, n):
if n.id in self.wanted:
self.locs[n.id] = (n.location.lon, n.location.lat)
ng = NodeGetter(gf.node_ids)
ng.apply_file(PBF)
# Build Gürtel linestrings
wgs84 = pyproj.CRS("EPSG:4326")
proj = pyproj.CRS(CRS_PROJ)
trans = pyproj.Transformer.from_crs(wgs84, proj, always_xy=True).transform
trans_back = pyproj.Transformer.from_crs(proj, wgs84, always_xy=True).transform
def build_line(node_ids, locs):
pts = [locs[n] for n in node_ids if n in locs]
if len(pts) < 2: return None
return geometry.LineString(pts)
lines = [build_line(wd['nodes'], ng.locs) for wd in gf.ways.values()]
lines = [l for l in lines if l]
guertel_union = unary_union(lines)
# Project, buffer, project back
guertel_proj = transform(trans, guertel_union)
buf_proj = guertel_proj.buffer(BUFF_M)
buf_wgs84 = transform(trans_back, buf_proj)
# --- PASS 3: Collect highways and PT route info ---
class Collector(osmium.SimpleHandler):
def __init__(self, buffer):
super().__init__()
self.buffer = buffer
self.highway_ways = {} # way_id -> {node_ids, tags}
self.all_highway_node_ids = set()
self.route_relations = [] # [{tags, member_way_ids}]
self.route_way_ids = set()
# store which way IDs are part of which PT route relations
self.way_to_route = {} # way_id -> list of relation indices
def way(self, w):
if 'highway' in w.tags:
nodes = [n.ref for n in w.nodes]
tags = dict((k,v) for k,v in w.tags)
self.highway_ways[w.id] = {'nodes': nodes, 'tags': tags}
self.all_highway_node_ids.update(nodes)
def relation(self, r):
if 'type' in r.tags and r.tags['type'] == 'route':
route = r.tags.get('route')
if route in ('bus', 'tram', 'subway', 'train', 'trolleybus', 'light_rail', 'railway'):
tags = dict((k,v) for k,v in r.tags)
member_ways = [m.ref for m in r.members if m.type == 'w']
idx = len(self.route_relations)
self.route_relations.append({'tags': tags, 'member_ways': member_ways})
self.route_way_ids.update(member_ways)
for wid in member_ways:
if wid not in self.way_to_route:
self.way_to_route[wid] = []
self.way_to_route[wid].append(idx)
coll = Collector(buf_wgs84)
coll.apply_file(PBF)
# We need node locations for all highway ways AND all route member ways
all_needed_nodes = coll.all_highway_node_ids | coll.route_way_ids
# Actually we need node locations for route member ways too
# But route member ways are a subset of highway ways? Not necessarily.
# PT routes can include ways that are not tagged with highway.
# So let's get node locations for all route member ways separately.
# Hmm, we need to re-read the PBF to get node locations for route member ways
# Actually, we also need to re-read to get the actual geometries of the route member ways
# because some route member ways might not be highway ways and won't be in highway_ways dict.
# Let me redo: I need to collect ALL ways that are either:
# a) tagged as highway, OR
# b) members of a PT route relation
# Then get their node IDs and locations.
# Let me redo pass 3 more carefully...
```
OK this is getting complex. Let me just write the complete script carefully.
Actually, let me simplify by reading the PBF multiple times. The PBF is 765MB, reading it maybe 4-5 times should take 5-10 minutes total, which should be acceptable.
Let me write a cleaner script:
Pass 1: Find Gürtel ways + their node IDs
Pass 2: Get Gürtel node locations
Pass 3: Find all highway ways + all PT route-related ways, collect their node IDs
Pass 4: Get all those node locations
Pass 5: Build geometries and intersect
Let me combine passes 3 and 4 differently. Actually, let me use `NodeLocationsForWays` which can handle large files if I give it enough memory.
Actually, let me just go ahead and write the script. I'll be pragmatic about it.
2:53
Write
/work/solve.py
content · 360 lines · py
"""
Extract highway network and PT route relations intersecting a 500m buffer
around Vienna's Gürtel (ways whose name ends with 'Gürtel').
Output: vienna_network.gpkg with two layers:
- highways (LineString, EPSG:31287)
- pt_routes (MultiLineString, EPSG:31287)
"""
import osmium
import geopandas as gpd
import pandas as pd
from shapely.geometry import LineString, MultiLineString, shape
from shapely.ops import transform, unary_union
import pyproj
import sys
import json
PBF = "/work/austria-latest.osm.pbf"
OUT = "/work/vienna_network.gpkg"
BUFF_M = 500
CRS_PROJ = "EPSG:31287" # MGI / Austria Lambert
CRS_WGS = "EPSG:4326"
# ---------------------------------------------------------------------------
# PASS 1 – Find all Gürtel ways (highway + name ends with 'Gürtel')
# ---------------------------------------------------------------------------
class GuertelFinder(osmium.SimpleHandler):
def __init__(self):
super().__init__()
self.ways = {} # way_id -> {tags, nodes}
self.node_ids = set()
def way(self, w):
if 'highway' not in w.tags:
return
name = w.tags.get('name')
if name is None or not name.endswith('Gürtel'):
return
tags = {k: v for k, v in w.tags}
nodes = [n.ref for n in w.nodes]
self.ways[w.id] = {'tags': tags, 'nodes': nodes}
self.node_ids.update(nodes)
print("Pass 1: Finding Gürtel ways...")
gf = GuertelFinder()
gf.apply_file(PBF)
print(f" Found {len(gf.ways)} Gürtel ways, {len(gf.node_ids)} node refs")
# ---------------------------------------------------------------------------
# PASS 2 – Get Gürtel node coordinates
# ---------------------------------------------------------------------------
class GuertelNodeGetter(osmium.SimpleHandler):
def __init__(self, wanted):
super().__init__()
self.wanted = wanted
self.locs = {} # node_id -> (lon, lat)
def node(self, n):
if n.id in self.wanted:
self.locs[n.id] = (n.location.lon, n.location.lat)
print("Pass 2: Getting Gürtel node coordinates...")
gng = GuertelNodeGetter(gf.node_ids)
gng.apply_file(PBF)
print(f" Got {len(gng.locs)} node locations")
# ---------------------------------------------------------------------------
# Build Gürtel geometry in WGS84, project, buffer 500 m, project back
# ---------------------------------------------------------------------------
project_fwd = pyproj.Transformer.from_crs(CRS_WGS, CRS_PROJ, always_xy=True).transform
project_rev = pyproj.Transformer.from_crs(CRS_PROJ, CRS_WGS, always_xy=True).transform
def build_line(node_ids, locs):
pts = [locs[n] for n in node_ids if n in locs]
if len(pts) < 2:
return None
return LineString(pts)
guertel_lines = [build_line(wd['nodes'], gng.locs) for wd in gf.ways.values()]
guertel_lines = [l for l in guertel_lines if l is not None]
print(f" Built {len(guertel_lines)} Gürtel linestrings")
guertel_union = unary_union(guertel_lines)
guertel_proj = transform(project_fwd, guertel_union)
buf_proj = guertel_proj.buffer(BUFF_M)
buf_wgs = transform(project_rev, buf_proj)
print(" Buffer computed")
# ---------------------------------------------------------------------------
# PASS 3 – Collect all highway ways + PT route relations + their member ways
# ---------------------------------------------------------------------------
class Collector(osmium.SimpleHandler):
def __init__(self, buffer):
super().__init__()
self.buffer = buffer
# Highway ways: way_id -> {nodes, tags}
self.highway_ways = {}
self.highway_node_ids = set()
# All PT route relations
self.route_relations = [] # list of {tags, member_way_ids}
# Way IDs that are members of PT routes (regardless of being highway)
self.route_member_way_ids = set()
# All ways that are either highway or PT route members
self.all_way_node_ids = set()
# Store the node refs for ALL ways of interest (highway + PT route members)
self.way_nodes = {} # way_id -> [node_refs]
self.way_tags = {} # way_id -> {tags}
# PT route relations that have at least some way in the area
self.pt_route_info = [] # list of dicts
def way(self, w):
tags = {k: v for k, v in w.tags}
nodes = [n.ref for n in w.nodes]
is_highway = 'highway' in tags
# Store all ways that are either highways OR might be needed by PT routes
# We'll filter later, for now store everything to be safe
# Actually, storing ALL ways would be too much. Let's just store highway ways
# and collect PT route member ways separately.
pass
def relation(self, r):
if 'type' not in r.tags or r.tags['type'] != 'route':
return
route = r.tags.get('route')
if route not in ('bus', 'tram', 'subway', 'train', 'trolleybus', 'light_rail', 'railway'):
return
tags = {k: v for k, v in r.tags}
member_way_ids = [m.ref for m in r.members if m.type == 'w']
self.pt_route_info.append({
'tags': tags,
'member_way_ids': member_way_ids,
})
self.route_member_way_ids.update(member_way_ids)
# Hmm, we need to collect both highway ways and PT route member ways.
# But we can't store ALL ways in the PBF - too many.
# Let me restructure: in a single pass, collect:
# 1. All highway way IDs, node refs, tags
# 2. All PT route relations and their member way IDs
# Then in another pass, get node locations for all those way node refs.
# Actually let me redesign. Let me do a first pass to collect way info.
print("Pass 3: Collecting highway ways and PT route relations...")
class Collector2(osmium.SimpleHandler):
def __init__(self):
super().__init__()
# Highway ways
self.hwy_ways = {} # way_id -> {'nodes': [node_ids], 'tags': dict}
self.hwy_node_ids = set()
# PT route relations
self.pt_routes = [] # list of {'tags': {}, 'member_way_ids': [int]}
self.pt_member_way_ids = set()
# All PT route member ways that are NOT already in hwy_ways
# We'll need to get their geometries too
self.extra_pt_ways = {} # way_id -> {'nodes': [node_ids], 'tags': dict}
self.extra_pt_node_ids = set()
def way(self, w):
tags = {k: v for k, v in w.tags}
nodes = [n.ref for n in w.nodes]
if 'highway' in tags:
self.hwy_ways[w.id] = {'nodes': nodes, 'tags': tags}
self.hwy_node_ids.update(nodes)
def relation(self, r):
if 'type' not in r.tags or r.tags['type'] != 'route':
return
route = r.tags.get('route')
if route not in ('bus', 'tram', 'subway', 'train', 'trolleybus', 'light_rail', 'railway'):
return
tags = {k: v for k, v in r.tags}
member_way_ids = [m.ref for m in r.members if m.type == 'w']
self.pt_routes.append({
'tags': tags,
'member_way_ids': member_way_ids,
})
self.pt_member_way_ids.update(member_way_ids)
coll = Collector2()
coll.apply_file(PBF)
print(f" Highway ways: {len(coll.hwy_ways)}")
print(f" PT route relations: {len(coll.pt_routes)}")
print(f" PT member way IDs: {len(coll.pt_member_way_ids)}")
# Now find which PT member ways are NOT already captured as highways
missing_pt_way_ids = coll.pt_member_way_ids - set(coll.hwy_ways.keys())
print(f" PT member ways not in highway ways: {len(missing_pt_way_ids)}")
# We need node locations for:
# - All hwy_node_ids (from highway ways)
# - Node IDs from extra PT ways
# For extra PT ways, we need to read the PBF again to get their node refs
if missing_pt_way_ids:
class ExtraWayCollector(osmium.SimpleHandler):
def __init__(self, wanted_way_ids):
super().__init__()
self.wanted = wanted_way_ids
self.ways = {}
self.node_ids = set()
def way(self, w):
if w.id in self.wanted:
nodes = [n.ref for n in w.nodes]
tags = {k: v for k, v in w.tags}
self.ways[w.id] = {'nodes': nodes, 'tags': tags}
self.node_ids.update(nodes)
print("Pass 3b: Collecting extra PT member way geometries...")
ewc = ExtraWayCollector(missing_pt_way_ids)
ewc.apply_file(PBF)
print(f" Found {len(ewc.ways)} extra ways, {len(ewc.node_ids)} node refs")
else:
ewc = None
all_needed_node_ids = coll.hwy_node_ids.copy()
if ewc:
all_needed_node_ids.update(ewc.node_ids)
print(f" Total unique node IDs needed: {len(all_needed_node_ids)}")
# ---------------------------------------------------------------------------
# PASS 4 – Get all needed node coordinates
# ---------------------------------------------------------------------------
class AllNodeGetter(osmium.SimpleHandler):
def __init__(self, wanted):
super().__init__()
self.wanted = wanted
self.locs = {}
def node(self, n):
if n.id in self.wanted:
self.locs[n.id] = (n.location.lon, n.location.lat)
print("Pass 4: Getting all needed node coordinates...")
ang = AllNodeGetter(all_needed_node_ids)
ang.apply_file(PBF)
print(f" Got {len(ang.locs)} node locations")
# ---------------------------------------------------------------------------
# Build highway geometries and filter by buffer intersection
# ---------------------------------------------------------------------------
print("Building highway geometries and filtering by buffer...")
hwy_features = []
for wid, wd in coll.hwy_ways.items():
line = build_line(wd['nodes'], ang.locs)
if line is None or line.is_empty:
continue
# Check intersection with buffer
if not line.intersects(buf_wgs):
continue
# Keep only the requested attributes (plus ID)
tags = wd['tags']
row = {
'osm_id': wid,
'name': tags.get('name'),
'highway': tags.get('highway'),
'maxspeed': tags.get('maxspeed'),
'lanes': tags.get('lanes'),
'surface': tags.get('surface'),
'oneway': tags.get('oneway'),
'geometry': line,
}
hwy_features.append(row)
print(f" Highway features intersecting buffer: {len(hwy_features)}")
# ---------------------------------------------------------------------------
# Build PT route geometries (MultiLineString per route relation)
# ---------------------------------------------------------------------------
print("Building PT route geometries and filtering by buffer...")
# First, build all way geometries we might need
all_way_geoms = {} # way_id -> LineString
# From highway ways
for wid, wd in coll.hwy_ways.items():
line = build_line(wd['nodes'], ang.locs)
if line and not line.is_empty:
all_way_geoms[wid] = line
# From extra PT ways
if ewc:
for wid, wd in ewc.ways.items():
line = build_line(wd['nodes'], ang.locs)
if line and not line.is_empty:
all_way_geoms[wid] = line
print(f" Total way geometries available: {len(all_way_geoms)}")
pt_features = []
for ri, rinfo in enumerate(coll.pt_routes):
tags = rinfo['tags']
member_way_ids = rinfo['member_way_ids']
# Collect geometries of member ways that intersect buffer
route_lines = []
for wid in member_way_ids:
if wid in all_way_geoms:
line = all_way_geoms[wid]
if line.intersects(buf_wgs):
route_lines.append(line)
if not route_lines:
continue
# Combine into a MultiLineString
# If only one line, still make it a MultiLineString for consistency
mls = MultiLineString(route_lines)
row = {
'osm_id': ri,
'ref': tags.get('ref'),
'name': tags.get('name'),
'operator': tags.get('operator'),
'route': tags.get('route'),
'geometry': mls,
}
pt_features.append(row)
print(f" PT route features intersecting buffer: {len(pt_features)}")
# ---------------------------------------------------------------------------
# Write to GeoPackage
# ---------------------------------------------------------------------------
print("Writing to GeoPackage...")
# Highways layer
if hwy_features:
gdf_hwy = gpd.GeoDataFrame(hwy_features, crs=CRS_WGS, geometry='geometry')
# Project to Austrian CRS
gdf_hwy = gdf_hwy.to_crs(CRS_PROJ)
# Ensure LineString type
gdf_hwy = gdf_hwy[gdf_hwy.geometry.type == 'LineString']
gdf_hwy.to_file(OUT, layer='highways', driver='GPKG')
print(f" Written {len(gdf_hwy)} highways to {OUT}")
else:
print(" WARNING: No highway features found!")
# PT routes layer
if pt_features:
gdf_pt = gpd.GeoDataFrame(pt_features, crs=CRS_WGS, geometry='geometry')
gdf_pt = gdf_pt.to_crs(CRS_PROJ)
# Ensure MultiLineString type
# Convert any LineString to MultiLineString
mask = gdf_pt.geometry.type == 'LineString'
if mask.any():
gdf_pt.loc[mask, 'geometry'] = gdf_pt.loc[mask, 'geometry'].apply(
lambda g: MultiLineString([g])
)
gdf_pt = gdf_pt[gdf_pt.geometry.type == 'MultiLineString']
gdf_pt.to_file(OUT, layer='pt_routes', driver='GPKG')
print(f" Written {len(gdf_pt)} PT routes to {OUT}")
else:
print(" WARNING: No PT route features found!")
print("Done!")
Wrote /work/solve.py (13368 chars).
2:59
Let me run the script first to see if it works. It's processing a 765MB PBF file, so it might take some time.
2:59
Bash
cd /work && python3 solve.py 2>&1
<exit_code>-1</exit_code> <error>Command timed out after 600000ms</error>
13:27
The script timed out after 600 seconds. The PBF file is 765 MB and we're doing multiple passes. The main issue is likely that we're doing full-file passes which each take a long time. Let me think about optimizations. Actually, osmium applies to files by reading them sequentially. Each pass reads the entire 765 MB PBF. We're doing: - Pass 1: Find Gürtel ways (full scan) - Pass 2: Get Gürtel node coordinates (full scan) - Pass 3: Collect highway ways + PT route relations + PT member ways (full scan) - Pass 3b: Extra PT ways collection (full scan) - Pass 4: All needed node coordinates (full scan) That's 5 full scans of a 765 MB file. This is very slow. We can optimize significantly. Let me think about the approach: 1. Instead of multiple passes, we can combine some. But the issue is that we need node locations - osmium processes nodes first before ways, but the callback order is: nodes, ways, relations (depending on how they are stored in the PBF). Actually, in osmium, `apply_file` processes elements in file order. In PBF files, nodes are typically first, then ways, then relations. So if we try to find Gürtel ways in the first pass and then need their node coordinates, we can't get them in the same pass because nodes come before ways in the file. Wait, actually in OSM PBF files, nodes, ways, and relations are typically stored in that order but not guaranteed. However, osmium's SimpleHandler will call the callbacks in the order the elements appear in the file. Let me think about a better approach: Option 1: Use `osmium.NodeLocationsForWays` which can build node locations on-the-fly. This is a handler that stores node locations and makes them available to subsequent way processing. Option 2: Use osmnx which might handle all this more efficiently. Option 3: Use a more efficient approach with fewer passes. Let me try using `osmium.NodeLocationsForWays` which can handle the node-to-location resolution automatically. This would let us get way geometries in a single pass. Actually, looking at osmium docs more carefully: `osmium.NodeLocationsForWays` is a handler that stores node locations and makes them available as way nodes have their location set automatically when passed to the handler. So we can chain handlers. But the issue is we need to: 1. Find Gürtel ways first (to know which ways to focus on) 2. Then filter highways and PT routes Let me think about using osmnx instead. osmnx can download data from OSM and work with it directly. But we have the local PBF file - osmnx can't directly work with PBF files. Alternative: Use `osmium` with a two-pass approach but optimize: Pass 1: Find all ways (highway ways and PT route member ways) that we might need, along with their node IDs. Also collect PT route relations. Pass 2: Get node coordinates for all needed node IDs. This is essentially what I'm doing but the issue is the 5 full scans. Let me see if I can reduce to 2 scans. Actually, looking at my code more carefully: - Pass 1: Find Gürtel ways - needs to scan all ways - Pass 2: Get Gürtel node coords - scans all nodes - Pass 3: Collect highway ways + PT routes + PT member ways - scans all ways and relations - Pass 3b: Extra PT ways - scans all ways again - Pass 4: All needed node coords - scans all nodes We can combine Pass 3 and 3b into a single pass. But the main issue is we have 5 full scans. Let me redesign: Pass 1: Scan ways to find Gürtel ways (those ending in 'Gürtel' with highway tag). Also scan relations to find PT route relations. Also scan ways to find all highway ways (tagged with highway). We can do this in a single pass since osmium calls way() and relation() callbacks. But wait, to find which ways are relevant, we need to know the Gürtel geometry first (to build the buffer). So we need at least: - Pass 1: Find Gürtel ways and their node refs - Pass 2: Get Gürtel node coords, build buffer - Pass 3: Scan all ways again, check which intersect the buffer, and also collect PT route relation info and their member ways - Pass 4: Get node coords for all needed ways That's 4 passes minimum. But we can try to speed up the file reading. Actually, osmium can process files in chunks with `osmium.FileProcessor` or we can use `osmium.apply` with multiple handlers. Let me think about using `osmium.NodeLocationsForWays` handler. Actually, the simplest optimization is to use `osmium.NodeLocationsForWays` handler and process everything in 2 passes: Pass 1: Find Gürtel ways + get their node locations using NodeLocationsForWays - Actually NodeLocationsForWays only works if nodes come before ways in the file - In PBF, nodes typically come before ways - So a handler chain of [NodeLocationsForWays, GuertelWayFinder] should work Pass 2: Find all highway ways intersecting buffer + PT routes + their member ways - Use NodeLocationsForWays + a handler that checks buffer intersection - Build geometries on the fly Let me try this approach. Actually, let me just try to speed things up by combining passes and using a more efficient approach. Let me look at what took so long. The file is 765 MB. Reading it sequentially at ~100 MB/s would take about 7-8 seconds per pass. With 5 passes, about 40 seconds of I/O time. But Python overhead and the handler processing might slow things down significantly. The timeout was 600 seconds (10 minutes) which is a lot. Something must be very slow. Let me check: in Pass 3, I'm collecting ALL highway ways and ALL PT route relations from the entire PBF of Austria. Austria has a LOT of highway ways. Let me see if the issue is memory or processing speed. Actually, the Collector2 handler processes ALL ways in Austria looking for highway tags. That could be millions of ways. Each way callback creates a dict of tags and a list of nodes. For all 2M+ ways in Austria, this consumes a lot of memory and time. Let me think about a smarter approach: 1. Find Gürtel ways first (small subset) 2. Build the buffer 3. In the second pass, only scan ways and check if they intersect the buffer roughly But we can't easily check intersection without knowing the way's geometry, which requires node coords... Let me try using osmnx instead. osmnx can: - Download from OSM within a bounding box/polygon - Or work with OSM data directly But we need to use the local PBF file. osmnx can't read PBF directly. Another approach: Use `osmium` with a handler that uses `NodeLocationsForWays` and processes in a single pass. Let me think again about the two-pass approach: **Pass 1**: Find Gürtel ways (ways with highway tag and name ending 'Gürtel') - We scan all ways. For each way with highway tag and name ending Gürtel, we record its node IDs. - Then we scan all nodes (or have a second pass) to get those node locations. - Build the buffer. **Pass 2**: Find highways and PT routes intersecting buffer - Use NodeLocationsForWays to get way geometries - For each way (with highway tag), check if it intersects the buffer. If yes, collect. - For each relation (with route tag in our list), record it and its member way IDs. - Since NodeLocationsForWays resolves nodes for ways, we can get way geometries. But in Pass 2, we need to know which member ways of PT routes are relevant. We could: - Collect all PT routes (there aren't that many) - For each PT route, check its member ways' geometries (which we now have from NodeLocationsForWays) - Check if any member way intersects the buffer So the approach would be: Pass 1: Find Gürtel ways (simple, no need for node coords yet), extract their node IDs, then scan again for those node coords. Build buffer. Actually, let me try a completely different approach. Use `osmium` to extract just the Vienna area first using a bounding box, then work with that smaller dataset. Vienna is roughly at lat=48.2, lon=16.37. The Gürtel is a ring road around the inner districts. Let me extract a bounding box around Vienna first. Or better yet, let me use `osmium` with a filtered approach. The `osmium` tool (not Python library) has command-line tools that can filter by bounding box. Let me check if `osmium` is installed. Actually let me check what's available:
13:27
Bash
which osmi* 2>/dev/null; pip list 2>/dev/null | grep -i osmi; apt list --install...
osmium 4.3.1
13:39
The script timed out after 600s. The PBF is 765MB and we're scanning it multiple times. Let me think about how to optimize this. The main issues: 1. We're scanning the 765MB PBF multiple times (Pass 1, Pass 2, Pass 3, Pass 3b, Pass 4 = up to 5 passes!) 2. The buffer computation might be slow Let me optimize by: 1. Using a single pass to collect everything needed (Gürtel ways, highway ways, PT routes) 2. Using a second pass for node locations 3. Maybe using `osmium` tool command line utilities for faster processing Or I could use OSMnx which is already installed and might be faster for this use case. Actually, let me think about this differently. OSMnx can easily: 1. Get the Gürtel streets by querying name 2. Get the surrounding network within a buffer 3. Get the PT routes But OSMnx works with Overpass API, not local files. Let me check if we can use the local PBF with OSMnx or if we need to use osmium/pyosmium differently. Actually, let me try a more efficient approach: use osmium to do it in fewer passes, and use the osmium Python bindings more efficiently. Let me rewrite to: 1. First pass: Find Gürtel ways (way IDs, node IDs, tags) 2. Second pass: Get Gürtel node coordinates 3. Build buffer 4. Third pass: In ONE pass, collect highway ways that intersect the buffer (or store all ways and filter later) AND PT route relations AND their member ways' node refs 5. Fourth pass: Get all needed node coordinates 6. Build geometries The issue was probably that Pass 3 stores ALL highway ways (which could be millions) or that the file reading is slow. Let me think about osmium more carefully. The `osmium` tool (command line tool) might not be installed, but the Python `osmium` module is installed. Actually, the timeout at 600s (10 min) is probably due to iterating over the whole 765MB PBF multiple times. Let me try a smarter approach: 1. First, let me just use `osmium` command-line tools to filter, since they're much faster than Python. 2. Or, I can use OSMnx with the local file via a custom approach. Wait, let me check if osmnx can work with local files. Actually, looking at the osmium Python module more carefully, I should use `osmium.FileProcessor` which is faster. Also, the issue might be that `apply()` on the full file just takes a long time in Python. Let me try a different approach: use the `osmium` command-line tool (if available) to extract just Vienna, then work with that smaller dataset. Let me check what's available.
13:39
Bash
dpkg -l | grep -i osmi 2>/dev/null; which osmium 2>/dev/null; which osmose 2>/de...
(no output)
13:48
The osmium Python package is installed but the osmium command-line tool isn't. The Python-based approach is very slow for a 765MB PBF because it reads the file 4 times. Let me think about a more efficient approach. Actually, the issue is that reading the entire PBF multiple times from Python is slow. I could: 1. Use `osmium` command-line tool if available, but it's not. 2. Use `osmnx` which can filter by bounding box, but we need to define the buffer first. 3. Use a more efficient approach: maybe read the PBF once with all the filtering logic in one handler. Let me rethink. The slowest part is probably reading the PBF 4 times. Let me try to combine passes. Actually, the timeout could also be because the script is just too slow due to Python overhead. Let me think about alternatives. Option 1: Use osmnx to download the data directly from Overpass API. But the user specifically asked to use the Geofabrik PBF. Option 2: Use osmium to filter to just Vienna area, then process. Option 3: Use a single pass through the PBF with the handler doing everything. Let me try using osmnx's graph_from_place or similar, or use a spatial filter with osmium. Actually, osmium's SimpleHandler.apply_file can be slow. But maybe we can use pyosmium (the lower-level C++ bindings) more efficiently, or use a separate tool. Let me think about this differently. The Python osmium bindings read the PBF in one pass, so we can combine all our handlers into one. Let me rewrite: Pass 1 (one read): - Find Gürtel ways, collect their node IDs - After the pass, build Gürtel geometry and buffer - Can't do this because we need to know the buffer before filtering other features Alternative: Two passes: 1. First pass: Find Gürtel ways, get their node locations, build buffer 2. Second pass: Filter all highways and PT relations by the buffer That's 2 reads instead of 4. Let me try that. But even 2 reads of a 765MB file might be slow with Python. Let me check if there are faster alternatives. Actually, maybe I can use osmnx's `graph_from_point` or `graph_from_address` to get the Gürtel network, but the user specifically wants the Geofabrik file. Another option: Use osmium's `NodeLocationsForWays` handler which caches node locations, so in a single pass we can: 1. Collect Gürtel way node refs 2. After the pass, reconstruct the Gürtel geometry, buffer it 3. But we'd need to pass through the file again to collect intersecting features Hmm. Let me try a different approach: use the pyosmium with NodeLocationsForWays to store node locations, and do everything in two passes. Actually, let me try with a smaller test first to see if the script logic works, then optimize. Wait, actually maybe the issue is that the first pass (finding Gürtel ways) took forever because the Python PBF reader is slow when iterating over the entire file. Let me check if we can use the command-line osmium tool. Let me install osmium-tool.
13:48
Bash
apt-get install -y osmium-tool 2>&1 | tail -5
E: Could not open lock file /var/lib/dpkg/lock-frontend - open (13: Permission denied) E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), are you root?
13:58
The osmium Python library is iterating over the full PBF which is huge. Each pass touches every object in the file. Let me check sizes. The PBF is 765M. Each pass through the whole file could be slow. But 600 seconds should be enough... Let me check if the process is still running or what happened. Actually it timed out. Let me look at what's happening differently. Maybe I can use a more targeted approach. Let me use `osmium` command-line tool instead, or use a filter to only extract relevant data. But I don't have the CLI tool installed... Let me try using OSMnx instead which might be more efficient for this specific query? Or I could use pyosmium with more efficient filtering. Actually, maybe the issue is that the Python osmium library's `apply_file` is very slow with large files when using SimpleHandler. Each pass reads the entire file. Let me try a more efficient approach: use one pass only, collecting everything at once. Actually wait - the issue might be that the first pass takes too long. Let me check if there's a simpler way. Let me try using OSMnx first to get the Gürtel buffer and then use osmium filtering. Actually, let me think about this differently. The osmium Python library applies_file with SimpleHandler. For a 765MB PBF, reading it once should take maybe 30-60 seconds. But I'm doing 4 passes over the file! That's 4x 60s = 240s minimum, and probably more. Let me combine the passes: - Pass 1: Find Gürtel ways, get their nodes -> Done in one pass - Pass 2: Get Gürtel node coords -> One pass - Pass 3: Collect highway ways + PT route relations + their nodes -> One pass - Pass 4: Get all needed node coords -> One pass That's 4 passes. Can I reduce this? Actually, I can do: 1. Find Gürtel ways and their nodes (pass 1) 2. Get Gürtel node coords (pass 2) 3. Collect highways, PT routes, their member ways, and get all their node coords in ONE pass Wait, but I can't get node coords in the same pass as ways and relations because ways and relations reference nodes, but the nodes might come after the ways in the file. Actually, the osmium library uses `NodeLocationsForWays` handler which can store node locations and use them when processing ways. So I can do it in 2 passes: 1. Find Gürtel ways and their node coords 2. Collect highways + PT routes, using node locations to build geometries and test intersection Let me restructure. But first, let me check if the first pass even completes. Actually, the issue is the timeout of 600s. Let me check if the process was killed or what happened. Exit code -1 means timeout. Let me try running just the first pass to see how long it takes.
13:58
(no output)
Bash
cd /work && timeout 120 python3 -c "
cd /work && timeout 120 python3 -c "
import osmium, time
t0 = time.time()
class Counter(osmium.SimpleHandler):
def __init__(self):
super().__init__()
self.n = 0
def node(self, n):
self.n += 1
c = Counter()
c.apply_file('/work/austria-latest.osm.pbf')
print(f'Nodes: {c.n}, time: {time.time()-t0:.1f}s')
" 2>&1
16:08
The file is 765MB and reading it with osmium is taking extremely long. This is a huge file. Let me check if I can use a faster approach - maybe using osmnx which can download just Vienna, or use osmosis/osmium-tool command line which might be faster. Or I could try to approach this differently. Let me think about this differently. The Geofabrik PBF for all of Austria is huge. Instead of reading the whole file with Python, I could: 1. Use osmium tool (command line) to filter just the Vienna area first, then work with a smaller file 2. Or use osmnx to get Vienna data directly from the API Actually, osmnx can download from the OSM API directly. Let me try that approach - it'll be much faster for a city-scale extraction. Let me think about what we need: 1. Find Gürtel ways - ways whose name ends with "Gürtel" in Vienna 2. Buffer them by 500m 3. Get all highways that intersect that buffer 4. Get all PT route relations that cross that buffer osmnx can: - Get streets by name - Get the graph within a bounding box - etc. But for PT routes, osmnx might not be ideal. Let me think about a hybrid approach. Actually, let me try using osmnx to get the Gürtel geometry first, then use that to filter. Or even better - use osmium-tool (the C++ command-line tool) to filter the PBF to just Vienna, then process the smaller subset. Let me see if osmium-tool is available or can be installed via pip.
16:08
Bash
pip install --user osmium-tool 2>&1 | tail -5
ERROR: Could not find a version that satisfies the requirement osmium-tool (from versions: none) ERROR: No matching distribution found for osmium-tool
16:36
The Python osmium library is very slow for the full Austria PBF (765MB). Let me try a different approach using `osmnx` (which uses OSMnx to download and filter) or using `osm2pgsql` or something. Actually, let me think about this more carefully. The issue is that `osmium` in Python reads the entire PBF, and the full Austria file is huge. The problem is that in the first pass, we're reading all nodes/ways/relations which takes too long. Let me try a different approach: 1. Use `osmnx` to get the Gürtel ways and the buffer area 2. Use `osmnx` to get highways in that area 3. For PT routes, use a more efficient approach Actually, osmnx uses the Overpass API which requires internet connectivity. Let's check if that's available. Wait, actually the timeout was 600 seconds (10 minutes) - that should be enough. But maybe the Python osmium library is just very slow for reading 765MB of data. Let me try with a much more optimized approach. Actually, let me try a completely different strategy. Let me use `osmnx` to query Vienna directly, since osmnx can filter by custom filters and fetch from Overpass. Let me think about what's more practical: 1. OSMnx can fetch by polygon/bbox and custom tags 2. The Overpass API will be faster for this kind of targeted query Let me try osmnx first.
16:36
Bash
cd /work && timeout 60 python3 -c "
cd /work && timeout 60 python3 -c "
import osmnx as ox
# Try to just get the Gürtel using osmnx
print('Gef')
gdf = ox.features_from_place('Vienna, Austria', tags={'highway': True})
print(gdf.columns.tolist()[:10])
" 2>&1
Gef
17:42
The osmium Python library is very slow for full-file reading. Let me try a different approach. Maybe use `osmium` command-line tool or use `geopandas` with `pyogrio` to read the PBF, or use `osmnx` with a smaller bounding box approach. Actually, let me check if there's a way to use osmnx or direct reading with geopandas/pyogrio more efficiently. The PBF is 765 MB which is big but should be manageable. Let me try using pyogrio to read the file directly, or use duckdb's spatial extension. Actually let me try a different approach - use osmnx with a bounding box query for Vienna, or use duckdb to filter the data. Let me check what tools are actually fast enough for this.
17:42
Bash
cd /work && timeout 120 python3 -c "
cd /work && timeout 120 python3 -c "
import pyogrio
# Just test reading speed
t0 = __import__('time').time()
# Read just a few hundred features
layers = pyogrio.list_layers('/work/austria-latest.osm.pbf')
print('Layers:', layers)
print(f'Time: {__import__(\"time\").time()-t0:.1f}s')
" 2>&1
Layers: [['points' 'Point'] ['lines' 'LineString'] ['multilinestrings' 'MultiLineString'] ['multipolygons' 'MultiPolygon'] ['other_relations' 'GeometryCollection']] Time: 0.0s
17:53
Interesting! Pyogrio can read OSM PBF directly as layers! That's much faster. Let me use that approach instead of iterating through the PBF with osmium. Let me think about the approach: 1. Use pyogrio to read the `lines` layer, filter for `highway` ways with `name` ending in 'Gürtel' 2. Buffer those by 500m 3. Read all lines that intersect the buffer (highway ways) 4. Read relations to find PT route relations, then get their member ways But pyogrio might not handle relations well. Let me check what pyogrio can do with the PBF. Actually, pyogrio uses GDAL/OGR's OSM driver which has limitations: - The `lines` layer contains all ways as linestrings (already built geometries!) - The `multilinestrings` layer contains relations that form multilinestrings - The `other_relations` layer has route relations etc. So I can: 1. Read the `lines` layer with pyogrio, query for highway ways with name like '%Gürtel' 2. Buffer those 3. Read all lines intersecting buffer (filter by highway) 4. For PT routes, read `other_relations` and find route relations, then use their member ways Let me check what attributes are available.
17:53
Bash
cd /work && timeout 120 python3 -c "
cd /work && timeout 120 python3 -c "
import pyogrio
# Read a small sample to see structure
df = pyogrio.read_dataframe('/work/austria-latest.osm.pbf', layer='lines', where=\"name LIKE '%Gürtel'\", columns=['osm_id','name','highway'], max_features=20)
print(list(df.columns))
print(df.head(10))
print('---')
print(df.geometry.type.unique())
" 2>&1
['osm_id', 'name', 'highway', 'geometry']
osm_id ... geometry
0 3992920 ... LINESTRING (15.47705 47.03298, 15.4766 47.0327...
1 3993614 ... LINESTRING (15.45692 47.06284, 15.45699 47.062...
2 3997191 ... LINESTRING (15.41307 47.03302, 15.4132 47.03311)
3 4100111 ... LINESTRING (15.41945 47.03287, 15.41946 47.033...
4 4302806 ... LINESTRING (15.42089 47.0325, 15.4209 47.03255...
5 4408068 ... LINESTRING (16.35377 48.23216, 16.35367 48.232...
6 4463859 ... LINESTRING (16.39368 48.18759, 16.3936 48.1876...
7 4583427 ... LINESTRING (16.35486 48.23308, 16.35485 48.233...
8 4583668 ... LINESTRING (16.33792 48.20866, 16.33798 48.208...
9 4583862 ... LINESTRING (16.34997 48.22711, 16.35014 48.227...
[10 rows x 4 columns]
---
<ArrowStringArray>
['LineString']
Length: 1, dtype: str
18:26
Great! pyogrio can read directly from the OSM PBF and supports SQL WHERE clauses. Let me try a much faster approach using pyogrio (which internally uses GDAL/OGR's OSM driver). Actually, pyogrio can read from OSM PBF files using GDAL's OSM driver. Let me check if the SQL syntax works for filtering by name LIKE '%Gürtel' and highway. Let me design a much more efficient approach: 1. Read all lines with name ending in 'Gürtel' to find the Gürtel ways 2. Compute their buffer 3. Read lines intersecting the buffer (for highways) 4. Read relations (PT routes) and their member ways Actually, pyogrio can read lines, but the question is about relations. Let me check what the other_relations layer contains. Let me also try using DuckDB with spatial extension which can read PBF files directly.
18:26
18:26
Bash
cd /work && timeout 120 python3 -c "
cd /work && timeout 120 python3 -c "
import pyogrio
# Check the other_relations layer schema
df = pyogrio.read_dataframe('/work/austria-latest.osm.pbf', layer='other_relations', max_features=5)
print(list(df.columns))
print(df.iloc[0].to_dict())
" 2>&1
['osm_id', 'name', 'type', 'other_tags', 'geometry']
{'osm_id': '3801', 'name': nan, 'type': 'bridge', 'other_tags': '"layer"=>"2"', 'geometry': <GEOMETRYCOLLECTION (LINESTRING (13.744 47.96, 13.746 47.961, 13.749 47.962)...>}
19:36
Hmm, the OSM PBF layer "other_relations" has `other_tags` as a string column. The route relations might be there. Let me check the "multilinestrings" layer too which might have route relations. Actually, the standard OGR OSM driver reading of a PBF has layers: points, lines, multilinestrings, multipolygons, other_relations. Route relations could be in "multilinestrings" or "other_relations". Let me check. Also, I need to find the Gürtel ways first. Let me query them.
19:36
Bash
cd /work && timeout 120 python3 -c "
cd /work && timeout 120 python3 -c "
import pyogrio
# Get Gürtel ways
sql = \"SELECT * FROM lines WHERE name LIKE '%Gürtel'\"
df = pyogrio.read_dataframe('/work/austria-latest.osm.pbf', sql=sql)
print(f'Found {len(df)} ways with Gürtel in name')
print(df[['osm_id','name','highway']].head(20))
print('---')
# Check which ones actually end with Gürtel
mask = df['name'].str.endswith('Gürtel')
print(f'End with Gürtel: {mask.sum()}')
print(df[mask][['osm_id','name','highway']].to_string())
" 2>&1
Found 395 ways with Gürtel in name
osm_id name highway
0 3992920 Sankt-Peter-Gürtel primary
1 3993614 Waltendorfer Gürtel secondary
2 3997191 Weblinger Gürtel primary
3 4100111 Weblinger Gürtel service
4 4302806 Weblinger Gürtel service
5 4408068 Währinger Gürtel primary
6 4463859 Landstraßer Gürtel primary
7 4583427 Döblinger Gürtel residential
8 4583668 Lerchenfelder Gürtel secondary
9 4583862 Währinger Gürtel living_street
10 4787753 Wiedner Gürtel primary
11 4787756 Wiedner Gürtel residential
12 4788847 Wiedner Gürtel primary
13 4788848 Wiedner Gürtel primary_link
14 4986573 Lerchenfelder Gürtel tertiary
15 5010598 Döblinger Gürtel tertiary
16 5095033 Gumpendorfer Gürtel primary
17 5110620 Sankt-Peter-Gürtel primary
18 5118511 Sankt-Peter-Gürtel service
19 5120239 Weblinger Gürtel primary
---
End with Gürtel: 395
osm_id name highway
0 3992920 Sankt-Peter-Gürtel primary
1 3993614 Waltendorfer Gürtel secondary
2 3997191 Weblinger Gürtel primary
3 4100111 Weblinger Gürtel service
4 4302806 Weblinger Gürtel service
5 4408068 Währinger Gürtel primary
6 4463859 Landstraßer Gürtel primary
7 4583427 Döblinger Gürtel residential
8 4583668 Lerchenfelder Gürtel secondary
9 4583862 Währinger Gürtel living_street
10 4787753 Wiedner Gürtel primary
11 4787756 Wiedner Gürtel residential
12 4788847 Wiedner Gürtel primary
13 4788848 Wiedner Gürtel primary_link
14 4986573 Lerchenfelder Gürtel tertiary
15 5010598 Döblinger Gürtel tertiary
16 5095033 Gumpendorfer Gürtel primary
17 5110620 Sankt-Peter-Gürtel primary
18 5118511 Sankt-Peter-Gürtel service
19 5120239 Weblinger Gürtel primary
20 5120240 Weblinger Gürtel primary
21 5120242 Weblinger Gürtel primary
22 5120243 Weblinger Gürtel primary
23 5142423 Weblinger Gürtel service
24 5848368 Wiedner Gürtel primary
25 5848371 Wiedner Gürtel primary
26 5848402 Wiedner Gürtel primary_link
27 5848412 Wiedner Gürtel primary_link
28 8096027 Währinger Gürtel primary
29 8113177 Döblinger Gürtel primary
30 9866259 Hernalser Gürtel residential
31 10881576 Eggenberger Gürtel primary
32 13449449 Landstraßer Gürtel primary
33 18343136 Lerchenfelder Gürtel primary_link
34 23256066 Währinger Gürtel service
35 23311963 Währinger Gürtel primary
36 23311966 Hernalser Gürtel primary
37 23788822 Währinger Gürtel living_street
38 23842012 Lerchenfelder Gürtel secondary
39 24365774 Währinger Gürtel pedestrian
40 24409219 Döblinger Gürtel residential
41 24409220 Döblinger Gürtel primary
42 24412287 Währinger Gürtel primary
43 24813023 Weblinger Gürtel service
44 24813025 Weblinger Gürtel service
45 24813027 Weblinger Gürtel service
46 24813028 Weblinger Gürtel service
47 24813043 Weblinger Gürtel service
48 24867844 Währinger Gürtel service
49 25004367 Weblinger Gürtel service
50 25004483 Weblinger Gürtel primary
51 25004484 Weblinger Gürtel primary
52 25004485 Weblinger Gürtel primary
53 25004486 Weblinger Gürtel primary
54 25191366 Gaudenzdorfer Gürtel residential
55 25510841 Mariahilfer Gürtel tertiary
56 25716043 Weblinger Gürtel primary
57 26322159 Sankt-Peter-Gürtel primary
58 26332665 Lerchenfelder Gürtel primary
59 26477854 Weblinger Gürtel service
60 26477858 Weblinger Gürtel service
61 26738034 Lerchenfelder Gürtel tertiary
62 26738383 Lerchenfelder Gürtel primary
63 26738443 Hernalser Gürtel primary
64 26738933 Mariahilfer Gürtel primary
65 26739444 Mariahilfer Gürtel tertiary
66 26739449 Mariahilfer Gürtel primary
67 26739452 Sechshauser Gürtel primary
68 26739482 Gaudenzdorfer Gürtel primary
69 27621514 Gürtel residential
70 27621539 Gürtel residential
71 27779723 Gürtel residential
72 27950566 Mariahilfer Gürtel primary
73 27950571 Mariahilfer Gürtel primary
74 29076759 Landstraßer Gürtel primary
75 30281919 Döblinger Gürtel primary
76 30282137 Währinger Gürtel primary
77 30282227 Währinger Gürtel primary
78 31275229 Mariahilfer Gürtel primary
79 32787497 Weblinger Gürtel service
80 32787502 Weblinger Gürtel service
81 33071983 Weblinger Gürtel service
82 37519303 Währinger Gürtel residential
83 38050648 Währinger Gürtel pedestrian
84 45362690 Wiedner Gürtel primary
85 45362691 Wiedner Gürtel residential
86 45362773 Wiedner Gürtel primary
87 45732108 Wiedner Gürtel primary
88 45732109 Landstraßer Gürtel primary
89 47379823 Lerchenfelder Gürtel primary
90 47379824 Lerchenfelder Gürtel secondary
91 51017530 Sankt-Peter-Gürtel primary
92 51237226 Weblinger Gürtel service
93 51237230 Weblinger Gürtel service
94 51342841 Sechshauser Gürtel primary_link
95 52296261 Währinger Gürtel residential
96 56693271 Eggenberger Gürtel primary
97 56693272 Eggenberger Gürtel primary
98 69901867 Weblinger Gürtel primary
99 70843155 Lerchenfelder Gürtel secondary
100 73261335 Lerchenfelder Gürtel primary
101 75839314 Währinger Gürtel busway
102 76776648 Weblinger Gürtel service
103 76776649 Weblinger Gürtel service
104 76776650 Weblinger Gürtel service
105 80200559 Landstraßer Gürtel residential
106 80201081 Landstraßer Gürtel residential
107 80202102 Landstraßer Gürtel primary
108 80202364 Landstraßer Gürtel primary_link
109 90376903 Weblinger Gürtel service
110 90664625 Sankt-Peter-Gürtel primary
111 90664647 Liebenauer Gürtel primary
112 90664671 Sankt-Peter-Gürtel primary
113 96058968 Landstraßer Gürtel primary
114 97841317 Waltendorfer Gürtel service
115 97841324 Waltendorfer Gürtel service
116 97841329 Waltendorfer Gürtel service
117 99209608 Hernalser Gürtel residential
118 107234652 Weblinger Gürtel primary
119 107234654 Weblinger Gürtel primary
120 107234658 Weblinger Gürtel primary
121 107234665 Weblinger Gürtel primary
122 108090671 Sankt-Peter-Gürtel service
123 108777214 Eggenberger Gürtel primary
124 110150743 Eggenberger Gürtel primary
125 110150775 Eggenberger Gürtel primary
126 110540217 Weblinger Gürtel service
127 111496281 Weblinger Gürtel service
128 111496283 Weblinger Gürtel service
129 111496286 Weblinger Gürtel service
130 115431153 Weblinger Gürtel primary
131 115431154 Weblinger Gürtel primary
132 120906988 Landstraßer Gürtel primary
133 129889438 Weblinger Gürtel service
134 131491167 Sankt-Peter-Gürtel primary
135 131702533 Waltendorfer Gürtel secondary
136 134122998 Sankt-Peter-Gürtel primary
137 134123000 Sankt-Peter-Gürtel service
138 134123001 Sankt-Peter-Gürtel service
139 135336622 Eggenberger Gürtel primary
140 135336623 Eggenberger Gürtel primary
141 135336627 Eggenberger Gürtel primary
142 135336631 Eggenberger Gürtel primary
143 135336632 Eggenberger Gürtel primary
144 135336636 Eggenberger Gürtel primary
145 135336639 Eggenberger Gürtel primary
146 135336643 Eggenberger Gürtel primary
147 135336646 Eggenberger Gürtel primary
148 135336648 Eggenberger Gürtel primary
149 136512214 Sankt-Peter-Gürtel primary
150 136512215 Sankt-Peter-Gürtel primary
151 136842585 Eggenberger Gürtel primary
152 138554935 Weblinger Gürtel primary
153 138554936 Weblinger Gürtel primary
154 138554937 Weblinger Gürtel primary
155 138554938 Weblinger Gürtel primary
156 140768326 Lerchenfelder Gürtel primary
157 142244827 Wiedner Gürtel primary
158 146985785 Hernalser Gürtel secondary
159 148092692 Lerchenfelder Gürtel residential
160 155488474 Landstraßer Gürtel primary
161 157013290 Wiedner Gürtel primary
162 160782892 Waltendorfer Gürtel secondary
163 161604958 Lerchenfelder Gürtel service
164 163290668 Eggenberger Gürtel primary
165 163982100 Döblinger Gürtel residential
166 164014602 Weblinger Gürtel service
167 164953275 Weblinger Gürtel service
168 164953306 Weblinger Gürtel service
169 172758112 Landstraßer Gürtel footway
170 176620231 Mariahilfer Gürtel tertiary
171 176620234 Mariahilfer Gürtel tertiary
172 179875779 Döblinger Gürtel residential
173 179875780 Döblinger Gürtel residential
174 179875781 Döblinger Gürtel residential
175 179875783 Döblinger Gürtel residential
176 183723746 Gaudenzdorfer Gürtel primary
177 184637567 Währinger Gürtel primary
178 187770190 Weblinger Gürtel primary
179 187770191 Weblinger Gürtel primary
180 187770200 Weblinger Gürtel service
181 187770202 Weblinger Gürtel service
182 187770208 Weblinger Gürtel service
183 187770209 Weblinger Gürtel service
184 187770215 Weblinger Gürtel service
185 187770219 Weblinger Gürtel service
186 187770221 Weblinger Gürtel service
187 187770222 Weblinger Gürtel service
188 188231739 Währinger Gürtel pedestrian
189 191802786 Währinger Gürtel living_street
190 194755313 Liebenauer Gürtel primary
191 194755316 Liebenauer Gürtel primary
192 194755317 Liebenauer Gürtel primary
193 194755318 Liebenauer Gürtel primary
194 194755320 Sankt-Peter-Gürtel primary
195 194755321 Sankt-Peter-Gürtel primary
196 194755323 Sankt-Peter-Gürtel primary
197 194755325 Sankt-Peter-Gürtel primary
198 194755326 Sankt-Peter-Gürtel primary
199 194755327 Sankt-Peter-Gürtel primary
200 194782696 Weblinger Gürtel primary
201 194782697 Weblinger Gürtel primary
202 194782698 Weblinger Gürtel primary
203 194782699 Weblinger Gürtel primary
204 194782700 Weblinger Gürtel primary
205 194818645 Weblinger Gürtel primary
206 194818646 Weblinger Gürtel primary
207 194818653 Weblinger Gürtel service
208 194997778 Wiedner Gürtel service
209 195213163 Wiedner Gürtel primary
210 195638384 Sankt-Peter-Gürtel primary
211 196295288 Sankt-Peter-Gürtel service
212 196295289 Sankt-Peter-Gürtel service
213 197927540 Weblinger Gürtel service
214 197927566 Weblinger Gürtel service
215 197927567 Weblinger Gürtel service
216 200247615 Eggenberger Gürtel primary
217 200247616 Eggenberger Gürtel primary
218 200247617 Eggenberger Gürtel primary
219 200247618 Eggenberger Gürtel primary
220 200247619 Eggenberger Gürtel primary
221 200247620 Eggenberger Gürtel primary
222 200247621 Eggenberger Gürtel primary
223 202213256 Weblinger Gürtel service
224 202213260 Weblinger Gürtel service
225 202213261 Weblinger Gürtel service
226 205999088 Hernalser Gürtel service
227 214792843 Landstraßer Gürtel primary
228 215070841 Landstraßer Gürtel primary
229 217654982 Eggenberger Gürtel primary
230 218017203 Währinger Gürtel service
231 221456199 Eggenberger Gürtel primary
232 221456200 Eggenberger Gürtel primary
233 224715391 Hernalser Gürtel service
234 224715392 Hernalser Gürtel service
235 227698376 Hernalser Gürtel secondary
236 228861213 Landstraßer Gürtel primary
237 228861214 Landstraßer Gürtel primary
238 228861215 Landstraßer Gürtel primary
239 228861783 Wiedner Gürtel primary
240 228861786 Wiedner Gürtel primary
241 228861789 Wiedner Gürtel primary
242 228862000 Wiedner Gürtel primary
243 231385925 Währinger Gürtel primary
244 231385930 Währinger Gürtel primary
245 237206540 Waltendorfer Gürtel secondary
246 237626849 Hernalser Gürtel secondary
247 237976509 Döblinger Gürtel residential
248 239911077 Mariahilfer Gürtel primary
249 241598449 Weblinger Gürtel service
250 245115220 Hernalser Gürtel secondary
251 245251291 Gaudenzdorfer Gürtel primary
252 245251292 Gaudenzdorfer Gürtel primary
253 246114842 Wiedner Gürtel primary_link
254 250864445 Lerchenfelder Gürtel pedestrian
255 269512026 Liebenauer Gürtel primary_link
256 272668393 Sechshauser Gürtel primary
257 272668398 Sechshauser Gürtel primary
258 284586653 Sankt-Peter-Gürtel service
259 284586767 Sankt-Peter-Gürtel service
260 284790191 Liebenauer Gürtel primary_link
261 291241344 Gaudenzdorfer Gürtel residential
262 304166628 Weblinger Gürtel service
263 315358866 Währinger Gürtel service
264 317461423 Währinger Gürtel primary
265 324276028 Wiedner Gürtel primary_link
266 324297226 Lerchenfelder Gürtel primary
267 324549972 Währinger Gürtel primary
268 335528227 Sankt-Peter-Gürtel service
269 337654522 Währinger Gürtel primary
270 363738922 Wiedner Gürtel primary
271 376410648 Hernalser Gürtel secondary
272 378265326 Währinger Gürtel primary
273 378265328 Währinger Gürtel primary
274 378272959 Währinger Gürtel primary
275 378275518 Hernalser Gürtel primary
276 384292260 Hernalser Gürtel primary
277 384292261 Hernalser Gürtel primary
278 384796925 Währinger Gürtel primary
279 384796926 Währinger Gürtel primary
280 385800228 Währinger Gürtel primary
281 391813231 Gaudenzdorfer Gürtel primary
282 404881562 Liebenauer Gürtel primary
283 419250569 Karlauer Gürtel platform
284 419250570 Karlauer Gürtel platform
285 425297959 Währinger Gürtel primary
286 450081200 Landstraßer Gürtel residential
287 491714226 Währinger Gürtel primary
288 499396828 Weblinger Gürtel service
289 502906882 Landstraßer Gürtel primary
290 502918368 Währinger Gürtel primary
291 502918372 Währinger Gürtel primary
292 515537558 Landstraßer Gürtel primary
293 515537559 Landstraßer Gürtel primary
294 515537561 Landstraßer Gürtel primary
295 516473645 Sankt-Peter-Gürtel primary
296 516473646 Sankt-Peter-Gürtel primary
297 516473647 Sankt-Peter-Gürtel primary
298 537689830 Lerchenfelder Gürtel primary
299 537689831 Lerchenfelder Gürtel primary
300 537715715 Mariahilfer Gürtel primary
301 537715719 Gaudenzdorfer Gürtel primary
302 537715723 Gaudenzdorfer Gürtel primary
303 537932457 Währinger Gürtel primary
304 537932460 Währinger Gürtel primary
305 537978830 Wiedner Gürtel primary
306 537978831 Wiedner Gürtel primary
307 568246885 Döblinger Gürtel tertiary
308 576332930 Hernalser Gürtel primary
309 628371786 Hernalser Gürtel primary
310 631910642 Sankt-Peter-Gürtel primary
311 639501875 Wiedner Gürtel primary_link
312 657079587 Währinger Gürtel primary
313 657079588 Währinger Gürtel primary
314 667715002 Weblinger Gürtel primary
315 671154398 Wiedner Gürtel primary_link
316 679596246 Währinger Gürtel primary
317 679596247 Währinger Gürtel primary
318 681257994 Währinger Gürtel primary
319 687707607 Gumpendorfer Gürtel primary
320 688172642 Landstraßer Gürtel primary
321 743329426 Wiedner Gürtel primary_link
322 752996637 Waltendorfer Gürtel service
323 752996638 Waltendorfer Gürtel service
324 772129790 Sankt-Peter-Gürtel service
325 772143638 Sankt-Peter-Gürtel service
326 785306583 Lerchenfelder Gürtel primary
327 785306584 Lerchenfelder Gürtel primary
328 810541795 Währinger Gürtel primary
329 810647295 Hernalser Gürtel primary
330 810647296 Lerchenfelder Gürtel primary
331 812392882 Gumpendorfer Gürtel primary
332 813057438 Mariahilfer Gürtel primary
333 813057439 Mariahilfer Gürtel primary
334 813057440 Mariahilfer Gürtel primary
335 813507907 Hernalser Gürtel primary
336 813507908 Hernalser Gürtel primary
337 813588389 Hernalser Gürtel secondary
338 813671729 Währinger Gürtel primary
339 818917142 Sankt-Peter-Gürtel primary
340 818917143 Sankt-Peter-Gürtel primary
341 834641547 Gürtel residential
342 895346238 Weblinger Gürtel service
343 895346239 Weblinger Gürtel service
344 905123200 Währinger Gürtel residential
345 937875334 Währinger Gürtel cycleway
346 938334758 Landstraßer Gürtel footway
347 945831227 Wiedner Gürtel primary_link
348 968143419 Sankt-Peter-Gürtel primary
349 972115137 Gaudenzdorfer Gürtel primary
350 972115138 Gaudenzdorfer Gürtel primary
351 1000010354 Währinger Gürtel primary
352 1024961073 Währinger Gürtel busway
353 1029057330 Landstraßer Gürtel primary
354 1051095497 Gaudenzdorfer Gürtel primary
355 1056947093 Döblinger Gürtel primary
356 1056947094 Döblinger Gürtel residential
357 1120606320 Währinger Gürtel primary
358 1120606329 Währinger Gürtel primary
359 1175919639 Döblinger Gürtel primary
360 1188504330 Landstraßer Gürtel footway
361 1188901254 Lerchenfelder Gürtel primary
362 1207424231 Sankt-Peter-Gürtel primary
363 1207424234 Sankt-Peter-Gürtel primary
364 1207424235 Sankt-Peter-Gürtel primary
365 1213763396 Hernalser Gürtel secondary
366 1214356761 Gaudenzdorfer Gürtel primary
367 1220403824 Brücke über den Mariahilfer Gürtel NaN
368 1258030644 Weblinger Gürtel primary
369 1258030645 Weblinger Gürtel primary
370 1258030646 Weblinger Gürtel primary
371 1258030648 Weblinger Gürtel primary
372 1266482478 Währinger Gürtel primary
373 1276193240 Währinger Gürtel service
374 1278105687 Döblinger Gürtel residential
375 1316166510 Weblinger Gürtel service
376 1316168064 Weblinger Gürtel service
377 1316168065 Weblinger Gürtel service
378 1425539855 Sechshauser Gürtel primary
379 1425539923 Mariahilfer Gürtel primary
380 1425540045 Gumpendorfer Gürtel primary
381 1425914136 Währinger Gürtel primary
382 1429360646 Hernalser Gürtel secondary
383 1453275730 Wiedner Gürtel primary
384 1453275731 Landstraßer Gürtel primary
385 1453275733 Wiedner Gürtel primary
386 1456542225 Weblinger Gürtel service
387 1456992118 Hernalser Gürtel secondary
388 1456992120 Hernalser Gürtel secondary
389 1482276855 Landstraßer Gürtel primary
390 1497694252 Wiedner Gürtel primary
391 1497694253 Wiedner Gürtel primary
392 1497694254 Wiedner Gürtel primary
393 1497694255 Wiedner Gürtel primary
394 1516213049 Lerchenfelder Gürtel primary
20:41
The query is working. But it's finding Gürtel ways all over Austria (Graz has a Gürtel too - Weblinger Gürtel, Waltendorfer Gürtel, etc.). I need to focus on Vienna's Gürtel. Let me check what Gürtel ways are in Vienna. Actually, the user says "Vienna's Gürtel" which is specifically the ring road around the inner city districts. The ones in Vienna are: Währinger Gürtel, Döblinger Gürtel, Hernalser Gürtel, Lerchenfelder Gürtel, Mariahilfer Gürtel, Gumpendorfer Gürtel, Sechshauser Gürtel, Gaudenzdorfer Gürtel, Wiedner Gürtel, Landstraßer Gürtel. The ones in Graz (Weblinger Gürtel, Waltendorfer Gürtel, Eggenberger Gürtel, etc.) are clearly not in Vienna. So I should filter by the bounding box of Vienna or by the actual location. Let me use a different approach: first get the Gürtel ways that are in/near Vienna by doing a spatial filter, or just find all that end with 'Gürtel' and then filter by those whose geometry is within Vienna's general area. Actually, let me think about this more carefully. The user says "Vienna's Gürtel, defined as the highway ways whose name ends in 'Gürtel'". So they want ALL highways whose name ends in Gürtel, and then buffer around them. But many Gürtel ways are in Graz, not Vienna. The user says "Vienna's Gürtel" - so they're referring to the specific ring road in Vienna. Let me filter by a bounding box around Vienna first. Let me use osmnx to get the Vienna boundary, or just use a bounding box. Actually, I think the simplest approach: get all Gürtel ways from the PBF, filter those whose centroids fall within a bounding box around Vienna, then proceed.
20:41
20:41
Write
/work/solve.py
content · 220 lines · py
"""
Extract highway network and PT route relations intersecting a 500m buffer
around Vienna's Gürtel (ways whose name ends with 'Gürtel').
Output: vienna_network.gpkg with two layers:
- highways (LineString, EPSG:31287)
- pt_routes (MultiLineString, EPSG:31287)
Uses pyogrio for fast OSM PBF reading.
"""
import pyogrio
import geopandas as gpd
import pandas as pd
import numpy as np
from shapely.geometry import LineString, MultiLineString, box, shape
from shapely.ops import unary_union, transform, linemerge
import pyproj
import json
import re
PBF = "/work/austria-latest.osm.pbf"
OUT = "/work/vienna_network.gpkg"
BUFF_M = 500
CRS_PROJ = "EPSG:31287" # MGI / Austria Lambert
CRS_WGS = "EPSG:4326"
# ---------------------------------------------------------------------------
# Step 1 – Get all Gürtel highway ways from the lines layer
# ---------------------------------------------------------------------------
print("Step 1: Finding Gürtel ways...")
sql1 = "SELECT * FROM lines WHERE name LIKE '%Gürtel' AND highway IS NOT NULL"
gdf_all_guertel = pyogrio.read_dataframe(PBF, sql=sql1)
print(f" Found {len(gdf_all_guertel)} ways worldwide with 'Gürtel' in name and highway tag")
# Filter: name must end with 'Gürtel' (the SQL LIKE matches anywhere)
mask_name = gdf_all_guertel['name'].str.endswith('Gürtel')
gdf_all_guertel = gdf_all_guertel[mask_name].copy()
print(f" Of which {len(gdf_all_guertel)} end with 'Gürtel'")
# Spatially filter to Vienna area (~bbox of Vienna)
# Vienna approximate bounding box in WGS84
VIENNA_BBOX = box(16.1, 48.1, 16.6, 48.35)
gdf_all_guertel = gdf_all_guertel[gdf_all_guertel.intersects(VIENNA_BBOX)].copy()
print(f" Of which {len(gdf_all_guertel)} are in the Vienna area")
if len(gdf_all_guertel) == 0:
print(" ERROR: No Gürtel ways found in Vienna area!")
# Fallback - use all Gürtel ways
gdf_all_guertel = gdf_all_guertel_orig
# ---------------------------------------------------------------------------
# Step 2 – Compute the 500 m buffer around the Gürtel
# ---------------------------------------------------------------------------
print("Step 2: Computing buffer...")
project_fwd = pyproj.Transformer.from_crs(CRS_WGS, CRS_PROJ, always_xy=True).transform
project_rev = pyproj.Transformer.from_crs(CRS_PROJ, CRS_WGS, always_xy=True).transform
# Merge all Gürtel lines into one geometry
guertel_union = unary_union(gdf_all_guertel.geometry.values)
guertel_proj = transform(project_fwd, guertel_union)
buf_proj = guertel_proj.buffer(BUFF_M)
buf_wgs = transform(project_rev, buf_proj)
print(f" Buffer area (WGS84): {buf_wgs.area:.2f} sq deg")
# Save buffer for debugging
buf_gdf = gpd.GeoDataFrame({'geometry': [buf_wgs]}, crs=CRS_WGS)
buf_gdf.to_file(OUT, layer='_debug_buffer', driver='GPKG')
# ---------------------------------------------------------------------------
# Step 3 – Get highway ways intersecting the buffer
# ---------------------------------------------------------------------------
print("Step 3: Getting highway ways intersecting buffer...")
# We'll stream the lines layer in chunks using pyogrio with SQL
# But first let's just get all lines and filter spatially - might be big
# Let's try a different approach: read bbox-filtered data
# Get the bounding box of the buffer to limit reads
buf_bounds = buf_wgs.bounds # (minx, miny, maxx, maxy)
print(f" Buffer bounds: {buf_bounds}")
# We'll read all lines and filter. The lines layer has ~3M features.
# Let's try with a WHERE clause using the bbox if possible.
# pyogrio supports spatial filtering via the 'bbox' parameter.
sql_hw = "SELECT * FROM lines WHERE highway IS NOT NULL"
gdf_hw = pyogrio.read_dataframe(PBF, sql=sql_hw, bbox=buf_bounds)
print(f" Read {len(gdf_hw)} highway features within buffer bbox")
# Now filter by actual intersection with the buffer
mask = gdf_hw.intersects(buf_wgs)
gdf_hw_filtered = gdf_hw[mask].copy()
print(f" {len(gdf_hw_filtered)} intersect the buffer")
# Select only the requested columns
keep_cols_hw = ['osm_id', 'name', 'highway', 'maxspeed', 'lanes', 'surface', 'oneway', 'geometry']
# Make sure all exist
for col in keep_cols_hw:
if col not in gdf_hw_filtered.columns:
gdf_hw_filtered[col] = None
gdf_hw_out = gdf_hw_filtered[keep_cols_hw].copy()
gdf_hw_out = gdf_hw_out.to_crs(CRS_PROJ)
# Only LineString geometries
gdf_hw_out = gdf_hw_out[gdf_hw_out.geometry.type == 'LineString']
print(f" Output highways: {len(gdf_hw_out)} features")
# ---------------------------------------------------------------------------
# Step 4 – Get PT route relations
# ---------------------------------------------------------------------------
print("Step 4: Getting PT route relations...")
# PT route types we care about
pt_routes_of_interest = ('bus', 'tram', 'subway', 'train', 'trolleybus', 'light_rail', 'railway')
# Read the other_relations layer filtered by type='route'
# We need to parse the 'other_tags' field to find route=bus/tram/etc
# or use the 'type' column which is 'route' for route relations
sql_rel = "SELECT * FROM other_relations WHERE type = 'route'"
gdf_routes = pyogrio.read_dataframe(PBF, sql=sql_rel, bbox=buf_bounds)
print(f" Found {len(gdf_routes)} route relations within buffer bbox")
# Parse the other_tags field (hstore-like string: "key"=>"value","key2"=>"value2")
def parse_other_tags(tag_str):
"""Parse OSM hstore format to dict."""
if pd.isna(tag_str) or not tag_str:
return {}
result = {}
# Pattern: "key"=>"value"
pairs = re.findall(r'"([^"]+)"\s*=>\s*"([^"]*)"', tag_str)
for k, v in pairs:
result[k] = v
return result
# Add parsed tags
parsed = gdf_routes['other_tags'].apply(parse_other_tags)
gdf_routes['route'] = parsed.apply(lambda d: d.get('route'))
gdf_routes['ref'] = parsed.apply(lambda d: d.get('ref'))
gdf_routes['route_name'] = parsed.apply(lambda d: d.get('name'))
gdf_routes['operator'] = parsed.apply(lambda d: d.get('operator'))
# Filter to PT route types
mask_pt = gdf_routes['route'].isin(pt_routes_of_interest)
gdf_pt_routes = gdf_routes[mask_pt].copy()
print(f" Of which {len(gdf_pt_routes)} are PT route relations")
if len(gdf_pt_routes) == 0:
print(" WARNING: No PT routes found via other_tags parsing.")
print(" Trying alternative: reading from the multilinestrings layer...")
# ---------------------------------------------------------------------------
# Step 5 – For each PT route, we need to build its geometry from member ways
# ---------------------------------------------------------------------------
# The OSM PBF as read by pyogrio doesn't give us relation membership directly.
# We need a different approach. Let's use osmium for the relation handling.
# Actually, let's try a different approach entirely. Use osmium to get the relations,
# build geometries, and then write out. But osmium was too slow earlier...
#
# Let me try a smarter approach: use pyogrio to get all data, then use osmium
# only for the relation-to-way mapping.
# Alternative: the multilinestrings layer might contain route geometries already.
print(" Checking multilinestrings layer for PT routes...")
sql_ml = "SELECT * FROM multilinestrings WHERE route IN ('bus','tram','subway','train','trolleybus','light_rail')"
try:
gdf_ml = pyogrio.read_dataframe(PBF, sql=sql_ml, bbox=buf_bounds)
print(f" Found {len(gdf_ml)} multilinestrings with PT route tag")
if len(gdf_ml) > 0:
cols = ['osm_id', 'ref', 'name', 'operator', 'route', 'geometry']
for c in cols:
if c not in gdf_ml.columns:
gdf_ml[c] = None
gdf_pt_out = gdf_ml[cols].copy()
gdf_pt_out = gdf_pt_out.to_crs(CRS_PROJ)
# Ensure MultiLineString
mask_ls = gdf_pt_out.geometry.type == 'LineString'
if mask_ls.any():
gdf_pt_out.loc[mask_ls, 'geometry'] = gdf_pt_out.loc[mask_ls, 'geometry'].apply(
lambda g: MultiLineString([g])
)
gdf_pt_out = gdf_pt_out[gdf_pt_out.geometry.type == 'MultiLineString']
# Filter by buffer intersection
buf_proj_wgs84 = transform(project_rev, buf_proj) # wait, buf_proj is already projected
# Actually we need to filter in WGS84 or projected. Let's just use the WGS84 buffer.
mask_pt = gdf_pt_out.to_crs(CRS_WGS).intersects(buf_wgs)
gdf_pt_out = gdf_pt_out[mask_pt]
print(f" Output PT routes: {len(gdf_pt_out)} features")
else:
gdf_pt_out = gpd.GeoDataFrame({'osm_id': pd.Series(dtype='int'),
'ref': pd.Series(dtype='str'),
'name': pd.Series(dtype='str'),
'operator': pd.Series(dtype='str'),
'route': pd.Series(dtype='str'),
'geometry': pd.Series(dtype='object')}, crs=CRS_WGS)
except Exception as e:
print(f" Error reading multilinestrings: {e}")
gdf_pt_out = gpd.GeoDataFrame({'osm_id': pd.Series(dtype='int'),
'ref': pd.Series(dtype='str'),
'name': pd.Series(dtype='str'),
'operator': pd.Series(dtype='str'),
'route': pd.Series(dtype='str'),
'geometry': pd.Series(dtype='object')}, crs=CRS_WGS)
# ---------------------------------------------------------------------------
# Step 6 – Write to GeoPackage
# ---------------------------------------------------------------------------
print("Step 6: Writing to GeoPackage...")
gdf_hw_out.to_file(OUT, layer='highways', driver='GPKG')
print(f" Written {len(gdf_hw_out)} highways to {OUT}")
if len(gdf_pt_out) > 0:
gdf_pt_out.to_file(OUT, layer='pt_routes', driver='GPKG')
print(f" Written {len(gdf_pt_out)} PT routes to {OUT}")
else:
print(" No PT routes to write")
print("Done!")
Wrote /work/solve.py (9943 chars).
20:54
The approach using pyogrio with multilinestrings might work for PT routes, but the multilinestrings layer may not contain all route relations properly. Let me think about this more carefully. In OSM PBF files read by OGR/GDAL (pyogrio): - `lines`: ways that are simple linestrings (most highways, etc.) - `multilinestrings`: ways that are multilinestrings (generally not common) - `other_relations`: relations of type other than boundary, multipolygon, etc. These include route relations - `multipolygons`: multipolygon relations For PT routes (route relations), the `other_relations` layer has them. But they have `GeometryCollection` geometries. Each relation has member ways, and OGR creates a geometry collection from the member way geometries. Wait, actually OGR for OSM does create geometry for route relations from their member ways. Let me check if the other_relations layer already has proper geometries for route relations. Let me test this.
20:54
Bash
cd /work && timeout 120 python3 -c "
cd /work && timeout 120 python3 -c "
import pyogrio
import pandas as pd
import re
# Check what the other_relations look like for route type
sql = \"SELECT * FROM other_relations WHERE type = 'route' LIMIT 20\"
df = pyogrio.read_dataframe('/work/austria-latest.osm.pbf', sql=sql)
print('Columns:', list(df.columns))
print('---')
for i in range(min(5, len(df))):
print(f'Row {i}:')
print(f' osm_id: {df.iloc[i][\"osm_id\"]}')
print(f' name: {df.iloc[i][\"name\"]}')
print(f' type: {df.iloc[i][\"type\"]}')
print(f' other_tags: {df.iloc[i][\"other_tags\"]}')
print(f' geometry type: {df.iloc[i][\"geometry\"].geom_type if df.iloc[i][\"geometry\"] else None}')
print(f' geometry: {df.iloc[i][\"geometry\"]}')
print()
" 2>&1
Columns: ['osm_id', 'name', 'type', 'other_tags', 'geometry'] ---
21:33
21:33
Bash
cd /work && timeout 120 python3 -c "
cd /work && timeout 120 python3 -c "
import pyogrio
# Check the other_relations layer
sql = \"SELECT * FROM other_relations WHERE type = 'route' LIMIT 50\"
df = pyogrio.read_dataframe('/work/austria-latest.osm.pbf', sql=sql)
print(f'Total: {len(df)}')
for i in range(len(df)):
ot = df.iloc[i]['other_tags']
if ot and 'route' in ot:
print(f'{df.iloc[i][\"osm_id\"]}: name={df.iloc[i][\"name\"]}, type={df.iloc[i][\"type\"]}, tags={ot[:200]}')
" 2>&1
Total: 0
22:11
Hmm, the other_relations layer with type='route' returns 0 results. Let me check the schema of the multilinestrings layer and other layers properly.
22:11
Bash
cd /work && timeout 120 python3 -c "
cd /work && timeout 120 python3 -c "
import pyogrio
# Check all layers and their columns
for layer in ['points', 'lines', 'multilinestrings', 'multipolygons', 'other_relations']:
try:
meta = pyogrio.read_info('/work/austria-latest.osm.pbf', layer=layer)
print(f'Layer {layer}: {meta[\"features\"]} features')
cols = pyogrio.list_fields('/work/austria-latest.osm.pbf', layer=layer)
print(f' Columns: {cols[:20]}')
except Exception as e:
print(f'Layer {layer}: error - {e}')
print()
" 2>&1
Layer points: -1 features Layer points: error - module 'pyogrio' has no attribute 'list_fields' Layer lines: -1 features Layer lines: error - module 'pyogrio' has no attribute 'list_fields' Layer multilinestrings: -1 features Layer multilinestrings: error - module 'pyogrio' has no attribute 'list_fields' Layer multipolygons: -1 features Layer multipolygons: error - module 'pyogrio' has no attribute 'list_fields' Layer other_relations: -1 features Layer other_relations: error - module 'pyogrio' has no attribute 'list_fields'
22:17
Let me check how pyogrio handles OSM PBF more carefully. It seems the other_relations layer might be empty or filtered differently. Let me look at the actual data.
22:17
Bash
cd /work && timeout 120 python3 -c "
cd /work && timeout 120 python3 -c "
import pyogrio
# Check what's in other_relations
meta = pyogrio.read_info('/work/austria-latest.osm.pbf', layer='other_relations')
print('Meta:', meta)
print()
# Try reading without filter
df = pyogrio.read_dataframe('/work/austria-latest.osm.pbf', layer='other_relations', max_features=10)
print('Columns:', list(df.columns))
print(f'Rows: {len(df)}')
for i in range(len(df)):
print(f'Row {i}: osm_id={df.iloc[i][\"osm_id\"]}, type={df.iloc[i][\"type\"]}, other_tags={df.iloc[i][\"other_tags\"][:100] if df.iloc[i][\"other_tags\"] else None}')
" 2>&1
Meta: {'layer_name': 'other_relations', 'crs': 'EPSG:4326', 'encoding': 'UTF-8', 'fields': array(['osm_id', 'name', 'type', 'other_tags'], dtype=object), 'dtypes': array(['object', 'object', 'object', 'object'], dtype=object), 'ogr_types': ['OFTString', 'OFTString', 'OFTString', 'OFTString'], 'ogr_subtypes': ['OFSTNone', 'OFSTNone', 'OFSTNone', 'OFSTNone'], 'fid_column': '', 'geometry_name': '', 'geometry_type': 'GeometryCollection', 'features': -1, 'total_bounds': (9.52678, 46.369793, 17.164078, 49.02403), 'driver': 'OSM', 'capabilities': {'random_read': False, 'fast_set_next_by_index': False, 'fast_spatial_filter': False, 'fast_feature_count': False, 'fast_total_bounds': True}, 'layer_metadata': None, 'dataset_metadata': None}
Columns: ['osm_id', 'name', 'type', 'other_tags', 'geometry']
Rows: 10
Row 0: osm_id=3801, type=bridge, other_tags="layer"=>"2"
Row 1: osm_id=8535, type=tunnel, other_tags="wikidata"=>"Q2460065","wikimedia_commons"=>"Category:Tunnel Tradenberg","wikipedia"=>"de:Tunnel Tra
Row 2: osm_id=12267, type=restriction, other_tags="bicycle"=>"yes","restriction"=>"no_left_turn"
Traceback (most recent call last):
File "<string>", line 13, in <module>
print(f'Row {i}: osm_id={df.iloc[i]["osm_id"]}, type={df.iloc[i]["type"]}, other_tags={df.iloc[i]["other_tags"][:100] if df.iloc[i]["other_tags"] else None}')
~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
TypeError: 'float' object is not subscriptable
23:27
The other_relations layer has type='route' but I see the SQL filter returned 0 results. Let me check what types are available.
23:27
Bash
cd /work && timeout 120 python3 -c "
cd /work && timeout 120 python3 -c "
import pyogrio
import pandas as pd
df = pyogrio.read_dataframe('/work/austria-latest.osm.pbf', layer='other_relations', max_features=1000)
print(f'Total read: {len(df)}')
# Check types
types = df['type'].value_counts()
print('Types:', types.head(20))
print()
# Check for route type
route_mask = df['type'] == 'route'
print(f'Route relations: {route_mask.sum()}')
if route_mask.any():
for i in df[route_mask].index[:5]:
print(f' osm_id={df.iloc[i][\"osm_id\"]}, name={df.iloc[i][\"name\"]}, other_tags={df.iloc[i][\"other_tags\"]}')
" 2>&1
Total read: 1000 Types: type public_transport 490 restriction 328 station 36 enforcement 36 TMC 34 waterway 24 site 11 lane_assist 9 network 7 bridge 5 roadAccess 5 destination_sign 4 skitour 3 building 2 tunnel 1 street 1 power 1 communication_tower_group 1 route_master 1 cluster 1 Name: count, dtype: int64 Route relations: 0
24:42
So there are no "type=route" relations in the first 1000 features of other_relations. They're probably stored differently. Let me check the multilinestrings layer - that's where OSM route relations typically end up in the GDAL OSM driver.
24:42
Bash
cd /work && timeout 120 python3 -c "
cd /work && timeout 120 python3 -c "
import pyogrio, pandas as pd
# Check multilinestrings layer structure
meta = pyogrio.read_info('/work/austria-latest.osm.pbf', layer='multilinestrings')
print('MultiLineString columns:', meta['fields'])
# Read a sample
df = pyogrio.read_dataframe('/work/austria-latest.osm.pbf', layer='multilinestrings', max_features=20)
print('Sample columns:', list(df.columns))
print('First 5:')
for i in range(5):
print(f' osm_id={df.iloc[i][\"osm_id\"]}, name={df.iloc[i][\"name\"]}, type={df.iloc[i][\"type\"]}')
" 2>&1
MultiLineString columns: ['osm_id' 'name' 'type' 'other_tags'] Sample columns: ['osm_id', 'name', 'type', 'other_tags', 'geometry'] First 5: osm_id=2381, name=Moorwanderweg Tanner Moor, type=route osm_id=5738, name=Zirben-Atem-Trail, type=route osm_id=8503, name=Kreuttal Radroute, type=route osm_id=8524, name=Rund um den Bisamberg, type=route osm_id=8526, name=S2: Mödling => Laa, type=route
25:59
The multilinestrings layer has route-type relations but with their geometry already built. The other_relations layer is for non-route relations (bridges, tunnels, etc.). Let me check if route relations appear in multilinestrings. The issue is that I need to find the route relations that are of type bus/tram/subway/etc. Let me check:
25:59
Bash
cd /work && timeout 120 python3 -c "
cd /work && timeout 120 python3 -c "
import pyogrio, re
# Read more multilinestrings to find PT routes
df = pyogrio.read_dataframe('/work/austria-latest.osm.pbf', layer='multilinestrings', max_features=50000)
print(f'Total: {len(df)}')
# Parse other_tags for route value
def parse_other_tags(tag_str):
if pd.isna(tag_str) or not isinstance(tag_str, str):
return {}
result = {}
pairs = re.findall(r'\"([^\"]+)\"\s*=>\s*\"([^\"]*)\"', tag_str)
for k, v in pairs:
result[k] = v
return result
parsed = df['other_tags'].apply(parse_other_tags)
df['route'] = parsed.apply(lambda d: d.get('route'))
df['ref'] = parsed.apply(lambda d: d.get('ref'))
df['rname'] = parsed.apply(lambda d: d.get('name'))
df['operator'] = parsed.apply(lambda d: d.get('operator'))
pt_types = ('bus', 'tram', 'subway', 'train', 'trolleybus', 'light_rail', 'railway')
mask = df['route'].isin(pt_types)
print(f'PT routes: {mask.sum()}')
print()
# Show some
pt_df = df[mask]
for i in pt_df.index[:10]:
row = pt_df.loc[i]
print(f' osm_id={row[\"osm_id\"]}, ref={row[\"ref\"]}, name={row[\"rname\"]}, route={row[\"route\"]}, operator={row[\"operator\"]}')
" 2>&1
Total: 34516
Traceback (most recent call last):
File "<string>", line 18, in <module>
parsed = df['other_tags'].apply(parse_other_tags)
File "/usr/local/lib/python3.14/site-packages/pandas/core/series.py", line 5084, in apply
).apply()
~~~~~^^
File "/usr/local/lib/python3.14/site-packages/pandas/core/apply.py", line 1520, in apply
return self.apply_standard()
~~~~~~~~~~~~~~~~~~~^^
File "/usr/local/lib/python3.14/site-packages/pandas/core/apply.py", line 1578, in apply_standard
mapped = obj._map_values(mapper=curried)
File "/usr/local/lib/python3.14/site-packages/pandas/core/base.py", line 1020, in _map_values
return arr.map(mapper, na_action=na_action)
~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.14/site-packages/pandas/core/arrays/arrow/array.py", line 1753, in map
return super().map(mapper, na_action)
~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.14/site-packages/pandas/core/arrays/base.py", line 2745, in map
return map_array(self, mapper, na_action=na_action)
File "/usr/local/lib/python3.14/site-packages/pandas/core/algorithms.py", line 1715, in map_array
return lib.map_infer(values, mapper)
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
File "pandas/_libs/lib.pyx", line 3071, in pandas._libs.lib.map_infer
File "<string>", line 10, in parse_other_tags
if pd.isna(tag_str) or not isinstance(tag_str, str):
^^
NameError: name 'pd' is not defined. Did you mean: 'id'?
27:22
27:22
Bash
cd /work && timeout 120 python3 -c "
cd /work && timeout 120 python3 -c "
import pyogrio, re, pandas as pd
# Read more multilinestrings to find PT routes
df = pyogrio.read_dataframe('/work/austria-latest.osm.pbf', layer='multilinestrings', max_features=50000)
print(f'Total: {len(df)}')
# Parse other_tags for route value
def parse_other_tags(tag_str):
if pd.isna(tag_str) or not isinstance(tag_str, str):
return {}
result = {}
pairs = re.findall(r'\"([^\"]+)\"\s*=>\s*\"([^\"]*)\"', tag_str)
for k, v in pairs:
result[k] = v
return result
parsed = df['other_tags'].apply(parse_other_tags)
df['route'] = parsed.apply(lambda d: d.get('route'))
df['ref'] = parsed.apply(lambda d: d.get('ref'))
df['rname'] = parsed.apply(lambda d: d.get('name'))
df['operator'] = parsed.apply(lambda d: d.get('operator'))
pt_types = ('bus', 'tram', 'subway', 'train', 'trolleybus', 'light_rail', 'railway')
mask = df['route'].isin(pt_types)
print(f'PT routes: {mask.sum()}')
print()
# Show some
pt_df = df[mask]
for i in pt_df.index[:10]:
row = pt_df.loc[i]
print(f' osm_id={row[\"osm_id\"]}, ref={row[\"ref\"]}, name={row[\"rname\"]}, route={row[\"route\"]}, operator={row[\"operator\"]}')
" 2>&1
Total: 34516 PT routes: 12543 osm_id=8526, ref=S2, name=None, route=train, operator=ÖBB-Personenverkehr AG osm_id=20309, ref=U3, name=None, route=subway, operator=Wiener Linien osm_id=22494, ref=U1, name=None, route=subway, operator=Wiener Linien osm_id=23294, ref=1, name=None, route=tram, operator=nan osm_id=27010, ref=7, name=None, route=tram, operator=Graz Linien osm_id=27030, ref=58, name=None, route=bus, operator=Graz Linien osm_id=27114, ref=U2, name=None, route=subway, operator=Wiener Linien osm_id=27176, ref=5, name=None, route=tram, operator=nan osm_id=27194, ref=4, name=None, route=tram, operator=nan osm_id=27222, ref=3, name=None, route=tram, operator=Graz Linien
28:56
The multilinestrings layer contains PT routes with their geometries already built. This is great! Now I need to: 1. Find all Gürtel ways (highway, name ends with Gürtel) in Vienna 2. Compute 500m buffer 3. Read highway lines intersecting buffer 4. Read PT route multilinestrings intersecting buffer Let me rewrite the script properly.
28:56
28:56
Write
/work/solve.py
content · 162 lines · py
"""
Extract highway network and PT route relations intersecting a 500m buffer
around Vienna's Gürtel. Output: vienna_network.gpkg with highways (LineString)
and pt_routes (MultiLineString), EPSG:31287 (MGI / Austria Lambert).
"""
import pyogrio
import geopandas as gpd
import pandas as pd
import numpy as np
import re
import pyproj
from shapely.geometry import LineString, MultiLineString, box
from shapely.ops import unary_union, transform
PBF = "/work/austria-latest.osm.pbf"
OUT = "/work/vienna_network.gpkg"
BUFF_M = 500
CRS_PROJ = "EPSG:31287"
CRS_WGS = "EPSG:4326"
project_fwd = pyproj.Transformer.from_crs(CRS_WGS, CRS_PROJ, always_xy=True).transform
project_rev = pyproj.Transformer.from_crs(CRS_PROJ, CRS_WGS, always_xy=True).transform
# ---------------------------------------------------------------------------
# 1. Find Vienna Gürtel ways (highway, name ends with 'Gürtel')
# ---------------------------------------------------------------------------
print("1. Finding Gürtel ways in Vienna...")
sql = "SELECT * FROM lines WHERE highway IS NOT NULL AND name LIKE '%Gürtel'"
gdf = pyogrio.read_dataframe(PBF, sql=sql)
print(f" Found {len(gdf)} potential Gürtel ways")
# Name must end with 'Gürtel'
gdf = gdf[gdf['name'].str.endswith('Gürtel', na=False)].copy()
# Spatial filter to Vienna area
vienna_bbox = box(16.1, 48.1, 16.6, 48.35)
gdf = gdf[gdf.intersects(vienna_bbox)].copy()
print(f" Vienna-area Gürtel ways: {len(gdf)}")
if len(gdf) == 0:
raise RuntimeError("No Gürtel ways found in Vienna area")
# ---------------------------------------------------------------------------
# 2. Compute 500m buffer in projected CRS
# ---------------------------------------------------------------------------
print("2. Computing buffer...")
guertel_union = unary_union(gdf.geometry.values)
guertel_proj = transform(project_fwd, guertel_union)
buf_proj = guertel_proj.buffer(BUFF_M)
# Keep buffer in both CRS for later filtering
buf_wgs = transform(project_rev, buf_proj)
buf_bounds_wgs = buf_wgs.bounds # (minx, miny, maxx, maxy)
print(f" Buffer bounds WGS84: {buf_bounds_wgs}")
# ---------------------------------------------------------------------------
# 3. Read highway ways within buffer bounding box, then filter
# ---------------------------------------------------------------------------
print("3. Getting highway ways intersecting buffer...")
sql_hw = "SELECT * FROM lines WHERE highway IS NOT NULL"
gdf_hw = pyogrio.read_dataframe(PBF, sql=sql_hw, bbox=buf_bounds_wgs)
print(f" Highway ways in buffer bbox: {len(gdf_hw)}")
# Spatial filter: actually intersect the buffer
gdf_hw = gdf_hw[gdf_hw.intersects(buf_wgs)].copy()
print(f" Highway ways intersecting buffer: {len(gdf_hw)}")
# Keep only requested columns
keep = ['osm_id', 'name', 'highway', 'maxspeed', 'lanes', 'surface', 'oneway', 'geometry']
for col in keep:
if col not in gdf_hw.columns:
gdf_hw[col] = None
gdf_hw_out = gdf_hw[keep].copy()
# Project and ensure LineString
gdf_hw_out = gdf_hw_out.to_crs(CRS_PROJ)
gdf_hw_out = gdf_hw_out[gdf_hw_out.geometry.type == 'LineString'].copy()
print(f" Output highways: {len(gdf_hw_out)}")
# ---------------------------------------------------------------------------
# 4. Read PT route multilinestrings
# ---------------------------------------------------------------------------
print("4. Getting PT route relations...")
# Parse other_tags hstore
def parse_tags(tag_str):
if pd.isna(tag_str) or not isinstance(tag_str, str):
return {}
result = {}
for m in re.finditer(r'"([^"]+)"\s*=>\s*"([^"]*)"', tag_str):
result[m.group(1)] = m.group(2)
return result
# Read in chunks to avoid memory issues with huge PBF reads
# Actually, let's read the whole multilinestrings layer within buffer bbox
# But multilinestrings might be large. Let's just read all and filter.
# The multilinestrings layer has routes as well as other things.
# Let's use SQL to filter by type='route' which is the 'type' column.
sql_ml = "SELECT * FROM multilinestrings WHERE type = 'route'"
gdf_ml = pyogrio.read_dataframe(PBF, sql=sql_ml, bbox=buf_bounds_wgs)
print(f" Route multilinestrings in buffer bbox: {len(gdf_ml)}")
# Parse tags
parsed = gdf_ml['other_tags'].apply(parse_tags)
gdf_ml['route'] = parsed.apply(lambda d: d.get('route'))
gdf_ml['ref'] = parsed.apply(lambda d: d.get('ref'))
gdf_ml['rname'] = parsed.apply(lambda d: d.get('name'))
gdf_ml['operator'] = parsed.apply(lambda d: d.get('operator'))
# Filter to PT route types
pt_types = ('bus', 'tram', 'subway', 'train', 'trolleybus', 'light_rail', 'railway')
gdf_pt = gdf_ml[gdf_ml['route'].isin(pt_types)].copy()
print(f" PT routes in bbox: {len(gdf_pt)}")
# Spatial filter: intersect buffer (convert to WGS84 for the test)
gdf_pt_wgs = gdf_pt.to_crs(CRS_WGS)
gdf_pt = gdf_pt[gdf_pt_wgs.intersects(buf_wgs)].copy()
print(f" PT routes intersecting buffer: {len(gdf_pt)}")
# Build output columns
out_cols = ['osm_id', 'ref', 'name', 'operator', 'route', 'geometry']
for col in out_cols:
if col not in gdf_pt.columns:
if col == 'osm_id':
gdf_pt[col] = 0
else:
gdf_pt[col] = None
gdf_pt_out = gdf_pt[out_cols].copy()
# Rename rname -> name if needed
if 'rname' in gdf_pt_out.columns and 'name' in gdf_pt_out.columns:
# name column already exists from OSM, use it
pass
gdf_pt_out = gdf_pt_out.to_crs(CRS_PROJ)
# Ensure MultiLineString type (convert LineStrings)
mask_ls = gdf_pt_out.geometry.type == 'LineString'
if mask_ls.any():
gdf_pt_out.loc[mask_ls, 'geometry'] = gdf_pt_out.loc[mask_ls, 'geometry'].apply(
lambda g: MultiLineString([g])
)
gdf_pt_out = gdf_pt_out[gdf_pt_out.geometry.type == 'MultiLineString'].copy()
print(f" Output PT routes: {len(gdf_pt_out)}")
# ---------------------------------------------------------------------------
# 5. Write GeoPackage
# ---------------------------------------------------------------------------
print("5. Writing GeoPackage...")
gdf_hw_out.to_file(OUT, layer='highways', driver='GPKG')
print(f" Highways: {len(gdf_hw_out)} features written")
if len(gdf_pt_out) > 0:
gdf_pt_out.to_file(OUT, layer='pt_routes', driver='GPKG')
print(f" PT routes: {len(gdf_pt_out)} features written")
else:
print(" WARNING: No PT routes to write")
print("Done!")
Wrote /work/solve.py (6336 chars).
29:05
Wait, the multilinestrings layer's `name` column might be the route name already. Let me check the column names more carefully. The issue is that when we read the multilinestrings layer, it has `osm_id, name, type, other_tags, geometry`. The `name` column is the OSM name tag (which could be the route name). The parsed `rname` from `other_tags` would be `name` from other_tags, but that might be redundant. Let me check what `name` in the multilinestrings layer actually is. Also, I need to be careful: the `name` column in the output should be the route name from the OSM name tag. And the `ref` should be from the ref tag (which is in other_tags). Let me run the script and check.
29:05
Bash
cd /work && timeout 300 python3 solve.py 2>&1
1. Finding Gürtel ways in Vienna... Found 394 potential Gürtel ways Vienna-area Gürtel ways: 218 2. Computing buffer... Buffer bounds WGS84: (16.32985446408538, 48.175697404107744, 16.40540552225453, 48.240822338117724) 3. Getting highway ways intersecting buffer... Highway ways in buffer bbox: 33273 Highway ways intersecting buffer: 9230 Output highways: 9230 4. Getting PT route relations... Route multilinestrings in buffer bbox: 672 PT routes in bbox: 487 PT routes intersecting buffer: 383 Output PT routes: 383 5. Writing GeoPackage... Highways: 9230 features written PT routes: 383 features written Done!
31:32
Let me verify the output to make sure everything is correct.
31:32
31:32
Bash
cd /work && python3 -c "
cd /work && python3 -c "
import pyogrio, geopandas as gpd
# Check layers
layers = pyogrio.list_layers('/work/vienna_network.gpkg')
print('Layers:', layers)
# Check highways
hw = pyogrio.read_dataframe('/work/vienna_network.gpkg', layer='highways')
print(f'\nHighways: {len(hw)} features')
print(f'CRS: {hw.crs}')
print(f'Geometry types: {hw.geometry.type.value_counts()}')
print(f'Columns: {list(hw.columns)}')
print(f'Sample:')
print(hw[['osm_id', 'name', 'highway', 'maxspeed', 'lanes', 'surface', 'oneway']].head(10))
# Check PT routes
pt = pyogrio.read_dataframe('/work/vienna_network.gpkg', layer='pt_routes')
print(f'\nPT routes: {len(pt)} features')
print(f'CRS: {pt.crs}')
print(f'Geometry types: {pt.geometry.type.value_counts()}')
print(f'Columns: {list(pt.columns)}')
print(f'Route types: {pt[\"route\"].value_counts()}')
print(f'Sample:')
print(pt[['osm_id', 'ref', 'name', 'operator', 'route']].head(10))
" 2>&1
Layers: [['highways' 'LineString']
['pt_routes' 'MultiLineString']]
Highways: 9230 features
CRS: EPSG:31287
Geometry types: LineString 9230
Name: count, dtype: int64
Columns: ['osm_id', 'name', 'highway', 'maxspeed', 'lanes', 'surface', 'oneway', 'geometry']
Sample:
osm_id name highway maxspeed lanes surface oneway
0 3220517 Anton-Schmid-Promenade cycleway None None None None
1 4407998 NaN trunk_link None None None None
2 4407999 Brigittenauer Lände primary None None None None
3 4408002 Koppstraße secondary None None None None
4 4408068 Währinger Gürtel primary None None None None
5 4408190 Heiligenstädter Straße primary None None None None
6 4408191 NaN primary None None None None
7 4463859 Landstraßer Gürtel primary None None None None
8 4463860 NaN motorway_link None None None None
9 4463862 NaN motorway_link None None None None
PT routes: 383 features
CRS: EPSG:31287
Geometry types: MultiLineString 383
Name: count, dtype: int64
Columns: ['osm_id', 'ref', 'name', 'operator', 'route', 'geometry']
Route types: route
bus 219
train 86
tram 49
railway 17
subway 10
light_rail 2
Name: count, dtype: int64
Sample:
osm_id ref ... operator route
0 8526 S2 ... ÖBB-Personenverkehr AG train
1 20309 U3 ... Wiener Linien subway
2 22494 U1 ... Wiener Linien subway
3 50332 35A ... Wiener Linien bus
4 61525 40A ... Wiener Linien bus
5 63301 S2 ... ÖBB-Personenverkehr AG train
6 63302 S1 ... ÖBB-Personenverkehr AG train
7 63304 S4 ... ÖBB-Personenverkehr AG train
8 63305 S3 ... ÖBB-Personenverkehr AG train
9 63311 S7 ... ÖBB-Personenverkehr AG train
[10 rows x 5 columns]
31:49
The output looks good. Let me also check: 1. That the `name` column in pt_routes is from OSM (which comes from the OSM 'name' tag), not from the parsed other_tags. Actually looking at the output, the `name` column shows None for most routes - that's because the OSM `name` field in the multilinestrings layer is the OSM name tag. For routes, the name tag often isn't set directly but rather in other_tags. Let me check what's happening with the name column. Looking at the code, I used `gdf_pt[out_cols]` where `out_cols` includes 'name'. The `name` column in the multilinestrings layer is the OSM `name` tag. For route relations, the name is often stored in the `name` tag (OSM column) or in `other_tags`. Let me verify. Actually, looking at the sample output, the routes have `name=None`. But some might have names. Let me check if the 'name' column in the original OSM multilinestrings layer is actually the OSM `name` tag or something else. From the earlier test, multilinestrings has columns: ['osm_id', 'name', 'type', 'other_tags', 'geometry']. The 'name' column is the OSM `name` tag. For route relations, the `name` tag is often not set - it's the `ref` tag that's more commonly used. But wait - the user asked for `name` attribute on PT routes. The OSM name tag for routes might be empty. Let me see if we should use the name from other_tags instead. Actually, looking at the code more carefully: I have both 'name' (from OSM multilinestrings) and 'rname' (parsed from other_tags). But in the output I used 'name' not 'rname'. Let me fix this to use the parsed name if the OSM name is None. Actually wait - looking at the code output for `pt_routes`, the `name` column is present but shows empty for most routes. The column I wanted was the OSM `name` tag. But actually for route relations, the `name` tag might be in either place. Let me check what we have. Let me also double-check the buffer is reasonable - 218 Gürtel ways in Vienna area seems plausible. And 9230 highway features in the buffer seems reasonable. Actually, I notice there's a potential issue: the user asked for the highway ways from Vienna's Gürtel specifically (ways whose name ends in 'Gürtel'), and then EVERY highway intersecting a 500m buffer around those Gürtel ways. That's what we did. Let me also verify the PT routes have the correct name column. Let me check what names exist.
31:49
31:49
Bash
cd /work && python3 -c "
cd /work && python3 -c "
import pyogrio, re
# Check what names exist in PT routes
pt = pyogrio.read_dataframe('/work/vienna_network.gpkg', layer='pt_routes')
print('Routes with non-null name:', pt['name'].notna().sum())
print('Routes with non-null ref:', pt['ref'].notna().sum())
print()
# Check some with names
named = pt[pt['name'].notna()]
if len(named) > 0:
print('Sample named routes:')
print(named[['ref', 'name', 'operator', 'route']].head(10))
# Also check if there are route names in other_tags that we might be missing
# Let's look at the original multilinestrings data again
sql = \"SELECT * FROM multilinestrings WHERE type = 'route' LIMIT 10\"
df = pyogrio.read_dataframe('/work/austria-latest.osm.pbf', sql=sql)
print()
print('Sample raw multilinestrings routes:')
for i in range(min(10, len(df))):
print(f' osm_id={df.iloc[i][\"osm_id\"]}, name={df.iloc[i][\"name\"]}, other_tags={df.iloc[i][\"other_tags\"]}')
" 2>&1
<exit_code>-1</exit_code> <error>Command timed out after 30000ms</error>
32:28
The command timed out because reading the PBF again. Let me check just the output file without reading the PBF again.
32:28
Bash
cd /work && timeout 30 python3 -c "
cd /work && timeout 30 python3 -c "
import pyogrio
pt = pyogrio.read_dataframe('/work/vienna_network.gpkg', layer='pt_routes')
print('Routes with non-null name:', pt['name'].notna().sum())
print('Routes with non-null ref:', pt['ref'].notna().sum())
named = pt[pt['name'].notna()]
if len(named) > 0:
print('Sample named:')
print(named[['ref', 'name', 'operator', 'route']].head(10))
" 2>&1
Routes with non-null name: 383 Routes with non-null ref: 382 Sample named: ref ... route 0 S2 ... train 1 U3 ... subway 2 U1 ... subway 3 35A ... bus 4 40A ... bus 5 S2 ... train 6 S1 ... train 7 S4 ... train 8 S3 ... train 9 S7 ... train [10 rows x 4 columns]
32:41
The output looks good. Let me check if the 'name' column in PT routes actually has the route names (not just the ref). Looking at the sample, the `name` column is populated for all 383 routes, but I notice in the output the names aren't showing - probably because they're long and the display truncates. Let me check more carefully. Actually, looking at the code again, the `name` column in the output is from the OSM `name` tag of the relation, not the `rname` I parsed from other_tags. Let me check if the OSM `name` field in the multilinestrings layer actually has meaningful route names. Let me also verify that the highways have their attributes properly (not all None).
32:41
Bash
cd /work && timeout 30 python3 -c "
cd /work && timeout 30 python3 -c "
import pyogrio
pt = pyogrio.read_dataframe('/work/vienna_network.gpkg', layer='pt_routes')
# Check name column properly
print('Name values sample:')
print(pt['name'].head(20).tolist())
print()
# Check ref
print('Ref values sample:')
print(pt['ref'].head(20).tolist())
" 2>&1
Name values sample: ['S2: Mödling => Laa', 'U3: Ottakring => Simmering', 'U1: Oberlaa => Leopoldau', 'Bus 35A: Spittelau => Salmannsdorf', 'Bus 40A: Schottentor => Döblinger Friedhof/Felix-Dahn-Straße', 'S2: Laa => Mödling', 'S1: Marchegg => Wien Meidling', 'S4: Wiener Neustadt => Tullnerfeld', 'S3: Hollabrunn => Wiener Neustadt', 'S7: Mistelbach => Wolfsthal', 'Nachtbus N41: Schottentor => Pötzleinsdorf', 'Nachtbus N43: Schottentor => Neuwaldegg', 'Nachtbus N49: Oper, Karlsplatz => Hütteldorf', 'S1: Wien Meidling => Marchegg', 'S40: St. Pölten => Wien', 'Bus 80A: Praterstern => Neu Marx', 'Bus 74A: St. Marx => Stubentor', 'Nachtbus N6: Westbahnhof => Enkplatz, Grillgasse', 'Nachtbus N8: Alterlaa => Handelskai', 'Tram 49: Ring, Volkstheater => Hütteldorf'] Ref values sample: ['S2', 'U3', 'U1', '35A', '40A', 'S2', 'S1', 'S4', 'S3', 'S7', 'N41', 'N43', 'N49', 'S1', 'S40', '80A', '74A', 'N6', 'N8', '49']
32:50
The output looks good. The PT routes have names and refs. Let me check that the highways also have proper data and verify the file size. Let me also double-check that the 'name' column in PT routes is populated (it looks like it is - "name" column from the OSM multilinestrings layer is being used, which has nice descriptions like "U3: Ottakring => Simmering"). Let me also check that the highways layer is indeed LineString (not MultiLineString) and the PT routes are MultiLineString. The script worked well. Let me verify the output file size and do a quick sanity check.
32:50
Bash
ls -lh /work/vienna_network.gpkg
-rw-r--r-- 1 agent agent 22M Jun 18 06:43 /work/vienna_network.gpkg
33:06
Everything looks good. Let me verify the output one more time and check that all the required attributes are present and not truncated.
33:06
33:06
Bash
cd /work && timeout 30 python3 -c "
cd /work && timeout 30 python3 -c "
import pyogrio
hw = pyogrio.read_dataframe('/work/vienna_network.gpkg', layer='highways')
pt = pyogrio.read_dataframe('/work/vienna_network.gpkg', layer='pt_routes')
print('=== HIGHWAYS LAYER ===')
print(f'Features: {len(hw)}')
print(f'Geometry type: LineString (verified)')
print(f'CRS: {hw.crs}')
print(f'Columns: {list(hw.columns)}')
for c in ['name','highway','maxspeed','lanes','surface','oneway']:
nonnull = hw[c].notna().sum()
print(f' {c}: {nonnull}/{len(hw)} non-null')
print()
print('=== PT ROUTES LAYER ===')
print(f'Features: {len(pt)}')
print(f'Geometry type: MultiLineString (verified)')
print(f'CRS: {pt.crs}')
print(f'Columns: {list(pt.columns)}')
for c in ['ref','name','operator','route']:
nonnull = pt[c].notna().sum()
print(f' {c}: {nonnull}/{len(pt)} non-null')
print(f' Route categories:')
print(pt['route'].value_counts().to_string())
" 2>&1
=== HIGHWAYS LAYER === Features: 9230 Geometry type: LineString (verified) CRS: EPSG:31287 Columns: ['osm_id', 'name', 'highway', 'maxspeed', 'lanes', 'surface', 'oneway', 'geometry'] name: 2554/9230 non-null highway: 9230/9230 non-null maxspeed: 0/9230 non-null lanes: 0/9230 non-null surface: 0/9230 non-null oneway: 0/9230 non-null === PT ROUTES LAYER === Features: 383 Geometry type: MultiLineString (verified) CRS: EPSG:31287 Columns: ['osm_id', 'ref', 'name', 'operator', 'route', 'geometry'] ref: 382/383 non-null name: 383/383 non-null operator: 369/383 non-null route: 383/383 non-null Route categories: route bus 219 train 86 tram 49 railway 17 subway 10 light_rail 2
33:20
The attributes are in the right fields but the maxspeed, lanes, surface, and oneway columns show 0 non-null values. This seems odd - many roads should have these tags. Let me check if the original OSM data has these tags but maybe they're stored differently. Let me check a few Gürtel way features directly from the PBF.
33:20
33:20
Bash
cd /work && timeout 30 python3 -c "
cd /work && timeout 30 python3 -c "
import pyogrio
# Check some Gürtel way from PBF directly for these attributes
sql = \"SELECT * FROM lines WHERE name LIKE '%Gürtel' AND highway IS NOT NULL LIMIT 50\"
df = pyogrio.read_dataframe('/work/austria-latest.osm.pbf', sql=sql)
print('All columns:', list(df.columns))
print()
# Show one row fully
for c in df.columns:
vals = df[c].dropna().unique()
if len(vals) > 0:
print(f'{c}: {vals[:5]}')
" 2>&1
All columns: ['osm_id', 'name', 'highway', 'waterway', 'aerialway', 'barrier', 'man_made', 'railway', 'z_order', 'other_tags', 'geometry']
osm_id: <ArrowStringArray>
['3992920', '3993614', '3997191', '4100111', '4302806']
Length: 5, dtype: str
name: <ArrowStringArray>
[ 'Sankt-Peter-Gürtel', 'Waltendorfer Gürtel', 'Weblinger Gürtel',
'Währinger Gürtel', 'Landstraßer Gürtel']
Length: 5, dtype: str
highway: <ArrowStringArray>
['primary', 'secondary', 'service', 'residential', 'living_street']
Length: 5, dtype: str
z_order: [7 6 0 3 4]
other_tags: <ArrowStringArray>
['"bicycle"=>"use_sidepath","cycleway:left"=>"separate","foot"=>"use_sidepath","lanes"=>"7","lanes:backward"=>"2","lanes:forward"=>"5","lit"=>"yes","maxspeed"=>"60","oneway"=>"no","ref"=>"B67a","sidewalk:left"=>"no","sidewalk:right"=>"separate","surface"=>"asphalt","turn:lanes:forward"=>"left|left|through|through|right"',
'"cycleway:left"=>"no","cycleway:right"=>"separate","foot"=>"use_sidepath","lanes"=>"3","lanes:backward"=>"2","lanes:forward"=>"1","lit"=>"yes","maxspeed"=>"50","oneway"=>"no","priority_road"=>"yes","ref"=>"B67c","sidewalk"=>"separate","surface"=>"asphalt","turn:lanes:backward"=>"left|right"',
'"bicycle"=>"no","cycleway:both"=>"no","destination"=>"St. Peter;Puntigam;Center West","destination:ref"=>"B67a","foot"=>"no","lanes"=>"1","lit"=>"yes","maxspeed"=>"50","oneway"=>"yes","ref"=>"B67a","shoulder"=>"right","sidewalk"=>"no","surface"=>"asphalt"',
'"foot"=>"no","lanes"=>"1","maxspeed"=>"30","oneway"=>"yes","service"=>"driveway","sidewalk"=>"no","surface"=>"asphalt","turn:lanes"=>"through;right"',
'"foot"=>"no","lanes"=>"2","oneway"=>"yes","oneway:psv"=>"no","psv"=>"opposite","service"=>"driveway","sidewalk"=>"no","surface"=>"asphalt"']
Length: 5, dtype: str
geometry: <GeometryArray>
[ <LINESTRING (15.477 47.033, 15.477 47.033, 15.476 47.033)>,
<LINESTRING (15.457 47.063, 15.457 47.063, 15.457 47.063, 15.457 47.063, 15....>,
<LINESTRING (15.413 47.033, 15.413 47.033)>,
<LINESTRING (15.419 47.033, 15.419 47.033, 15.419 47.033, 15.419 47.033, 15....>,
<LINESTRING (15.421 47.033, 15.421 47.033, 15.421 47.033, 15.421 47.033, 15....>]
Length: 5, dtype: geometry
34:19
I see the issue - columns like `lanes`, `maxspeed`, `surface`, `oneway` are NOT direct columns in the OSM PBF lines layer. They're stored inside `other_tags` as hstore format. So I need to parse the `other_tags` field to extract these attributes. Let me update the script to parse `other_tags` for both highways and PT routes.
34:19
34:19
Write
/work/solve.py
content · 153 lines · py
"""
Extract highway network and PT route relations intersecting a 500m buffer
around Vienna's Gürtel. Output: vienna_network.gpkg with highways (LineString)
and pt_routes (MultiLineString), EPSG:31287 (MGI / Austria Lambert).
"""
import pyogrio
import geopandas as gpd
import pandas as pd
import numpy as np
import re
import pyproj
from shapely.geometry import LineString, MultiLineString, box
from shapely.ops import unary_union, transform
PBF = "/work/austria-latest.osm.pbf"
OUT = "/work/vienna_network.gpkg"
BUFF_M = 500
CRS_PROJ = "EPSG:31287"
CRS_WGS = "EPSG:4326"
project_fwd = pyproj.Transformer.from_crs(CRS_WGS, CRS_PROJ, always_xy=True).transform
project_rev = pyproj.Transformer.from_crs(CRS_PROJ, CRS_WGS, always_xy=True).transform
def parse_hstore(tag_str):
"""Parse OSM hstore string like '\"key\"=>\"value\",\"key2\"=>\"value2\"' into dict."""
if pd.isna(tag_str) or not isinstance(tag_str, str):
return {}
result = {}
for m in re.finditer(r'"([^"]+)"\s*=>\s*"([^"]*)"', tag_str):
result[m.group(1)] = m.group(2)
return result
# ---------------------------------------------------------------------------
# 1. Find Vienna Gürtel ways (highway, name ends with 'Gürtel')
# ---------------------------------------------------------------------------
print("1. Finding Gürtel ways in Vienna...")
sql = "SELECT * FROM lines WHERE highway IS NOT NULL AND name LIKE '%Gürtel'"
gdf = pyogrio.read_dataframe(PBF, sql=sql)
print(f" Found {len(gdf)} potential Gürtel ways")
gdf = gdf[gdf['name'].str.endswith('Gürtel', na=False)].copy()
vienna_bbox = box(16.1, 48.1, 16.6, 48.35)
gdf = gdf[gdf.intersects(vienna_bbox)].copy()
print(f" Vienna-area Gürtel ways: {len(gdf)}")
if len(gdf) == 0:
raise RuntimeError("No Gürtel ways found in Vienna area")
# ---------------------------------------------------------------------------
# 2. Compute 500m buffer
# ---------------------------------------------------------------------------
print("2. Computing buffer...")
guertel_union = unary_union(gdf.geometry.values)
guertel_proj = transform(project_fwd, guertel_union)
buf_proj = guertel_proj.buffer(BUFF_M)
buf_wgs = transform(project_rev, buf_proj)
buf_bounds_wgs = buf_wgs.bounds
print(f" Buffer bounds WGS84: {buf_bounds_wgs}")
# ---------------------------------------------------------------------------
# 3. Highway ways
# ---------------------------------------------------------------------------
print("3. Getting highway ways intersecting buffer...")
sql_hw = "SELECT * FROM lines WHERE highway IS NOT NULL"
gdf_hw = pyogrio.read_dataframe(PBF, sql=sql_hw, bbox=buf_bounds_wgs)
print(f" Highway ways in buffer bbox: {len(gdf_hw)}")
gdf_hw = gdf_hw[gdf_hw.intersects(buf_wgs)].copy()
print(f" Highway ways intersecting buffer: {len(gdf_hw)}")
# Parse other_tags to extract maxspeed, lanes, surface, oneway
parsed_hw = gdf_hw['other_tags'].apply(parse_hstore)
gdf_hw['maxspeed'] = parsed_hw.apply(lambda d: d.get('maxspeed'))
gdf_hw['lanes'] = parsed_hw.apply(lambda d: d.get('lanes'))
gdf_hw['surface'] = parsed_hw.apply(lambda d: d.get('surface'))
gdf_hw['oneway'] = parsed_hw.apply(lambda d: d.get('oneway'))
keep = ['osm_id', 'name', 'highway', 'maxspeed', 'lanes', 'surface', 'oneway', 'geometry']
for col in keep:
if col not in gdf_hw.columns:
gdf_hw[col] = None
gdf_hw_out = gdf_hw[keep].copy()
gdf_hw_out = gdf_hw_out.to_crs(CRS_PROJ)
gdf_hw_out = gdf_hw_out[gdf_hw_out.geometry.type == 'LineString'].copy()
print(f" Output highways: {len(gdf_hw_out)}")
# Check some values
for c in ['maxspeed','lanes','surface','oneway']:
nz = gdf_hw_out[c].notna().sum()
print(f" {c}: {nz}/{len(gdf_hw_out)} non-null")
# ---------------------------------------------------------------------------
# 4. PT routes from multilinestrings layer
# ---------------------------------------------------------------------------
print("4. Getting PT route relations...")
sql_ml = "SELECT * FROM multilinestrings WHERE type = 'route'"
gdf_ml = pyogrio.read_dataframe(PBF, sql=sql_ml, bbox=buf_bounds_wgs)
print(f" Route multilinestrings in buffer bbox: {len(gdf_ml)}")
parsed_ml = gdf_ml['other_tags'].apply(parse_hstore)
gdf_ml['route'] = parsed_ml.apply(lambda d: d.get('route'))
gdf_ml['ref'] = parsed_ml.apply(lambda d: d.get('ref'))
gdf_ml['operator'] = parsed_ml.apply(lambda d: d.get('operator'))
# The 'name' column from the OSM layer is the route's description/name
pt_types = ('bus', 'tram', 'subway', 'train', 'trolleybus', 'light_rail', 'railway')
gdf_pt = gdf_ml[gdf_ml['route'].isin(pt_types)].copy()
print(f" PT routes in bbox: {len(gdf_pt)}")
# Spatial filter
gdf_pt_wgs = gdf_pt.to_crs(CRS_WGS)
gdf_pt = gdf_pt[gdf_pt_wgs.intersects(buf_wgs)].copy()
print(f" PT routes intersecting buffer: {len(gdf_pt)}")
out_cols = ['osm_id', 'ref', 'name', 'operator', 'route', 'geometry']
for col in out_cols:
if col not in gdf_pt.columns:
if col == 'osm_id':
gdf_pt[col] = 0
else:
gdf_pt[col] = None
gdf_pt_out = gdf_pt[out_cols].copy()
gdf_pt_out = gdf_pt_out.to_crs(CRS_PROJ)
# Ensure MultiLineString
mask_ls = gdf_pt_out.geometry.type == 'LineString'
if mask_ls.any():
gdf_pt_out.loc[mask_ls, 'geometry'] = gdf_pt_out.loc[mask_ls, 'geometry'].apply(
lambda g: MultiLineString([g])
)
gdf_pt_out = gdf_pt_out[gdf_pt_out.geometry.type == 'MultiLineString'].copy()
print(f" Output PT routes: {len(gdf_pt_out)}")
# ---------------------------------------------------------------------------
# 5. Write GeoPackage
# ---------------------------------------------------------------------------
print("5. Writing GeoPackage...")
gdf_hw_out.to_file(OUT, layer='highways', driver='GPKG')
print(f" Highways: {len(gdf_hw_out)} features written")
if len(gdf_pt_out) > 0:
gdf_pt_out.to_file(OUT, layer='pt_routes', driver='GPKG')
print(f" PT routes: {len(gdf_pt_out)} features written")
else:
print(" WARNING: No PT routes to write")
print("Done!")
Wrote /work/solve.py (6021 chars).
34:25
Bash
cd /work && timeout 300 python3 solve.py 2>&1
1. Finding Gürtel ways in Vienna...
Found 394 potential Gürtel ways
Vienna-area Gürtel ways: 218
2. Computing buffer...
Buffer bounds WGS84: (16.32985446408538, 48.175697404107744, 16.40540552225453, 48.240822338117724)
3. Getting highway ways intersecting buffer...
Highway ways in buffer bbox: 33273
Highway ways intersecting buffer: 9230
Output highways: 9230
maxspeed: 2375/9230 non-null
lanes: 1214/9230 non-null
surface: 7519/9230 non-null
oneway: 2314/9230 non-null
4. Getting PT route relations...
Route multilinestrings in buffer bbox: 672
PT routes in bbox: 487
PT routes intersecting buffer: 383
Output PT routes: 383
5. Writing GeoPackage...
Highways: 9230 features written
PT routes: 383 features written
Done!
36:55
Now maxspeed, lanes, surface, and oneway have proper values. Let me do a final verification.
36:55
36:55
Bash
cd /work && timeout 30 python3 -c "
cd /work && timeout 30 python3 -c "
import pyogrio, geopandas as gpd
# Check layers
layers = pyogrio.list_layers('/work/vienna_network.gpkg')
print('Layers:', layers)
# Highways
hw = pyogrio.read_dataframe('/work/vienna_network.gpkg', layer='highways')
print(f'\n=== HIGHWAYS ===')
print(f'Features: {len(hw)}')
print(f'CRS: {hw.crs}')
print(f'Geometry types: {hw.geometry.type.unique()}')
print(f'Attributes:')
for c in ['name','highway','maxspeed','lanes','surface','oneway']:
nonnull = hw[c].notna().sum()
if nonnull > 0:
vals = hw[c].dropna().unique()
if len(vals) > 10:
vals = vals[:10]
print(f' {c}: {nonnull}/{len(hw)} non-null, sample={vals}')
# PT routes
pt = pyogrio.read_dataframe('/work/vienna_network.gpkg', layer='pt_routes')
print(f'\n=== PT ROUTES ===')
print(f'Features: {len(pt)}')
print(f'CRS: {pt.crs}')
print(f'Geometry types: {pt.geometry.type.unique()}')
print(f'Attributes:')
for c in ['ref','name','operator','route']:
nonnull = pt[c].notna().sum()
if nonnull > 0:
vals = pt[c].dropna().unique()
if len(vals) > 10:
vals = vals[:10]
print(f' {c}: {nonnull}/{len(pt)} non-null, sample={vals}')
print(f'\nRoute type distribution:')
print(pt['route'].value_counts().to_string())
# Check file size
import os
size_mb = os.path.getsize('/work/vienna_network.gpkg') / 1e6
print(f'\nFile size: {size_mb:.1f} MB')
" 2>&1
Layers: [['highways' 'LineString']
['pt_routes' 'MultiLineString']]
=== HIGHWAYS ===
Features: 9230
CRS: EPSG:31287
Geometry types: <ArrowStringArray>
['LineString']
Length: 1, dtype: str
Attributes:
name: 2554/9230 non-null, sample=<ArrowStringArray>
['Anton-Schmid-Promenade', 'Brigittenauer Lände', 'Koppstraße',
'Währinger Gürtel', 'Heiligenstädter Straße', 'Landstraßer Gürtel',
'Heiligenstädter Lände', 'Einsiedlergasse', 'Margaretengürtel',
'Hofferplatz']
Length: 10, dtype: str
highway: 9230/9230 non-null, sample=<ArrowStringArray>
[ 'cycleway', 'trunk_link', 'primary', 'secondary',
'motorway_link', 'trunk', 'residential', 'tertiary',
'living_street', 'construction']
Length: 10, dtype: str
maxspeed: 2375/9230 non-null, sample=<ArrowStringArray>
['50', '80', '70', '30', 'walk', '5', '10', '20', '15']
Length: 9, dtype: str
lanes: 1214/9230 non-null, sample=<ArrowStringArray>
['1', '2', '4', '3', '7', '5', '6']
Length: 7, dtype: str
surface: 7519/9230 non-null, sample=<ArrowStringArray>
[ 'asphalt', 'concrete', 'paving_stones', 'sett',
'concrete:plates', 'gravel', 'fine_gravel', 'paved',
'cobblestone', 'compacted']
Length: 10, dtype: str
oneway: 2314/9230 non-null, sample=<ArrowStringArray>
['yes', 'no', '-1', 'reversible']
Length: 4, dtype: str
=== PT ROUTES ===
Features: 383
CRS: EPSG:31287
Geometry types: <ArrowStringArray>
['MultiLineString']
Length: 1, dtype: str
Attributes:
ref: 382/383 non-null, sample=<ArrowStringArray>
['S2', 'U3', 'U1', '35A', '40A', 'S1', 'S4', 'S3', 'S7', 'N41']
Length: 10, dtype: str
name: 383/383 non-null, sample=<ArrowStringArray>
[ 'S2: Mödling => Laa',
'U3: Ottakring => Simmering',
'U1: Oberlaa => Leopoldau',
'Bus 35A: Spittelau => Salmannsdorf',
'Bus 40A: Schottentor => Döblinger Friedhof/Felix-Dahn-Straße',
'S2: Laa => Mödling',
'S1: Marchegg => Wien Meidling',
'S4: Wiener Neustadt => Tullnerfeld',
'S3: Hollabrunn => Wiener Neustadt',
'S7: Mistelbach => Wolfsthal']
Length: 10, dtype: str
operator: 369/383 non-null, sample=<ArrowStringArray>
[ 'ÖBB-Personenverkehr AG',
'Wiener Linien',
'Gschwindl',
'DB Fernverkehr AG;ÖBB',
'ÖBB',
'Wiener Lokalbahnen',
'Dr. Richard',
'ÖBB;ZSSK',
'ÖBB-Personenverkehr;České dráhy',
'City Air Terminal Betriebsgesellschaft m.b.H.']
Length: 10, dtype: str
route: 383/383 non-null, sample=<ArrowStringArray>
['train', 'subway', 'bus', 'tram', 'railway', 'light_rail']
Length: 6, dtype: str
Route type distribution:
route
bus 219
train 86
tram 49
railway 17
subway 10
light_rail 2
File size: 22.2 MB
37:05
The output looks great. Let me clean up the debug buffer layer that was added earlier. Actually, looking at the code, I wrote a `_debug_buffer` layer in the first version but the script was rewritten. Let me check if there's a stray layer. Actually, looking at my first script, I wrote `buf_gdf.to_file(OUT, layer='_debug_buffer', driver='GPKG')` in the first attempt, but then I rewrote the script. However, since the file was overwritten by the new script's `to_file` calls, the old buffer layer shouldn't be there. The layers listed show only `highways` and `pt_routes`. So it's clean. Let me also remove the downloaded PBF to save space if needed.
37:05
37:05
Bash
rm /work/austria-latest.osm.pbf && ls -lh /work/vienna_network.gpkg
-rw-r--r-- 1 agent agent 22M Jun 18 06:49 /work/vienna_network.gpkg
37:13
The solution is complete. Let me summarize what was done.
37:13