Skip to content

Add Getting Data section with Ibis #104

@ncclementi

Description

@ncclementi

Hello, I work on Ibis and with the default backend (duckdb) we can query overture maps very easily.

Would you be open to a PR contribution to your docs, to add this to the "Getting Data Section"? I can work on multiple examples, but to start it would be something like this.

# install instructions
$ pip install 'ibis-framework[duckdb,geospatial]'
import ibis
from ibis import _

# by default it gets duckdb backend
con = ibis.get_backend()

# example of how to get infrastructure data of `subtype="power"` for a `bbox` around Washington DC 
# and selecting only a few columns

url = "s3://overturemaps-us-west-2/release/2024-07-22.0/theme=base/type=infrastructure/*"
t = con.read_parquet(url, table_name="infra-power-dc")

# so far no execution 
expr = t.filter(_.bbox.xmin > -77.119795,
                _.bbox.xmax < -76.909366,
                _.bbox.ymin > 38.791631,
                _.bbox.ymax < 38.995968,
                _.subtype=="power"
                ).select(["names",
                          "geometry", 
                          "sources", 
                          "class"])

con.to_parquet(expr, "infra-power-dc-ibis.parquet")

t_power_dc = con.read_parquet("infra-power-dc-ibis.parquet") 

# turn interactive mode to preview data
ibis.options.interactive = True   

# inspect data where `class=="power_line", and select only few columns

t_infra.filter(t_infra["class"]=="power_line")["names", "geometry","class"]

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ namesgeometryclass      ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ struct<primary: string, common: map<string, string>, rules: array<struct<varian… │ geospatial:geometrystring     │
├──────────────────────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────────┼────────────┤
│ NULL<LINESTRING (-77.113 38.805, -77.113 38.805, -77.112 38.806, -77.111 38.806,...>power_line │
│ NULL<LINESTRING (-77.11 38.818, -77.11 38.818, -77.111 38.818, -77.112 38.819, -...>power_line │
│ NULL<LINESTRING (-77.098 38.848, -77.097 38.848, -77.097 38.848, -77.097 38.848,...>power_line │
│ {'primary': '230kV transmission line', 'common': None, ... +1}                   │ <LINESTRING (-77.097 38.848, -77.096 38.848, -77.095 38.847, -77.094 38.847,...>power_line │
│ {'primary': '230kV transmission line', 'common': None, ... +1}                   │ <LINESTRING (-77.059 38.796, -77.061 38.796)>power_line │
│ NULL<LINESTRING (-77.06 38.799, -77.06 38.799, -77.061 38.797, -77.061 38.796)>power_line │
│ {'primary': '230kV transmission line', 'common': None, ... +1}                   │ <LINESTRING (-77.057 38.795, -77.059 38.796)>power_line │
│ {'primary': '230kV transmission line', 'common': None, ... +1}                   │ <LINESTRING (-77.055 38.797, -77.055 38.796, -77.055 38.796, -77.055 38.795,...>power_line │
│ {'primary': '230kV transmission line', 'common': None, ... +1}                   │ <LINESTRING (-77.055 38.797, -77.055 38.796, -77.055 38.795, -77.057 38.795)>power_line │
│ {'primary': '230kV transmission line', 'common': None, ... +1}                   │ <LINESTRING (-77.087 38.844, -77.086 38.844, -77.085 38.844, -77.084 38.844,...>power_line │
│ …                                                                                │ …                                                                                │ …          │
└──────────────────────────────────────────────────────────────────────────────────┴──────────────────────────────────────────────────────────────────────────────────┴────────────┘                     

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions