Dewey Dunnington's Avatar

Dewey Dunnington

@paleolimbot

Scaling spatial computing at Wherobots, Inc. ApacheArrow PMC, #gischat, #rstats, ex Voltron Data.

218
Followers
81
Following
69
Posts
29.10.2024
Joined
Posts Following

Latest posts by Dewey Dunnington @paleolimbot

SedonaDB 0.3.0 Release - Apache Sedona Apache Sedona is a cluster computing system for processing large-scale spatial data. Sedona extends existing cluster computing systems, such as Apache Spark, Apache Flink, and Snowflake, with a set of...

Check all this out and more in the release post! sedona.apache.org/latest/blog/...

11.03.2026 02:31 πŸ‘ 0 πŸ” 0 πŸ’¬ 0 πŸ“Œ 0
Post image

Finally, we added the beginnings of an #rstats DataFrame API that can be used to implement a #dplyr backend. This is my favourite feature of SedonaDB 0.3.0 because I love R, dplyr, and because I never properly learned SQL 😬

11.03.2026 02:31 πŸ‘ 3 πŸ” 0 πŸ’¬ 1 πŸ“Œ 0
Post image

#rstats users will be pleased to know that you can now read anything sf can piped directly into SedonaDB via GDAL's @arrow.apache.org integration. This makes the SedonaDB R package considerably more useful!

11.03.2026 02:31 πŸ‘ 6 πŸ” 1 πŸ’¬ 1 πŸ“Œ 0
Post image

We also added a lot of functions (with full 4D and geometry type support with PostGIS integration tests, as usual!)

11.03.2026 02:31 πŸ‘ 0 πŸ” 0 πŸ’¬ 1 πŸ“Œ 0
Post image

We've always had a great Parquet writer; however, Parquet is new on the block and sometimes you just need a Shapefile (or GeoPackage, or FlatGeoBuf). This was always possible with SedonaDB and pyogrio's Arrow integration but in 0.3.0 we gave it some nice defaults and made it easy to do.

11.03.2026 02:31 πŸ‘ 0 πŸ” 0 πŸ’¬ 1 πŸ“Œ 0
Post image

Ever since reading @opencholmes.bsky.social's fantastic GeoParquet best practices guide, I've wanted to make that trivial to do with SedonaDB's Parquet writer. After this release...it is! Sort, compress, and reduce your row group size based on the results of any query all in one go!

11.03.2026 02:31 πŸ‘ 1 πŸ” 0 πŸ’¬ 1 πŸ“Œ 0
Post image

One of the most commonly requested features by those of us who wrote SedonaDB code frequently was parameterized SQL queries! Now you can add a placeholder and bind just about any spatial object you can think of (CRS included!). Let us know if we missed your favourite one!

11.03.2026 02:31 πŸ‘ 0 πŸ” 0 πŸ’¬ 1 πŸ“Œ 0
Post image

SedonaDB can now represent geometries with a separate CRS per row (like @postgis.bsky.social/EWKB), including transforming to and from with CRSes derived from a column. If you've ever been sent a spreadsheet with UTM coordinates with a "zone" column, this feature is for you!

11.03.2026 02:31 πŸ‘ 0 πŸ” 0 πŸ’¬ 1 πŸ“Œ 0
Post image

The new spatial join gives more flexibility for running SedonaDB in memory constrained environments and increases the size of the data you can swing around on your laptop. It's currently opt-in (requires setting a memory limit)...give it a go and let us know how it goes!

11.03.2026 02:31 πŸ‘ 0 πŸ” 0 πŸ’¬ 1 πŸ“Œ 0
Post image

We're chuffed to announce Apache SedonaDB 0.3.0! This release features a rewritten join that supports larger-than-memory spatial/KNN joins courtesy of Kristin Cowalcijk, new functions, parameterized SQL queries, GDAL/pyogrio reads, GDAL/sf based reads in R, and the beginnings of an R DataFrame API!

11.03.2026 02:31 πŸ‘ 4 πŸ” 2 πŸ’¬ 1 πŸ“Œ 0
Post image

Does SedonaDB's KNN join help at all? I haven't wired up nice dplyrish syntax for everything yet but the SQL isn't too bad.

24.02.2026 04:26 πŸ‘ 1 πŸ” 0 πŸ’¬ 1 πŸ“Œ 0
Post image

πŸ“– Apache Parquet recently added native support for Geospatial. This post explains what that means and why it is important: parquet.apache.org/blog/2026/02...

13.02.2026 13:56 πŸ‘ 13 πŸ” 2 πŸ’¬ 0 πŸ“Œ 0
Native Geospatial Types in Apache Parquet Native Geospatial Types in Apache Parquet

Great inaugural post about the geospatial types on the Parquet blog.

Thank you Jia Yu, Dewey Dunnington , Kristin Cowalcijk, Feng Zhang.

More posts coming !

parquet.apache.org/blog/2026/02...

14.02.2026 00:36 πŸ‘ 8 πŸ” 2 πŸ’¬ 0 πŸ“Œ 0
Post image

Released this week: Version 22 of the ADBC libraries and drivers.

This release includes updates to the ADBC libraries for 8 languages, and improvements to the 4 ADBC drivers that are maintained in the apache/arrow-adbc repository. See the blog for more details: arrow.apache.org/blog/2026/01...

11.01.2026 00:18 πŸ‘ 13 πŸ” 3 πŸ’¬ 0 πŸ“Œ 0
Preview
GitHub - brownag/gdalcli: An R Frontend for the GDAL CLI An R Frontend for the GDAL CLI. Contribute to brownag/gdalcli development by creating an account on GitHub.

Introducing gdalcli by Andrew Brown -- an R frontend to GDAL’s unified CLI (β‰₯3.11) 🌐

Compose and execute GDAL workflows with pipe-friendly functions.

Learn more: github.com/brownag/gdal...

#RStats #GDAL #Geospatial #OpenSource #RSpatial

04.01.2026 15:01 πŸ‘ 37 πŸ” 13 πŸ’¬ 1 πŸ“Œ 0
Preview
Testing the R-universe build workflow from your own GitHub repository We refactored the R-universe CI workflows to make it possible to run the exact same workflow from your own GitHub repository.

ropensci.org/blog/2026/01...

03.01.2026 18:37 πŸ‘ 6 πŸ” 3 πŸ’¬ 0 πŸ“Œ 0
Post image

Apache SedonaDB 0.2.0 is now available. Download here: buff.ly/k9LRlyC

SedonaDB is the first open source, single-node analytical database engine that treats spatial data as a first-class citizen. It is developed as a subproject of Apache Sedona. #opensource

23.12.2025 23:30 πŸ‘ 6 πŸ” 2 πŸ’¬ 0 πŸ“Œ 0

I taught a 70 person section of intro to computer science and the automated checks were a total godsend. (I also ran out of GitHub Actions minutes almost immediately...)

19.12.2025 21:00 πŸ‘ 2 πŸ” 0 πŸ’¬ 0 πŸ“Œ 0
SedonaDB 0.2.0 Release - Apache Sedona Apache Sedona is a cluster computing system for processing large-scale spatial data. Sedona extends existing cluster computing systems, such as Apache Spark, Apache Flink, and Snowflake, with a set of...

For the full scoop, check out our release post! sedona.apache.org/latest/blog/...

09.12.2025 20:03 πŸ‘ 0 πŸ” 0 πŸ’¬ 0 πŸ“Œ 0
Post image

Finally...we're now on crates.io! We're still learning how to support Rust projects that are interested in spatial support...leave a note on our repo if you're interested!

09.12.2025 20:03 πŸ‘ 0 πŸ” 0 πŸ’¬ 1 πŸ“Œ 0
Post image

We were fortunate to have over 40 functions contributed since our last release, but if we didn't get to your favourite you can now patch it in via pyarrow and/or shapely!

09.12.2025 20:03 πŸ‘ 1 πŸ” 0 πŸ’¬ 1 πŸ“Œ 0
Post image

On the write side, SedonaDB can now write GeoParquet 1.1 files with spatial sorting and limited row group size to optimize partial reads when hosting or viewing large files.

09.12.2025 20:03 πŸ‘ 1 πŸ” 0 πŸ’¬ 1 πŸ“Œ 0
Post image

Just like our GeoParquet suport, if your query contains one or more WHERE ST_Intersects() (or similar), this is automatically pushed into the data source to read as little as possible!

09.12.2025 20:03 πŸ‘ 1 πŸ” 0 πŸ’¬ 1 πŸ“Œ 0
Post image

We're proud to officially be GDAL in a trenchcoat! You can now query GeoPackage, FlatGeoBuf, Shapefile, or anything else supported by GDAL/OGR, including URLs, zipped files, directories, and globs.

09.12.2025 20:03 πŸ‘ 0 πŸ” 0 πŸ’¬ 1 πŸ“Œ 0
Post image

Excited to announce SedonaDB 0.2.0! Highlights include reading spatial file formats via GDAL/OGR, improved GeoParquet IO, Python @arrow.apache.org UDFs, and 40+ additional functions.

09.12.2025 20:03 πŸ‘ 6 πŸ” 2 πŸ’¬ 1 πŸ“Œ 0
Apache DataFusion Boston Meetup: September 12, 2025
Apache DataFusion Boston Meetup: September 12, 2025 YouTube video by Andrew Lamb

Here are the slides and recordings from our Boston DataFusion Meetup in September:

Youtube: youtu.be/wCAud478Dg8
Slides (pdf): drive.google.com/file/d/18KGH...

04.12.2025 11:51 πŸ‘ 10 πŸ” 2 πŸ’¬ 0 πŸ“Œ 0
Preview
FOSS4G Auckland 2025 Kia ora! Last week, we were in Auckland, New Zealand, for FOSS4G 2025. We had a great time and gave...

At FOSS4G Auckland 2025, we, MIERUNE Inc., had four talks about various topics from GTFS to GeoArrow. This is a quick summary of the presentation titles and the links to the slidesπŸ”—

dev.to/mierune/foss...

30.11.2025 02:41 πŸ‘ 3 πŸ” 1 πŸ’¬ 0 πŸ“Œ 0
Preview
Implement QgsArrowIterator to iterate over features as batches of ArrowArray by paleolimbot Β· Pull Request #63749 Β· qgis/QGIS Description As encouraged by @nyalldawson! https://mastodon.social/@nyalld/115459416976982489 The motivation is to eliminate the need for per-feature iteration in Python to maintain the fidelity of...

Wohoo, QGIS Arrow support has been merged: https://github.com/qgis/QGIS/pull/63749

Thanks @paleolimbot & @nyalld et al.

Looking forward to testing it out soon 🀩

#QGIS #Arrow #GISChat

28.11.2025 08:44 πŸ‘ 11 πŸ” 9 πŸ’¬ 0 πŸ“Œ 0
Preview
QOD November 2025 QGIS is a free, open source, cross platform (lin/win/mac) geographical information system (GIS) - qgis/QGIS

Starring in a few minutes: It’s QGIS Open Day today! πŸŽ‰ Don’t miss our two awesome talks. Grab all links and info on the wiki: https://github.com/qgis/QGIS/wiki/QOD-November-2025

#QGIS #GISChat

28.11.2025 13:38 πŸ‘ 6 πŸ” 3 πŸ’¬ 0 πŸ“Œ 0