Geospatial Data in Apache Iceberg: Geometry, Geography, and GeoParquet

A logistics team stores 40 million delivery stops in an Apache Iceberg table. Every row has a latitude and a longitude. The analyst wants every stop inside a polygon that outlines one metro area. The query engine scans every data file in the table, because nothing in the table metadata tells it which files contain points inside that polygon. Forty million rows get read to return two hundred thousand. That was the normal state of spatial data on the lakehouse for most of a decade. Coordinates lived in two double columns or in an opaque binary column. The table format did not know the column was spatial. The file format did not know either. Every optimization that Iceberg applies to timestamps, integers, and strings, from min/max pruning to partition transforms, simply did not apply. Iceberg format version 3 changes this by adding two native primitive types: geometry and geography. Apache Parquet 2.11 added matching logical types at the file level. Together they give spatial data the same standing as any other column: a declared type, a coordinate reference system that travels with the schema, and per-file bounding-box statistics that let an engine skip files before reading a single shape. This article explains the mechanism. It covers what the two types mean, how coordinate reference systems and edge interpolation are encoded, how bounding boxes are stored and used for pruning, how the Iceberg types relate to Parquet and to the older GeoParquet convention, and what breaks when you deploy this in production. I work at Dremio, which ships Iceberg v3 support, but the material here is spec-level and applies to any engine. How Spatial Data Lived in Tables Before v3 Before format version 3, an Iceberg table had no vocabulary for a shape. Teams picked from a short list of workarounds, and every option lost something. The simplest approach stored longitude and latitude as two double columns. This works for points and nothing else. A polygon, a route, or a service boundary cannot fit in two numbers. Min/max statistics on the two columns do give you crude bounding-box pruning for point data, which is why many teams stuck with this pattern for years. The more general approach stored shapes as Well-Known Binary (WKB) in a binary column, or Well-Known Text (WKT) in a string column. WKB is the Open Geospatial Consortium (OGC) standard byte encoding for points, lines, polygons, and their multi-part variants. Every spatial library reads it. The problem is that the Iceberg schema saw only binary. The manifest recorded byte-wise min and max bounds for the column, which are meaningless for pruning. No engine skipped a file based on those bounds. Every spatial predicate became a full scan followed by row-by-row geometry parsing. The coordinate reference system (CRS) was the other casualty. A CRS defines how a pair of numbers maps to a location on Earth. Longitude 30, latitude 10 means one place under WGS84 and a completely different place under a projected national grid. With a plain binary column, the CRS lived in a wiki page, a column comment, or someone's memory. Two teams writing to the same table with different assumptions produced silent corruption that no validation caught. Engines with spatial support, such as Apache Sedona, built their own conventions on top of Iceberg to fill the gap. Sedona's Havasu extension added CRS metadata, bounding-box statistics, and format annotations through a fork of Iceberg. This worked for Sedona users but did not travel. A Sedona-written table opened in another engine went back to being bytes. The v3 spec work pulled these ideas into the standard. The design was driven largely by the Wherobots team, who had run the Havasu approach in production since 2022 and contributed the design upstream to both Parquet and Iceberg. The Parquet logical type proposal collected over 400 review comments. The Iceberg type spec collected 240 more. That review volume is a sign of how many decisions hide inside "just add a geometry type." Geometry Versus Geography: Two Types, Two Models of the Earth Iceberg v3 defines two spatial types rather than one because there are two different ways to compute with coordinates, and mixing them produces wrong answers. The geometry type treats coordinates as points on a flat plane. Distance is Euclidean. A line between two points is straight in the coordinate space. This is the right model for data in a projected CRS such as a state plane or UTM zone, where the projection has already flattened a region of the Earth onto a plane. It is also the right model for non-geographic data such as floor plans, chip layouts, or any coordinate system where "the Earth is round" is not a relevant fact. The geography type treats coordinates as positions on the surface of an ellipsoid or sphere. A line between two points follows a geodesic, the shortest path over the curved surface, rather than a straight line in longitude and latitude. Distance is computed along that surface. This is the right model for global data stored in longitude and latitude, where a "straight" line across a thousand kilometers in planar math bends noticeably away from the true shortest path. The difference shows up in ordinary queries. Take two airports 8,000 kilometers apart. Planar distance on raw longitude and latitude gives a number in degrees that means nothing. Geodesic distance gives kilometers. Take a polygon that covers Alaska. Under planar math its western edge crosses the antimeridian at longitude 180 and the polygon appears to wrap around the entire planet. Under geographic math the polygon is a small region on a sphere and behaves correctly. The spec encodes this distinction in the type definitions. geometry(C) is parameterized by a CRS C. geography(C, A) is parameterized by a CRS C and an edge-interpolation algorithm A. Both default the CRS to OGC:CRS84, which means longitude and latitude on the WGS84 datum with longitude first. Geography defaults the algorithm to spherical. The choice between them is not cosmetic. An engine reading a geometry column runs Cartesian computations regardless of what CRS string is attached. The spec states this directly: for geometry, the CRS does not affect geometric calculations. The CRS is carried as metadata so downstream tools can reproject or display correctly, but the storage layer computes on a plane. If your longitude-latitude data needs correct global distances and containment, geography is the type that asks for that. Coordinate Reference Systems and Edge Interpolation in the Schema The CRS parameter is a string, and the spec is deliberate about what that string can and cannot contain. The recommended form is :. Examples from the spec are OGC:CRS84, EPSG:4326, IGNF:ATI, and SRID:0. The EPSG registry (originally the European Petroleum Survey Group) is the most widely used catalog of CRS definitions, and EPSG:4326 is the code for WGS84 with latitude-first axis order. OGC:CRS84 is the same datum with longitude-first order, which matches the WKB convention of X then Y. The default is OGC:CRS84 for exactly that reason: WKB always stores X (longitude or easting) before Y (latitude or northing), so the default CRS declares the same order. For a custom CRS that does not have a registry code, the spec allows a reference of the form projjson:. PROJJSON is the JSON encoding of a CRS definition from the PROJ library. The definition itself goes in a table property under that name, and the type string only points to it. The spec forbids inlining PROJJSON directly into the type string and forbids implementations from parsing the type string as PROJJSON. The reason is size. A full PROJJSON definition runs to kilobytes, and the schema is embedded in every metadata file and every manifest list. Inlining it bloats metadata reads across the whole table. For geography, the CRS has an added constraint: it must be geographic, with longitudes in [-180, 180] and latitudes in [-90, 90]. A projected CRS on a geography column is invalid. The edge-interpolation algorithm A on geography selects how the engine computes the curve between two vertices. The spec lists five values: spherical: edges are geodesics on a perfect sphere. Cheapest to compute, accurate to within about 0.3 percent for most distances. The default. vincenty: Vincenty's iterative formulae on the ellipsoid. Accurate to millimeters, fails to converge for nearly antipodal points. thomas: Paul Thomas's 1970 spheroidal geodesic method. andoyer: Thomas's 1965 navigation model, a lower-cost ellipsoidal approximation. karney: Charles Karney's 2013 algorithm as implemented in GeographicLib. Converges everywhere and is accurate to nanometers. Most teams never change this from spherical. The parameter exists so that two engines reading the same table agree on what "the edge between these two points" means. If a writer computed containment using Karney geodesics and a reader used spherical ones, a point sitting a few meters from a polygon boundary flips between inside and outside depending on who asks. Storing the algorithm in the type removes that ambiguity. In the schema JSON, the types serialize as strings. A geometry column in a default CRS is written as "geometry". With a custom CRS it becomes "geometry(srid:4326)". A geography column with both parameters looks like "geography(srid:4326, spherical)". Any engine that already parses Iceberg type strings extends its parser to handle the parenthesized parameters. What the Type Changes in Metadata, Files, and Partitioning Adding a type to a table format touches more than the schema. Several rules in the v3 spec exist only because these two types exist. Default values are restricted. Iceberg v3 introduced initial-default and write-default so a column added later can be populated for old rows without rewriting files. For geometry and geography, along with variant and unknown, the spec requires that both defaults be null. A non-null default for a shape column is invalid. This avoids embedding WKB byte strings inside the schema JSON, and it sidesteps the question of what a "default polygon" even means. Partition transforms are limited. The identity transform is defined for every primitive type except geometry and geography. The bucket transform's list of valid source types does not include them either. You cannot partition directly on a shape column. The reasons are practical. Identity partitioning on a polygon produces one partition per distinct polygon, which is useless. Bucketing by hash of the WKB bytes scatters spatially adjacent shapes across buckets at random, which defeats the point of spatial locality. Spatial partitioning is done today through derived columns, covered later in this article. Physical storage is WKB everywhere. In Avro, both types map to bytes in WKB. In Parquet, both map to binary, annotated with the GEOMETRY or GEOGRAPHY logical type where the writer supports it. In ORC, both map to binary with an iceberg.binary-type attribute set to GEOMETRY or GEOGRAPHY, because ORC has no native spatial logical type. Single-value serialization for partition values and bounds uses WKB. JSON serialization, used in places like default values and some REST catalog payloads, uses WKT so the value is human-readable. The Parquet logical type is what makes cross-engine reads work. This point deserves emphasis. If a writer produces a Parquet file with a plain binary column and no logical type annotation, a reader that opens that file without the Iceberg schema sees bytes. The PyIceberg implementation notes this explicitly: binary columns cannot be distinguished from geometry without the Iceberg schema metadata. When the writer applies the Parquet GEOMETRY logical type, the file itself declares the column as spatial, and any Parquet reader that understands Parquet 2.11 recognizes it. That is the difference between spatial data that works in one engine and spatial data that works everywhere. The Parquet logical type also carries the CRS. Parquet's GEOMETRY and GEOGRAPHY types have their own CRS field and, for geography, their own edge algorithm field. Iceberg writers set these to match the Iceberg type parameters. A file written for a geography(OGC:CRS84, karney) column carries that same CRS and algorithm in its Parquet footer. Readers that trust the Parquet footer and readers that trust the Iceberg schema arrive at the same answer. Bounding Boxes: How Files Get Skipped The most valuable thing the v3 types add is a per-file bounding box that the query planner reads from the manifest. This is the mechanism that turns a 40-million-row scan into a handful of files. For every primitive column, Iceberg manifests store lower_bounds and upper_bounds. For an integer column these are the smallest and largest values in the file. For a geometry or geography column, the spec defines the bounds as two points. The lower bound is a point whose X, Y, and optional Z and M coordinates are each the minimum of that coordinate across every shape in the file. The upper bound is the point of maximums. Together they define the axis-aligned bounding box that contains every object in the file. Z is elevation and M is a fourth measure such as a milepost or timestamp. Both are optional in WKB. The spec handles missing dimensions carefully. Null or NaN coordinate values are skipped during bound computation. If a dimension has only null or NaN values across the whole file, that dimension is omitted from the box. If either X or Y is missing entirely, no bounding box is produced at all, because a box without both planar axes cannot prune anything. In v3, the two bound points are serialized as raw binary: an x:y:z:m concatenation of 8-byte little-endian IEEE 754 doubles. X and Y are mandatory. The encoding shrinks to x:y when Z and M are absent, x:y:z when only M is absent, and x:y:NaN:m when only Z is absent. The NaN placeholder keeps the byte offsets unambiguous. In v4, the bounds move into typed structs called geo_lower and geo_upper inside the new content_stats structure. Each struct has required x and y doubles and optional z and m doubles. The struct field IDs are assigned by fixed offsets within the column's stats ID range, so a geometry column with field ID 4 gets its lower-bound X at stats ID 10,810 and its upper-bound X at 10,814. The information is the same as v3. The difference is that engines read typed fields instead of parsing a variable-length byte array. The geography type has one special rule for bounding boxes that catches people out. For geography columns, the X value of the lower bound is allowed to be greater than the X value of the upper bound. This encodes a box that crosses the antimeridian at longitude 180. A file containing shapes around Fiji, which straddles that line, gets a lower X of 178 and an upper X of negative 179. Under normal min/max logic that box is empty. Under the geography rule, an object matches if its X satisfies x >= xmin OR x