Data Commons on Spanner Graph: Unify Public & Private Data

Ever tried to mix your own data with the world’s public datasets? It’s like trying to merge a swamp with the ocean—messy, murky, and full of unexpected currents. Google’s Data Commons just made that a whole lot easier with the general availability of Data Commons on Spanner Graph and a preview of the new Data Commons Platform.

Think of it as a friendly capybara building a bridge between your private pond and the vast public reservoir. You get to keep your data isolated while tapping into over 400 billion data points from 100+ authoritative sources like the UN, World Bank, and NOAA—all normalized and ready to swim with.

Why Spanner Graph? Because Caches Are So Last Season

Data Commons used to rely on Bigtable caching to handle massive lookups. That worked, but it was like storing the whole swamp in a bucket—rigid and hard to update. Now, with Spanner Graph, they’ve moved to a native graph model that supports SQL-like queries and GQL (Graph Query Language).

This means you can run complex relationship queries directly in the database, without pre-computed caches. It’s like having a living map of the swamp that updates itself as new data flows in, instead of a static snapshot that goes stale.

Key Benefits of the New Architecture

• Unified storage & incremental updates: Say goodbye to full refreshes. Update specific datasets without rebuilding the whole graph.
• Dynamic graph traversals via GraphRAG: Multi-hop queries like continent → country → state → county → city run on the fly, enabling natural language queries to map directly to graph paths.
• Consistent data snapshots: Spanner’s TrueTime and stale reads ensure you always see a consistent version of the data, even during ingestion.
• Operational analytics at scale: The columnar engine scans massive time-series data efficiently, and BigQuery federation via Data Boost keeps production traffic isolated.

SDMX 3.0: Speaking the Language of Statistics

Data Commons now supports SDMX 3.0, the ISO standard for statistical data exchange. This means out-of-the-box integration with tools like Tableau, Flourish, and Observable. Two new endpoints—the availability API and the data API—let you discover dimensions and fetch observations without breaking your code when new variables are added.

Private Instances: Your Data, Your Rules

Want to build your own private Data Commons instance? You can now spin one up using the same scalable tech that powers Google’s public instance. You keep full control over your data, limit access as needed, and still query across both your private graph and Google’s public one.

Imagine a retail company combining national GDP trends with their own sales data to spot untapped markets. That’s the kind of cross-pollination that makes this a game-changer for analysts and developers alike.

Get Started

• Explore the public Data Commons at datacommons.org.
• Check out Spanner Graph use cases and the setup guide.
• Request preview access to the Data Commons Platform by emailing [email protected].

This is a big leap for knowledge graphs, and it’s great to see Google making public data more accessible and combinable with your own. If you’re already swimming in the Google Cloud pond, this might be the perfect time to dive deeper. And if you’re comparing cloud options, check out our Google Cloud review to see how it stacks up against the caimans.

Original announcement published on Google Cloud.