Meet Google Spanner: The Largest, Single Database Ever Created

The evolution of Google from a search engine company renting a garage, to one of the largest companies on the planet, is a story that can truly boggle the mind. One cause of Google's growth is their investment in developing new technology. This is the case with the Google Spanner, which is a new technology designed to manage Google's growth.

At the heart of the Google empire is data. Google houses all of their data from all of their various ventures (YouTube, Gmail, Google Docs., etc.), in giant warehouses all around the world called data centers. Each data center is packed full of servers that stores the world's data on thousands of hard drives. Google's showcase data center, in their hometown of Mountain View California, has an estimated 45,000 servers, and this is just one of perhaps a dozen or more data centers around the world.

Google has hundreds of thousands of servers operating all around the globe, Datacenterknowledge.com estimates Google operates 900,000 servers based on power consumption reports released by Google. Herein lies the challenge, almost a million servers housed in data centers around the globe, serving the information needs of people in almost every country, and all of this data needs to be synced across the world's 24 different time zones.

The Google Spanner is a massive database that is designed to efficiently scale this complex flow of data to a single time. Spanner will timestamp all of Google's data from all around the world using True Time API, which uses an atomic clock combined with a GPS clock in order to sync every server in every Google data center. Google released a very detailed 14 page paper explaining how Spanner works; here is an excerpt with a technical explanation (you might want to put on your thinking hat for this one):

Data is stored in schematized semi-relational tables; data is versioned, and each version is automatically timestamped with its commit time; old versions of data are subject to configurable garbage-collection policies; and applications can read data at old timestamps. Spanner supports general-purpose transactions, and provides a SQL-based query language.

And breathe. In order to achieve this colossal data syncing feat, every one of Google's 900,000+ servers will need to be fitted with a GPS antenna and an atomic clock. Therefore, it might be a good move to find out who manufactures these devices and purchase stock in that company. The Google Spanner is incredibly efficient, able to automatically balance the demands for data across all of Google's servers. Spanner also uses a data structure that lets the applications control the data locality, this means the data retrieval time for users will be at a maximum.

There are enough technical details on how the Spanner works to overheat your graphing calculator. One last detail that we would like to point out is the sheer scope of the Spanner's processing power. Google has designed the Spanner to handle up to 10 million servers! The Spanner is Google's effort to lay the groundwork for future company growth, ensuring they will be able to grow ten times their current size. Considering how far Google has come from a few servers crammed into a garage in 1998, to their current incarnation, it should not be too long before Google reaches 10 million servers and has to invent yet another technology to keep up with its growth.

While you may not be facing the same ginormous server challenges as Google, you may have servers in your office that are underutilized. Just like the Google Spanner is an attempt by Google to maximize efficiency from all of its servers, so too can XFER evaluate your server infrastructure and determine best practices that can reduce power consumption. Through a process called virtualization, we can even reduce by at least 50% the number of server units that you are using. Call XFER at 734-927-6666 / 800-GET-XFER and let us show you how you can get the most out of your technology.