Joseph Smarr, former CTO of Plaxo (which explains why I recognized his picture), in I’m a technical lead on the Google+ team. Ask me anything, reveals the stack used for building Google+:
While you don’t have Google’s stack available to you, you do have some open source options. HBase is replacement for BigTable. Then there’s Hadoop MapReduce. Colossusis Google’s next generation file system, a replacement for GFS. Since we don’t know much about Colossus, it’s hard to say what a suitable replacement would be. There’s the Hadoop distributed file system HDFS. And if you are looking for some of the cloud like infrastructure glue there’s OpenStack (which also has storage system).
Google probably uses a custom Java servlet container, but the choice here doesn’t matter that much. Most of the work will be spawned in parallel and performed on other servers implemented in C++, Java, or Python.
Whereas most communication with Google is non-existent, the Google+ development team is noticeably more responsive, turning out visible improvements quickly and consistently. Joseph tells us why: We put extra emphasis on engineering speed/agility–we try to release code updates on a daily basis while still keeping quality/stability/latency as high as you’d expect from google. This helps us move fast and respond quickly to user feedback. We try to do a full push (almost) every day, and we sometimes sneak in patch releases too if needed. But there are humans in the loop, it’s not a “auto-push if all tests pass” situation or anything like that.
For Google+’s most innovative feature, video conferencing with Hangouts, GigaOM has a good article on that stack, which is based on Announcing Google+ Hangouts, written by Tech Lead Justin Uberti. Unlike Skype, which runs on a cost effective P2P model, Hangouts is completely hosted by Google. This must cost a staggering amount of money. You are on your own here. Nobody can replace the bandwidth being donated by the Google fairy.
That’s Google in a box. Then again, an ex-Googler thinks you can do better using MessagePack, JSON, Hadoop, jQuery, and MongoDB. If you can do better for a world wide base of a billion users is a completely different question.
- Closure Library / Closure Templates / Closure Compiler
- VMware exposes its plans to be the OS for the cloud – another infrastructure option.
- Colossus of Rhodes – wonder of the world.
- Colossus Computer – electronic computing device used by British codebreakers to help read encrypted German messages during World War II. They used vacuum tubes (thermionic valves) to perform the calculations.
- Colossus: The Forbin Project – SF film about a massive American defense computer, named Colossus, becoming sentient and deciding to assume control of the world.
- This post On Hacker News
- This post On Reddit
- OpenGSE Released – Something that may be of interest to Open Source enthusiasts is the recent release of the Open Google Servlet Engine (OpenGSE). Once upon a time engineers within Google developed their own fast, lightweight servlet engine which we inside Google refer to as simply the “Google Servlet Engine” or GSE.
- On Quora: Is Google’s software infrastructure obsolete? TLDR: Nope. Hector Yee has a good summary: The Google infrastructure is far more stable and well documented than the open source variants. They work together as a coherent whole. It’s much easier to have a stable, massively scalable system built in the Google framework than with the equivalent open source tools. On the other hand, it’s easy to chafe at the restrictions imposed by a big company with a defined way of doing things. Both view points can be true.