Ming the Mechanic:
Gargantuan Google

The NewsLog of Flemming Funch
 Gargantuan Google2004-04-28 11:58
picture by Flemming Funch

The Guardian has an article about how Google seems to cover up what kind of horsepower they're really sitting on.
[T]ake the number of servers that Google operates. The only figure the company will admit to is '10,000+'. They also claim to have '4+ petabytes' of disk storage, and have let slip that each server is fitted with two 80 gigabyte hard drives. Now a petabyte is 10 to the power of 15 bytes, so if Google had only 10,000 servers, that would come to 400 Gb per server. So again the numbers don't add up. I could go on, but you will get the point. But what it all comes down to is this: Google has far more computing power at its disposal than it is letting on. In fact, there have been rumours in the business for months that the Google cluster actually has 100,000 servers - which if true means that the company's technical competence beggars belief.

Now the interesting question raised by all this is: why the reticence? Most companies lose no opportunity to brag about their technology. (Think of all those Oracle ads.) Is this an example of Google behaving ultra-responsibly - being careful not to hype its prospects prior to an IPO? Or is it a sign of a deeper commercial strategy? The latter is what Garfinkel suspects. 'After all,' 'he says, 'if Google publicised how many pages it has indexed and how many computers it has in its data centres around the world, search competitors such as Yahoo!, Teoma, and Mooter would know how much capital they had to raise in order to have a hope of displacing the king at the top of the hill.' If truth is the first casualty of war, openness is the first casualty of going public.
It is an interesting conundrum, as Google indeed appears as a very open and accommodating company. But it seems their secret weapon is not just a clever indexing algorithm, but the fact that they've constructed the vastest distributed computer system in history, making them able to apparently easily and quickly do feats that previously were considered completely unrealistic. It is intriguing that it has considerable business value for them to greatly understate their own accomplishments. And it is intriguing that the Internet no longer is just a flexible routing protocol. The Google hypercomputer has become an unavoidable key to providing what we expect from the net.

[< Back] [Ming the Mechanic]



28 Apr 2004 @ 14:46 by sharie : There's other suspicions about Google...
so I just don't use them. I use Yahoo to search.

Thanks for the post.  

28 Apr 2004 @ 16:50 by Dan @ : A few quick calculations
ok so 4 million gigabytes is 4 petabytes right? (10,000 servers times 400 gigs per server.)

Ok now assuming 700 megabytes for two hours of film, that is approxamitely 1,300 YEARS of film. You could setup a video camera in the middle ages and let it run till now and not run out of disk space. Can you imagine how big that is? That is mindblowing.

I bet you could store everything ever written, every movie ever made, and every song ever sung and still have room to spare. What do they do with all that space? Truly mindblowing.  

28 Apr 2004 @ 19:56 by maxtobin : Why one asks?
There are many reasons could be behind this little 'tip of the ice berg' fact!
What power to be the channel through which the connected internet world works, what can one do with such computational power in terms of real time traffic analysis? Most people use only the first page of search results so the game of getting placement and exposure therefore are all important. What people do with their online time may be of interest to more than just the terror snoopers, there would be much information mining potential with such capacity and all that traffic. I must admit that I try to remember to use other search engines for certain searches (kind of keeping a random habit pattern to maintain the illusion (for myself mostly perhaps!) that I am a slightly invisible being.  

17 Jun 2004 @ 23:14 by Tech Watcher @ : Unbelievable
Firstly, we should assume that there is a reasonable if not probable chance that all this talk of unimaginable computer power is merely strategic market-seeding by business execs, or better yet, simply inaccurate heresay. Think about it -- the converse is true also; if your competitors believe they need 100,000 servers, and you only need 1k or 10k, then they're in deep debt before they know it, and your company has the last laugh.

That being said, if they do indeed have this capability, there is one obvious and paramount danger: With 50 terahertz of processing power (assuming 100k comps have 500mhz processing power, a very minimal assumption), even some of our best encryption routines could be cracked in a matter of hours, if not minutes. Suddenly no business, including google's competitors, is safe. Now I doubt google would ever find itself in a situation like I've just suggested, but isn't it a matter of national security if it's at that level? Now I'm sure google's clusters are "impenetrable" and have countless "failsafe" protections, but where have we heard that before? This kind of power at a hacker's disposal (including foreign nations or terrorists) could spell all kinds of trouble for our economy and even safety. And what about a google employee who understands the depths of a system such as this and would find such power handy for his own decryption routines or other serious mischief. As a programmer with reasonable experience in the security field, I know that it isn't of great difficulty to hide things such as processing cycles by disguising them as unused cycles or as, for instance, masqueraded searches. So at this moment, someone might have the capacity to, at their will, calculate the "unbreakable" packet sequences of those who are accessing this server (requires enormous processing power, such as a terabyte or so), snoop the encrypted passwords used to remotely access this server from the administrator's packet stream, feed such data into a decryption routine, and delete this post... all within minutes and invisible to google. Now this last example is very "conspiracy theory" in nature, and next to impossible in terms of likelihood, but it should sufficiently demonstrate the possiblities of such processing power. One might even make an AI program of sufficient proficiency to learn to do the prior task on its own (think nueral networks). The final and probably most obvious example I'll make is the risk to DoD, NSA ect.'s firewalls. Or perhaps military contractors such as Boeing which are responsible for concealing hundreds of military secrets. I'll leave my commentary at that... Unlikely they have this type of speed, but if they do... I (and you should) have some serious concerns.

If you do have similar concerns, the best way to have an effect is to post your concerns on other websites as well. Why not write your congressman? I'm all for free markets ... laissez faire ... but as time has taught us -- no single entity can be trusted to look out for the interests of the population as a whole.  

19 Dec 2014 @ 10:52 by Nour @ : MKJddAJhFwIibBu
Thanks alot - your answer solved all my problems after several days stingglrug  

23 Dec 2014 @ 07:59 by Bob @ : QZkgYXwdOpMmgri
Yup, that'll do it. You have my apoiceiatrpn.  

Other stories in
2007-12-26 21:19: LeWeb3
2007-07-04 23:14: Orbo free energy
2007-06-28 22:03: Wide-angle Immersive Stereoscope
2007-06-24 16:51: Spy box
2007-06-14 01:00: Photosynth
2007-04-25 14:01: Open source hardware
2007-03-22 16:33: The Air Car
2007-03-09 23:44: Web 2.0
2007-02-07 16:09: Talkr
2007-01-12 00:15: Open Source Desktop Fabricator

[< Back] [Ming the Mechanic] [PermaLink]? 

Link to this article as: http://ming.tv/flemming2.php/__show_article/_a000010-001226.htm
Main Page: ming.tv