Home » Articles » Internet
Google Breaks Speed Record, Sorting 1PB Data In 6 hours
Published on 11/29/08 at 19:00:01 by On
Internet
15/05/2009: Hosting and registrar fraud by Parava Networks
19/02/2009: Pirate Bay Trial goes LIVE
01/02/2009: Google’s misplaced slash banned the entire Internet
17/05/2007: DreamHost: The Big 5-oh-oh-oh-oh-oh!
InternetGoogle Systems Infrastructure Team has announced that they have been able to sort 1 terabyte of data across one-thousand computers in a mere 68 seconds. (Previously, the same record of 1TB was across 910 computers and took 209 seconds.) For one petabyte of data, it took Google 6 hours and 2 minutes to do the sorting—and the sorted data gets moved to 48,000 hard drives.

Google also acknowledges that they had triple redundancy—the data was backed up on 3 hard drives. Could it be that if you reduced that redundancy, this would all be much quicker? Not necessarily, since storing may not be part of the sorting algorithm that was recorded (and storing the data could have still been ongoing after the sort had been completed).

According to the Google Systems Infrastructure Team during sorting experiments they have followed the rules of a standard terabyte (TB) sort benchmark. The standardized experiments helped understand and compare the benefits of various technologies and also add a competitive spirit. You can think of it as an Olympic event for computations. By pushing the boundaries of these types of programs, one can learn about the limitations of current technologies as well as the lessons useful in designing next generation computing platforms. This, in turn, should help everyone have faster access to higher-quality information.

Read more here: Grzegorz Czajkowski, Systems Infrastructure Team - Sorting 1PB with MapReduce

1 comment, (4098 reads) All Articles by, On
 
Printer Friendly version - Google Breaks Speed Record, Sorting 1PB Data In 6 hours   Log in to use this feature 
Sign Up
Comments on this article: Comments on this article:

1. abendkleider katalog Written on 08/02/12 at 22:43:19 by carmencha





The comments are owned by the poster. We aren't responsible for its content.
Only registered members may comment on articles.
Login / Register
V
Username:

Password:

Remember Me?


Forgot password?
Register?

Main Menu
 * Home
 * Recent Discussions
 * Articles
 * Forums
 * Links
 * Downloads
 * Top 10
 * Palm Pilot
 * User Help Docs
 * Download WebAPP
Online Now
V
2 Members/Guests
From: 1 countries
Online Now.
Log in to see who's on.

Most ever on: 900
Membership: 8269

New Users:
  alliancepetr
  andrewjames
  Neighbours
  anilpaliwal
  sergiomazapura
  aussietroll
  yuyuiuy
  asss
  edjackson6143
  July128