bit-tech.net

We stored 295 exabytes of data by 2007, says study

We stored 295 exabytes of data by 2007, says study

The digital revolution has seen a huge surge in data storage.

The BBC has reported on a study published in the journal Science, which has calculated the total amount of data stored in 2007. The figure the researchers worked out is 295 exabytes, which is equivalent to 295 billion gigabytes or 295 million terabytes.

The data doesn't just take PCs into account, though. A total of 60 technologies, from DVDs to paper adverts and books, were included in the research. To provide a sense of scale, Dr Martin Hilbert of the University of Southern California, told the BBC's Science in Action: 'If we were to take all that information and store it in books, we could cover the entire area of the US or China in 13 layers of books.'

Terabytes come after gigabytes, petabytes come after terabytes and exabytes come after that, so we're talking about a colossal amount of data here. The study not only demonstrates a huge increase in the amount of data stored in the last decade, but also points out that 75 per cent of information was stored on an analogue format such as video cassettes in 2000, but that 94 per cent of it was digital by 2007.

The article also states that 'the fastest growing area of information manipulation has been computation. During the two decades the survey covers, global computing capacity increased by 58 per cent per year.'

With the average size of a hard disk increasing by around 50 times since then (the average size is 1TB now as opposed to around 20GB in 2000), the PC has undoubtedly made a huge contribution to these figures. Of course, since 2007, the average hard disk capacity has doubled again, so the total amount of data will now be far in excess of 300 exabytes.

What's the total storage capacity of your PC? What fills your hard disks apart from software? Are you a lean SSD speed freak or multi-terabyte hoarder? Let us know in the forums.

28 Comments

Discuss in the forums Reply
Xir 14th February 2011, 12:49 Quote
and how much of that is contributed to Pr0n? :D
proxess 14th February 2011, 12:55 Quote
I reckon pr0n only contributed until roughly 3 years ago... now pip0l just stream their pr0nz. I currently have 1TB external HDD + 640GB internal HDD + 120GB iPod Classic. Anime is what occupies most space, tho I rarely watch Anime now a days.
dicobalt 14th February 2011, 13:08 Quote
Ok human race don't congratulate yourselves just yet. The vast majority (over 95%) of that "data" is video and pictures which do not contribute to "knowledge" of the human race in the least bit.
Javerh 14th February 2011, 13:13 Quote
The magic keyword here is data. Just because you can produce and store oomphs of data doesn't mean it's useful in any way. I'd say most of that data is input from some temperature probe or security camera recordings of your local mall.
NickCPC 14th February 2011, 13:45 Quote
Quote:
Originally Posted by dicobalt
Ok human race don't congratulate yourselves just yet. The vast majority (over 95%) of that "data" is video and pictures which do not contribute to "knowledge" of the human race in the least bit.
Sorry, disagree massively. You try and learn third year university physics without pictures or video, and watch how your understanding stays at a constant 0. Pictures and graphs are fantastic at displaying very advanced concepts or lots of data extremely concisely. Video is obviously pictures in sequential form, again extremely useful for conveying advanced concepts.

Note how I'm not saying all video or pictures are useful, but to dismiss all video and pictures as not useful is just stupid.
dicobalt 14th February 2011, 13:48 Quote
Quote:
Originally Posted by NickCPC
Quote:
Originally Posted by dicobalt
Ok human race don't congratulate yourselves just yet. The vast majority (over 95%) of that "data" is video and pictures which do not contribute to "knowledge" of the human race in the least bit.
Sorry, disagree massively. You try and learn third year university physics without pictures or video, and watch how your understanding stays at a constant 0. Pictures and graphs are fantastic at displaying very advanced concepts or lots of data extremely concisely. Video is obviously pictures in sequential form, again extremely useful for conveying advanced concepts.

Note how I'm not saying all video or pictures are useful, but to dismiss all video and pictures as not useful is just stupid.

Yea YouTube, MySpace and Facebook are full of university physics enthusiasts. I was referring to useless video and pictures more than educational material. Because there is way more useless than useful.
dogknees 14th February 2011, 13:49 Quote
Of course images contribute to knowledge. What about all the images sent back from satellites and other space craft. All the weather satellites, landsat and it's successors. Hubble and other observatories,....

Then you have all the images generated by colliders and other scientific apparatus,....

No contribution? On the contrary, this data underlies a huge amount of knowledge. Then you have documentaries and other TV content that is full of useful information.

To say nothing about art of all sorts. To say that the visual arts have contributed nothing to the human condition and our knowledge of ourselves and our relationships with others is ridiculous.
Th3Maverick 14th February 2011, 13:51 Quote
define useless.

Anyway, cool article. Utterly amazing amount of data, regardless of its perceived usefulness.
steveo_mcg 14th February 2011, 14:11 Quote
The question is how much of that is duplication.

Every copy of a blu-ray file is 30gb (hyperbole i know) I don't know how many films are sold each year but thats a lot of duplicate data being summed. Then there is the torrent swarm how much data could be saved if it were possible to reliably stream everything that existed in the torrent swarm with out having to download it.

I wouldn't be surprised if it were possible to reduce the sum of every thing stored to a few exabytes if there were some magical way of storing one copy of everything in the cloud.
SMIFFYDUDE 14th February 2011, 14:30 Quote
"I can has cheezburger?" type pics could be called usless
Guinevere 14th February 2011, 14:31 Quote
Quote:
Originally Posted by dicobalt
Yea YouTube, MySpace and Facebook are full of university physics enthusiasts. I was referring to useless video and pictures more than educational material. Because there is way more useless than useful.

Actually, I've got quite a few professional physicists friends who are on my Facebook.

Just because "you" choose to only experience a "useless" selection of material, it doesn't mean everyone's the same.

For example my upload to the net contribution to the knowledge of the world has been in the last month:

- A few pictures of my kids
- A video of myself and two friends wading through 50m of neck deep mud as part of an extreme charity run
- A couple of page edits to wikipedia
- Emails to friends
- Match history of online gaming

None of them are "useless" they all have a use to me, even if they are not life changing.

And anyway, who's to say face-plants and keyboard cats are useless anyway? They make us laugh or cringe don't they?

Shared data is shared knowledge, even if it's just cultural voyeurism as in "OMG Did you see that crazy red-neck with the home-made roller-coaster"

I say get rid of the proper damaging stuff (From the real bad stuff down through happy slapping down to teasing) and keep everything else.

I love you internet - never change.
maximus09 14th February 2011, 14:34 Quote
thats a good point, how much of this would be duplicate data?
Guinevere 14th February 2011, 14:35 Quote
Quote:
Originally Posted by SMIFFYDUDE
"I can has cheezburger?" type pics could be called usless

Yeah... kind of... but they are a form of entertainment. I say we don't put them into room 101 just yet, as where do you draw the line? It's okay to have "high art" but not "low art"?

Keep the catz!

(I can't believe I'm defending this topic)
Xir 14th February 2011, 16:28 Quote
Quote:
Originally Posted by Guinevere
For example my upload to the net contribution to the knowledge of the world has been in the last month:

- A few pictures of my kids
- A video of myself and two friends wading through 50m of neck deep mud as part of an extreme charity run
- A couple of page edits to wikipedia
- Emails to friends
- Match history of online gaming

None of them are "useless" they all have a use to me, even if they are not life changing.

Yeah, but a worthy contribution to the Worlds knowledge base?
As in, not only usefull to you and those that directly know you, but the general public?

Data wise, what probably takes up most space is the video (who will care about this particular video in 100 years time?)
What takes up probably the least space might actually be interesting to the rest of the world (the Wikipedia edits) :D
borandi 14th February 2011, 18:16 Quote
My work simulations generate a few 100gb of data each night. And that's just one person.
theevilelephant 14th February 2011, 18:25 Quote
I wonder how much of that data is the same? Would be interesting (and probably impossibly hard) to see an estimate that doesn't include duplicate data. Not that I'm even sure how you classify duplicate data :D
dullonien 15th February 2011, 01:29 Quote
The price per GB of hard drive space has gotten so low now that it's simply too easy to store almost everything we need. I've got plenty of stuff I would have simply deleted in the past, but it's just so much easier to simply buy a new hard drive instead now.

The amount of storage I have has snowballed in recent years. I've now got 4x1.5TB, 2x1TB, 2x500GB, 320GB, 200GB and 80GB hard drives. If I include SD cards, external hard drives or thumb stricks, and even my phone, I'm sure I'm touching the 10TB mark!!! The 4x1.5TB drives are in Raid5 so the total storage space is a little less. Whilst they aren't all full, they are filling up at an alarming rate.

The majority of it is media such as films, tv shows and music, but the amount of space required to even store work has spiralled aswell. My second year uni folder ended up being over 13GB in size, whilst my third year folder is already fast approaching 10GB. How the hell did we used to survive storing work on floppies?
Gradius 15th February 2011, 01:44 Quote
I'm at around 20TB atm, and I'm planning to go to 50TB in 2 years (or even less) from now.
Altron 15th February 2011, 02:21 Quote
As the availability of storage space expands, it allows for more data to be archived, and generally in a smaller format. This, measuring it simply in bytes isn't really a fair comparison.

As others have said, pictures and video have rapidly increased the amount of storage space required. Two thousand years ago, the library of alexandria had a vast amount of human knowledge stored on scrolls. We could scan those babies into a computer and run OCR and burn them onto a single DVD. On the converse, you could view the contents of the BluRay copy of "Jackass 3D" in binary, and print it out on scrolls and fill the entire library of alexandria with it. It's the same overall amount of data, but the format is such that there's far more useful knowledge stored in a given amount of text data than there is in a given amount of video data.

I could probably fit all of human knowledge up to the invention of movable type on a big flash drive. Everything from movable type until the invention of the computer on a decently sized hard drive. They didn't have as much storage space, so they condensed their knowledge to have less data. They used written words and static drawings, as these are more efficient on filesize to convey the same amount of knowledge.

But, as storage availability increases, more ancillary data is preserved. While it may seem silly to retain all of this data, it's probable that historians of the future will know far more about us than we do about our predecessors. If you watch a documentary on the neanderthals, you'll only see cave drawings and skeletons. If you watch a documentary on the Renaissance, all you'll see are written works and paintings, as they couldn't store anything more complicated. If you watch one on World War I, you'll see rudimentary video and sound. If you watch a documentary on Ronald Reagan, you'll see clear color video with matching voice recordings. We know far more about recent history than the far past because of how much more data has been preserved from it.
LordPyrinc 15th February 2011, 07:09 Quote
Quote:
Originally Posted by Guinevere
Shared data is shared knowledge, even if it's just cultural voyeurism as in "OMG Did you see that crazy red-neck with the home-made roller-coaster"

Eyes ben tryn ta bild ma own rolla costa fo yeers now. Can ya sin me tha blu.. blup... blupi..... shii.. drawins?

Devolve 15th February 2011, 09:24 Quote
isn't it a bit daft to say a lot of these images/videos do not contribute. At the end of the day no matter what is stored, it is stored on a product which has been bought, boosting economy. There is entertainment value aswell perhaps? I really dont think anyone can say stuff is useless, just certain things may be morally questionable?
p0Pe 15th February 2011, 09:49 Quote
Quote:
Originally Posted by LordPyrinc
Eyes ben tryn ta bild ma own rolla costa fo yeers now. Can ya sin me tha blu.. blup... blupi..... shii.. drawins?

Now i need to go look for a video of that :)
SMIFFYDUDE 15th February 2011, 10:19 Quote
Quote:
Originally Posted by Guinevere
Quote:
Originally Posted by SMIFFYDUDE
"I can has cheezburger?" type pics could be called usless

Yeah... kind of... but they are a form of entertainment. I say we don't put them into room 101 just yet, as where do you draw the line? It's okay to have "high art" but not "low art"?

Keep the catz!

(I can't believe I'm defending this topic)

OK OK we keep the lolcatz, but we chuck out all the BBC 3 and ITV output.
Silverster 15th February 2011, 11:33 Quote
Sorry to ruin the atmosphere here~ but I just want to make it sure: 295 exabytes is the number of data we have stored TILL 2007 or just in the SINGLE YEAR 2007?
And for a film-DVD example, it contains 4.7GB data, but produced million copies, will it be considered as 4.7GB data or 4.7PB data?
Hakuren 15th February 2011, 15:48 Quote
I'm just curious about "...13 layers of books.".

Just wonder what kind of books were used. 13 layers of stories for kids or elementary school math book or Harry Potter or Quantum Physics theories or 54 volumes of V.I. Lenin 'Collected Works'?

Please define the phrase >layer of books<. :D How the hell they calculated that magic '13 layers'? LOL
jadawgis732 20th February 2011, 04:48 Quote
Quote:
Originally Posted by dicobalt
Ok human race don't congratulate yourselves just yet. The vast majority (over 95%) of that "data" is video and pictures which do not contribute to "knowledge" of the human race in the least bit.

Oh, really? You never learned anything by looking at an image with graphs and charts? You never gained any knowledge by watching a documentary?
jadawgis732 20th February 2011, 04:51 Quote
Quote:
Originally Posted by Hakuren
I'm just curious about "...13 layers of books.".

Just wonder what kind of books were used. 13 layers of stories for kids or elementary school math book or Harry Potter or Quantum Physics theories or 54 volumes of V.I. Lenin 'Collected Works'?

Please define the phrase >layer of books<. :D How the hell they calculated that magic '13 layers'? LOL

You know it's like Lenin said "you look for the people who are going to benefit, and...." "I am the walrus" "and...." "I am the walrus" "DONNIE STFU! V.I. Lenin! VLADIMIR ILYICH ULIANOV! Those mf's I did not watch my buddies dive face down in the mud so that this struffet, this whore! could strut around!" "What got into Walter?"

For anyone who has NO idea what I'm talking about, no I'm not crazy, you just need to catch up on your best comedies of all time list.
jadawgis732 20th February 2011, 04:56 Quote
Sorry, I cant edit.

My guess would be that they took the average number of words per square inch on pages over a certain amount of books that made the data significant, and just applied to the total area of land. It wouldn't matter how big the pages were because you could just make one page 3.7 million square miles (roughly equal to the are of China or the US)*(average words per square mile on a page) and found that that there was 13x that amount. But yeah, I'd like to know whether it was 13 pages or 13 books, with x number of pages.
Log in

You are not logged in, please login with your forum account below. If you don't already have an account please register to start contributing.



Discuss in the forums