The Big Data Evolution (updated)

In 1969 (the year I was born), an 8″ floppy disc by IBM was able to capture 80 KB (or 80 * 1024 = 81.920 byte). Only IBM could write to it; for “normal” people it was read-only.

In 1976, an 5,25″ single-side floppy disc could store 110 KB (112.640 byte); in 1984 it had made it to double-side and high-density with the famous capacity of 1.200 KB (or 1.228.800 byte).

The 3,5″ double-density classic appeared in the same year. Unforgettable its epic storage capacity of 720k (or 737.280 bytes). It took until 1987 for the high-density brother to come up with 1.440 KB (1.474.560 byte). MS DOS needed 3 of those floppy discs. My Amiga computer back then still worked with DD types of 880k and all of my private data in 1990 when I seriously started to program fitted onto only 1 of those little HD wonders.

I do not really recall the tape drive era; however I recall that guy responsible for source control in our projects in the office back then who could watch flying sheets of paper in Windows Explorer for hours when he backuped our VSS database (probably onto 2,4 GB – 2.516.582.400 byte – Sony tape).

More or less at the same time I discovered the tremendiously brilliant invention of the IoMega Zip Drive, bought a 100MB (104.857.600 byte) type of it and was utterly happy to be able to do backups of not only all of my private data but also all of my source code of one project onto just one disc.

And just when I had purchased the second Zip Drive for a – then – reasonable price in order to be able to carry data between home and office without carrying a drive, our company had ordered the first CD RW writer – an epic piece made by — ? not sure ? — Plextor, I think. What I however do remember clearly is an audio CD project which I did in that time, using the utterly famous “Feurio!“. For an 8th grade school class I sampled English songs with strong lyrics like “Free Electric Band” or “Me and Bobby McGee” for their English lesson and as a school’s-out present. I think I destroyed like 20 raw discs before getting 13 working ones 😉

It was 1997++ and there were the 74 and the 80 minute CD-RWs. 74 equalled to around 640 MB (671.088.640 byte) and 80 equalled to 700 MB which by “overburning” could be extended to 730 MB (765.460.480 byte) sometimes – involving the risk that the target drive couldn’t read it anymore.

It was in those years that I started backing up my data onto disc around once a year (incl. some transfer of some very old projects from the early 90ies) and the backups captured from 1995 until 2010 hold 159.666 files or 40,3 GB (43.363.611.776 byte) of data – which is

• 57 times the 80 minute CD
• 414 times my Zip Drive
• 28.408 times the 3,5″ HD
• and 529.341 times the 8″ floppy disc

Which is still a ridicolously little amount of data compared to a 59,- EUR 1TB (1.099.511.627.776 byte) external HDD or my 800,- EUR 4-bay 6 TB Raid-5 NAS (6.597.069.766.656 byte).

One Windows Azure Storage Account has a limited capacity of 200 TB (219.902.325.555.200 byte) – which obviously is still quite small compared to the overall data storage capacity of all Azure data centers worldwide. And facebook is reported to hold the equivalent of over 100 PB (112.589.990.684.262.400 byte) of data while its more than a billion users utilize 7 PB of foto storage each and every month only.

Which means, that in 45 years time the storage used by a private person for private reasons has increased by a factor of more than 96 billion!

I just do hope this isn’t all food and cat pics.

Update:

I’ve just purchased a MiniSD for my mobile. 16GB at the size of a fingernail (0,3″ roughly or 5,7% the size of a 5,25″ floppy disk). It cost EUR 7,70. I digged out the price of a 5,25″ double-side high-density; according to my records that was about EUR 0,17 back in 1995. Which equals a price drop per Gigabyte by more than 99% throughout those years … Gives an interesting twist to the quote “this is worth nothing anymore” …