Sunday 16 December 2012

The History of Personal Storage Mbs > Gbs > TBs

As a teenager I messed about with cassette tapes on Sinclair ZX Spectrum and BBC micro storing roughly 617kb on one side of a C90 cassette. In those days you often types the raw code out by hand from magazines. Not very realistic when games have 3million lines of code to type (ref). Tapes and paper cards were used in mainframes until IBM's Alan Shugart and David Noble invented the floppy. The floppy disk came to the home market in the 80s. In 1989  I was introduced to a few 5.25" floppys on pre-windows machines at work (this was a Sony design, introduced in 1982) but the world quickly moved to the 720kb "double density" 3.5" disk which could store what seemed like a respectable 1.44mb (2.88mb at the end of its life). I built up several hundred of these fragile disks as some installations came 20-30 disks! Just swapping between them was a pain and why did not one invent the multidisk reader? Seems everyone was happy loading one at a time.



The 1.44mb floppy sold millions (make that hundred of millions) and never quite died (see http://news.bbc.co.uk/1/hi/magazine/8646699.stm) but imagine having instead of that 16gb USB stick you had on 11,700 floppys instead. In fact, what would a hard drive of floppys look like? A modest 1.5 terabyte desktop hard drive (cost now about £100) would be about a million floppies or a solid cube of 3.5metres in size and weighing about 15,700kg (15.7 metric tonnes)!



Then in about 1995 iomega's Zip arrived  with capacities of 100mb (later versions increased what seemed like an amazing 250 MB then 750 MB). These were great when they worked but a "little unreliable" and a few times I lost a whole disk of data with the "click of death" which seemed like a lifetime's worth (see http://en.wikipedia.org/wiki/Click_of_death). I became scared to use the 750mb in case of provoking a failure. Something was clearly wrong with the design. It offered such potential when 750mb was otherwise a pipedream! Zip had a transfer rate of about  1mb per sec maybe 20x that of the floppy but 20x less than a hard disk. The Zip disk was killed by two factors. Suicide (by unreliability) and also by recordable optical drives, namely CDR which cost a fortune when they came out (over £1000 for the recorder). I save up and bought one about 1996 for £600; a year after launch. Making your own CDs seemed like the big time; almost like you were a manufacturer yourself. I remember when I backup the whole of my infant version of itunes to CDR and later DVD-R (taking about 10 DVD-R's.....so about 50gb...and that was about 5000 mp3s).

 External hard drives weren't common at all and USB sticks tiny. In fact in the 90s it seems there was no easy way to back up your data. Using another hard drive seemed crazy because they were really expensive. In 1996 a 2gb drive cost about £400...rapidly falling to £300 for £20gb in about 2000.




Then in 1999 Amir Ban, Dov Moran and Oron Ogdan, of M-systems (with the help of IBM) invented the USB stick launched on December 15, 2000 at a humble 8mb. 8mb not 8gb. USB sockets were universal on computers by then, so USB sticks quickly became capable of carrying all you needed from office to office. I remember that before 2000 I had 4x four drawer cabinets for my work papers (largely 4000 scientific papers photocopied and meticulously filed from medical journals). Of course, you could never find the exact paper you wanted, no matter how good the filing system.


In 2004 someone at a conference showed me a 1gb USB stick. I was stunned by its large capacity, not to mention the cost, about £200. That was a lot to carry around. But by 2005 I had one and all these papers had effectively fitted onto one 1gb usb stick. In 2008 I could fit most everything I needed in the office onto a 4gb then 8gb and by 2010 I was carrying two 16gb sticks (one main, one backup) at all times with more than 30,000 files. Up to 2005 I had always felt the need to carry my laptop with me, sometimes with disasterous consequences. An interesing aside, large organizations
loose 265 laptops  per year:
43% of laptops were lost off-site (working from a home office or hotel room)
33 percent lost in transit or travel
Rates are highest in education and research; health and pharmaceuticals
7 percent of all assigned laptops in benchmarked companies will be lost or stolen.

As of 2012 USB drive of 256gb are available and its not just capacity that is up. Its speed too. Lets say you have recorded a HD video in 1080p resolution, a typical file will be 1gb for 120 mins compressed (all consumer camcorders compress the video). Professionals like to boast about raw recording which could be 24-48 frames per second (on frame is close to 8MB, the image 24 per seconds for 120minutes) would be about 1 to 1.5TB....remember those 1million floppies? Well lets assume you got them all onto 100mb Zip disks (yes all 10,500 of them) then at 1mb/s it would take 12 days to transfer the data (day and night assuming you didn't have to swap them!).....this is about the  time it would take with the first generation of slow sticks (USB1.0). Currently most people have USB2.0 and 1TB was on USB2.0 would take about 8 hours to transfer to you HDD.


 Most HDD are going to struggle to transfer this file indeed to good old mechanical HDD days are truely numbered. They are going to be replaced completed by solid state drives. SSDs are like mega RAM / super USB on your HDD. Only two years ago, this was ridiculous because SSD were 100x the price of regular HDD. The first 128gb SSD cost.....wait for it....$4600 in 2007 (review). However prices are dropped already, I just bought the Sandisk 480gb for £250 (amazon) and the increase in speed and responsiveness in windows7 is quite amazing.



SSDs are ideal are HDDs because they are robust, compact and not subject to mechanical failure. I must have lost more than 5 internal HDD due to failue, sometimes loosing everythingt, sometimes with warning signs (clicks, intermitant offs etc). I have learned the hard way to keep back ups. I know have 3x 500gb external drives, 3xTB drives 1x2TB drive, 10+ 16Gb USB sticks, 10+32gb SD cards, and over 100gb of cloud storage. Great, but a little slow at times, which doesn't matter for docs, but can matter for streaming video or large backups. The solution? SSD are also coming to external devices (like camcorders) and as USB SSDs (see OCZ Enyo or Iomega SSD or Transcend SSD). I have a 128gb iomega on order, but no reason you cannot replace the internal SSD with a 512gb very soon as prices fall. These make no sense unless USB3.0 / eSata / Firewire connections can be made with the device in question because SSD can potentially transfer (read write) at up to 5gb/sec (although 500mb/s in more realistic). That means a 1TB in as little as 3 minutes (more realistic figures would be 30-60minutes).....which is whole lot less than the 12 days cites above!

What a journey. In 1990 it was possible to carry 1mb in your pocket, now by 2013 it is quite possible to carry half a million in the form of a 512Gb external SSD. Unbelievable.