So this is my first post of what will likely be a long series of articles about Bacula. My interest in Bacula started a few years ago when I began exploring the world of LTO Tape Backups. For the average person backups these days are written to Cloud services or sometimes external hard drives. I’ve been bashing away at Computers for more than 30 years and the concepts of long term backups of data and programs have been around as long as the personal computer, longer even. I started writing external data to Cassette tapes, and then Floppy Disks, then smaller floppy disks, then massive external scsi drives, LTO Tape storage, then removable cartridge spindle drives zip/ syquest. I thought the writeable optical media we now know as CD/DVD storage would change it all but it didn’t, we soon found out that the coating just didn’t last and CD’s written and stored for years gave unreliable results. Along came SD cards and something called bubble memory storage, once again great performance but long term reliability proved non-existent.
If You’ve got any of your memories or critical documents in ANY format other than LTO tape or reliable Cloud Storage, don’t delay get those files moves somewhere reliable or by the time you need them they may have decayed beyond rescue.
For Most people Cloud Storage is the way to go these days. The Cloud isn’t some magical thing, it’s really just somebody else’s computer. You’re paying a monthly fee not only for the physical storage media but also the infrastructure that backs up that storage in a scheduled and automatic fashion as an insurance policy for disaster recovery. Amazon, Apple, IBM and any other massive Tech corporation isn’t immune from Hurricanes, Earthquakes, or any man made disasters, but they have implemented a backup and redundancy infrastructure so that anything short of a Global Armageddon situation means the data can all be recovered. You see 1 copy of your Cloud Based File Folder, but in reality it’s mirrored in 3 or 4 different places and each place is writing backups to in all probability and LTO Tape Library system in a 5th or 6th location. Choose your Cloud Data Storage service wisely, iCloud, Onedrive or Amazon S2 storage solutions are priced competitively, if someone is offering the same thing for significantly less then they probably don’t have the same disaster proof infrastructure. Their Disaster Recovery plan is likely just an Insurance claim, a bankruptcy filing and thousands of customers who will have lost their photo memories and critical documents forever.
The 4th item in my list of Historical backup media was LTO tape and that’s significant. IT came along went away and now it’s back again, but why? LTO stands for Linear Tape Open, and it’s a technical specification standard. What that means is that Hardware and Media that conform to the standard are for the most part interchangeable. Cassette audio tapes, CD’s, VHS tapes, all these things conform to a standard so that media written that fits into and can be written by 1 device can be read by anything that conforms to the standard. Storing Information on Magnetic tape pre-dates the entire computer industry. Magnetic tape was invented in Germany in 1928, it has been leveraged to Store most of master recordings of every song you’ve probably ever heard anywhere, and of course in the 70’s and 80’s it was adapted to store video as well as audio information and that resulted in the Beta and VHS tape systems. Up until 2002 Magnetic tape had an expected lifespan of roughly 10-20, far better than anything else but still not perfect.
LTO Tapes are an extension of the very first storage media for mainframe computers. Open reel tape systems were developed by most mainframe computer manufacturers, but there was no standard, so an IBM system might use 1 size of tape and write it’s data in it’s own proprietary format. but Cray , HoneyWell, Fujitsu, DEC and NCR were also selling mainframes and each had its own format of Open Reel tape backups. Magnetic Tape was abandoned as a long term solution because the Density of Data storable was severely limited by the square area of the tape. To backup your iPhone you would need 3-4 reels of circa 1970 1 inch magnetic tape, each reel was about 18″ in diameter and weighed 15-20 lbs.
in 2002 a corporation then called Imation, who where actually a spinoff of 3M, was granted about 12 million dollars by a US Tech innovation department (NIST), to investigate and develop higher data density storage onto magnetic tapes. Wisely Imation as a part of their research process adopted a standard for how data would be organized and of course standard sizes and shapes of the tape cartridges. This LTO standard is now controlled by IBM, Quantum and HP with some government oversight. Over the years improvements in the actual tape material and the coatings used to store the information, means that today the life of LTO Tape stored at room temperature and 50% humidity is at a minimum 15 years and more commonly as long as 30 years.
So I haven’t talked very much about Bacula, but I did mention that my interest began with my conversion to LTO Tape as a backup medium. Because LTO is almost entirely an enterprise level technology, the operating system drivers and software to manage these devices is maintained and distributed only by the biggest Hardware corporations (IBM, DELL, HP). Even Quantum who manufacture and sell almost all of the tapes don’t release any software that can effectively write to them in an organized way. 95% of the commercially available software is windows server based and is very very expensive ($1000 + for single licenses). Now the majority of the internet actually runs on operating systems collectively know as ‘nix systems, short for UNIX based. Nix system are branded by names you know and probably associated with the Linux kernel, Red Hat, Ubuntu, CentOS, etc. Because of the open source nature of these operating systems the LTO tape Consortium was forced to release device drivers and API standards so that open source operating systems could take advantage of LTO backup hardware. While there are dozens of open source software solutions available today, Bacula is probably the oldest and it’s certainly the most feature rich. A single bacula server can organize and control hundreds of client instances, and write these backups to a common storage solution. In a large enterprise every computer is detecting file system changes on a Daily, or even hourly basis, the send only the DIFFERENCES to the main server and that server writes these incremental backups to a common storage place (usually a Tape drive or a Robotic Tape Library).
By using repurposed hardware and open source solutions I’ve managed to develope my own backup that mimics this enterprise behaviour and my TL2000 Robotic Tape Autochanger affords me a total of 136 TeraBytes of backup space, my journey is chronicled in this series of blog articles.