AVCHD Backup Strategy

Back in December I purchased a new video camera–the High Definition Canon Vixia HF100. Naturally, I did a lot of research before purchasing it as I wasn’t very familiar with the AVCHD format–a high definition H.264 codec that is very efficient and is able to store a lot of video data using a minimal amount of memory. The codec does this by only recording changes in the images rather than full frames. I don’t know the intricate details of the technology, however, it is quite impressive how good the quality is for such an inexpensive camera. You can see the quality this camera can produce by going to Vimeo.com and searching for the tag HF100. So if the video is so great at such a decent price, what’s the catch?

Simply put, the catch is that there are no applications available currently that can edit the AVCHD format natively on the Mac. The Canon camera, and other AVCHD cameras like it, create files that use the .mts file extension. Again, I’m not familiar with all of the technical details, however, I do know the only way to work with the files on the Macintosh is to convert them to some other format. iMovie 08/09 both handle the AVCHD format by importing the .mts files and converting them to the Apple Intermediate Codec (AIC).

iMovie provides several different options for importing the video data, however, if you decide to go with the “Full HD” option when importing, which I recommend if you want to have high quality output, your files will be GINORMOUS! Mine run in the neighborhood of about 1GB per minute of video. That’s huge!

Back in the days of tape, yes probably most people are still using tape, you could simply fill a tape with video and then put a date on it and add it to your physical media archive on a shelf somewhere. This works well as the shelf life for tape, while not infinite, is really quite good. This is exactly how I’ve done archiving in the past (and when I still use my old DV camera for various tasks). The question for me has become, how can I archive such massive amounts of data as efficiently and with some level of confidence that the data won’t get lost. Here are the the various issues I considered in coming up with my backup strategy:

  • Hard Drive: This is a viable option for sure as disks keep getting cheaper, however, I have heard enough stories of failed hard drives that this one makes me a bit nervous. I can see using it as a secondary backup strategy, but I don’t trust hard drives enough. RAID seems like a reasonable solution, but that requires more money as you need redundancy which requires multiple disks.
  • Burning AIC Format to DVD: This strategy would just be too much work. Remember that you can only hold 4.7GB on a single layer DVD-R/DVD+R. This means that you would be able to store less that 5 minutes of video on a single disk. While I worked hard to force myself to record short clips and keep things maintainable at the clip level, manually splitting all of the clips up in order to only get 5 minutes of video per disc seems just unmanageable.
  • Burning AVCHD Format to DVD: The shelf life of optical media is also questionable, however, this is the strategy I am pursuing at the moment. Ideally solid state drives will get much bigger and cheaper at which point I will move everything over. For now, however, burning to DVD is the strategy I’m going to use

So I’ve decided that burning the original files to DVD is the best way to go for now as the shelf life of optical media is reasonable enough that my data should keep in the near term. And I’ve chosen to archive the video in its compressed AVCHD format. Here, however, is where the strategy gets interesting. I want to have a way to get the clips off of the DVD without too much hassle while using iMovie. Your strategy may differ if you use some other editing software. Here is what I mean.

iMovie automatically detects the existence of AVCHD movie clips when you either plug in your camera or mount the SD card with a card reader and loads the import utility. What I have done is taken the data from the SD card; copied it to my hard drive; and then made a disk image (.dmg) file out of it. When this disk image is mounted while iMovie is open, it will automatically detect that the image contains the AVCHD video files and load the import utility just as if it were physical media being mounted. Pretty cool eh? So, yes there is a bit of work on the front end, however, when you burn these disk images to a DVD, you can just mount them directly when you put the DVD in at some point in the future when you are ready to import and start working with the clips. So here are the steps you need to get this done.

  1. Plug in your camera or mount the SD card with a card reader. Then run iMovie
  2. When iMovie runs, it will automatically detect your camera or SD card and will load the import utility. Click the “Archive All…” button at the bottom of the window. Choose your Desktop as the save location (you’ll see why shortly) and give the file a name and iMovie will back up the data to your hard drive.
    Archive All...
    Archive All...
  3. The backup process will take a little while depending upon how much data you are saving. Once it has finished, you need to open Terminal.app, which is located in /Applications/Utilities.
  4. Now we need to create the disk image and will do so using a command line utility called hdiutil. You need to change directories to your Desktop. To do so, at the command prompt in Terminal type cd Desktop and press Enter. You are now in the Desktop folder where we saved the archive.
  5. To save the archive folder as a disk image, type the following command followed by the enter key: hdiutil create -srcdir ArchiveFolder AVCHDImage.dmg. This command will take a little while depending upon how much data your are archiving. You will know the process has finished when the command prompt returns. You should expect it to take a while if you are saving several gigabytes worth of data so be patient as the progress that hdiutil shows isn’t very informative. It is, however, working.

    Note: ArchiveFolder is the name of the folder you saved when you had iMovie create the file on your Desktop. This command will not work if you specified spaces in your folder name. If you do use spaces, you will need to escape them. This means that if you created a folder called ‘Archive Folder’ for example, your command would need to be: hdiutil create -srcdir Archive\ Folder AVCHDImage.dmg. Notice the backslash before the space. Add this backslash before any spaces in your archive folder’s name.

  6. Once hdiutil has finished creating the disk image, you can now burn it to a DVD. Insert a blank DVD and open a new burn folder on the desktop or somewhere handy. Then drag the disk image to the burn folder and click burn.
  7. Once your burn is finished, insert the DVD back into your drive (if it was ejected) and let it mount on the desktop. Open the disk and double click the .dmg file. This will mount the image. You can click Skip during the verification process if you want to have it simply mount without verification. I usually skip it myself as I’m impatient.
  8. Open iMovie again. iMovie should detect the disk image and automatically load the import utility. If it doesn’t then you may have done something wrong. Shoot me specific questions in the comments if you need help here.

There is one caveat I should mention to using this technique. I suggest you keep your data size on your SD card to 4GB or so. In fact I use a 4GB card for precisely this reason–it fits well with my backup-to-DVD strategy since a single layer DVD only holds 4.7GB. If you have a hard drive based camera, you’ll have to keep an eye on the size of the data in order to ensure that your data will fit on a single DVD, but this shouldn’t be too difficult. I have a 16GB SD card for those cases where I need to capture longer shots like when capturing a long presentation, but most of what I do allows me to keep smaller files and therefore reasonable backup sizes.

So that’s it. It’s not a perfect strategy as the shelf life for DVDs is a little disconcerting, however, I have every confidence that solid state memory as it becomes more and more ubiquitous will be the best backup/archival methodology once the drives get bigger and the costs continue to go down. In the mean time, this strategy seems to be the best one for backing AVCHD video. If you have other strategies that have worked for you, share them in the comments. I would love to hear your ideas.

Good luck.

Guitar Hero On The Commodore 64?

Shredz64Wow, some people have some serious hacking skills. That, and a lot of time. This guy built his own interface that allows him to connect the Guitar Hero controller for the Playstation up to his Commodore 64 on which he plays a Guitar Hero clone he calls Shredz64.

That’s very geeky… and cool!

Get more information at Toni Westbrook’s website and here is more about his Shredz64.

Zarra Studios releases iWeb Buddy, the post-processor for iWeb

iWeb Buddy Now AvailableColorado Springs, Colorado – Zarra Studios LLC announced today the release of iWeb Buddy. iWeb Buddy is a post processor application for iWeb that is designed to bring additional functionality to iWeb that professionals and “pro-sumers” have come to need on their web sites.

iWeb Buddy allows you to have multiple domain files with iWeb so that you can keep your sites separated into their own files. iWeb Buddy also post processes web pages created in iWeb; allowing the user to add Google Analytics, Social Bookmarking, Haloscan comments and custom HTML code to your website automatically.

In addition, iWeb Buddy remembers your settings, account numbers, etc. making it easy to update your website whenever you make a change in iWeb.

With iWeb Buddy, a user can “monetize” their websites that they have created in iWeb by being able to track the traffic on their sites, add custom html code for shopping carts and more.

Explains Marcus Zarra, owner of Zarra Studios, “iWeb is a great website design application. It is geared for the consumer but it is so close to being truly viable for professionals to use but it is missing a few key features – iWeb Buddy brings those features to iWeb.”

* Use multiple “domain” files with iWeb
* Add Google Analytics to your website
* Redirect your RSS feeds to sites like Feedburner
* Fix the RSS links on your iWeb blog pages
* Add Social Bookmarking to your blog posts
* Add Haloscan comments to your pages/blog posts
* Add Custom code to your web pages (WITHOUT IFRAMES)
* Add Mint site tracking to your webpages

Pricing and Availability
iWeb Buddy is available today for $25.00 USD for a single user license. A family license is also available for $35.00 USD. A demonstration copy is fully functional for the first 30 days and can downloaded from Zarra Studios website.

Congratulations on this release, Marcus and Zarra Studios!

Self Tuning Guitar

So this is pretty cool. My friend Kip says that it would make me look like a dork if I showed up with one of these at an open mike night, however, I don’t think I’ve ever been to an open mike night where *everyone* didn’t look like a dork. It’s a cool piece of gear. And by the way, I’ve never done and don’t plan on doing an open-mike night, no matter what Kip says.

Twitter: The Long Tail of Celebrity?

If you’re not familiar with Chris Anderson’s The Long Tail, you should take a look at his book or at least read the about page on his site to understand this post. In a nutshell from Chris’s site:

The theory of the Long Tail is that our culture and economy is increasingly shifting away from a focus on a relatively small number of “hits” (mainstream products and markets) at the head of the demand curve and toward a huge number of niches in the tail.

In an attempt to explain what Twitter is to those who are not familiar, I’ve said that it is the long tail of celebrity. Ok, so that is probably pretty dumb since now I also have to also explain what the long tail is. I probably need to keep searching for a better definition, but when I say that it’s the long tail of celebrity, I simply mean that everyone who uses Twitter can be a celebrity…

to a certain extent.

Once you have followers on Twitter, you could be considered a celebrity of sorts. You’re not one of the big names, the hits, but you could certainly be considered a niche–especially in the case where your followers are only friends or family.

Thanks to Marcus I have started to follow some independent software developers for the Mac on Twitter and find many of their tweets (Twitter posts/updates) to be interesting. There’s lots of noise too, but you can quickly filter through that. What I’ve learned from following these guys is just the sheer amount of time these guys seem to spend coding. I know that’s what it takes to be successful as an indie, but it’s just fascinating to me to watch these guys update what’s new in their development processes.

To me, guys like Wil Shipley, creator of the award winning Delicious Library and Daniel Jalkut, developer of MarsEdit are celebrities of sorts. I don’t know Wil or Daniel personally and they don’t know me from Adam, but Twitter has enabled me to see what’s going on with the most recent developments in the latest version of each of the applications they develop and maintain. It’s highly interesting to me.

I see them as successful at doing that which I would personally like to do as well. I’m not star struck by these guys as tends to be the case when people are fascinated with Hollywood celebs, however, there is certainly an element of admiration as they are the real deal in independent Mac OS X development. They actually have applications available for purchase and are living the dream.

If my theory is right, then everyone can be a celebrity by using Twitter. Just don’t go taking yourself too seriously, throwing public tantrums, and getting caught driving drunk like the regular celebrities do every time your followers number rises. Remember, you’re still in the long tail–you’re a niche, not a hit.

You can catch me on Twitter at http://twitter.com/perlmunger. It’s not terribly interesting, but if you follow me, I’ll be your celebrity.