I don't miss ...

I don't miss ... from Martin Koch on Vimeo.

My Mac talks to the HF100

If you're tired of going into the menu over and over again just to set something there's a solution. Let your Mac do it. All you need is a ZephIR^TM. This USB device enables you to use your computer to control almost any piece of equipment designed for infrared remote control. You can train it's software to simulate any remote control and on top of that everything can be controlled via AppleScript.

My ZephIR arrived today and after an hour of "training" and trial and error my HF100 listens carefully to my Mac. I recorded all of the 20 buttons of the Canon WL-D88 wireless controller and also submitted the component to the ZephIR data base.To download it open the ZephIR application, click on "WebztIR" then navigate to Canon > Camera > HF100_WL_D88 and double-click the remote (you must be connected to the internet).

For instance the following command would toggle the on-screen displays:

fire zephir command "DISP" of component "Canon HF100 WL-D88"

The really interesting thing is that the supplied software allows to build custom remotes with buttons that execute a batch of AppleScript commands at once. To get a reliable response I recommend adding a delay before fireing a subsequent command.

If you use File > Open Dictionary in the Script Editor and select the ZephIR application you'll see that it offers a repeating and delaying option:

fire zephir command v : Send a fire command to the ZephIR™.
fire zephir command text : name of command
of component text : the target component
[repeating integer] : override the designated repeat count of the code (minimum 1, maximum 35)
[delaying integer] : wait the number of ticks (60 ticks per second) before firing (minimum 0, maximum 300,default is 10)
→ text : The reply returns a status string if the request fails otherwise it is empty.

Below is a first try which allows to set normal autofocus or instant autofocus with a single mouse click. All I have to do is to make sure that the ZephIR aims at the remote sensor of the HF100 before I press a button. Watching the ZephIR wading through the menus is fun and about ten seconds later the new AF mode is set. Note that the connection depends on a light path and that the ZephIR acts blindly. If there's an interruption in the connection either through bad placement or by unintentionally concealing the connection you'll end up anywhere in the menu. Also note that the following example is for the HF100. The HF10 let's you choose between the SD card and the internal memory so there may be one additional menu item. Adjust the code accordingly.

The AppleScript for setting iAF looks like this:

(*SET iAF*)
fire zephir command "MENU" of component "Canon HF100 WL-D88"
fire zephir command "RIGHT" of component "Canon HF100 WL-D88" delaying 120
fire zephir command "DOWN" of component "Canon HF100 WL-D88" delaying 60
fire zephir command "DOWN" of component "Canon HF100 WL-D88" delaying 60
fire zephir command "DOWN" of component "Canon HF100 WL-D88" delaying 60
fire zephir command "SET" of component "Canon HF100 WL-D88" delaying 60
fire zephir command "UP" of component "Canon HF100 WL-D88" delaying 60
fire zephir command "SET" of component "Canon HF100 WL-D88" delaying 60
fire zephir command "MENU" of component "Canon HF100 WL-D88" delaying 60

As you can see the required AppleScript is straightforward and the possibilities are numerous. Everything you can do with the wireless controller can be simulated and even more.

Controlling a timelapse recording where the HF100 takes a photo every minute for one hour requires just four lines of AppleScript. Note that storing a photo takes about 4 seconds so the minimum possible interval is 5s.

(*TIME LAPSE*)
repeat 60 times
   fire zephir command "PHOTO" of component "Canon HF100 WL-D88"
   delay 60
end repeat

The HF100 only offers a two second SELF TIMER delay when it is triggered by the wireless controller. You also have to activate the SELF TIMER in the menu before each and every recording. The following code allows one-click self timed recordings with a delay of 10 seconds. The "beep sound" is the actual system alert sound which can be altered in the OS X System Preferences.

(*SELF TIMER*)
repeat 10 times
   beep
   delay 1
end repeat
beep 2
fire zephir command "START-STOP" of component "Canon HF100 WL-D88"

How's about a remote controlled 10s recording?

(*TIMED RECORDING*)
fire zephir command "START-STOP" of component "Canon HF100 WL-D88"
delay 10 + 1
fire zephir command "START-STOP" of component "Canon HF100 WL-D88"

Smooth computer controlled zooming is now easy (set the zoom speed to the slowest speed ">" in advance):

(*ZOOM IN*)
fire zephir command "ZOOM T" of component "Canon HF100 WL-D88" repeating 35 delaying 1

Precise focus shift or rack focus (press set and toggle to FOCUS first). The correct repeating number must be found by trial and error:

(*FOCUS SHIFT*)
fire zephir command "UP" of component "Canon HF100 WL-D88"
fire zephir command "RIGHT" of component "Canon HF100 WL-D88" delaying 60 repeating 20

The following code initalizes the card (use it with caution!):

(*INITIALIZE CARD*)
display dialog "This will initialize the card. Continue?" 
fire zephir command "MENU" of component "Canon HF100 WL-D88"
fire zephir command "DOWN" of component "Canon HF100 WL-D88" delaying 120
fire zephir command "RIGHT" of component "Canon HF100 WL-D88" delaying 60
fire zephir command "DOWN" of component "Canon HF100 WL-D88" delaying 60
fire zephir command "SET" of component "Canon HF100 WL-D88" delaying 60
fire zephir command "SET" of component "Canon HF100 WL-D88" delaying 60
fire zephir command "DOWN" of component "Canon HF100 WL-D88" delaying 60
fire zephir command "SET" of component "Canon HF100 WL-D88" delaying 60
delay 5
display alert "The card has been initialized." 
fire zephir command "SET" of component "Canon HF100 WL-D88"
fire zephir command "MENU" of component "Canon HF100 WL-D88" delaying 60

The ZephIR application itself is scriptable. A slightly altered version of the 10s recording example above would look like this in Apples Script Editor (or any application that supports AppleScript):

AppleScript is very powerful just take a look at the AppleScript Language Guide for details. Right now I'm only scratching the surface.

Next you can see the AppleScript integrated into an Automator workflow that copies the recordings to your computer. Unfortunately you can't access the SD card in recording mode. You must manually switch to playback mode and connect the USB cable. The camcorder turns into a "dumb" harddisk and only becomes responsive again when you eject the camera and disconnect the USB cable. Considering this it's far easier to take the SD card out and put it into a card reader.

So far I only controlled recording but imagine using the ZephIR connected to a MacBook Air to control a High-Def presentation delivered by the HF100.

HF100 color comparison

AVCHD support on the Mac

Editing MPEG4 needs a lot of processing power. It's highly compressed and to achieve this just a few frames contain the entire image information. In the majority of frames just the differences from previous frames as well as predictions for future frames is stored. That means that the editing software must constantly decode a lot of neighbouring frames to restore the frame you're currently working on. This is much more demanding than MPEG2 editing and even the latest Mac Pro has a hard time to do this fluidly. iMovie 08 can edit pure MPEG4 video with AAC audio natively (not AVCHD!) but forget about the scrubbing feature if you try this. Of course, over time when computers get faster this will change.

If you consider the current situation, it's nothing but clever by Apple to convert all AVCHD clips to the Apple Intermediate Codec (AIC) since you wouldn't be satisfied with the user experience of native AVCHD editing on todays computers. The AIC is a high quality MPEG2 intra frame codec that is easy to handle on all current Macs. Intra frame means that each frame is compressed individually which is very edit friendly. The conversion is not lossless but you will not see any quality loss just like you will not see a quality loss in JPEG compressed images at high quality settings. It's generally a good idea to leave the highly compressed MPEG4 codec as soon as possible.

AIC clips are QuickTime movies that can be played on Windows or OS X. The screenshot below shows the QuickTime Player info window of an AVCHD clip that was converted to the Apple Intermediate Codec by iMovie 08. As you can see the data rate is very high which ensures quality. The Dolby encoded audio of the HF100 is converted to 48 kHz uncompressed stereo audio.

The only disadvantage of the conversion is that your compact .MTS clips become much, much larger. Typically the factor is eight so a 1GB AVCHD clip will occupy 8 GB of hard disk space after conversion. An external 500 GB disk that is dedicated to video editing for instance will hold about 7 hours of AIC encoded video from the HF100.

As of May 2008 all video editing solutions from Apple convert AVCHD. iMovie 08 and Final Cut Express 4 convert to the Apple Intermediate Codec while FinalCut Pro has an additional ProRes option with even higher quality (and file size). Note that the current Final Cut Express 4 doesn't support the FXP mode of the HF100. It still uses 1440x1080 (the HDV standard) intern and scales up to full HD.

AVCHD import is only supported on Intel Macs. Users of older Macs or iMovieHD can use the utility VoltaicHD from shedworx.com

At the moment (May 2008) only the video player of Toast 9 can play .MTS AVCHD files directly.

Update November 2008: Adobe Premiere Pro CS4 allows native AVCHD editing on a Mac.

Backup of HF100 AVCHD clips

If your video editing software is able to import .MTS clips directly then just copy the STREAM folder on your SDHC card to your computer and rename the folder. Give it an unique name (e.g. event_date). Then you can burn up to half an hour of AVCHD clip folders on a DVD-ROM or up to one hour on a double-sided one and store it away.

iMovie 08 and Final Cut Express 4 don't import .MTS files directly. They import from a supported camcorder, the SDHC card, a mounted disk image or a DVD-ROM. In order to be able to re-import the HF100 clips you must backup the entire BDMV folder of each card. This folder contains the STREAM folder as well as thumbnail and clip info folders that are needed by the import window. To backup the videos on a SDHC card just copy the BDMV folder to an unique named folder on your computer.

To be able to re-import into iMovie you can burn the BDMV folder to a accordingly labeled DVD-ROM. iMovie 08 will recognize a mounted DVD as shown below and automatically open its AVCHD import window.

The second possibility is to use a disk image. Open Disk Utility and go to File > New > New Blank Image... Choose a volume size thats large enough for your BDMV folder and click OK.

Copy the BDMV folder to the mounted disk image and start iMovie 08. iMovie 08 will recognize a disk image as shown below and automatically open its AVCHD import window. You can also archive such disk images.

Another option is to make a disk image of the entire SDHC card using Disk Utility. To do this open Disk Utility, click on the SDHC card then go to File > New > New Image from ... An exact copy of your SDHC card will be saved to your boot drive. Archive, mount and import such an image as explained above. The only disadvantage of this method is that the disk image size is always the size of the SDHC card even if it's half full.

I've tested this with iMovie 08 version 7.1.1 but I strongly recommend that you test it too on your computer before relying on it.

UPDATE: If you have to backup a 16 GB SDHC card to standard DVDs move .mts files that do not fit on the first DVD into DVD2, DVD3 and DVD4 folders and burn just the .mts clips to additional DVDs (I'm assuming that no single clip is longer than 30 minutes e.g. <4.7 GB).
To restore the card copy "DVD1" with the original folder structure to the harddisk then fill the STREAM folder with the missing .mts clips from DVD 2, 3 and 4.
For import into iMovie '08 use "Make image from folder" in Disk Utility and mount it. I don't know if Final Cut Express 4 needs a mounted image but Final Cut Pro 6 imports just fine from a folder with the original structure.

Original Canon HF100 AVCHD clip

Original Canon HF100 AVCHD clip from Martin Koch on Vimeo.

Just in case you want to test AVCHD import before buying this camcorder.

HF100_PF25_NEUTRAL.MTS (10MB)
Original .mts file for direct import or for conversion using VoltaicHD from shedworx.com

HF100_SDHCcard.dmg (10MB)
Intel Mac users can mount this disk image to import the clip into iMovie 08.

Enjoy!

Please note that the corners are blurry because I zoomed in too far with the 0.5x wide angle converter on the camcorder. This was nonsense since a wide angle converter is not designed for tele settings (I learned this by now) and does blur the corners when you zoom in too far. I also didn't use a required 37/37mm adapter ring. That's why the corners are so blurry. The HF100 alone will not blur any edges regardless of zoom setting.

Where are the video files and photos?

The screenshots show the folder structure of a SDHC card recorded by the HF100

Buried deep, the video files:

The place where JPG compressed photos end up:

Using a gray card

Using a gray card with the HF100 from Martin Koch on Vimeo.

A gray card is a great way to adjust a "technically" correct exposure in controlled lighting conditions that don't change during shooting. It can also be useful to set the exposure in difficult scenes like e.g. an object in front of a bright background. You'll find gray cards at any serious photo store.

Put the gray card under the same light as your main object. Point the camera at the card, step close or zoom in on it and adjust the exposure in manual mode.

On the HF100 there's no manual mode. You can either lock a certain arperture (Av mode) or shutter speed (Tv mode). In the video above I show you how to display the exposure settings, how to lock the shutter speed to 1/50s and then how to also lock the arperture (and gain) setting chosen by the HF100.

How the HF100 controls exposure

If you press the Photo button on top of the camera half down the current arperture and shutter speed get displayed. You will notice that these two settings change very little when you point the camera to brighter or darker scenes.

Unlike a professional camcorder the HF100 does a lot behind the curtain. To control exposure it uses:

• Shutter speed
• Arperture
• Gain in low light
• Most likely a variable neutral density (ND) filter in bright light

So rather than changing the arperture or shutter speed, gain or ND filter come into play. A ND filter does no harm it only controls the amount of light but gain is bad. Gain is electronic amplification of the sensor signal similar to the ISO setting on a photo camera. As you probably know a small sensor and high ISO means noise. The same is true with high gain. Just like you want to shoot at ISO 100 to get good looking pictures with a minimum of noise you want to shoot at zero gain.

I don't know a trick to display the gain used by the HF100. If you read this article about controling exposure on the HV20 you'll see that the only way to expose the gain setting of a particular recording was to put the tape into another camcorder model and to display the info there.

The Remote Control

I appreciate that camcorders always come with wireless remote controllers. It's so handy to be able to control the HF100 without ever touching it. The remote sensor on the camera is large and placed at the upper left corner of the display. I would prefer a slightly larger remote though.

Since I find the placement of the Photo button on top of the camera not very convenient I prefer to trigger my photo shots with the remote control. This has the added benefit that there's no camera shake due to pressing the button.

The Battery Pack

The HF100 is tiny and so is the supplied BP-809 battery pack (7.4V 890mAh Li-ion). The battery is "intelligent" and reports the remaining usage time with an accuracy of one minute to the camcorder. A fully charged battery reports 80 minutes but in real life it's much less.

I took the HF100 with a fully charged battery and a 4GB SDHC card to a trip to Graz. My concern that the card would be too small was unneccessary. I recorded about 100 very short clips over a time span of several hours, didn't review them and closed the LCD between most recordings. Note that closing the LCD turns on a power safe mode. I also completely turned off the camcorder when not in use for a longer period. Mind you I still had recording space for about 15 minutes when the battery was empty. So unless you let the camcorder record continuously the effective output of the standard battery will only be between 15 to 30 minutes of footage since camcorder preparation and framing shots use up a considerable amount of battery time.

Canons marketing people obviously know that you need a second battery so they price this spare part accordingly. Alternatively you can buy the very expensive BP-819 battery pack with more capacity that will protrude from the back. Currently there are no third party Li-ion batteries available but that should be only a matter of time. If you can pass on the "intelligence" cheaper alternatives with their own charger will certainly do.

Update: I ordered a cheap battery plus charger last week from a Hong Kong eBay dealer and will post my experience when it arrives.

The battery gets charged in the camcorder within 155 minutes. That means you can't use it while you are charging. If you use two batteries the camera is occupied twice as long and when the second one is charged the first one no longer is "fresh". An external charger is available as an - again rather over-priced - accessory.

If you connect the camcorder to the supplied compact power adapter you can use it without inserted battery.

A less tall alternative to the BP-809 is called BP-808.

UPDATE: My battery problem is solved!

HF100 + Wide Angle Converter

Doesn't it look cool? I've chosen the Raynox HD-5050PRO. It matches the camera perfectly and it costs less than the WD-H37C 0.8x wide-converter offered by Canon. The HF100 now feels very professional and valuable because of the added weight.

Read the comment by Gareth about the supplied 37/37mm adapter ring then read my observation.

The widest angle of the HF100 lens equals 43 mm on a 35 mm camera. This is very little wide angle - in fact that's no wide angle at all - so a wide angle converter will help you out in tight places. A 0.5x wide angle converter will widen the view to a 22 mm equivalent and show twice as much.

The HF100 doesn't display the actual zoom values, just a tiny bar. Below is the approximate 35 mm equivalent zoom range of the HF100 without and with 0.5X wide angle converter. I retrieved the values from the JPEG Exif data and rounded the figures for the 1/4, 1/2 and 3/4 positions of the zoom display so you can easier memorize them. Since the the midpoint between W and T is the only zoom position that can be set with a certain accuracy these very rough figures will suffice.

Basically this means you can use an 0.5x wide angle converter up to half of the zoom range. The rest is (and should be) covered by the HF100 without converter.

Below is a wide angle shot at f8 with the Raynox HD-5050PRO mounted. I think the converter is pretty good. It's considerable sharp at the borders and there's no darkening of corners visible. Note that this is only true for wide angle shots. Don't use such a converter in tele mode because it's not designed for this and the image gets very unsharp at the corners. Download the original JPEG.

Below are three scenes without and with the converter. The zoom was always at it's widest setting.

HF100 without and with 0.5x wide angle converter from Martin Koch on Vimeo.

Note that 22mm is an extreme wide angle and if you put a filter on the converter it will be visible at full wide angle.

My very first movie

Chickens from Martin Koch on Vimeo

I have yet to master the manual settings so I relied on P mode and auto white balance. I used the "Neutral" digital effect to get less saturated colors. Download a less compressed version at Vimeo (after logging in).

All in all the quality is impressive, especially when you consider how tiny the sensor is. You can download a sample clip that has been transcoded to the Apple Intermediate Codec without visual loss of quality. This 185 MB clip can be watched at full size and 25 fps on a very fast computer using the QuickTime Player.

First impressions

The HF100 is small. Images can't give an impression how small this camcorder really is. You must see it to get a feeling for its size. Although there are smaller AVCHD devices I wouldn't want a camcorder any smaller.

The casing is made of plastic material with a solid, quality feel. The articulating display opens and closes easily. It's held close by spring tension, there's no latch. I like the fact that there's no need for a lens cap.

413 grams with battery

Difference between the HF100 and the HF10 model

The HF10 differs from the HF100 in two things

1. The HF10 has additional 16 MB internal memory
2. The HF10 is black

I prefer the HF100. I like the much simpler memory management. Having to manage two storage locations looks confusing to me. SDHC cards are so small that it's no problem to have spare cards in the camera bag. In case the camera dies I also see a problem because I will not be able to erase video recorded to the internal memory. I don't feel comfortable when I imagine a service technican watching my private video. So I'm not loosing anything when I buy the cheaper model. In fact it's the more attractive model for me.

Storage Format

The HF100 compresses video to the very efficient MPEG-4 AVC/H.264 compression format. AVC stands for Advanced Video Codec.

AVCHD on the other hand is just a strict recording format specification for high definition (HD) digital video cameras to ensure compatibility with editing software.

AVCHD Format Specifications¹

Video Signal	1080/60i, 1080/50i, 1080/24p
Pixels	1920x1080, 1440x1080
Aspect ratio	16:9
Video compression	MPEG-4 AVC/H.264
Sampling structure	4:2:0
Audio compression	Dolby Digital (AC3)
Audio bit rate	64 to 640 kbits/s, ?
Audio channels	1, 2, 3, 4, 5.1 channels
System	MPEG-2 Transport Stream
System bit rate	up to 24 Mbits/s (17 Mbits/s)

¹I only included the specifications that apply to the HF100. Specifications of the HF100 PAL model are in bold. The full format specification overview can be found at http://www.avchd-info.org/format/index.html

The HF100 uses only 60% of the maximal allowed AVCHD system bitrate of 24 Mbits/s. As you can see there's no 25p in the specification. In fact the "progressive" frames of the "PF25" mode are converted and stored in interlaced 50i format. This camera doesn't store real progressive frames!

Storage Media

The HF100 records on small SDHC cards. HC stands for "High Capacity". The old SD standard only allowed storage capacities of up to 4 GB while the SDHC standard allows capacities up to 32 GB. If you use a card reader to transfer the video to the computer make sure it is compatible with SDHC cards.

An important thing is the speed class specification printed on the SDHC card because it guarantees minimum transfer rates. A Class 2 card must write and read at least 2 MB/s, a Class 4 must handle 4 MB/s and a Class 6 must offer at least 6 MB/s.

The maximum bit rate of the HF100 is 17 mbits/s which gives 2.13 MB/s (17/8bit). That's a little over the Class 2 specification so you need a Class 4 card to record video in the highest possible quality. The advantage of using a Class 6 card in this camera is faster transfer speed to your computer. The Class 6 SDHC card shown above for instance can handle up to 20 MB/s and will copy your video files five times faster than a Class 4 card.

A 4 GB card holds about 30 minutes of video in the FXP (17 mbits/s) recording mode.

Sensor Size

The HF100 has a 1/3.2" sensor but I have no clue how large the sensor really is because of the odd way sensor sizes are defined. Mind you the 1/3.2" is not the diagonal size of the sensor but the standard diameter of a TV camera tube from the 50's! Find an explanation at dpreview.com.

I also don't know for sure if the sensor has a 4:3 or 16:9 format. Canon doesn't tell in the specifications. The video of the HF100 is 16:9 but it also has a 4:3 photo mode.

I show the size of a typical 1/3.2" 4:3 sensor above. If the sensor in the HF100 has a 16:9 format it could be somewhat less tall and somewhat wider.

Nevertheless a 1/3.2" sensor is unbelievable small. It's a mircacle how manufacturers can squeeze pictures out of them. Try to draw the size shown above using a metric ruler to get an impression. Do it!