man daviesh

Disclaimer: This is partly for my benefit in case I lose my batch files and have to re-derive these techniques at some point. As such, the language is pretty techy, but this might be handy for other people.

These days we need to format-shift content a million different ways. Xbox, Playstation, iPod, etc all require different formats to get the most out of the platform. The Xbox 360 can’t deal with modern container formats like .mkv and .mp4, while the Playstation and iPod don’t implement all the codec features available to .mp4 files. There are a number of specialised tools available on the ‘net for around the US$20 that can convert anything you throw at it to a particular format, but none of them give you complete control. A program I won’t name for my iPod decided to file 16×9 video to my iPod screen by chopping the sides of the image off. That’s not what I wanted!! Fortunately, open source tools I originally found in linux land can do everything I want at no charge and switch to a fresh Windows 7 install proved a good time to throw out my old way of doing things. This was also inspired by Ashton Mills’ final X-Ray in issue 104 of Atomic on transcoding (pick it up, he explains things better than I can!). Before we start, remember to only tinker with content you have the rights to and everything from here on works equally well in Windows and Linux.

I entered this with two needs, firstly to archive videos into a format that doesn’t take up too much space and secondly to have a robust method of shifting content onto my iPod. I have had a fair bit of experience with the xvid and divx video codecs in the past, but the quality isn’t so good at smaller file sizes (say, 1-1.5GB for 100 minutes of video). Fortunately, we have the x264 codec which is a new generation codec and the open source version of the h264 standard. It also just so happens to be the codec that iPods and the like expect in the .mp4 container! Another huge reason to use x264 is that it multi-threads very well, so can make the best use of my quad core processor. I find older codecs struggle to load up more than 2 of my processors. The downside of x264 is that it is not really supported in the venerable .avi container, so don’t expect .avi files with a x264 video stream to work 100% of the time unless you are using a smart media player.

The other tools we need for the job is to ensure all the codecs we need are installed (grab missing ones here at free-codecs.com) as well as mencoder and mp4creator. I already us MPlayer as my media player in Windows and Linux, so using the encoder that comes with it seems logical. Windows binaries of mplayer/mencoder are available here and in linux can grabbed from your package manager (in Fedora/Red Hat yum install mplayer mplayer-gui mplayer-doc mencoder, similar apt-get command in Ubuntu or Debian). The final puzzle piece of mp4creator is available from Sourceforge. Note that all these are command line tools, so it is a good idea to add the directories where you installed mencoder and mp4creator to your Windows PATH or you will need to type out the full command path EVERY TIME in your batch files. All the commands I describe from here on can be just added to a batch file or shell script and left to run.

So lets look at my basic archiving sequence step by step, then I will add some sample batch files for other tasks at the end. We will use 2-pass encoding for very good reasons.

mencoder -sws 9 -vf scale=768:432,hqdn3d=4:3:6,
unsharp=l7×5:1,pp=ha:128:7/va/dr/al,harddup ovc x264
-x264encopts pass=1:turbo:bitrate=600:bframes=1:
me=umh:partitions=all:trellis=1:subq=6:frameref=5:
direct_pred=auto:keyint=120:threads=auto -oac faac
-faacopts br=192:mpeg=4:object=2 -channels 2
-srate 48000 inputfile -o outputfile.avi

OK, so this command (all on one line!!!) is the first pass of encoding process. A lot of the options I copied from Ashton’s guide in Atomic and only changed stuff that didn’t work for me. Important things to note:

• scale=768:432 - This at the start of the video filter (vf) chain scales the video to the width x height you want. This value is for a 16×9 movie at DVD resolution, use scale=720:576 for a 4×3 movie. For reference, 720p is scale=1280:720.
• hqdn3d - this is the high quality denoising filter. This can help encoding a lot by cleaning out the noise and making the frames more similar
• harddup - this option at the end of vf chain forces duplicate frames to be copied. This may not be needed for every container format and codec but without this option duplicate frames can be dropped, which saves file space but causes the video and audio to drift out of sync. Not good!
• pass=1 - this tells the codec to run through the video and write out a log file.
• turbo - I don’t care about video quality on the first pass as all I want is the log file, so the codec can run in turbo mode.
• threads=auto - Tells the codec to use as many CPUs as it can find. If you want to restrict it, you can use a number instead of ‘auto’.
• inputfile - This can be any video file you like, even a DVD title. If you want title 1 from the DVD in D:, the inputfile would be -dvd-device D: dvd://1.

Other options we could have used is -aid to specify the ID number of the audio track we want, -sid for the subtitle track, -af volume=15 to add 15db to the audio track. If instead of the 2 channel AAC audio we wanted to keep source audio (like the surround sound track from a DVD), we just replace everything from -oac to inputfile with -oac copy. If the source video is interlaces (only every second line of the video is refreshed at a time), we can deinterlace it with yadif (Yet Another DeInterlacing Filter) at the start of the vf chain.

mencoder -sws 9 -vf scale=768:432,hqdn3d=4:3:6,
unsharp=l7×5:1,pp=ha:128:7/va/dr/al,harddup ovc x264
-x264encopts pass=2:bitrate=1000:bframes=1:
me=umh:partitions=all:trellis=1:subq=6:frameref=5:
direct_pred=auto:keyint=120:threads=auto -oac faac
-faacopts br=192:mpeg=4:object=2 -channels 2
-srate 48000 inputfile -o outputfile.avi

For the second pass, all we do is drop the turbo and increase the bitrate, as we actually care about the output now. We also need to set pass=2, so the codec knows to read from the logfile we created in the first pass. This leaves us with a .avi file that has x264 video and aac audio. To convert to mp4, we first need to extract the video and audio streams with mplayer:

mplayer outputfile.avi -dumpaudio -dumpfile outputfile.aac
mplayer outputfile.avi -dumpvideo -dumpfile outputfile.h264

And mux them back together with mp4creator:

mp4creator -create=outputfile.aac outputfile.mp4
mp4creator -create=outputfile.h264 -rate=25.0 outputfile.mp4
mp4creator -optimize outputfile.mp4

The -rate argument to mp4creator is the framerate, PAL content runs at 25fps.

At the end of all this, we have a roughly 1.2-1.5GB file which gives a good quality archive of video. Using the same technique, we can get a bunch of different formats. My most common first passes are below:

iPod - note that iPods are superduper picky about your x264 encoding options. The -ofps is the output fps (iPods expect 30fps) and the format=ipod will take care of the messiest configuration. We output an mp4 directly in this command, no post-processing.
mencoder -ofps 30 -sws 9 -of lavf -lavfopts
format=ipod -vf scale=-10:240,dsize=320:240:0,
harddup -ovc x264 -x264encopts bitrate=768:
vbv_maxrate=1500:vbv_bufsize=2000:nocabac:me=hex:
subq=4:frameref=2:trellis=1:level_idc=30:global_header:
threads=auto:pass=1 -oac faac -faacopts
mpeg=4:object=2:br=160:raw -channels 2 -srate 48000
inputfile -o outputfile.mp4

Xvid - Does essentially the same as the original command, but as an xvid .avi file with MP3 audio for devices that can only read that. We only need the output from the second pass, no post-processing. As this is the xvid codec, most of the options are different to the x264 command.
mencoder -sws 9 -vf scale=768:432,hqdn3d=4:3:6,
unsharp=l7×5:1,pp=ha:128:7/va/dr/al -ovc xvid
-xvidencopts bvhq=1:vhq=2:chroma_opt:quant_type=mpeg:
turbo:pass=1 -oac mp3lame -lameopts vbr=3
inputfile -o outputfile.avi

Have a play around until you find what works. Every source is different, so keep batch files for common tasks that you can customise and optimise for the current task.

Notes:
mplayer/mencoder docs: http://www.mplayerhq.hu/DOCS/HTML/en/index.html
mencoder man page: http://linux.die.net/man/1/mencoder

Today I decided to pull out a spare hard drive I had lying around and install the Windows 7 release candidate I had downloaded a couple of weeks ago. My experience with Vista has not exactly been smooth, so I was interested to see if my problems had been addressed and to look at the new task bar that has been discussed so much in the IT media.

The process for installing Windows was considerably improved with Windows Vista. Windows 7 sees this even more refined, basically all you need to do is enter your language and keyboard layout, go away for 20 mins then come back and enter your username and a couple of other minor settings. Quick and smooth with no need for you to hang around while Windows installs.

It seems that pretty much any hardware that works under Vista works under Win7 with no problems. Windows detected that my computer was 64-bit capable, so I have 64-bit Win7 installed. I am not sure if that is intentionally automatic. The latest Microsoft certified drivers automatically appeared in Windows Update.

The new task bar is great, all the windows of the same program get stacked together and clicking on the icon of the program give you a preview of all the open windows, allowing easy jumping between windows. This is still not as flexible has having multiple workspaces like KDE and GNOME have under Linux, but this is still a good improvement. Minor changes to the start menu are welcome, with it now obvious whether the ‘turn off’ button is actually going to turn off the machine.

Windows Media Player has been improved and seems to play all the common media codecs out of the box. The is fantastic, as most of my Vista media problems were due to it not playing nice with codecs like DivX. In the end, I compiled the linux program SMplayer for windows to give me a more reliable media player. The only common codec I have had to install myself so far has been for .mkv files.

So far, the new Windows gets a cautious two thumbs up. I have to admit I am waiting for it to blow up in my face, but so far so good.

Recently I have replaced my aging Dell Inspiron 8600 with a new Studio 15 laptop. Things have changed a lot in the laptop market in the last 4 years, I bought a laptop in the same relative place in the product lineup for half what I paid for the old one. The one I have is ‘Midnight Blue’, one of a VAST array of colours and styles. It is a sort of dark blue, but the colour changes a bit in different light.

First off, some of the good points. I have never been a fan of glossy laptop screens due to glare issues, but the screen on this machine is quite good. I got the upgrade to a 1440×900 native resolution screen which give crisp, clear images, especially while watching media. The selection of ports on the sides of the laptop are good, with plenty of USB ports, wireless and wired ethernet, bluetooth, firewire, eSATA (which will become more important in the future) and mostly importantly, HDMI. The HDMI output is SO much better than the old DVI/D-Sub out on other laptops, you plug in the cable and it just works. Sound is also pumped through HDMI. The keyboard is a good size, and looks good backlit (I have this option) while the trackpad is also responsive. Finally, the upgraded 9 cell battery lasts a good 4 hours or so on light work and movie watching (from hard disk, not DVD) and raises the base of the laptop into a comfortable typing position.

On the downside, my biggest gripe is with the speakers. They are tinny and have little frequency response in the lower half of the audible range. Other minor things are the use of a slot loading DVD drive to save space (I prefer tray loaders). The graphics card I have is the best you can get, an ATI Radeon 3450. This means you can’t play the newest games, but it is good enough for most applications. This reinforces the purpose of the laptop as a multimedia machine, not a hardcore gaming laptop. Importantly, the 3450 has the grunt for multimedia work (what I got the laptop for) and driving the HDMI port.

Overall, the Studio 15 is a good buy, with the usual excellent Dell service and customisation options. As with all Dell laptops, it is worth waiting until a good online special comes up. It may not be a true desktop replacement able to do anything that is thrown at it, but it does balance well battery life and portability with the flexibility of a desknote.

Hey guys, it’s been a long time since I did one of these. Not much has been happening, just chugging my way through the work roster. Some of you know I have joined the league of cyclists a couple of months ago. I have a really nice bike, a Giant Cypress. Now that it is kitted out with a bag rack and some pannier bags, it is a great way to get to and from work (except during a build-up storm!).

Speaking of the weather, we are well and truly in the build-up now. The first week of October was horrible with 36+ degree days, then this last week has been hard to forecast, with storms popping up where they seemingly shouldn’t. It has been fairly busy and, of course, all the concreters are ringing again to find a good time to pour. The answer of course is “the dry season”!

A couple of cool things I have come across recently. Firstly, Dr Horrible’s Sing Along Blog is FINALLY available on the Australian iTunes store and is FANTASTIC. It was created by Joss Whedon (Buffy, Angel, Firefly, etc) during the writers strike and is only available as a web series. It stars Neil Patrick Harris (How I Met Your Mother, Starship Troopers, Harold & Kumar), Nathan Fillion (Malcolm Reynolds in Firefly) and the extremely talented Felicia Day (Buffy and her own web series The Guild). Great, catchy songs hold together an intentionally cheesy plot, making for a funny 45 minutes in total.

The other thing is Star Wars: The Force Unleashed on the 360. It has been getting decidedly average reviews on the various websites, but really I found this to be the Star Wars game I always wanted to play. Kickass graphics, cool looking force powers, great boss fights and an ultimate choice between good and evil make this a really compelling game. The storyline beats the 3 most recent Star Wars movies, showing the introduction of a lot of characters into the canon (Mon Mothma and Garm Bel Iblis), the formation of the Rebel Alliance, Vader’s Jedi hunt and more. If you have a next gen console and even remotely like Star Wars, this one is for you.

So I have been to see the latest movie that has bolted out of the Wachowski stable and I have to say………I liked it! This is totally a Wachowski film, with a lot of really vibrant, stylistic colour and amazing effects. It is pretty obvious that most of the movie was shot in front of a green screen.

Spritle Racer and Chim Chim (pictured above) are well used in the movie to provide the laughs every time them movie heads into ’serious’ territory or starts to slow down. Those guys pop up EVERYWHERE, and whoever trained the chimp deserves major kudos. As a whole, the movie is great with stacks of action sequences and crazy racing. John Goodman is a great Pops Racer and Roger Allam has been given a great role (the Wachowskis must have loved working with him for ‘V for Vendetta’).

Don’t expect a high class movie, go to see it expecting a fun romp through a fantasy world and you won’t be disappointed. It is an appropriate homage to the animated series, and the remix of the original theme at the end of the movie is kind of catchy.

It’s been a while since I have done one of these, but I guess there has not really been a lot to talk about. I’ve been playing a bit of golf (badly!) with the guys from work, but mostly I have been in the routine of going to work, then catching up on all my domestic stuff on my days off. I have been to see Ironman at the movies, it is really awesome, anyone vaguely interested in action/comic book movies MUST see this, Robert Downey Jr really is great as a smart-arse Tony Stark.

Lately I have been playing a bit of Mario Kart Wii. If you have played a Mario Kart game before, you pretty much know what to expect. The new tracks are even more outlandish and change slightly on each lap. The ‘retro’ tracks are great blasts from the past, especially the old SNES battle mode course. One of the bigger changes is upping the number of competitors in each race from 8 to 12. This makes it more likely that you will suffer a CASCADE of adverse item effects sending you rapidly from frontrunner to distant last in a race. The online modes are the biggest new feature, the Nintendo WiFi Connection is implemented seamlessly and playing against ‘real’ people is so much fun.

Another game I started playing a day or so ago is Penny Arcade Adventures - On a Rain-Slicked Precipice of Darkness: Episode One. Short title there. As you may have gathered, it is based on the characters of the Penny Arcade web comic and is very true to the humour and animation style of the comic. Very funny game, and at roughly $20 it’s a real steal.

OK, I haven’t done one of these in a while. I’ve just been too busy!

Darwin is coming out of the wet season now and becoming a much more comfortable place to live. You can comfortably breathe the air again instead of drinking it and the nights are getting colder. I went on a round of golf with some of the guys from work, and the daytime heat was not that oppressive, showing that the large drop in humidity makes a huge difference. With the weather clearing up, the markets are starting up again at Mindil Beach again next week, and the Deckchair Cinema has reopened.

Just came across a really good cheap game on Steam called AudioSurf. Basically you can pick any music file you like on your computer and the program turns it into a puzzle game where you have to collect coloured blocks as you race down a track. The blocks are proxies for notes in the song, the track undulates to the beat and as the song builds up or winds down, the track goes up or downhill. Depending on the car you use, you have different powerups, some of them are an easy mode where there is only one colour of blocks, others have the ability to scramble up blocks or delete certain colours. It is a GREAT way to listen to your music and is only US$10. I created a short video showing how AudioSurf interpreted ‘Your are a Pirate’.

Anyway, check it out, it’s a great break from Hellgate and will tide me over until MarioKart Wii comes out!

…Blu-Ray! In a statement released by Toshiba today, HD-DVD is the new Betamax video tape.

Toshiba Corporation today announced that it has undertaken a thorough review of its overall strategy for HD DVD and has decided it will no longer develop, manufacture and market HD DVD players and recorders….”We carefully assessed the long-term impact of continuing the so-called ‘next-generation format war’ and concluded that a swift decision will best help the market develop,” said Atsutoshi Nishida, President and CEO of Toshiba Corporation.

The interesting thing about this is that HD-DVD is actually the official successor to DVD, with the standard ratified by the DVD Forum. HD-DVD devices were cheaper than Blu-Ray devices because much of the tech was lifted out of conventional DVD players. Goes to show what marketing can do for you. Sony ‘won’ with Blu-Ray because it negotiated with movie studios and made the cheapest Blu-Ray player (the PS3). Stunts like throwing in a free PS3 with a big Bravia screen over Christmas also helped.

What does this all mean? Hopefully, it means that as the market concentrates on one format, the consumer install base hits critical mass and everything can become cheaper. At the moment, the price of discs and players is similar to what DVD was in the late 90s, this needs to change for the format to gain traction. Also, this will probably motivate fence-sitters to go out and buy a PS3.

On a seperate, but not totally unrelated note, I got my Austar PVR called ‘MyStar’ installed last week. It was pretty glitchy the first couple of days, but has settled down now that I know when I need to baby it. The series link function is great for automatically recording a whole series and the 2 free-to-air tuners are pretty good. Hopefully the firmware updates keep coming, I am still having to whack the reset button every couple of days. It would also be good if access to Foxtel’s HD channels will be enabled via firmware update. My only real gripe with the unit is the lack of outputs, but it has component connections, so the quality isn’t bad.

Let me put it this way. Spamalot is INCREDIBLE. It is a very tough call to say if it exceeds ‘The Producers’, but I would happily go and see either of them over and over again. There are a lot of Python in-jokes, but 99% of it works even if you haven’t seen a bit of Python material in your entire life.

Lucinda Shaw as the Lady of the Lake is incredible, I have no idea how she has any voice left after a show! Bille Brown is a great King Arthur. Together with the rest of the cast they keep the charm of the original movie while adding their own unique spin to it. The songs are incredibile and fit beautifully with the storyline and the song and dance sequence at Camelot is just crazy fun. They have also stretched out parts of the movie that people loved so much, like the French taunting.

Anyone who has the chance to see the show I STRONGLY recommend it, it’s in Melbourne for a while longer and is also still playing in the US on Broadway and in Vegas, as well as in London.

About this time last month I posted a gut reaction to Kevin Rudd’s grand plan for mandatory internet filtering. I was reminded of this on the Whirlpool forums today after reading some posts from one of the guys from Internode (a large internet service provider). This one in particular addresses things well: Data Blocking

Essentially, the point is that copyright holders have a vested interest in keeping their material off the net, yet with billions of dollars, fairly uniform copyright laws across the globe and complete cooperation from ISPs everywhere, they have been 100% unsuccessful in stopping the flood of pirated movies, music, etc. Now with adult content, you have no uniform standards of what is acceptable around the world, different personal standards and far less in the way of monetary resources to ‘clean’ the internet. No filter is going to work on that basis, and we haven’t even looked at technical issues yet (which are discussed here).

I still stand by the position that it is not the job of government to force ISPs to play parent. Parents should be responsible for monitoring what their kids do online. If they want a filter, fine, download the free one from the government you can install on your PC, but don’t force me to deal with slower internet and subsidise ISP filtering for you.

For more on Comrade Kevin’s rise to power, watch this video: