Tuesday, December 14, 2010

UPS ruined my Christmas!

This is some epic courier FAIL right here folks. I know it's popular to overstate failage these days, but if anything ever deserved the term EPIC FAIL, it's this tale right here.

Settle in folks! This will take a little while. As you read, bear one thing in mind: this all happened or came to light the same day -- today.

For Christmas I ordered my wife a fondue pot from Amazon. (It's okay, she knows what it is now. Read on.) It was supposed to have arrived by the 7th, but they shipped it USPS so I couldn't track it. It never came and never came. Finally today I went back to Amazon to file a request for help, when I noticed that I had been mistaken and it actually shipped via UPS. So I tracked it!

Delivery location: "Rear door".

We do not have a rear door.

We have a sliding glass door that opens onto a small patio with a 4.5-foot high, 8-inch thick concrete privacy wall. Surrounded by some bushes. And about 3' of landscaped ground cover.

The UPS delivery dude:

  1. Walked through the landscaping.
  2. Stepped over or through the bushes.
  3. Climbed over the wall.
  5. ...out from underneath the roof overhang.

So it's been outside in the Bay Area rainy season getting rained on for a week. I brought it inside, and had to open it to see how fuckered it is (which is why Aimee knows what it is.) Jury's still out, but the actual product might have survived. Let's just say that the box is over. GOOD JOB, UPS!

You probably think that's it, don't you?  Oh no. Oh ho no. Ha ha ha ha, no. I'm just getting started. Read on, my friends.

For Christmas, I also ordered my dad $MULTIPART_GIFT from $LOCAL_VENDOR. Now, $LOCAL_VENDOR is a small chain and has several warehouses. So my $MULTIPART_GIFT order was split into 3 separate UPS shipments.

Package #1 arrived on time, on the 7th. Huzzah!

Package #2 was supposed to arrive on the 8th. Since my mom told me it still hadn't arrived, I tracked it. It reached Latham, NY (the nearest distribution center) on the 9th. The tracking history says:


Please note that this is the exact same address -- verbatim -- as the package that had been delivered 2 days earlier. What then? Well, it was delivered, today, the 14th, as follows:
Delivered On: 12/14/2010 11:05 A.M.
Signed By:SEAN
Shipped/Billed On:12/07/2010
Now I sure hope that means that they delivered it to my apartment complex office, but at this point all bets are off. There was a note on our door so hopefully that is what happened, but so much for 2-day UPS shipping. If I had wanted it myself I would have driven to the store to pick it up. GOOD JOB, UPS!!

But what's that you say? There were three packages? Why, yes, as a matter of fact there were.

BEHOLD the fate of Package #3:

  • From Gloversville, NY (the other nearby UPS distribution center):
    (Note: false. My mother was home all day.)
  • And then the next day:
    ...and then finally: DELIVERED
DELIVERED?? Huzzah, that's gre-- hold up.
Delivered On: 12/14/2010 3:03 P.M.
Delivered To: GALWAY, NY, US
Shipped/Billed On:12/09/2010
I have no idea who $NOT_MY_PARENTS is. Galway is -- according to Google Maps -- 10.2 miles from my parents' house.  GOOD JOB, UPS!!

So UPS -- that's the United Parcel Service -- left my wife's Christmas gift in the rain after delivering it to a wildly inappropriate location without protection from the elements. And then UPS -- that's the ups of www.ups.com -- shuffled one of my dad's $MULTIPART_GIFT across the continent twice and delivered it to me instead of him. And then finally, UPS -- that is, the well-known and now apparently corner-cutting courier who seems to have lowered the hiring bar on both coasts -- delivered the third part of my dad's $MULTIPART_GIFT to $NOT_MY_PARENTS in a town 10 miles away.


Monday, December 6, 2010

Google Books

Google Books launched today. I'm a Googler so I've been using it for a while, and so far I like it quite a bit. But okay, whatever, you expect that from me. So here, at least, is an explanation of why I like it.

Rewind a few weeks. I'd been using Google Books for several months by now. My mentality had been: well it's neat, but you know... it's just a Kindle clone, right? And the book scanning project is interesting and cool, but whatever -- just a novelty really, or maybe of interest to academics and librarians. That all seemed sort of outside my world, so I wasn't paying much attention.

But then I searched for The Call of the Wild, figuring I'd reread a favorite book from my childhood. That search turned up something else: In the Maine Woods, a sort of half-travelogue half-advertisement published by the Bangor & Aroostook Railroad periodically. This version was from 1906.

My family is from Maine. And not just like Bar Harbor here, I'm talking about Maine as in, Millinocket, Milo, Island Falls. Aroostook potato people.  My grandfather spent more or less his entire career at the B&A; he worked his way up to yardmaster and trainmaster in Millinocket. Millinocket was barely incorporated in 1906. The book talks about nearby landmarks but doesn't mention Millinocket because it didn't yet amount to much.

This book is hilarious, awesome, and fascinating. Not to mention all the incredible photos, like 2 guys with no less than 14 trout, wearing the dour portrait faces of the day. I can't adequately describe what it is like to read through this book about my family's stomping grounds apparently 104 years after it was published. I can't even really decide how I'm feeling about it.

This book was apparently scanned from Harvard's library as part of the book scanning project. Amazon has a print version of it, but it's a later edition, and costs $23. And anyway I had to look it up -- I would never have just found it, let alone read it for free. Finding this book at random like this was so unexpected but strangely meaningful, if only to me.

So, I get it now. This is what the Books scanning project is all about.

Sunday, November 7, 2010

Starting a New Thing

I just bought one of these:

As my wife said while we were walking out of the Fry's, "It blows my mind that you can buy a whole computer in a little box like that." I don't think I can or want to add anything to that.

I got Ubuntu Lucid Server up and running on without really any trouble at all, even with the configuration I chose, of booting the installer from a thumb drive and installing to an SD card. (I'm trying to keep it as low power as possible; we'll see if the SD card pans out.)

My plan is to write some server code for this thing (in Go!) and hook up some peripherals like a web cam, door opening monitor circuits, etc. My aim is somewhere between home automation and a homebrew security system. Most likely I will just end up spying on the cats. Heh heh...

So far it's a pretty impressive little widget. Hopefully things will pan out and I'll get something spiffy out of it all.

Wednesday, November 3, 2010

BEHOLD the Future All Over Again

This article is pretty much spot on: http://open.neurostechnology.com/content/future-hardware

I was especially interested that the author recognizes that the events that are now getting under way will mirror the PC model in outcome, but not in the gory details. This is something that's been on my mind for some time, too.

The PC world took off because there were various curated standards, like ISA/PCI/PCI Express, ATA/IDE/SATA, CGA/EGA/VGA, and so on. But the consumer electronics industry of today does not have any analog of that. Processors are wired directly to peripheral chips -- at least, those peripherals not directly integrated into the SoC -- and just about the only "off the shelf" widget you can buy that just works in multiple devices is a flash storage card.

This has been (in my view) the primary drag on rapid innovation in the consumer electronics industry over the past 10 years or so:  no common "interface" where hardware and software could meet and act as the stable foundation to build interesting products on.

The fundamental insight of the article (to my mind at least) is that in the PC world, this "interface" evolved just on the hardware side of the border. But in the mobile/electronics world, various economic forces prevented that (which is a whole other story.) So what Android is really doing is establishing an equivalent "interface", except on the software side of the border.

To put it another way, modern OSes for PCs include a "hardware abstraction layer" to gloss over the minor differences between mostly-compatible hardware. But Android is a hardware abstraction layer that paves over the huge differences between consumer electronics devices. (This is why it's impossible to create a generic version of Android that is installable on any phone. You can't install a consumer electronics OS, you have to port it.)

From the point of vantage point of developers and device builders sitting on top of this, the view is very much the same as the PC world: they see a common set of APIs and a well-defined execution environment that they can build on. Since the view looks so similar, the outcome should likewise be similar.

Of course they mostly don't realize just how much the structure below what they can see differs from the PC world; but the beauty of Android is, they don't need to.

This is indeed an exciting time.

Sunday, August 1, 2010

Den Suck

We bought a new bedroom set from IKEA to replace an old multiple-hand-me-down set. This turned out to be a bit of an odyssey.

The assembly of the bed is good in theory: it has a wooden headboard, footboard, and sides that form a box to surround the actual bed, which is supported by some metal infrastructure. There are 2 angle brackets on the length of the side boards and a fairly robust metal beam down the middle that hold the actual weight of the bed.

Unfortunately the angle brackets are each too narrow by probably 5mm, which allows the box spring to drift and basically fall off one side or the other. This drops the box spring and mattress (and occupants, if any) onto some cross members which are designed to keep the box rigid and not to support weight; damage occurs.

The root cause seems to be that this is a European style bed, and Over There they don't use box springs; mattresses sit directly on planks or slats. So they apparently don't quite get box springs, and things like this happen.

Anyway I tried to work around it but ultimately the angle brackets are just too narrow. I didn't feel like going out and buying a bunch of planks to fix IKEA's bad construction, so instead I figured I'd try just taking the basic metal bed frame we used previously. Since it's nothing more than a metal rectangle that the bed sits on and is exactly as wide as the box spring, it should drop right into the new IKEA bed box.

Well, almost. It didn't quite fit, because the mounting brackets for the headboard on the old frame add just a tiny little bit of width, but it's enough to keep it from fitting.

But I pretend to be a Maker, and I smelled a project! So I figured I'd just remove the headboard brackets and then it'd fit. Unfortunately those brackets are riveted to the frame sides.

I tried drilling out the rivets, but my cordless drill didn't have the power and I didn't have a steel-drilling bit anyway. Then I whipped out the Dremel and put on a cutting tool, and cut off the rivet heads on one side, but the rivet material was pretty well alloyed with the frame metal, so I couldn't push the rivets out once I'd cut the head off. I tried just cutting the bracket off, but a Dremel as old as I am vs. steel is slow going, to say the least.

At this point I'd spent probably an hour and a half on all this. It had come down to it. I was sitting on the floor in the middle of the bed box staring at this frame, out of options. I was going to have to admit defeat.

Then I looked at the frame, and suddenly remembered that it was actually in 2 parts. The left and right angle brackets have arms that join together with a simple keyhole-and-pin slide lock. So I just pushed them in slightly to make the frame narrower, dropped it into the bed box, and then dropped in the box spring. This took a grand total of 90 seconds.

So, lesson learned. Again.

Always look for the simple solution before jumping into the hard one.

A Shocking Exposé!

Oh jeez.

Consider this Apple product, selling for $29: http://www.apple.com/battery-charger/

Basically this is a 2x NiMH AA battery charger, with 6 AA batteries included. The batteries by their description sound a lot like (and may well be) rebranded low-self-discharge Sanyo Eneloop batteries. By "a lot like", I mean that the recharge cycle specs is very similar to Eneloop, and while Apple provides no specifics (such as capacity, discharge characteristics, behavior under load), the description sounds a lot like the "pre-charged" marketing message used by Eneloops.

Why, they're practically inviting comparison! Let's investigate.

Sanyo Eneloop 2x charger with 2 Eneloops, $13.18:

4x AA Eneloops, $9.15:

Total: $22.33.
Apple markup: $6.67, a ~23% premium.

For $6.67, you can upgrade to a 4x charger and still come in under Apple by a couple bucks.

Now, Apple's claiming a low standby load, which at least is a potential difference from the competitor. The problem is, their competitors don't publish standby loads (at least that I can find) for their chargers, and they themselves never define the "average" charger they're comparing themselves to. Did they test a good random sampling of chargers? Or did they test a hand-picked set of dubious chargers they picked up at a market in Taiwan or something?

So in other words, it's cute marketing, but it's a 23% markup for a product which has no actual, verifiable advantage over the competitor.  Although it does look cute I suppose, if you're into the whole Apple wall-wart aesthetic.

When Apple does this kind of thing with a Mac, you can at least point to the user experience and make non-specific non-falsifiable claims of enhanced productivity or lower frustration or better build quality or whatever. And that's fine; I don't have a problem if people find that those things make the difference for them in their daily usage. Human factors are very hard to quantify.

But when they do it with a battery form factor that was defined as a commodity in 1947, has been exhaustively characterized, is utterly well-understood, and can be trivially compared apples-to-apples (pun intended) with competitors, I find it kind of laughable and silly.

And honestly, if they apply the "Apple premium" to something as obnoxiously over the top as this, it makes me question whether there really is an Apple premium on anything they sell, or if Apple is just a giant snow-job.

Friday, June 18, 2010

Now I can manage my life from the command line

GoogleCL (http://code.google.com/p/googlecl) is entirely too cool.

sudo apt-get install dcraw graphicsmagick python-gdata
sudo dpkg -i http://googlecl.googlecode.com/files/googlecl_0.9.5-1_all.deb
for nef in *.NEF; do
jpg=`echo $nef | sed 's/\.NEF$/\.jpg/'`;
gm convert $nef $jpg;
google picasa post --title="Vegas ATV Trip" $jpg;
rm $jpg;

.NEF is Nikon's RAW file format. dcraw is a utility that can convert NEF to standard formats, and graphicsmagick is a utility that can do various manipulations of images, including transcoding.

The script above converts all the NEF files in the current directory to temporary JPEG files, and then uploads them to my Picasa Web Albums account, in the "Vegas ATV Trip" album. That is, it's an extremely simple but effective Picasa uploader.

I think I will be having some fun with this thing. I'm even posting this via GoogleCL.

Sunday, June 6, 2010

On 'On Android Compatibility'

[I'm going to talk about Android a bit in this post. You should know that I'm Android's Open-Source and Compatibility Program Manager, so this is near and dear to my heart. But remember that this is still my personal blog, and thus my personal opinion. I'm not necessarily speaking for Google here. This post describes some of my personal perspectives on the mobile industry.]

On Monday we posted a blog I wrote about Android's approach to compatibility over on the official Android blog.

The results were interesting. Tom Gibara made the great point that in exchange for handling device diversity you get a much larger audience in return. Engadget has presented both sides of this topic, including the good and the bad. Although my personal favorite example of the negative point of view is Harry McCracken's somewhat histrionic (IMO) post.

A Name for the Ailment
The gist of the main criticism is that there are multiple versions of Android available at the same time. What I think has been missed is that I never said this was a non-issue; I just said that calling it "fragmentation" is too vague to be helpful (and then moved on, since Monday's official blog post wasn't about this.) Regardless, I actually liked the word Mr. McCracken used to describe this -- "versionitis" -- because it's a pretty evocative but descriptive term.

So, what about versionitis? It's certainly true that there are several Android versions extant at the moment, but I believe this isn't a big deal. For one thing, the risk of user confusion is extremely overblown; you've really got to be cynical if you think the buying public is somehow too stupid to comprehend the concept of last year's model, as applied to mobiles.

From a developer's point of view it's admittedly a bit trickier, though. You can do without features from later versions and rely on Android's forward compatibility to target the largest possible audience; or you can make a flashy cutting edge app for the latest version and wait for audience size to catch up. Although sometimes you don't even have to choose, because with a little extra work you can make an app that will run on old versions AND take advantage of new features if they are available. Nobody likes extra work, but at least as Tom Gibara pointed out, you're getting something in return.

But all that said, it comes down to choice: consumers and developers have to make some decisions, and the argument is that this all treads dangerously close to a tyranny of choice. Personally, I don't think it's nearly that bad, but of course that's just my opinion, and I respect that some people don't agree. And anyway, in an ideal world we'd like to see all devices running the latest version of Android -- it's not as if we are somehow opposed to the idea.

Wherein We Invoke a Business Cliche
But the main reason why this issue just isn't that big a deal is because we're right in the middle of a sea change that will wash this all away.

Rewind to about 5 years ago. This is recent enough you had feature phones and smartphones mostly as we do today, but long enough ago that it was before the mobile revolution really got underway. Back then as today, it was practically unheard of for a feature phone to ever get a software update. As for smartphones, it wasn't unheard of to get an update, but it was pretty uncommon (at least in the consumer space.)

Why? The reason was that the smartphone platform vendors controlled the software. It was exceedingly difficult for OEMs to differentiate on software because they had little control over the software. It was difficult for them to differentiate on features because they could only ship features supported by the OS they were using. But it was still a fiercely competitive market and they still innovated as hard as they could.

So they innovated on the only dimension they had control over: hardware and industrial design. The inevitable result was that as soon as an OEM released one phone, it was immediately on to the next. The follow-on to a successful device was another device. "Version 2.0" wasn't an upgrade, it was a replacement. This is how we ended up with 16 different RAZR models (if I counted correctly), each with different hardware. It was easier for OEMs to rev hardware than software, because they had little control over the software.

Think about that. Easier to rev hardware than software! This flies in the face of everything we've learned from the personal computer. It's almost like the cell phone world ran counter to the laws of nature, so as soon as someone offered the OEMs the chance to control their own software destiny they jumped at the chance. And that's exactly what Android as an open platform is doing.

And the Point Is...?
So what does this have to do with versionitis? Simply this: it takes time to rebuild a manufacturing process.

When all you do is rev hardware, you get pretty good at it. You optimize around it. You make being good at it a competitive strength. The OEMs are transitioning to a more PC-like world where they control their own software destiny, but to do this they need to update their procedures and tools.

This takes time, and today we're right in the middle of it. Device manufacturers want your brand loyalty. They also know they aren't going to get it unless their users have a good experience, and they know that users are expecting that the new $500 tiny computer they just bought comes with a year or two of support. OEMs know this, they are hardly dumb.

The updates are coming. You can see it already, with devices like the Samsung Moment and HTC Hero (for Sprint) getting 2.1, and commitments to upgrade other models. Section 12 in the Compatibility Definition Document even requires that compatible Android devices be upgradeable, which at least sets the stage. Device manufacturers are getting better and better at doing these updates -- and I mean, it's not like they are going to get slower at it.

That's why it just doesn't make sense to me to worry much about this versionitis thing. It's certainly true that today some people aren't getting the updates they want, but that will change over time. And while it may be poor consolation when your buddy's got a later Android version than you, your current phone isn't just going to stop working. And of course the reality is that to the vast majority of the millions of regular-Joe Android users, this isn't even an issue.

So for some people it's painful today, but really they're just impatient for the future to arrive. We have to take the long view, though -- and as far as I can see, the view is pretty good.

Thursday, April 29, 2010

On Being Open

"Diversity" is more than the absence of racism. Diversity does not mean grudgingly tolerating the equality of those different from you, it means recognizing the real, additive value that disparate perspectives and habits bring to your life, and others' lives. It doesn't mean tolerating differences, it means valuing differences. Tolerance is just the first step on the road to diversity.

Freedom of speech does not mean waiting for the other guy to say something before you shout him down. Free speech means, as the saying goes, disagreeing with what someone says but defending to the death his right to say it. You only really value freedom of speech when you quietly thank your higher power of choice that the other guy has the right to make your blood boil.

So it is with open platforms.

Openness is more than the absence of closed. Openness does not come automatically   just because you deploy an industry standard. Openness means you recognize the real, additive value that disparate products and businesses bring to your platform. It doesn't mean tolerating competition, it means valuing competition. You only really value openness when you quietly thank your higher power of choice that your competitor just took advantage of the opportunity to destroy your business on your own platform. Implementing standards is just the first step on the road to openness.

Steve Jobs, you fail at openness.

Sunday, April 18, 2010


I got Issue 22 of Make on Friday, wherein is contained a project for a cat toy that "senses and Twitters felinogenic perturbations". (Heh.) I recently discovered -- while making a cigar box guitar from Issue 21, naturally -- that one of our cats, Sammy, luuuuuurves to play with guitar strings, since they dance all around and hold up well to clawing and nomming, being made of steel and all. As the cat toy project mostly consists of a paper bird on the end of a guitar string, I figured this one was a no-brainer.

It's a pretty straightforward project, although I did encounter one snafu. The core of the project is just a steel molly bolt installed into an enclosure, with a steel guitar string projecting up through the center. When the cat whacks the toy, the guitar string bends and makes contact with the molly bolt, which closes an electrical connection. It turns out to be a bit tricky to get the guitar string, molly bolt, and heat-shrink tubing lined up correctly so that the guitar string is not in contact with the bolt while at rest yet still makes contact easily when whacked. I ended up screwing up the first attempt and had to try again. But, I got it to mostly work on my second try.

Now, I'm no dummy. I know better than to spend hours soldering up a whole project, installing it into an enclosure, and then writing Arduino code for a cat toy, before finding out if the cat actually likes it. (Measure twice, cut once, as it were.)

So instead I took a decapitated USB cable that I have lying around that I use to prototype with a breadboard, and hooked up USB power/ground to the breadboard, and an LED in series with the toy's signal leads. I then plugged it into my laptop, which made for a quick and dirty 5V power source for testing.

The LED lit when I whacked it, so everything seemed installed correctly. I whacked it around a few more times watching the LED light up, until I realized I was playing with a cat toy. *ahem*  I then went to find Sammy.

By now the astute reader knows where this is going, of course. Sammy would have nothing to do with this thing. He could tell I wanted him to play with it (since I kept picking him and putting him back in front of it whenever he started to walk away), so he batted at it half-heartedly a couple times, then looked at me and meowed for dinner.

To confirm, I grabbed another guitar string and held it out for him, and boy howdy did he have a good time with that one. The toy, unfortunately, just doesn't have enough whiplash action to interest him. Since it stands vertically, the string's length has to be short so that it doesn't bend over under its own weight, but Sammy likes his strings to be full-length, I guess.

So, today I made a cat toy that was more entertaining for me than for the cat. My name is Dan, and I'm a Maker.  (Hello, Dan!)

All is not lost though, I have a couple ideas for how I can make a variant of this thing that Sammy will like. I think I will use a dowel for a hard vertical shaft with the guitar string placed horizontally on the top. Pulling on the string will make the dowel lever back and forth, which I can use to trigger a switch. This might actually even be easier and more durable than the vertical version, and has the advantage of not electrically connecting the cat to 5V DC.

When (if?) I get this working, I doubt I'll use it to Tweet, though. Or at least, if I do I intend to at least use it to trigger a webcam that grabs a snapshot or video of the cat at play. In this era of "video or it didn't happen", can I do any less?

Sunday, April 11, 2010

Cable winders

My wife and I have been collecting corks for a while to fill in some of those cork-board things you can buy, where you glue corks into a wooden frame to make a trivet or a corkboard for messages. This weekend we finally had enough corks so I assembled those, but I also had a few corks left over.

Since corks don't compost well, I wanted to put some of them to another use. At the same time I've been having a problem with damaging and losing USB cables for my various Android phones, since I've been carrying them loose in my Timbuk2 bag. I suppose could have done a better job of winding them up carefully, but that's annoying to do, given how often I use them.

So instead I made some cable winders out of some of the corks. It's extremely straightforward, and I won't belabor the point, since I made a YouTube video: http://www.youtube.com/watch?v=8Yf6AaXevyY

I can't decide if this is a clever idea, or something lame that belongs on regretsy.com.

Sunday, March 7, 2010

Watching a film on your F*CKING telephone

Before we begin, please watch this important safety video. If Mr. Lynch's cautionary tale has not dissuaded you, read on, with the understanding that you should not try this at home, that I am not responsible if your house burns down or you get arrested or sued, and that I offer no warranty, including for merchantability or fitness for a particular purpose.

I have a 45+ minute commute, twice a day. By way of sanity management, I use my phone (typically a Droid or Nexus One these days) to listen to music... and watch some videos. (No, I don't drive, I take transit, but your interest in safety is reassuring.) Specifically, I've been ripping DVDs to watch on my phone.[1]  It took me quite some doing to find a way to rip DVDs so that they'd play on my phone, and I figured I would blog about it for two reasons: so that others may perhaps benefit, but also so that when my computer dies the next time, I'll have a copy in this Cloud all the kids these days are talking about.

So, here is how to use Ubuntu to rip DVDs to MP4 files that will work on an Android device. This approach uses command line programs: no GUI required. Mostly this is because I prefer it this way so that I can rip DVDs from my headless server, but it's also partly because I couldn't find a GUI program that was capable of ripping a file that Android can understand. (I suspect it's because they all use mencoder behind the scenes -- read on.)

The output we're looking for is an MPEG4 SP file, with video and audio encoded in a supported format. Any supported formats will work, but practically speaking we want H264 for video, and AAC for audio. You could use MP3 for audio, but... why would you do that to yourself? Note that many (most? all?) DVDs use AC3 for audio, but we are transcoding that to AAC.

The basic idea here is that you use the mencoder program to do the ripping. However, mencoder has a bug that generates broken MP4 container files, so Android devices don't recognize them as movies and can't play them. So, after you rip a DVD to AVI, you need to repackage it as a legitimate MP4 file.

To begin, you need to install a few packages:
  • libdvdcss2 - for the ability to decrypt DVDs [2]
  • lsdvd - for looking at the contents of a DVD
  • mencoder - to rip DVDs to AVI files (don't bother trying to rip to MP4, it won't work)
  • faac - used by mencoder to rip audio to AAC
  • mplayer - for extracting the audio track from the AVI file 
  • mpeg4ip-server - for its mp4creator tool which can re-merge audio and video into a correct MP4
These can be installed via apt-get or aptitude or whatever floats your boat.

In slightly more detail, here is the actual procedure:
  1. Use 'mencoder dvd://' to rip the DVD to an AVI file containing H264 video and AAC audio
  2. Use 'mplayer -dumpaudio' to extract the AAC data from the AVI
  3. Use 'mencoder -of rawvideo' to extract the H264 video from the AVI
  4. Use 'mp4creator -create' to create an MP4 file from the AAC audio track
  5. Use 'mp4creator -create' again to merge the video in to the new MP4
It's not quite as simple as the above, but since I don't really want to bother getting into the gory command-line details, I have instead included a simple Python script that automates these. You pass it a single integer as a command line argument, which indicates which track on the DVD to rip.

In the script, I set the H264 bitrate to 768. This is too high for first-generation devices like the G1, Magic, and Hero, which don't have the hardware to handle 768kbit. However, Droid and Nexus one can actually handle higher, but I set it at 768kbit as a compromise between quality and file size. A half-hour TV show episode is about 275MB with this configuration. (Wow, my first hard drive was 20MB.)

[1] - I do not condone ripping DVDs which you do not own. Seriously. I have ripped 800+ CDs to our disk server, but in each case we own the physical disk, or purchased it legitimately if it is digital. The same is true for DVDs I've ripped. I do this as a form of time-shifting (which I believe is fair use, or ought to be), and because I've bought the media in each case I have no moral issues doing this. Don't be evil!

[2] - See [1].

Maybe one day mencoder will get fixed and steps 2 - 5 will become obsolete. Until that day, here is a Python script:

import os, sys
tracknum = int(sys.argv[1])
args = {'tracknum': tracknum}

print 'mencoder...'
s = 'mencoder dvd://%(tracknum)s -dvd-device /dev/dvd -aid 128 -o ripping-%(tracknum)s.avi -ovc x264 -x264encopts bitrate=1024:nocabac:direct_pred=auto:me=umh:frameref=2:level_idc=21:partitions=all:subq=6:threads=auto:trellis=1:vbv_maxrate=768:vbv_bufsize=244:bframes=0 -oac faac -faacopts br=192:mpeg=4:object=2 -channels 2 -srate 48000 -vf harddup' % args
f = os.popen(s)
x = f.read()
while x:
  x = f.read()

print 'dumping audio...'
s = 'mplayer "ripping-%(tracknum)s.avi" -dumpaudio -dumpfile "ripping-%(tracknum)s.aac"' % args
f = os.popen(s)
x = f.read()
while x:
  x = f.read()

print 'dumping video...'
s = 'mencoder ripping-%(tracknum)s.avi -of rawvideo -nosound -ovc copy -o ripping-%(tracknum)s.h264' % args
f = os.popen(s)
x = f.read()
while x:
  x = f.read()

print 'creating mp4 from aac...'
s = 'mp4creator -create ripping-%(tracknum)s.aac ripping-%(tracknum)s.mp4' % args
f = os.popen(s)
x = f.read()
while x:
  x = f.read()

print 'merging h264 to mp4...'
s = 'mp4creator -create ripping-%(tracknum)s.h264 -rate 29.97 ripping-%(tracknum)s.mp4' % args
f = os.popen(s)
x = f.read()
while x:
  x = f.read()

print 'cleaning up...'
os.remove('ripping-%(tracknum)s.avi' % args)
os.remove('ripping-%(tracknum)s.h264' % args)
os.remove('ripping-%(tracknum)s.aac' % args)

Sunday, February 28, 2010

This makes me sad

My wife and I recently bought a Honda Civic Hybrid. Among other features, it has the ability to play music files off a USB mass storage device. Yay! Easy way to play the music on my Android phone through the car speakers, right?

Wrong. You see, about a year ago I upgraded the disk server to 1.5TB and re-ripped our 800+ CD music collection to FLAC. At the same time I also transcoded to AAC, because AAC turned out slightly better to my ears than MP3 in some comparative testing I did.

Now, the Civic's owner manual claims that the stereo can play AAC files, and even lists the bitrates, sampling rates, etc. that are supported. However when I actually tried to play these files, it gives the error "UNPLAYABLE FORMAT".

If you look more closely, you notice the disclaimer that says only iTunes AAC files are supported. As best I can tell, what's going on specifically is that iTunes doesn't support encoding AAC files with VBR, despite the fact that VBR is required by the AAC spec (according to WikiPedia, at least.)

Rather than support VBR, Honda's stereo vendor chose to simply implement AAC without VBR. I have to wonder why. There are plenty of spec-compliant AAC implementations out there, including some that are open source. Did Apple actually give them this code? The thing supports deeper integration with iPods so it's possible they got some code for that from Apple that included playback of Apple-forked AAC files, and are just reusing that same AAC-playback for USB mass storage playback too. This is just speculation of course -- I have no way of knowing, except that the manual says straight up that only Apple's non-standard fork is supported.

Regardless of what the technical reality is, the retail leader in digital audio forked a standard, and then used their market power (whether intentionally, or accidentally) to cause another vendor to propagate their fork. Shame on Apple for not supporting AAC properly, and shame on Honda for propagating the breakage.

Now please excuse me while I go transcode my FLACs to MP3, just so I can try to play them in my car. God help a regular person who tried to do this, without the know-how to write a Python script to do it...

Update: It was pointed out to me that iTunes does support VBR, and has an option for it in the UI. However, according to WikiPedia, it's not true VBR. Here's the quote from WikiPedia, to make things clear as mud:
iTunes supports a "Variable bit rate" (VBR) encoding option which encodes AAC tracks in an "Average bit rate" (ABR) scheme. As of September 2009, Apple has added support for HE-AAC (which is fully part of the MP4 standard) but iTunes still lacks support for true VBR encoding. The underlying QuickTime API does offer a true VBR encoding profile however.
So it could be that, or it could be one of the other changes Apple made to the AAC file format that I've read about. But as I said, I'm just speculating; all I know for sure is that my car only supports whatever hacked format Apple's introduced, and it makes me sad.

Saturday, February 27, 2010

Licensing is hard; let's go shopping!

While I was working on my recent Arduino project, I happened to end up looking at some of the code that's available. This was.... interesting? Scary? Depressing? I'm not sure exactly how to describe it, but I thought it was worth talking about.

I'm talking about licensing. Here's a header file from an optional, but common, Arduino add-on library. The copyright holder's (i.e. author's) name is elided, because my goal is not to call people out:

  Foo.h - Arduino library for XXXXXXXXXXXXXXXX
  Copyright (c) John Doe.  All right reserved.

  This library is distributed in the hope that it will be useful,
  but WITHOUT ANY WARRANTY; without even the implied warranty of

With apologies to Inigo Montoya, "You keep using this license. I do not think it does what you think it does."

Obviously this developer's heart was in the right place, and he's properly protected himself by disclaiming warranty; so far so good. The problem, though, is three little words: "all right [sic] reserved".

By the brevity, I presume the author's goal was either to abdicate copyright to the public domain, or license it under a maximally permissive BSD-style license. However, that's not what's happened. This copyright actually isn't even a license at all, it's just a copyright header. This isn't a contract because the reader/user isn't agreeing to anything and there are no conditions. The author has reserved all rights to himself, and granted me none.

The bit about hoping it will be useful is nice I suppose, but since I have not actually been permitted to use it for anything, it's a moot point. Even if there were a grant of license in there somewhere, that clause is merely stating an unverifiable, irrelevant fact, and it is a total no-op as far as copyright and licensing is concerned.

So, okay, having made my point -- does it matter? This is an interesting question. The code itself is fairly simple, even trivial. What's more, the culture of the Arduino project is that a lot of... uh, let's call them "makers" (since many of them aren't professional software engineers) make various kinds of contributions. So the author of this particular piece of code manifestly knows how to write basic C++ but just as manifestly does not understand software licensing. From this I conclude he's more or less an amateur software developer, and that he probably doesn't attach a particularly high intrinsic value to the code he's contributed. And of course, he did contribute it knowing full well it would be publicly visible, after all. All of which is to say, it's almost certain he intended this to be open-source, in some form.

So there's very little practical legal risk of using this code. The likelihood that I would get sued for using this software in my personal little Arduino project is vanishingly small. If that were my only concern, then I wouldn't worry about it and carry on.

Unfortunately, that's definitely not the only concern.

Suppose I contacted the author of this software, with a polite message: "Hey, I don't think this header in this file is doing what you think it is; you might want to consider changing it." Let's assume the author is willing to fix it. In that case... fix it how?

If he fixes it by making it public domain or using a BSD-style license, then this story ends, and we all go about our business. However, most of the rest of the core Arduino software appears to be LGPLv2, so it would be quite reasonable for the author to also choose LGPL for his contributed code. If he were to do that, he actually would screw me.

See, my preferred license is Apache 2.0. I prefer this license because it's basically BSD-style with some anti-patent-burn language. Or to put it another way, it's essentially the maximally-permissive BSD license updated to address the various IP realities of our modern times. Unfortunately the patent clause also happens to make it incompatible with GPLv2, and (I'm pretty sure), LGPLv2. This means that if the author of this code chooses to license it under LGPLv2 -- a very fair and reasonable decision -- it means I either have to alter my own license choice, or else find or write some other software to use.  (Unless of course he chooses LGPLv3, which introduces a whole new set of complexities.)

My point here isn't to start a flame war about open-source licenses, by the way. I'm not saying that I'm right and the other guy's wrong, just that our license preferences conflict (at least in the hypothetical scenario I just described). Backing away from hypotheticals, what I can say concretely is that there's too much ambiguity here for me to be comfortable.

To put it another way, even though I might be comfortable taking the legal risk of getting sued because I feel it's obvious that the author meant his code to be open source, I nonetheless can't be comfortable using the code anyway because I don't know how the author meant it to be open-source. He might very well have meant it to be open-source in a way incompatible with my actual usage, and so in the absence of surety it's ethically wrong (as well as technically legally wrong) for me to use it.  Plus, Google Code Project Hosting has a policy that you may not use it to host non-open-source code, and so I couldn't include that software in my chosen SCM site, anyway. (N.B. - I actually did have it included for a day or two, until I noticed this problem and removed it.)

In this case, I did not attempt to locate and contact the author to clarify because it was easier for me to just write the code out of my app. In other cases it might not be so easy. And by the way, this isn't the only Arduino code that does this: there's a sample Makefile intended to be copied and customized for building projects from the command line. That actually is included in the core Arduino distribution, and it has no copyright heading at all, let alone a license.

Perhaps it sucks that the world we live and code in makes this stuff important. But, like it or not, it is important. I'm not trying to pick on the Arduino project; it's a fabulous project that has done awesome work. But this kind of thing is one reason why my day job is sometimes so hard: license compliance is often not easy, and with a project of the scope of Android, it gets to be damned hard.

So I guess I wrote all this to make a plea: please please developers, if you release source code, think carefully about what you are releasing, and make sure you choose your license carefully so that it reflects not only your principles, but also how you hope and expect it to be actually used by others. But most of all, PLEASE take the time to get the details right.

Garduino Redux

A while back I made a "Garduino", based on an article Luke Iseman wrote for Make Magazine #18. It's an Arduino-controlled plant growing rig.

Luke's original version monitors moisture and automatically waters the plants when they need it, and it monitors light level via a photoresistor and turns on a supplementary grow light as necessary. My version includes the light monitoring, but I omitted the moisture sensors and water pump in favor of an Earthbox self-watering planter. I figured that since Luke's version involved dumping water into a large bucket, manual refill is manual refill and I might as well just refill an Earthbox. I'm stupid, so I need to keep it simple, you see.

Anyway that's all old news. A while back, I put up some videos on YouTube, documenting the growing process:
Since recording those videos, I moved the circuitry from my breadboard to a permanent home on an Arduino protoshield, but until fairly recently that was about it. It's been running great for months now. It turns out that even with the grow light I don't get a terrible lot of light over there, so the basil's growing slowly enough that I only just now have enough of it to make some pesto. But alas, this is the best I can do unless we move somewhere with better lighting. I credit the Garduino with this being possible at all.

However there's been an annoyance: periodically the thing will just reset itself. I think this was because the host would occasionally brownout the USB voltage, making the Arduino reset. But whatever the cause, this has 2 problems: it means that I need to reset the Arduino's onboard time (since it doesn't have battery backup or even a clock), which in turn means I need to hook it up to a computer to access the serial port. For various reasons I have only one computer which can program the thing, which means I have to haul out a laptop to do this, and then physically sit next to the thing, tethered, to do the reset.

I resolved to solve this.

The lightbulb went on when Brian Jepson told me about the Sparkfun BlueSMiRF, which is a Bluetooth RFCOMM-to-serial-TTL bridge. In practice, it takes your "Serial.print" output from an Arduino program and makes it available over Bluetooth to a PC..... or a phone. Android, meanwhile, added an API for Bluetooth RFCOMM in version 2.0. My wife got me a BlueSMiRF for my birthday, and it was off to the races.

I rewrote the Arduino code into a simple finite state machine, and added the ability to accept commands over serial. It's a very simple project, so there are only 2 commands: reset, and set current time. I encountered some interesting open-source related issues in doing this, but that's another post.

I then wrote a spiffy little Android app that pairs up to the BlueSMiRF, reads the state dumped from the Arduino every 3 seconds, and then makes a pretty little Android UI. It shows me a Sun, Moon, or Clock depending on which state the Arduino is in (daytime, nighttime, or waiting for clock data), and reports the other status fields like light intensity and status.

All told, it's working pretty well! There are a couple minor bugs I need to fix, but it's doing what I need it to do, and it's pretty fun to whip out my phone to see how my grow lights are doing. A little pointless maybe, but fun.

My source is up on Google Code, if anyone cares.

Monday, February 22, 2010

Call me the GIT-arr man

I'll start the revivifying with the cigar box guitar I built.

First, know that despite appearances this is not a remarkable thing; after all: http://makezine.com/21/cbg/ And I will cut the suspense as well, by revealing the output of this procedure: http://www.youtube.com/watch?v=wkmUAUJGHsI

I generally just followed the instructions in Make Magazine, which I will neither repeat nor paraphrase. What I do want to do is just describe some of the ways in which my experience compared to the article.

I shall begin where I began: at the cigar store. I started out with a search in Google Maps (where else? heh heh) for "cigar". This revealed some Maps place spam which it was my privilege and duty to report to the Maps folks, but I shall not digress.

I ended up at Grant's Tobacconist up on Market St. I wandered in on a Saturday afternoon, whereupon I discovered two things. First, that the guy who runs the place may or may not know what these people are doing with all these empty cigar boxes, but he does know enough to make us pay for them. $10 got me 3, of which I don't mind saying that one may well be the platonic Form of a cigar box, at least from a guitar point of view. More on this later.

Second, I learned that, wow, it's smoky in there. It was almost funny: I was in a race against time, I had to check out quickly and escape before I busted out coughing and offended everyone in there. I mean, you don't walk into a cigar shop and then start coughing ostentatiously and glancing around to see if people take your hint. But I survived, and hopefully without spreading offense.

Building a CBG requires a modest variety of handtools, of which to my shame I was not already an owner. I went to Lowes, that magnificent Store of Stores, vast savannah of tooling. There, I found only FAIL. They had like 1/3 of what I needed, so I bought the oak stick I needed and left it otherwise as I found it.

Then I went to tiny-ass Cole Hardware, clown-car of retailers, wedged all Harry Potter style into a 47-inch gap between buildings on 4th St. They had everything I needed. Go fig.

Having procured my tools, the construction went pretty much as the Make guys described. I will say that I wished more than once for my dear Father's table saw. By my rough calculations, a table saw would have reduced my prep time on the neck from about 97 and a half fortnights down approximately 0.0000000 seconds, for a percentage improvement of roughly INFINITY. Alas.

Honestly though half my problem was that I didn't know how to use a coping saw and I wasn't sure of the best way to proceed, since you have to basically slice the oak stick in half the long way, twice, to carve out a couple chunks. On one end, I ended up using chisels to hack out the wood and then sanded the bejesus out of it. This worked but was time (and kilocalorie) consuming. The other end I used the coping saw on, which worked pretty well. With practice or a steadier hand, one might even call the results good.

Yeah, I screwed some things up. But some were not my fault, I swear! Behold my shame, in no particular order.

For one thing, it turns out that cigar boxes smell like cigars. And I don't like cigars. Wanting to not have to deal with a Pigpen-like cloud of cigar odor every time I picked up my guitar, I wanted to deodorize it a bit. To do this I just dumped a couple ounces of baking soda inside the box and let it sit for two or three days, then dumped it out. The odor's not gone, but now it's a gentle musk instead of a heinous stank. Problem: SOLVED!

As for the neck, well... I wasn't really thinking when I drilled the holes for the tuning pegs. Normal people arrange the pegs with 2 on one side and 1 on the other. I thought it would be hilarious (absolutely. hilarious! Seriously, I thought this would be totally funny) to put the 3rd peg on -- get this -- on the TOP of the headstock!!! So I drilled the hole up there only to realize that I drilled it too far away from the end of the headstock for the actual thumbscrew to clear the wood. I actually had to use the coping saw again to cut off a slice of the headstock at a jaunty angle to make room for the tuning peg. I wasn't laughing anymore, although in the end I think it lends the finished product a certain ghetto je ne sais quoi.

Now, you have to understand: I had never seen a guitar string before. Ever. I had no idea how the little barrel plug thing on the end was supposed to work, and I was like, "do you just tie the string onto the hinge through the screw holes? Or what?" Then I got them in the mail and I was like, "Ah. Uh. Hmm." Apparently the hinge that Mark Frauenfelder used in the Make article had tiny little holes, or else he had weird strings because there was no way those strings were staying attached to MY hinge through geometry alone. I ended up just punching 3 holes in the hinge that were smaller than the barrel plugs. This worked, but makes me wonder if I shouldn't have just used something other than a hinge, but whatever. Fun factoid though: a faux gold/brass hinge that would look decidedly low-budget on actual cabinetry looks surprisingly kickass on a cigar box guitar.

And now, I have to talk about the frets. Dear Make Magazine photographers: you guys know I love you, but YOU LIE! The article clearly depicts the fret slots being cut with the coping saw. Now, maybe it was just my particular saw, but this ended up cutting a slot too wide for the fretwire to take grip. When I used the Xacto hobbyist saw that came with my miter box, which has a significantly narrower kerf, the fretwire stuck fine. Thing is, I couldn't find replacement coping saw blades that vary in kerf, so I think this might have just been a photo staging error. Fortunately I (wisely) only cut the first slot with the too-wide slaw, and all the others turned out fine. I still need to glue that first fret in, but I think I may leave it as-is. As a reminder. Never forget! Anyway, "Measure Twice, Cut Once" paid off -- thank you, Norm Abram!

Though this reminds me of my only real gripe about the article, which is that it was insufficiently precise in some of the parts list. And yes I realize that there are variable widths of fretwire, but I also ordered the wire that the article advised, as far as I could tell. I think. Maybe? I tried anyway. Point being, the article just sort of said "hey go to elderly.com and buy some fretwire. Or maybe cbgitty.com if you like." A little more guidance on specific strings and fretwire to buy would have helped us hapless n00bz.

Another minor issue I encountered was that the screws securing the cigar box to the neck really wanted to dig right through the cigar box's wood, because it was pretty lightweight. (Okay fine, technically the platonic Form of a cigar box from a guitar point of view would be made of wood that sounds great but is hard enough to hold a screw well. So maybe it's not quite the platonic Form. But more on this later.) Fortunately I have screwed enough screws in my day to expect this, and I didn't let the screws bite straight through the cigar box entire.

Finally, Cigar Box Guitar protip: remember that shipping things (such as, hypothetically, guitar strings and tuning pegs) thousands of miles takes time. And also that if you don't order everything you need at once, you have to make another order. And then wait for the second order to arrive. Yeah. Keep that in mind...

Ultimately, battle was joined, and a hard-fought victory was won. Which is actually a lie, the victory came quite easily. My opponent was made of wood after all. Aside from the quibbles above, the whole thing went remarkably like the Make article, except for the preparation of the cigar box itself.

The box I chose was not a typical box where the lid closes flat and flush. I chose one that actually was like a chest with a shallow lid with a lip that closes onto the bottom. The result is that it has a nice hollow reverb just by itself, when you knock it with a knuckle. The edges are also nicely beveled, giving it a great appearance and feel in the hands. And finally, the branding is actually branded into the wood, as opposed to printed on paper glued to the outside of the box. All in all, it is a Cadillac among cigar boxes. It ought to have been though, since it cost me the princely sum of $4 -- American!!

Anyway this box structure complicated the process of making the neck flush with the box top. The trickiest part was that ultimately I only needed to remove about 3mm of material from the portion of the next that goes into the box, which is why I tried the chisels I mentioned earlier. But whatever, I soldiered on, and all is well.

I'm extremely satisfied with how it came out. That cigar box was all I could have hoped for, and it sounds pretty freakin' awesome. I expected kind of a course, ragged, thin sound, at least compared to a real guitar. I strummed this sucker and I was completely shocked -- it actually sounds like a guitar!! A real honest to gods guitar!

Then I tuned it. With my Google™ Nexus One™, powered by Android™, using an app named "G-strings" I obtained from Android Market™, the leading mobile applications marketplace. *cough*

The only other issue I had was that while tuning it, the high G string was twanging. I poked around at this and realized this was because the g-string wasn't pulled tight across the nut. (What?? Get your mind out of the gutter.) So I restrung that thread onto the peg's lower hole, and boy howdy, does it sound awesome now.

So that's it! It went pretty much according to plan, and I truly can't believe how well it turned out. It sounds awesome, was fun to build, and required only hand tools, except for a cordless drill.

Now I just need to learn to play guitar.


I stopped blogging more or less when I started tweeting. This, frankly, was the easy way out. Better to be pithy but without substance in 140 characters, than to be dull at novel length.

But, lately I've been doing some projects, so I'm going to use this blog as a build log for stuff I'm fooling around with.

You've been warned.