The Android Developer Phone

I started with Android programming couple of weeks ago and was itching to get hold of the actual phone and this week my wish was granted.

Being an iPhone owner, I was very excited with the possibilities and opportunities an open platform brings. While the iPhone provides a great user experience, the lack of support for third-party background services puts a limit to the kind of applications one can write. Anyway, back to Android.

The developer phone, also called ADP1, is a SIM-unlocked and hardware-unlocked. So, a SIM card from any network. I used it with Airtel SIM and it worked without any problems (well not really but more on this later). The other great advantage with ADP1 is that since the bootloader is unlocked, advanced users can build Android from the source and flash the phone with custom or bleeding edge builds. Exciting!!

After I got the Android, I booted it up only to find that there was no way to get past the initial setup wizard. The setup wizard ties your Google account to the phone. This essentially allows for download (and sending) of emails on Gmail and syncing of the calendar app on the phone with Google Calendar and the contacts app on the phone with Google Contacts with the registered account. The only problem is that it requires a data connection (aka. EDGE) to be enabled on the SIM. There is no 'skip' button (atleast not with the stock ADP1 firmware) to skip this step, enable WiFi and register.

But I had the SDK installed on my system. So, I used Android Debug Bridge or 'adb' to skip this screen. I connected the phone to my computer and opened a terminal window and typed:

adb shell

This connected my computer to the shell on the phone. The following commands disabled the app wizard:

$ su
# echo "app.setupwizard.disable=1" > /data/local.prop
# reboot

The 'adb shell' opens a shell on the remote device from the computer. The 'su' was required since the default user on the device did not have the necessary permissions to create the file in '/data'.

This connected my computer to the shell on the phone. The following commands disabled the app wizard:

That did the trick. The home screen appeared after reboot. I played around with the phone for some time before I started itching for 'cupcake'.

'cupcake' had a couple of features I wanted to try out including a soft keyboard and the A2DP bluetooth profile.

So, I downloaded and built the Android source code. What was amazing was that the entire source code built without any issues the first time. Well, almost. The default Makefile target was not set for the device. I read through the documentation and found that there were some additional steps for building Android for the device.

I followed the instructions as is and after about 45 minutes or so, I had the 'cupcake' version built and ready for flashing.

Here are the instructions for flashing the device. Please try it at your own risk :-)

1. Go to the Android source code root folder on your machine.
   
2. Ensure that 'out/target/product/dream' exists. We are interested in 'system.img', 'boot.img' and 'userdata.img'. Make sure all the three files exist. If they don't then follow these instructions exactly and try again.
3. Locate 'fastboot'. On my system I found it under 'out/host/darwin-x86/bin/fastboot'.
   
4. Execute the following commands (I included the outputs I got):
   

$ ln -s out/host/darwin-x86/bin/fastboot out/target/product/dream/fastboot
$ cd out/target/product/dream/fastboot
$ ./fastboot -w
erasing 'userdata'... OKAY
erasing 'cache'... OKAY
$ ./fastboot flash system system.img
sending 'system' (53946 KB)... OKAY
writing 'system'... OKAY
$ ./fastboot flash boot boot.img
sending 'boot' (1418 KB)... OKAY
writing 'boot'... OKAY
$ ./fastboot flash userdata userdata.img
sending 'userdata' (2 KB)... OKAY
writing 'userdata'... OKAY
$ ./fastboot flash reboot
rebooting...

After reboot I had the latest (and not necessarily the greatest) version of Android :-).

My experience with Android (and the device) has been positive so far. I have been updating the Android source code on my machine daily and am seeing quite a few changes being committed every day which is great.

My next goal is to try out the custom firmware by JesusFreke, an xda developer, which appear to be much better than the stock firmware. I will keep everyone posted on how that experience turns out.

I finally 'git' it

I started work on several personal projects of mine at home and very soon realized that managing code for these projects was getting out of hand.

After making sure that the code worked, I tarballed the folders and stored them a couple of times. However, once I started coding more regularly, I realized that the tarballing approach will not work. It is difficult to maintain and very impractical.

I wanted to have a revision control system in place that would be easy to use and required very little work from my side. Since, I was working on this alone, I also wanted something that would work on my machine but is easy to set up on another machine with very little effort. I used Perforce extensively for a very long time and was comfortable with it. But setting it and having to back it up regularly would have taken up too much of my time.

I tried CVS and after the first day, I started not really liking it. Since I was working on Android and used 'git', ableit very briefly, I wanted to give it a shot.

I did not realize till I downloaded the source code to build it that it was designed and developed by Linus Torvalds for Linux kernel development. I found this great video of Linus' talk on git when he visited the Google campus.

I setup 'git' on my machine in less than 5 minutes and completed my first checkin soon after that. Within the hour, I also setup 'gitweb' to access git repositories via the browser.

After a week's use I can say that I love 'git'. It is very simple to setup and use and moreover it is distributed making a central repository totally unnecessary. In fact, Linus in his talk said that he considered several alternatives to BitKeeper but anything that was not distributed was discarded immediately.

'git' can be downloaded and built from here and you can find the documentation here and here.

Update: Yet another great document on how to use 'git' and 'repo'.

It is all 'fsck'ed up

A couple of days ago, my MacBook Pro(MBP) refused to boot up. When booting from the Mac OS X partition, I got the usual spinning logo, then it would shut off completely. I did not realize there was an issue until after about five or so attempts at booting, all of which failed.

One of my options would have been to boot up the MBP in target disk mode, mount the volume on another Mac and run Disk Utility to find out any potential issues. Since, I had no other Mac machine handy, I decided to boot from the Mac OS X install disk and run the Disk Utility app that is in there.

Disk Utility detected a rather cryptic problem called 'invalid sibling link' error while verifying the catalog file. Also, interestingly, the 'Repair Disk' option in Disk Utility was grayed out.

I was desperate now. There was a lot of data in that partition which I needed and had no backup of. Thankfully, I was able to fix the problem and here is how I did it.

But, what is 'invalid sibling link' error?

To understand this, let us first, very briefly, look at how a HFS+ volume is structured.

An HFS+ volume contains five special files:

1. Catalog file: Describes the folder and file hierarchy of the volume. It is organized as a "balanced tree" for fast and efficient searches.
2. Extents overflow file: This keeps track of the allocation blocks that are allocated to each file as extent (extent-based file systems allocate disk blocks in large groups at a single time, which forces sequential allocation, of course this is somewhat of an oversimplification).
3. Allocation file: Specifies whether an allocation block is free. This is stored in a bitmap, specifying a free allocation block with a "clear bit".
4. Attributes file: Contains attribute information regarding files or folder.
5. Startup file: Allows computers to boot that do have built in support for HFS+ file systems

Catalog files, Extents overflow files and Attribute files are stored in data structures that are technically known as B-trees (technically the HFS B-Trees are a variant of B+-Trees which Apple's technical documentation calls them B*-Trees.)

The Catalog tree (which describes a file or a folder) contains records which, for efficiency, are grouped into fixed length blocks called nodes. The B-trees are structured in such way that the records are in the leaf nodes. These records are sorted and indexed for fast searching which could physically scatter the nodes around the B-tree file. So, all the nodes are linked together to form a doubly linked list which allows the traversal of leaves in the same directory.

If, for whatever reason, the link between the leaves breaks (technically, becomes an invalid or dangling pointer), then we end up with an error like 'invalid sibling link'.

Now that we understand what the problem really is (through the rather oversimplified write-up that preceded this), here is how to fix the problem.

1. Start in single user mode by holding down Command-S while booting (in my case the problem occurred on the boot volume so this was required).
   
2. Once the text scrolls all the way down you should be at a command prompt
3. Enter fsck -fy (fsck is a file system consistency and repair tool. the 'f' option is to force checks even on a clean file system and the 'y' is to just blindly say 'yes' to every option that may come up. Read the man document on fsck for more detailed explanations).
   
4. If it reports 'FILE SYSTEM WAS MODIFIED' repeat from step 3 again.
   
5. Keep repeating until it says 'The volume appears to be OK' (or something to that extent) or you give up ;-).
6. If successful, type reboot to reboot.
   

Running fsck multiple times may be necessary. In my case I had to run it 4 times before the problem was fixed.

Googling for 'invalid sibling link' returns thousands of results but none of them really explain what causes the problem. Here is another good solution (and a great discussion) to the same problem.

If neither of these solutions work, then Alsoft's DiskWarrior (which will set up back by $99.95) could be another option although even this tool has varying degrees of success.

The first thing I did, after successfully rebooting, was to setup Time Machine. After this episode, I realized the need for a good backup solution. If things are beyond repair, then I at least know that my data is safe. :-)

Last week Palm announced their much anticipated Palm Pre mobile phone. The Palm Pre features a slide-out QWERTY keypad to complement its 3.1-inch touchscreen display, a 3-megapixel camera with flash, EV-DO Rev. A/HSDPA connectivity, Bluetooth, GPS, WiFi, Bluetooth 2.1 with A2DP and EDR, proximity and light sensor, accelerometer, 8GB internal memory, and runs on their latest mobile operating system, the WebOS. The Pre will be one of the first smartphones to feature wireless charging, achieved using electromagnetic induction and the use of an optional wireless charging dock, dubbed the 'Touchstone'.

While watching the Palm Pre CES launch video, I couldn't help but feel that Pre had features that iPhone could have easily had. The card view makes multi-tasking a cinch and 'Synergy' which brings all your email accounts and contact and calendar information from various sources into one place is not all that difficult for Apple to implement. Maybe in firmware v3.0 along with cut and paste.

WebOS looks very promising. Development for WebOS will be through an yet-to-be-released-to-public RAD framework called Mojo. Support for background apps and 'direct access to the device's capabilities' has been confirmed by Ars.

Slated for release in the first half of 2009 in the US, Palm Pre definitely brings innovation and life back to the struggling company. Here are some positive and not so positive reviews about the Palm Pre.

Personally, I think the CDMA only support (atleast for now) is a bummer. While waiting for Palm to bring out the GSM model, I cannot wait to get hold of the SDK and play around with it.

Update: Good article on Engadget titled 'What Apple could learn from Palm's WebOS'.

Regex and Prime Numbers

I was looking for articles and examples on advanced regular expressions when I came across this amazing expression that prints prime numbers:

perl -e 'print "PRIME" if (1 x shift) !~ /^(11+)\1+$/' xxxx

I knew regular expressions were powerful but this is pure ninja stuff. The origin of this expression is attributed to Abigail who also maintains the Regex::Common module. Abigail is a frequent poster to the comp.lang.perl.misc newsgroup and this example originated as a one-line program in one of Abigail’s sign-off signatures. Since then, it has been commented on within the group on a few occasions and with a few other variations.

You can find explanations on how and why this works here or here.

Perl vs Python

Perl vs Python is a classic debate that has flared up on many forums, websites and user groups. Here is my $0.02 on the whole affair.

I was a very devoted Perl user till 2005 and then switched to Python, mainly because everyone else in the project was using it.

Like, so many others before me, I had difficulty getting used to the 'no bracket, indentation only' approach. But after some initial hiccups (mainly strong denial and prejudice against Python), I started writing some useful programs in Python. The indentation stopped bothering me after a while, but I never really bothered mastering Python the way I did Perl.

However, my opinion about Perl (and, indirectly, Python) changed last year. I had to generate some code coverage data for the Mac project I was working on but raw 'gcov' data files were not really helpful. I found that 'lcov' could create well formatted HTML pages showing code coverage data and annoted source code. I was very happy with lcov but going through lcov source code, written in Perl, left my head spinning.

I started losing track of the context within which certain variables were being used. The fact that I hadn't used Perl in while and had forgotten quite a lot of the idiosyncrasies native to the language did not help matters either. But I felt very guilty. I loved Perl before and I did not have any complaints against it earlier, so what happened now?

I went back to my old source code 'repository' (actually it was a bunch of zip files on a CD :-)) and had a look at the Perl code I had written. What was interesting was none of the scripts I wrote were more than 200 or so lines of code. They were effective but never long and I had never re-used any script. That made sense to me. I actually enjoyed using Perl as long as the scripts were small, I was the only one using them and I did not have to re-use them.

Henry Spencer, a well known computer programmer, called Perl a 'Swiss-Army Chainsaw, intending to convey his evaluation of the language as exceedingly powerful but ugly and noisy and prone to belch noxious fumes'. I think his observation is spot-on.

But once I started working with a team, Python seemed better. Python was a lot easier to read and understand and the learning curve was lot less steep. Python has everything that Perl offers but without the eccentricities. Moreover, Python has a better implementation of object orientation and is inherent to the language where as Perl's implementation has a very kludgy feel to it.

I will still use Perl for what it was originally intended for, mostly scripts that munge data, I will keep them small and will probably be the only one using it. I will, probably, use Python for everything else that is more involved and has complex tasks.

Update: I found this article, by Eric Raymond, which presents an argument in Python's favor more articulately.

Laughter's really the best medicine

Marissa Mayer's laughter compilation from her Stanford speech :-)



But this, by far, is the funniest laugh I have ever heard.

QR Codes

I met up with Srikanth at Cafe Coffee Day last week. While we were sipping some overpriced beverages, our conversation went around to the topic of 'mobile tagging'.

The image below explains what mobile tagging is all about.


You just point your phone's camera to a 2D bar code, take a picture of it and decode it. There are several 2D bar code formats but one of the most popular is QR (or Quick Response) code.

QR codes can contain anything from plain alphanumeric text (or a website address or an SMS message) to airline tickets. The Japanese even use QR codes on graves. Check out this interesting article about potential uses of QR code in India.

There are several QR code generators out there including one from Google. Download a QR code-reader for your phone from BeeTagg and find out where the code below takes you :-).