Sideloading Nexus Devices with the Android SDK


Android 5.0 "Lollipop" was released yesterday and I wanted to check it out on my Nexus 7 2013 WiFi tablet without having to wait for the update to be pushed. I've done this several times with various Nexus devices now, but so infrequently that I always have to look up how to do it. So this time I'm taking some notes. It should be noted that most of this information is just scraped from the flash-all.{bat,sh} included in the factory image.

First off, Nexus factory images are listed here. Start off by pulling the proper one down, verifying the integrity, and then extracting the tarball:

~ $ mkdir tmp && cd tmp && wget -q
~/tmp $ md5sum razor-lrx21p-factory-ba55c6ab.tgz 
fd868b03bd00074dd7f70e31ad73b25a  razor-lrx21p-factory-ba55c6ab.tgz
~/tmp $ tar xf razor-lrx21p-factory-ba55c6ab.tgz 
~/tmp $ ls razor-lrx21p
bootloader-flo-flo-04.04.img  flash-all.bat

To start the bootloader will need to be unlocked. This will wipe the data on the device, but once unlocked this is no longer an issue when flashing in the future. Sideloading the image requires the Android SDK platform tools.

First thing's first: get into the bootloader. Begin by plugging the tablet into the PC and verifying that it's connected.

~/android-studio/sdk/platform-tools $ ./adb devices -l
List of devices attached 
0a16e983               device usb:1-1.2 product:razor model:Nexus_7 device:flo

Now reboot into the bootloader:

~/android-studio/sdk/platform-tools $ ./adb reboot bootloader

Once in the bootloader make sure the device is connected/detected:

~/android-studio/sdk/platform-tools $ sudo ./fastboot devices -l
0a16e983               fastboot usb:1-1.2

If the bootloader is not already unlocked this will need to be done first (note: this wipes the device, so be sure to back things up beforehand). Unlock the bootloader with the following command (follow the prompts on the device; it'll warn about wiping the device clean):

~/android-studio/sdk/platform-tools $ sudo ./fastboot oem unlock
(bootloader) Unlocking bootloader...
(bootloader) erasing userdata...
(bootloader) erasing userdata done
(bootloader) erasing cache...
(bootloader) erasing cache done
(bootloader) Unlocking bootloader done!
OKAY [ 65.559s]
finished. total time: 65.559s

Now flash the bootloader image downloaded earlier:

~/android-studio/sdk/platform-tools $ sudo ./fastboot  flash bootloader ~/tmp/razor-lrx21p/bootloader-flo-flo-04.04.img 
sending 'bootloader' (3911 KB)...
OKAY [  0.192s]
writing 'bootloader'...
OKAY [  1.936s]
finished. total time: 2.128s

And reboot into the bootloader again:

~/android-studio/sdk/platform-tools $ sudo ./fastboot reboot-bootloader
rebooting into bootloader...
OKAY [  0.006s]
finished. total time: 0.006s

Next flash the zip image:

~/android-studio/sdk/platform-tools $ sudo ./fastboot -w update ~/tmp/razor-lrx21p/ 
archive does not contain 'boot.sig'
archive does not contain 'recovery.sig'
archive does not contain 'system.sig'
Bootloader Version...: FLO-04.04
Baseband Version.....: none
Serial Number........: 0a16e983
checking product...
OKAY [  0.003s]
checking version-bootloader...
OKAY [  0.004s]
sending 'boot' (7154 KB)...
OKAY [  0.347s]
writing 'boot'...
OKAY [  0.420s]
sending 'recovery' (7740 KB)...
OKAY [  0.369s]
writing 'recovery'...
OKAY [  0.297s]
erasing 'system'...
OKAY [  1.510s]
sending 'system' (799121 KB)...
OKAY [ 37.893s]
writing 'system'...
OKAY [ 35.392s]
erasing 'userdata'...
OKAY [ 23.679s]
formatting 'userdata' partition...
Creating filesystem with parameters:
    Size: 28856791040
    Block size: 4096
    Blocks per group: 32768
    Inodes per group: 8192
    Inode size: 256
    Journal blocks: 32768
    Blocks: 7045115
    Block groups: 215
    Reserved block group size: 1024
Created filesystem with 11/1761280 inodes and 154578/7045115 blocks
sending 'userdata' (139085 KB)...
writing 'userdata'...
OKAY [ 13.485s]
erasing 'cache'...
OKAY [  0.406s]
formatting 'cache' partition...
Creating filesystem with parameters:
    Size: 587202560
    Block size: 4096
    Blocks per group: 32768
    Inodes per group: 7168
    Inode size: 256
    Journal blocks: 2240
    Blocks: 143360
    Block groups: 5
    Reserved block group size: 39
Created filesystem with 11/35840 inodes and 4616/143360 blocks
sending 'cache' (10984 KB)...
writing 'cache'...
OKAY [  1.033s]

finished. total time: 114.857s

And now it's finished. If the bootloader was already locked previously, then you can remove the -w flag in the last command to avoid wiping the device (note: I've not tried this yet).

Debugging CGI / CGit


I've been using CGit as a git web front-end for a bit now and just recently started looking into customizing the root header. Assuming I'd have to hack the code to do so, I decided to pull down the source and build it.

~ $ git clone
~ $ cd cgit/
~/cgit $ make get-git && make

After perusing cgit.c and ui-shared.c I was pleasantly surprised to find I wouldn't have to hack anything; it seems that just about everything I wanted to change was configurable via the cgit config file (default: /etc/cgitrc). For example for changing the root title and description, just set root-title and root-desc:

~ $ grep 'root' /etc/cgitrc
root-desc=git repository browser

This worked fine, but what about changing the logo? Just overwriting the logo.png in the resources directory should work, but I want to use an animated gif. The file extension then would be technically incorrect, the worst kind of incorrect. Fortunately, it too looks like this could be set with logo in cgitrc as well. But this didn't seem to work. What's going on?

To run cgit from the CLI, simply do the following. HTML output is printed to stdout and errors are printed to stderr. This will generate the root index content.

~/cgit $ CGIT_CONFIG="./cgitrc" ./cgit 1>stdout.html 2>stderr.log

As an example, to see the content generated for a specific repo, do the following:

~/cgit $ CGIT_CONFIG="./cgitrc" QUERY_STRING="url=bunkergate" ./cgit 1>stdout.html 2>stderr.log
~/cgit $ CGIT_CONFIG="./cgitrc" QUERY_STRING="url=bunkergate/tree/index.html" ./cgit 1>stdout.html 2>stderr.log

It's worth noting that this is probably a basic template for debugging any CGI.

Setting logo in the configuration file didn't seem to have the desired effect. How can we see what's going on? Simple, run it in GDB:

~/cgit $ CGIT_CONFIG="./cgitrc" QUERY_STRING="url=bunkergate/tree/index.html" gdb ./cgit

So what was happening? Well after setting a watchpoint on ctx.cfg.logo I noticed something peculiar. The variable was getting set to the value I expected, but then later it was getting set yet again to the undesired value. Turns out that the problem was that I had set logo twice in the configuration file. Ugh.

Shell Wildcard Channel Attacks


Using wildcards with a shell can open you up to channeling attacks (for example, sql injection is a type of channeling attack). When using the * wildcard in particular in a directory containing argument-like-filenames (e.g. -rf) can lead to wild results.

zum Beispiel:

[root@defensecode public]# ls -al
total 20
drwxrwxr-x.  5 leon   leon   4096 Oct 28 17:04 .
drwx------. 22 leon   leon   4096 Oct 28 16:15 ..
drwxrwxr-x.  2 leon   leon   4096 Oct 28 17:04 DIR1
drwxrwxr-x.  2 leon   leon   4096 Oct 28 17:04 DIR2
drwxrwxr-x.  2 leon   leon   4096 Oct 28 17:04 DIR3
-rw-rw-r--.  1 leon   leon      0 Oct 28 17:03 file1.txt
-rw-rw-r--.  1 leon   leon      0 Oct 28 17:03 file2.txt
-rw-rw-r--.  1 leon   leon      0 Oct 28 17:03 file3.txt
-rw-rw-r--.  1 nobody nobody    0 Oct 28 16:38 -rf

[root@defensecode public]# rm *
[root@defensecode public]# ls -al
total 8
drwxrwxr-x.  2 leon   leon   4096 Oct 28 17:05 .
drwx------. 22 leon   leon   4096 Oct 28 16:15 ..
-rw-rw-r--.  1 nobody nobody    0 Oct 28 16:38 -rf

Because rm * expands to:

[user@defensecode WILD]$ rm DIR1 DIR2 DIR3 file1.txt file2.txt file3.txt -rf

This type of attack used in conjunction with seemingly innocuous utilities like tar can lead to execution of arbitrary commands.

gvim + ibus woes / building gvim from source


My main development machine is running Ubuntu 12.04 and for a while now I've had issues with using gvim 7.3.x in conjunction with ibus. Specifically upon starting gvim there's a long start up latency (30s+) while the ibus daemon is running. Shutdown the ibus daemon and gvim starts up near instantly. Here is the ticket describing the issue in more detail in Launchpad.

Yesterday vim 7.4 was just released and according to the aforementioned ticket the issue was resolved in 7.3.530. Unfortunately neither vim 7.4 nor 7.3.530 are yet in the Ubuntu repos, so I decided to build 7.4 from source. Along the way I'll attempt to jot down some notes here for my future self.

I didn't know before I began, but gvim is actually built from the same source tree as vim source. Before we start, we'll need to make sure we have all the necessary development libraries required to build gvim. Ubuntu docs suggest the following command should work -- I can't say for sure as I had a lot of requisite development libraries installed as other projects I'm working on depend on them as well.

~ $ sudo apt-get build-dep vim

We'll need mercurial to pull down the latest vim sources. I'm going to build the sources in my ~/working directory. Since I doubt I'll need the repo metadata (the included .hg directory) so I'll remove that to reclaim a little space.

~ $ sudo apt-get install mercurial
~ $ cd ~/working/
~/working $ hg clone vim
~/working $ cd vim
~/working/vim $ du -sh
139M	.
~/working/vim $ rm -rf .hg
~/working/vim $ du -sh
65M	.

Then to build, change into the src directory and run configure with the options below. I want to build with pretty much all the features so I'll go with the option --with-features=huge; optionally you can use tiny, small, normal, or big instead; a breakdown of the differences is given here. The options of the form --enable-XXXinterp are support for vim plugins written in XXX. The --enable-gui=gtk2 option is obviously for gvim support.

~/working/vim $ cd src
~/working/vim $ ./configure --with-features=huge --enable-rubyinterp  \
                           --enable-pythoninterp --enable-perlinterp \
                           --enable-luainterp --enable-gui=gtk2

Once the configuration script runs without errors, we can then build with make. Note the VMRUNTIMEDIR variable passed to make. This is the location of default system runtime director for vim plugins, docs, dictionaries, etc... for Ubuntu.

~/working/vim $ make VIMRUNTIMEDIR=/usr/share/vim/vim73

At this point one could optionally install vim on their system using the typical make install target. But this feels dirty to me. Plus I prefer to keep all my development tools installed within my home directory. This makes migrating my my development environment much less painful. So I'll define an alias in my ~/.bashrc and be good to go.

~/working/vim $ alias | grep gvim
alias gvim='~/working/vim/src/vim -g -p'


A Solid Breakdown of the Linux Font Rendering Stack


This article saved me. Explains hinting and anti-aliasing quite well, also includes an excellent description of the Linux font rendering stack.

Succint Description of UTF-8 Encoding


UTF-8 is actually a pretty neat way to encode text. Its clever design leads to several interesting properties. Notably of which, at least to me, is that all ASCII files are already UTF-8 encoded.

  Unicode code points    |       UTF-8 encoding (binary)
        00-7F ( 7 bits)  |                            0tuvwxyz
    0080-07FF (11 bits)  |                   110pqrst 10uvwxyz
    0800-FFFF (16 bits)  |          1110jklm 10npqrst 10uvwxyz
010000-10FFFF (21 bits)  | 11110efg 10hijklm 10npqrst 10uvwxyz