Customized Ubuntu distributions part three

Apparently most of the stuff in my previous posting was the wrong approach. I think I've finally got what I want. All this time, I've been trying to figure out how to do a live Linux CD that (a) includes some code we've been developing at work, and (b) boots very quickly and simply to where the user can use that code. The goal is to provide software tools to a partner company where everybody's laptop runs Windows, but all our stuff is written in Linux.

First I tried to build our code in Cygwin using the Windows version of libusb. I found that fraught with complexities of all sorts and eventually decided the Live CD approach sounded a lot easier. Besides, we wanted the Live CD/bootable USB stick anyway for some later plans.

Theoretically there are small Linux distributions (the most famous being Damn Small Linux) that can be used for this sort of thing. As soon as I started getting into that, I found that DSL is no longer maintained, the documentation for it is insufficient and the pieces that do exist contradict one another. I struggled to resolve dependency and version issues in porting our code to DSL and finally gave up. By that time, I had already discovered how to make an Ubuntu Live CD, and so I delivered one with the first piece of our code to our partner.

But I really wanted a much shorter boot time. I don't need X Windows or networking or OpenOffice or a web browser. I'd prefer to have a development environment on there in case the code required modification but even that is unnecessary.

In the past several days I've tinkered with about a dozen Linux distributions claiming to be "small" and found them all deficient in one way or another. I've tried dozens and dozens of permutations of dumb little tricks involving VirtualBox and QEMU and Ubuntu Customization Kit and burning CD-Rs and USB sticks. I've looked at what feels like hundreds of different web pages and blog postings, each claiming to have an authoritative and trustworthy solution to my problem. Each involves failures to account for discrepancies between versions, or the document I've found is old and inapplicable to what I'm doing, or the author made several minor assumptions that don't work in my environment.

Currently I'm looking at something called Ununtu Mini Remix which looks promising. It's looking very good so far, as I am remastering it with the information on the Ubuntu help website. Adding a shell script to /bin to make sure I can, and adding an "echo HELLO" to /etc/skel/.bashrc to make sure it appears when the disk boots into a bash session.

Everything was going great and then mksquashfs got hung up on the proc directory -- AH, this happened because when you finish a chroot session you must do three umounts (dev/pts, proc, sys) even if you weren't aware of having mounted them. Apparently chroot mounts them without telling you. Umount those in the chroot environment, exit, then umount edit/dev, and the mksquashfs goes just dandy.

So my two dumb tricks in /bin and /etc/skel/.bashrc worked like a champ in VirtualBox and now I'm going to try to make a bootable USB stick. Ubuntu's Startup Disk Creator likes the file (it's very picky about what ISO files are considered bootable) and the USB stick works great in my Windows laptop. Now we make the Angry Birds WHEEEE noise, however dumb some people might find the game. The next step is to make my tweaks into a Deb file using this HOWTO so they go in painlessly.

Grazie mille to Fabrizio Balliano for creating Ubuntu Mini Remix.

Ubuntu Mini Remix - The tiny Ubuntu you can build on!

165MB Ubuntu livecd containing only the minimal Ubuntu package set. Wonderful as a rescue disk or as a base for Ubuntu derivative creation.

Customized Ubuntu distributions part two

Turns out I was mostly still spinning my wheels here -- move on to part three of this tale for the solution to the problem.

After a brief visit to the world of Damn Small Linux and subsequent narrow escape from eternal damnation, I returned to Ubuntu with the idea of reducing the size (not as big a priority as I initially thought), speeding up the boot time, and running a program when the user logs into X.

First a quick note about the 9.10/10.04 UCK thing. What's working well for me is to run UCK on a box that has 10.04 installed, and apply it to a 9.10 ISO. Do not attempt to run UCK inside a VMware instance, it's just a lot of pain. You can run the resulting ISO in QEMU but don't forget the "-boot d" argument.

To shrink the distro, I removed OpenOffice, Ubuntu Docs, and Evolution. Things I added included autoconf, emacs, git, guile, and openssh-server.

The low-hanging fruit for speeding up boot time is to bring up networking as a background process, described here. Doing this in Ubuntu 9.10 means editing /etc/init.d/networking, adding "&" here:
case "$1" in
start)
        /lib/init/upstart-job networking start &
        ;;

People have given a lot of thought to other approaches to speed up Ubuntu's boot time, and maybe I'll blog more about that as I investigate it further. I really need to dig deeper into the boot time topic. It will likely warrant another blog post.

Having an app start immediately when the user logs in is somewhat interesting. The X session startup stuff is all in /etc/X11/Xsession.d/ and the main thing here is /etc/X11/Xsession.d/40x11-common_xsessionrc where we find a call to the user's ".xsessionrc" file. The user's directory is populated from /etc/skel, so the trick here is to create /etc/skel/.xsessionrc:
export LC_ALL=C
${HOME}/hello.sh &

where hello.sh is a sample shell script just to make sure I've got the principle down pat:

#!/bin/sh
sleep 2   # wait for other xinit stuff to finish
xterm -geometry 120x50+0+0 -e "echo HELLO WORLD; sleep 5"

Hmm... that worked for a bit, then stopped working. I've since discovered another file, /etc/gdm/PreSession/Default, which seems more relevant. But that starts the app just a little too early, before the user is actually logged into the X session, so maybe I should put a time delay in my app? Annoying.

Fun with customized Ubuntu distributions

At work we have an interesting problem. We are working with some companies in Taiwan. Obviously there's a language difference, but there is another difference as well. We are an Ubuntu Linux shop, and they all have Windows laptops. Periodically we have bits of test code that they need to use, and the OS gulf needs to be overcome.

My first whack at this issue was to try to use Cygwin to rebuild our tools from source on a Windows platform. But after I'd spent a few days dealing with libusb, and making not a whole lot of progress, a co-worker suggested a bootable USB stick. The Taiwanese folks get to keep their Windows laptops, but with a quick reboot they can temporarily use Linux machines just like ours. So I set about learning the art of bootable USB sticks, which in Ubuntu 9.10 is pretty painless. (This is not the case with Ubuntu 10.04. If you need to do this, stick with 9.10.)

Not to keep you in suspense, the two magical things are

• Ubuntu Customization Kit, (sudo apt-get install uck) which produces an ISO file suitable for burning a CD or DVD which you can boot from, and
  
• USB Startup Disk Creator (already present in your System>Administration menu) which puts that ISO file onto a USB stick and makes the stick bootable.
  

These are amazingly easy-to-use tools, given the complexity of what they're doing. In the bad old days, the Knoppix distribution existed solely for the purpose of rendering this feat possible for mortals. That said, I learned a few tricks about these things which I'll pass along here. Do NOT use Ubuntu 10.04, as there is a serious bug in that version of UCK plus a handful of annoying behavioral oddities. These are fixed in a future UCK release, but that's not available in the Ubuntu 10.04 repositories. In order to produce a USB stick which could be used with a Windows laptop to produce another bootable USB stick, I put a copy of the ISO file onto the USB stick. The instructions for copying the USB stick then go like this.

• Boot into Windows and insert the first USB stick. Copy the ISO file somewhere memorable. Restart the laptop.
  
• Boot into Ubuntu using the USB stick. Once you're booted, insert the second USB stick.
  
• Bring up USB Startup Disk Creator. The original ISO file on the first USB stick (from which you are now running) will not be visible in the file system. But the Windows hard drive will be readable, so dig around in it to find the ISO file copy you just used. Use that as the source, and select the 2nd USB stick as the destination. Push the button.
  
• Once that installation is complete, copy the ISO file from the Windows hard drive onto the second USB stick. Voila, a copy.
  

Using the first part (making an ISO image) I was able to produce a DVD with some of our tools for the Taiwanese folks to use. I set up Traditional Chinese and English as languages, with the default to boot into Traditional Chinese. But then because it had some of our source code, I encrypted this entire 725 MB file, which is ironic given that Ubuntu is open source. But there had to be a way to encrypt only the proprietary stuff.

On the next boot image I send them (which will be a USB stick, not a DVD, since USB sticks are oh so much sexier), the contents of the stick will be open source, and the proprietary stuff will be pulled down from a little tarball on some handy little server. The thing that pulls down the tarball and handles security is my little tarball runner script. The new ISO is at http://willware.net/tbr-disk.iso, and if you need to share some closed-source Linux code with people in China or Taiwan, feel free to use it.

To use this bit of cleverness, build some code on your Linux box, package it up as a tarball (including a run.sh shell script at the root level, in case you need to do installation stuff), and if necessary, encrypt it using (my tweaked version of) the Twofish algorithm found on Sourceforge. Then post it to the Internet and email the password only to your intended recipients.

If I find the time and energy, I'll package up the tarball runner and the Twofish module as a Deb package to make the installation painless.

Preliminary things from the Life Extesion Conference

This weekend I'm attending Christine Peterson's Life Extension Conference in San Francisco. Chris wanted to put together information that is both scientifically valid and actionable, so she lined up a lot of really high-quality speakers. One thing I learned pretty quickly is that there are a large number of areas of expertise, generally interrelated, all pretty deep. I'll try to do a series of blog postings about these topics so this one will just skim a few highlights.

Here are some very quick bits of advice.

• Completely stop eating sugar.
  
• Exercise.
  
• Eat spinach and other leafy greens, take vitamin D and drink green tea.
  
• The health of your brain is crucial to your overall health. Meditation is better for your brain than puzzles and games.
  
• Intermittent fasting (e.g. 24 hours every 2 or 3 day) is good for you.
  

The popular aging theory that our bodies wear out over time is false. We know this because there are animals and plants thousands of years old which may die from accidents or mishaps, but they do not age biologically. Michael Rose has been breeding long-lived "Methuselah" fruit flies for over 30 years and he discussed his approach. There were a lot of great talks but I found this one clarified some bsic information about aging for me.

Simplistically assume that flies always begin reproducing at age A and always stop reproducing at age B. Any heritable cause of death that takes effect before age A will be strongly selected against, and any heritable cause of death that takes effect after age B will face no selection pressure at all. What Rose did was to tinker with A and B, delaying both, and discardiing the flies who didn't live very long, and he did this from 1980 to the present day. I think I'll have more to say about this when I've gone over my notes more, but a few quick things about these Methuselah flies.

We couldn't do this in 1980 but we can now sequence the DNA of these flies and compare it to the DNA of normal flies. What you see is that there are a lot of teeny differences widely spread over the genome. This leads me to think that there's no silver bullet longevity gene, but rather a lot of small tweaks that address a large number of heritable causes of death.

More stuff to come as I sift through my notes. Chris has talked about posting all the slides online and making the presentation videos available as a DVD.