Installing Gentoo On An Encrypted Root Partition


There are times when one needs to be able to know that if a computer's hard drive falls into the wrong hands that the information on it cannot be harvested. This is increasingly needed for laptops, desktops, and servers whose hard drives may be storing required-to-be confidential material. Of course, it need not only be such, but, one may wish to have the peace-of-mind that his/her contact information and other personal data is also not accessible (e.g., should their laptop be stolen). The purpose of this post is to detail how configure the Gentoo Linux operating system so that it boots from an encrypted root partition.

DISCLAIMER: Use of this post in any manner is completely and totally at your own risk. You are responsible for what you do with your own equipment/installations. Always read all documentation before using any commands.

Installation Media

In order to install the operating system, one first needs to be able to:

  • boot from a CD/DVD capable of installing Gentoo,
  • be able to partition the hard drive using software on the CD/DVD,
  • be able to run cryptsetup from the CD in order to format the hard drive with encryption,
  • be able to run LVM (Logical Volume Management) software from the CD/DVD, and,
  • be able to format the logical volumes created so the operating system can be installed.

Unlike a normal operating system installation, one cannot reboot the computer and boot into the operating system until enough software is installed so the operating system can actually mount the root partition. Further until such is made to work the only recourse one has is to have reliable boot CDs/DVDs with the required software programs needed to mount partitions so any issues can be fixed.

IMPORTANT: Ensure you have working bootable media on hand before you start. This is especially important if you need to stop and resume things later: make sure you have the necessary bootable media!

The simplest way to ease the burden of an installation is to use the latest Gentoo LiveDVD as it boots a fully working copy of Gentoo Linux with KDE, etc. It also has all of the tools, including cryptsetup, as well as the necessary kernel modules for encryption compiled into the Linux kernel. Without the proper kernel modules, it will not be possible to format the root partition with encryption.

NOTE: The minimal CD for Gentoo is very minimal and nearly everything has to be done manually. Unless you are used to this, I would not recommend using it. The LiveDVD, despite its large size, is much nicer as it configures the network (if it uses DHCP), has firmware for various network cards, can be used with KDE or simply as a text console, has all of the tools needed to encrypt, format, and mount hard drive paritions, etc. This post assumes the use of the LiveDVD. Briefly, the LiveDVD is nice, and convenient whereas the minimal CD is tedious.

Assumptions And Definitions

It is assumed herein:

  • the installation media is the Gentoo Linux LiveDVD (or equivalent),
  • the hard drive is to only have Linux installed on it,
  • there is only one hard drive,
  • Internet access is present, and,
  • a USB memory stick (or equivalent) is on-hand to hold encryption keys needed to boot the system.

You must have a USB memory stick! Do not store any keys on the hard drive.

The following variables are used below to represent a number of things that are system dependent:

Variable Definition
BOOT_DEV Device name of the (unencrypted) boot partition.
ENC_HDD A name given to the cryptsetup-created encrypted device made after calling luksOpen.
INST_HDD Installation hard drive's device name, e.g., /dev/sda. /dev/hdc, etc.
KEY_DEV USB memory stick device name.
KEY_FILE Full path to encryption key.
KEY_MOUNT Mounted directory path for $KEY_DEV.
ROOT_DEV Device name of the (encrypted) root partition.
ROOT_NAME The name of the root logical volume, e.g., root.
SWAP_NAME The name of the swap logical volume, e.g., swap.
VG_NAME The name of the volume group used with LVM, e.g., vg_hostos.

If You Get Stuck…

Since this post is very long, this helpful section is given before the start of any instructions so that it will be noticed and used if needed. If you get stuck after the encrypted partitions have been created and had to shutdown or reboot, then these are the steps to get back up-and-running:

  1. Boot the LiveDVD.
  2. Open a Konsole/Terminal window.
  3. If the network is not already set up, configure it.
  4. Insert and mount your USB key that holds the encryption key.
  5. Mount the encrypted partition so it can be used:
    cryptsetup --key-file $KEY_MOUNT/key.bin luksOpen $ROOT_DEV $ENC_HDD
  6. Activate the volumne group:
    vgchange -a y $VG_NAME
  7. Activate the swap partition:
    swapon /dev/$VG_NAME/$SWAP_NAME
  8. Run:
    mkdir -p /mnt/gentoo
  9. Mount the (encrypted) root partition:
    mount /dev/$VG_NAME/$ROOT_NAME /mnt/gentoo
  10. Run:
    mkdir -p /mnt/gentoo/boot
  11. Mount the (unencrypted) boot partition:
    mount $BOOT_DEV /mnt/gentoo/boot
  12. Copy resolv.conf over from the LiveDVD set up:
    cp -L /etc/resolv.conf /mnt/gentoo/etc/
  13. Mount /proc so it will be accessible from the chroot jail:
    mount -t proc none /mnt/gentoo/proc
  14. Mount /dev so it will be accessible from the chroot jail:
    mount --rbind /dev /mnt/gentoo/dev
  15. Enter a chroot jail on the encrypted drive:
    chroot /mnt/gentoo /bin/bash
  16. Use the environment from the hard drive:
  17. Source /etc/profile:
    source /etc/profile
  18. Fix the prompt to indicate that the chroot jail is active:
    export PS1="(chroot) $PS1"

and then you can perform any required work. Obviously, if you did not create the swap partition or other details, some of the above commands given will not work.

Installation Steps

Partition The Hard Drive

After booting the LiveDVD, use any partition manager to partition the hard drive (e.g., fdisk, gparted, etc.). If you want to partition the drive with GPT then you must install it first:

emerge -av gptfdisk

Create two partitions:

Partition Notes
$BOOT_DEV FDISK type = 83, GPT type = 8300.
Minimum size = 16 MiB. Will remain unencrypted and needs to be large enough to hold contents of /boot.
$ROOT_DEV FDISK type = 7, GPT type = 0700.
Fill rest of hard drive.

Modern hard drives are faster if these partitions are aligned on 2048-byte (or megabyte in gparted) boundaries. This can be set in the expert menu in fdisk and gdisk.

Populate $ROOT_DEV With Random Data

This step is optional but one would likely want to have any previous hard drive content to appear no different than random data. To do this run the following command:

dd if=/dev/urandom of=$ROOT_DEV bs=2048

Create The Encryption Key

In order to encrypt the hard drive one needs to use one or more of:

  • a (long) passphrase, and/or,
  • a random file of binary data which can be encrypted with a passphrase using GPG later.

Ideally, one wants to have a very strong key and would normally choose to create a file with random data. The size of this file depends on the encryption method used. Herein we'll use the AES (i.e., the aes-xts-plain kernel module) with 512 bytes of random data. This key should be generated somewhere other than the installation hard drive (i.e., use the USB memory stick). One can do this with the following steps:

  • Insert and mount the USB memory stick.
  • Change the current directory to the USB memory stick.
  • Create a key file using /dev/random (which requires activity on the computer for entropy):
    head -c 512 /dev/random >keyfile.bin
  • Note that $KEY_FILE is $KEY_MOUNT/keyfile.bin below.

Seriously consider giving the keyfile a meaningful name. I would also suggest incorporating the hard drive's make, model, serial number, and partition number into its name. One can obtain such information without opening the computer by installing and running lshw.

If a passphrase is preferred, then don't specify any --key-file options to cryptsetup below. You can have up to eight (8) key files and/or passphrases set on any partition. If using passphrases, ensure that they are very long so as to have enough entropy. Instead of using passphrases, seriously consider using key files encrypted with GnuPG with a passphrase instead as used in this post.

Create And Mount The Encrypted Partition

Before any disk partition and data can be put onto the encrypted partition, it must first be formated using cryptsetup:

cryptsetup --cipher aes-xts-plain -s 512 \
  --key-file $KEY_FILE luksFormat $ROOT_DEV

NOTE: The Cryptsetup FAQ in response to the question, "Can I resize a dm-crypt or LUKS partition?" notes that aes-xts-plain can only be used for partitions less than 2 TiB. If the encrypted partition is to be greater than or equal to 2 TiB in size then use aes-xts-plain64.

Once the partition has been formatted (with respect to its encryption only), it can then be mounted to provide a device that appears to not be encrypted. Of course, the device is actually encrypted, but, for any programs to make use of it, it needs to have an unencrypted "view". The mount is accomplished using the following command:

cryptsetup --key-file $KEY_FILE luksOpen $ROOT_DEV $ENC_HDD

NOTE: Set $ENC_HDD to something meaningful. For example, if $ROOT_DEV is /dev/sda2 then perhaps set $ENC_HDD to hdd_a2.

The $ENC_HDD representing the unencrypted device is now online. If logical volume management (LVM) is used on it then $ENC_HDD should be thought of as a virtual encrypted hard drive. The full path of the device is: /dev/mapper/$ENC_HDD

Use LVM Create The Partitions To Install The Operating System

Instead of treating $ENC_HDD as a single partition, logical volume management (LVM) will be used to allow (i) the creation of a swap device and (ii) the root partition that the operating system will be installed on. One may not want to have Linux installed into one partition. If so, then instead of creating a single root partition, simply create all of the partitions, appropriately sized, as is appropriate for your circumstances.

Currently, there is only one "physical" (encrypted) volume, /dev/mapper/$ENC_HDD, and the LVM system must be told this (i.e., to create a physical volume):

pvcreate /dev/mapper/$ENC_HDD

With LVM, physical volumes are placed into (various) volume groups within which logical volumes (i.e., "partitions") are created. Since no previous volume groups have been created yet, one needs to be created:

vgcreate $VG_NAME /dev/mapper/$ENC_HDD

In order to do anything with a volume group, it must be activated:

vgchange -a y $VG_NAME

The swap partition should be on $ENC_HDD so it is encrypted. Should anything be swapped out to the hard drive, which will contain data from running programs, it will also be encrypted. Additionally, the swap partition should be contiguously stored, thus, the command used to create the swap partition is slightly different:

lvcreate -L$SWAP_SIZE --contiguous y -n$SWAP_NAME $VG_NAME

NOTE: $SWAP_SIZE will be a value such as 24G (which would be 24 GiB). See the lvcreate man page for further details.

The swap partition's device name is now /dev/$VG_NAME/$SWAP_NAME.

In order to create the root partition in a manner that uses all remaining space available in the volume group, the number of physical extents available needs to be obtained:

  vgdisplay | \
  egrep 'Free[[:space:]]+PE / Size' | \
  awk '{ print $5 }' \

Now the root partition can be created:

lvcreate -l $PESIZE -n$ROOT_NAME $VG_NAME

The root partition's device name is /dev/$VG_NAME/$ROOT_NAME.

Formatting And Mounting Required Partitions

In order to install and use the operating system a chroot jail will be needed and this will require (re-)mounting /dev and /proc in addition to the root and boot partitions so they are accessible from inside the chroot jail:

  1. Format the swap partition:
    mkswap /dev/$VG_NAME/$SWAP_NAME
  2. Turn on the swap partition:
    swapon /dev/$VG_NAME/$SWAP_NAME
  3. Format the root partition:
    mkfs.ext4 /dev/$VG_NAME/$ROOT_NAME
  4. Mount the root partition:
    mkdir -p /mnt/gentoo
    mount /dev/$VG_NAME/$ROOT_NAME /mnt/gentoo
  5. Format the boot partition:
    mkfs.ext4 $BOOT_DEV
  6. Mount the root partition:
    mkdir -p /mnt/gentoo/boot
    mount $BOOT_DEV /mnt/gentoo/boot
  7. Copy resolv.conf over from the LiveDVD set up:
    cp -L /etc/resolv.conf /mnt/gentoo/etc/
  8. Mount /proc so it will be accessible from the chroot jail:
    mount -t proc none /mnt/gentoo/proc
  9. Mount /dev so it will be accessible from the chroot jail:
    mount --rbind /dev /mnt/gentoo/dev
  10. Enter a chroot jail on the encrypted drive:
    chroot /mnt/gentoo /bin/bash
  11. Use the environment from the hard drive:
  12. Source /etc/profile:
    source /etc/profile
  13. Fix the prompt to indicate that the chroot jail is active:
    export PS1="(chroot) $PS1"

Install The Operating System

At this point one should follow the normal installation procedure for Gentoo Linux starting immediately after the steps of formatting and mounting all partitions. Ensure that these packages are also installed:

  • sys-fs/cryptsetup – Must be the same version or higher as what was on the LiveDVD.
  • sys-apps/busybox – If you want a decent shell within initramfs.
  • eix – An available package query tool.
  • sys-kernel/genkernel – Needed to easily build the kernel.
  • sys-kernel/gentoo-sources or sys-kernel/hardened-sources – Linux kernel sources.
  • app-crypt/gnupg – Needed for security (i.e., to have a passphrase on the keyfile).
  • sys-kernel/module-rebuild – Tool to build out-of-kernel kernel modules.
  • app-portage/portage-utils – Additional portage package management tools.
  • sys-apps/util-linux – For /sbin/blkid

Ensuring cryptsetup Version Is Correct

It is possible for the version of cryptsetup on the LiveDVD to be different from the version that is considered "stable" in the operating system. If an older version relative to the LiveDVD of cryptsetup is installed then the system will likely not be able to boot properly. In order to determine if the version of cryptsetup is the same or newer than the one on the LiveDVD, do the following:

  1. Open another console window (i.e., don't use any chrooted terminal session) and run this command as root to obtain the version of cryptsetup on the LiveDVD:
    cryptsetup --version
  2. In the chroot jail console, run:
    emerge cryptsetup
    cryptsetup --version

If the version of cryptsetup in the chroot jail console is older than the one in the LiveDVD console, then Gentoo must be told to use the newer version:

  1. Edit /etc/portage/package.accept_keywords/crypto to contain this line:
    sys-fs/cryptsetup ~amd64
  2. Re-emerge cryptsetup:
    emerge -av --autounmask=y --autounmask-write=y cryptsetup

Configuring And Building The Linux Kernel

After running emerge -av gentoo-sources or emerge -av hardened-sources the linux kernel will have been installed into /usr/src/linux. The steps to configure a Linux kernel are:

  • Run make menuconfig.
  • Set any and all settings.
  • Save and exit the tool.
  • Build the kernel, its modules, and initramfs.
  • Install such to /boot.
  • Ensure the boot loader (e.g., grub) is able to boot the kernel.

While one can roll their own kernel configuration, this post focuses on using genkernel with the gentoo-sources or hardened-sources packages for these reasons:

  • the genkernel utility completely automates everything, except this post will intentionally run make menuconfig manually,
  • booting using an encrypted root partitions as described in this post will require the following to be built and installed in initramfs:
    • statically compiling sys-fs/cryptsetup,
    • statically compiling app-crypt/gnupg,
    • statically compiling (optionally) sys-apps/busybox,
    • statically compiling in disklabel (UUID) support, and,
    • writing a script/program to prompt the user for a key-file or passphrase in order to be able to mount the encrypted system.

In short, using genkernel automates a number a lot of extra work that most won't need to be concerned with on practical level, especially if time-constrained. In order to have genkernel automate the kernel building process, do the following:

  1. Change the current working directory to /usr/src/linux:
    cd /usr/src/linux
  2. Copy the LiveDVD's kernel configuration to the current directory:
    zcat /proc/config.gz >.config
  3. Configure the kernel by running make menuconfig and ensure the following options are set to Yes to avoid issues when booting:
    • Cryptographic API –> CBC support
    • Cryptographic API –> ECB support
    • Cryptographic API –> XTS support
    • Cryptographic API –> HMAC support
    • Cryptographic API –> all CRC32c options
    • Cryptographic API –> MD5 digest algorithm
    • Cryptographic API –> Michael MIC keyed digest algorithm
    • Cryptographic API –> SHA1 digest algorithm
    • Cryptographic API –> SHA224 and SHA256 digest algorithm
    • Cryptographic API –> SHA384 and SHA512 digest algorithms
    • Cryptographic API –> all AES options
  4. Before running genkernel, ensure these settings are set in /etc/genkernel.conf:
  5. If you use MAKEOPTS in /etc/make.conf, then ensure you set the same in genkernel as genkernel does not use /etc/make.conf at all. (See this post on how to set MAKEOPTS.)
  6. Build the kernel, its modules, necessary statically linked software, and install it to /boot:
    genkernel all
  7. Determine any external-to-the-kernel modules for your system:
    module-rebuild populate
  8. Rebuild any external-to-the-kernel modules:
    module-rebuild rebuild

Configuring /etc/fstab and grub

Determining All Partitions' UUIDs

Run blkid and note the UUIDs for the following partitions:

  • $BOOT_DEV – This is the unencrypted boot partition.
  • $ROOT_DEV – This is the encrypted root partition UUID which grub will need in order to find the partition at boot time.
  • /dev/mapper/$VG_NAME/$SWAP_NAME – This is the swap partition.
  • /dev/mapper/$VG_NAME/$ROOT_NAME – This is the root partition that the booted operating system uses.

NOTE: The output from blkid has quotation marks surrounding them. You'll need to remove the quotation marks when placing the UUIDs in /etc/fstab and in grub.conf.

Configuring /etc/fstab

Edit /etc/fstab so that it contains the following:

UUID=ROOT_NAME_UUID   /            ext4    errors=remount-ro     0 1
UUID=BOOT_DEV_UUID    /boot        ext4    noauto,noatime        0 1
UUID=SWAP_NAME_UUID   none         swap    sw                    0 0
/dev/cdrom            /mnt/cdrom   auto    noauto,user,ro        0 0
proc                  /proc        proc    defaults              0 0
shm                   /dev/shm     tmpfs   nodev,nosuid,noexec   0 0


  • ROOT_NAME_UUID is the UUID value for /dev/$VG_NAME/$ROOT_NAME,
  • BOOT_DEV_UUID is the UUID value for $BOOT_DEV, and,
  • SWAP_NAME_UUID is the UUID value for /dev/$VG_NAME/$SWAP_NAME.

Protecting The Keyfile

When the key was generated earlier, it was generated as a binary file. One, however, does not want to walk around with a key that that has no passphrase in case it is lost or stolen, one will need to crack the passphrase in order to obtain the key. To convert the key to a GPG protected key, do the following:

  1. Mount the USB memory stick with the key.
  2. Run:
    gpg --symmetric -o $KEY_FILE.gpg $KEY_FILE

Configuring grub

Except for editing /boot/grub/grub.conf installing and configuring grub is straight-forward:

  • If grub has not been installed yet, do so:
    emerge -av grub
  • Using the UUID for $ROOT_DEV from the previous section, edit /boot/grub/grub.conf (see below).
  • Copy the active mounts:
    cat /proc/mounts >/etc/mtab
  • Install grub to the hard drive's boot sector:
    grub-install --no-floppy $INST_HDD

Examine $ROOT_DEV and note the drive letter and partition number which are the last two characters in its name. The grub boot loader uses numbers to identify drives and partitions starting from 0, therefore, drive letters 'a', 'b', 'c', etc. correspond to 'hd0', 'hd1', 'hd2', etc. and partition numbers '1', '2', '3', etc. correspond to '0', '1', '2' etc. This matters since the line with root (hd0,1) needs to be set to the correct drive and partition number as per the system being installed. Further, the appropriate file names, UUID, $VG_NAME, $KEY_DEV, $KEY_FILE, $KEY_MOUNT, $ROOT_NAME must be set/replaced with the proper values for grub to boot. Knowing this, entries in /boot/grub/grub.conf will be similar to:

title Gentoo Linux (Keyfile Boot)
root (hd0,0)
kernel /boot/kernel-genkernel-x86_64-2.6.39-gentoo-r3 root=/dev/ram0 crypt_root=UUID=ROOT_DEV_UUID real_root=/dev/mapper/$VG_NAME-$ROOT_NAME root_keydev=$KEY_DEV root_key=$KEY_FILE.gpg rootfstype=ext4 key_timeout=0 dolvm
initrd /boot/initramfs-genkernel-x86_64-2.6.39-gentoo-r3

title Gentoo Linux (Passphrase Boot)
root (hd0,0)
kernel /boot/kernel-genkernel-x86_64-2.6.39-gentoo-r3 root=/dev/ram0 crypt_root=UUID=ROOT_DEV_UUID real_root=/dev/mapper/$VG_NAME-$ROOT_NAME rootfstype=ext4 key_timeout=0 dolvm
initrd /boot/initramfs-genkernel-x86_64-2.6.39-gentoo-r3


Finally, the system can be rebooted. If everything was set correctly, the system should fully reboot. If not, follow the instructions in the "If You Get Stuck…" section to be able to access the contents of the encrypted partition so any issues can be fixed.

Kindly contact me should any typos or errors be found within this post.

Parallel Builds With Gentoo's Emerge


Gentoo allows one to specify the degree of concurrency to be employed when emerging packages with Portage in two ways by specifying:

  • the -j and the -l options in the MAKEOPTS variable in /etc/make.conf, or,
  • the --jobs and --load-average options on the emerge command line or in the EMERGE_DEFAULT_OPTS variable in /etc/make.conf.

The article describes some of my experiences with using the above parameters.

Determining Hardware Limits On Parallelism

Assuming the operating system being used to run Portage is Linux, one needs to determine the number of CPU cores and theads available for parallelism. This can be determined by executing the following:

grep '^processor' /proc/cpuinfo | sort -u | wc -l

For brevity, this amount will be referred to as NJOBS below, i.e., as if this were run:

NJOBS=$(grep '^processor' /proc/cpuinfo | sort -u | wc -l)

Maximizing Processor Saturation

Ideally one would want all processors busy performing work as this would allow one to minimize the total amount of time compiling code. However, one does not want the system load to become excessive for the following reasons:

  • to minimize process switching, and,
  • to keep the system responsive to user input.

The first step in this process is to set MAKEOPTS to have as its maximum number if parallel tasks to be NJOBS+1 and to limit starting any new jobs if the load is NJOBS or higher in /etc/make.conf:

MAKEOPTS="-j$((NJOBS+1)) -l${NJOBS}"

where the -j option sets the maximum number of parallel jobs that can be run via make and the -l prevents any new parallel job starting unless the load is below the amount specified. (The reason the number of jobs is set to one higher than the number of processors is to help ensure saturation of processor utilization.)

There is no need to set an NJOBS variable in /etc/make.conf as hardware will not change, so one should simply substitute the proper values in MAKEOPTS. For example, MAKEOPTS suitable for an i7 processor would be:

MAKEOPTS="-j9 -l8"

Setting MAKEOPTS is very safe as there are very few packages with parallel make issues. Typically, if there are issues, the ebuild for that package will turn off the option, or, will give notice that -j1 must be used.

Setting MAKEOPTS will improve the build times of a number of packages, however:

  • many packages don't perform parallel makes, and,
  • many packages' build procedures won't saturate all processors with load.

This is most apparent when running long-build tasks such as emerge -eav world or when building large software programs such as Firefox, Chromium, or LibreOffice. In my experience an i7 processor will have a load between 1.00 and 2.00 most of the time if only MAKEOPTS is set. This is not even close to ideal.

To better utilize processors, one has to tell emerge to also run parallel jobs. This is done using the --jobs and --load-average options with emerge:

emerge --jobs=${NJOBS} --load-average=${NJOBS} world

Here, unlike MAKEOPTS, one need only set the jobs and the load limit to the number of processors as tasks will be run within such that will generate load. Since there is a load average limit specified here and in MAKEOPTS the system should not become overly busy and start thrashing or need to excessive amounts of process switching. (If such is an issue, then lower the load average settings.)

Since emerge's command line overrides anything set in /etc/make.conf, I set EMERGE_DEFAULT_OPTS in /etc/make.conf as follows:

EMERGE_DEFAULT_OPTS="--jobs=${NJOBS} --load-average=${NJOBS}"

i.e., for an i7 this would be:

EMERGE_DEFAULT_OPTS="--jobs=8 --load-average=8"

so I don't have to specify such on the command line. For packages that have issues being compiled in parallel, one need only override --jobs on the command line setting it to 1:

emerge -j1 world

Dealing With Failed Builds

There are some instances of packages that will simply not build when emerged as parallel jobs. This poses a significant issue if one was performing an emerge world of hundreds or thousands of packages. Fortunately, emerge has the ability to resume a compile as well as to skip the first package when resuming. Assuming the packages are already built on the computer, you can safely skip the package and rebuild it later with -j1. If you are extra careful not to do such more than once (or you'll lose the ability to resume), you can even do this to build the package (e.g., in another window) and when done, emerge -rav --skipfirst to resume the build process.

Want More Output?

When the number of jobs is greater than one, there is only a summary output. To see the tasks actually being performed, run:

qlop -c

or the continuously updating:

while true ; do clear ; qlop -c ; sleep 2 ; done

To see the actual build output that one sees with emerge -j1, simply run tail for the package, PKG, being built:

tail -f /var/tmp/portage/*/${PKG}*/temp/build.log

by replacing ${PKG} with the name of the package.

Different Values For Jobs and Load Averages

Elsewhere on the Internet, I've seen people use different values such as having the number of processors and assigning half to MAKEOPTS and the other half to emerge. Unfortunately, this will on average saturate and utilize only half of the processors. In my experience, even this "average" does not happen frequently!

Logically, it is much better to set the jobs to the number of processors and then to use load average settings to limit the load to limit the total number of parallel tasks created during the build process. This strategy, as described above, allows all processors to be used without excessive loads being created (e.g., on an i7 the highest loads I've seen with the above settings is approximately 9.0). It also allows parallelism to occur when packages are being built that don't use parallel make (or are limited in how much parallelism they can do). This is why the emerge load average value is set as it is above –as the MAKEOPTS load average ensures parallel tasks only start running if the load is not too high. My experience with such are very positive with good to excellent load averages when things can be done in parallel.


In my experience, using the settings outlined above significantly speeds up compiles on multi-core machines. There are only a couple of packages that don't like the parallel emerge where I need to intervene but the rest build without a problem and my cores/processors become mostly utilized instead of being mostly idle.

Installing Ubuntu In A VirtualBox Session For Web Development


Often when I teach a web development class I get a number of questions related to what is a good way to install software suitable for web development on one's computer. Of course, officially, this is all provided on the school's computers, computer science students (i) want to learn how to set up such on their own machines and (ii) want to be able to do web development even when they are not connected to the Internet. Although outside the official support of any web course(s) that I may be teaching, the purpose of this article is to outline how one would set out to do this using Ubuntu and VirtualBox.

Disclaimer: While I have made the effort to provide accurate information, it is entirely possible there are errors, etc. Do read the documentation for everything mentioned herein and your use of such is completely at your own risk.

Step I: Acquire VirtualBox & Ubuntu

Download and Install VirtualBox

This is the easiest step! Simply go to and download the latest VirtualBox program and install it. If VirtualBox will be run on a 64-bit computer, be sure to download the 64-bit version.

VirtualBox is a software program that is capable of running an entire PC inside a virtual machine. This means other operating systems can be installed inside of it, booted, and then run –all inside a window on your computer. Nicely this allows one to have and use other operating systems on your computer without actually having to resize, create, and format partitions, set up your computer for dual booting, etc. This also allows you to use the operating system you use the most at the same time as other operatings systems (one per virtual machine that you run) at the same time –which is otherwise impossible using only a single computer.

Download Ubuntu

Before one can do anything with VirtualBox, one needs installation disc(s) for an operating system so that it can be installed. For this article, download the Ubuntu CD-ROM ISO image from Again, if this will be run on a 64-bit computer, be sure to download the 64-bit "amd64" version of Ubuntu. The "amd64" version will work with Intel or AMD64 chipsets.

Now that VirtualBox has been installed and the Ubuntu installation CD-ROM ISO file, the next step is to create a virtual machine session in VirtualBox and install Ubuntu!

Step II: Establish The Virtual Machine

Create The Virtual Machine

Unlike a real computer, with VirtualBox one can configure the which hardware resources as well as the amount of RAM, hard disk space, etc. that the virtual machine will be allowed to have. To use any software program that let's you run virtual machines, it is highly recommended that you have extra of all of the resources required to run the operating system and the software in it. Currently, to boot your computer and to run software you computer needs to have a certain amount of RAM available and a certain amount of hard disk space available + free RAM and free hard disk space to do things. When running a virtual machine, the same is true –but one will also need an additional amount of RAM and hard disk space that the virtual machine needs as well.

Important: If you don't have enough RAM or disk space to do this step, then you will have to acquire it in order to proceed further.

The amount hard disk space available to your virtual machine is easy to set at the beginning and harder to increase later. Since VirtualBox has a "grow" option for hard disk space that increases in size as more space is used, one is best to create an appropriately large hard drive. As for RAM, this can be easily changed at a later point in time.

To create the virtual machine in VirtualBox, follow these steps:

  1. Run VirtualBox.
  2. Click the New button to create a virtual machine.
  3. A Create New Virtual Machine dialog will appear.
  4. Click the Next > button.
  5. Type in a Name for the virtual machine (e.g., Ubuntu).
  6. Select the Operating System (OS) Type for the machine (e.g., Operating System: Linux, Version: Ubuntu or Ubuntu 64-bit).
  7. Click the Next > button.
  8. Select the Base Memory Size. If unsure, go with the recommended amount (e.g., 384 MiB).
  9. Click the Next > button.
  10. Ensure that the Boot Hard Disk checkbox is selected.
  11. Select the Create new hard disk radio button.
  12. Click the Next > button.
  13. A Create New Virtual Disk dialog window will appear. Click the Next > button.
  14. Under Storage Type select the Dynamically expanding storage radio button.
  15. Click the Next > button.
  16. Choose a Location to store the virtual hard disk image if the default is not liked.
  17. Choose a Size for your the disk (e.g., 8 GiB). Remember: this is not easily changed later!
  18. Click the Next > button.
  19. Click the Finish button to close the Create New Virtual Disk dialog.
  20. Click the Finish button to close the Create New Virtual Machine dialog.
  21. Ensure that the newly created virtual machine is selected.
  22. Click the Settings button in the tool bar.
  23. Click the Storage item in the left-hand side listbox.
  24. Click the Empty CD-ROM disc under IDE Controller.
  25. Click the CD-ROM drop-down button and select Choose a virtual CD/DVD disk file….
  26. Find and select the ISO file that was downloaded for Ubuntu.
  27. Click OK in the Settings dialog window.
  28. Click Start button in the toolbar to start the virtual machine to start installing Ubuntu!

Step III: Install Ubuntu

Installing Ubuntu

The last step in the previous section started the virtual machine. Just like a real computer would, you can see it (quickly!) count up the memory, look for the disk drives present, and boot off the Ubuntu installation CD-ROM. There are two modes to run Ubuntu off the CD-ROM: (i) as a LiveCD or to (ii) install. You can install Ubuntu with either option, but, the LiveCD option let's you run try Ubuntu without installing anything on your computer. There isn't a need to select the LiveCD option when using a virtual machine –except if you need to "rescue" an unbootable computer, so choose just to install Ubuntu.

Do the following steps when the installation screen appears:

  1. Click on English and choose Install Ubuntu.
  2. On the Time page choose the appropriate region (e.g., Canada) and an appropriate timezone city/locale (e.g., Toronto).
  3. Click the Forward button.
  4. On the keyboard page, choose the appropriate keyboard layout (e.g., USA keyboard layout).
  5. Click the Forward button.
  6. On the disk partitioning screen, select Use the entire disk.
  7. Click the Forward button.
  8. Enter in your full name, a login, a password, and a machine name.
  9. Click the Forward button.
  10. Click Install.
  11. Wait for all of the files to be downloaded and copied.
  12. Click Restart Now.
  13. Since your CD-ROM isn't real, removing it is a little different than what you may be used to! When prompted to remove the CD-ROM do the following:
    1. From the VirtualBox machine window's pull down menu, select the Devices menu.
    2. Select the CD/DVD Devices menu item.
    3. Uncheck the CD-ROM ISO file. This will "eject" the installation CD-ROM.
    4. Press enter inside the virtual machine window to reboot.
  14. After rebooting, login using your login and password.
  15. If prompted, install any updates.

Step IV: Post-OS-Install Software Configuration

Software Installation

With Ubuntu installed, all that remains is to install Apache web server, PHP, PostgreSQL, and a set of handy utilities! Here are the steps to do this:

  1. In Ubuntu, open a Terminal window via Applications | Accessories | Terminal.
  2. Run:
    sudo apt-get install aptitude
  3. Run:
    sudo aptitude install openssh-server
  4. Run:
    sudo aptitude install apache2 apache2-doc php5 \
    php5-mcrypt php5-ps php5-timezonedb php5-xmlrpc php5-xsl \
    php5-cli php5-curl php5-dev php5-gd php5-imagick php5-mysql php5-pgsql \
    php5-tidy php5-xmlrpc php5-xsl php5-xdebug php5-uuid php5-gmp php5-recode \
    sablotron libapache2-modxslt
  5. To install the PostgreSQL database run:
    sudo aptitude install postgresql postgresql-client postgresql-doc php5-pgsql
  6. To install the MySQL database run:
    sudo aptitude install mysql-client mysql-server php5-mysql
  7. Run:
    sudo aptitude install tinyca
  8. Run:
    sudo aptitude install vim ctags vim-doc vim-scripts
  9. Run:
    sudo aptitude install xsel
  10. Run:
    sudo aptitude install gedit-plugins
  11. Run:
    sudo aptitude install gimp
  12. Run:
    sudo aptitude install dwww w3-recs doxygen doxygen-doc
  13. Run:
    sudo aptitude install w3c-dtd-xhtml w3c-markup-validator
  14. Run:
    sudo aptitude install csstidy xmlstarlet
  15. Run:
    sudo aptitude install mutt postfix qpopper
    1. Choose Local only as the delivery option.
    2. Type in the name exactly the same as what was given the system during the install.
    3. Run:
      sudo nano -w /etc/postfix/
    4. Edit the line starting with mydestination= to read:
      mydestination = 60-334, localhost.localdomain, localhost,,
    5. Run:
      sudo /etc/init.d/postfix reload
    6. Run:
      sudo nano -w /etc/aliases
    7. Ensure that the aliases file has this content (with YOUR_LOGIN_NAME replaced with your login):
      root: YOUR_LOGIN_NAME
      postmaster: YOUR_LOGIN_NAME
      webmaster: YOUR_LOGIN_NAME
      webadmin: YOUR_LOGIN_NAME
  16. Before the mail program can be configured to deliver (localhost) mail, an email needs to be sent as follows:
    1. Run:
    2. Answer No to the Create mail folder in home folder question.
    3. Hit m to generate an email to send.
    4. Enter your login name in the To: line.
    5. Enter Test in the Subject: line.
    6. Enter This is a test. in the body.
    7. Hit Ctrl-X and save the message (using the given file name).
    8. Hit y to send the message.
    9. Hit q to quit.
  17. With an email waiting, configure the Evolution mail client as follows:
    1. Open Applications | Office | Evolution Mail and Calendar.
    2. Since this is the first time you are opening Evolution, a dialog window will appear to allow you to configure its settings. Click Forward.
    3. Click Forward (if there is a backup screen).
    4. Enter your Full Name, your email address within the virtual machine (i.e., YOUR_LOGIN_NAME@localhost), and make it the default account.
    5. Click Forward.
    6. Set the Server Type to Local delivery and the path to /var/mail/spool/YOUR_LOGIN_NAME.
    7. Click Forward.
    8. Check messages every minute.
    9. Click Forward.
    10. Set the Outgoing Mail Server Type to SMTP and the Server Name to localhost.
    11. Click Forward.
    12. Click Forward.
    13. Click Apply.
  18. Right-click Applications | Office | Evolution Mail and Calendar and choose Add to Panel. This will add the Evolution icon to the panel at the top of the screen.
  19. Run Firefox:
    1. Create a bookmark for dwww:
    2. Create a bookmark for the w3-recs package's directory (via dwww):
    3. Create a bookmark for dwww's Help | Standards menu (i.e., for w3-recs package files):
    4. Create a bookmark for the W3C Markup Validator:
    5. Install these Add-Ons: Firebug, FirePHP, Firefinder for Firebug, FireQuery, FireXPath, Inline Code FInder for Firebug, and Web Developer.
  20. Configure the Apache web server for virtual hosting:
    1. Ensure that /etc/hosts has in it (e.g., sudo nano -w /etc/hosts) while noting that any 127/8 address will work: www
    2. Run:
      sudo nano -w /etc/apache2/sites-available/

      and add:

        ServerAdmin webmaster@localhost

        DocumentRoot /home/YOUR_LOGIN_NAME/web/public
        <Directory /home/YOUR_LOGIN_NAME/web/public>
          Options Indexes FollowSymLinks MultiViews
          AllowOverride None
          Order allow,deny
          Allow from all

        LogLevel warn
        ErrorLog /home/YOUR_LOGIN_NAME/web/logs/error.log
        CustomLog /home/YOUR_LOGIN_NAME/web/logs/access.log combined
        ServerSignature Off

        <IfModule mod_php5.c>
          php_flag magic_quotes_gpc Off
          php_flag magic_quotes_runtime Off
          php_flag file_uploads On
          php_flag short_open_tag On
          php_flag session.auto_start Off
          php_flag session.bug_compat_warn Off

          php_value upload_max_filesize 16M
          php_value post_max_size 16M

          php_value error_log /home/YOUR_LOGIN_NAME/web/logs/php_errors.log
          php_flag display_errors Off
          php_flag display_startup_errors Off

        <IfModule mod_dir.c>
            DirectoryIndex index.php index.xml index.html index.htm
  21. Run:
    sudo nano -w /etc/apache2/ports.conf

    to have the following line in it (which must match the IP in the /etc/hosts file):

  22. Run:
    mkdir -p /home/YOUR_LOGIN_NAME/web/{public,log}
  23. Run:
    sudo chown -R www-data:www-data /home/YOUR_LOGIN_NAME/web/log
  24. Run:
    sudo a2ensite
  25. Run:
    sudo /etc/init.d/apache2 force-reload

Step V: Accelerate VirtualBox (Optional)

Installing VirtualBox Additions

VirtualBox provides a number of device drivers that provide features (e.g., Shared Folders) and accelerate the speed of the virtual machine. (The instructions below applied to VirtualBox version 3.x. I've not checked that they still apply to version 4.x.) To install these VirtualBox Additions do the following:

  1. Ensure you are logged into the virtual machine.
  2. Choose from the VirtualBox Session window Devices | Install Guest Additions….
  3. In the Ubuntu window, click on the Places menu.
  4. Open the CD-ROM menu item.
  5. After the CD-ROM drive appears, open Applications | Accessories | Terminal.
  6. Run:
    cd /media
  7. Run:
  8. Note the CD-ROM name, e.g., VBOXADDITIONS_3.1.8_61349.
  9. Run:
    cd VBOX*
  10. Run (if a 32-bit install, otherwise change the name appropriately):
    Run: sudo bash ./
  11. Wait for the drivers to build.
  12. Run:
  13. In the CD-ROM window, click on the Eject icon for the CD-ROM. You may have to make the Window larger to see it.
  14. In the VirtualBox window menu, click on Devices | CD / DVD Devices… and uncheck VBoxGuestAdditions.iso.
  15. If you get slow screen updates, then turn off 3D in Ubuntu by:
    1. Opening System | Preferences | Appearance.
    2. Click on the Visual Effects tab.
    3. Choose None.
    4. Click Close.
  16. If VirtualBox is upgraded, remember to redo the above to upgrade your Guest Additions!

Step VI: Enable Shared Folders And Bridged Networking

Web development tasks can be simplified considerably by enabling the Bridged Networking feature and using VirtualBox' Shared Folders. Both of these are found in the Settings window.

The bridged networking feature allows the host OS to talk directly to the guest OS (i.e.. Ubuntu). After the the network setting is changed from NAT to Bridged and the computer restarted, the IP address of Ubuntu can be found by running:

sudo /sbin/ifconfig

and then using that to connect to your virtual machine from the host OS.

The Shared Folders feature makes it very easy to move files to/from the guest and host OSes. To configure Shared Folders do the following:

  1. Before booting Ubuntu, edit the Shared Folders settings and add the desired folder(s). Be sure to write down each shared folder's name as that will be needed below.
  2. One can mount the shared folder by running in a Terminal window:
  3. Know that to mount a device in Linux the A_MOUNT_DIRECTORY must previously exist as a directory:
    sudo mkdir /media/windows
  4. For example, if the A_MOUNT_DIRECTORY was /media/windows and the share SHARED_FOLDER_NAME was myspace, you could mount it using:
    sudo mount -t vboxsf -o rw,uid=1000 myspace /media/windows
  5. You could also have it permanently mount the share every time you boot by adding something like this to /etc/fstab (where uid=1000 is only true if you are the only in your virtual machine):
    public_html /home/YOUR_LOGIN_NAME/web/public vboxsf rw,auto,uid=1000 0 0

Step VII: Read Documentation

There are a lot of commands and details above. Be sure to read the installed online documentation accessing via dwww right on the virtual machine. Additionally, seek out the documentation on the Internet for the various programs. Kindly do not send me, "How do I do this [but it is already on the page]? Can you do this for me [because I don't want to invest any time towards such]?" emails especially if you did not read all relevant documentation for the items of concern and give serious efforts towards resolving the issues you are trying to resolve. (Sadly people do send emails like that and they are very inappropriate.)

If you are new at and are learning how to do all of this, truly the best way is to struggle through it on your own especially armed with the level of detail I have provided on this page. If you do, you will learn a lot and you will remember it.

Linux Containers, /dev/pts/ptmx, and Gentoo

For virtualization technologies (e.g., environments, machines) Linux now supports containers which allow multiple instances of the devpts filesystem. This allows the /dev/pts instance indices in one virtual environment to be independent of the indices allocated in another environment.

As detailed in the kernel file Documentation/filesystems/devpts.txt, this requires the kernel option:


and the "-o newinstance" mount option when mounting devpts.

Gentoo's sys-apps/openrc-0.8.3-r1 configuration has a /dev/init.d/devfs script in the sysinit runlevel. Despite it name and the fact that the system uses udev, the devfs initscript states that it mounts "required stuff as user may not have [them] in /etc/fstab". Unfortunately, the devfs script does not check for entries in fstab before mounting the devpts and shm filesystems. Fortunately, the Documentation/filesystem/devpts.txt nicely mentions what to do if initscripts are not updated to handle CONFIG_DEVPTS_MULTIPLE_INSTANCES. Since one doesn't likely want to do such things manually after every reboot and it is generally a bad idea to start editing a distribution's installation scripts, here's an initscript that performs the devpts.txt file's instructions:


description="Multiple PTY fix for /dev/ptmx"

  after udev devfs

  if [ -e /dev/pts/ptmx ]; then
    # Use next line if devpts is in /etc/fstab...
    mount -o remount /dev/pts
    # Or uncomment next line...
    #chmod 0666 /dev/pts/ptmx
    rm /dev/ptmx
    ln -s pts/ptmx /dev/ptmx
    return 0
    echo "/dev/ptmx does not exist!"
    return 1

which also requires this line to be in /etc/fstab (but adjust the perms to your liking):

devpts /dev/pts devpts rw,relatime,gid=5,mode=666,ptmxmode=666 0 0

and to run in the sysinit runlevel:

# rc-update add ptmx-fix sysinit

Test it by rebooting and checking that /dev/ptmx is a symlink to /dev/pts/ptmx and the permissions of /dev/pts/ptmx are 0666. 🙂

Printing: Duplex & Margins

If you are using KDE4, you may discover that a number of applications that use GTK/GNOME (e.g., Firefox and Thunderbird) to print, their Print…|Page Setup screens will not let you select any Two-Sided options. Since there is no nice global KDE4 printer configuration tool yet, the fix is to:

  1. Access CUPS using a web browser via http://localhost:631/.
  2. Click on the Printers tab.
  3. For the desired printer, select Set Printer Options.
  4. Change the Duplexer Installed setting from Not Installed to Installed.
  5. Click Set Printer Options.

Also, if you are using HPLIP and are having issues with the print margins, then this may help:

  1. Upgrade to the latest stable CUPS, if you haven't already done so.
  2. Upgrade to the latest stable HPLIP, if you haven't already done so.
  3. Run the hp-toolbox tool.
  4. Delete all of the relevant HP printers.
    • This will delete all settings and PPD files for those printers.
  5. Recreate all of your HP printers.
    • This will recreate all default printer settings, PPD files, etc.

I recently encountered this issue on a computer which was only resolved by (i) ensuring the latest stable releases of CUPS and HPLIP were installed and then (ii) deleting all HP printers and recreating them. With hp-toolbox, deleting and recreating all HP printers is quick and easy: simply use the round "-" (to delete) and "+" (to create) buttons.

NOTE: If you do this for an HP printer supporting duplex printing, remember to follow the directions given earlier to set the Duplexer Installed setting to Installed! 🙂

Finally, avoid using "testing" or "unstable" versions of CUPS or HPLIP unless you have a newly released printer that was unsupported, or, there is a critical bug fix for your printer that must be fixed.