How to create an Owl-based OS template for OpenVZ

This wiki page is for “Owl hackers”. New users are advised to proceed to the "Getting started" page instead.

NB: Almost all of the content below is obsoleted by Owl's integration of OpenVZ on 2009/11/23 and by implementation of the “make vztemplate” target in the Owl build environment on 2010/01/28. Things have become much simpler now. This wiki page needs to be updated accordingly.

This text is based on the Galaxy's instruction on OS template creation published on owl-users mailing list in 2007, with some corrections and additional explanations.

Disclaimer: This instruction may be incomplete and may not follow the best possible practice; it is even possible it simply won't work. This text is neither endorsed nor authorized by any persons from OpenVZ or Openwall.

First of all, make sure you've got OpenVZ ready to work, that is, you run OpenVZ kernel, the module vzctl is active (either compiled in or modprobed) and the packages vzctl and vzquota are installed.

Before installing the packages please note that all the VPS containers typically live under the /vz directory which should (at least it is a very good idea, see this http://wiki.openvz.org/Quick_installation#Filesystems for detail) be placed on a separate filesystem. So, if you want to follow this recommendation, then first create the appropriate filesystem, mount it under /vz (perhaps with the “noexec,nosuid,nodev” options to avoid inadvertent or malicious use of container-created programs or device files on the host system) and then proceed with installing OpenVZ modules.

The kernel which worked for me (Croco) can be taken from http://download.openvz.org/kernel/branches/2.6.18/stable/, however as of this writing it is obsolete. Only OpenVZ's "rhel5" branch kernels are currently recommended for production use. If you take one of OpenVZ's pre-built kernels, please note that you'll have to make sure the module vzdev is loaded at the boot time, which is not obvious for Owl as it usually uses monolithic kernels. FIXME: There are Openwall patches to OpenVZ kernels “in the works”, which are to be made public soon, at which point this paragraph will need to be updated accordingly. There's one known bug when running the Owl userland with a 2.6 kernel (including OpenVZ) without the not-yet-released Openwall patches: klogd fails to run as non-root, so it exits. As a temporary workaround, klogd may be re-configured to run as root (by editing /etc/init.d/syslog to remove -u klogd).

The other packages are found here: http://download.openvz.org/current/ and you need three of them: vzctl, vzctl-lib and vzquota. Simply install them with

 rpm -Uvh vz* --nodeps

(the instruction provided by Galaxy says the deps can be safely ignored, and it also worked for me)

Now create an empty OS template with the following commands:

 # cd /vz/template/cache
 # touch empty.txt
 # tar czf empty.tar.gz empty.txt
 # rm empty.txt

Having the empty template, create an OpenVZ container using the template:

 # vzctl create 100 --ostemplate empty

(there will be some errors during the execution of this command but they can be ignored)

The next step is to actually install Owl into the container. To do so you'll need the Owl build/install environment which is found under /usr/src/world on the Owl CD but perhaps is not installed into your system. The simplest possible way to get it is to use the Owl CD as such (or its uncompressed image mounted somewhere using the loopback mount option); however, you'll need to change the installworld.conf file so the /usr/src/world tree must be copied to your HDD. FIXME I realize this can be done without the CD image, downloading only the required things from ftp, not the whole CD image. Anyone who knows the actual step-by-step procedure is welcome to update this text.

Let's assume you mounted the Owl CD under /mnt/cdrom. Then (as root) do the following:

su - build
cd /mnt/cdrom/usr/src/world
tar -cSf - . | tar -C /usr/src/world -xpf -

You've got the build/install tree under your /usr/src/world. Now update the installworld.conf file:

su - build # if you exited it before, otherwise just ''cd'' to get back to ''/usr/src/world'' (user ''build'' home directory)
rm -f installworld.conf # in case it was created as a symlink
cp native/Owl/build/installworld.conf .
vi installworld.conf 

Somewhere at the top of the file (say, within the first 10 lines of it) you'll see the line

ROOT=/owl

Comment it out and replace with the following:

ROOT=/vz/private/100

Alternatively, you could have temporarily created /owl as a symlink to /vz/private/100.

You're ready to install the packages. Just issue the command

make installworld

and go have a cup of tea or coffee as it typically takes a few minutes (we've observed 30 seconds on the fastest machines to 20+ minutes on obsolete ones).

Once the install is completed, add the line

route add default venet0

to the file /vz/private/100/etc/rc.d/rc.local and make this file executable:

chmod +x /vz/private/100/etc/rc.d/rc.local

This is a dirty workaround that may or may not be needed depending on version of vzctl (its scripts) and the OpenVZ kernel. Alternatively, GATEWAYDEV=venet0 may be added to /etc/sysconfig/network within the container. Once we integrate OpenVZ containers support into Owl and debug/test it, no such tricks will be needed in any case.

Also be sure to comment out getty lines from /etc/inittab; your VE has no terminals to run getty on, so if your init tries to run them, this will flood your /var/log/messages and /var/log/wtmp with lots of junk. So, edit the file /vz/private/100/etc/inittab, locate the following lines:

# /sbin/getty invocations for the runlevels.
#
# The "id" field MUST be the same as the last
# characters of the device (after "tty").  
#
# Format:
#  <id>:<runlevels>:<action>:<process>
1:2345:respawn:/sbin/mingetty tty1
2:23:respawn:/sbin/mingetty tty2
3:23:respawn:/sbin/mingetty tty3
4:23:respawn:/sbin/mingetty tty4
5:23:respawn:/sbin/mingetty tty5
6:23:respawn:/sbin/mingetty tty6

and comment them out with a #, or just remove them — you'll never need them with your template anyway.

The next step is to form the correct /etc/fstab for your template. Strictly speaking, this is not critical, but you should perform this step if you want, e.g., the df command to work properly. Open /vz/private/100/etc/fstab in your text editor (there is likely already some content provided by the installed packages) and replace its entire content with the following:

simfs           /                       ext3    usrquota,grpquota       0 1
proc            /proc                   proc    nosuid,noexec,gid=110   0 0

It is the first of them which is OpenVZ-specific. It is also recommended to have the following lines:

devpts          /dev/pts                devpts  gid=5,mode=620          0 0
tmpfs           /tmp                    tmpfs   noexec,nosuid,nodev     0 0
sysfs           /sys                    sysfs   noexec,nosuid           0 0

The container is almost ready – at least you can start it. So do it, and change right into it to see how it looks inside:

vzctl start 100
vzctl enter 100

By the way, it is the right time to change permissions of the root directory within your new container, as it typically has root of 700 mode (so you won't be able to add any new users and execute any commands from a non-root user). So, either being inside the container, do

chmod 755 /

or being outside it, do

chmod 755 /vz/private/100

BTW, be sure to remove that empty.txt file from the container's root :-) you don't need that file, do you?

You can make any other improvements to the container's system if you wish; then, leave the container (e.g., pressing Ctrl-D to end the shell) and stop it:

vzctl stop 100

Now, before you actually turn the container's contents into a template, please remove ssh host keys as they should be generated from scratch for each VPS (and will be unless you leave them in the template – so be sure to remove them):

rm /vz/private/100/etc/ssh/ssh_host*

FIXME Maybe it was preferable to not boot the would-be-template container up in the first place? Booting it up also creates log files with a few records.

Change to the container's root and actually create the template:

cd /vz/private/100
tar -czSf /vz/template/cache/owl-XXX.tar.gz --one-file-system -C /vz/private/100 .

(replace XXX with your version of Owl). Your template is ready. You can try it in work, creating another container:

vzctl create 101 --ostemplate owl-XXX

(again, replace XXX with your version or whatever word you used when created it).

FIXME Everything should be fine now, but it might be not bad to check for the permissions of your new container's root. They must be 755 (or at least 711), or else you'll get fine problems. Well, if you follow the explained procedure, create the container and everything's fine with permissions, feel free to remove this notice from this page.

Now start it and go to http://wiki.openvz.org/Basic_operations_in_OpenVZ_environment page for some useful things you can do now.