LILO MBRs on RAID drives

FIXME This is a little bit edited copy of Solar's message on one of the lists where he described the result of his analysis of LILO's source code. Eventually, we need to convert this to a Wiki article.
Date: Wed, 21 Oct 2009 03:25:51 +0400
From: Solar Designer
 
Galaxy,
 
I read up on LILO (the source code, disassembly of the MBRs,
values/flags embedded into the MBRs).
 
I found out that LILO's MBR (first.S in this case) is capable of using
the following approaches to determine the BIOS drive number to read
"the rest" from:
 
1. In theory, it can use the drive number as passed by the BIOS in the
DL register upon MBR code startup.  However, in our case this
functionality is disabled.  The first.S code checks for FLAG_MAP_ON_BOOT,
and this flag appears to not be set in our MBRs, perhaps because of this:
 
/* if the state of the BIOS is DL_GOOD, always mark when boot==map
   if the state of the BIOS is < DL_GOOD, never mark */
 
        if ( bios_boot == bios_map  &&
                (bios_passes_dl == DL_GOOD
                 || (do_md_install && !(extra==X_MBR_ONLY ))
                  ) )
            bsect.par_1.prompt |= FLAG_MAP_ON_BOOT;
 
It appears that DL_GOOD is never set in the current code.  The code
which might set it is #if 0'ed:
 
#if 0
        if ( mask &&  ( extra == X_NONE ||
                        (extra == X_AUTO  &&  autocount == 0) ) )  {
            if (bios_passes_dl > DL_BAD) bios_passes_dl = DL_GOOD;
        }
#endif
 
2. The MBRs have the BIOS drive numbers embedded in them, at offset 30
(decimal).  However, it appears that these are only used as a fallback
in case the search described below (#3) fails.
 
Right now, the numbers are: 82, 82, 82, 83 for MBRs of sda, sdb, sdc,
and sdd, respectively.
 
3. There's code to search all hard drives for the 32-bit "VolumeID" or
"serial no" (it is referred to differently) that is encoded in MBRs at
offset 440 (decimal).  This is documented:
 
http://en.wikipedia.org/wiki/Master_boot_record
 
"01B8   0670    440     Optional Disk signature         4"
 
LILO also stores the value it will search all MBRs for at offset 24
(decimal).  For our four hard drives, these are currently set to:
 
Offset 440 (actual unique ids):
 
0b6b879c
5d690ae1
09f27048
70a523b3
 
Offset 24 (the values being searched for, with offset 440 ones being
checked against these):
 
09f27048
09f27048
09f27048
70a523b3
 
That is, even if sda or sdb are used for reading the MBR from, they will
then try to read the rest from sdc and, failing that, from sdd.
 
Only if neither sdc nor sdd are found (by their VolumeIDs), then there
will be fallback to the drive number encoded in the MBR, which for both
sda and sdb is currently 0x82 (not good given the circumstances we're
considering here, although with the root fs on sdc+sdd only this does
not matter anyway).
 
I haven't figured out the exact effect of raid-extra-boot yet (didn't
spend time on it).  I am not sure if our use of it is appropriate.
 
Some of my commands were:
 
dd if=/dev/sda of=sda-mbr count=1
objdump -D -b binary -m i8086 sda-mbr | less
cmp -l sda-mbr sdb-mbr
dd if=sda-mbr bs=1 skip=30 count=1 | od -tx1
 
(and ditto for other hard drives)
 
for n in a b c d; do dd if=sd$n-mbr bs=1 skip=440 count=4 2> /dev/null | od -tx4; done
for n in a b c d; do dd if=sd$n-mbr bs=1 skip=24 count=4 2> /dev/null | od -tx4; done