These notes assume you wish to install a fresh debian system on a machine (whether a server or a workstation). In practice, always choose a netinstall ISO for the Debian installer.

Here, these notes were done on the basis of a Debian Squeeze installer (6.0).

Accessing the console for the installation (via a KVM switch, a BMC card such as DELL DRAC or HP iLO) or mounting the installer ISO is outside the scope of these notes.

Reference documentation

Prepare the Installer ISO

Download the latest ISO of the stable Debian installer here.

Typically select the “Small CDs” ISO (in the AMD64 architecture).

Note: If you install on the latest DELL server with a Broadcom NetXtreme II network interface, you will encounter a problem during the installation as the bnx2 firmware has licenses issues that makes it not present in the default installer.

To deal with this issue, you can:

modify the installer ISO to include the missing firmware such that everything is handle transparently
prepare a separate ISO containing the missing firmware and the debian package firmware-bnx2. Then, on installer demand, mount the ISO (this assumes you used a separate USB stick typically) such that the installation can continue.

The first method is obviously more convenient. In both case, I assume you do the following operations on a linux machine (Debian or Ubuntu).

Debian installer ISO modification (Ressource)

Again, the idea is to modify the install ISO (debian-6.0.0-amd64-netinst.iso) to contain the missing deb package.

download the file firmware-bnx2_0.28_all.deb
Now install isomaster (apt-get install isomaster)
Launch isomaster, open the debian installer ISO and modify it to copy in the firmware/ directory the missing firmware deb firmware-bnx2_0.28_all.deb
Quit and Save the ISO image (to debian-6.0.0-amd64-netinst-with-bnx2.iso for instance)

You can now use this ISO as you primary installer (burn it or attach it on boot)

Separate ISO containing the missing firmware (Ressource)

Download the file firmware-bnx2_0.28_all.deb then:

$> mkdir firmware-bnx2
$> cd firmware-bnx2
$> ar xv ~/Download/firmware-bnx2_0.28_all.deb 
x - debian-binary
x - control.tar.gz
x - data.tar.gz
$> tar xf data.tar.gz
$> ls lib/firmware/
bnx2  bnx2-06-4.0.5.fw  bnx2-09-4.0.5.fw

The missing file in my case was in lib/firmware/bnx2/bnx2-mips-09-5.0.0.j3.fw

Create the ISO containing the firmware AND the debian package:

$> mkisofs -o /tmp/firmware-bnx2-R610.iso -J -R ./firmware-bnx2/lib/firmware/bnx2/bnx2-mips-09-5.0.0.j3.fw firmware-bnx2_0.28_all.deb

Now you can use this ISO on debconf demand to add the missing firmware.

Console access

Obviously, you need to access the console to start an installation. If you have a screen and a keyboard attached to the machine you are about to install, jump to the next section dealing with the BIOS settings. Otherwise, you probably use some kind of KVM. The following notes may interest you in this case

DELL DRAC

Tested with DRAC 6.

You need unfortunately to rely on IE 32 bits to effectively use the console redirection AND the attached Virtual Media.

Update: I now managed to make it work on Firefox under Mac OS X. The hint came from here i.e. when you are prompt by firefox to open the file viewer.jnpl(xxxxxxxx), simply select to open the file with /System/Library/CoreServices/Java Web Start (always).You just can’t attach the virtual Media.

Log on iDrac web server and launch virtual console (configured as a java applet rather than an ActiveX component). The advantage is also that you can attach a given ISO to the console, as follows:

launch virtual console (java)
Menu “Virtual Media” / “Start Virtual Media”; Add Image + path to ISO and check “Map”
Let the panel open and go to iDrac web, select “Systeme” / “Configuration” and select “Virtual CD/DVD/Image ISO” as first boot
reboot the machine to start on the ISO

DRAC Console redirection over SSH:

Ressource: this is useful when working from home. I tried to configure the redirection of the appropriate ports (443, 623, 5900, 5901 and 3668/3889) as follows:

ssh  -p 22  -L 443:<DRAC_IP>:443 -L 623:<DRAC_IP>:623 -L 5900:<DRAC_IP>:5900 -L 5901:<DRAC_IP>:5901 -L 3668:<DRAC_IP>:3668 -L 3669:<DRAC_IP>:3669 -l <login> -N <server>

Whereas it’s possible the to access the DRAC web server via https://localhost (which is still useful to configure various aspects of the )

HP iLO

TODO

Work on firefox (Mac OS / Linux)

Preliminary setup: BIOS settings and Hardware RAID

BIOS setup

cf https://support.mayfirst.org/wiki/bios_changes

Except if you know what you’re doing:

Processor settings: Ensure virtualization is enabled
Boot settings: Only booting from disk
Disabled PXE option from Integrated Devices
Keyboard errors: do not report
Ensure that the disks that are installed are detected
- Note: if you are using a disk controller, your disks may be plugged into the controller, not directly into the sata ports that the server knows about. So - if bios doesn’t see disks, don’t panic, keep bios configured to “off” for the Sata ports.
Serial communications (to redirect output to serial console)
- Serial Communication: On with Console Redirection via COM1
- Redirection after boot: disabled

TODO: see BMC config to authorize SOL over SSH.

Hardware RAID

On DELL server, hit CTRL+R on demand.

Recent DELL servers integrate a PERC H700 controller, now fully supported by the stable Debian installer.

Recommended RAID level: RAID 1, 5, 10, 50

Booting the installer

Several choices at this level:

burn the ISO on a CD
prepare a USB stick
attach the ISO as a virtual disk and map it (iDRAC)

In all case, you have to configure the boot order to ensure you will run the installer.

Debian installation

Once on the installer boot menu, select “Advanced options / Expert install”.

Using the expert installation has several advantages:

it authorizes to run the installer over SSH, which is a huge benefit to bypass keyboard layout issues that wan intervene as soon as you use any kind of java-applet based console (DRAC, iLO, Avocent DSView etc.)
it’s currently the only way to setup a GPT-based partitioning to permits partition size of more than 2.2 TB (which is not possible with classical BIOS MBR (msdos) as you can select a partition method based on parted instead of fdisk

Configuring the install process

Choose the language (“English”), the country (“other > Europe > Luxembourg”) and the default locale (US en_US.UTF-8)
Select the keyboard layout (type: “PC-style”, keymap: French). Note: this setting can be changed later once the installation continue over SSH
Detect and mount the CD-ROM (no specific PCMCIA settings)
Load installer components from CD: select at least the following elements:
- ai-console-setup-udeb and ai-kdb-chooser (delayed config of the keyboard)
- choose-mirror (choose the closest mirror for Debian packages)
- network-console (continue installation remotely using SSH)
- parted-udeb (manual partition based on GNU parted)
Detect and Configure the network (either by DHCP or manually depending on your configuration)
Now select “Continue the installation remotely using SSH”
- enter a remote installation password

Continuing the installation by SSH

Operating over SSH has several advantages:

no longer issues with the keymap
same interface in the terminal (thanks to ncurses)
copy-paste in a terminal is easier
you can easily run a separate shell (ash)
you have therefore a finer control on the steps operated (typically, they are more detailed and you can operate some of them such as the partionning of the disk “by hand” on a separate shell)
you can still access the logs (the one you can see on the fourth console (accessible by ALT+F4 – the installer run on the first console) as they are located in /var/log/syslog. If you want to see them in live, juste run in a shell:
```
     $> tail -f /var/log/syslog
```

So to continue the installation, you just have to connect by SSH to your node as the user installer and choose to run the Installer.

Here is the sequential order of operations to perform:

Choose a mirror of the Debian archive: Select the appropriate mirror
Set up users and passwords:
- Enable shadow passwords
- DO NOT ALLOW login as root
- Create a local user for further administration tasks (i.e. he will be part of the sudo group)
Configure the clock, typically by NTP
Detect disks
Partition disks: prefer a manual partitioning and select the device to partition.
- if your asked to create a new empty partition table on this device, accept and select gpt as type of partition table (again, it permits partitions of capacity that exceed the limit of 2.2TB inherent to classical BIOS MBR partition).
- you should now have a single area on your device labelled “FREE SPACE”
For the partitioning in itself, proceed as follows:
- select the “FREE SPACE” area to “Create a new partition” of size “2MB” (at the “Beginning” of the available space). Use the following partition settings in this case:
```
Name:           biosboot      
Use as:         Reserved BIOS boot area  
Bootable flag:  off 
```
This partition is required for Grub to interface GPT
- now create an ext3 partition for /boot (avoid to put it on a LVM partition as this requires the lvm module to be available to reach your kernel on boot, which can pause problem on some configuration. On the contrary, ext2 partitions (and by extension the journaled version ext3) are always supported.). Select the “FREE SPACE” area to “Create a new partition” of size “1 GB” (at the “Beginning” of the available space). Use the following partition settings in this case:
```
Name:             boot
Use as:           Ext3 journaling file system
Mount point:      /boot  
Mount options:    defaults
Label:            none
Reserved blocks:  1% 
Typical usage:    standard
Bootable flag:    on
```
the rest of the space should be configured as an LVM partition. Alway let some free space (1% of the total capacity typically) at the end for further usage.

Eventually, if you plan to install Xen, you may want to rely on two LVM Volume Groups:
1. one for the system of the dom0, labeled lvm_dom0 (plan a partition size of around 20-30 GB)
2. one for the guests domU images, labeled lvm_domU (use typically 99% of the rest of the space)
In all cases, to create an LVM partition, select the “FREE SPACE” area to “Create a new partition” for the expected size at the “Beginning” of the available space. Here is an example of the final state of the partition:
```
 #1     2.0 MB     f  biosgrub    biosboot 
 #2     1.0 GB  B  f  ext3        boot      /boot 
 #3    30.0 GB     K  lvm         lvm_dom0 
 #4     1.7 TB     K  lvm         lvm_domU
        17.7 GB        FREE SPACE 
```

Now select “Configure the Logical Volume Manager” to setup LVM.

Accept to “Write the changes to disks and configure LVM”
Create volume group. Ex: vg_hostname_0 or vg_hostname_dom0
Create logical volume (on top of the previous volume group) for each part of the system (swap, root, var, tmp and eventually home and data for the remaining space).

When you have finished this setup, you will have to precise manually the configuration of each logical volume. Always choose to “Erase data on this partition” before finishing the configuration of a given partition. Here is a typical configuration:

	LV Name		Size 		Filesystem	 Mount point	Reserved Block
	---------------------------------------------------------------
	swap		<2*RAM>GB	swap	
	root		6 GB		ext4		/				1%	
	var			4 GB		ext4		/var			0%
	tmp			4 GB		ext4		/tmp			0%
	data		<rest> 		ext4 		/data			0%

Once everything is finished, you should of course accept to “Write the changes to disks”.

You can now continue the installation

Install the base system. Select the kernel linux-image-2.6-amd64 and include all available drivers in the initrd.
Configure the package manager. In particular, “ Use a network mirror” and “Use non-free software”. Include also the security and volatile updates.
Select and install software. Refuse the popularity contest and do not configure mandb to be installed ‘setuid man’. In the “Software selection” panel, you probably want to have only a SSH server in addition to the Standard system utilities.
Install the GRUB boot loader on a hard disk (confirm to “Install the GRUB boot loader to the master boot record”).

If you install the system on brand new disks (or devices) where Grub has never been installed before, then for some reason this step will probably fail. See the following section “FAQ: Dealing with Boot loader install error” for an hint at this level. * Finish the installation (set the system clock to UTC) and reboot

That’s all folks ;) You can now personalize the installed system.

Post install

Once logged on your newly installed system, run as root:

$> apt-get install vim subversion git-core sudo bash-completion nmap iotop dnsutils figlet

Installing exim4-* packages on squeeze illustrates this bug:

 	[...]
	Setting up exim4-config (4.69-9+lenny4) ...
	dpkg-statoverride: syntax error: unknown group 'Debian-exim' in statoverride file Quoting the post:

Ah-ha - I built this system (it’s a xenu) using xen-utils - it looks like this was a xen-create-image bug where it creates users but not groups - #404521

To correct it, create the Debian-exim as follows (adapt the gid accordingly)

 	$> addgroup --gid 107 --force-badname Debian-exim
	$> apt-get install

Edit /etc/vim/vimrc to uncomment (remove the " character) the line syntax on.

Protect access to the root account:

$> chmod 700 /root

[eventually] sudo configuration

Update the sudo configuration for the user localuser to prevent this user to enter the password, add the line (after the %sudo definition):

localuser	ALL=(ALL)	NOPASSWD:ALL

Mail configuration

You probably want to be notified by mail of anything that can happen on your system.At least install apticron as follows:

$> apt-get install apticron

Now proceed to the configuration of exim4:

Exim4 configuration (run dpkg-reconfigure exim4-config)
- Config: mail sent by smarthost; no local mail
- Name: put the full hostname of the system hostname.domain
- IP-addresses to listen on for incoming SMTP connections: 127.0.0.1 ; ::1
- Other destinations for which mail is accepted: hostname.domain
- Visible domain name for local users: hostname.domain
- IP address or host name of the outgoing smarthost: smtp.uni.lu
- Keep number of DNS-queries minimal (Dial-on-Demand)? No
- Split configuration: No
Update /etc/aliases (“root: Sebastien.Varrette@uni.lu”) and run newaliases
Test the mail configuration:

echo "Test from `hostname`" | mail -s "Test" root

Disable the root account

If you followed the previous steps at the installation of the system, nothing special should be done here.

Otherwise:

to disable the root account, run sudo passwd -l root
to re-enable the root account, run sudo passwd -u root

Update the motd (message of the day)

Update /etc/motd.tail and run

/etc/init.d/bootlogs (on Squeeze).
/etc/init.d/bootmisc.sh on Lenny

Update SSH configuration

Edit /etc/ssh/sshd_config (“Port 8022”, “PasswordAuthentication no”, “PermitRootLogin no”)
Edit /etc/ssh/ssh_config (“HashKnownHosts no”)

BASH configuration

Update the bash configuration for root and localuser - cf my github config:

boot message logs

Activate boot logs by having in /etc/default/bootlogd

BOOTLOGD_ENABLE=Yes

Protection against SSH attacks

$> apt-get install denyhosts logcheck

TODO: finalize rules etc.

Detect rootkits

$> apt-get install rkhunter

Note: in case of too much false positive, edit /etc/rkhunter.conf and add:

ALLOWHIDDENDIR=/dev/.udev 
ALLOWHIDDENDIR=/dev/.static

Iptables firewall

Setup firewall for this host (typically based on iptables). You can use my personal script available on Github

Typical installation:

$> git clone git://github.com/Falkor/init.d_scripts.git 
$> cd init.d_scripts/firewall
$> cp default/firewall /etc/default
$> ln -s `pwd`/init.d/firewall /etc/init.d/firewall
$> update-rc.d firewall defaults 98

No you can specialize the behavior of the script (especially the ports to be opened) in /etc/default/firewall.

FAQ

Dealing with Boot loader install error

Encounter error on the step “Grub boot loader installation” when proceeding on brand new disks which MBR as never been initialized. You’ll probably get the error

grub-install /dev/sdX failed !

So it should be done by hand on a separate shell. If you take a look at the logs (in /var/log/syslog), you will see that the error comes from an issue linked to the absence of /dev on the target system (in /target).

To solve the issue:

save the /dev directory of the target system
```
 mv /target/dev /target/dev.old
```
bind the actual /dev (of the installer) on the target system
```
 mount --bind /dev /target/dev
```
chroot on the target system
```
 chroot /target /bin/bash
```
install grub on the appropriate partition
```
 grub-install /dev/sdX
```

On Reboot: Busybox initramfs prompt and “Unable to find LVM volume `vg_hostname_dom0`”

For some reason, on boot, the system fails sometimes to find the volume group containing the root system. This is reported here for instance. This is probably due to a delay in the initialization of the LVM and the configuration of LVM volumes. Following this discussion, it appears that there is a race condition error in initramfs.

If things goes bad, you obtain the message

Unable to find LVM volume <...>

followed that the Busybox initramfs prompt.

Resolution

Once on the initramfs shell, run (replace vg_hostname_dom0 with the volume group containing your root system.)

   vgchange -a y vg_hostname_dom0

exit the shell and press Enter to continue.

Then, you’ll have to apply the patch lvm2_wait-lvm.patch provided in this bug report, i.e. :

backup the script /usr/share/initramfs-tools/scripts/local-top/lvm2
apply the patch from mentioned bug report
```
 patch -p0 < lvm2_wait-lvm.patch
```
run update-initramfs -u

HPC @ Uni.lu

Generic Debian Installation Note

Prepare the Installer ISO

Debian installer ISO modification (Ressource)

Separate ISO containing the missing firmware (Ressource)

Console access

DELL DRAC

DRAC Console redirection over SSH:

HP iLO

Preliminary setup: BIOS settings and Hardware RAID

BIOS setup

Hardware RAID

Booting the installer

Debian installation

Configuring the install process

Continuing the installation by SSH

Post install

[eventually] sudo configuration

Mail configuration

Disable the root account

Update the motd (message of the day)

Update SSH configuration

BASH configuration

boot message logs

Protection against SSH attacks

Detect rootkits

Iptables firewall

FAQ

Dealing with Boot loader install error

On Reboot: Busybox initramfs prompt and “Unable to find LVM volume `vg_hostname_dom0`”

Recent Posts

GitHub Repos

Prepare the Installer ISO

Debian installer ISO modification (Ressource)

Separate ISO containing the missing firmware (Ressource)

Console access

DELL DRAC

DRAC Console redirection over SSH:

HP iLO

Preliminary setup: BIOS settings and Hardware RAID

BIOS setup

Hardware RAID

Booting the installer

Debian installation

Configuring the install process

Continuing the installation by SSH

Post install

[eventually] sudo configuration

Mail configuration

Disable the root account

Update the motd (message of the day)

Update SSH configuration

BASH configuration

boot message logs

Protection against SSH attacks

Detect rootkits

Iptables firewall

FAQ

Dealing with Boot loader install error

On Reboot: Busybox initramfs prompt and “Unable to find LVM volume vg_hostname_dom0”

Recent Posts

GitHub Repos

On Reboot: Busybox initramfs prompt and “Unable to find LVM volume `vg_hostname_dom0`”