EDG LCFGng Server Installation using RedHat 7.3

C. Aiftimiei, E. Ferro, C. Loomis

22 September 2003

1 Introduction

Here is a procedure for setting up an LCFGng installation server for the first time, using some shell scripts and package lists to reduce the amount of manual work involved. This server must be installed with Red Hat 7.3 and it will be able to install clients with Red Hat 7.3 via diskette or PXE.

The main steps of the LCFGng server installation procedure are:

2 Prerequisites

Before beginning the server installation, collect the following information: Having the information available before starting will make the installation and testing proceed more smoothly.

3 Install Red Hat 7.3 release

Some hints for the installation of Red Hat 7.3:

4 Set up SSH and NTP

Edit /etc/ntp.conf to configure the NTP daemon and set your time server: 
server  <YourNTPServer>
To setup NTP and SSH: 
/usr/sbin/ntpdate <YourNTPServer>
/sbin/chkconfig --level 2345 ntpd on
/sbin/chkconfig --level 2345 sshd on
/sbin/service ntpd start
/sbin/service sshd start

5 Install edg-populate-serverng and updaterpms-static-server packages

These two packages should be installed. Get the latest version of updaterpms-static-server from: while for edg-populate-serverng:

6 Download the EDG machine profiles and RPM lists

EDG manages the machine configuration files and associated RPM lists for each release of the EDG software through the edg-release module in CVS. It also contains some utility scripts to make use of the LCFGng server easier.

CVS tags on this module define the EDG release. Production releases have tags of the form: v2_0_1. Testing tags have a time stamp like t20030910_1145. Note that the testing tags and the head version of edg-release are not guaranteed to describe a working release! Use a production tag if you plan on joining the EDG application testbed.

Complete instructions for CVS access are available, but to get started quickly put the following into the " /.ssh/config" file for root: 

Host datagrid.in2p3.fr
Protocol 1
Port 2222
PasswordAuthentication yes
RSAAuthentication no
Compression no
User anoncvs
After doing so, the following command will checkout the edg-release module: 
export CVS_RSH=`which ssh`
cvs -d anoncvs@datagrid.in2p3.fr:/cvs co -r v2_0_0 edg-release
where "v2_0_0" is replaced by the version you want to checkout.

The machine profiles are in the ng_source subdirectory, the RPM lists in the ng_rpmlist subdirectory, and useful scripts in ng_updaterep.

Copy the rpm lists to the directory /opt/local/linux/7.3/rpmcfg/ and the machine profiles to /var/obj/conf/server/source (also copying the examples subdirectory and contents).

7 Populate the RPM repository with updateRPMcache

updateRPMcache is a perl script to populate the RPM repository in the LCFGng server, downloading only those RPMs which are necessary to satisfy the machine configuration files passed on the command line. Its default configuration file is /etc/updaterep.conf.

Both the script and the configuration file are stored at in the ng_updaterep directory in the edt-release module downloaded at the previous step.

Select the CVS tag corresponding to the EDG version to install, download it and replace the existing /etc/updaterep.conf.

The command options are:

updateRPMCache.pl [-help] [-clean, -noclean] [-base] [-days] [-rpmcfg] <configuration files>

where:

To download the RPMs needed to install the LCFGng server: 

./updateRPMCache.pl --rpmcfg /opt/local/linux/7.3/rpmcfg/ngserver73-rpm
./updateRPMCache.pl --rpmcfg /opt/local/linux/7.3/rpmcfg/nginstallroot73-rpm

To download the RPMs needed by your gridelements run: 

./updateRPMCache.pl /var/obj/conf/server/source/<hostname>

Type ./updateRPMCache.pl -help for more details. The packages will be put in /opt/local/linux/7.3/RPMS/.

8 Installation/upgrade of server packages

The most of the work is done by this script: 

/opt/local/linux/utils/installpackages-ngserver73
It will install additional packages required (they are listed in /opt/local/linux/7.3/rpmcfg/ngserver73-rpm) and server-side part of components. Please note that in LCFGng they are called 
edg-lcfg-<component>-defaults-s[12]
or 
lcfg-<component>-defaults-s[12]
while in the old LCFG they were called obj-<component>-server.

To update in the future all the packages *-defaults-*.rpm you can use the script checkServerRPMs.pl available in the same url of updateRPMCache.pl.

9 Build nginstallroot file system for 7.3

To build the installroot required by 7.3 installations run this script: 

/opt/local/linux/utils/create-nginstallroot-rh73
In a Red Hat 7.3 LCFGng you cannot create the installroot for 6.2 (so the script create-nginstallroot-rh62 does not work). Please note that during rpm installation process you can see some error messages like ``Cannot create temp file'', ``execution of %post scriptlet failed''; do not worry about them. The nginstallroot is placed in /opt/local/linux/nginstallroot/7.3. The default installparams file assumes a US keyboard and the CET time zone; if you want to change to change this: 
cd /opt/local/linux/nginstallroot/7.3/etc
cp installparams.default installparams
vi installparams

10 Set up DHCP and NFS servers

You can find an example of /etc/dhcpd.conf in /etc/dhcpd.conf.ngexample. The user-class option contains the URL of the installation server. Update this file as needed by your local network configuration adding all your nodes.

Start and configure DHCP daemon: 

/sbin/service dhcpd start
/sbin/chkconfig --level 2345 dhcpd on

For NFS configuration add these lines to /etc/exports (they are present also in /etc/exports.ngexample73): 

/opt/local/linux/7.3                *(ro,no_root_squash)
/opt/local/linux/nginstallroot/7.3  *(ro,no_root_squash)
Start and configure NFS daemons: 
/sbin/service nfs start
/sbin/service nfslock start
/sbin/chkconfig --level 345 nfs on
/sbin/chkconfig --level 345 nfslock on

11 Set up HTTP server

Stop the web server (Apache) if it is running: 

/sbin/service httpd stop

Edit its configuration file /etc/httpd/conf/httpd.conf, so that its root directory points to LCFGng profile and status.

Please note that you can use the example /etc/httpd/conf/httpd.conf.ngexample73.

The lines about cgi are needed if you'll use cgi scripts to manage the PXE client installation using pxelinux: these scripts are provided by edg-pxelinux-supp package for installations via PXE.

Replace: By:
DocumentRoot "/var/www/html" DocumentRoot "/var/obj/conf/server/web"
<Directory "/var/www/html"> <Directory "/var/obj/conf/server/web">
Alias /icons/ "/var/www/icons/" #Alias /icons/ "/home/httpd/icons/"
#AddHandler cgi-script .cgi AddHandler cgi-script .cgi
<Directory "/var/www/cgi-bin/"> <Directory "/var/obj/conf/server/web/install/">
Options None Options ExecCGI

If you want to use PXE installation, add the following lines for security reasons, to protect the install.cgi script. This script provides a basic interactive interface to select which nodes should be installed by changing the pxelinux configuration files to be used (install the machine/boot from the hard disk): 

<Files "install.cgi">
    AuthUserFile /etc/httpd/.htpasswd
    AuthType BASIC
    AuthName "User and Password needed"
    require user lcfgng
</files>
In order to create the password file you need to issue the following command: 
htpasswd -c /etc/httpd/.htpasswd lcfgng
Of course you can change the location of the password file, the user name or setup a better protection for the cgi script. Another script (ack.cgi) is called by the clients at the end of the installation to restore the boot from the harddisk.

Restart and enable the web server: 

/sbin/service httpd start
/sbin/chkconfig --level 345 httpd on

12 Compile and update LCFGng profiles

The source files and RPM lists have been already downloaded in previous chapters.

There should be some header files (*.h), and a file for each LCFGng client computer testbed004 (its name is the unqualified host name of the client).

The file site-cfg.h must be configured with all configuration specific of your site.

If you have SCSI controller, look at the beginning of redhat73-cfg.h and select the corresponding kernel module.

Start and enable mkxprof daemon on the LCFGng server with the command: 

/sbin/service mkxprofd start
/sbin/chkconfig --level 345 mkxprofd on

This will run mkxprof as daemon but without building the profiles. For building the XML files profiles you have to reload it: 

/sbin/service mkxprofd reload

You have to run this command every time you change the configuration files and you want to deploy them to the clients. In LCFGng the result of the XML profiles creation is available only via web on: 

http://<LCFGServer>/status/

Once all the client profiles are compiled, you can populate the RPM repository with the client RPMs using updateRPMcache: 

./updateRPMcache.pl /var/obj/conf/server/source/grid*

where your client machines are called grid01, grid02 and grid04.

13 Installing a client using a diskette

These steps are needed only if you are installing your nodes via a floppy disk, to install via PXE see the next paragraph. for Red Hat 7.3.1: 

cd /tmp
wget http://datagrid.in2p3.fr/distribution/datagrid/wp4/edg-lcfg/packages/\
external/diskettes/RH73/bootdisk_rh73_04122002.img
dd if=bootdisk_rh73_22112002.img of=/dev/fd0 bs=1024

14 Installing a client node via PXE

The installation via PXE is done with pxelinux. The first step is to put the pxelinux loader (provided by the package syslinux, already installed with installpackages-ngserver73) in the /tftpboot directory : 

cp /usr/lib/syslinux/pxelinux.0 /tftpboot/

TFTP is used by PXE NIC to download the pxelinux.0 loader and then by this one to download its configuration file (from /tftpboot/pxelinux.cfg) and a kernel (from /tftpboot/kernel). To enable it modify /etc/xinetd.d/tftp by changing the "disable" parameter to "no" and reload the xinetd configuration: 

vi /etc/xinetd.d/tftp
/sbin/service xinetd restart

With a browser choose the installation type for your nodes: 

http://<LCFGngServer>/install/install.cgi

Enable the boot via PXE on the machines. To check if the process is working put a look to /var/log/messages on the LCFGng server to see the DHCP requests and the mount. In case of process with the download of the XML profiles put a look on /var/log/httpd/access_log

15 Next steps

The procedure described in this note sets up an LCFGng server which can be used to install a DataGrid testbed but it does not cover higher levels of DataGrid software, nor does it deal with the maintenance of LCFGng server software over time. For an installation guide of the testbed see:


http://www.lnl.infn.it/datagrid/wp4-install

16 History and acknowledgements

LCFG and LCFGng were developed at Edinburgh University, where they are used to manage several hundred computers. The principal authors are Paul Anderson and Alastair Scobie.

LCFG and LCFGng have been adapted for use in the European DataGrid by staff from INFN and CERN, including Massimo Biasotto, German Cancio, Andrea Chierici, Jan Iven, Mohammad Jaudet, Maite Barroso Lopez, Louis Poncet and Marco Serra. Other EDG collaborators have written DataGrid specific LCFGng components.

The installation of a LCFGng server on a Red Hat 7.3 was done for the first time by Emanuale Leonardi.

17 References

The documentation about WP4 and LCFGng:

http://datagrid.in2p3.fr/distribution/datagrid/wp4/edg-lcfg/documentation/

Here you can find more informations about the differences between LCFG and LCFGng, documentation for each component, ``Writing LCFG Components'' by Paul Anderson of Edinburgh University.


The EDG LCFGng package repository:

http://datagrid.in2p3.fr/distribution/datagrid/wp4/edg-lcfg

The CVSweb access to EDG LCFGng components:

http://datagrid.in2p3.fr/cgi-bin/cvsweb.cgi/fabric_mgt/edg-lcfg

The EDG configuration and rpmlist files:

http://datagrid.in2p3.fr/cgi-bin/cvsweb.cgi/edg-release/ng_source

http://datagrid.in2p3.fr/cgi-bin/cvsweb.cgi/edg-release/ng_rpmlist