I needed to prepare 30 PCs with Linux (Ubuntu was chosen) as a base OS for data entry application written in Java. PCs were equipped with two hard disks, intended to work as a RAID1 matrix. I created the base installation that was supposed to be cloned.
I thought that I will have to use advanced tools, but I didn't have to, thanks to the beauty of KISS (Keep It Simple, Stupid) I could do that with set of standard system utilities. Create partitions with sfdisk, bind them together with mdadm, create a file system with mkreiserfs, mount it, get the system files with netcat, pipe them to tar and populate the RAID matrix, chroot into the new partition and install lilo boot loader. That's it. Put all those commands together in a shell script, boot Ubuntu Live CD and run the script. Couldn't be simpler.
Why lilo, you ask. Well, when it comes to RAID matrices, GRUB sucks! It does understand file systems, but it doesn't understand RAID and it causes it to hang on "GRUB loading stage 2". News archives show several guys with the same problem with no straight answers, so I decided to bail out from GRUB and go for lilo which served me well in many RAID installations. Lilo doesn't understand file systems, doesn't this and can't that, but at the end it can boot my RAID while GRUB can't. Worse is better.
I tried to optimize the cloning process. I didn't transfer whole 40GB disk images, I transferred only the system files from a single 840MB gzipped tar file. File was served from my laptop. I expected the network or the disk reading speed to be the bottlenecks, but it was nothing like that. The bottleneck was in Ubuntu Live CD booting duration and the number of available monitors. The booting took few minutes (multiply it by 30 computers!) and it required a monitor to be connected, because Live CD has to detect the monitor model to set up the display properly. Also physical unpacking, setting up and packing back took us some time, but there was no way to avoid that.
Ubuntu Live CD had one flaw. It wouldn't close the system properly. After selecting Shutdown, CD drive would buzz and the screen would went black, but the machine wouldn't reboot. No other way than holding the power button for 10 seconds. After such reboot the RAID matrix was unclean and had to be synchronized, which took about 20 minutes.
The file copying was the smallest part of the process and was took under 15 minutes per one computer.
This is the largest number of Linux computers I've ever been installing at one time.