28 Feb 2014

Creating A Flexible OpenBSD USB Installer

PREAMBLE

Chris Cappuccio recently submitted to Undeadly his recipe for creating a bootable USB flash disk OpenBSD installer for the amd64 and i386 architectures. More accurately, he gave instructions for creating a bootable disk image using the install.iso file as a base. While there is utility in this, I prefer my particular method for creating a USB installer, and so I am sharing the process in this article.

My method for preparing a USB installer involves creating a small, bootable OpenBSD FFS partition large enough to hold several RAM disk kernels (bsd.rd files). The remainder of the USB disk is allocated to the lowest common denominator universal FAT32 format, which optionally houses the OpenBSD installation sets.

This method is more flexible than others for a number of reasons:
  • Most of the USB disk is partitioned in FAT32, giving it cross-platform appeal. Some other methods partition the entire USB device exclusively with OpenBSD's Fast File System (FFS);
  • Disk preparation only needs to be performed once (barring major changes to OpenBSD). To update to the latest release the user simply replaces the existing installation files with the latest files;
  • It is possible to update this USB installer using only the USB installer itself. This can be handy if the user does not have immediate access to an OpenBSD system and has not yet updated to the latest release; and
  • It is possible to carry around multiple releases on a single USB disk, and the user can even mix i386 and amd64 architectures.

I have detailed the steps to create such an installer below. I've been purposely verbose in my instructions to ensure that they are easily followed by even the novice OpenBSD user (albeit one familiar with the installation process).

NOTE 1: Any existing data on the drive will be lost.

NOTE 2: The bsd.rd file has nearly doubled in size since amd64 was first included in OpenBSD. Assuming that it will continue to grow at the same rate, an OpenBSD partition of 50 MB should be sufficient to hold at least two bsd.rd files for a significant number of subsequent releases. While not necessary in most instances, some users may want to create a larger OpenBSD partition to comfortably allow for even more bsd.rd files into the foreseeable future.


DIRECTIONS

1. Prepare the USB flash drive using fdisk, disklabel, newfs, and installboot.

This step only needs to be performed once to prepare the USB installer disk layout. Upgrades of an existing USB installer should begin at step 2.

Perform the following commands with the USB disk plugged in, assuming that the disk attaches at sd3:
# fdisk -e sd3
fdisk :*1> reinit
fdisk :*1> edit 3
Partition id ('0' to disable) [0 - FF]: [A6] (? for help)
Do you wish to edit in CHS mode? [n]
Partition offset: [64]
Partition size: [7871786] 102400
fdisk :*1> edit 0
Partition id ('0' to disable) [0 - FF]: [0] (? for help) c
Do you wish to edit in CHS mode? [n]
Partition offset: [0] 102464
Partition size: [0] *
fdisk :*1> w
fdisk :*1> q
# disklabel -E sd3
> a a
offset: [64]
size: [102400]
FS type: [4.2BSD]
> w
> q
# newfs sd3a
# newfs_msdos sd3i
# mount /dev/sd3a /mnt
The commands to install the bootstrap if you are running OpenBSD 5.4 or earlier:
# cp /usr/mdec/boot /mnt
# cd /usr/mdec; ./installboot -v /mnt/boot biosboot sd3
If you are running anything above OpenBSD 5.4, then use this command instead:
# installboot -vr /mnt sd3 /usr/mdec/biosboot /usr/mdec/boot
The USB disk preparation is complete and should not be necessary in future upgrades.


2. Copy the desired bsd.rd files to the OpenBSD partition.

The user may only need the latest version of bsd.rd for a single architecture. In such a case, renaming the bsd.rd file to "bsd" will cause the USB disk to automatically boot that file. The following commands would apply to create a USB installer for only the 5.4 release of amd64 (use a closer FTP mirror if not located in Canada):
# cd /mnt
# ftp -a ftp.ca.openbsd.org:/pub/OpenBSD/5.4/amd64/bsd.rd
# mv bsd.rd bsd
# cd /
# umount /mnt
Alternatively, the user may choose to have more than one RAM disk available. Thanks to similarities between i386 and amd64, it is even possible to mix the two architectures on the same USB disk. The trick is to rename the individual bsd.rd files so that they are distinguishable by release and architecture. The next set of commands will place both the amd64 and i386 bsd.rd files for the 5.4 release on the USB disk:
# cd /mnt
# ftp -a ftp.ca.openbsd.org:/pub/OpenBSD/5.4/amd64/bsd.rd
# mv bsd.rd 5.4_amd64.bsd.rd
# ftp -a ftp.ca.openbsd.org:/pub/OpenBSD/5.4/i386/bsd.rd
# mv bsd.rd 5.4_i386.bsd.rd
# cd /
# umount /mnt
At this point the USB installer has everything needed to install OpenBSD using remote install sets. However, if the user wants to use installation sets located on the USB installer itself then step 3 should be followed.


3. Copy the corresponding installation sets to the FAT32 partition.

This step may be performed using any OS that writes to FAT32. The process is demonstrated below using OpenBSD.

First, prepare the folders that will hold the installation sets on the FAT32 partition. Assuming that the user requires the 5.4 release of the i386 and amd64 architectures:
# mount_msdos /dev/sd3i /mnt
# mkdir -p /mnt/5.4/amd64; mkdir /mnt/5.4/i386
If the user has ample free space on the FAT32 partition of the USB disk, then the entire installation folder for each release may be fetched:
# cd /mnt/5.4/amd64
# ftp -a ftp.ca.openbsd.org:/pub/OpenBSD/5.4/amd64/*
# cd ../i386
# ftp -a ftp.ca.openbsd.org:/pub/OpenBSD/5.4/i386/*
The various disk image and network boot files can be left out if the user needs to be more selective due to space limitations. The following commands will replicate the necessary files for a local disk installation:
# cd /mnt/5.4/amd64
# ftp -a ftp.ca.openbsd.org:/pub/OpenBSD/5.4/amd64/*.tgz
# ftp -a ftp.ca.openbsd.org:/pub/OpenBSD/5.4/amd64/bsd*
# ftp -a ftp.ca.openbsd.org:/pub/OpenBSD/5.4/amd64/INSTALL*
# ftp -a ftp.ca.openbsd.org:/pub/OpenBSD/5.4/amd64/SHA256*
# cd ../i386
# ftp -a ftp.ca.openbsd.org:/pub/OpenBSD/5.4/i386/*.tgz
# ftp -a ftp.ca.openbsd.org:/pub/OpenBSD/5.4/i386/bsd*
# ftp -a ftp.ca.openbsd.org:/pub/OpenBSD/5.4/i386/INSTALL*
# ftp -a ftp.ca.openbsd.org:/pub/OpenBSD/5.4/i386/SHA256*
Once the desired files have been copied to the USB installer, the user must unmount the FAT32 filesystem.
# cd /
# umount /mnt
The USB installer is now ready to be used.


4. Using the USB installer to perform an OpenBSD install.

If the user has prepared multiple bsd.rd files for booting, the user will need to specify at the boot prompt the RAM disk file to be booted. This is done at the boot prompt:
>> OpenBSD/amd64 BOOT 3.28
boot> boot /5.4_amd64.bsd.rd
If the user intends to use remote installation sets, the user may complete an installation as normal once the RAM disk has booted.

Alternatively, if the user wishes to use installation sets located on the FAT32 partition of the USB disk, the user simply chooses the FAT32 partition at the appropriate point in the installation script. The continuing assumption made here is that the disk continues to attach at sd3, but remember that this can change when we reboot or re-plug devices:
Location of sets? (disk ftp http or 'done') [ftp] disk
Is the disk partition already mounted? [no]
Available disks are: sd0 sd1 sd2 sd3.
Which one contains the install media? (or 'done') [sd0] sd3
  a:           102400               64  4.2BSD   2048 16384    1
  i:          7784384           102464   MSDOS
Available sd3 partitions are: a i.
Which one has the install sets? (or 'done') [a] i
Pathname to the sets? (or 'done') [5.4/amd64]
Assuming that the sets were stored in the directory hierarchy shown above, choosing the default path will bring up the familiar set selection portion of the installer.

The only problem I've ever run into appears to be an incompatibility in the way that OpenBSD reads long file names under FAT32. If the installation sets are retrieved using OpenBSD, as above, then this error will not be triggered. However, if the user copies the sets onto the USB disk using Windows 7 (and possibly other versions), then the following message might appear:
Pathname to the sets? (or 'done') [5.4/amd64]
INSTALL.amd64 not found. Use sets found here anyway? [no] yes
Choosing to proceed in spite of the error message has always worked fine for me. While I haven't read the actual install script, I suspect that confirming the existence of the file is merely a sanity check.


CONCLUSION

You now have a very flexible OpenBSD installer in your possession. Use it to spread the "Free, Functional, Secure" goodness throughout your networks!

Some may recall that I alluded in the preamble to the possibility of updating the USB installer using only the USB installer itself. One merely requires a PC capable of booting from USB, with a compatible network device, connected to a network serving DHCP addresses, and with access to the Internet. I realize that some novices may have trouble achieving this. I may be motivated to write a follow up piece with directions on how to perform such an update. I simply need to receive enough interest in the comments below.

11 comments :

  1. Nice Article I am going to install OpenBSD 5.5 this way. My net-book does not have optical media so this will work well.

    ReplyDelete
  2. Joe, no need for 5.5:
    "Disk images which can be written to a USB flash (miniroot55.fs and install55.fs) are now provided for i386 and amd64."
    See http://www.openbsd.org/55.html

    ReplyDelete
  3. Thanos, you are correct that the project now provides images, but you might be assuming that Joe only needs flash media. If you read my article and fully comprehend it you should understand why I used the word "flexible" in my title. The project supplied images are not equally flexible.

    Still, you've provided a good link for those who don't require the additional flexibility.

    ReplyDelete
  4. Marvellous instructions, thanks so much, this how-to worked without any problem. Great way to create flexible Architecture OpenBSD USB Installer. I only tried it for i386 excessively for fun. Thanks again. For my part the miniroot55.fs and install55.fs is the way to go for a quick install.

    What I've accounted with both miniroot55.fs and install55.fs is that during the installation process it warned me out: pckbcintr:nodev for slot 1 many times but I can still perform all the installation on the usb key. I've not had any good success running out Openbsd on a usb flash in the past. But will definitely try to play more with it soon. Nonetheless my transparent Openbsd firewall inside Soekris 5501 is performing as stellar for over 3 years now with CF card.

    ReplyDelete
  5. Thanks for the appreciation, Petit Nono. I use my method mostly to keep more than one version of the amd64 installer around (for testing), but I also keep around the latest i386 installer just in case I'm dealing with older hardware. I volunteer at the local community radio station, and one of their audio logging / music streaming servers is running on an older Celeron which is only 32 bit. (Yes, I've set up all their streaming audio infrastructure to run on OpenBSD: two local encoding servers that store 90 days of logs, plus a remote server that receives streams from the encoding servers and shares those mp3 and vorbis streams with the world.)

    With respect to the pckbc error that you are getting with the OpenBSD Project's miniroot55.fs and install55.fs, there are a couple of things you can test. First, if you don't use a PS/2 keyboard, your BIOS may be configured to emulate your USB keyboard as an older PS/2 keyboard. Disable any legacy keyboard emulation setting that might exist and try the installers again.

    Otherwise, if you are using a USB keyboard and not a PS/2, you could disable the pckbc device at the initial boot prompt, as follows:

    boot> -c
    UKC> disable pckbc
    UKC> quit

    I'm travelling right now and I can't test this myself, but I can't see any reason why it won't work.

    You really shouldn't get those errors with the OpenBSD official files, so it would be good of you to file a bug report with the OpenBSD project, too.

    On the other hand, another benefit of my method is that it isn't as susceptible to increases in the size of the OpenBSD flash drive installer files. Assume that you use the current 5.5 files to prepare a USB installer with the remainder of the flash drive allocated to other file systems. Later, the 5.6 or higher files are larger than the 5.5 files, in which case you will have to back up the non-OpenBSD portion of the flash drive and repartition everything from scratch. PITA.

    Of course, this is only a problem if your OpenBSD flash drive installer is used for other file systems. If you're using a flash drive exclusively for installing OpenBSD then it doesn't matter if later versions of the files are larger in size, because you simply overwrite your flash drive with the new files without any concern for what is being overwritten.

    ReplyDelete
  6. Your streaming audio infrastructure sounds amazing, well done. :) what speed is the Celeron and what kind of connection this radion station has? Are the mp3 streaming at 128 kbps or something? I'm so interesting to do this kind of setup for myself.

    For now I'm testing on lenovo x230. Amazing portable by the way. I will try to resort this pckbc warning as you mentioned earlier to me. I really appreciate your help :)

    I've so many usb flash drives, the limiting sizes is not a problem at all. I like to test as many with BSD. I simply love the way you are doing it. You get more control of what you do and it help to prevent failure too.

    Two or three time I've installed Openbsd via vitualbox, but two days ago, I tried Openbsd 5.5 but none of my several usb several keys were recognizable. Even though I'am using VirtualBox 4.3.12 for OS X hosts. So I came to try your way. I've done a similar way in the past which work great and went to install all the sets over the network. Here is the instructions I followed: http://undeadly.org/cgi?action=article&sid=20100404103735

    I tried Preboot eXecution Environment one day, it was a pain for me, but it did work.

    My firewall still run Open5.2, I'm trying to set the log file to an other server so I can disable as much services as I can.

    Have a nice trip :)

    I will report soon.

    ReplyDelete
  7. I tried your trick on boot and it was successful:

    boot> -c
    UKC> disable pckbc
    UKC> quit

    No more warning of pckbcintr: no dev for slot 1

    Thanks again. I will dig this up.

    ReplyDelete
  8. Glad to help.

    The Celeron is an old 2.0 GHz single core CPU, approximately 10 years old. I intend to post a future article about the radio station audio streaming setup, so keep an eye out for an announcement on the Google Plus OpenBSD community!

    ReplyDelete
  9. Thanks Breen for such interesting dedication of your part. I'm following you great work. :-)

    ReplyDelete
  10. Thank you Breen. I frequently used a usb device to alternatively install OpenBSD or Redhat Ent Linux in my Thinkpad laptops, now I can keep multiple versions of OpenBSD in the same usb drive and use the rest of the disk for other files as well. Earlier creating a OpenBSD partition meant the device could not be used anywhere else.

    ReplyDelete