In response to an article in the OpenBSD Journal (and inspired by the "RAM Raid" article by Bo Adler in SysAdmin 6/2002), i worked my way through setting up an OpenBSD system running in an mfs filesystem. This how-to outlines how i did it.
How To Use RAIDframe In OpenBSD To Get A Full System Running Only In RAM
by francisco roque last modified 08/19/2002
Table Of Contents
1. Introduction
2. Necessary Equipment
3. Instructions
3.1 Installing the Base System
3.2 Compiling New Kernel
3.3 Initializing the Raid Device
3.4 Setting up the Raid Device
3.5 Adding the Memory File System
3.6 Final Steps
4. Benchmarks
5. Conclusion
6. References
7. Author, Copyright, License
1. Introduction
This document will walk you through using up all your RAM as a filesystem
instead of using hard drive space. It is done under OpenBSD 3.1 i386 but
should be easily translatable to other platforms. The same concepts should
also carry over to other OS's (i.e. this would probably work with linux
too).
Though you will still need a drive to boot from and to hold the original
copy of your data, when the machine is finally up it will be running solely
in RAM. The benefits of using RAIDframe over using a ramdisk in kernel is
that ramdisks are limited in their size but the RAIDframe can be of any
size up to the total of your RAM and swap space. Additionally, using
RAIDframe requires only 2 changed lines in yuor kernel config, and ramdisks
are a littel trickier to set up.
2. Necessary Equipment
A computer with 160 megs RAM, 500 megs hd space.
An OpenBSD install on which you can compile a new kernel for your platform.
It is assumed that you know how to read man pages, compile kernels, edit
files, and other simple unix tasks.
3. Instructions
3.1 Installing the Base System
Install OpenBSD. For minimization of space, install only the following
packages:
base31.tgz etc31.tgz bsd
The techniques outlined in this doc will work for full installs too, but
i am assuming most people do not have spare workstations with 2 gb RAM
that they can try this on. An install demanding 160 mb RAM seems more
feasible. Of course, you could do a full install using only 160 mb RAM
and still follow all these instructions (increasing fs sizes where
appropriate) provided you have enough swap, but that would eliminate the
benefit of a memory only system.
It will be handy to have another machine on which you can view OpenBSD man
pages. Remember that man pages are also available via the web.
Use the disk space as follows:
a - 128 megs
d - 220000 cylinders, fs type RAID
b - remainder
For example, the disklabel might look like:
# using MBR partition 3: type A6 off 63 (0x3f) size 2048193 (0x1f40c1)
# /dev/wd0c:
type: ESDI
disk: ESDI/IDE disk
label: VMware Virtual I
flags:
bytes/sector: 512
sectors/track: 63
tracks/cylinder: 15
sectors/cylinder: 945
cylinders: 2168
total sectors: 2048760
rpm: 3600
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0 # microseconds
track-to-track seek: 0 # microseconds
drivedata: 0
16 partitions:
# size offset fstype [fsize bsize cpg]
a: 261702 63 4.2BSD 1024 8192 16 # (Cyl. 0*- 276)
b: 1566491 481765 swap # (Cyl. 509*- 2167*)
c: 2048760 0 unused 0 0 # (Cyl. 0 - 2167)
d: 220000 261765 RAID # (Cyl. 277 - 509*)
3.2 Compiling New Kernel
On a separate machine, compile a new kernel with the following additions
to the kernel config:
pseudo-device raid 4 # RAIDframe disk driver
option RAID_AUTOCONFIG
The first line provides us with the raid drivers. The second lets us
configure the raid devices at boot time from the kernel as well as
allowing us to use the raid device as our root device.
On the machine we're setting up, place the new kernel in /bsd but
remember to keep a copy of the old kernel as /bsd.old, just in case.
Boot into the new kernel.
If you have trouble with the new kernel then at the boot> prompt you
can type in:
boot> boot hd0a:/bsd.old
this will boot up the system with the old kernel. Fix the problems with
the new kernel and boot into it.
3.3 Initializing the Raid Device
Create /etc/raid0.conf with the following data:
#BEGIN /etc/raid0.conf
START array
# numRow numCol numSpare
1 2 0
START disks
/dev/wd0d
/dev/wd0p
START layout
# sectPerSU SUsPerParityUnit SUsPerReconUnit RAID_level_1
128 1 1 1
START queue
fifo 100
#END /etc/raid0.conf
This file is used to configure the raid device. We are creating a raid
device mirrored across 2 disks, one of which does not exist. This will
give us errors but is ok.
Run the following commands to initialize the raid device:
raidctl -C /etc/raid0.conf raid0
raidctl -I 1234 raid0
raidctl -iv raid0
raidctl -A root raid0
3.4 Setting up the Raid Device
Now we can use /dev/raid0c as if it were any other disk, so we proceed
to disklabel(8), newfs(8), mount(8) and place data on the disk:
disklabel -R /dev/raid0c /root/raid0c-disklabel
newfs /dev/rraid0c
mkdir /mnt/raid
mount /dev/raid0a /mnt/raid
The file /root/raid0c-disklabel contains the following:
# /dev/raid0c:
type: RAID
disk: raid
label: fictitious
flags:
bytes/sector: 512
sectors/track: 128
tracks/cylinder: 8
sectors/cylinder: 1024
cylinders: 214
total sectors: 219904
rpm: 3600
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0 # microseconds
track-to-track seek: 0 # microseconds
drivedata: 0
16 partitions:
# size offset fstype [fsize bsize cpg]
a: 219904 0 4.2BSD 1024 8192 16 # (Cyl. 0 - 214*)
c: 219904 0 unused 0 0 # (Cyl. 0 - 214*)
Place data on the raid device:
cd /
tar -cXf - . | tar -xpf - -C /mnt/raid
It will take a little bit for that tar command to complete.
Finally, edit /mnt/raid/etc/fstab to change /dev/wd0a to /dev/raid0a
You will also want to add a line to /mnt/raid/etc/fstab indicating that
swap is on /dev/wd0b. The following line should work:
/dev/wd0b none swap sw 0 0
Reboot now. The system will come up using the raid device as the root
device.
It will complain about the raid device being corrupt. That's ok.
3.5 Adding the Memory File System
Now we create the memory filesystem, place a file on it, link that file
up to a vnd device, and label that device.
mkdir /mnt/mfs
mount_mfs -s 234000 swap /mnt/mfs
dd of=/mnt/mfs/myfs if=/dev/zero count=220000
vnconfig -c svnd0 /mnt/mfs/myfs
disklabel -R /dev/svnd0c /root/svnd0c-disklabel
The file /root/raid0c-disklabel contains the following:
# /dev/svnd0c:
type: SCSI
disk: vnd device
label: fictitious
flags:
bytes/sector: 512
sectors/track: 100
tracks/cylinder: 1
sectors/cylinder: 100
cylinders: 2200
total sectors: 220000
rpm: 3600
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0 # microseconds
track-to-track seek: 0 # microseconds
drivedata: 0
16 partitions:
# size offset fstype [fsize bsize cpg]
c: 220000 0 unused 0 0 # (Cyl. 0 - 2199)
d: 220000 0 RAID # (Cyl. 0 - 2199)
Last steps, we attach the memory filesystem as a spare to the raid device,
rebuild the raid device, and then dettach the hard drive from the raid
device.
raidctl -a /dev/svnd0d raid0
raidctl -F component1 raid0
raidctl -f /dev/wd0d raid0
NOTE: The last step is dettaching the hard drive from the raid device. It
must be done after the recovery from the step before. You can check the
status of the recovery by running:
raidctl -S raid0
Additionally, if you want a system that kind of makes sense, don't get rid
of the hard drive. You will have benefits from the mfs side of the raid
while not losing data on powerdown.
3.6 Final Steps
Your computer is now running off of a raid device as its root filesystem.
The raid device consists of a vnode disk in an mfs file system and a
failed hard drive partition. That is, it should be using only the memory
portion of the raid device.
You may also wish to automate step 3.5. Add the following lines to your
/etc/rc.local file and then touch /memonly. To not run in mmemory only
mode, boot into single user mode and remove the /memonly file.
#mount mfs and use as spare dev for raid dev, recover onto the mfs
#dev, then remove the hd dev.
if [ -e /memonly ]; then
echo mounting mfs
mount_mfs -s 234000 swap /mnt/mfs
sleep 1
echo creating vnode
dd of=/mnt/mfs/myfs if=/dev/zero count=220000
vnconfig -c svnd0 /mnt/mfs/myfs
disklabel -R /dev/svnd0c /root/svnd0c-disklabel
echo adding/recovering vnode to raid
raidctl -a /dev/svnd0d raid0
raidctl -F component1 raid0
echo watching reconstruction
while [ `raidctl -S raid0 | grep Reconstruction | cut -f3 -d' '` != '100%' ]
do
raidctl -S raid0 | grep Reconstruction | cut -f3 -d' '
sleep 1
done
#removing the physical drive from our setup
raidctl -f /dev/wd0d raid0
fi
You should now go back, increase the filesystem sizes, and perform a full
install. You may also choose to create multiple partitions inside the
raid device. It might also be possible to use a cd instead of /dev/wd0c
Of course for best performance you will want a lot of RAM.
4. Benchmarks
The following benchmarks were performed on:
Sun Blade 100: UltraSparc IIe 500mhz, 1 20G 7200 RPM IDE disk, 2G RAM,
OpenBSD 3.1, GENERIC kernel plus the 2 aforementioned raid lines.
It was configured with a 900mb hd partition and/or a 900mb mfs partition.
4 runs were performed:
run 1: disk only, no raid
run 2: disk only, raid
run 3: disk+mfs, raid
run 4: mfs only, raid
Iozone was used to test IO speed. It was run as:
iozone -Ra -g 64m > filename
Filesize was limited to 64m because the test machine needed to be in
production soon. Unfortunately, this limited the tests in that buffering
most likely biased some tests.
Graphs:
Write,
Re-Write,
Read,
Re-Read,
Random Read,
Random Write,
Backward Read,
Record Re-Write,
Stride Read,
Fwrite,
Re-Fwrite,
Fread,
Re-Fread
Eventually i hope to get some benchmarks for an Intel system, 128 mb RAM.
5. Conclusion
You now have a workstation that can be booted into a mode in which
everything is running in memory. Compiles should be quicker since disk
access is really memory access, as should any other process which requires
a lot of disk access.
The drawback is that any changes will be lost on reboot, but you could
use raidctl to regularly sync changes to disk, or regulary copy data to
a separate machine. I should note that trying to reenable the physical
disk causes kernel panics.
A better option may be developing a kernel and ramdisk with mfs support, and
then creating mfs partitions for major partitions. However, creating kernel
ramdisk images is a more intensive process than simply adding two lines to a
kernel config.
6. References
disklabel(8), newfs(8), mount_mfs(8), raidctl(8), vnconfig(8), options(4),
boot(8), raid(4), mount(8), tar(1), vnd(4), zero(4), dd(1)
Iozone disk io benchmark - www.iozone.org
For more info on setting up mirror raid, please refer to this thread:
http://marc.theaimsgroup.com/?l=openbsd-misc&m=105064683521765&w=2
7. Author, Copyright, License
The author of this document is francisco roque, who can be found at
http://www.blackant.net/
This document is copyright francisco roque 2002.
This document is released under BSD style license and may be used in whole or
part for anything so long as this section remains intact.
The author takes no responsibility for damages incurred by following the
steps in this document, including but not limited to the loss of data or
damage to hardware. Good Luck.
