Saturday, January 15, 2011

OpenBSD - Compact Flash Media Installation w/Read-Only Configuration

The core "Installation Guide" (coming soon) is compliments of Lin.  He was able to compile the information from various sources and has proven it out multiple times through various installations.  

I recently had reason to install a Alix2D3 and followed his tutorial.   I strayed slightly from the instructions and will note it at the beginning along with a few oddities that I encountered. 


If you choose not to configure the CF Card as read-only be advised that Flash memory, regardless of format, is limited to a finite number of erase/write cycles for any "block," before that block can no longer be written to successfully.  In other words, the life expectancy of your system could be very short.  

I had initially setup a device in a read-write state intending to return at a later date to configure it as read-only.  This proved catastrophic as I exceeded the write cycles of the CF card in a rather short period of time.  You have been sufficiently warned...better to learn from others miss fortune.

Component List:
  • VMware Fusion - MacBook Pro
  • OpenBSD 4.8 ISO 
  • 2G CF Card
  • ALIX2D3 - PCEngines mainboard & Case
  • NULL Modem Cable 
  • USB -> Serial cable
  • Compact Flash Card Reader
Helpful Hints based on my most recent installation:
    Each time I partitioned and formatted the CF card without a reboot, I encountered write errors on the CF card and the installation failed.  If you experience this same issue, I found that partitioning followed by an immediate restart of the installation was the answer.  This may simply be a problem related to Fusion or the method I am using for installation.

    To minimize the installation footprint, I did not install any "x" packages.  If you do find something missing, it is a trivial matter to add at a later date.

    I opted to use a 9600 baud rate throughout instead of 19200.

    While using both "screen /dev/tty.usbserial 9600" and putty under windows to console to the Alix2D3 device, I experienced times where the output to the console was clean but it wouldn't accept any input from the keyboard.  This led me on a wild goose chase for a time but I was able to determine that this happens on both a windows and Mac OSX platform.  Resetting the terminal would not resolve the issue, it required that I move the usb -> serial cable to a different port.  No reset of the device was necessary.

    I also utilized another OpenBSD virtual machine to mount the CF Card after the installation to make the minor modifications necessary in fstab.  If you choose to make additional changes or symbolic links while mounted in this way, please use extreme caution as any mistake could lead to an unusable installation.

    If setting up as a firewall remember the following:
      • /etc/mygate
      • rc.conf (pf=YES)
      • sysctl.conf (ip forwarding)
      • pf.conf requires syntax changes for ver. 4.7 and later. 


    -----------------------------------------------
    Installing OpenBSD on CompactFlash
    ----------------------------------------------
    First you need to get a compactflash or other media installed with OpenBSD

    Download latest .iso image and save it on a box with vmware or other vm technology

    Create a virtual machine with no disk drives, and point the cdrom to the OpenBSD install .iso file.

    Boot from cd on the vm, thus entering the OpenBSD install process.

    When you get to the (I)nstall, (U)pgrade or (S)hell? prompt, connect/insert the usb device holding your intended media to the usb hub of the virtual machine.  OpenBSD should recognize the media correctly if you insert it after boot.  Otherwise you might get errors asking about the geometry of the disk.

    Proceed with normal install on sd0 or other appropriate device.

    Make sure you include the ramdisk kernel in the install bsd.rd or whatever.

    -----------------------------------------
    Booting the New OpenBSD install
    -----------------------------------------
    After install insert your media (compact flash) into a running OpenBSD machine.  Mount the sd0a (if that is /) partition for example and enter the /etc dir.

    Create a boot file (/etc/boot.conf) with the following, unless you answered yes to the questions regarding the console output when prompted during installation:

    stty com0 19200
    set tty com0

    Now when you boot from the new install, the serial port will be used for the console, so you can connect from another PC via a serial cable using screen thus:

    sudo screen /dev/ttyS0 19200

    My Mac required syntax of

    screen /dev/tty.usbserial 19200

    A regular boot from the new install will probably fail as your media is now the boot disk and will be referenced differently (not sd0) by the OS, while the /etc/fstab file probably still says sd0a or whatever.

    In order to fix this and other problems, you actually want to boot the ramdisk first.  To do so, use the following as the boot line in the above boot.conf to begin with.

    boot wd0a:/bsd.rd 

    or type

    wd0a:/bsd.rd

    at the boot prompt

    When you get to the prompt, choose "S" for shell.

    Now you can work with things to prepare for a real boot.

    -------------------------------------------
    Get into a Sane Working Environment
    -------------------------------------------
    Mount your partitions so that /mnt is the root of the newly installed system, and everything is mounted correctly under that.

    Next, we chroot to /mnt thus:

    # /mnt/usr/sbin/chroot /mnt

    Set some environmental variables thus:

    # export TERM=vt220
    # export HOME=/

    vi should work for you now, but if not it's located at /usr/bin/vi

    Update the /etc/fstab file to reflect how your OS views your install media now.

    /dev/wd0a / ffs rw 1 1 

    Note: that the sd0a from the usb device specified to install OpenBSD on originally has been changed to wd0a

    Find out what your network interfaces are by running ifconfig

    Now copy /etc/hostname.xxx to create files for your interfaces such as

    /etc/hostname.dc0
    /etc/hostname.vr0
    /etc/hostname.vr1

    or whatever they are - you can even edit them to assign IP's etc if you like.

    My notes indicate the following is important in the /etc/ttys file (baud rate):

    tty00   "/usr/libexec/getty std.19200"  vt220    on secure"

    YOUR SYSTEM SHOULD BOOT NORMALLY NOW, AND YOU SHOULD BE ABLE TO GET TO IT FROM A SERIAL INTERFACE LIKE HYPERTERMINAL, MINICOM, OR SCREEN. (But only if you edit the /etc/boot.conf file to boot regularly, and not to the ramdisk)

    To do so, remove the boot line - default is to boot normally after 5 seconds

    Note: While this should be true, I found that I had to add the boot parameter to the last line of my boot.conf file in order for my system to boot without manually typing return at the >boot prompt.

    ---------------------------------------------------
    Tweak OpenBSD to boot from read only disk
    ---------------------------------------------------

    Edit the /etc/fstab as follows to provide a 50MB ram disk /mfs

    Note: that wd0a has been changed to read only (ro)

    /dev/wd0a / ffs ro,softdep 1 1
    #/dev/wd0a / ffs rw,noatime 1 1
    swap /mfs mfs rw,nosuid,-P=/skel,-s=102400 0 0

    Make the dir for the ram disk above

    # mkdir /mfs

    Now, we have written in fstab that the content of /skel (-P=/skel)

    Directory must be copied in /mfs at boot.

    So, now, all the files will be stored in /skel. Start by making the directories:

    # mkdir /skel
    # mkdir /skel/dev

    Now we can make the rights link, but first we make a symbolic link to
    the real path:


    # cd /mfs
    # ln -s /skel/dev/ .

    ---------------------
    Create the devices
    ---------------------
    # mv /dev/ptyp* /skel/dev/
    # mv /dev/ttyp* /skel/dev/
    # cd /dev/
    # for i in /mfs/dev/* ; do ln -s $i . ; done
    # rm -rf /mfs/dev

    ------------------------------
    Important RW Directories
    ------------------------------

    In the / (root) dir:

    # mv /root/ /skel/
    # mv /tmp/ /skel/
    # cd /
    # ln -s /mfs/root .
    # ln -s /mfs/tmp .

    In the /var dir:

    # mkdir /skel/var
    # mv /var/backups/ /skel/var/
    # mv /var/cron/ /skel/var/
    # mv /var/db/ /skel/var/
    # mv /var/empty/ /skel/var/
    # mv /var/log/ /skel/var/
    # mv /var/mail/ /skel/var/
    # mv /var/msgs/ /skel/var/
    # mv /var/named/ /skel/var/
    # mv /var/run/ /skel/var/
    # mv /var/spool /skel/var/
    # mv /var/tmp/ /skel/var/

    # ln -s /skel/var/ /mfs/var
    # cd /var/

    Note:  The next line did not work under ksh.  I had to create the symbolic links manually.

    # for i in /mfs/var/* ; do ln -s $i . ; done
    # rm -rf /mfs/var/

    That should pretty much do it.  You will still need to boot in read-write mode to create your ssh keys and a few other things might have to be done in rw mode...

    Create lock and unlock files thus

    mkdir /usr/local/conf

    echo "mount -o rw,noatime /dev/wd0a /" > /usr/local/conf/unlock.txt

    echo "mount -o ro,softdep /dev/wd0a /" > /usr/local/conf/lock.txt

    now you can lock and unlock when needed as follows:

    `cat /usr/local/conf/unlock.txt`
    or

    `cat /usr/local/conf/lock.txt`

    You might want to leave the following line in fstab, just commented out

    #/dev/wd0a / ffs rw,noatime 1 1

    So you can easily adjust if you need to reboot in RW mode.

    ----------------------------------------
    Examples of what stuff should look like
    ----------------------------------------

    # ls -lhF /var/
    total 40

    # ls -lhF /skel
    total 16
    drwxr-xr-x   2 root  wheel   2.0K Jul  9 22:16 dev/
    drwx------   2 root  wheel   512B Jul  9 22:24 root/
    drwxrwxrwt   2 root  wheel   512B Jul  9 22:25 tmp/
    drwxr-xr-x  13 root  wheel   512B Jul  9 22:29 var/

    # ls -lh /mfs/
    #

    drwxr-xr-x   2 root  wheel      512B Jul  9 21:01 account/
    drwxrws---   2 root  wheel      512B Jul  9 21:01 audit/
    drwxrwx---   2 root  authpf     512B Jul  9 21:01 authpf/
    lrwxr-xr-x   1 root  wheel       16B Jul  9 22:37 backups@ -> /mfs/var/backups
    drwxr-xr-x   3 root  wheel      512B Jul  1 17:54 cache/
    drwxrwx---   2 root  wheel      512B Feb 23 08:26 crash/
    lrwxr-xr-x   1 root  wheel       13B Jul  9 22:37 cron@ -> /mfs/var/cron
    lrwxr-xr-x   1 root  wheel       11B Jul  9 22:37 db@ -> /mfs/var/db
    lrwxr-xr-x   1 root  wheel       14B Jul  9 22:37 empty@ -> /mfs/var/empty
    drwxrwxr-x   5 root  games      512B Feb 23 08:26 games/
    lrwxr-xr-x   1 root  wheel       12B Jul  9 22:37 log@ -> /mfs/var/log
    lrwxr-xr-x   1 root  wheel       13B Jul  9 22:37 mail@ -> /mfs/var/mail
    lrwxr-xr-x   1 root  wheel       13B Jul  9 22:37 msgs@ -> /mfs/var/msgs
    lrwxr-xr-x   1 root  wheel       14B Jul  9 22:37 named@ -> /mfs/var/named
    drwxr-x---   2 root  operator   512B Jul  9 21:01 quotas/
    lrwxr-xr-x   1 root  wheel       12B Jul  9 22:37 run@ -> /mfs/var/run
    drwxr-xr-x   2 root  wheel      512B Jul  9 21:01 rwho/
    lrwxr-xr-x   1 root  wheel       14B Jul  9 22:37 spool@ -> /mfs/var/spool
    lrwxr-xr-x   1 root  wheel       12B Jul  9 22:37 tmp@ -> /mfs/var/tmp
    drwxr-xr-x  10 root  daemon     512B Jul  9 21:01 www/
    drwxr-xr-x   2 root  wheel      512B Jul  9 21:09 yp/
     

    Thursday, January 13, 2011

    Pf Tips

    Some basic notes and commands to aid in diagnosing PF.

    pfctl -d Disable the packet filter
    pfctl -e Enable the packet filter
    pfctl -Fa -f /etc/pf.conf Flush all (nat, filter, queue, state, info, table) rules and reload from the file /etc/pf.conf
    pfctl -s rules Report on the currently loaded filter ruleset.
    pfctl -s nat Report on the currently loaded nat ruleset.
    pfctl -s state Report on the currently running state table (very useful).
    pfctl -v -n -f /etc/pf.conf This does not actually load any rules, but allows you to check for errors in the file before you do load the ruleset. This is obviously good for testing.


    pflogd logs in tcpdump binary format

    To watch only packets that match a certain interface use the following: 

    tcpdump -n -e -ttt -i pflog0 host 192.168.0.1

    To watch packets that match a specific port:

    tcpdump -n -e -ttt -i pflog0 port 443

    To determine a specific rule as identified from tcpdump.  For example rule #100 below.  Note:  This is not the same as simply editing the pf.conf file and going to line #100.  PF expands rules due to macros etc...

    pfctl -vvs rules|grep \@100