How To setup a Diskless Swap System.

09-07-2008

Registered User

2, 0

Join Date: Sep 2008

Last Activity: 13 December 2008, 6:36 PM EST

Posts: 2

Thanks Given: 0

Thanked 0 Times in 0 Posts

How To setup a Diskless Swap System.

This is a guide for setting up your computer with a fast swap system using usb memory sticks. A detailed study of the theories behind this setup can be found here. While this How-To does not include the CFLRU replacement algorithm, it does show the performance benefits of using NAND flash memory and will hopefully increase interest and speed development.

It is assumed you know how to format your USB NAND storage drives as swap, how to find the serial number and kernel names(i.e. sdc). This was written for Ubuntu, most should stay the same for other distros except maybe for path names and you will probably need to make your own boot script in the appropriate location.

1.) Create a persistent device node for each swap device using udev. This will allow the devices to move around but won't effect your swap system.

Code:

sudo gedit /etc/udev/rules.d/15-swap.rules

Then add the following lines but using your serial numbers:

Code:

SUBSYSTEM=="block", ATTRS{serial}=="46d2cef7d5a461", NAME="%k", SYMLINK+="SwpNodA"
SUBSYSTEM=="block", ATTRS{serial}=="SNDKA96735075A203908", NAME="%k", SYMLINK+="SwpNodB"
SUBSYSTEM=="block", ATTRS{serial}=="SNDKA940252CCCC05806", NAME="%k", SYMLINK+="SwpNodC"
SUBSYSTEM=="block", ATTRS{serial}=="SNDKA96735037AA03903", NAME="%k", SYMLINK+="SwpNodD"

2.) Update fstab:

Code:

sudo gedit /etc/fstab

You should see a line something like this:

Quote:

UUID=921f422a-5816-4040-97df-850e569c4687 none swap sw 0 0

it is your current swap drive, comment it out with a '#' at the front. Underneath place the following lines:

Quote:

/dev/SwpNodA none swap sw,pri=0 0 0
/dev/SwpNodB none swap sw,pri=0 0 0
/dev/SwpNodC none swap sw,pri=0 0 0
/dev/SwpNodD none swap sw,pri=0 0 0

3.) Edit sysfs:

Code:

sudo gedit /etc/init.d/bootmisc.sh

find the lines that read:

Quote:

# Remove bootclean's flag files.
# Don't run bootclean again after this!

and add these lines

Code:

for x in A B C D
do
Node=`udevinfo --query=name --name=SwpNod$x | cut -c -3`
echo -n 1024 | dd ibs=4096 conv=fdatasync,sync of=/sys/block/$Node/device/max_sectors 2>/dev/null
blockdev --setra 0 /dev/$Node
echo deadline > /sys/block/$Node/queue/scheduler
done

4.) As of kernel ~2.6.20 the Virtual Memory settings might need to be modified. Check your values with

Code:

sudo sysctl -A | grep vm

Add the following to /etc/sysctl.conf if the values don't match these

Quote:

vm.dirty_writeback_centisecs=1000
vm.dirty_background_ratio=3
vm.dirty_ratio=40

5.) Reboot and check your swap system with

Code:

swapon -s

Quote:

Filename Type Size Used Priority
/dev/sde1 partition 503400 69252 0
/dev/sdf1 partition 976884 69024 0
/dev/sdb1 partition 497972 69140 0
/dev/sdd1 partition 497972 69008 0

Comments:

Udev

When the kernel starts it talks to all the devices and gathers information to build sysfs with. Udev is a front end for us to use as a way of managing this information, without needing to know a lot about the kernel. Here we identify our device by serial number and make a pseudo device node with a name "SwpNod". The pseudo node is just a system link pointing to the kernel device, %k.

fstab

Here our node is referenced for mounting purposes, it has the options of being a swap partition and has equal priority with the other nodes. When the priorities are equal, the kernel will use them in sequence, first to last then back to the first and so on until all the data written. In this situation, the work is spread across the devices evenly which multiplies read/write speeds of page clusters by the number of devices (i.e. x4).

Boot Script

This function sets 3 kernel values for each device. The first tells the kernel to transfer data to the device in bulk packages of up to 1024 sectors(usually 512b). The second turns off device read ahead, which is a technique used by disk devices to help eliminate data access lag but is useless for swap devices in the long run due to the natural tendency of swap data to become fragmented over time (See the study for more info). The third value tells the kernel to make read operations more important then write operations. This is very true for a swap system since data being written is "dormant" and data being read is "critical".

Sysctl

The default writeback time was 500 centisecs but is increased to 1000 to cut down on cpu operations and make better use of the max_sector value. The other options should never have changed. While "tweaking" these values may increase performance for one application in the short run, it will effect other applications including the desktop application in the long run. Basically, the linux swap system is a simple system that "just works" and while the developers allow for configuration changes, it would be better to add to the code.

End Notes:

a) Blocks will "burn out" based on use over time, a check can be made and a resize of the partition can be done. In most cases, an idle desktop with no background processes will not require a swap file and the `swapoff` command can safely be used. `mkswap -c /dev/xxx1` will remake the swap file and check for errors. For an exhaustive check, format the drive to ext2 and check with `mke2fs -c -c /dev/xxx1`. I thought that over years of heavy use, the resulting sector loss was ~15MB of drive space off each device but this may be partition mechanics, device and partition reserve space since the reported size has not changed... Keep an eye on the value and knock 100MB off the partition if paranoid.

b) This setup has been stress tested with large flash video files, World of Warcraft, Firefox with large cache, Mplayer playing DVD, all running simultaneously. It was also used during development of this script. It should be noted that a number of bugs appeared during early development of that script which produced some extremely abnormal operating environments and I smoked a drive... the final version ran without issue. The script is not intended for use, it is linked only to document testing conditions.

c) There is talk about turning swapping off since the cost of system RAM has decreased dramatically. Wrong. Swapping not only frees up system RAM, it splits the page lookup table into a table of "active" pages and a table of "non-active" pages. This helps reduce the workload on a cpu by eliminating look-ups on dormant pages. The idea should be taken further with a small area of system RAM (possibly the write cache) as a 'Level 1' swap area, NAND memory as 'Level 2' and finally a disk as 'Level 3'. Here pages we normally would write to disk are kept in RAM(Level 1) "just in case", then expire to Level 2 and finally Level 3. Things like access frequency, read or write access and possible access would determine what level(and table) the pages should reside. The idea is to have important pages circle in RAM, fall to a fast access Level 2 when deemed non-important and then finally hit a disk that is organised for fast read. Memory Management kernel functions should be a candidate for co-processing.

Further Reading:

Code:

man fstab; man udev; man swapon

Sysctl VM

Last edited by Johnny_Thumbs; 12-13-2008 at 08:31 PM..

Johnny_Thumbs

View Public Profile for Johnny_Thumbs

Find all posts by Johnny_Thumbs

09-29-2008

Registered User

2,157, 51

Join Date: Feb 2007

Last Activity: 6 September 2017, 5:43 AM EDT

Location: Innsbruck, Austria

Posts: 2,157

Thanks Given: 12

Thanked 51 Times in 48 Posts

Cool. Better yet, buy more RAM and turn off swapping.

otheus

View Public Profile for otheus

Find all posts by otheus

10-09-2008

Registered User

2, 0

Join Date: Sep 2008

Last Activity: 13 December 2008, 6:36 PM EST

Posts: 2

Thanks Given: 0

Thanked 0 Times in 0 Posts

Excellent question, one I don't have a complete answer for since I've never tried it. Why not load the whole operating system and all the data a person would ever use in there as well? Knoppix has a boot option which allows you to copy all the iso files into RAM. Grab the DVD, boot it toram(if possible on DVD), install WoW into home; that should be about 20GB. When I look at this situation I wonder what is happening on the Front Side Bus and how much of the important data is getting down the channels ahead of non-important. DSL is a derivative of knoppix with the same boot option and when it is booted toram there is still lag due to 100% cpu; Moore's law if you will. How much of the cpu usage is user instructions or system duties like memory management. Filling the tables with pointers to dormant data would increase the Big "O" and place even more demand on an already taped out tech.

Looking at a Motherboard we see it is covered in circuits. For the machine to be operating at 100% efficiency, all the circuits should be pulsing with data simultaneously. The MPU should be at 100% as well as cache memory. One way of side stepping Moore's Law is multitasking the hardware better and increasing bandwidth. Offloading common operations onto hardcoded processors with a direct connection to main cpu and system bus, etc.. But I digress, the most important reason to do this is as a means of upgrading an old computer at low expense. I highly recommend trying to run DSL on an older cpu, it seems the world needed better software a lot more then faster processor speeds. Older machines can act as Firewall/Router/Gateways, web servers, tor servers, file servers, wireless monitors, etc.. Don't sell or throw away that xbox because it has many years left running the latest kernel code.

Last edited by Johnny_Thumbs; 10-09-2008 at 12:41 AM.. Reason: ...spelling...

Johnny_Thumbs

View Public Profile for Johnny_Thumbs

Find all posts by Johnny_Thumbs

Filesystems, Disks and Memory

How To setup a Diskless Swap System.

10 More Discussions You Might Find Interesting

1. Solaris

File system full, swap

Discussion started by: nikhil kasar

2. Red Hat

Static IP Address setup for vm as well as the host system

Discussion started by: rparavastu

3. UNIX for Dummies Questions & Answers

swap issues, system is running at 99%

Discussion started by: samnyc

4. Solaris

increase SWAP on ZFS file system

Discussion started by: osmanux

5. UNIX for Advanced & Expert Users

Swap file system

Discussion started by: bluescreen

6. Solaris

swap space in veritas file system

Discussion started by: samy123

7. Solaris

/tmp: File system full, swap space limit exceeded

Discussion started by: frustrated1

8. IP Networking

Setup 2 NIC cards in one UNIX system

Discussion started by: texaspanama

9. UNIX for Advanced & Expert Users

What swap is my system running on?

Discussion started by: abhijeetkul

10. UNIX for Advanced & Expert Users

System wide CDE setup

Discussion started by: meyersp