[sf-lug] About USB partitioning and formatting

[Ken Shaffer]

Jim, more notes on optimizing a usb stick once it's been set up:

Some Optimizations for Linux Running on a 4G Flash Memory

(Summary of flash setup for 128k erase blocks: partition info is 4 heads, 16
sectors/track, start at sector 8192, to end, no swap, ext2 filesystem (4k
blocks 32k groups (which are the defaults I think), install to the flash,
rejecting offers to repartition or format.)

Linux installations on flash cards or USB thumbdrives raise the issue of
longevity, since after some number of writes to a flash block, it wears out.
Miminize these writes to flash by using ramdisks. Laptops these days
frequently have 2G-4G of memory, more than enough for typical useage. Using
a part of this ram for ramdisks mounted at locations of high write activity
is a reasonable way to extend the life of the flash media.

With 2G of ram on my laptop, I create my (4G) thumbdrives without swap, with
no adverse consequences, but this is just a matter of not wasting space --
my normal processing load does not use swap anyway. Two obvious locations
with high write activity are /tmp and /var/log. Putting the /var/log on a
ramdisk means that system logs would be dumped at each shutdown, so remember
to save any logs needed for problem investigation. Since both of these
locations are used early in the boot sequence, getting them mounted as soon
as possible makes sense. Two lines in /etc/fstab will be all that is needed:

tmpfs        /tmp        tmpfs    size=256M,mode=1777    0    0
ramfs        /var/log    ramfs    mode=755        0    0

update-manager cannot see any space in /tmp with the ramfs filesystem -- a
problem (bug 495131) caused by the underlying os.statvfs function. Use
tmpfs, even if there is no swap. I think the default size of the /var/log
ramdisk is 32M or 64M, which is more than sufficient for me, but df -a
reports 0 size, so it too seems to have a problem with ramfs. These are the
only two programs I run regularly in which I have noticed any problems --
the system runs normally otherwise. Some applications (e.g. Postgresql ) may
have problems if their logs are in a non-existent directory. These problems
may be addressed on a case by case basis, adding the appropriate directory
creations and ownerships to the /etc/rc.local startup file.

The directory that update-manager uses for downloaded packages may be moved
to a ramdisk. With 256M in /tmp, just create a link from the
/var/cache/apt/archives directory to /tmp/debs. The existing archives
directory may be renamed or removed.

  sudo mv /var/cache/apt/archives /var/cache/apt/archives-orig
  sudo ln -s /tmp/debs /var/cache/apt/archives

Finally, create the link target by add the below mkdir line to the end of
/etc/rc.local, just before the "exit 0" (since the /tmp ramdisk starts empty
at boot).

  mkdir -p /tmp/debs/partial

The /var/log ramdisk is remade from scratch at every boot. No problems were
noticed until trying to update/run postgresql-8.4, which needed a postgresql
directory which was not made on the fly. This fix consists of adding the
directory creation to the rc.local file. Instead, I just removed Postgresql,
which is just used for a few demos.

  mkdir -m775 /var/log/postgresql
  chgrp postgres /var/log/postgresql

Another directory with lots of writes is my browser cache directory. A link
from this cache directory to /tmp/Cache-username removes those writes from
flash, and speeds up browser performance too.

  rm ${HOME}/.mozilla/firefox/xxxxx.default/Cache
  ln -s ${HOME}/.mozilla/firefox/xxxxx.default/Cache /tmp/Cache-username

Create the link target target in /etc/rc.local:
  mkdir /tmp/Cache-username

Gnome and various applications create/modify lots of files, so a more
radical step would be to move an entire user home directory to /tmp. Create
a new user, e.g. guest, and login to the new account.

  adduser guest
  ...

Tailor the things you care about like the desktop, bookmarks, mail, ...
Assuming the guest user does not have sudo capability, logout, and log back
in as your normal user, who does have sudo capability. Tar up the guest home
directory:

  cd /home
  sudo tar -czf /usr/local/guest.tgz guest
  sudo chown guest:guest /usr/local/guest.tgz

Edit /etc/rc.local, to unpack the guest.tgz file into /tmp:
  sudo vi /etc/rc.local
  # create the guest home directory in /tmp
  cd /tmp
  tar -xpzf /usr/local/guest.tgz


Finally, edit the /etc/passwd file and change the guest home directory
to be in /tmp:
  sudo vi /etc/passwd
  ...
  guest:x:1001:1001:Guest,,,:/tmp/guest:/bin/bash

Now reboot and try to login as guest. A pwd command at a terminal should
give you /tmp/guest. Browser performance should be good, but remember, no
further changes to bookmarks, etc. will be permanent. Permanent changes may
be made by re-taring the /tmp/guest directory and replacing the
/usr/local/guest.tgz file. Of course, a file may still be copied to
/home/guest/... to permanently save it. If you do want to change the guest
home directory, make changes immediately after logging in, to keep unwanted
temporary files out of the tarball. After running as guest for awhile, you
can see what files get created with a find command:

  find . -newer ./.cache/notify-osd.log -ls

My typical browser/email usage for less than an hour resulted in 270
file creations/modifications, of which 180 were browser cache files.

links to permanent directories back in /home/guest may be added and the
/tmp/home re-tarred to provide locations with persistance. Another approach
would be to login as guest, run xhost to allow some otheruser with $HOME on
flash to use the display, then su to this otheruser. This approach keeps a
lot of the Gnome/app temporary files in memory instead of being written onto
flash.

As guest, use xhost to add username (or just open up the display to
everyone with xhost + ):

  xhost +SI:localuser:otheruser
  su otheruser

Now run firefox from a command line, (ignore the error popup) and see
otheruser's bookmarks.  Firefox is easy, but other X programs
(e.g. evolution) which rely heavily on Gnome's environment/sockets
will need more work to run this way.  The benefits are obvious --
the flash-access LED blinks far less often.

Duplicating one of these sticks is quick and easy with the dd program.
A stick to stick copy is possible, but I make a copy on a hard disk
to allow faster copies on multiple sticks.  Use the 128k erase block
size to write (read) to the sticks, and the number of writes should
match the partition information on the stick (1/4 the number of 32k
cylinders).

  dd if=/dev/sdx of=file     ibs=128k

Switch sticks.

  dd if=file     of=/dev/sdx obs=128k

I have been very satisified running my daily operations off a 4G stick for
the last few months. I still have some concern about how long the flash will
last with this normal workload, but even so, I run update-manager ever few
days, so updates represent the heaviest write load on the stick.

Since hardware is expected to fail at some point, backup is not optional.
Spinning metal platters are fragile too, a friend lost a Terabyte of his
data when he tipped over the external disk enclosure. My laptop has
developed a (motherboard?) problem seeing its internal (Windows) disk, and
fails to boot with it present. Solution: remove the disk (which is good and
still accessible via a USB enclosure) and continue to use the laptop by
booting off Linux USB devices (it runs cooler now ;) ).

Ken Shaffer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://linuxmafia.com/pipermail/sf-lug/attachments/20100207/d9213606/attachment.html>