[sf-lug] About USB partitioning and formatting
Ken Shaffer
kenshaffer80 at gmail.com
Mon Feb 8 16:38:41 PST 2010
Here's some info on my planning process for setting up these sticks, and
some instructions.
Have Fun
Ken
Bootable USB Stick Setup
Of course, the Ubuntu "System/Administration/USB Startup Disk Creator" menu
item will create a "Live USB" stick which will (eventually) boot Linux, but
a
much quicker booting Linux system may be created by installing directly to a
4G stick. A little initial preparation of the stick should allow for
increased life and snappier performance.
Assumptions
1) The USB stick will be set up from a running Linux system, and standard
Linux/Hurd tools are available (fdisk, cfdisk, mke2fs, cp, df, dmesg, free,
mount, tail, vi, ... )
2)The USB stick will be the target of a live media install.
3) The install system can access a usb drive, but does not have to be able
to boot from one.
4) Explicit instructions below assume a Gnome desktop -- use the equivalent
actions for your system.
5) Not all usb sticks/cards have equal performance. I have used Kingston
Datatraveler sticks, Kingston SD4 cards, and HP sticks with good success.
Check the packaging or vendor web site for any indication of the media
speed.
The cards usually indicate class 4 (4MB/sec) or class 6 (6MB/sec). I seldom
see any speed indications on the usb sticks. Kingston's web site indicated
the Datatraveler stick reads at 10MB/sec and writes at 5 MB/sec (just over
class 5). http://en.wikipedia.org/wiki/Secure_Digital#Speeds has the table
mapping the speed to class. Notice that read speeds are faster than
write speeds, and ideally, the vendor will report both.
6) The flash memory erase block size is 128k. This may actually be 256k,
but
I don't know how to tell. The larger (8G+) sticks may have the larger size.
Planning
Flash media has a limited number of writes, and blocks are grouped -- a
write of one block forces all the blocks in the group to be rewritten. Some
thought is needed to position partitions and filesystem parts to minimize
unnecessary group rewrites. Space is a critical resource on these USB
sticks,
so consider forgoing the standard /boot and swap partitions. A standard
Ubuntu installation will fit nicely on a 4G card or stick.
Historically, a /boot partition allowed positioning the kernel within the
first 1024 cylinders of a disk for BIOSes whose address space was limited.
Some older machines may still need this. If the 1024 limit is not a factor
on your machine, then doing away with a boot partition will not waste the
unused space in such a partition. If you are setting up a generic stick
which
will boot in the maximum number of (older) machines, maybe you do want a
boot
partition for the first partition. The below assumes no separate boot
partition is needed.
Swap space simulates having more, slow, memory on your computer.
It may not even be needed, but if it is, may be added later as a
file or as a separate USB stick. If you are on a Linux/HURD machine
with similar memory to the one on which you will be running the USB
stick, check swap usage with "free", "swapon -s", or "cat /proc/swap".
Newer machines probably have memory to spare, so seeing no
swap activity is typical. If swap activity is seen, then decide if
the convenience of having swap on the usb boot stick is worth the space and
possibly shortened lifespan of the stick (assuming the writes to the swap
device wear the stick out before anything else). The alternative solution
is to use a second small (cheap) 0.5-2G stick for swap. This may
extend the life of the larger (more expensive) boot stick.
The stick partition start (and end) will be selected to be aligned with
the 128k flash block groups. I use a non-journaling filesystem, ext2,
the with block size set to 4096 (default) and the groupsize set to 32768
(also the default).
Partitioning the USB stick
Insert the USB stick drive (or memory card). If the media is automounted,
an icon will appear on the desktop, and a window may open. Use df, mount,
or dmesg to determine the device:
df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sdb7 14112620 6632504 6763220 50% /
...
/dev/sdc1 3969024 32 3968992 1% /media/disk
or
mount
...
/dev/sdc1 on /media/disk type vfat
(rw,nosuid,nodev,uhelper=hal,shortname=mixed,uid=1000,utf8,umask=077,flush)
See below for the dmesg information.
close the window if one opened at the mount, and unmount the media (the icon
should disappear).
-- right click on the drive desktop icon, then select "safely remove" if
present, or
"Unmount volume" if not.
If the automount does not run (it is a configuration item), get the device
from the system messages.
Run dmesg to see what device your drive was (in the below example it was
sdc).
-- dmesg |tail -23
...dmesg output for a USB stick:
[ 5524.328108] usb 1-3: new high speed USB device using ehci_hcd and address
5
[ 5524.463443] usb 1-3: configuration #1 chosen from 1 choice
[ 5524.464185] scsi6 : SCSI emulation for USB Mass Storage devices
[ 5524.464674] usb-storage: device found at 5
[ 5524.464679] usb-storage: waiting for device to settle before scanning
[ 5529.464419] usb-storage: device scan complete
[ 5529.466864] scsi 6:0:0:0: Direct-Access Multi Flash Reader
1.00 PQ: 0 ANSI: 0
[ 5529.797821] sd 6:0:0:0: [sdc] 7954432 512-byte hardware sectors: (4.07
GB/3.79 GiB)
[ 5529.799666] sd 6:0:0:0: [sdc] Write Protect is off
[ 5529.799672] sd 6:0:0:0: [sdc] Mode Sense: 03 00 00 00
[ 5529.799678] sd 6:0:0:0: [sdc] Assuming drive cache: write through
[ 5529.804687] sd 6:0:0:0: [sdc] 7954432 512-byte hardware sectors: (4.07
GB/3.79 GiB)
[ 5529.808673] sd 6:0:0:0: [sdc] Write Protect is off
[ 5529.808679] sd 6:0:0:0: [sdc] Mode Sense: 03 00 00 00
[ 5529.808685] sd 6:0:0:0: [sdc] Assuming drive cache: write through
[ 5529.808698] sdc: sdc1
[ 5529.810505] sd 6:0:0:0: [sdc] Attached SCSI removable disk
[ 5529.810662] sd 6:0:0:0: Attached scsi generic sg3 type 0
...dmesg output for a directly inserted memory card (automounts :
[ 4833.619838] mmc0: new SD card at address e624
[ 4833.646052] mmcblk0: mmc0:e624 SD02G 1.89 GiB
[ 4833.646179] mmcblk0: p1
(The below assumes sdc for the device -- make sure you have the correct
device for your system.)
The information from the below link Jim mentioned:
http://wiki.laptop.org/go/How_to_Damage_a_FLASH_Storage_Device
resulted in the selection of the partition parameters. I guessed my
4G devices were of the 128k write block instead of the 256k block variety,
but I haven't figured out how to tell.
Examine the device to determine the default parameters and partitioning. a
4G
Kingston stick had h=5 s=32 and the partition start at 81920 sectors of 512
bytes -- about a 40 meg offset, and not even a power of 2. I choose to use
h=4, s=16 and partition start at 8192, so I zeroed the existing partition
table and reset the heads and sectors/track. This may be done within fdisk
when the partition is made, or on the command line:
cfdisk -z -h 4 -s 16 /dev/your_drive_here
Note, the above command performed as I expected on a 4G sd card inserted in
mmcblk0, but several 4G usb sticks failed to get the expected h and s. This
unexpected resetting of h and s has also occurred when using fdisk. Reset
them in fdisk when setting up the partition if necessary. See the below
output. There appears to be some buffering which does not get flushed as
expected, as I have seen this sort of problem frequently. Try removing the
media right after exiting fdisk, and reinserting. Check with fdisk to ensure
the proper number of heads and sectors/track is present, and quit without
saving. Resolve this before making the filesystem, because it may be the
wrong size, and may be too small to hold the installation.
If you decide to make a swap area on the usb stick, figure out its size in
sectors so you can make the root (first partition) smaller by that amount.
With fdisk, the m command is for help, which lists all the commands.
Ensure that the heads and sectors are still set, or set them now, in the
x=advanced section of commands (h for heads, s for sectors, r to return to
main menu).
$ sudo fdisk /dev/sdc
[sudo] password for user:
If not done with cfdsk, set the heads and sectors/track
x (advanced commands)
h (set the heads to 4)
s (set the sectors/track to 16)
r (return to main menu)
Set the units to sectors (which start at 0) with the u command,
then make a root partition:
u (toggle units to sectors from cylinders)
Delete the old partition if not already done with cfdisk
d (delete the existing partition)
Create a new partition
n ( new partition),
p (primary partition),
1 (first primary),
8192 (start sector, 4M and a power of 2),
XXXX (end sector at end of the available sectors as one less than a
multiple of 8192) (which may be the last available sector,
or a few sectors back, depending on the stick, or back by the
swap size in sectors.
The type of the partition should default to linux type -- check with p to
print
out the partition information.
Set it explicitly with the t command with type = 83.
Make it bootable:
a (select first partition)
If you want a swap partition, make it now as the next primary.
n (new partition)
p (primary)
2 (number 2)
XXXX+1 (starting on the next free sector beyond the end of the first,
which
should be a multiple of 8192.
YYYY (end of the swap partition, (as one less than a multiple of 8192),
Set its type to 82, with the t command.
t (Set the type)
2 (partition 2)
82 (swap type)
Exit fdisk saving the new configuration:
w (write out the new info, and exit)
Example, checking with fdisk after setting everything:
$ sudo fdisk /dev/sdc
[sudo] password for user:
The number of cylinders for this disk is set to 122496.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Command (m for help): p
Disk /dev/sdc: 4013 MB, 4013948928 bytes
4 heads, 16 sectors/track, 122496 cylinders
Units = cylinders of 64 * 512 = 32768 bytes
Disk identifier: 0x00000000
Device Boot Start End Blocks Id System
/dev/sdc1 129 122496 3915776 83 Linux
Command (m for help): u
Changing display/entry units to sectors
Command (m for help): p
Disk /dev/sdc: 4013 MB, 4013948928 bytes
4 heads, 16 sectors/track, 122496 cylinders, total 7839744 sectors
Units = sectors of 1 * 512 = 512 bytes
Disk identifier: 0x00000000
Device Boot Start End Blocks Id System
/dev/sdc1 8192 7839743 3915776 83 Linux
Command (m for help): q
Make the filesystem now:
sudo mkfs.ext2 -b 4096 -g 32768 /dev/sdc1
mke2fs 1.41.4 (27-Jan-2009)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
244800 inodes, 978944 blocks
48947 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=1002438656
30 block groups
32768 blocks per group, 32768 fragments per group
8160 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736
Writing inode tables: done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 25 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
The usb stick or card is now ready for a standard installation from
a live media. Reject all offers to repartition or format the card,
so what was set up above will be used. Continue through all warnings
about missing swap, or illegal attributes on the filesystem.
Installation
Boot your install media, and perform an install to the stick which you set
up. If your install media is also a USB stick, and you have problems
booting
with two USB sticks present, you might need to insert the target stick after
the boot starts. There will be several warnings of problems with no swap
space (assuming you have no swap partition), and invalid file attributes,
all
of which may be ignored. Do not checkoff the format box for the partition,
since you have already created the filesystem with your selected attributes.
Always check the location grub will put its boot block -- click on the
Advanced button after partitioning. It needs to be the target stick, and it
better be a real device, /dev/sdx, not hd0 (Karmic bug). Regardless of
your
install media, hd0 is most likely the Windows hard disk, and that is not
a place most people want to rewrite.
After installation completes, remove the install media, check that the boot
order has the USB stick before the hard disk, and try booting from the
stick.
First order of business is to edit the /etc/fstab file and change the
"relatime" to "noatime" on the line starting "UUID=" followed by a single
"/". This change prevents the "access time" for files from getting written
into the file system, and remember, we are trying to minimize writes.
With only 4G total, you need to be more aware of buildup of unwanted files.
After every update run, all the package files are left in
/var/cache/apt/archives. Lots of old copies of system log files are also
kept. There are options to limit the number of old copies of log files,
but check out the Optimization post first. Writing the log files
to a ramdisk means they do not build up over time.
After everything has been set up, and works, consider moving the most
active write directories (like /tmp and /var/log) onto a ramdisk (See
the Optimizing post).
Ken Shaffer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://linuxmafia.com/pipermail/sf-lug/attachments/20100208/92ef44d8/attachment.html>
More information about the sf-lug
mailing list