[conspire] Lying drive issues (was Re: The practice of making ext4 a default needs to die an excruciating and gruesome death)

[Tony Godshall]

OK, so the posting below was probably a bit cryptic, but
I was at a conference at the time.  ZFS day, in fact
where they were discussing the ZFS file system that
originated at Sun and is now available in Linux (in both
FUSE and kernel-module variants), and BSD/Apple/
Illumos/OpenIndiana/Oracle (Solaris and forks)

Anyway, the gist if I recall correctly was that some
larger hard drives (typ 2TB and larger) (and most flash
drives) emulate 512-byte sectors while really doing 4K
or larger sectors under the covers and that some
actually lie to the OS so the OS can't know it's doing that.
Which means every write on such drives is really a
read-write cycle with possible disastrous consequences
on power-loss and also some weirdnesses on partitioning,
especially old-style 63-sector offset partitioning which
pretty much guarantees poorly aligned partitioning
(some drives have an "XP mode" jumper or internal setting)

Those interested in more details might visit zfsday.com
to identify the presenters and then google them for more
details.

The change I've made myself is to reduce the amount
of write-in-place I use- I'm using ext3 with full journaling
(tune2fs -o journal_data <drive>), which by the way
does not have as much performance concern as I used
to assume it did (double-write is offset by reduced seek
as the journal can be written close to the ) and I'll be
starting to use ZFS For Linux (the kernel module version)-
it's in production in the largest computer system in the
worlds, so it's stable enough for me to try at home.

And it checksums top to bottom.

> Subject: Lying drive issues Re: [conspire] The practice of making ext4 a
>  default needs to die an excruciating and gruesome death
>
> ZFSday quote...
>
> Y4K? ? George Wilson
>
> For over 30 years, hard drives have designated the smallest storage
> location as 512 bytes. In January 2011, all major hard drive manufactures
> began shipping their hard drive platforms using a new standard called
> Advanced Format. To aid in the transition, these new hard drives provide a
> 512 byte emulation mode that allows the drives to advertise themselves as a
> 512 byte addressable devices. This can severely impact write performance
> resulting in the need for read-modify-write operations for any misaligned
> or partial writes that are issued.
>
> The problem is not limited to just physical hardware. Other storage
> platforms may also provide LUNs (logical unit number) that presents
> themselves as a 512 byte addressable devices when, in fact, they use a 4K
> sector size internally. Although ZFS has built-in support for 4K sectors,
> it has no automatic way of dealing with the lies that the storage devices
> tell. This talk will focus on the methods that have been developed to work
> around the lies that hard drive storage platforms tell and will discuss the
> challenges and drawbacks that come with using 4K sectors.
>
> ----- End forwarded message -----
>
> _______________________________________________
> conspire mailing list
> conspire at linuxmafia.com
> http://linuxmafia.com/mailman/listinfo/conspire



-- 
Best Regards.
This is unedited.