[sf-lug] Dynamically Scalable Data?

[Kristian Erik Hermansen]

On Fri, Apr 18, 2008 at 11:50 AM, Ernest De Leon <edeleonjr at gmail.com> wrote:
> For some reason, I'm perfectly ok with the scalability from the server and
> application side.  (That may or may not be a bad thing)  :)  I guess what
> I'm more after is tackling the issue of dynamic data availability (say due
> to a spike in search traffic for a particular video or image).  Even within
> that, I'm not really concerned with bandwidth per se, but more with the
> ability of the data storage device (a SAN in this case) to scale it's output
> on a certain finite data set within the larger warehouse.  With new tools
> available in the storage world like dynamic data optimization, it is now
> possible to redistribute data across the spindles in a SAN to increase
> throughput out of the device itself onto the fiber network.  I am fleshing
> out a way to take a dynamic event like a surge in search queries for a given
> finite data set and translate that into an automated process or system than
> can integrate with the storage management platform and dynamically scale the
> throughput on the fly.  Then perhaps when the spike in interest subsides,
> redistribute the data 'normally' to allow for the next surge to take
> precedence.

If you have money, you can use what the movie studios are
transitioning to :-)  The IBRIX file system does some very cool
things, and I cannot tell you about all of them here...
http://www.ibrix.com/

* DISCLAIMER: My brother Jon Hermansen is the youngest employee at
IBRIX and so I am biased toward his input.  However, I still encourage
you to check them out and see what they are doing.  PIXAR is one of
their main customers, and have raved about the file system speed
increases they have gotten.  In fact, some new movies are
better-looking because IBRIX is able to speed up the rendering by 100x
or more...
http://www.ibrix.com/pages/customers.php
-- 
Kristian Erik Hermansen
--
"Clever ones don't want the future told. They make it."