Interesting observation

[markw at mohawksoft.com]

> On Mon, Apr 07, 2008 at 10:06:07PM -0400, markw at mohawksoft.com wrote:
>>
>> I have a compressed data source (freedb, netflix, geo), I have a small
>> extraction program that parses the data and formats the insertion
>> strings.
>> Using the insertion strings, I add the data to a database.
>>
>> Now, up until the 1g ethernet, it was faster to run the loader and the
>> database on the same machine. Now that the data transfer is
>> theoretically
>> 10 times faster than 100mb, the data transfer (assuming separate
>> machines)
>> is no longer larger than the processing time.
>>
>> I'm not sure if this is earth shattering news, but it is an interesting
>> metric.
>
> Cycle of reincarnation: about the time 10 Mb/s ethernet became popular,
> a friend noted that actual throughput via NFS was significantly faster
> than the disk interfaces he had on the low-end machines.

Well, I remember when "Stac" disk compression was used to speed up reads
because disks were so slow. Which works really only (and still does) with
highly compressible data.

However, up until now, I believe, disks have always been faster than
network. When ethernet was 10 mbit/s, disks were 2 mbyte/s, with ethernet
100mbit/s disks are topped at 20mbyte/s, with SATA disks are around
40mbyte/s, but gb ethernet is 1000/mbit a second, roughly and (depending
on protocol, switching, IP stack, etc.) potentially twice as fast.

Now that hurdle is cleared, it makes a physical disk with high speed
interface potentially not as fast as a network service. Which, if you
think about issues of scaling for performance, creates a whole new
paradigm.

Just something to think about.

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.