Asynchronous File I/O on Linux

[richard.pieri-Re5JQEeQqe8AvxtiuMwx3w at public.gmane.org (Richard Pieri)]

On May 18, 2010, at 2:06 PM, Bill Bogstad wrote:
> 
> Please re-read the end of my last message.   Take a look at pread()
> (POSIX) and readahead() (Linux only).
> It turns out you do not need separate file handles.    Threads may
> still be required to make it non-blocking.

I did look at pread().  While it doesn't reset the file handle's seek pointer it still effectively does a lseek() to the offset.  So that's not really a performance win.


> Not necessarily   I would like random chunks of data from this file
> (perhaps NEED it at some specific computation point in the future),
> but I have some other computation I can do in the meantime.   Please
> start the disk IO now.  Don't make me create multiple FDs for a single
> file.    At my option, I would like you to:

This seems a little silly to me.  I mean, if your processing takes more time then the reads then the hand-carved optimizations aren't a win.  On the other hand, if they don't then the reads will block.  If you aren't caching anything at that point then I think it's time to reconsider your storage format because this is a problem that's been solved before.

--Rich P.