Boston Linux & Unix (BLU) Home | Calendar | Mail Lists | List Archives | Desktop SIG | Hardware Hacking SIG
Wiki | Flickr | PicasaWeb | Video | Maps & Directions | Installfests | Keysignings
Linux Cafe | Meeting Notes | Blog | Linux Links | Bling | About BLU

BLU Discuss list archive


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Asynchronous File I/O on Linux



On May 18, 2010, at 11:02 PM, Bill Bogstad wrote:
> 
> I didn't say pread()  was faster.  However, I believe it could safely
> allow concurrent/independent IO requests via the same underlying file
> descriptor from multiple threads.   Since the offset is explicit with
> pread() there should be no reason for conflicts on the single shared
> file offset to occur like with read() from multiple threads.   Whether
> it actually works that way or not, I don't know.

Ah-hah!  I get it now.  One file descriptor shared by multiple threads using pread() would give you more or less concurrent read operations on the file.  I still think that you're going to get better results with discrete file handles if you ever need to do a second sequential read() on that portion of the file. YMMV.


> Caching won't help me if I only want to look at each chunk once.  If
> the data was in the file sequentially then
> the built-in kernel readahead would help.  If the file format is fixed
> and I want to process the data in some other order
> then sequential then the simplistic kernel readhead isn't going to
> help (and may make things slower).

Yeah... see... the problem now is the file storage format.  What you really want now is an index into the actual data: find what you want from the index and use that pointer to jump immediately to the data you want instead of having to seek across Ghu knows how much file.  As I said, this has been solved before.

--Rich P.








BLU is a member of BostonUserGroups
BLU is a member of BostonUserGroups
We also thank MIT for the use of their facilities.

Valid HTML 4.01! Valid CSS!



Boston Linux & Unix / webmaster@blu.org