The Linux streams library
Mark Veltzer
mark@veltzer.net
Streams and performance
● Since each FILE structure is held in user space
it could be accessed by two threads at the
same ...
Options for design – best option
● Don't use the same stream in different threads
● If you use the standard stream operati...
Options for design – little contention
● If your application is not doing lots of IO then you
can just go ahead and use th...
Options for design – reducing
contention
● If you do contend on streams then you can
manage the locking yourself and get s...
Locking manually for transactional
IO
● If you have multiple threads that want to access the same
files using the streams ...
References
● man unlocked_stdio from the Linux manual
pages
● 'I/O on streams' from 'info libc'.
Upcoming SlideShare
Loading in …5
×

Streams

897 views

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
897
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Streams

  1. 1. The Linux streams library Mark Veltzer mark@veltzer.net
  2. 2. Streams and performance ● Since each FILE structure is held in user space it could be accessed by two threads at the same time. ● So the streams library locks each FILE to avoid race conditions. ● You must take these locks into consideration when designing a multi-threaded streams using application.
  3. 3. Options for design – best option ● Don't use the same stream in different threads ● If you use the standard stream operation (without the _unlocked suffix) then you get quite good performance. ● This is due to the fact that uncontended locks in Linux are acquired very fast in user space (futex). ● Or, you can go a step further and just use the *_unlocked operations and get no locking at all.
  4. 4. Options for design – little contention ● If your application is not doing lots of IO then you can just go ahead and use the same streams in different threads with little contention. ● Remember that streams buffer in user space and so most of the time you are causing very slight delays (memcpy). ● In this case you must not use the _unlocked versions of the functions.
  5. 5. Options for design – reducing contention ● If you do contend on streams then you can manage the locking yourself and get some more performance. ● Use flockfile(3) and funlockfile(3) and between them use the *_unlocked operations. ● If you have multiple IO operations between each lock and unlock then you are saving the need to acquire and release the lock multiple times.
  6. 6. Locking manually for transactional IO ● If you have multiple threads that want to access the same files using the streams library and you want each to perform several IO operation without being interrupted then you must resort to manual locking. ● Use flockfile(3) and funlockfile(3) and between them use the *_unlocked operations. ● Note that this does not protect you from other processes accessing the same file. ● If you put the FILE structures and their buffers (using setbuffer) into shared memory you can use the streams library for multi-processed designs.
  7. 7. References ● man unlocked_stdio from the Linux manual pages ● 'I/O on streams' from 'info libc'.

×