Os Reindersfinal


Published on

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Os Reindersfinal

    1. 1. Exploiting Parallelism with Multi-core Technologies <ul><li>James Reinders </li></ul><ul><li>Date: Thursday, July 26 </li></ul><ul><li>Time: 2:35pm - 3:20pm </li></ul><ul><li>Location: E142 </li></ul>
    2. 2. Coding with TBB Contest <ul><li>threadingbuildingblocks.org </li></ul><ul><li>Challenge : Integrate TBB in open source projects through 8/31/07 </li></ul><ul><li>Judging: Best implementation and most benefit achieved from using TBB </li></ul><ul><li>Grand Prize : A multi-core laptop and recognition at the upcoming Intel Developers Forum </li></ul><ul><li>Details : threadingbuildingblocks.org </li></ul>
    3. 3. Problem <ul><li>Gaining performance from multi-core requires parallel programming </li></ul><ul><li>Even a simple “parallel for” is tricky for a non-expert to write well with threads. </li></ul><ul><li>Two aspects to parallel programming </li></ul><ul><ul><li>Correctness : avoiding race conditions and deadlock </li></ul></ul><ul><ul><li>Performance : efficient use of resources </li></ul></ul><ul><ul><ul><li>Hardware threads (keep busy with real work) </li></ul></ul></ul><ul><ul><ul><li>Memory space (share between cores) </li></ul></ul></ul><ul><ul><ul><li>Memory bandwidth (efficient cache line use) </li></ul></ul></ul>
    4. 4. Three Approaches for Improvement <ul><li>New language: Cilk, NESL, Haskell, Erlang, Fortress, … </li></ul><ul><li>Language extensions / pragmas: OpenMP </li></ul><ul><ul><li>Easier to get acceptance, but require special compiler </li></ul></ul><ul><li>Library: POOMA, Hood, … </li></ul><ul><ul><li>Easy to use </li></ul></ul><ul><ul><li>Domain specific </li></ul></ul>
    5. 5. Family Tree 1988 2001 2006 1995 Languages *Other names and brands may be claimed as the property of others Cilk space efficient scheduler cache-oblivious algorithms Threaded-C continuation tasks task stealing OpenMP* fork/join tasks OpenMP taskqueue while & recursion Pragmas Chare Kernel small tasks JSR-166 (FJTask) containers Intel® Threading Building Blocks STL generic programming STAPL recursive ranges ECMA .NET* parallel iteration classes Libraries
    6. 6. Enter Intel® Threading Building Blocks <ul><li>2 years from conception to 1.0 beta </li></ul><ul><ul><li>Born from a meeting of Principal Engineers who wanted to: </li></ul></ul><ul><ul><ul><li>Ease parallel programming for variable numbers of cores </li></ul></ul></ul><ul><ul><ul><li>Build an accessible technology to encourage adoption </li></ul></ul></ul><ul><ul><ul><li>Take advantage of existing research to meld ease of conception with performance </li></ul></ul></ul><ul><li>Version 1.0 launched August 2006 </li></ul><ul><li>Version 1.1 launched April 2007 </li></ul><ul><li>Version 2.0 Intel provides Threading Building Blocks as an Open Source project with more ports done, and more we can work on together. threadingbuildingblocks.org </li></ul>
    7. 7. Example (just a peek) void ApplyFoo( size_t n , int x ) { for(size_t i=range_begin; i<range_end; ++i) Foo( i,x ); } SERIAL VERSION void ParallelApplyFoo(size_t n, int x) { parallel_for( blocked_range<size_t>(0,n,10), <>(const blocked_range<size_t>& range) { for(size_t i=range.begin(); i<range.end(); ++i) Foo(i,x); } ); } PARALLEL VERSION (the way I wish I could write it)
    8. 8. Parallel Version (as it can be written) class ApplyFoo { public: int my_x; ApplyFoo( int x ) : my_x(x) {} void operator()(const blocked_range<size_t>& range) const { for(size_t i=range.begin(); i!=range.end(); ++i) Foo(i,my_x); } }; void ParallelApplyFoo(size_t n, int x) { parallel_for(blocked_range<size_t>(0,n,10), ApplyFoo(x)); }
    9. 9. Underlying concepts
    10. 10. Generic Programming <ul><li>Best known: C++ Standard Template Library </li></ul><ul><li>Write best possible algorithm - most general way </li></ul><ul><ul><li>parallel_for does not require specific type of iteration space, but only that it have signatures for recursive splitting </li></ul></ul><ul><li>Instantiate algorithm to specific situation </li></ul><ul><li>Enables distribution of broadly-useful high-quality algorithms and data structures </li></ul>
    11. 11. Key Features of Intel Threading Building Blocks <ul><li>Manage work as divisible tasks instead of threads </li></ul><ul><ul><li>Intel TBB maps such tasks onto physical threads </li></ul></ul><ul><ul><ul><li>Solving cache issues and load balancing for us </li></ul></ul></ul><ul><ul><li>Full support for nested parallelism </li></ul></ul><ul><li>Targets threading for robust performance </li></ul><ul><ul><li>portable, scalable perf. for computationally intense portions </li></ul></ul><ul><li>Interoperable with other threading packages </li></ul><ul><li>Emphasizes scalable, data parallel programming </li></ul>
    12. 12. Relaxed Sequential Semantics <ul><li>TBB emphasizes relaxed sequential semantics </li></ul><ul><li>Parallelism as accelerator , not mandatory for correctness. </li></ul>
    13. 13. Synchronization Primitives atomic, spin_mutex, spin_rw_mutex, queuing_mutex, queuing_rw_mutex, mutex Generic Parallel Algorithms parallel_for parallel_while parallel_reduce pipeline parallel_sort parallel_scan Concurrent Containers concurrent_hash_map concurrent_queue concurrent_vector Task scheduler Memory Allocation cache_aligned_allocator scalable_allocator
    14. 14. Serial Example static void SerialUpdateVelocity() { for( int i=1; i<UniverseHeight-1; ++i ) for( int j=1; j<UniverseWidth-1; ++j ) V[i][j] += (S[i][j] - S[i][j-1] + T[i][j] - T[i-1][j])*M[i]; }
    15. 15. Parallel Version blue = original code red = provided by TBB black = boilerplate for library struct UpdateVelocityBody { void operator()( const blocked_range <int>& range ) const { int end = range.end (); for( int i= range.begin (); i<end; ++i ) { for( int j=1; j<UniverseWidth-1; ++j ) { V[i][j] += (S[i][j] - S[i][j-1] + T[i][j] - T[i-1][j])*M[i]; } } } void ParallelUpdateVelocity() { parallel_for ( blocked_range<int> ( 1, UniverseHeight-1), UpdateVelocityBody(), auto_partitioner() ); } Task Parallel control structure Task subdivision handler
    16. 16. Range is Generic <ul><li>Requirements for parallel_for Range </li></ul><ul><li>Library provides blocked_range and blocked_range2d </li></ul><ul><li>You can define your own ranges </li></ul><ul><li>Partitioner calls splitting constructor to spread tasks over range </li></ul>Destructor R::~R() True if range is empty bool R::empty() const True if range can be partitioned bool R::is_divisible() const Split r into two subranges R::R (R& r, split ) Copy constructor R::R (const R&)
    17. 17. parallel_reduce parallel_scan parallel_while parallel_sort pipeline
    18. 18. Parallel pipeline <ul><li>Linear pipeline of stages </li></ul><ul><ul><li>specify maximum number of items that can be in flight </li></ul></ul><ul><ul><li>handling arbitrary DAG can take thought but is doable </li></ul></ul><ul><li>Each stage can be serial or parallel </li></ul><ul><ul><li>Serial stage = one item at a time, in order. </li></ul></ul><ul><ul><li>Parallel stage = multiple items at a time, out of order. </li></ul></ul><ul><li>Uses cache efficiently </li></ul><ul><ul><li>Each thread carries an item through as many stages as possible. </li></ul></ul><ul><ul><li>Biases towards finishing old items before tackling new ones. </li></ul></ul>
    19. 19. Parallel pipeline Parallel stage scales because it can process items in parallel or out of order. Serial stage processes items one at a time in order. Another serial stage. Items wait for turn in serial stage Controls excessive parallelism by limiting total number of items flowing through pipeline. Uses sequence numbers recover order for serial stage. Tag incoming items with sequence numbers Throughput limited by throughput of slowest serial stage. 1 3 2 4 5 6 7 8 9 10 11 12
    20. 20. Concurrent Containers, Mutual Exclusion Memory Allocator Task Scheduler
    21. 21. Concurrent Containers <ul><li>Intel TBB provides concurrent containers </li></ul><ul><ul><li>STL containers are not safe under concurrent operations </li></ul></ul><ul><ul><ul><li>attempting multiple modifications concurrently could corrupt them </li></ul></ul></ul><ul><ul><li>Standard practice: wrap a lock around STL container accesses </li></ul></ul><ul><ul><ul><li>Limits accessors to operating one at a time </li></ul></ul></ul><ul><li>TBB provides fine-grained locking and lockless operations where possible </li></ul><ul><ul><li>Worse single-thread performance, but better scalability. </li></ul></ul><ul><ul><li>Can be used with TBB, OpenMP, or native threads. </li></ul></ul>
    22. 22. Concurrent interface requirements <ul><li>Some STL interfaces are not designed to support concurrency </li></ul><ul><ul><li>For example, suppose two threads each execute: </li></ul></ul><ul><li>Solution: Intel TBB concurrent_queue has pop_if_present() </li></ul>extern std::queue q; if(!q.empty()) { item=q.front(); q.pop(); } At this instant, another thread might pop last element.
    23. 23. concurrent_vector <T> <ul><li>Dynamically growable array of T </li></ul><ul><ul><li>grow_by (n) </li></ul></ul><ul><ul><li>grow_to_at_least (n) </li></ul></ul><ul><li>Never moves elements until cleared </li></ul><ul><ul><li>Can concurrently access and grow </li></ul></ul><ul><ul><li>Method clear() is not thread-safe with respect to access/resizing </li></ul></ul>// Append sequence [begin,end) to x in thread-safe way. template<typename T> void Append( concurrent_vector <T> &x, const T *begin, const T *end ) { std::copy(begin, end, x.begin() + x. grow_by (end-begin) ) } Example
    24. 24. concurrent_queue <T> <ul><li>Preserves local element order </li></ul><ul><ul><li>If one thread pushes and another thread pops two values, they come out in the same order as they went in. </li></ul></ul><ul><li>Two kinds of pops </li></ul><ul><ul><li>blocking </li></ul></ul><ul><ul><li>non-blocking </li></ul></ul><ul><li>Method size() returns signed integer </li></ul><ul><ul><li>If size() returns – n, it means n pops await corresponding pushes. </li></ul></ul><ul><li>Caution: queues are cache coolers </li></ul>
    25. 25. concurrent_hash <Key,T,HashCompare> <ul><li>Associative table allows concurrent access for reads and updates </li></ul><ul><ul><ul><li>bool insert ( accessor &result, const Key &key) to add or edit </li></ul></ul></ul><ul><ul><ul><li>bool find ( accessor &result, const Key &key) to edit </li></ul></ul></ul><ul><ul><ul><li>bool find ( const_accessor &result, const Key &key) to look up </li></ul></ul></ul><ul><ul><ul><li>bool erase ( const Key &key) to remove </li></ul></ul></ul><ul><li>Reader locks coexist – writer locks are exclusive </li></ul>
    26. 26. struct MyHashCompare { static long hash ( const char* x ) { long h = 0; for( const char* s = x; *s; s++ ) h = (h*157)^*s; return h; } static bool equal ( const char* x, const char* y ) { return strcmp(x,y)==0; } }; typedef concurrent_hash_map <const char*,int,MyHashCompare> StringTable; StringTable MyTable; Example: map strings to integers void MyUpdateCount( const char* x ) { StringTable::accessor a; MyTable. insert ( a, x ); a->second += 1; } Multiple threads can insert and update entries concurrently. accessor object acts as smart pointer and writer lock.
    27. 27. Synchronization Primitives <ul><li>Shared Data </li></ul><ul><ul><li>use mutual exclusion to avoid races </li></ul></ul><ul><li>TBB mutual exclusion regions are protected by scoped locks </li></ul><ul><ul><li>The range of the lock is determined by its lifetime (scope) </li></ul></ul><ul><ul><li>Leaving lock scope calls the destructor, making it exception safe </li></ul></ul><ul><ul><li>Minimizing lock lifetime avoids possible contention </li></ul></ul><ul><li>Several mutex behaviors are available </li></ul><ul><ul><li>Spin vs. queued </li></ul></ul><ul><ul><li>Writer vs. reader/writer </li></ul></ul><ul><ul><li>Scoped wrapper of native mutual exclusion function </li></ul></ul>
    28. 28. Example: spin_rw_mutex promotion <ul><li>spin_rw_mutex MyMutex; </li></ul><ul><li>int foo () { </li></ul><ul><li>spin_rw_mutex::scoped_lock lock (MyMutex, /*is_writer*/ false); </li></ul><ul><li>… </li></ul><ul><li>if (!lock. upgrade_to_writer ()) </li></ul><ul><li>{ /* reacquire state */ } </li></ul><ul><li>/* perform desired write */ </li></ul><ul><li>return 0; </li></ul><ul><li>/* Destructor of ‘lock’ releases ‘MyMutex’ */ </li></ul><ul><li>} </li></ul><ul><li>Exceptions occurring within the locked code range automatically release the lock ( lock passes out of scope), avoiding deadlock </li></ul><ul><li>Any reader lock may be upgraded to writer lock; upgrade_to_writer() fails if the lock had to be released before it can be locked for writing </li></ul>
    29. 29. Scalable Memory Allocator <ul><li>Problem </li></ul><ul><ul><ul><li>Memory allocation can bottle-neck a concurrent environment </li></ul></ul></ul><ul><ul><ul><ul><ul><li>Thread allocation from a global heap requires global locks. </li></ul></ul></ul></ul></ul><ul><ul><li>Solution </li></ul></ul><ul><ul><ul><li>Intel® Threading Building Blocks provides tested, tuned, scalable, per-thread memory allocation </li></ul></ul></ul><ul><ul><ul><li>Scalable memory allocator interface can be used… </li></ul></ul></ul><ul><ul><ul><ul><li>As an allocator argument to STL template classes </li></ul></ul></ul></ul><ul><ul><ul><ul><li>As a replacement for malloc/realloc/free calls (C programs) </li></ul></ul></ul></ul><ul><ul><ul><ul><li>As a replacement for new and delete operators (C++ programs) </li></ul></ul></ul></ul>
    30. 30. Task Scheduler <ul><li>Underlying the generic task structure is a task scheduler </li></ul><ul><li>Core to task scheduling is a thread pool whereby Intel TBB maximizes thread efficiency and manages its complexity </li></ul><ul><li>Task scheduler is designed to address common performance issues of parallel programming with native threads </li></ul>TBB Approach Problem Task chunking and work-stealing help balance load Load imbalance Programmer specifies tasks, not threads. Program complexity “ Greedy” scheduling often wins “ Fair” scheduling One scheduler thread per hardware thread Oversubscription
    31. 31. Task Scheduler <ul><li>Intel TBB task interest is managed in the task_scheduler_init object </li></ul><ul><li>Thread pool construction also tied to the life of this object </li></ul><ul><ul><li>Nested construction is reference counted, low overhead </li></ul></ul><ul><ul><li>Put Init object scope high to avoid pool reconstruction overhead </li></ul></ul><ul><ul><li>Construction specifies thread pool size automatic , explicit or deferred </li></ul></ul><ul><ul><li>Dynamic init object lifetime management offers thread pool size control </li></ul></ul>#include “tbb/task_scheduler_init.h” using namespace tbb; int main() { task_scheduler_init init; … . return 0; }
    32. 32. Tasking Development tools <ul><li>Intel TBB offers facilities to accelerate development </li></ul><ul><ul><li>Linking with libtbb_debug.so (or Win/Mac equivalents) adds checking </li></ul></ul><ul><ul><li>TBB_DO_ASSERT macro extends checking into the header/inline code </li></ul></ul><ul><ul><li>TBB_DO_THREADING_TOOLS adds hooks for Intel Thread Analysis tools </li></ul></ul><ul><ul><li>The tick_count class offers convenient timing services. </li></ul></ul><ul><ul><ul><li>tick_count:: now () returns current timestamp </li></ul></ul></ul><ul><ul><ul><li>tick_count::interval_t:: operator- (const tick_count &t1, const tick_count &t2) </li></ul></ul></ul><ul><ul><ul><li>double tick_count::interval_t:: seconds () converts intervals to real time </li></ul></ul></ul><ul><ul><li>ITT_NOTIFY events can be useful with Intel Thread Analysis tools </li></ul></ul>
    33. 33. open source tour
    34. 34. Open Source – quick ‘tour’ <ul><li>Source library organized around 4 directories </li></ul><ul><ul><ul><li>src – C++ source for Intel TBB, TBBmalloc and the unit tests </li></ul></ul></ul><ul><ul><ul><li>include – the standard include files </li></ul></ul></ul><ul><ul><ul><li>build – catchall for platform-specific build information </li></ul></ul></ul><ul><ul><ul><li>examples – TBB sample code </li></ul></ul></ul><ul><li>Top level index.html offers help on building and porting </li></ul><ul><ul><li>Build prerequisites: </li></ul></ul><ul><ul><ul><li>C++ compiler for target environment </li></ul></ul></ul><ul><ul><ul><li>GNU make </li></ul></ul></ul><ul><ul><ul><li>Bourne or BASH-compatible shell </li></ul></ul></ul><ul><ul><ul><li>Some architectures may require an assembler for low-level primitives </li></ul></ul></ul>
    35. 35. Coding with TBB Contest <ul><li>Challenge : Integrate TBB in open source projects through 8/31/07 </li></ul><ul><li>Judging: Best implementation and most benefit achieved from using TBB </li></ul><ul><li>Grand Prize : A multi-core laptop and recognition at the upcoming Intel Developers Forum </li></ul><ul><li>Details : threadingbuildingblocks.org </li></ul>
    36. 36. Learn more… <ul><li>INTEL booth (today until 5pm!) many Intel engineers – come see us! </li></ul><ul><li>threadingbuildingblocks.org </li></ul>