1. A Pattern System forA Pattern System for
Enumeration ProcessingEnumeration Processing
Expression and ParallelisationExpression and Parallelisation
Jean-Lin Pacherie
6. MotivationsMotivations
Parallel Programming, Why?
The solution should conforms
with usual programming
environment
Constraints for the solution
Standard compilation tools
No language extensions
Parallel Programming, How?
12. RequirementRequirement
Knowledge and Reusability
Data Parallelism
Context : Data collection processing
1) Cut-out
From iteration start until iteration end loop
action ( current item )
next iteration
end
Collection :
Aggregation of
homogeneous items
14. RequirementsRequirements
Knowledge and Reusability
Data Parallelism
Data Driven Distribution
Parallel Code
Collection
Sequential Code
?
Code Proc 2Code Proc 2
Code Proc 1Code Proc 1
Code Proc 3Code Proc 3
Data Layout
Owner Compute rule
Remote Data Refreshing
SPMD
Principles
15. ThesisThesis
Design patterns for data parallelism
programming
Identify and organize the design pattern dedicated
to data collection processing
Apply the DDD mechanism to these patterns:
Parallelization at the software architecture level
Use the design patterns of serialUse the design patterns of serial
applications to introduce parallelismapplications to introduce parallelism
16. Design Model for EnumerationDesign Model for Enumeration
ProcessingProcessing
Operator Design Pattern
Consequences
Introduction
Design Model for ProcessingDesign Model for Processing
Pattern System
Impact of the DDD
Fundamental Services for Data Parallelism
Conclusion
17. Towards a pattern...Towards a pattern...
Description of behavior
Behavior vs.. Implementation
Instaciation delegation trough the Factory Method pattern
AbstractClient
FactoryMethod : AbstractProduct
AnOperation
AbstractProduct
Service
…
product := FactoryMethod
product.service
...
ConcreteClient
FactoryMethod :
ConcreteProduct
ConcreteProduct
Servic
e
!ConcreteProduct!Result
19. Factory Pattern
Extracting iteration domain through the Iterator pattern
ArrayIteratorArray
ConcreteClient
Independence from data structures
AbstractClient
Collection Iterator
collection := make_collection
iteration := collection.make_iterator
from iteration.start until iteration.exhausted loop
…
iteration.next
end
Towards a pattern...Towards a pattern...
Description of behavior
21. Extracting control structure
AbstractClient
Collection Operator
collection := make_collection
iteration := collection.make_iterator
from iteration.start until iteration.exhausted loop
…
iteration.next
end
Iterator
Extracting iteration domain through the Iterator pattern
Towards a pattern...Towards a pattern...
Description of behavior
22. AbstractClient
Collection Operator
collection := make_collection
iteration := collection.make_iterator
operator.attach(iteration)
operator.run
Iterator Operator
Extracting control structure
Extracting iteration domain through the Iterator pattern
Towards a pattern...Towards a pattern...
Description of behavior
26. Operators ?
Enumeration Free Monoïd [ A* , ( ) , . ]
Operator Morphism toward [ P , e , + ]
••••[A*, ( ) , . ]
••••[P*, ( ) , . ]
+ + + +[ P , e , + ]
R
Valuation +Valuation +
List of A
List of P
. +
f : A P
Redefining implementation
ConsequencesConsequences
Separation of the responsibilities
28. Pattern SystemPattern System
Designing collections and iterators
Expressing Operators
Using Enumerations
Introduction
Design Model for Processing
Pattern SystemPattern System
Impact of the DDD
Fundamental Services for Data Parallelism
Conclusion
29. MethodMethod
Identification of significant characteristics
Accessor
Capacity
Indexed
Hashed
LinearIndexed
Dictionary Bounded
Dynamic
FIFO
FILO
Classification with families
Relations between characteristics
30. Specification of abstractions (Collections & Iterators)
A
B
C
a
b
c
Abstraction
MethodMethod
Identification of significant characteristics
Classification with families
Relations between characteristics
31. Application to collectionsApplication to collections
deferred class ACCESSOR
feature
General definitions
end
deferred class ACCESSOR
feature
General definitions
end
FamilyFamily
Families:
Abstract classes
Characteristic
deferred class INDEXED [ITEM,INDEX]
inherit
ACCESSOR
feature
read ( i : INDEX ) : ITEM is
deferred end
write ( i : INDEX ; e : ITEM ) is
deferred end
end
deferred class INDEXED [ITEM,INDEX]
inherit
ACCESSOR
feature
read ( i : INDEX ) : ITEM is
deferred end
write ( i : INDEX ; e : ITEM ) is
deferred end
end
Characteristics:
Progressive specialization
deferred class COLLECTION [ ITEM ]
inherit
ACCESSOR
CAPACITY
../.. -- Other families
feature ../..
end
deferred class COLLECTION [ ITEM ]
inherit
ACCESSOR
CAPACITY
../.. -- Other families
feature ../..
end
collection
AccesseurAccesseur
capacitycapacity
Collections:
Abstract classes using
families and specialization
using characteristics
33. Building an OperatorBuilding an Operator
Operator properties
• Items type
• First order function
• Second order function
• Type of enumeration
Implementation • Generic type
• Virtual method (deferred)
• Classification (BMF, etc.)
• Iterator type
34. Pattern for IteratorsPattern for Iterators
Composable operator and Iterator
Compositions
Factorization
static
dynamic
Multiplexer Demultiplexer
35. Impact of the DDDImpact of the DDD
Deriving the operator pattern
Equivalence Constraint
Introduction
Design Model for Processing
Pattern System
Impact of the DDDImpact of the DDD
Fundamental Services for Data Parallelism
Conclusion
36. Deriving the Operator patternDeriving the Operator pattern
Reminder : Operator structure
Redefinition of collections : data distribution
Parallelisation of processing
AbstractClient
AbstractCollection OperatorIterator
collection := make_collection
iteration := collection.make_iterator
operator.attach(iteration)
operator.run
ConcreteOperator
ParaOperator
ConcreteIterator
SpreadIterator
SpreadCollection
Enumeration of
local data
Enumeration of
remote data
37. Conformity constraints
Spread collections and Local collections
Sequential and Parallel Operators
Sequential and Parallel execution
Deriving the Operator patternDeriving the Operator pattern
Reminder : Operator structure
38. Derivation
Factory method : select distributed services
Invocation : local computations
ParaOperator
Spread_Collection
ParaClient
make_collection : Spread_Collection
make_operation : ParaOperator
SeqClient
make_collection : ACollection
AnOperation
make_operation : AnOperator
ConcreteOperator
Collection
collection := make_collection
operator := make_operator(collection)
operator.run
make_operation ( collection) : AnOperator is
do
!!Result
Result.attach(Collection.items)
end
make_operation ( collection) : ParaOperator is
do
!!Result
Result.attach(Collection.local_items)
end
Deriving the Operator patternDeriving the Operator pattern
Reminder : Operator structure
Conformity constraints
39. Parallel Evaluation SchemeParallel Evaluation Scheme
[A*, () , . ]
[ P , e , + ]
F
[ A**, () , . ][ A**, () , . ]
DistDist
[ P*, () , . ]
F*
Aggregation
• Morphism (f,e,+)
• Distribution
Data :
Intermediate :
• Computation of the
local contributions
Result :
• Aggregation of the
local contributions
41. Particular situations
Propriety of morphisms
Commutative operator
Morphism
+F ( ) F ( )
+{F ( ) F ( )} F ( )+
F ( ) F ( ) F ( )// Evaluation
Aggregation
Order of aggregation = initial order
Is there a problem???
Equivalence ConstraintEquivalence Constraint
42. F ( ) F ( ) F ( )//Evaluation
Inner composition rule is commutative
& : Commutative
&F ( ) F ( )
&{F ( ) F ( )} F ( )&
Aggregation
Order of aggregation =/= initial order
&F ( ) F ( )=?=
Particular situations
Propriety of morphisms
Is there a problem???
Equivalence ConstraintEquivalence Constraint
43. General situation
An operator defines a partial order
The distribution layout implements this order
Warning : collaboration between operator and layout manager
Particular situations
Is there a problem???
Equivalence ConstraintEquivalence Constraint
44. Fundamental Services for DataFundamental Services for Data
ParallelismParallelism
Spreading of the Collections:
Spread Layout
Model of Memory and Communication
Introduction
Design Model for Processing
Pattern System
Impact of the DDD
Fundamental Services for Data ParallelismFundamental Services for Data Parallelism
Conclusion
45. Spreading of the CollectionsSpreading of the Collections
Proc
Collection
P1
P0 P2
Instance
P1
Instance
P2
Instance
P0
Conformity
Ubiquity of spreaded collections - Local Collection
46. Conformity
Separation of concerns
Spread model
Invocation
Id
Key
Owner
Layout
Proc 1 Proc 2
Layout
Dynamic DistributionDynamic DistributionLocal SharingLocal Sharing
Spreading of the CollectionsSpreading of the Collections
47. Memory model
Shared Memory
Local Storage
Shared Collections
Conformity
Separation of concerns
Spreading of the CollectionsSpreading of the Collections
Spread model
48. Local memoryLocal memory
Local memory
Local memory
Local copy Original
Distributed Collection
Memory model
Conformity
Separation of concerns
Spreading of the CollectionsSpreading of the Collections
Spread model
49. Redefinition of the accessors
Granularity
Fundamental
Behavior
• read
• write
Use case
• sequential (SPMD)
• parallel
Interpretation
• local
• remote (distributed, shared, etc.)
• atomic
• enumerative
Conformity
Separation of concerns
Spreading of the CollectionsSpreading of the Collections
52. ExperimentationExperimentation
Linear Algebra
Genome
Forest Simulation
Contract Intel SSD
Collaboration D. Lavenier
Collaboration M.-A. Moravie
Laboratory of Biometrics Lyon
N-Body Application derivation
Shared Memory API User, GC extensions
Paragon, Origin2000,
PowerChallenge
Origin2000, Clusters
Paragon
Clusters (Myrinet, ATM),
Origin2000
Distributed Memory API User (POM) Origin2000, Myrinet, etc.
Theme Context Plate form
NoW
53. AssessmentAssessment
Pattern system for enumeration processing
Application of the patterns for data parallelism
Patterns dedicated to data parallelism
Method to design collections, iterators and operators
54. PerspectivesPerspectives
Research :
Industry :
Integration of research in the area of functional programming
Validation and conformity of parallel code
Dealing with data dependencies
A pattern system for data collection processing
Library for data collection processing (C++ & STL, Java & JGL, etc.)
Toward an automatic production of the code for parallel patterns
