The Concurrent Constraint Programming Research Programmes -- Redux

IBM Research: Computing as a Service
The Concurrent Constraint Programming
Research Programmes -- Redux
The use of constraints for communication and
control in concurrent programming
Vijay Saraswat
<firstname>@<lastname>.org
IBM TJ Watson
Sep 9, 2014
© 2005 IBM Corporation Computing as a Service 1

CCP Research Programmes
Constraint Programming
 Constraint Programming is programming with
(probabilistic) partial information, using logic-based
combinators
– contra functional programming
 Constraint Programming is for general-purpose
application programming
– Not just constraint solving, combinatorial problem solving
© 2009 IBM Corporation IBM Research
2
Thesis: Time has come to design a new general-purpose
constraint programming language for probabilistic analytic
applications involving big data. Its been 30 years since
Prolog!

Acknowledgements
Martin Rinard, Prakash Panagaden, Ken Kahn, Jacob
Levy, Saumya Debray, Clifford Tse, Radha Jagadeesan,
Vineet Gupta, Patrick Lincoln, Mary Dalrymple, Bjorn
Carlson, Markus Fromherz, Pascal van Hentenryck, Yves
Deville, Yumi Iwasaki, Adam Farquhar, Danny Bobrow,
Catuscia Palamidessi, Frank Valencia, Francesca Rossi,
Gopalan Nadathur
C10 project in collaboration with Francesca Rossi, Daniel
Diaz, Salvador Abreu, Radha Jagadeesan, Vineet Gupta,
Prakash Panangaden, Philippe Codognet
4

Agenda
 Where have these years gone?
 CCP Research – selected review
 C10
5

The past twenty Years
6

Cloud
Mobile Social
Research Analytics
IBM © 2014 International Business Machines Corporation
© 2009 IBM Corporation The CAMS Mega-trends
Based fundamentally on scale-out architectures

Scale out architectures Tianhe-2:
18,688 16-core AMD 6274
+ Tesla K20X 2688-core
GPUs
~ 50M cores
GigaFlops (109)
Asychrony
• async S
Research
Locality
• at (P) S
IBM © 2009 IBM Corporation Atomicity
• when (c) S
Order
• finish S
• clocks
Global data-structures
points, regions,
distributions,
arrays

Big Data: More than just volume
© 2009 IBM Corporation IBM Research © 2014
International
Business
Volume
Terabytes to
exabytes of existing
data to process
Velocity
Streaming data,
milliseconds to
seconds to
respond
Variety
Structured,
unstructured,
text & multimedia
Veracity
Uncertainty from
inconsistency,
ambiguities,
etc.
Spurred the development of resilient, parallel application
frameworks – Hadoop, Pregel, MillWheel, …

Net flix Challenge
Predict user ratings
for films based on
previous ratings
Cyber and Ot her
Siri
Voice recognition and
Natural Language
Processing (NLP)
Research IBM 8
10
© 2009 IBM Corporation Machine Learning is Ubiquitous and Very Useful
I SR Natural Language Processing Predict ive Analyt ics
Y a h o o ’ s B a y e s i a n S p a m F i l t e r i n g: Self-adapting
system based on word probabilities
C y b e r G e n o m e L i n e a g e: Reverse engineer
malware samples to find shared “genetic"
features between different malware samples
G e n e S e q u e n c i n g: Determine order of
nucleotides in a DNA molecule
N u c l e a r T e s t B a n T r e a t y C o m p l i a n c e: Deduce
set of seismic events given detections and
misdetections on a network of stations
I m a g e S e a r c h / A c t i v i t y D e t e c t i o n: Find and
identify objects and actions in video
O b j e c t T r a c k i n g: Follow vehicles as they
move through a city and are recorded in
multiple video streams (DARPA CZTS)
P a t t e r n s o f L i f e: Process wide area aerial
surveillance data and associated tracks to
infer location types and object dynamics
B i r d M i g r a t i o n P a t t e r n s: Model spatio-temporal
distribution of birds (by species);
involves large-scale sensor integration
D A R P A L A G R: Vision-based robot navigation
G o o g l e G l a s s e s : Perform searches based on
images taken by user cell phone cameras
M i c r o s o f t M a t c h b o x: Match players based on
their gaming skill set
P r e d i c t i v e D a t a b a s e: Understand information
based on causal relationships in data
B i n g I m a g e S e a r c h: Search for images on the
web by selecting text in word document
A m a z o n R e c o m m e n d a t i o n E n g i n e:
Recommend items based on consumer data
DARPA Grand
Challenge
Fully autonomous
ground vehicles
competition
ORNL’s At t ack Variant
Det ector: Discover
compromised systems
W a t s o n: Computer system capable of
answering questions posed in natural
language
T o p i c M o d e l s: Statistical model for
discovering the abstract "topics" that occur in
a collection of documents
D i s t r i b u t e d T o p i c M o d e l s: Asynchronous
distributed topic discovery
C i t a t i o n A n a l y s i s: Given citations, extract
author, title, and venue strings and identify
co-reference citations
E n t i t y R e s o l u t i o n: Discovering entities that
are mentioned in a large corpus of news
articles
N L P S e q u e n c e T a g g i n g: Tagging parts of
speech and recognizing named entities in
newspaper articles
.....
© Apple
© Netflix
D i s c l a i m e r : I m a g e s o f s p e c i f i c p r o d u c t s a r e u s e d f o r i l l u s t r a t i o n o n l y . U s e o f t h e s e
Ack: DARPA slides from Kathleen Fisher
i m a g e s d o e s n o t i m p l y e n d o r s e m e n t o f i n h e r e n t t e c h n i c a l v u l n e r a b i l i t i e s .
Approved for Public Release; Distribution Unlimited

11
We’re Missing a Tool to Write These Applications
I SR
1. Seismic Monitoring for
Nuclear Test Ban Treaty
2. Image Search and
Activity Detection
PerSEAS, VIRAT,
VMR
3. Object tracking (video) PerSEAS, VIRAT,
CZTS
4. Patterns of Life ONR
5. Bird Migration Patterns
Natural Language Processing
6. Watson
7. Topic Models CALO
8. Distributed Topic Models CALO
9. Citation Analysis BOLT, DEFT,
MADCAT, RATS
10. Entity Resolution BOLT, DEFT,
MADCAT
11. NLP Sequence Tagging BOLT, DEFT,
MADCAT
Predict ive Analyt ics
12. Microsoft Matchbox
13. Netflix Challenge
14. Predictive Database XDATA
Cyber and Ot her
15. Cyber Genome Lineage Cyber Defense
16. Gene Sequencing
Too Complex to Imagine
Requires
Heroic Effort
Inexpressible with Current Tools
Approved for Public Release; Distribution Unlimited

The Promise of Probabilistic Programming Languages
Sources:
• Bayesian Data Analysis, Gelman, 2003
• Pattern Recognition and Machine Learning,
Bishop, 2007
• Science, Tanenbaum et al, 2011
Research IBM 15
12
© 2009 IBM Corporation • Shorter: Reduce LOC by 100x for machine learning applications
• Seismic Monitoring: 28K LOC in C vs. 25 LOC in BLOG
• Microsoft MatchBox: 15K LOC in C# vs. 300 LOC in Fun
• Faster: Reduce development time by 100x
• Seismic Monitoring: Several years vs. 1 hour
• Microsoft TrueSkill: Six months for competent developer vs. 2 hours with Infer.Net
• Enable quick exploration of many models
• More I nformat ive: Develop models that are 10x more sophisticated
• Enable surprising, new applications
• Incorporate rich domain-knowledge
• Produce more accurate answers
• Require less data
• Increase robustness with respect to noise
• Increase ability to cope with contradiction
• With less expert ise: Enable 100x more programmers
• Separate the model (the program) from the solvers (the compiler),
enabling d o m a i n e x p e r t s without machine learning PhDs to write applications
Probabilistic Programming could empower domain experts a n d ML experts
DISTRIBUTION STATEMENT F. Further dissemination only as directed by DARPA, (February 20, 2013) or higher DoD authority.

How Watson Works: DeepQA Architecture
Answer
Scoring
Model
s
Research Responses
with
Confidence
IBM © 2014 International Business Machines Corporation
© 2009 IBM Corporation Inquiry
Evidenc
e
Sources
Model
s
Model
s
Model
s
Model
s
Model
s
Primary
Search
Candidate
Answer
Generation
Hypothesis
Generation
Hypothesis and Evidence
Scoring
Final Confidence
Merging &
Ranking
Synthesis
Answer
Sources
Inquiry/Topi
c Analysis
Evidence
Retrieval
Deep
Evidence
Scoring
Learned Models
help combine and
weigh the Evidence
Hypothesis
Generation
Hypothesis and Evidence
Scoring
Inquiry
Decompositio
n
1000’s of
Pieces of Evidence
Multiple
Interpretations
of a question
100,000’s Scores from
many Deep Analysis
Algorithms
100’s
sources
100’s Possible
Answers
Balance
& Combine
Another new dawn for AI…?

14

Constraint system
 A constraint system is a set of
formulas (with first-order structure)
with an entailment relation: c1,…, cn
|- c that satisfies certain obvious
properties
– Reflexivity
– Transitivity
– Cylindric properties
 Gentzen constraint system:
– A1,…, An |- A iff A=Ai
Research  Herbrand
– Un-interpreted first-order terms
(labeled, fixed-arity trees)
IBM – f(s,…,s)=g(t,…,t) iff
1m1nf=g, m=n, and s=iti
15
© 2009 IBM Corporation  Other examples: finite domains,
propositional logic (SAT), quantified
SAT, Arithmetic constraints (Naïve,
linear, nonlinear), Interval
arithmetic, Orders, Temporal
Intervals, Hash-tables, Arrays,
Graphs, …
 Constraint systems (as systems of
partial information) are ubiquitous
in computer science
– Type systems
– Compiler analysis
– Symbolic computation
– Concurrent system analysis

Concurrent Constraint Programming
 Use constraints for
communication and control
between concurrent agents
operating on a shared store.
 Two basic operations
– Tell c: Add c to the store
– Ask c then A: If the store is
strong enough to entail c,
reduce to A.
– (also hiding)
(Agents) A ::= c
if (c) A
A,B
{x:T, A}
(proc calls)
Research IBM Saraswat 89; POPL 87, POPL 90, POPL 91
16
© 2009 IBM Corporation

Operational Semantics
Configuration
(Config) Γ ::= A,…, A (multi-set of agents)
Γ, {val x:T, A}  Γ,A (x not free in Γ)
Γ, {A, B}  Γ,A,B
Γ,c1,…,cn,if (c) A  Γ,A (c1,…,cn |- c)
LHS disjunction handled similarly
Fundamental Theorem of CCP:
Execution of A leads to addition of c to store iff L[[ A]] |- c.
[[A]] = function mapping initial store to final store (or limit)
Denotational Semantics via closure operators.
Parallel composition represented by set intersection  semantic reflection
of the simplicity of constraint-based communication

In this section we introduce the basic structure of a MiniZinc model using two simple exam-ples.
Propagators are agents
propagators, indexicals,
N=3, WA:1..N, NSW:1..N, NT:1..N, V:1..N, SA:1..N, T:1..N, Q:1..N,
WA != NT, WA!=SA, NT != SA, NT != Q, SA != Q, SA != NSW,
SA != NSW, SA != V, Q != NSW, NSW != v |- ?
Research WA=1, NT=2, SA=3, QA=1, NSW=2, V=1, T=1 ;
…
no
propagator =>
all (X:1..3) if (X!=1,X !=2) X=3
IBM 18
© 2009 IBM Corporation 2.1 Our First Example
Figure 1: Australian states.
As our first example, imagine that we wish to colour a map of Australia as shown in
Figure 1. It ismade up of seven different states and territories each of which must be given
a colour so that adjacent regions have different colours.
We can model this problem very easily in MiniZinc. Themodel is shown in Figure 2. The
first line in the model is a comment. A comment starts with a ‘%’ which indicates that the
rest of the line is a comment. MiniZinc has no begin/ end comment symbols (such as C’s / *
and * / comments).
The next part of the model declares the variables in the model. The line
3
(From Mini-Zinc tutorial)
cc(FD) – Hentenryck, Saraswat, Deville 1994

RCC – Integrating CCP and logic programming
 CCP performs recursive computations “on the left”, logic
programming “on the right” – can we combine the two?
 Key idea: View an Intuitionistic Logic (IL) sequent A1,…, An |- G as
asserting “the system of concurrently executing agents Ai can pass
the test G”
 Problem: How do we make sense of (Id) A |- A?
 Solution: Restrict agents and tests to communicate only through
constraints! Replace with c1,…,cn |- c (if this holds in the underlying
constraint system)
Research
 Hence, Disjoint Vocabulary Condition: agents, tests, constraints are
drawn from disjoint vocabularies.
IBM © 2009 IBM Corporation 19
“Testing Concurrent Systems: An interpretation of intuitionistic logic”
Jagadeesan, Nadathur, Saraswat FSTTCS 2005

RCC – Integrating CCP and logic programming
E (A):: LHS (RHS) atomic
formulas, representing user-defined,
possibly recursive,
agents (goals).
RCC extends HH (Harrop
formulas of Dale Miller et
al). Very powerful
framework with deep
guards, universals.
20
All extensions
preserve Fundamental
Theorem of CCP
(Agent) D ::= true | D,D | D;D | all X D | some X D | E | c | if G D | E => D | G => H
(Goal) G ::= true | G,G | G;G| all X G | some X G | H | c | if D G

Discrete Timed CCP (1993)
environment
system
 Synchronicity principle
– System reacts instantaneously to the
environment
 Semantic idea
– Run a (bounded) default CCP
program at each time point to
determine instantaneous response
and program for next time instant
(resumption)
– Add: A ::= hence A
– No connection between the store at
one point and the next.
– Future cannot affect past.
Berry’s Synchrony Hypothesis
 Semantics
– Non-empty, prefix-closed sets of
sequences of (pairs of) constraints
– P after s =d= {e | s.e in P} must be
denotation of a Default CC program
 Determinacy guaranteed if unless
used only with next:
– unless (c) next A;
 Reintroduces mutation but in a
controlled way, only when the clock
ticks.
Research IBM Move to first-order linear temporal logic (A::= hence A)
21

HCC: Move to Continuous Time, Continuous Change (1995)
 No new combinator needed
– Constraints are now permitted to
vary with time (e.g. x’=y)
 Semantic intuition
– Run default CCP at each real time
instant, starting with t=0.
– Evolution of system is piecewise
continuous: system evolution
alternates between point phase
and interval phase.
– In each phase program determines
output of that phase and program
to be run in next phase.
 Point phase
– Result determines initial conditions for
evolution in the subsequent interval
phase and hence constraints in effect
in subsequent phases.
 Interval phase
– Any constraints asked of the store
recorded as transition conditions.
– ODE’s integrated to evolve time-dependent
variables.
– Phase ends when any transition
condition potentially changes status.
– (Limit) value of variables at the end of
the phase can be used by the next
point phase.
Research IBM Gupta, Jagadeesan, Saraswat Science of Computer Programming 1998
22

Probabilistic CCP (Concur 96, POPL 99)
23
Gupta, Jagadeesan, Saraswat “Probabilistic Concurrent Constraint
Programming”, Concur 1996.

Significant amount of additional theory work
 Proving properties of CC programs
– de Boer, Gabbrielli, Marchiori, Palamidessi TOPLAS
1997; Etalli, Gabbrielli, Meo TOPLAS 2001
 Abstract interpretation
– Falaschi, Olarte, Valencia, PPDP 09; Falaschi, Olarte,
Palamidessi, 2011
 Declarative debugging
– Fromherz, 1995
 Abstract diagnosis
– Comini et al, TPLP 2011, Titolo PhD thesis
24
See also survey paper: Gabbrielli, Palamidessi, Valencia “Concurrent and
Reactive Constraint Programming”, 2010

C10
25

Desiderata
 Intended for (probabilistic) analytic applications involving big
data
– cf new implementations of R, Matlab, DML
– Implementation must exploit available intra-node parallelism and
scale to multiple nodes in a cluster
– Implementation must recover automatically from node-failure
– In-memory scalability (contra disk-level scalability) is ok
 Must support high productivity
– Must have high-level, declarative abstractions
– Strongly type-checked, support type inference
– Determinate by design (no race conditions, deadlock)
– Must not require explicit concurrency or distribution constructs
Research
(annotations are ok)
– Must support high-level (declarative) debugging
IBM © 2009 IBM Corporation 26

combinators
27
applications involving big data.

C10 High-level design
Pure
Strongly typed
Object-oriented
Timed
Probabilistic
Implicitly concurrent
language
over a rich logic
Supporting set
comprehensions
Pure logical interpretation. Like Prolog, without cut, assert.
Based on augmented X10, using its generic, dependent type-system.
Programs are collections of classes and interfaces
whose members are constructors, field and method declarations
Crucial difference: all variables, fields are logic variables.
TCC, HCC
Probabilistic CCP
No explicit concurrency or distribution constructs
RCC
(New to C10)
28
Should compile to (Resilient) X10, leverage existing (distributed,
concurrent) X10 libraries, exploiting LHS/RHS And/Or, Data parallelism
 Dictates OO structure, type structure
Next few slides illustrate some key features of the language

Basic syntactic categories
29
<AgentDef> ::=
<Qualifiers> agent <Name> [ ( <ArgList> ) ] { [ <Agent> ] }
<Qualifiers> agent <Name> [ ( <ArgList> ) ] [ : <Type> ] = <Term> .
<Term> ::= <Var> | <Term> ? <Term> : <Term> | old <Term>
| [ <Primary> . ] <Fun> ( <TermList> )
| new <ClassName> ( <TermList> )
| <Var> ! <Constraint> -- a constrained variable
| (<ArgList>){<Goal>} => <Term> -- a function literal
| (<ArgList>){<Goal>} => <Term> groupby <Term> -- with grouping
<Agent> ::= <Constraint>
| <AgentName> [ ( <TermList> ) ]
| <Primary> . <AgentName> [ ( <TermList> ) ]
| <Agent> , <Agent>
| <Agent> ; <Agent>
| if (<Goal>) <Agent> [ else <Agent> ]
| all(<ArgList>) <Agent> | hence <Agent>
Similar definitions for goals and constraints

Example “conventional” CCP: quicksort
class Cons[T](h:T, t:List[T])
implements List[T] {
agent qsort()= Y!{
Pair(S,B)=t.split(h),
Y=s.qsort()
.append(Cons(h,b.qsort()))}
agent split(i:T){T<:Comparable[T]}=Y!{
Pair(A,B)=t.split(i),
Y = h < i ?
Pair(Cons(H,A), B)
: Pair(A, Cons(H,B))}
agent
append(L:List[T])=Cons(h,t.append(L)).
…
class Null[T] implements List[T] {
agent qsort()=this.
agent append(L:List[T])=L.
agent split(i:T)=this.
…
}
class Pair[S,T](a:S,b:T) {}
Research }
Invocation
B:Cons[Int], A=B.qsort(), B=Cons(1,C), C=Cons(45,D),D=Null[Int]()
IBM Method invoked with target an unbound promise Information about target computed incrementally;
triggers evaluation of qsort body
30

Functions as set comprehensions
 Key insight: (partial) functions implicitly represent a
bag (their range).
– Functions with finite domains represent finite bags
– Reduction operations (e.g. max) can be directly applied to
functions
– Functions with groupby clauses represents Rails (maps) of
sets
def delta(A:Vector, B:Vector)=
max((i:A.domain)=>Math.abs(A(i)-B(i))).
31
def histogram(N:Int, A:Rail[Int(1,N)]) =
sum ((i:A.domain)=> 1 groupby A(i)).

A simple “Hello world”
32
class MontePi {
static agent pi(N:Int, M:Random) = avg((1..N)=> R ! {
X = M.nextDouble(),
Y = M.nextDouble(),
R = X*X + Y*Y <=1 ? 1 : 0
}
public static agent main(args:Rail[String]) {
N = Integer.parseInt(args(0)),
Console.OUT.println(”Hello world! Pi is ~"
+pi(N, new Random()))
}
}
Shows data parallelism

Kmeans – a clustering algorithm
33
class KMeans(N:Int, P:Int, K:Int, pts:Rail[Vector(N)](P)) {
type Vector = Rail[Double](N).
def delta(A:Vector, B:Vector)=
max((i:A.domain)=>Math.abs(A(i)-B(i))).
agent kmeans=kmeans((i:0..(K-1))=>pts(i) groupby i).
agent kmeans(old:Rail[Vector](K))= means ! {
T = avg ((i:0..(P-1))=>
pts(i) groupby
argmin((j:0..(K-1))=> delta(pts(i), old(j)))),
means=delta(old, T) < epsilon? T: kmeans(T)
}
}
Same basic pattern for many ML algorithms – Gaussian Non-negative matrix
factorization, linear regression, logistic regression, …

Crossgrams: words with anagrams for every letter
Use of a D=>G goal
Assert word/4 facts
on the fly, based on
the dictionary
34
class Crossgram {
@Gentzen def word(Atom, Char, Char, Atom).
goal crossgrams(Dict: List[Atom]) = R!{
if (all (W in Dict) {
W = name(Wls),
Wsls = Wls.msort(),
word(W, Wls.head, Wsls.head, name(Wsls))
})
words(R)
}
goal words(R:List[Atom]) {
word(A,C,C,Ws), A=name(Wls),
R=List(A, Wls.tail.map((L:Char)=>W!{word(W,L,_,Ws)}))
}}
word(emits, e, e, eimst)
word(items, i, e, eimst)
word(mites, m, e, eimst)
word(smite, s, e, eimst)
word(times, t, e, eimst)
…
Goal solved in
context of generated
clauses

Jacobi iteration on a shared mutable array
35
class Jacobi(N:Int, G:Array2[Double](N,N)) {
R=1..(M-1);
agent jacobi {
all (i,j in R*R)
always
G(i,j) = old 0.25*(G(i-1,j)+G(i+1,j)
+G(i,j-1)+G(i,j+1))
}
}

Constraint maintenance
agent thermometer {
always temp = mercury.height/scale,
always whiteRect(thermometer),
always greyRect(mercury),
always displayNumber(temp),
always
mercury.top = mouse.button==down?
mouse.location.y : old mercury.top
}
36
“Kaleidoscope: A Constraint Imperative Language” Lopez, Freeman-
Benson, Borning, 1993

Bayes Net – The backache example
37
public class BackAche {
val True = Boolean.TRUE, False = Boolean.FALSE.
static type PV=ProbabilisticValue[Xboolean].
def backache(Chair:Boolean, Sport:Boolean, Worker:Boolean, Back:Boolean, Ache:Boolean) {
Chair ~ new PV([True~0.8,False~0.2]),
Sport ~ new PV([True~0.02,False~0.98]),
Worker ~ new BooleanCasePD(Chair, [new PV([True~0.9,False~0.1]),
new PV([True~0.01,False~0.99])]),
Back ~ new BooleanCasePD2(Chair, Sport, [new PV([True~0.9,False~0.1]),
new PV([True~0.2,False~0.8]),
new PV([True~0.9,False~0.1]),
new PV([True~0.01,False~0.99])
]),
Ache ~ new BooleanCasePD(Back, [new PV([True~0.7,False~0.3]),
new PV([True~0.1,False~0.9])])
}
agent run() {
Chair = new Boolean("Chair"), Sport = new Boolean("Sport"), Worker=new Boolean("Worker"),
Back=new Boolean("Back"), Ache=new Boolean("Ache”),
Chair = True, Sport = True,
backache(Chair,Sport,Worker,Back,Ache)
}
}
Bayes’ nets can be solved explicitly with symbolic computation of C10 programs

Open Problems -- Implementation
 C10 implementation
– Need coarsening techniques, formalism exposes very fine-grained concurrency.
– Need to analyze when not to create promises.
– Need efficient implementation of suspension
– Valuable for compiler to use a symbolic execution engine
– Develop practical implementation of declarative debugger
– Use nested relational algebras to compile programs with set formers.
• Note: implementation can reuse X10 scheduler (fork-join, work-stealing), concurrent allocator, garbage
Research collector, multi-place implementation
IBM C10 is a very ambitious attempt to develop a modern constraint language.
Please join us!
38

Open Problems – Theory
 Theory
– Develop declarative debugging for timed programs.
– Develop extended static checking for CCP
– Develop implementations of abstract interpretation for (T)CC
(Falaschi et al)
– Develop theory of determinate default programs
– Integrate “soft constraints”, preferences into CCP theory.
– Develop theory of “sketching” (another use for symbolic
execution engine)
Research IBM C10 is a very ambitious attempt to develop a modern constraint language.
Please join us!
39

combinators
40
applications involving big data.

IBM Research: Computing as a Service
Background
© 2005 IBM Corporation Computing as a Service 41

X10 2.2: An APGAS language
– Class-based single-inheritance OO
– Structs
– Closures
– True Generic types (no erasures)
– Constrained Types (OOPSLA 08)
– Type inference
– User-defined operations
– Structured concurrency
Research IBM Java-like productivity, MPI-like performance
42
© 2009 IBM Corporation Asychrony
• async S
Locality
• at (P) S
Atomicity
• atomic S
• when (c) S
Order
• finish S
• clocks
Global data-structures
• points, regions,
distributions,
arrays
 Basic model is now well established
– PPoPP 2011 paper shows best known
speedup numbers for UTS upto 3K cores.
– Global Matrix Library shows substantial
speedup over Hadoop for data analytics
kernels.
– Similar performance improvement for Main
Memory Map Reduce engine (M3R) over
Hadoop.
– SATX10 – better than plingeling on 8 cores,
significant perf improvement at 16,32,64,128
cores (x86 multicore, P7 cluster).
class HelloWholeWorld {
public static def main(s:Array[String]) {
finish
for (p in Place.places())
async
at (p)
Console.OUT.println("(At " + p + ") "
+ s(0));
}
}

Logical Semantics: An agent is a formula
L[[ c ]] = c
L[[{val x:T, A}]]= exists x A
L[[ A, B]] = A ∧ B
L[[if (c) A]] = c implies A
Γ |- c Γ, A |- G
Γ, c implies A |- G
Γ, A |- G
Γ, exists X A |- G
(X not
free
below
line)
Γ, A, B |- G
Γ, A ∧ B |- G
Research Fundamental Theorem of CCP:
Execution of A leads to addition of c to store iff L[[ A]] |- c.
IBM Saraswat, Lincoln 1992
44

Denotational Semantics
[[A]] = function mapping initial store to final store (or limit)
[[ c ]] = {d | d ≥ c }
[[ when (c) A]] = { d | d ≥ c implies d in [[A]]}
[[ {val X:T, A} ]] = { d | exists X d = exists X c, c in [[A]]}
[[ {A,B} ]] = [[ A ]] intersect [[ B ]]
Observation: Function is a closure operator
(monotone, extensive, idempotent)
Observation: Closure operator representable by
a single set (its fixed points). (P(a) is just the
least fixed point of P above a.)
Observation: Parallel composition is just set
intersection!
Concurrency =
Interleaving

Declarative Debugging
 Declarative debugging
techniques can be applied to
logic programs, functional
programs, CCP.
– Ueda 98 (CCP)
– Fromherz 93 (LNCS 749)
– Falaschi et al ICLP 07
 Basic idea is to summarize an
execution through an execution
tree
– Node = procedure call
– Children = calls made in the
Research body.
– Node associated with some data
about subtree, e.g. pair of
input/output constraints.
IBM 46
© 2009 IBM Corporation  Debugging
– Query oracle (user, specification)
whether data with node is
correct.
– Identify node with incorrect data
whose children have correct data
…. BUG!


Declarative Debugging of CCP
sift(X)=Y {X=[2, 3,4,5], Y=[2,3,4,5]}
sift(Z1)=Y1 {X1=[3,4,5], Y1=[3,4,5]} filter(X1,2)=Z1 {X1=[3,4,5], Z1=[3,4,5]}
filter(X2,2)=Z2 {X2=[4,5], Y2=[4,5]}
filter(X3,2)=Y3 {X3=[5], Y3=[5]}
filter(X4,2)=Y4 {X4=[], Y4=[]}
Data associated with node is
just a constraint!
Research agent sift(Ns:List[Int]):List[Int]
= Ns.null() ? Null[Int]()
: Cons(Ns.head, sift(filter(Ns.tail, Ns.head)));
agent filter(Ns:List[Int], N:Int):List[Int] {
IBM : 0==x % N ? Cons(Ns.head, filter(Ns.tail,N))
: Cons(Ns.head, filter(Ns.tail,N));
47
© 2009 IBM Corporation Observed fixed
points
Intended fixed
points

Live Debugging
sift(X)=Y {X=[2, 3,4 |Xr], Y=[2,3,4 |Yr]}
sift(Z1)=Y1 {Z1=[3,4|Zr], Y1=[3,4|Yr]} filter(X1,2)=Z1 {X1=[3,4|Xr], Z1=[3,4|Zr]}
filter(X2,2)=Z2 {X2=[4|Xr], Y2=[4|Zr]}
filter(X3,2)=Y3 {X3=Xr, Y3=Zr}
Research sgent sift(Ns:List[Int]):List[Int]
: Cons(Ns.head, sift(filter(Ns.tail, Ns.head)));
agent filter(Ns:List[Int], N:Int):List[Int] {
IBM : 0==x % N ? Cons(Ns.head, filter(Ns.tail,N))
: Cons(Ns.head, filter(Ns.tail,N));
48
© 2009 IBM Corporation gen(Xr)
Stores can be incomplete!
Can debug a subcomputation even with live concurrent agents
stuck for now
Observed fixed
points

Default CCP
 A ::= unless(c) A
– Run A, unless c holds at end
– ask c / A
 Leads to nondet behavior
– unless(c) c
• No behavior
– unless(c1)c2,unless(c2)c1
• gives c1 or c2
 Leads to non-mononoticity
– unless(c) d : gives d
– c, unless(c) d : gives c
Research  Determinacy restriction:
– A must
IBM 49
© 2009 IBM Corporation  [A] = set S of pairs (c,d)
satisfying
– Sd = {c | (c,d) in S} denotes
a closure operator.
– We still have a simple
denotational semantics!
 Operational implementation:
– Backtracking search
– Compile-time determinacy
analysis (not implemented)
– Open question:
• Efficient compile-time
analysis (cf causality
analysis in Esterel)
• Use negation as failure

Hybrid Systems
 Hybrid Systems combine
both
– Discrete control
– Continuous state evolution
– Intuition: Run program at every
real value.
• Approximate by:
– Discrete change at an
instant
– Continuous change in an
interval
 Primary application areas
– Engineering and Control systems
 Traditional Computer Science
– Discrete state, discrete change
(assignment)
– E.g. Turing Machine
– Brittleness
• Small error  Major impact
• Devastating with large code
• Primary application areas
 Traditional Mathematics
– Continuous Variables (Reals)
– Smooth state change
• Mean-value theorem
• E.g. computing rocket trajectories
Research – Robustness in the face of change
• Paper transport
– Stochastic systems (e.g. Brownian
• Autonomous vehicles…
motion).
– Biological Computation.
IBM – Programmable Matter?
Emerged in early 90s in the work of Nerode, Kohn, Alur, Dill, Henzinger…
50

Volterra-Lotka model – non-linear differential equations
class Volterra {
agent main(Array[String]){
#SAMPLE_INTERVAL_MAX 0.005
val py=8, // prey
val pd=2, // predator
val pd'=0.2,
always py'= py*(0.08-0.04*pd),
always {
cont(pd),
pd’ = -pd*(pd >=0.5*py?0.1:0.06 -0.02*py),
}
sample(pd), sample(py)
Decay (=death) proportional to
population size.
Research }}
IBM 51
© 2009 IBM Corporation Exponential term (natural
growth, assuming enough food)
Decay proportional to the rate at
which predator eats prey
Growth proportional to the rate
at which prey are consumed.
Execution introduces adaptive discretization

State dependent rate equations
 Expression of gene x inhibits
expression of gene y; above a
certain threshold, gene y
inhibits expression of gene x:
if (y < 0.8)
x’= -0.02*x + 0.01,
if (y >= 0.8) {
x’=-0.02*x,
y’=0.01*x
Research }
IBM Bockmayr and Courtois: Modeling biological systems in hybrid concurrent
constraint programming
52

An “internal view”
 What if we treat X~PD as a constraint? Now store can
make many inferences:
– X~Uniform(M,N), X >= L |- X~Uniform(max(M,L), N)
– X~Gaussian(M,S2), Y~Gaussian(M,S11
22
2) |- X+Y ~ Gaussian(M1
+ M2, S1
2 + S2
2)
 And, in some cases (e.g. Bayesian networks), we can
execute programs without sampling
– Programs: X~PD1, Y~PD2, Z~case (X,Y) of v1: t1 | … |
vn:tn
– Now add to the constraint solver the rule:
• X~(v1~p1&…&vn~pn), Y~case X of v1: t1 | … | vn:tn |-
Research
IBM Y~(t~p+ … + t~p)
11 nn© 2009 IBM Corporation 53
(cf bind operator for PV[T] monad)

Gayle Shapley algorithm
 Two sets of people – men, women, each of size N. Each woman y totally
orders all men x according to her preference (w(x,y) is a number, higher
number = stronger preference).
 At any time, a person can be engaged or free. Once a woman is engaged
she can never be free.
 Initially, all men and women are free.
 Non-deterministically select a free man. He proposes to the first woman on
his list he has not yet proposed to. If the woman accepts, the two are
engaged, else the man is free.
 A woman accepts a proposal iff she is free or engaged to a less worthy man
(in which case she breaks her existing engagement to the less worthy man,
who becomes free).
54
Theorem: terminates in O(n2) in man-optimal, stable matching.
Rich theory -- there is a lattice of stable matches, results extended
to many-one case.

Gale Shapley as a C10 program, using recursion
55
class StableMarriage(M:Int, N:Int) {
type RT=Int(1,M), HT=Int(1, N).
agent gs2 {for (x in RT) man(x, N)}
agent man(x:RT, lev:HT) {
y=atLev(x,lev),
w(y,x)<=lw(y),
if (w(y,x)<lw(y)) man(x,lev-1)
}
}
Key idea: Identify monotonicities, and express them as constraints.
Here lw(y) can only rise as computation progresses. Hence, use one-sided
constraints, w(x,y) <= lw(y) (captures: x proposes to y)
An engagement (x,y) is rejected if store entails w(y,x) < lw(y)
That’s all!

Gale Shapley as a C10 program
56
class StableMarriage(M:Int, N:Int) {
type RT=Int(1,M), HT=Int(1, N).
goal free(x:RT,y:HT) {
all (y0:HT) if (m(x,y0)>m(x,y)) w(y,x)<lw(y)
}
agent gs {
for (x,y in RT*HT)
if (free(x,y)) w(y,x)<=lw(y)
}
}

Selected Bibliography
 Saraswat, Rinard, Panangaden
“Semantics of Concurrent
Constraint Programming”, POPL
1991
 Falaschi, Gabbrielli, Marriott,
Palamidessi “Compositional
analysis for CCP”, LICS 1993
 Fromherz “Towards declarative
debugging of CCP”, 1995
 Saraswat, Jagadeesan, Gupta
“Timed Default CCP”, Journal
Symbolic Comp., 1996
 de Boer, Gabbrielli, Marchiori,
Palamidessi “Proving concurrent
constraint programs correct”,
TOPLAS 1997
 Etalli, Gabbrielli, Meo
“Transformations of CCP
programs”, TOPLAS 2001
 Falaschi, Olarte, Valencia
“Framework for abstract
interpretation for Timed CCP”,
PPDP 09
 Gabbrielli, Palamidessi, Valencia
“Concurrent and Reactive
Constraint Programming”, 2010
 Recent PhD theses – Carlos Olarte
(LiX) (universal TCC), Sophia
Knight (LiX), Laura Titolo (U Udine)
“Abstract Interpretation Framework
for Diagnosis … of Timed CC
languages”

HCC references
– Gupta, Jagadeesan, Saraswat “Computing with Continuous
Change”, Science of Computer Programming, Jan 1998, 30 (1—
2), pp 3--49
– Saraswat, Jagadeesan, Gupta “Timed Default Concurrent
Constraint Programming”, Journal of Symbolic Computation,
Nov-Dec1996, 22 (5—6), pp 475-520.
– Gupta, Jagadeesan, Saraswat “Programming in Hybrid
Constraint Languages”, Nov 1995, Hybrid Systems II, LNCS
999.
– Alenius, Gupta “Modeling an AERCam: A case study in modeling
with concurrent constraint languages”, CP’98 Workshop on
Modeling and Constraints, Oct 1998.

Spatial HCC: Move to continuous space
 Add A::= atOther A
– Run A at all other points.
(atAll A = A, atOther
A)
– Constraints may now use
partial derivatives.
– All variables now implicitly
depend on space
parameters (e.g. x,y,z)
 Semantic intutions
– Computation now uniformly
extended across space.
– At each point, run a Default
CC program.
– Program induces its own
discretization of space (into
open and closed regions).
 Programming intuition
– Program with vector fields, specifying
how they vary across space-time.
 Programming Matter realization
– Atoms represent dense computational
grid.
– Signals represented as memory cells
in each Atom
– Atoms use epidemic algorithms to
diffuse signals (possibly with non-zero
gradients) across space.
– Atoms use neighborhood queries to
sense local minima
– Atoms integrate PDEs by using
chaotic relaxation (Chazan/Mirankar).
– Compiler produces FSA for each atom
Research from input program.
IBM 59

The Concurrent Constraint Programming Research Programmes -- Redux

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to The Concurrent Constraint Programming Research Programmes -- Redux

Similar to The Concurrent Constraint Programming Research Programmes -- Redux (20)

Recently uploaded

Recently uploaded (20)

The Concurrent Constraint Programming Research Programmes -- Redux

Editor's Notes