Your systems. Working as one.




Data-centric Invocable Services



Workshop on Real-Time, Embedded and              Rick Warren
Enterprise-Scale Time-Critical Systems   rick.warren@rti.com
                                                   Rajive Joshi
April 2012; Paris                              rajive@rti.com
Data-Centricity by Example: Calendaring

Imperative Process:
1. Email: “Meet Monday at 10:00.”
2. Email: “Meeting moved to Tuesday.”
3. Email: “Here’s conference call info…”

4. You: “Where do I have to be? When?”
5. You: (sifting through email…)


                     © 2012 RTI            2
Data-Centricity by Example: Calendaring

Data-centric Process:
1. Calendar: (add meeting Monday at 10:00)
2. Calendar: (move meeting to Tuesday)
3. Calendar: (add dial-in info)

4. You: “Where do I have to be? When?”
5. You: (check calendar)


                   © 2012 RTI                3
Data-centric vs. Imperative Styles
• Imperative                                       Functions
                                                   coupled;
  – Tell someone to do something                 State implicit
  – Check whether they did it
• Data-centric                                     Functions
                                                  decoupled;
  Canonical    SQL      HTTP           DDS
                                                 State explicit
    Create    INSERT    POST           write
    Read      SELECT      GET          read
   Update     UPDATE      PUT          write
    Delete    DELETE    DELETE       dispose
   on_event   TRIGGER      —         on_data_
                                     available
                        © 2012 RTI                                4
Conflict?
• Data-centricity benefits system designers
  –   (Eventually) Consistent “truth”
  –   Scalability and elasticity, because of stateless logic
  –   Evolvability, because of functional decoupling
  –   Robustness to communication failures
• Programmers think imperatively
  try {
  doSomethingThatMayFail();
  doSomethingOnSuccess();
  } catch (FailureOccurred) {
  doSomethingElseOnFailure();
  }
                            © 2012 RTI                         5
Conflict?

  Are these models fundamentally
            in conflict?

                 No.




              © 2012 RTI           6
Reconciliation

      “Dude, instructions are just data.”
             — John Von Neumann, 1945


• Desires are also just data
• Objectives are also just data

Best practice: Don’t tell someone to do
something; assert your desired future state

                      © 2012 RTI              7
Reconciliation
Imperative View                Data-centric View
try {               structThingDesired{
doSomething();      string thingWanted;
doOnSuccess();      }
} catch (Failure) {
                    structThingHappened{
doOnFailure();
                    string thingWanted;
}
                               booleansuccess;
                               };

                               …

                  © 2012 RTI                       8
Reconciliation
Imperative View                Data-centric View
try {                          …
                               create(
doSomething();                    thingWanted = “something”)
                               on_event(
doOnSuccess();                    thingWanted == “something”
                                  &&
} catch (Failure) {               success == true) {
doOnFailure();                 update(
                                  thingWanted = “onSuccess”)
}                              }
                               on_event(
                                  thingWanted== “something”
                                  &&
                                  success == false) {
                               update(
                                  thingWanted = “onFailure”)
                               }
                  © 2012 RTI                             9
Reconciliation
Imperative View                      Data-centric View
   This way is easier                    This way is easier
 …to write the first time                  …to scale out
  …to understand top-                       …to extend
  down, sequentially                 …to maintain incrementally
                                      …to understand bottom-
                                              up, causally




                        © 2012 RTI                           10
Reconciliation
Imperative View                                 Data-centric View

Express application                                       Express interop.
code this way                   GOOD                             this way


     This is what we mean by
 “data-centric invocable service”:
 • Request looks like explicit “call” (RMI
   or similar)
 • Interoperability on the basis of
   data, not functional implementation
 • Caller and Callee may use different
   programming models
                                   © 2012 RTI                                11
Reconciliation
Imperative View                     Data-centric View

Express application                             Express interop.
code this way          GOOD                            this way


Make app programmers                            Express interop.
wire that up           HARD                            this way


Express application                       Express interop. based
code this way          BAD                    on one app’s code


                       © 2012 RTI                                  12
Process
Architect            Programmer            Platform
  1. Define data       4. Generate           6. Govern data
     model                code from             flows based
  2. Define and           architect-            on architect-
     group data           defined               defined
     streams using        contract(s)
                          (OPTIONAL)            contract(s)
     those types
  3. Define            5. Produce and
     deployment of        respond to
     producers and        data according
     consumers of         to app algo’s
     those streams




                        © 2012 RTI                              13
Process Example: WSDL
      Architect                                    Programmer

                                             class MyService {
                                                                     Standard WSDL
                                               Result1 doThis(        bindings exist
                                                 Args1) {                  for
                                                 …                   SOAP, REST, and
                                               }                         SMTP.
                                                   Result2 doThat(     Additional
                                                     Args2) {
                                                                     bindings could
                                                     …
                                                   }                 be defined for
                                                   …                  DDS or other
                                             }                       technologies.
Figure courtesy of ICONIX,
                                                      © 2012 RTI                   14
http://www.iconixsw.com/Articles/SOARoadmap.html
Hypothetical DDS IDL Example

structFoo {                  local interface Svc {
  @Key long id;                @InTopic("FooTpc")
long x;                        @OutTopic("BarTpc")
double y;                      Bar doFoo(in Foo f);
};                           };

structBar {
  @Key string who;                Control binding to topics for
                                   easy subscription, filtering,
short baz;                              and QoS control
};

                     © 2012 RTI                                    15
Hypothetical DDS IDL Example

structFoo {                  local interface Svc {
                               @InTopic(
  @Key long id;                   value="T1”,
                                  key="42")
long x;                        @OutTopic(
double y;                         value="T2",
                                  key="me")
};                             Bar doFoo1(in Foo f);

                             @InTopic(
structBar {                       value="T1”,
                                  key="29")
  @Key string who;             @OutTopic(
short baz;                        value="T2",
                                  key="you")
};                             Bar doFoo2(in Foo f);
                             };
                     © 2012 RTI                        16
Hypothetical DDS IDL Counter-Example

local interface Svc {              union Svc_Type
                                   switch (WhichOpEnum){
Foo doFoo(
                                   case Svc_doFoo_CALL:
    out string x,                  Svc_doFoo_InStruct
    inout long y,                      sdfis;
    in double z); Reverse          case Svc_doFoo_RETURN:
  Bar doBar(      Engineer         Svc_doFoo_OutStruct
                                       sdfos;
    in long a,                     case Svc_doBar_CALL:
    out float b,                   Svc_doBar_InStruct
    inout short c);                    sdbis;
};                                 case Svc_doBar_RETURN:
                                   Svc_doBar_OutStruct
                                       sdbos;
                                   };
                      © 2012 RTI                       17
Why is This Gross?
         Insane data type
   to make a DBA cringe in horror            union Svc_Type
                                             switch (WhichOpEnum){
  Segregates data model: human-
                                             case Svc_doFoo_CALL:
  written vs. machine-generated
                                             Svc_doFoo_InStruct
                                                 sdfis;
 (Someone’s) API constraints baked
                                             case Svc_doFoo_RETURN:
    into shared interop. interface
                                             Svc_doFoo_OutStruct
      Heavily influenced by                      sdfos;
     scalability assumptions                 case Svc_doBar_CALL:
  …which may or may not be valid             Svc_doBar_InStruct
Over-reliance on complex filtering on            sdbis;
           subscriber side                   case Svc_doBar_RETURN:
                                             Svc_doBar_OutStruct
Fine-grained QoS governance almost               sdbos;
             impossible                      };
                                © 2012 RTI                       18
Summary
• Data-centricity remains an              • These two sets of benefits
  architectural best practice               are not exclusive
   – Many of its benefits accrue             – Design system interop.
     at the systems level                      interface
• App programmers may be                     – Design programming
  more comfortable with                        interface
  higher-level API                           – Bind them together
                                          • Proven examples exist:
                                             – Architecture
                                             – Workflow
                                             – Data and service
                                               description
                                             – Programming model
                             © 2012 RTI                                  19

Data-centric Invocable Services

  • 1.
    Your systems. Workingas one. Data-centric Invocable Services Workshop on Real-Time, Embedded and Rick Warren Enterprise-Scale Time-Critical Systems rick.warren@rti.com Rajive Joshi April 2012; Paris rajive@rti.com
  • 2.
    Data-Centricity by Example:Calendaring Imperative Process: 1. Email: “Meet Monday at 10:00.” 2. Email: “Meeting moved to Tuesday.” 3. Email: “Here’s conference call info…” 4. You: “Where do I have to be? When?” 5. You: (sifting through email…) © 2012 RTI 2
  • 3.
    Data-Centricity by Example:Calendaring Data-centric Process: 1. Calendar: (add meeting Monday at 10:00) 2. Calendar: (move meeting to Tuesday) 3. Calendar: (add dial-in info) 4. You: “Where do I have to be? When?” 5. You: (check calendar) © 2012 RTI 3
  • 4.
    Data-centric vs. ImperativeStyles • Imperative Functions coupled; – Tell someone to do something State implicit – Check whether they did it • Data-centric Functions decoupled; Canonical SQL HTTP DDS State explicit Create INSERT POST write Read SELECT GET read Update UPDATE PUT write Delete DELETE DELETE dispose on_event TRIGGER — on_data_ available © 2012 RTI 4
  • 5.
    Conflict? • Data-centricity benefitssystem designers – (Eventually) Consistent “truth” – Scalability and elasticity, because of stateless logic – Evolvability, because of functional decoupling – Robustness to communication failures • Programmers think imperatively try { doSomethingThatMayFail(); doSomethingOnSuccess(); } catch (FailureOccurred) { doSomethingElseOnFailure(); } © 2012 RTI 5
  • 6.
    Conflict? Arethese models fundamentally in conflict? No. © 2012 RTI 6
  • 7.
    Reconciliation “Dude, instructions are just data.” — John Von Neumann, 1945 • Desires are also just data • Objectives are also just data Best practice: Don’t tell someone to do something; assert your desired future state © 2012 RTI 7
  • 8.
    Reconciliation Imperative View Data-centric View try { structThingDesired{ doSomething(); string thingWanted; doOnSuccess(); } } catch (Failure) { structThingHappened{ doOnFailure(); string thingWanted; } booleansuccess; }; … © 2012 RTI 8
  • 9.
    Reconciliation Imperative View Data-centric View try { … create( doSomething(); thingWanted = “something”) on_event( doOnSuccess(); thingWanted == “something” && } catch (Failure) { success == true) { doOnFailure(); update( thingWanted = “onSuccess”) } } on_event( thingWanted== “something” && success == false) { update( thingWanted = “onFailure”) } © 2012 RTI 9
  • 10.
    Reconciliation Imperative View Data-centric View This way is easier This way is easier …to write the first time …to scale out …to understand top- …to extend down, sequentially …to maintain incrementally …to understand bottom- up, causally © 2012 RTI 10
  • 11.
    Reconciliation Imperative View Data-centric View Express application Express interop. code this way GOOD this way This is what we mean by “data-centric invocable service”: • Request looks like explicit “call” (RMI or similar) • Interoperability on the basis of data, not functional implementation • Caller and Callee may use different programming models © 2012 RTI 11
  • 12.
    Reconciliation Imperative View Data-centric View Express application Express interop. code this way GOOD this way Make app programmers Express interop. wire that up HARD this way Express application Express interop. based code this way BAD on one app’s code © 2012 RTI 12
  • 13.
    Process Architect Programmer Platform 1. Define data 4. Generate 6. Govern data model code from flows based 2. Define and architect- on architect- group data defined defined streams using contract(s) (OPTIONAL) contract(s) those types 3. Define 5. Produce and deployment of respond to producers and data according consumers of to app algo’s those streams © 2012 RTI 13
  • 14.
    Process Example: WSDL Architect Programmer class MyService { Standard WSDL Result1 doThis( bindings exist Args1) { for … SOAP, REST, and } SMTP. Result2 doThat( Additional Args2) { bindings could … } be defined for … DDS or other } technologies. Figure courtesy of ICONIX, © 2012 RTI 14 http://www.iconixsw.com/Articles/SOARoadmap.html
  • 15.
    Hypothetical DDS IDLExample structFoo { local interface Svc { @Key long id; @InTopic("FooTpc") long x; @OutTopic("BarTpc") double y; Bar doFoo(in Foo f); }; }; structBar { @Key string who; Control binding to topics for easy subscription, filtering, short baz; and QoS control }; © 2012 RTI 15
  • 16.
    Hypothetical DDS IDLExample structFoo { local interface Svc { @InTopic( @Key long id; value="T1”, key="42") long x; @OutTopic( double y; value="T2", key="me") }; Bar doFoo1(in Foo f); @InTopic( structBar { value="T1”, key="29") @Key string who; @OutTopic( short baz; value="T2", key="you") }; Bar doFoo2(in Foo f); }; © 2012 RTI 16
  • 17.
    Hypothetical DDS IDLCounter-Example local interface Svc { union Svc_Type switch (WhichOpEnum){ Foo doFoo( case Svc_doFoo_CALL: out string x, Svc_doFoo_InStruct inout long y, sdfis; in double z); Reverse case Svc_doFoo_RETURN: Bar doBar( Engineer Svc_doFoo_OutStruct sdfos; in long a, case Svc_doBar_CALL: out float b, Svc_doBar_InStruct inout short c); sdbis; }; case Svc_doBar_RETURN: Svc_doBar_OutStruct sdbos; }; © 2012 RTI 17
  • 18.
    Why is ThisGross? Insane data type to make a DBA cringe in horror union Svc_Type switch (WhichOpEnum){ Segregates data model: human- case Svc_doFoo_CALL: written vs. machine-generated Svc_doFoo_InStruct sdfis; (Someone’s) API constraints baked case Svc_doFoo_RETURN: into shared interop. interface Svc_doFoo_OutStruct Heavily influenced by sdfos; scalability assumptions case Svc_doBar_CALL: …which may or may not be valid Svc_doBar_InStruct Over-reliance on complex filtering on sdbis; subscriber side case Svc_doBar_RETURN: Svc_doBar_OutStruct Fine-grained QoS governance almost sdbos; impossible }; © 2012 RTI 18
  • 19.
    Summary • Data-centricity remainsan • These two sets of benefits architectural best practice are not exclusive – Many of its benefits accrue – Design system interop. at the systems level interface • App programmers may be – Design programming more comfortable with interface higher-level API – Bind them together • Proven examples exist: – Architecture – Workflow – Data and service description – Programming model © 2012 RTI 19

Editor's Notes

  • #10 Pseudo-code!
  • #16 Since many here are interested in DDS and use IDL…
  • #17 Since many here are interested in DDS and use IDL…