eScience consists of computation-intensive workflows executing on highly distributed networks. Service compositions aggregate web services to automate scientific and enterprise business processes. Along with the increased demand for data quality and Quality of Service (QoS) for an accurate outcome in a shorter completion time, execution of the eScience workflows and service compositions are also required to be distributed efficiently across various geo-distributed nodes. This paper presents Mayan, a Software-Defined Networking (SDN) based approach for service composition. Mayan i) facilitates an adaptive execution of scientific workflows, ii) offers a more efficient service composition by leveraging distributed execution frameworks, in addition to the traditional web service engines, and iii) enables a very large-scale reliable service composition by finding and consuming the current best-fit among the multiple implementations or deployments of the same service.

1. Building Blocks of Mayan:
Componentizing the eScience Workflows Through
Software-Defined Service Composition
Pradeeban Kathiravelu*, Tihana Galinac Grbac+, Luís Veiga*
*INESC-ID Lisboa & Instituto Superior Técnico, Universidade de Lisboa, Portugal
+University of Rijeka, Croatia
23rd IEEE International Conference on Web Services (ICWS 2016)
June 27 - July 2, 2016, San Francisco, USA.
Pradeeban Kathiravelu (IST-ULisboa) Software-Defined Service Composition 1 / 28

2. Overview
1 Introduction
2 Mayan Approach
3 Evaluation
4 Conclusion
Pradeeban Kathiravelu (IST-ULisboa) Software-Defined Service Composition 2 / 28

3. Introduction
Introduction
eScience workflows
Computation-intensive.
Execute on highly distributed networks.
Complex service compositions aggregating web services
To automate scientific and enterprise business processes.
Pradeeban Kathiravelu (IST-ULisboa) Software-Defined Service Composition 3 / 28

4. Introduction
Motivation
Increasing demand for
Data quality and Quality of Service (QoS).
Better Performance (Shorter completion times and higher throughput).
Geo-distribution (workflows and compositions).
Need for additional control and flexibility.
Exploring Trade-off: Efficiency vs. Accuracy.
Leveraging Software-Defined Approaches (from SDN).
Pradeeban Kathiravelu (IST-ULisboa) Software-Defined Service Composition 4 / 28

5. Introduction
Goals
Scalable Distributed Executions.
High Scalability.
Better orchestration.
Data Quality Assurance.
Multi-Tenanted Environments.
Isolation Guarantees.
Differentiated Quality of Service (QoS).
Pradeeban Kathiravelu (IST-ULisboa) Software-Defined Service Composition 5 / 28

6. Introduction
Contributions
Support for,
Adaptive execution of scientific workflows.
Flexible service composition.
Reliable large-scale service composition.
Efficient selection of service instances.
Pradeeban Kathiravelu (IST-ULisboa) Software-Defined Service Composition 6 / 28

7. Mayan Approach
Mayan
Extensible SDN approach for cloud-scale service composition
Driven by:
Loose coupling
Message-oriented Middleware (MOM)
Availability of a logically centralized control plane
Leveraging OpenDaylight SDN controller as the core.
Modular, as OSGi bundles.
Additional advanced features.
State of executions and transactions stored in the controller distributed
data tree.
Clustered and federated deployments.
Pradeeban Kathiravelu (IST-ULisboa) Software-Defined Service Composition 7 / 28

8. Mayan Approach
Services as the building blocks of Mayan
Prototypical Example:
Pradeeban Kathiravelu (IST-ULisboa) Software-Defined Service Composition 8 / 28

9. Mayan Approach
Software-Defined Service Composition
Pradeeban Kathiravelu (IST-ULisboa) Software-Defined Service Composition 9 / 28

10. Mayan Approach
Multiple Implementations and Deployments of a Service
Pradeeban Kathiravelu (IST-ULisboa) Software-Defined Service Composition 10 / 28

11. Mayan Approach
Software-Defined Service Composition
Pradeeban Kathiravelu (IST-ULisboa) Software-Defined Service Composition 11 / 28

12. Mayan Approach
Services as the building blocks of Mayan
Prototypical Example:
Pradeeban Kathiravelu (IST-ULisboa) Software-Defined Service Composition 12 / 28

13. Mayan Approach
Too many requests on the fly?
Prototypical Example:
Pradeeban Kathiravelu (IST-ULisboa) Software-Defined Service Composition 13 / 28

14. Mayan Approach
Alternative Deployment/Implementation
Prototypical Example:
Pradeeban Kathiravelu (IST-ULisboa) Software-Defined Service Composition 14 / 28

15. Mayan Approach
Mayan Services Registry: Modelling Language
Pradeeban Kathiravelu (IST-ULisboa) Software-Defined Service Composition 15 / 28

16. Mayan Approach
Service Composition Representation
<Service3,(<Service1, Input1>, <Service2, Input2>)>
Pradeeban Kathiravelu (IST-ULisboa) Software-Defined Service Composition 16 / 28

17. Mayan Approach
Alternative Implementations and Deployments
Pradeeban Kathiravelu (IST-ULisboa) Software-Defined Service Composition 17 / 28

18. Mayan Approach
Mayan Higher Level Deployment Architecture:
Multi-Domain Workflows
Pradeeban Kathiravelu (IST-ULisboa) Software-Defined Service Composition 18 / 28

19. Mayan Approach
Connecting Services View with the Network View
Pradeeban Kathiravelu (IST-ULisboa) Software-Defined Service Composition 19 / 28

20. Mayan Approach
Connecting Services View with the Network View
Pradeeban Kathiravelu (IST-ULisboa) Software-Defined Service Composition 20 / 28

21. Evaluation
Evaluation System Configurations
Evaluation Approach:
Smaller physical deployments in a cluster.
Larger deployments as simulations and emulations (Mininet).
Evaluated Deployment:
Service Composition Implementations.
Web services frameworks.
Apache Hadoop MapReduce.
Hazelcast In-Memory Data Grid.
OpenDaylight SDN Controller.
Pradeeban Kathiravelu (IST-ULisboa) Software-Defined Service Composition 21 / 28

22. Evaluation
Preliminary Assessments
A workflow performing distributed data cleaning and
consolidation [PK 2015].
A distributed web service composition.
vs.
Mayan approach with the extended SDN architecture.
Pradeeban Kathiravelu (IST-ULisboa) Software-Defined Service Composition 22 / 28

23. Evaluation
Speedup and Horizontal Scalability
No negative scalability in larger distributions.
100% more positive scalability for larger deployments.
Pradeeban Kathiravelu (IST-ULisboa) Software-Defined Service Composition 23 / 28

24. Evaluation
Memory consumption in the Service Nodes
Initial coordination overhead in memory for smaller deployments.
Minimal overhead for larger deployments.
Pradeeban Kathiravelu (IST-ULisboa) Software-Defined Service Composition 24 / 28

25. Conclusion
Related Work
MapReduce for efficient service compositions [SD 2014].
But we should not forget the registry!
Palantir: SDN for MapReduce performance with the network proximity
data [ZY 2014].
A multi-domain deployment of SDN for community
networks [PK 2016].
Pradeeban Kathiravelu (IST-ULisboa) Software-Defined Service Composition 25 / 28

26. Conclusion
Conclusion
SDN-based approach that enables large scale flexibility with
performance
Components in eScience workflows as building blocks of a distributed
platform.
Service composition with web services and distributed execution
frameworks.
Multi-tenanted multi-domain executions.
Pradeeban Kathiravelu (IST-ULisboa) Software-Defined Service Composition 26 / 28

27. Conclusion
Conclusion
SDN-based approach that enables large scale flexibility with
performance
Components in eScience workflows as building blocks of a distributed
platform.
Service composition with web services and distributed execution
frameworks.
Multi-tenanted multi-domain executions.
Future Work
Mayan should further be deployed and evaluated on physical
geo-distributed nodes.
Extending Software-defined service composition for the network
functions in service composition of middlebox actions.
Load balancing.
Firewalls.
Adapting as an NFV framework for service function chaining.
Pradeeban Kathiravelu (IST-ULisboa) Software-Defined Service Composition 27 / 28

28. Conclusion
References
PK 2015 Kathiravelu, Pradeeban, Helena Galhardas, and Luís Veiga. "∂u∂u Multi-Tenanted Framework: Distributed
Near Duplicate Detection for Big Data." On the Move to Meaningful Internet Systems: OTM 2015
Conferences. Springer International Publishing, 2015.
SD 2014 Deng, Shuiguang, et al. "Top-Automatic Service Composition: A Parallel Method for Large-Scale Service
Sets." Automation Science and Engineering, IEEE Transactions on 11.3 (2014): 891-905.
ZY 2014 Yu, Ze, et al. "Palantir: Reseizing network proximity in large-scale distributed computing frameworks using
sdn." 2014 IEEE 7th International Conference on Cloud Computing (CLOUD). IEEE, 2014.
PK 2016 Kathiravelu, Pradeeban, and Luıs Veiga. "CHIEF: Controller Farm for Clouds of Software-Defined
Community Networks." Software Defined Systems (SDS), 2016 IEEE International Symposium on. IEEE,
2016.
Thank you!
Questions?
Pradeeban Kathiravelu (IST-ULisboa) Software-Defined Service Composition 28 / 28