IBM Streams 4.1 expands support for building streaming applications to include Java and Scala. Developers can now create applications using these languages alongside existing support for SPL. The streamsx.topology project supports building topologies in Java and Scala, and applications can integrate data across languages. IBM Streams features like windowing, parallel regions, and checkpointing are supported for Java and Scala applications.

1. Expanding beyond SPL
IBM Streams 4.1

2. Multiple Language Support
1
IBM Streams Enterprise Ready
Distributed Platform
IBM Streams Application Features
SPL Java Scala Python
(experimental) …

3. Integration with Publish & Subscribe
2
SPL Java Scala
SPL Job
Java Job
Scala Job
Java Job with
SPL & Scala Elements
SPL Job

4. Agenda
• Streamsx.topology project
• Java Applications
• Scala Applications
• Data Interchange
• Streams Features
• SPL integration

5. Streamsx.topology
• A project that supports building streaming topologies
(applications) for IBM Streams in different programming
languages, such as Java and Scala.
 https://github.com/IBMStreams/streamsx.topology
• Open Source
• Agile Development
 Frequent releases
• Not tied to IBM Streams product release cycle
• 1.1 release included in IBM Streams 4.1 – November 20th

6. Java Applications for
IBM Streams

7. Java Application API
• Build streaming applications for IBM Streams with Java
 Natural for Java developer
 Make the common case simple
 Supports Java 8
 Supports IBM Streams 4.0+
 No SPL knowledge required

8. Streams of Java objects
7
TStream<String>
“#stayshere” “#vegas” “#ibminsight”
TStream<HashTag>
{“#stayshere”, 4567, 2015/10/26 03:42}
{“#ibminsight”, 8364, 2015/10/26 08:23}
{“#pbr”, 1324, 2015/10/25 16:23}

9. • Streams are sourced, transformed and sinked by functions
implemented in Java
• Represented as:
 A lambda expression (Java 8)
 A method reference (Java 8)
 An instance of an interface
• Anonymous class
• Class that implements the interface
Functional Programming
8
ƒ

10. Functional Filter
9
test(“Goal! #3lions” )= false
tuple discarded
(not present on resultant stream)

11. Functional Transformation (10/1)
10
apply( 2839000 “2.839E+6”
)=
String will appear on strings stream

12. Functional Transformation (10/N)
11
apply(“It is not in the stars to hold our
destiny but in ourselves”
“It”, “is ”, “not”, “in”, “the”,
“stars”, “to”, “hold”, “our”,
“destiny”, “but”, “in”,
“ourselves”
)=
Each word will appear on
words stream as a tuple

13. Building & Deploying
Java Streaming Applications
1. Declare Topology

14. Building & Deploying
Java Streaming Applications
1. Declare Topology
2. Submit Topology

15. Demonstration:
Java Applications for
IBM Streams

16. Scala Applications for
IBM Streams

17. Scala is a JVM language
• Very good interoperability between the two languages
 http://www.codecommit.com/blog/java/interop-between-java-and-scala
• Java can use Scala classes/methods
• Scala can use Java classes/methods
16

18. IBM Streams Scala Application support
• Scala support is
therefore through the
Java Application API
• API also provides
implicit conversions to
allow Scala functions to
be used as functional
elements
17
https://github.com/IBMStreams/streamsx.topology/wiki/Scala-Support

19. Demonstration:
Scala Applications for
IBM Streams

20. Data Interchange

21. • Streams instances typically contain multiple applications
 Exchanging data using export/import streams
Multiple Applications
20
Ingest
App
Analytic
App
Analytic
App
Analytic
App
Decision
App
Decision
App
Ingest
App

22. • Publish a stream by
 Type – Type of the stream
 Topic – A string – Recommendation to use MQTT style names
• vehicle/auto/sensor/engine/temp
• Subscribe to a stream by
 Type
 Topic – Single topic, no MQTT wildcard support
Publish/Subscribe
21

23. Interchangeable Data Types
22
SPL Java Scala
SPL Tuple   
JSON   
XML   
String   
Binary   
Java Class   
Scala Class   

24. Example: Publish/Subscribe
23
SPL
Java
Scala
SPL
SPL
Java
SPL

25. IBM Streams Features
supported by
Java & Scala

26. IBM Streams Features
• Windowing
 Partitioned
• Parallel Regions
 Round-robin, Key or Hash Partitioned
• Submission time values
• Fan-in, Fan-out of streams
• Checkpoint of operator state
• Resource Placement by tags
• Control over mapping of functions to processes (PEs)

27. • EMBEDDED – Application must only contain Java code
• DISTRIBUTED
• ANALYTICS_SERVICE – Direct submission from Java application to
• STANDALONE
• BUNDLE
• STANDALONE_BUNDLE
• TOOLKIT
• EMBEDDED_TESTER
• STANDALONE_TESTER
• DISTRIBUTED_TESTER
Execution Contexts
26
Creation of a bundle
for later submission
Testing of applications
using JUnit or similar

28. SPL Integration

29. SPL Streams within the Java Application
• Re-use of the IBM Streams Java Operator API
• Created by:
 Invocations of SPL operators
 Subscribing to an stream of SPL tuples

30. • Re-use existing SPL toolkits
 C++ primitive, Java primitive, SPL composite
• Create SPL composites to wrap SPL sub-graphs
 That can then be invoked from a Java/Scala application
SPL Operator Invocation
29

31. Summary

32. IBM Streams
expanded to a wider developer pool
31
IBM Streams Enterprise Ready
Distributed Platform
IBM Streams Application Features
SPL Java Scala …
Integration across languages
• Make the simple - simple
• Re-use of existing SPL toolkits
• Variety of execution mechanisms
Python
(experimental)

33. © 2015 IBM Corporation
Thank You