This presentation provides an overview of the new capabilities in IBM Streams V4.3. Topics include dynamic and elastic scaling, programming model, Streams runner for Apache Beam, operations and system management, and toolkit enhancements.

1. © 2018 IBM Corporation1
Streams V4.3
What’s New Overview
August 2018
Mike Spicer
Distinguished Engineer, Lead Architect - IBM Streams

2. © 2018 IBM Corporation2
Please note IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice
and at IBM’s sole discretion.
Information regarding potential future products is intended to outline our general product direction and it should
not be relied on in making a purchasing decision.
The information mentioned regarding potential future products is not a commitment, promise, or legal obligation
to deliver any material, code or functionality. Information about potential future products may not be
incorporated into any contract.
The development, release, and timing of any future features or functionality described for our products remains
at our sole discretion.
Performance is based on measurements and projections using standard IBM benchmarks in a controlled
environment. The actual throughput or performance that any user will experience will vary depending upon
many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O
configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given
that an individual user will achieve results similar to those stated here.
7/30/20182

3. © 2018 IBM Corporation3 © 2018 IBM Corporation3
Serverless Workloads
• Deploy applications without pre-assigning resources to a Streams Instance
• Resources freed when applications are cancelled
• Shared and exclusive assignment simplifies workload Isolation and fine grain resource allocation
Dynamic User Defined Parallelism (UDP)
• Dynamically change the width of a parallel region at runtime
• Dynamically change the resources available to the parallel region
• The Streams Scheduler may change the fusion of operators when the width is changed
Roadmap
• A first step on the path to being fully dynamic
• Manual commands and “hooks” with a number of restrictions
• We plan to increase function and remove restrictions over future releases
• Goal is to have seamless and automatic capabilities with minimal constraints and impacts on applications
• Other systems have constraints and implications
• Stateless or Immutable state;
• Lack of ordering or processing guarantees;
• High overhead or latency impact;
Dynamic and Elastic Scaling

4. © 2018 IBM Corporation4 © 2018 IBM Corporation4
SPL Optional Data Type
• Allows an attribute to have no value, similar to SQL NULL
• Improves interactions with external systems that support the concept of data fields which have no value
• Changes to SPL and a number of toolkits
SPL Event-Time Window Processing
• Simplifies support for window operations based on the time of an event
• Tuples may include an event time attribute that is different from the time it is received or processed
• Simplifies support for out of order and late arriving data
• New SPL window type and stream annotation
Java and Python Topology API
• A number of improvements developed in the open at http://ibmstreams.github.io/streamsx.topology/
Programming Model

5. © 2018 IBM Corporation5 © 2018 IBM Corporation5
Support for Beam-2.4 Java SDK
• Run Beam applications on your local Streams instance
• Use Beam's S3 IO connector to store results to an IBM Cloud Object Storage service
Parallel Beam Pipelines and Transforms
• Specify parallel widths for your application pipeline or specific Beam transforms at compile time
Tuple Bundling
• Specify bundle sizes or time limit at compile time
Componentized Logging
• Configure translation-time and runtime trace levels separately
Submission-Time Parameters
• Create submission-time parameters using Beam application options
Streams Runner for Apache Beam New Features

6. © 2018 IBM Corporation6 © 2018 IBM Corporation6
Improved Scheduling
• Make decisions based on additional data
• Network Load Consideration in PE Placement
• Allow users to control the relative importance of different data for their applications and environment
• Multi Dimensional User Selectable Scheduling Scheme
Improved Rolling Upgrades
• Instances can be upgraded without stopping all applications and restarting the instance
Operations & System Management
6

7. © 2018 IBM Corporation7 © 2018 IBM Corporation7
New toolkits:
• Kafka, MessageHub, MQTT, RabbitMQ (primarily breaking out from streams.messaging toolkit)
• Cloud Object Storage
Toolkit extensions (new operators):
• Geospatial
• FlightPathEncounter
• Timeseries
• BATS forecasting, Holt Winters (new version)
• Inet
• InetSource, HTTPGetStream, HTTPPost, HTTPGetJSONContent, HTTPGetXMLContent, HTTPRequest
Toolkits
7

8. © 2018 IBM Corporation8
Thank You