This document provides an overview and agenda for an IBM Streams Github Projects presentation. It discusses the IBMStreams organization on Github, what's new in Streams Github projects including new language integration, adapters, parsers/formatters, analytics/processing toolkits, and utilities. It encourages attendees to get involved by providing feedback, contributing code/samples, and proposing new projects/features. The presentation aims to foster an open community around extending and sharing Streams resources.

1. © 2015 IBM Corporation
Github Projects Overview
IBM Streams 4.1
Samantha Chan
IBM Streams Community Architect
For questions about this presentation contact: chanskw@ca.ibm.com

2. 2 © 2015 IBM Corporation
Important Disclaimer
THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL
PURPOSES ONLY.
WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE
INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”, WITHOUT WARRANTY
OF ANY KIND, EXPRESS OR IMPLIED.
IN ADDITION, THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY,
WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE.
IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR
OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION.
NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECT OF:
• CREATING ANY WARRANTY OR REPRESENTATION FROM IBM (OR ITS AFFILIATES OR ITS OR
THEIR SUPPLIERS AND/OR LICENSORS); OR
• ALTERING THE TERMS AND CONDITIONS OF THE APPLICABLE LICENSE AGREEMENT
GOVERNING THE USE OF IBM SOFTWARE.
IBM’s statements regarding its plans, directions, and intent are subject to change or
withdrawal without notice at IBM’s sole discretion. Information regarding potential
future products is intended to outline our general product direction and it should not
be relied on in making a purchasing decision. The information mentioned regarding
potential future products is not a commitment, promise, or legal obligation to deliver
any material, code or functionality. Information about potential future products may
not be incorporated into any contract. The development, release, and timing of any
future features or functionality described for our products remains at our sole
discretion.
THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE.
IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION.

3. 3 © 2015 IBM Corporation
Agenda
 IBMStreams Organization
 What’s new in Streams Github Projects?
 Getting Started with Streams 4.1

4. 4 © 2015 IBM Corporation
IBMStreams
http://ibmstreams.github.io
 Open-source organization established on Github in March 2014
 Goals:
– Provide a platform and foster a community to extend and share Streams
programming resources (toolkits, samples, performance benchmark,
utilities, etc.)
– Allow us to deliver new toolkit functions in a more open, agile and rapid
manner
– Improve visibility of Streams programming resources and make them more
easily accessible.

5. 5 © 2015 IBM Corporation
IBMStreams
 March 2014
– Started the 3 repositories:
• 3 toolkits from the Streams product (HDFS,
Messaging, Inet)
 April 2015
– Total: 31 Projects
 Dec 2015
– 1 Language Integration
– 9 Adapters
– 6 Parsers and Formatters
– 12 Processing and Analytics
– 6 Utilities
– 8 Demos / Samples / Docs
– Total: 42 Projects

6. 6 © 2015 IBM Corporation
What’s New? – Language Integration
 Topology Project (streamsx.topology)
– Shipped as part of Streams v4.1
– A project that supports building streaming topologies (applications) for
Streams in different programming languages, such as Java and Scala.
– An experimental version of Python is available.

7. 7 © 2015 IBM Corporation
What’s New? - Adapters
 HBase Toolkit (streamsx.hbase)
– Bluemix Support
 HDFS Toolkit (streamsx.hdfs)
– Sequence File Support (Preview)
– Bluemix Support
 Messaging Toolkit (streamsx.messaging)
– RabbitMQ (Preview)
– Kafka 0.9
– MessageHub
 Inet Toolkit (streamsx.inet)
– The Inet toolkit provides support for common internet protocols. Supported protocols
include FTP, WebSocket, HTTP.
– Native Functions for httpPut, httpGet, httpPost, httpDelete
– Map visualization

8. 8 © 2015 IBM Corporation
What’s New? - Adapters
Multi-Connection TCP Server Toolkit (streamsx.tcp)
– This toolkit contains a TCPServer operator which allows for multi-
connections, and is a multi-threaded source operator. The operator accepts
and sends text or binary data from one or more TCP sockets.
– Added support for full duplex TCP communication. Operator can now
accept and send data via a TCP socket connection.

9. 9 © 2015 IBM Corporation
What’s New? - Adapters
 New repositories:
 CDC Toolkit (streamsx.cdc)
– Provides support for efficiently reading and writing data from InfoSphere
Change Data Capture (CDC)
 GraphDB (streamsx.graphdb)
– Under Construction
– Repository created to provide support for reading and writing data to graph
databases (e.g. IBM System G / Apache Tinkerpop)
 JDBC (streamsx.jdbc)
– JDBCRun operator to allow IBM Streams application to run queries from DB
via JDBC

10. 10 © 2015 IBM Corporation
What’s New? – Parsers and Formatters
 Parquet Toolkit (streamsx.parquet)
–Parquet is a columnar storage format for Hadoop. This repository is created for
hosting operators for reading and writing data in Parquet format.

11. 11 © 2015 IBM Corporation
What’s New? – Analytics and Processing
 SparkMLLib (streamsx.sparkmllib)
– This repository contains a toolkit for real-time scoring using SparkMLLib.
 Healthcare Toolkit (streamsx.health)
– This repository is under construction. This toolkit is intended to provide
building blocks and operators for real-time healthcare related analysis.
 Social Toolkit (streamsx.social)
– This repository provides support to ingest data from social media. Initial
support includes ingesting from Twitter

12. 12 © 2015 IBM Corporation
What’s New? – Analytics and Processing
 Anomaly Detection (streamsx.anomalyDetection)
– This repository contains operators and functions for anomaly detection. Initial
contribution would be based on the Local Outlier Function (LOF)
 Internet of Things Foundation Toolkit (streamsx.iotf)
– Provides the ability to have IBM Streams application to easily interact with
IoTF, either in Bluemix or on-premises.

13. 13 © 2015 IBM Corporation
Github Projects Overview – Utilities
 Network Toolkit (streamsx.network)
– Contains operators and functions for processing network data.
 Transform Toolkit (streamsx.transform)
– Contains building block operators to transform input data from one format to
another.
– Contains a high-performance Modify operator that can be used as a
replacement for Functor

14. 14 © 2015 IBM Corporation
Github Projects Overview – Samples / Demos / Docs
 Cybersecurity Starter Applications
(streamsx.cybersecurity.starterApps)
– Contains starter applications to get started with the IBM Streams
Cybersecurity Toolkit quickly
 Tutorials
– Contains Streams Introduction and SPSS labs
 Streamsx.documentation
– Community-based documentation
– Contains various getting started guides for Streams v4.1 features

15. 15 © 2015 IBM Corporation
Getting Started! - StreamsDev
https://developer.ibm.com/streamsdev/

16. 16 © 2015 IBM Corporation
Streamsx.documentation
 Community-based documentation project
 Complements StreamsDev and Knowledge Center
 Provides Getting Started Guides for new features in Streams
 Community contribution and feedback is much appreciated!
 http://ibmstreams.github.io/streamsx.documentation/

17. 17 © 2015 IBM Corporation
Get Involved!
 Many new and cool projects to try!
– E.g. Python Support, JDBC, Social
 We want your feedback and input!
– Report any issues you have found
– If you think it’s cool, let us know!
– Which toolkit do you use? How do you use it?
– Do you want the toolkit to be included in the product?
 Contribute code, samples and docs!
– Got a clever way to do things? Contribute it to the samples project!
– Need a new parameter? Need support for a new type of server / data format?
Work with us to try to implement it!
– Get a better way to explain things? Contribute to streamsx.documentation
 Got an idea?
– Propose a new project / new feature.

18. 18 © 2015 IBM Corporation
Questions?