Github Projects Overview and IBM Streams V4.1

© 2015 IBM Corporation
Github Projects Overview
IBM Streams 4.1
Samantha Chan
IBM Streams Community Architect
For questions about this presentation contact: chanskw@ca.ibm.com

2 © 2015 IBM Corporation
Important Disclaimer
THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL
PURPOSES ONLY.
WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE
INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”, WITHOUT WARRANTY
OF ANY KIND, EXPRESS OR IMPLIED.
IN ADDITION, THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY,
WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE.
IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR
OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION.
NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECT OF:
• CREATING ANY WARRANTY OR REPRESENTATION FROM IBM (OR ITS AFFILIATES OR ITS OR
THEIR SUPPLIERS AND/OR LICENSORS); OR
• ALTERING THE TERMS AND CONDITIONS OF THE APPLICABLE LICENSE AGREEMENT
GOVERNING THE USE OF IBM SOFTWARE.
IBM’s statements regarding its plans, directions, and intent are subject to change or
withdrawal without notice at IBM’s sole discretion. Information regarding potential
future products is intended to outline our general product direction and it should not
be relied on in making a purchasing decision. The information mentioned regarding
potential future products is not a commitment, promise, or legal obligation to deliver
any material, code or functionality. Information about potential future products may
not be incorporated into any contract. The development, release, and timing of any
future features or functionality described for our products remains at our sole
discretion.
THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE.
IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION.

Agenda
 IBMStreams Organization
 What’s new in Streams Github Projects?
 Getting Started with Streams 4.1

IBMStreams
http://ibmstreams.github.io
 Open-source organization established on Github in March 2014
 Goals:
– Provide a platform and foster a community to extend and share Streams
programming resources (toolkits, samples, performance benchmark,
utilities, etc.)
– Allow us to deliver new toolkit functions in a more open, agile and rapid
manner
– Improve visibility of Streams programming resources and make them more
easily accessible.

IBMStreams
 March 2014
– Started the 3 repositories:
• 3 toolkits from the Streams product (HDFS,
Messaging, Inet)
 April 2015
– Total: 31 Projects
 Dec 2015
– 1 Language Integration
– 9 Adapters
– 6 Parsers and Formatters
– 12 Processing and Analytics
– 6 Utilities
– 8 Demos / Samples / Docs
– Total: 42 Projects

What’s New? – Language Integration
 Topology Project (streamsx.topology)
– Shipped as part of Streams v4.1
– A project that supports building streaming topologies (applications) for
Streams in different programming languages, such as Java and Scala.
– An experimental version of Python is available.

What’s New? - Adapters
 HBase Toolkit (streamsx.hbase)
– Bluemix Support
 HDFS Toolkit (streamsx.hdfs)
– Sequence File Support (Preview)
– Bluemix Support
 Messaging Toolkit (streamsx.messaging)
– RabbitMQ (Preview)
– Kafka 0.9
– MessageHub
 Inet Toolkit (streamsx.inet)
– The Inet toolkit provides support for common internet protocols. Supported protocols
include FTP, WebSocket, HTTP.
– Native Functions for httpPut, httpGet, httpPost, httpDelete
– Map visualization

Multi-Connection TCP Server Toolkit (streamsx.tcp)
– This toolkit contains a TCPServer operator which allows for multi-
connections, and is a multi-threaded source operator. The operator accepts
and sends text or binary data from one or more TCP sockets.
– Added support for full duplex TCP communication. Operator can now
accept and send data via a TCP socket connection.

 New repositories:
 CDC Toolkit (streamsx.cdc)
– Provides support for efficiently reading and writing data from InfoSphere
Change Data Capture (CDC)
 GraphDB (streamsx.graphdb)
– Under Construction
– Repository created to provide support for reading and writing data to graph
databases (e.g. IBM System G / Apache Tinkerpop)
 JDBC (streamsx.jdbc)
– JDBCRun operator to allow IBM Streams application to run queries from DB
via JDBC

What’s New? – Parsers and Formatters
 Parquet Toolkit (streamsx.parquet)
–Parquet is a columnar storage format for Hadoop. This repository is created for
hosting operators for reading and writing data in Parquet format.

What’s New? – Analytics and Processing
 SparkMLLib (streamsx.sparkmllib)
– This repository contains a toolkit for real-time scoring using SparkMLLib.
 Healthcare Toolkit (streamsx.health)
– This repository is under construction. This toolkit is intended to provide
building blocks and operators for real-time healthcare related analysis.
 Social Toolkit (streamsx.social)
– This repository provides support to ingest data from social media. Initial
support includes ingesting from Twitter

What’s New? – Analytics and Processing
 Anomaly Detection (streamsx.anomalyDetection)
– This repository contains operators and functions for anomaly detection. Initial
contribution would be based on the Local Outlier Function (LOF)
 Internet of Things Foundation Toolkit (streamsx.iotf)
– Provides the ability to have IBM Streams application to easily interact with
IoTF, either in Bluemix or on-premises.

Github Projects Overview – Utilities
 Network Toolkit (streamsx.network)
– Contains operators and functions for processing network data.
 Transform Toolkit (streamsx.transform)
– Contains building block operators to transform input data from one format to
another.
– Contains a high-performance Modify operator that can be used as a
replacement for Functor

Github Projects Overview – Samples / Demos / Docs
 Cybersecurity Starter Applications
(streamsx.cybersecurity.starterApps)
– Contains starter applications to get started with the IBM Streams
Cybersecurity Toolkit quickly
 Tutorials
– Contains Streams Introduction and SPSS labs
 Streamsx.documentation
– Community-based documentation
– Contains various getting started guides for Streams v4.1 features

Getting Started! - StreamsDev
https://developer.ibm.com/streamsdev/

Streamsx.documentation
 Community-based documentation project
 Complements StreamsDev and Knowledge Center
 Provides Getting Started Guides for new features in Streams
 Community contribution and feedback is much appreciated!
 http://ibmstreams.github.io/streamsx.documentation/

Get Involved!
 Many new and cool projects to try!
– E.g. Python Support, JDBC, Social
 We want your feedback and input!
– Report any issues you have found
– If you think it’s cool, let us know!
– Which toolkit do you use? How do you use it?
– Do you want the toolkit to be included in the product?
 Contribute code, samples and docs!
– Got a clever way to do things? Contribute it to the samples project!
– Need a new parameter? Need support for a new type of server / data format?
Work with us to try to implement it!
– Get a better way to explain things? Contribute to streamsx.documentation
 Got an idea?
– Propose a new project / new feature.

Questions?

Github Projects Overview and IBM Streams V4.1

More Related Content

What's hot

Viewers also liked

Similar to Github Projects Overview and IBM Streams V4.1

More from lisanl

Recently uploaded

Github Projects Overview and IBM Streams V4.1