In this slide deck, Anjana will explore data integration with Ballerina.

1. Architect, WSO2
Ballerina: Exploring Data Integration
Anjana Fernando

2. • Source: Heterogeneous Data Sources
• Processing:
– Data Quality/Cleansing
– ETL
• Store/Mining:
– Data Warehouse
• Expose:
– Data Services
Data Integration Explained

3. • Built-in support for popular data formats,
XML/JSON/Tables
– Natural manipulation of data
• Extensive Data Connector Support (SQL, NoSQL)
• Transactions (Local, Distributed)
• Graphical data modelling with Composer
• Scheduled data processing jobs with tasks
• Rich HTTP REST service development support for
rapid data services exposure
How Ballerina Fits In

4. • Value Types
– int/float/boolean/string/blob
• Arrays
– type[]
• Maps
– map
• XML
– “xml” type with fully functional XML sequence API
Ballerina Data Types

5. • JSON
– “json” type with constrained JSON support for
schema enforcement
• Datatable
– “datatable” type in representing a resultset from
data connectors
• User defined types
– “struct” type for data binding
– Automatic data validation
– Record representation in datatable type
Ballerina Data Types

6. Data Connectors
• SQL Connector (built-in)
– MySQL, MSSQL, Oracle, DB2...
– Supports DML / DDL operations
– Stored procedures / batch
operations
– UDT support
– XA transaction support
• Multiple databases / Message
queues / topics

7. Data Connectors...
• SQL Connector

8. Data Connectors...
• Cassandra Connector (https://github.com/ballerinalang/connector-cassandra)
– CQL based data querying
– Streaming results with datatable integration

9. Data Connectors...
• Cassandra Connector

10. Data Connectors...
• MongoDB Connector (https://github.com/ballerinalang/connector-mongodb)
– MongoDB CRUD API support
• insert/find/update/delete
• Native JSON data handling
– Ballerina ‘json’ data type
• JSON streaming support

11. Data Connectors...
• MongoDB Connector

12. Transactions
• Local / XA Transactions
– Declarative transactions by defining a
transaction boundary in the program
– The mode is inferred from the participants in
the transaction scope
– Currently support SQL databases and
message brokers for distributed transactions

13. Transactions...

14. Graphical Data Modelling
Data Mapping with Ballerina Transformer

15. • ETL jobs can be scheduled with the Ballerina tasks
features:
– Timers: delay + interval scheduling
– Appointments: cron based scheduling
Scheduled Data Jobs
https://www.informatec.com/sites/default/files/inline-images/etl_2.jpg,
https://www.cosculture.com/static/images/Icon---Scheduled-Publisher@270.png

16. Scheduled Data Jobs
Timers:-

17. Scheduled Data Jobs
Appointments:-

18. • HTTP/WS server connectors
• Ability to generate REST APIs rapidly
• Native data streaming capability
– XML / JSON
• Batch data request support for high performance data
loading
• Power of a general purpose programming language
with the ease of a visual programming approach
– Data preprocessing, filtering, access control etc...
Data Services

19. Data Services...

20. • Ballerina provides an efficient set of connectors, tools
and technologies for easy data access/manipulation
and serving.
More information, visit https://ballerinalang.org/.
Summary

21. wso2.com