2. Background
Today’s data landscape for enterprises continues to grow exponentially in
volume, variety, and complexity.
Multiple geographic locations, on-premises and cloud
Combination of open source, commercial solutions and custom processing code
Can be expensive, hard to integrate and maintain.
Ever increasing volumes of data (terabytes, petabytes)
New ways of processing data (Hadoop, Spark etc.)
.NET Developers write large amounts of custom point-solution logic
Difficult to maintain and orchestrate
Performance bottlenecks
3. SparkPipe Framework
A development framework to deliver a .NET information production system
that co-ordinates all of this data and processing.
Familiar technologies for .NET developers including
.NET Framework 4.0
Windows Workflow Foundation
Task Parallel Library Dataflow
Drag and drop business process pipeline modeling
Designed for performance to scale across processor cores and servers
from the local data center to cloud providers such as Microsoft Azure
4. Build Solutions
Build data-driven workflows (pipelines) that join, aggregate and transform
data sourced from on-premises, cloud-based, and internet data stores.
Transform semi-structured, unstructured and structured data from diverse
data sources into trusted information.
Produce data that can be easily consumed by using business intelligence
(BI), analytics tools, and other applications.
Set up complex data processing through simple composing.
6. Built for “Cloud Scale”
Support for Microsoft Azure offerings including:
Azure SQL Server
HDInsight (HADOOP)
Blob, Tables, Queues and ServiceBus
Automatically spin-up cloud servers, process data and then shut down to
for cost-effective processing.
7. Support for Healthcare
Out of the box components include:
HL7 v2
Clinical Document Architecture
EDI 834
PGP Encryption
Secure FTP