In Cassandra Lunch #89, we will discuss how to store and parse semi-structured data in Cassandra using Spark Accompanying Blog: Coming Soon! Accompanying YouTube: https://youtu.be/ZhNnn51BRUc Sign Up For Our Newsletter: http://eepurl.com/grdMkn Join Cassandra Lunch Weekly at 12 PM EST Every Wednesday: https://www.meetup.com/Cassandra-DataStax-DC/events/ Cassandra.Link: https://cassandra.link/ Follow Us and Reach Us At: Anant: https://www.anant.us/ Awesome Cassandra: https://github.com/Anant/awesome-cassandra Cassandra.Lunch: https://github.com/Anant/Cassandra.Lunch Email: solutions@anant.us LinkedIn: https://www.linkedin.com/company/anant/ Twitter: https://twitter.com/anantcorp Eventbrite: https://www.eventbrite.com/o/anant-1072927283 Facebook: https://www.facebook.com/AnantCorp/ Join The Anant Team: https://www.careers.anant.us

1. Version 1.0
Semi-Structured Data in Cassandra
In Cassandra Lunch #89, we will discuss how to store and parse
semi-structured data in Cassandra using Spark
Stefan Nikolovski
Engineer @ Anant

2. Apache Spark
● Unified analytics engine for large-scale data processing.
● Achieves high performance for both batch and streaming
data, using a state-of-the-art DAG scheduler, a query
optimizer, and a physical execution engine.
● Offers over 80 high-level operators that make it easy to
build parallel apps. And you can use it interactively from
the Scala, Python, R, and SQL shells.
● Powers a stack of libraries including SQL and
DataFrames, MLlib for machine learning, GraphX, and
Spark Streaming. You can combine these libraries
seamlessly in the same application.
● You can run Spark using its standalone cluster mode, on
EC2, on Hadoop YARN, on Mesos, or on Kubernetes.
Access data in HDFS, Alluxio, Apache Cassandra, Apache
HBase, Apache Hive, and hundreds of other data sources.

3. Types of Data
● Structured (CSV)
● Semi-Structured (JSON,XML)
● Unstructured Data (Logs)

4. Structured Vs Unstructured Data
1. Defined Vs Undefined Data
2. Quantitative Vs Qualitative Data
3. Storage In Data Lakes Vs Data Warehouses
4. Ease Of Analysis

5. Data representation in Spark
○ RDD (Resilient Distributed Datasets)– RDD is a distributed collection of data elements spread across
many machines in the cluster. RDDs are a set of Java or Scala objects representing data.
○ DataFrame – A DataFrame is a distributed collection of data organized into named columns. It is
conceptually equal to a table in a relational database.
○ DataSet – DataSet is an extension of DataFrame API that provides the functionality of – type-safe,
object-oriented programming interface of the RDD API and performance benefits of the Catalyst query
optimizer and off-heap storage mechanism of a DataFrame API.

6. Demo
● Start Spark
● Load a Json File into a DataFrame
● Load that DataFrame into Cassandra
● Parse through xml inside Cassandra
● Load the data into a DataFrame
● Load that DataFrame into Cassandra

7. Strategy: Scalable Fast Data
Architecture: Cassandra, Spark, Kafka
Engineering: Node, Python, JVM,CLR
Operations: Cloud, Container
Rescue: Downtime!! I need help.
www.anant.us | solutions@anant.us | (855) 262-6826
3 Washington Circle, NW | Suite 301 | Washington, DC 20037