The document discusses big data, defining it as very large data sets with high volumes, velocities, and varieties that are complex to process. It notes trends like Moore's Law, Kryder's Law, and Butter's Law that support big data growth by increasing processing and storage capabilities while decreasing costs. Examples of big data include NASA climate simulations, Large Hadron Collider data, and Walmart customer data. Tools for working with big data are discussed, including Hadoop, NoSQL databases, and MPP databases. The future is predicted to include continued data growth, lower storage and processing costs, higher network capacities, and potential for data growth to exceed processing abilities.

1. Big Data
● What is it ?
● What trends in IT support it ?
● Some examples
● What tools are used ?
● The future ?

2. Big Data – What is it ?
● Very large data set volumes
● Very long / unacceptable processing times
● Very large data velocity ( inputs / outputs )
● Very large varieties of data
● High level of complexity

3. Big Data – Supporting Trends
● Moore's Law
An observation that the number of transistors on integrated circuits doubles
every two years.

4. Big Data – Supporting Trends
● Kryder's Law
The density of storage is increasing and the cost decreasing at a rate faster
than Moore's Law.

5. Big Data – Supporting Trends
● Butter's Law
Relates to network capacity and states that the cost of sending data over an
optical network halves every nine months.

6. Big Data – Supporting Trends
● Parallel Processing
Task parallelism, breaking the task down into its constituent parts and
processing them simultaneously.

7. Big Data – Examples
● NASA Climate Simulation
32 petabytes
● The Large Hadron Collider
25 petabytes annually, 200 petabytes after replication
● Wall mart
2.5 petabytes per hour

8. Big Data – Tools
● Hadoop
Hadoop is often used at the server level to organise the cluster along with a
NoSQL database for data storage.
● NoSQL
Databases ( non sql ) that use looser consistency models than relational
databases. Performance gains via simplification using key value stores.
● MPP
Massively parallel processing and analytics databases. Fast for data
aggregation but slow for data loading.

9. Big Data – The Future
● Data sets will continue to grow
● Storage unit costs will continue to decrease
● Processing costs will decrease
● Network capacity will continue to grow
● Data growth may exceed processing capacity

10. Contact Us
● Feel free to contact us at
– www.semtech-solutions.co.nz
– info@semtech-solutions.co.nz
● We offer IT project consultancy
● We are happy to hear about your problems
● You can just pay for those hours that you need
● To solve your problems