https://arcade.sqlbits.com/sessions/
Sql bits apache nifi 101 Introduction and best practices
11-March-2022 UK
https://www.datainmotion.dev/2020/06/no-more-spaghetti-flows.html
https://github.com/tspannhw/EverythingApacheNiFi
https://www.datainmotion.dev/2020/12/basic-understanding-of-cloudera-flow.html
https://www.datainmotion.dev/2020/10/top-25-use-cases-of-cloudera-flow.html
In this talk, we will walk step by step through Apache NiFi from the first load to first application. I will include slides, articles and examples to take away as a Quick Start to utilizing Apache NiFi in your real-time dataflows. I will help you get up and running locally on your laptop, Docker or in CDP Public Cloud.
I will cover:
Terminology
Flow Files
Version Control
Repositories
Basic Record Processing
Provenance
Backpressure
Prioritizers
System Diagnostics
Processors
Process Groups
Scheduling and Cron
Bulletin Board
Relationships
Routing
Tasks
Networking
Basic Cluster Architecture
Listeners
Controller Services
Remote Ports
Handling Errors
Funnels
Feedback LInk - https://sqlb.it/?7108
ROOM 04 Fri 12:00 - 12:50
2. Question Everything!
Session
Regular 50 minute session
Apache NiFi 101: Introduction
and Best Practices
Primary Speaker
Fri 12:00
Feedback Link
https://sqlb.it/?7108
3. Timothy Spann | Developer
Advocate
FLiP(N) Stack = Flink, Pulsar and NiFI Stack
Streaming Systems & Data Architecture Expert
Experience:
15+ years of experience with streaming technologies
including Pulsar, Flink, Spark, NiFi, Kafka, Big Data,
Cloud, ML, IoT and more.
Today, he helps to grow the Pulsar community sharing
rich technical knowledge and experience at both global
conferences and through individual conversations
4. streamnative.io
Passionate and dedicated team.
Founded by the original developers of
Apache Pulsar.
StreamNative helps teams to capture,
manage, and leverage data using Pulsarās
uniļ¬ed messaging and streaming
platform.
5. FLiP Stack Weekly
This week in Apache Flink, Apache Pulsar, Apache
NiFi, Apache Spark and open source friends.
https://bit.ly/32dAJft
8. Why Apache NiFi?
ā¢ Guaranteed delivery
ā¢ Data buffering
- Backpressure
- Pressure release
ā¢ Prioritized queuing
ā¢ Flow specific QoS
- Latency vs. throughput
- Loss tolerance
ā¢ Data provenance
ā¢ Supports push and pull
models
ā¢ 350+ processors
ā¢ Visual command and
control
ā¢ Over a 100 sources
ā¢ Flow templates
ā¢ Pluggable/multi-role
security
ā¢ Designed for extension
ā¢ Clustering
ā¢ Version Control
29. Toolkit Setup on Apache NiFi 1.15.3
Download NiFi Toolkit
Copy keystore and truststore information from your NiFi conf/nifi.properties
Create a nifi.properties file linked to the cli.sh
baseUrl=https://nvidia-desktop:8443
keystore=/home/nvidia/nvme/nifi-1.15.3/conf/keystore.p12
keystoreType=PKCS12
keystorePasswd=5325343412efaab3123c6892d93
keyPasswd=53134eee99da9dbe9349123aa17c6892d93
truststore=/home/nvidia/nvme/nifi-1.15.3/conf/truststore.p12
truststoreType=PKCS12
truststorePasswd=93498Dfdjfhujdhure8d8hfd84j3n43jd
37. Serverless Event
Streaming
Framework
ā¢ Lightweight computation similar
to AWS Lambda.
ā¢ Speciļ¬cally designed to use
Apache Pulsar as a message bus.
ā¢ Function runtime can be located
within Pulsar Broker.
ā¢ Java, Go, Python
https://streamnative.io/blog/engineering/2021-11-10-streaming-data-pipelines-with-pulsar-io/