10 minute command line apps with zio cli
•Download as PPTX, PDF•
0 likes•751 views
Command-line applications are pervasive, relied upon by developers,third-parties, and IT staff. While it’s easy to develop a basic command-line application, building one that features rich help, forgiving parsing, bulletproof validation, shell completions, and a clean and beautiful appearance is quite challenging. In this talk, Aiswarya Prakasan, contributor to ZIO CLI, will lead you on a journey to building powerful command-line applications in Scala. Through the power of composition and strong types, you will discover how to build type-safe command-line applications in just a few simple lines of code, which give you rich features of command-line applications like git.
Report
Share
Report
Share
Recommended
2022-06-23 Apache Arrow and DataFusion_ Changing the Game for implementing Da...
DataFusion is an extensible and embeddable query engine, written in Rust used to create modern, fast and efficient data pipelines, ETL processes, and database systems.
This presentation explains where it fits into the data eco system and how it helps implement your system in Rust
Data all over the place! How SQL and Apache Calcite bring sanity to streaming...
The revolution has happened. We are living the age of the deconstructed database. The modern enterprises are powered by data, and that data lives in many formats and locations, in-flight and at rest, but somewhat surprisingly, the lingua franca for remains SQL.
In this talk, Julian describes Apache Calcite, a toolkit for relational algebra that powers many systems including Apache Beam, Flink and Hive. He discusses some areas of development in Calcite: streaming SQL, materialized views, enabling spatial query on vanilla databases, and what a mash-up of all three might look like.
He also describes how SQL is being extended to handle streaming, and the challenges that will need to be solved if it is to become standard.
A talk given by Julian Hyde at Lyft, San Francisco, on 2018/06/27.
Understanding and Improving Code Generation
Code generation is integral to Spark’s physical execution engine. When implemented, the Spark engine creates optimized bytecode at runtime improving performance when compared to interpreted execution. Spark has taken the next step with whole-stage codegen which collapses an entire query into a single function.
Using LLVM to accelerate processing of data in Apache Arrow
Most query engines follow an interpreter-based approach where a SQL query is translated into a tree of relational algebra operations then fed through a conventional tuple-based iterator model to execute the query. We will explore the overhead associated with this approach and how the performance of query execution on columnar data can be improved using run-time code generation via LLVM.
Generally speaking, the best case for optimal query execution performance is a hand-written query plan that does exactly what is needed by the query for the exact same data types and format. Vectorized query processing models amortize the cost of function calls. However, research has shown that hand-written code for a given query plan has the potential to outperform the optimizations associated with a vectorized query processing model.
Over the last decade, the LLVM compiler framework has seen significant development. Furthermore, the database community has realized the potential of LLVM to boost query performance by implementing JIT query compilation frameworks. With LLVM, a SQL query is translated into a portable intermediary representation (IR) which is subsequently converted into machine code for the desired target architecture.
Dremio is built on top of Apache Arrow’s in-memory columnar vector format. The in-memory vectors map directly to the vector type in LLVM and that makes our job easier when writing the query processing algorithms in LLVM. We will talk about how Dremio implemented query processing logic in LLVM for some operators like FILTER and PROJECT. We will also discuss the performance benefits of LLVM-based vectorized query execution over other methods.
Speaker
Siddharth Teotia, Dremio, Software Engineer
Apache Calcite: One planner fits all
This document discusses how Apache Calcite makes it easier to write database management systems (DBMS) by decomposing them into modular components like a query parser, catalog, algorithms, and storage engines. It presents Calcite as a framework that allows these components to be mixed and matched, with a core relational algebra and rule-based optimization. Calcite powers systems like Apache Hive, Drill, Phoenix, and Kylin by translating SQL and other queries to relational algebra and optimizing queries using over 100 rules before executing them using configurable engines and data sources.
Handle Large Messages In Apache Kafka
Like many other messaging systems, Kafka has put limit on the maximum message size. User will fail to produce a message if it is too large. This limit makes a lot of sense and people usually send to Kafka a reference link which refers to a large message stored somewhere else. However, in some scenarios, it would be good to be able to send messages through Kafka without external storage. At LinkedIn, we have a few use cases that can benefit from such feature. This talk covers our solution to send large message through Kafka without additional storage.
Cloud Native Java GraalVM 이상과 현실
This document discusses GraalVM and its key components. GraalVM is a virtual machine that supports multiple languages including Java, JavaScript, Ruby, and Python. It includes Graal, a new just-in-time compiler that aims to replace the C2 compiler in HotSpot VM. GraalVM also features Truffle, a framework for building languages, and Native Image, an ahead-of-time compiler.
XStream: stream processing platform at facebook
XStream is Facebook's unified stream processing platform that provides a fully managed stream processing service. It was built using the Stylus C++ stream processing framework and uses a common SQL dialect called CoreSQL. XStream employs an interpretive execution model using the new Velox vectorized SQL evaluation engine for high performance. This provides a consistent and high efficiency stream processing platform to support diverse real-time use cases at planetary scale for Facebook.
Recommended
2022-06-23 Apache Arrow and DataFusion_ Changing the Game for implementing Da...
DataFusion is an extensible and embeddable query engine, written in Rust used to create modern, fast and efficient data pipelines, ETL processes, and database systems.
This presentation explains where it fits into the data eco system and how it helps implement your system in Rust
Data all over the place! How SQL and Apache Calcite bring sanity to streaming...
The revolution has happened. We are living the age of the deconstructed database. The modern enterprises are powered by data, and that data lives in many formats and locations, in-flight and at rest, but somewhat surprisingly, the lingua franca for remains SQL.
In this talk, Julian describes Apache Calcite, a toolkit for relational algebra that powers many systems including Apache Beam, Flink and Hive. He discusses some areas of development in Calcite: streaming SQL, materialized views, enabling spatial query on vanilla databases, and what a mash-up of all three might look like.
He also describes how SQL is being extended to handle streaming, and the challenges that will need to be solved if it is to become standard.
A talk given by Julian Hyde at Lyft, San Francisco, on 2018/06/27.
Understanding and Improving Code Generation
Code generation is integral to Spark’s physical execution engine. When implemented, the Spark engine creates optimized bytecode at runtime improving performance when compared to interpreted execution. Spark has taken the next step with whole-stage codegen which collapses an entire query into a single function.
Using LLVM to accelerate processing of data in Apache Arrow
Most query engines follow an interpreter-based approach where a SQL query is translated into a tree of relational algebra operations then fed through a conventional tuple-based iterator model to execute the query. We will explore the overhead associated with this approach and how the performance of query execution on columnar data can be improved using run-time code generation via LLVM.
Generally speaking, the best case for optimal query execution performance is a hand-written query plan that does exactly what is needed by the query for the exact same data types and format. Vectorized query processing models amortize the cost of function calls. However, research has shown that hand-written code for a given query plan has the potential to outperform the optimizations associated with a vectorized query processing model.
Over the last decade, the LLVM compiler framework has seen significant development. Furthermore, the database community has realized the potential of LLVM to boost query performance by implementing JIT query compilation frameworks. With LLVM, a SQL query is translated into a portable intermediary representation (IR) which is subsequently converted into machine code for the desired target architecture.
Dremio is built on top of Apache Arrow’s in-memory columnar vector format. The in-memory vectors map directly to the vector type in LLVM and that makes our job easier when writing the query processing algorithms in LLVM. We will talk about how Dremio implemented query processing logic in LLVM for some operators like FILTER and PROJECT. We will also discuss the performance benefits of LLVM-based vectorized query execution over other methods.
Speaker
Siddharth Teotia, Dremio, Software Engineer
Apache Calcite: One planner fits all
This document discusses how Apache Calcite makes it easier to write database management systems (DBMS) by decomposing them into modular components like a query parser, catalog, algorithms, and storage engines. It presents Calcite as a framework that allows these components to be mixed and matched, with a core relational algebra and rule-based optimization. Calcite powers systems like Apache Hive, Drill, Phoenix, and Kylin by translating SQL and other queries to relational algebra and optimizing queries using over 100 rules before executing them using configurable engines and data sources.
Handle Large Messages In Apache Kafka
Like many other messaging systems, Kafka has put limit on the maximum message size. User will fail to produce a message if it is too large. This limit makes a lot of sense and people usually send to Kafka a reference link which refers to a large message stored somewhere else. However, in some scenarios, it would be good to be able to send messages through Kafka without external storage. At LinkedIn, we have a few use cases that can benefit from such feature. This talk covers our solution to send large message through Kafka without additional storage.
Cloud Native Java GraalVM 이상과 현실
This document discusses GraalVM and its key components. GraalVM is a virtual machine that supports multiple languages including Java, JavaScript, Ruby, and Python. It includes Graal, a new just-in-time compiler that aims to replace the C2 compiler in HotSpot VM. GraalVM also features Truffle, a framework for building languages, and Native Image, an ahead-of-time compiler.
XStream: stream processing platform at facebook
XStream is Facebook's unified stream processing platform that provides a fully managed stream processing service. It was built using the Stylus C++ stream processing framework and uses a common SQL dialect called CoreSQL. XStream employs an interpretive execution model using the new Velox vectorized SQL evaluation engine for high performance. This provides a consistent and high efficiency stream processing platform to support diverse real-time use cases at planetary scale for Facebook.
Optimizing Delta/Parquet Data Lakes for Apache Spark
Matthew Powers gave a presentation on optimizing Delta and Parquet data lakes. He discussed the benefits of using Delta lakes such as built-in time travel, compacting, and vacuuming capabilities. Delta lakes provide these features for free on top of Parquet files and a transaction log. Powers demonstrated how to create, compact, vacuum, partition, filter, and update Delta lakes in Spark. He showed that partitioning data significantly improves query performance by enabling data skipping and filtering at the partition level.
Improving SparkSQL Performance by 30%: How We Optimize Parquet Pushdown and P...
The document discusses optimizations made to Spark SQL performance when working with Parquet files at ByteDance. It describes how Spark originally reads Parquet files and identifies two main areas for optimization: Parquet filter pushdown and the Parquet reader. For filter pushdown, sorting columns improved statistics and reduced data reads by 30%. For the reader, splitting it to first filter then read other columns prevented loading unnecessary data. These changes improved Spark SQL performance at ByteDance without changing jobs.
Postgresql database administration volume 1
The paperback version is available on lulu.com there http://goo.gl/fraa8o
This is the first volume of the postgresql database administration book. The book covers the steps for installing, configuring and administering a PostgreSQL 9.3 on Linux debian. The book covers the logical and physical aspect of PostgreSQL. Two chapters are dedicated to the backup/restore topic.
tmux
In this talk, Jason will introduce tmux, the terminal multiplexer. He’ll cover why you’d want to use it, and then teach you how to actually use it with practical examples. He’ll teach you his favorite key bindings, and then go into some incredibly handy plugins that you can use that will make your tmux experience so much better. Once you are done learning tmux, you’ll never understand how you got along without it.
Spark + Parquet In Depth: Spark Summit East Talk by Emily Curtin and Robbie S...
What if you could get the simplicity, convenience, interoperability, and storage niceties of an old-fashioned CSV with the speed of a NoSQL database and the storage requirements of a gzipped file? Enter Parquet.
At The Weather Company, Parquet files are a quietly awesome and deeply integral part of our Spark-driven analytics workflow. Using Spark + Parquet, we’ve built a blazing fast, storage-efficient, query-efficient data lake and a suite of tools to accompany it.
We will give a technical overview of how Parquet works and how recent improvements from Tungsten enable SparkSQL to take advantage of this design to provide fast queries by overcoming two major bottlenecks of distributed analytics: communication costs (IO bound) and data decoding (CPU bound).
Everyday I'm Shuffling - Tips for Writing Better Spark Programs, Strata San J...
Watch video at: http://youtu.be/Wg2boMqLjCg
Want to learn how to write faster and more efficient programs for Apache Spark? Two Spark experts from Databricks, Vida Ha and Holden Karau, provide some performance tuning and testing tips for your Spark applications
Frame - Feature Management for Productive Machine Learning
Presented at the ML Platforms Meetup at Pinterest HQ in San Francisco on August 16, 2018.
Abstract: At LinkedIn we observed that much of the complexity in our machine learning applications was in their feature preparation workflows. To address this problem, we built Frame, a shared virtual feature store that provides a unified abstraction layer for accessing features by name. Frame removes the need for feature consumers to deal directly with underlying data sources, which are often different across computing environments. By simplifying feature preparation, Frame has made ML applications at LinkedIn easier to build, modify, and understand.
[若渴計畫] Challenges and Solutions of Window Remote Shellcode
This document discusses challenges and solutions related to window remote shellcode. It outlines challenges posed by antivirus software, EMET, firewalls, and IDS/IPS systems. It then describes various techniques for bypassing these protections, such as encryption, obfuscation, non-standard programming languages, and the use of tools like Meterpreter and Veil Framework payloads. Specific bypass techniques covered include DLL injection, process hollowing, reflective loading, and the use of techniques like one-way shells and HTTP stagers.
Incremental View Maintenance with Coral, DBT, and Iceberg
These slides present how DBT, Coral, and Iceberg can provide a novel data management experience for defining SQL workflows. In this UX, users define their workflows as a cascade of SQL queries, which then get auto-materialized and incrementally maintained. Applications of this user experience include Declarative DAG workflows, streaming/batch convergence, and materialized views.
Apache Spark Architecture
This is the presentation I made on JavaDay Kiev 2015 regarding the architecture of Apache Spark. It covers the memory model, the shuffle implementations, data frames and some other high-level staff and can be used as an introduction to Apache Spark
Arbitrary Stateful Aggregations using Structured Streaming in Apache Spark
In this talk, we will introduce some of the new available APIs around stateful aggregation in Structured Streaming, namely flatMapGroupsWithState. We will show how this API can be used to power many complex real-time workflows, including stream-to-stream joins, through live demos using Databricks and Apache Kafka.
Bootstrapping state in Apache Flink
Apache Flink is a popular stream computing framework for real-time stream computing. Many stream compute algorithms require trailing data in order to compute the intended result. One example is computing the number of user logins in the last 7 days. This creates a dilemma where the results of the stream program are incomplete until the runtime of the program exceeds 7 days. The alternative is to bootstrap the program using historic data to seed the state before shifting to use real-time data.
This talk will discuss alternatives to bootstrap programs in Flink. Some alternatives rely on technologies exogenous to the stream program, such as enhancements to the pub/sub layer, that are more generally applicable to other stream compute engines. Other alternatives include enhancements to Flink source implementations. Lyft is exploring another alternative using orchestration of multiple Flink programs. The talk will cover why Lyft pursued this alternative and future directions to further enhance bootstrapping support in Flink.
Speaker
Gregory Fee, Principal Engineer, Lyft
Real-time Analytics with Trino and Apache Pinot
Trino summit 2021:
Overview of Trino Pinot Connector, which bridges the flexibility of Trino's full SQL support to the power of Apache Pinot's realtime analytics, giving you the best of both worlds.
Common Strategies for Improving Performance on Your Delta Lakehouse
The Delta Architecture pattern has made the lives of data engineers much simpler, but what about improving query performance for data analysts? What are some common places to look at for tuning query performance? In this session we will cover some common techniques to apply to our delta tables to make them perform better for data analysts queries. We will look at a few examples of how you can analyze a query, and determine what to focus on to deliver better performance results.
Spark Operator—Deploy, Manage and Monitor Spark clusters on Kubernetes
The document discusses the Spark Operator, which allows deploying, managing, and monitoring Spark clusters on Kubernetes. It describes how the operator extends Kubernetes by defining custom resources and reacting to events from those resources, such as SparkCluster, SparkApplication, and SparkHistoryServer. The operator takes care of common tasks to simplify running Spark on Kubernetes and hides the complexity through an abstract operator library.
Data Ingest Self Service and Management using Nifi and Kafka
We’re feeling the growing pains of maintaining a large data platform. Last year we went from 50 to 150 unique data feeds by adding them all by hand. In this talk we will share the best practices developed to handle our 300% increase in feeds through self service. Having self-service capabilities will increase your teams velocity and decrease your time to value and insight.
* Self service data feed design and ingest
* configuration management
* automatic debugging
* light weight data governance
Parquet performance tuning: the missing guide
Parquet performance tuning focuses on optimizing Parquet reads by leveraging columnar organization, encoding, and filtering techniques. Statistics and dictionary filtering can eliminate unnecessary data reads by filtering at the row group and page levels. However, these optimizations require columns to be sorted and fully dictionary encoded within files. Increasing dictionary size thresholds and decreasing row group sizes can help avoid dictionary encoding fallback and improve filtering effectiveness. Future work may include new encodings, compression algorithms like Brotli, and page-level filtering in the Parquet format.
Securing k8s With Kubernetes Goat
Kubernetes Cloud Native Jakarta Online Meetup #18
At this talk, i will give story about how to securing kubernetes using kubernetes goat playground with demo.
Url: https://www.meetup.com/jakarta-kubernetes/events/272455638/
NGINX ADC: Basics and Best Practices
On-Demand: https://www.nginx.com/resources/webinars/nginx-adc-basics-best-practices/
In this webinar, we help you get started with NGINX, industry’s most ubiquitous web server and API gateway. We cover best practices for installing, configuring, and troubleshooting both NGINX Open Source and the enterprise-grade NGINX Plus. We provide insights about using NGINX Controller to manage your NGINX Plus instances.
Watch this webinar to learn:
- How to create NGINX configurations for web server, load balancer, etc.
- About improving performance using keepalives and other NGINX directives
- How the NGINX Controller Load Balancing Module can manage NGINX Plus instances at scale
- About augmenting your existing ADC with NGINX
ceph::errorator<> throw/catch-free, compile time-checked exceptions for seast...
The document discusses an error handling library called "errorator" that provides compile-time checked exceptions for seastar futures. Errorator avoids exceptions by using error codes and allows type-safe error handling without performance penalties from exceptions. It works by creating a specialized future type for each set of allowed error codes. Errorator futures can safely chain callbacks while preserving error type information without exceptions. This avoids issues with scalability of exceptions in high concurrency applications like seastar.
Kandroid for nhn_deview_20131013_v5_final
The document provides information about high performance Android app development. It begins with a history of Android performance features from early versions through Jellybean and Project Butter. It then compares the three Android programming models (SDK, NDK, RenderScript) in terms of workflow, execution model, and performance. A case study on the performance features of the Google Chrome browser for Android is presented, covering its multi-process architecture, hardware acceleration, networking, and VSync scheduling. The document concludes with a questionnaire on topics like multi-core vs GPU, Android vs Chrome, and developments beyond Android.
DevOpsDays Taipei 2019 - Mastering IaC the DevOps Way
How to let IaC support Unit Testing by Terratest, Integration Testing by Open Policy Agent, and GitOps by Atlantis
More Related Content
What's hot
Optimizing Delta/Parquet Data Lakes for Apache Spark
Matthew Powers gave a presentation on optimizing Delta and Parquet data lakes. He discussed the benefits of using Delta lakes such as built-in time travel, compacting, and vacuuming capabilities. Delta lakes provide these features for free on top of Parquet files and a transaction log. Powers demonstrated how to create, compact, vacuum, partition, filter, and update Delta lakes in Spark. He showed that partitioning data significantly improves query performance by enabling data skipping and filtering at the partition level.
Improving SparkSQL Performance by 30%: How We Optimize Parquet Pushdown and P...
The document discusses optimizations made to Spark SQL performance when working with Parquet files at ByteDance. It describes how Spark originally reads Parquet files and identifies two main areas for optimization: Parquet filter pushdown and the Parquet reader. For filter pushdown, sorting columns improved statistics and reduced data reads by 30%. For the reader, splitting it to first filter then read other columns prevented loading unnecessary data. These changes improved Spark SQL performance at ByteDance without changing jobs.
Postgresql database administration volume 1
The paperback version is available on lulu.com there http://goo.gl/fraa8o
This is the first volume of the postgresql database administration book. The book covers the steps for installing, configuring and administering a PostgreSQL 9.3 on Linux debian. The book covers the logical and physical aspect of PostgreSQL. Two chapters are dedicated to the backup/restore topic.
tmux
In this talk, Jason will introduce tmux, the terminal multiplexer. He’ll cover why you’d want to use it, and then teach you how to actually use it with practical examples. He’ll teach you his favorite key bindings, and then go into some incredibly handy plugins that you can use that will make your tmux experience so much better. Once you are done learning tmux, you’ll never understand how you got along without it.
Spark + Parquet In Depth: Spark Summit East Talk by Emily Curtin and Robbie S...
What if you could get the simplicity, convenience, interoperability, and storage niceties of an old-fashioned CSV with the speed of a NoSQL database and the storage requirements of a gzipped file? Enter Parquet.
At The Weather Company, Parquet files are a quietly awesome and deeply integral part of our Spark-driven analytics workflow. Using Spark + Parquet, we’ve built a blazing fast, storage-efficient, query-efficient data lake and a suite of tools to accompany it.
We will give a technical overview of how Parquet works and how recent improvements from Tungsten enable SparkSQL to take advantage of this design to provide fast queries by overcoming two major bottlenecks of distributed analytics: communication costs (IO bound) and data decoding (CPU bound).
Everyday I'm Shuffling - Tips for Writing Better Spark Programs, Strata San J...
Watch video at: http://youtu.be/Wg2boMqLjCg
Want to learn how to write faster and more efficient programs for Apache Spark? Two Spark experts from Databricks, Vida Ha and Holden Karau, provide some performance tuning and testing tips for your Spark applications
Frame - Feature Management for Productive Machine Learning
Presented at the ML Platforms Meetup at Pinterest HQ in San Francisco on August 16, 2018.
Abstract: At LinkedIn we observed that much of the complexity in our machine learning applications was in their feature preparation workflows. To address this problem, we built Frame, a shared virtual feature store that provides a unified abstraction layer for accessing features by name. Frame removes the need for feature consumers to deal directly with underlying data sources, which are often different across computing environments. By simplifying feature preparation, Frame has made ML applications at LinkedIn easier to build, modify, and understand.
[若渴計畫] Challenges and Solutions of Window Remote Shellcode
This document discusses challenges and solutions related to window remote shellcode. It outlines challenges posed by antivirus software, EMET, firewalls, and IDS/IPS systems. It then describes various techniques for bypassing these protections, such as encryption, obfuscation, non-standard programming languages, and the use of tools like Meterpreter and Veil Framework payloads. Specific bypass techniques covered include DLL injection, process hollowing, reflective loading, and the use of techniques like one-way shells and HTTP stagers.
Incremental View Maintenance with Coral, DBT, and Iceberg
These slides present how DBT, Coral, and Iceberg can provide a novel data management experience for defining SQL workflows. In this UX, users define their workflows as a cascade of SQL queries, which then get auto-materialized and incrementally maintained. Applications of this user experience include Declarative DAG workflows, streaming/batch convergence, and materialized views.
Apache Spark Architecture
This is the presentation I made on JavaDay Kiev 2015 regarding the architecture of Apache Spark. It covers the memory model, the shuffle implementations, data frames and some other high-level staff and can be used as an introduction to Apache Spark
Arbitrary Stateful Aggregations using Structured Streaming in Apache Spark
In this talk, we will introduce some of the new available APIs around stateful aggregation in Structured Streaming, namely flatMapGroupsWithState. We will show how this API can be used to power many complex real-time workflows, including stream-to-stream joins, through live demos using Databricks and Apache Kafka.
Bootstrapping state in Apache Flink
Apache Flink is a popular stream computing framework for real-time stream computing. Many stream compute algorithms require trailing data in order to compute the intended result. One example is computing the number of user logins in the last 7 days. This creates a dilemma where the results of the stream program are incomplete until the runtime of the program exceeds 7 days. The alternative is to bootstrap the program using historic data to seed the state before shifting to use real-time data.
This talk will discuss alternatives to bootstrap programs in Flink. Some alternatives rely on technologies exogenous to the stream program, such as enhancements to the pub/sub layer, that are more generally applicable to other stream compute engines. Other alternatives include enhancements to Flink source implementations. Lyft is exploring another alternative using orchestration of multiple Flink programs. The talk will cover why Lyft pursued this alternative and future directions to further enhance bootstrapping support in Flink.
Speaker
Gregory Fee, Principal Engineer, Lyft
Real-time Analytics with Trino and Apache Pinot
Trino summit 2021:
Overview of Trino Pinot Connector, which bridges the flexibility of Trino's full SQL support to the power of Apache Pinot's realtime analytics, giving you the best of both worlds.
Common Strategies for Improving Performance on Your Delta Lakehouse
The Delta Architecture pattern has made the lives of data engineers much simpler, but what about improving query performance for data analysts? What are some common places to look at for tuning query performance? In this session we will cover some common techniques to apply to our delta tables to make them perform better for data analysts queries. We will look at a few examples of how you can analyze a query, and determine what to focus on to deliver better performance results.
Spark Operator—Deploy, Manage and Monitor Spark clusters on Kubernetes
The document discusses the Spark Operator, which allows deploying, managing, and monitoring Spark clusters on Kubernetes. It describes how the operator extends Kubernetes by defining custom resources and reacting to events from those resources, such as SparkCluster, SparkApplication, and SparkHistoryServer. The operator takes care of common tasks to simplify running Spark on Kubernetes and hides the complexity through an abstract operator library.
Data Ingest Self Service and Management using Nifi and Kafka
We’re feeling the growing pains of maintaining a large data platform. Last year we went from 50 to 150 unique data feeds by adding them all by hand. In this talk we will share the best practices developed to handle our 300% increase in feeds through self service. Having self-service capabilities will increase your teams velocity and decrease your time to value and insight.
* Self service data feed design and ingest
* configuration management
* automatic debugging
* light weight data governance
Parquet performance tuning: the missing guide
Parquet performance tuning focuses on optimizing Parquet reads by leveraging columnar organization, encoding, and filtering techniques. Statistics and dictionary filtering can eliminate unnecessary data reads by filtering at the row group and page levels. However, these optimizations require columns to be sorted and fully dictionary encoded within files. Increasing dictionary size thresholds and decreasing row group sizes can help avoid dictionary encoding fallback and improve filtering effectiveness. Future work may include new encodings, compression algorithms like Brotli, and page-level filtering in the Parquet format.
Securing k8s With Kubernetes Goat
Kubernetes Cloud Native Jakarta Online Meetup #18
At this talk, i will give story about how to securing kubernetes using kubernetes goat playground with demo.
Url: https://www.meetup.com/jakarta-kubernetes/events/272455638/
NGINX ADC: Basics and Best Practices
On-Demand: https://www.nginx.com/resources/webinars/nginx-adc-basics-best-practices/
In this webinar, we help you get started with NGINX, industry’s most ubiquitous web server and API gateway. We cover best practices for installing, configuring, and troubleshooting both NGINX Open Source and the enterprise-grade NGINX Plus. We provide insights about using NGINX Controller to manage your NGINX Plus instances.
Watch this webinar to learn:
- How to create NGINX configurations for web server, load balancer, etc.
- About improving performance using keepalives and other NGINX directives
- How the NGINX Controller Load Balancing Module can manage NGINX Plus instances at scale
- About augmenting your existing ADC with NGINX
ceph::errorator<> throw/catch-free, compile time-checked exceptions for seast...
The document discusses an error handling library called "errorator" that provides compile-time checked exceptions for seastar futures. Errorator avoids exceptions by using error codes and allows type-safe error handling without performance penalties from exceptions. It works by creating a specialized future type for each set of allowed error codes. Errorator futures can safely chain callbacks while preserving error type information without exceptions. This avoids issues with scalability of exceptions in high concurrency applications like seastar.
What's hot (20)
Optimizing Delta/Parquet Data Lakes for Apache Spark
Optimizing Delta/Parquet Data Lakes for Apache Spark
Improving SparkSQL Performance by 30%: How We Optimize Parquet Pushdown and P...
Improving SparkSQL Performance by 30%: How We Optimize Parquet Pushdown and P...
Spark + Parquet In Depth: Spark Summit East Talk by Emily Curtin and Robbie S...
Spark + Parquet In Depth: Spark Summit East Talk by Emily Curtin and Robbie S...
Everyday I'm Shuffling - Tips for Writing Better Spark Programs, Strata San J...
Everyday I'm Shuffling - Tips for Writing Better Spark Programs, Strata San J...
Frame - Feature Management for Productive Machine Learning
Frame - Feature Management for Productive Machine Learning
[若渴計畫] Challenges and Solutions of Window Remote Shellcode
[若渴計畫] Challenges and Solutions of Window Remote Shellcode
Incremental View Maintenance with Coral, DBT, and Iceberg
Incremental View Maintenance with Coral, DBT, and Iceberg
Arbitrary Stateful Aggregations using Structured Streaming in Apache Spark
Arbitrary Stateful Aggregations using Structured Streaming in Apache Spark
Common Strategies for Improving Performance on Your Delta Lakehouse
Common Strategies for Improving Performance on Your Delta Lakehouse
Spark Operator—Deploy, Manage and Monitor Spark clusters on Kubernetes
Spark Operator—Deploy, Manage and Monitor Spark clusters on Kubernetes
Data Ingest Self Service and Management using Nifi and Kafka
Data Ingest Self Service and Management using Nifi and Kafka
ceph::errorator<> throw/catch-free, compile time-checked exceptions for seast...
ceph::errorator<> throw/catch-free, compile time-checked exceptions for seast...
Similar to 10 minute command line apps with zio cli
Kandroid for nhn_deview_20131013_v5_final
The document provides information about high performance Android app development. It begins with a history of Android performance features from early versions through Jellybean and Project Butter. It then compares the three Android programming models (SDK, NDK, RenderScript) in terms of workflow, execution model, and performance. A case study on the performance features of the Google Chrome browser for Android is presented, covering its multi-process architecture, hardware acceleration, networking, and VSync scheduling. The document concludes with a questionnaire on topics like multi-core vs GPU, Android vs Chrome, and developments beyond Android.
DevOpsDays Taipei 2019 - Mastering IaC the DevOps Way
How to let IaC support Unit Testing by Terratest, Integration Testing by Open Policy Agent, and GitOps by Atlantis
Getting Started with the Angular 2 CLI
This document provides an overview of the Angular CLI and how to get started with it. It discusses how to install the Angular CLI, generate an initial project using the ng new command, and an overview of the file structure and common commands like ng serve, ng test, and ng build. It also briefly touches on generating components, services, pipes and more using the ng generate command.
How to build a tool for operating Flink on Kubernetes
Operating Flink on Kubernetes can be challenging. Which products are available? Do we need to build our own tool? Which tool do we need? How do we build it? This presentation provides some ideas how to build a Flink Operator and it contains a link to a proof of concept available on GitHub
Streaming SQL for Data Engineers: The Next Big Thing? With Yaroslav Tkachenko...
Streaming SQL for Data Engineers: The Next Big Thing? With Yaroslav Tkachenko | Current 2022
SQL is the lingua franca of data analysis, but should we use it more as data engineers?
Modern tools like dbt make it easier to express transformations in SQL, but streaming is more complicated than batch. Streaming pipelines usually require higher SLAs and many CI/CD and observability practices, so data engineers prefer to use familiar languages like Python, Java and Scala along with many useful frameworks and libraries. Can SQL replace that?
I was very skeptical when I first heard the idea of using SQL for writing somewhat complex stream-processing data application a few years ago. How do you unit test it? How do you version it?
Over the years, Spark SQL streaming, Flink SQL, ksqlDB and similar tools have matured, now they easily support complex stateful transformations. However, developer experience is still questionable: it’s easy to write a SQL statement, but how do you maintain it over the years as a long-running application?
In this presentation, I hope to share the discoveries I made over the years in this area, as well as working practices and patterns I’ve seen.
Streaming SQL for Data Engineers: The Next Big Thing?
The document discusses different approaches for using SQL in streaming data applications, including structured statements, dbt-style projects, notebooks, and managed runtimes. It evaluates each approach based on criteria like version control, code organization, testability, CI/CD, and observability. Overall, it recommends that for long-running streaming apps, developers should pay special attention to state management, avoid mutability, prioritize integration testing over unit testing, and embrace an SRE mentality. The document also notes that while notebooks are great for exploration, production code is better served by traditional programming frameworks, and that any managed runtime requires excellent developer experience.
Angular Extreme Performance - V2
Sabemos que os frameworks Javascript modernos estão usando o que há de melhor e mais moderno no mercado e disponibilizando no seu ecossistema para os dois maiores pilares da performance de uma aplicação front-end: network e runtime, mas quais são as ferramentas que eu posso usar no build do meu projeto? O que essa sopa de letrinhas: JIT, AOT, Tree-shaking, Uglify, GCC, Lazy loading, Ivy … são capazes de fazer para melhorar a performance de network do meu projeto? Também veremos sobre Server Rendering, trackBy, Angular OnPush Component, React Pure Component, Imutabilidade, AOT… Eles podem fazer milagres na performance de runtime do seu projeto, e várias dessas técnicas e ferramentas existem e são aplicáveis seja no Angular, React, Vue ou o velho Javascript vanilla.
Why use Gitlab
This document discusses running your own Gitlab runner fleet. It begins with an overview of Gitlab and Gitlab runners. The main benefits of running your own fleet are availability, security, cost optimization, and customization. The document then covers estimating costs, designing an auto-scaling fleet architecture using Kubernetes and docker+machine, and establishing service level objectives and metrics to monitor fleet performance.
Why and How to Run Your Own Gitlab Runners as Your Company Grows
In this session, Casey Zednick, Senior Software Engineer for NGINX, tells you everything you need to know about GitLab Runner, an app for running jobs in your CI/CD pipeline. Using your own runners gives you full control to meet your service level objectives (SLOs), increase security, and save money at scale. It's notoriously difficult to forecast costs for dynamic cloud services like an autoscaling fleet of runners, but Casey demos a tool from the NGINX tools team that calculates your costs based on a handful of numbers. He also shows how to clone our GitLab repos to set up a production-grade fleet in Azure and monitor its performance.
Speakers:
Casey Zednick
Serverless in production (O'Reilly Software Architecture)
AWS Lambda has changed the way we deploy and run software, but the serverless paradigm has created new challenges to old problems: How do you test a cloud-hosted function locally? How do you monitor them? What about logging and config management? And how do we start migrating from existing architectures?
Yan Cui shares solutions to these challenges, drawing on his experience running Lambda in production and migrating from an existing monolithic architecture.
Serverless testing @ serverlessdays Hamburg
This document discusses strategies for testing serverless applications both locally and in the cloud. It outlines pros and cons of each approach and recommends a hybrid strategy of testing certain services locally like API Gateway and Lambda functions for easy debugging and short deploy times while testing other services like DynamoDB and S3 in the cloud for a production-like environment. It also provides tips for effective testing such as using distributed tracing, load testing, and testing cold starts and function memory allocation.
Intuitive CLIs for gRPC APIs
Have you ever left out a comma in your payload calling an API on the command line? Have you cURL’d a REST API only to realize you forgot that crucial query parameter? If only the command line could take advantage of some structured description of the API model…
APIs can be challenging to understand and consume from the command line, a utility most developers use daily. In this session we will look at an approach to generating intuitive command line utilities for gRPC APIs.
Apache Airflow in the Cloud: Programmatically orchestrating workloads with Py...
Apache Airflow allows users to programmatically author, schedule, and monitor workflows or directed acyclic graphs (DAGs) using Python. It is an open-source workflow management platform developed by Airbnb that is used to orchestrate data pipelines. The document provides an overview of Airflow including what it is, its architecture, and concepts like DAGs, tasks, and operators. It also includes instructions on setting up Airflow and running tutorials on basic and dynamic workflows.
Terraform 101: What's infrastructure as code?
This document provides an overview of infrastructure as code (IAC). It defines IAC as managing infrastructure through machine-readable files rather than physical configuration. The document traces the history of IAC back to tools in the 1990s and early frameworks like CloudFormation. It outlines benefits of IAC such as cost reduction, speed, and reduced risk from errors. Examples are given of using IAC to create different environments and replicate common infrastructure components.
Дмитрий Лемешко. Comaqa Spring 2018. Continuous mobile automation in build pi...
В докладе я расскажу, как был организован запуск автоматических тестов (appium/javascript) в gitlab CI для нативного Android приложения на каждый Merge Request. Опишу, как можно встроить автотесты в существующий процесс сборки, как правильно настроить запуск тестов в docker image (тесты бегут в TestObject облаке), как произошла интеграция с клаудом и какие результаты это принесло. Tech stack: Gitlab CI, kubernetes, android, appium, javascript, testobject.
A GitOps Kubernetes Native CICD Solution with Argo Events, Workflows, and CD
Presented at Kubernetes and Cloud Native meetup in Toronto on December 4, 2019
See https://www.youtube.com/watch?v=YmIAatr3Who for a video recording of a similar talk.
Are you looking to get more flexibility out of your CICD platform? Interested how GitOps fits into the mix? Learn how Argo CD, Workflows, and Events can be combined to craft custom CICD flows. All while staying Kubernetes native, enabling you to leverage existing observability tooling.
AOT(Ahead Of Time)
This document discusses Ahead-of-Time (AOT) compilation in Angular. It explains that AOT analyzes templates and components and compiles them to JavaScript before deployment for faster rendering. With AOT, the browser does not need to compile code at runtime. The document provides steps to set up an Angular project with AOT using ngc compiler and Rollup to bundle the application.
Similar to 10 minute command line apps with zio cli (20)
DevOpsDays Taipei 2019 - Mastering IaC the DevOps Way
DevOpsDays Taipei 2019 - Mastering IaC the DevOps Way
How to build a tool for operating Flink on Kubernetes
How to build a tool for operating Flink on Kubernetes
Streaming SQL for Data Engineers: The Next Big Thing? With Yaroslav Tkachenko...
Streaming SQL for Data Engineers: The Next Big Thing? With Yaroslav Tkachenko...
Streaming SQL for Data Engineers: The Next Big Thing?
Streaming SQL for Data Engineers: The Next Big Thing?
Why and How to Run Your Own Gitlab Runners as Your Company Grows
Why and How to Run Your Own Gitlab Runners as Your Company Grows
Serverless in production (O'Reilly Software Architecture)
Serverless in production (O'Reilly Software Architecture)
Apache Airflow in the Cloud: Programmatically orchestrating workloads with Py...
Apache Airflow in the Cloud: Programmatically orchestrating workloads with Py...
Дмитрий Лемешко. Comaqa Spring 2018. Continuous mobile automation in build pi...
Дмитрий Лемешко. Comaqa Spring 2018. Continuous mobile automation in build pi...
A GitOps Kubernetes Native CICD Solution with Argo Events, Workflows, and CD
A GitOps Kubernetes Native CICD Solution with Argo Events, Workflows, and CD
Recently uploaded
Introducing Milvus Lite: Easy-to-Install, Easy-to-Use vector database for you...
Join us to introduce Milvus Lite, a vector database that can run on notebooks and laptops, share the same API with Milvus, and integrate with every popular GenAI framework. This webinar is perfect for developers seeking easy-to-use, well-integrated vector databases for their GenAI apps.
Climate Impact of Software Testing at Nordic Testing Days
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
Presentation of the OECD Artificial Intelligence Review of Germany
Consult the full report at https://www.oecd.org/digital/oecd-artificial-intelligence-review-of-germany-609808d6-en.htm
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Discover the seamless integration of RPA (Robotic Process Automation), COMPOSER, and APM with AWS IDP enhanced with Slack notifications. Explore how these technologies converge to streamline workflows, optimize performance, and ensure secure access, all while leveraging the power of AWS IDP and real-time communication via Slack notifications.
Securing your Kubernetes cluster_ a step-by-step guide to success !
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
Sudheer Mechineni, Head of Application Frameworks, Standard Chartered Bank
Discover how Standard Chartered Bank harnessed the power of Neo4j to transform complex data access challenges into a dynamic, scalable graph database solution. This keynote will cover their journey from initial adoption to deploying a fully automated, enterprise-grade causal cluster, highlighting key strategies for modelling organisational changes and ensuring robust disaster recovery. Learn how these innovations have not only enhanced Standard Chartered Bank’s data infrastructure but also positioned them as pioneers in the banking sector’s adoption of graph technology.
“I’m still / I’m still / Chaining from the Block”
“An Outlook of the Ongoing and Future Relationship between Blockchain Technologies and Process-aware Information Systems.” Invited talk at the joint workshop on Blockchain for Information Systems (BC4IS) and Blockchain for Trusted Data Sharing (B4TDS), co-located with with the 36th International Conference on Advanced Information Systems Engineering (CAiSE), 3 June 2024, Limassol, Cyprus.
Communications Mining Series - Zero to Hero - Session 1
This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered:
• Communication Mining Overview
• Why is it important?
• How can it help today’s business and the benefits
• Phases in Communication Mining
• Demo on Platform overview
• Q/A
Video Streaming: Then, Now, and in the Future
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
National Security Agency - NSA mobile device best practices
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
Removing Uninteresting Bytes in Software Fuzzing
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...
ここ3000字までしか入らないけどタイトルの方がたくさん文字入ると思います。
20240605 QFM017 Machine Intelligence Reading List May 2024
Everything I found interesting about machines behaving intelligently during May 2024
Microsoft - Power Platform_G.Aspiotis.pdf
Revolutionizing Application Development
with AI-powered low-code, presentation by George Aspiotis, Sr. Partner Development Manager, Microsoft
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Large Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial Applications.
Recently uploaded (20)
Introducing Milvus Lite: Easy-to-Install, Easy-to-Use vector database for you...
Introducing Milvus Lite: Easy-to-Install, Easy-to-Use vector database for you...
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...
みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Large Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial Applications
10 minute command line apps with zio cli
- 1. 10-Minute Command line apps with zio-cli Aiswarya Prakasan(Ash) Software Engineer ashprakasan
- 2. agenda ● Complexity of CLI Apps ● Unveiling ZIO-CLI ● Core Components ● Usage and Examples ● Salient Features ● Future Work
- 3. CLI Apps are Easy!
- 4. But, wait … Some apps like git are extremely powerful
- 5. HelpDoc
- 6. HelpDoc...
- 7. validations
- 10. parsing
- 11. CLI Apps are clearly NOT that easy! ● Documentation ● Validations ● Auto-Corrections ● Shell Completions ● Input Parsing
- 12. ZIO-CLI Rapidly build powerful command line applications! ● Functional Scala! ● Highly Composable ● Batteries Included ● ZIO Goodness!
- 13. Anatomy of a command git pull origin --rebase command sub-command argument option
- 14. WORD COUNT COMMAND wc -l -b -w ~/test-file.txt command argument options
- 16. Options wc -l -b -w ~/test-file.txt
- 17. Model
- 18. Arguments foo ./abc.txt 32 ‘bar’
- 19. Command Anatomy Command(“git”, configPath, Args.none) name options arguments
- 20. Command
- 21. Command Executor
- 22. CLI App
- 23. Bootstrap
- 24. Final!
- 26. Helpdoc
- 28. Help Doc DSL
- 29. Validations
- 30. Auto Correction
- 31. Future work ● Wizard - Select the options and arguments from a menu, instead of memorising. ● Installer - Installation script that works across different platforms. ● Auto-complete - Auto-complete support for both bash and zsh
- 32. CLI Apps are Quite easy! ● Documentation ● Validations ● Auto-Correct ● Input Parsing ● Shell Completions ● Purely Functional Design ● Highly Composable ● ZIO Goodness! Only with ZIO-CLI!