This document provides an overview of the Z shell (zsh) and highlights some of its advantages over other shells like Bash. It discusses zsh's improved tab completion for commands like git, its ability to expand and replace paths, its right prompt, spelling correction, advanced aliasing features, and syntax highlighting. The document also briefly outlines the history of shells like the Thompson shell, Bourne shell, C shell, Korn shell, and Bash before concluding that zsh offers many powerful features out of the box.
These slides accompanied a presentation by Steve Breker of Artefactual Systems, delivered as part of AtoM Camp Cambridge, a three-day boot camp held at St John's College, Cambridge University, May 9-11, 2017 For more information, see: https://wiki.accesstomemory.org/Community/Camps/SJC2017
These slides introduce users to the Access to Memory codebase. You can find AtoM's code at: https://github.com/artefactual/atom/
Ariel Waizel discusses the Data Plane Development Kit (DPDK), an API for developing fast packet processing code in user space.
* Who needs this library? Why bypass the kernel?
* How does it work?
* How good is it? What are the benchmarks?
* Pros and cons
Ariel worked on kernel development at the IDF, Ben Gurion University, and several companies. He is interested in networking, security, machine learning, and basically everything except UI development. Currently a Solution Architect at ConteXtream (an HPE company), which specializes in SDN solutions for the telecom industry.
These slides accompanied a presentation by Steve Breker of Artefactual Systems, delivered as part of AtoM Camp Cambridge, a three-day boot camp held at St John's College, Cambridge University, May 9-11, 2017 For more information, see: https://wiki.accesstomemory.org/Community/Camps/SJC2017
These slides introduce users to the Access to Memory codebase. You can find AtoM's code at: https://github.com/artefactual/atom/
Ariel Waizel discusses the Data Plane Development Kit (DPDK), an API for developing fast packet processing code in user space.
* Who needs this library? Why bypass the kernel?
* How does it work?
* How good is it? What are the benchmarks?
* Pros and cons
Ariel worked on kernel development at the IDF, Ben Gurion University, and several companies. He is interested in networking, security, machine learning, and basically everything except UI development. Currently a Solution Architect at ConteXtream (an HPE company), which specializes in SDN solutions for the telecom industry.
High-Performance Networking Using eBPF, XDP, and io_uringScyllaDB
In the networking world there are a number of ways to increase performance over naive use of basic Berkeley sockets. These techniques have ranged from polling blocking sockets, non-blocking sockets controlled by Epoll, all the way through completely bypassing the Linux kernel for maximum network performance where you talk directly to the network interface card by using something like DPDK or Netmap. All these tools have their place, and generally occupy a space from convenience to performance. But in recent years, that landscape has changed massively.. The tools available to the average Linux systems developer have improved from the creation of io_uring, to the expansion of bpf from a simple filtering language to a full-on programming environment embedded directly in the kernel. Along with that came something called XDP (express datapath). This was Linux kernel's answer to kernel-bypass networking. AF_XDP is the new socket type created by this feature, and generally works very similarly to something like DPDK. History lessons out of the way, this talk will look into, and discuss the merits of this technology, it's place in the broader ecosystem and how it can be used to attain the highest level of performance possible. This talk will dive into crucial details, such as how AF_XDP works, how it can be integrated into a larger system and finally more advanced topics such as request sharding/load balancing. There will be detailed look at the design of AF_XDP, the eBpf code used, as well as the userspace code required to drive it all. It will also include performance numbers from this setup compared to regular kernel networking. And most importantly how to put all this together to handle as much data as possible on a single modern multi-core system.
Ceph is a open source , software defined storage excellent and the only ( i would say ) storage backend as a cloud storage. Ceph is the Future of Storage. In this presentation i am explaining ceph and openstack briefly , you would definitely enjoy it.
Video: https://www.youtube.com/watch?v=JRFNIKUROPE . Talk for linux.conf.au 2017 (LCA2017) by Brendan Gregg, about Linux enhanced BPF (eBPF). Abstract:
A world of new capabilities is emerging for the Linux 4.x series, thanks to enhancements that have been included in Linux for to Berkeley Packet Filter (BPF): an in-kernel virtual machine that can execute user space-defined programs. It is finding uses for security auditing and enforcement, enhancing networking (including eXpress Data Path), and performance observability and troubleshooting. Many new open source tools that have been written in the past 12 months for performance analysis that use BPF. Tracing superpowers have finally arrived for Linux!
For its use with tracing, BPF provides the programmable capabilities to the existing tracing frameworks: kprobes, uprobes, and tracepoints. In particular, BPF allows timestamps to be recorded and compared from custom events, allowing latency to be studied in many new places: kernel and application internals. It also allows data to be efficiently summarized in-kernel, including as histograms. This has allowed dozens of new observability tools to be developed so far, including measuring latency distributions for file system I/O and run queue latency, printing details of storage device I/O and TCP retransmits, investigating blocked stack traces and memory leaks, and a whole lot more.
This talk will summarize BPF capabilities and use cases so far, and then focus on its use to enhance Linux tracing, especially with the open source bcc collection. bcc includes BPF versions of old classics, and many new tools, including execsnoop, opensnoop, funcccount, ext4slower, and more (many of which I developed). Perhaps you'd like to develop new tools, or use the existing tools to find performance wins large and small, especially when instrumenting areas that previously had zero visibility. I'll also summarize how we intend to use these new capabilities to enhance systems analysis at Netflix.
Session at ContainerDay Security 2023 on the 8th of March in Hamburg.
Hardening and securing Kubernetes requires expertise and experience. The talk takes an overview of how we contributed to Kubespray enabling cluster hardening, talking about features that have been introduced, the tools that we used to verify the cluster hardenization and our experience with the open-source community.
This presentation provides an overview of the Dell PowerEdge R730xd server performance results with Red Hat Ceph Storage. It covers the advantages of using Red Hat Ceph Storage on Dell servers with their proven hardware components that provide high scalability, enhanced ROI cost benefits, and support of unstructured data.
Delivered as plenary at USENIX LISA 2013. video here: https://www.youtube.com/watch?v=nZfNehCzGdw and https://www.usenix.org/conference/lisa13/technical-sessions/plenary/gregg . "How did we ever analyze performance before Flame Graphs?" This new visualization invented by Brendan can help you quickly understand application and kernel performance, especially CPU usage, where stacks (call graphs) can be sampled and then visualized as an interactive flame graph. Flame Graphs are now used for a growing variety of targets: for applications and kernels on Linux, SmartOS, Mac OS X, and Windows; for languages including C, C++, node.js, ruby, and Lua; and in WebKit Web Inspector. This talk will explain them and provide use cases and new visualizations for other event types, including I/O, memory usage, and latency.
Using eBPF for High-Performance Networking in CiliumScyllaDB
The Cilium project is a popular networking solution for Kubernetes, based on eBPF. This talk uses eBPF code and demos to explore the basics of how Cilium makes network connections, and manipulates packets so that they can avoid traversing the kernel's built-in networking stack. You'll see how eBPF enables high-performance networking as well as deep network observability and security.
Anatomy of a Container: Namespaces, cgroups & Some Filesystem Magic - LinuxConJérôme Petazzoni
Containers are everywhere. But what exactly is a container? What are they made from? What's the difference between LXC, butts-nspawn, Docker, and the other container systems out there? And why should we bother about specific filesystems?
In this talk, Jérôme will show the individual roles and behaviors of the components making up a container: namespaces, control groups, and copy-on-write systems. Then, he will use them to assemble a container from scratch, and highlight the differences (and likelinesses) with existing container systems.
Ceph Object Storage Performance Secrets and Ceph Data Lake SolutionKaran Singh
In this presentation, i have explained how Ceph Object Storage Performance can be improved drastically together with some object storage best practices, recommendations tips. I have also covered Ceph Shared Data Lake which is getting very popular.
High-Performance Networking Using eBPF, XDP, and io_uringScyllaDB
In the networking world there are a number of ways to increase performance over naive use of basic Berkeley sockets. These techniques have ranged from polling blocking sockets, non-blocking sockets controlled by Epoll, all the way through completely bypassing the Linux kernel for maximum network performance where you talk directly to the network interface card by using something like DPDK or Netmap. All these tools have their place, and generally occupy a space from convenience to performance. But in recent years, that landscape has changed massively.. The tools available to the average Linux systems developer have improved from the creation of io_uring, to the expansion of bpf from a simple filtering language to a full-on programming environment embedded directly in the kernel. Along with that came something called XDP (express datapath). This was Linux kernel's answer to kernel-bypass networking. AF_XDP is the new socket type created by this feature, and generally works very similarly to something like DPDK. History lessons out of the way, this talk will look into, and discuss the merits of this technology, it's place in the broader ecosystem and how it can be used to attain the highest level of performance possible. This talk will dive into crucial details, such as how AF_XDP works, how it can be integrated into a larger system and finally more advanced topics such as request sharding/load balancing. There will be detailed look at the design of AF_XDP, the eBpf code used, as well as the userspace code required to drive it all. It will also include performance numbers from this setup compared to regular kernel networking. And most importantly how to put all this together to handle as much data as possible on a single modern multi-core system.
Ceph is a open source , software defined storage excellent and the only ( i would say ) storage backend as a cloud storage. Ceph is the Future of Storage. In this presentation i am explaining ceph and openstack briefly , you would definitely enjoy it.
Video: https://www.youtube.com/watch?v=JRFNIKUROPE . Talk for linux.conf.au 2017 (LCA2017) by Brendan Gregg, about Linux enhanced BPF (eBPF). Abstract:
A world of new capabilities is emerging for the Linux 4.x series, thanks to enhancements that have been included in Linux for to Berkeley Packet Filter (BPF): an in-kernel virtual machine that can execute user space-defined programs. It is finding uses for security auditing and enforcement, enhancing networking (including eXpress Data Path), and performance observability and troubleshooting. Many new open source tools that have been written in the past 12 months for performance analysis that use BPF. Tracing superpowers have finally arrived for Linux!
For its use with tracing, BPF provides the programmable capabilities to the existing tracing frameworks: kprobes, uprobes, and tracepoints. In particular, BPF allows timestamps to be recorded and compared from custom events, allowing latency to be studied in many new places: kernel and application internals. It also allows data to be efficiently summarized in-kernel, including as histograms. This has allowed dozens of new observability tools to be developed so far, including measuring latency distributions for file system I/O and run queue latency, printing details of storage device I/O and TCP retransmits, investigating blocked stack traces and memory leaks, and a whole lot more.
This talk will summarize BPF capabilities and use cases so far, and then focus on its use to enhance Linux tracing, especially with the open source bcc collection. bcc includes BPF versions of old classics, and many new tools, including execsnoop, opensnoop, funcccount, ext4slower, and more (many of which I developed). Perhaps you'd like to develop new tools, or use the existing tools to find performance wins large and small, especially when instrumenting areas that previously had zero visibility. I'll also summarize how we intend to use these new capabilities to enhance systems analysis at Netflix.
Session at ContainerDay Security 2023 on the 8th of March in Hamburg.
Hardening and securing Kubernetes requires expertise and experience. The talk takes an overview of how we contributed to Kubespray enabling cluster hardening, talking about features that have been introduced, the tools that we used to verify the cluster hardenization and our experience with the open-source community.
This presentation provides an overview of the Dell PowerEdge R730xd server performance results with Red Hat Ceph Storage. It covers the advantages of using Red Hat Ceph Storage on Dell servers with their proven hardware components that provide high scalability, enhanced ROI cost benefits, and support of unstructured data.
Delivered as plenary at USENIX LISA 2013. video here: https://www.youtube.com/watch?v=nZfNehCzGdw and https://www.usenix.org/conference/lisa13/technical-sessions/plenary/gregg . "How did we ever analyze performance before Flame Graphs?" This new visualization invented by Brendan can help you quickly understand application and kernel performance, especially CPU usage, where stacks (call graphs) can be sampled and then visualized as an interactive flame graph. Flame Graphs are now used for a growing variety of targets: for applications and kernels on Linux, SmartOS, Mac OS X, and Windows; for languages including C, C++, node.js, ruby, and Lua; and in WebKit Web Inspector. This talk will explain them and provide use cases and new visualizations for other event types, including I/O, memory usage, and latency.
Using eBPF for High-Performance Networking in CiliumScyllaDB
The Cilium project is a popular networking solution for Kubernetes, based on eBPF. This talk uses eBPF code and demos to explore the basics of how Cilium makes network connections, and manipulates packets so that they can avoid traversing the kernel's built-in networking stack. You'll see how eBPF enables high-performance networking as well as deep network observability and security.
Anatomy of a Container: Namespaces, cgroups & Some Filesystem Magic - LinuxConJérôme Petazzoni
Containers are everywhere. But what exactly is a container? What are they made from? What's the difference between LXC, butts-nspawn, Docker, and the other container systems out there? And why should we bother about specific filesystems?
In this talk, Jérôme will show the individual roles and behaviors of the components making up a container: namespaces, control groups, and copy-on-write systems. Then, he will use them to assemble a container from scratch, and highlight the differences (and likelinesses) with existing container systems.
Ceph Object Storage Performance Secrets and Ceph Data Lake SolutionKaran Singh
In this presentation, i have explained how Ceph Object Storage Performance can be improved drastically together with some object storage best practices, recommendations tips. I have also covered Ceph Shared Data Lake which is getting very popular.
Most Rails Developers use the Unix shell, Bash (or a variant), every day. However, many newcomers don’t utilize the full potential of this ubiquitous command line interface to the Unix operating system. In this talk, I’ll discuss powerful features and tricks of the Bash shell to help improve your day to day life developing with Rails!
A multi-task and multi-user Operating System
Developed in 1969 at AT&T’s Bell Labs by
Ken Thompson (Unix)
Dennis Ritchie (C)
Douglas Mcllroy (Pipes - Do one thing, do it well)
Some other variants: System V, Solaris, SCO Unix, SunOS, 4.4BSD, FreeBSD, NetBSD, OpenBSD, BSDI
1. why
Z shell (zsh)
is cooler
than
your shell
Brendon Rapp - Cave Lunch #1
2. why
Z shell (zsh)
is cooler
than
your shell
(unless your shell is zsh)
Brendon Rapp - Cave Lunch #1
3. (alternate title)
she sells
Z shells
by the
sea shore
4. Donald Knuth
Professor Emeritus of
Computer Science at
Stanford
Author of The Art of
Computer Programming
"Father of algorithmic
analysis"
Creator of TeX
5. In 1986, Knuth was asked to write a guest
feature for the "Programming Pearls" column in
the Communications of the ACM journal.
The task was to write a program that would:
read a file of text, determine the n most
frequently used words, and print out a sorted
list of those words along with their frequencies.
6. Knuth produced a solution in Pascal that, when
printed, was about 10 pages in length. It was
well designed, thoroughly commented, and
used a novel data structure for managing the
word count list.
7. In response, Doug McIlroy wrote a shell script
that produced the same output.
8. In response, Doug McIlroy wrote a shell script
that produced the same output.
McIlroy's script was six lines long.
10. A brief history of shells
1971: Thompson shell
● Ken Thompson, Bell Labs, first Unix shell
● interactive interpreter, not scripting environment
1977: Bourne shell
● scripting language
● Version 7 Unix, PDP-11
● 1984: The UNIX Programming Environment, Kernighan & Pike
● The shell of commercial Unixes
○ System V, AIX, HP-UX, SCO, Solaris, SunOS
○ Still the default on some of these (that are still alive)
● /bin/sh
○ compatibility mode in modern shells
○ symlink or hard link to compatible shells in modern Unixes
11. A brief history of shells
1978: C shell
● BSD Unix
● More "C-like" scripting syntax (kinda)
● Command history
● Aliasing
● tcsh - newer C shell, default on FreeBSD, and OS X systems
10.0-10.2
1983: Korn shell
● Bell Labs (AT&T)
● Proprietary until 2000
● vi and emacs editing modes
● Lots of C shell features
● "middle road" between Bourne and C shell
● pdksh - default on OpenBSD
12. A brief history of shells
1989: Bourne Again shell (bash)
● GNU, GPL
● first legitimate Free shell (/bin/sh compatible)
○ shells like ksh and csh became Free only much later on
● standard shell for Linux distros, Mac OS X 10.3+
● TAB completion
● extended scripting syntax
1990: Z shell
● most closely resembles Korn shell
● /bin/bash compatibility, drop-in replacement for Bash
● "new" (despite being over 20 years old)
● awesome stuff I'll talk about next
16. ... your Bash is old!
(OS X 10.8.2... and many earlier OS X versions too)
17. OS X: GPL Wasteland
● no GPLv3 on OS X
● OS X bash: final version released as GPLv2
● Homebrew has latest Bash (but many use
situation as an excuse to try zsh instead)
28. bash: git completion
It's possible to get completion for git (and many other commands) in Bash
by installing bash-completion package, but the completion is still
rudimentary compared to zsh:
● no cycling through options with repeated tabs
● no accompanying info with commands, just a list
● breaks to new prompt line on each tab instead of updating in-place
There may be ways to improve that situation and bring it more in line with
zsh, but with zsh, you get it basically out-of-the-box, with a single command
in your .zshrc to enable completions.
59. Other zsh bullet points
● Simple configuration style
● Shared history
○ simple & fast, requires some monkeying to replicate
in bash
● Lots of additions for shell scripting
● Output redirection to multiple destinations
And, apparently, plenty of other stuff deeper than I've
gotten so far.
