Intel Theater Presentation - SC11

Talk by Brendan Gregg for All Things Open 2018. "At over one thousand code commits per week, it's hard to keep up with Linux developments. This keynote will summarize recent Linux performance features, for a wide audience: the KPTI patches for Meltdown, eBPF for performance observability and the new open source tools that use it, Kyber for disk I/O sc heduling, BBR for TCP congestion control, and more. This is about exposure: knowing what exists, so you can learn and use it later when needed. Get the most out of your systems with the latest Linux kernels and exciting features."

Talk for AWS re:Invent 2014. Video: https://www.youtube.com/watch?v=7Cyd22kOqWc . Netflix tunes Amazon EC2 instances for maximum performance. In this session, you learn how Netflix configures the fastest possible EC2 instances, while reducing latency outliers. This session explores the various Xen modes (e.g., HVM, PV, etc.) and how they are optimized for different workloads. Hear how Netflix chooses Linux kernel versions based on desired performance characteristics and receive a firsthand look at how they set kernel tunables, including hugepages. You also hear about Netflix’s use of SR-IOV to enable enhanced networking and their approach to observability, which can exonerate EC2 issues and direct attention back to application performance.

PROSEEric Van Hensbergen

containerit at useR!2017 conference, Brussels

Daniel Nüst

**Webpage** https://github.com/o2r-project/containerit/ **Abstract** Reproducibility of computations is crucial in an era where data is born digital and analysed algorithmically. Most studies however only publish the results, often with figures as important interpreted outputs. But where do these figures come from? Scholarly articles must provide not only a description of the work but be accompanied by data and software. R offers excellent tools to create reproducible works, i.e. Sweave and RMarkdown. Several approaches to capture the workspace environment in R have been made, working around CRAN’s deliberate choice not to provide explicit versioning of packages and their dependencies. They preserve a collection of packages locally (packrat, pkgsnap, switchr/GRANBase) or remotely (MRAN timemachine/checkpoint), or install specific versions from CRAN or source (requireGitHub, devtools). Installers for old versions of R are archived on CRAN. A user can manually re-create a specific environment, but this is a cumbersome task. We introduce a new possibility to preserve a runtime environment including both, packages and R, by adding an abstraction layer in the form of a container, which can execute a script or run an interactive session. The package containeRit automatically creates such containers based on Docker. Docker is a solution for packaging an application and its dependencies, but shows to be useful in the context of reproducible research (Boettiger 2015). The package creates a container manifest, the Dockerfile, which is usually written by hand, from sessionInfo(), R scripts, or RMarkdown documents. The Dockerfiles use the Rocker community images as base images. Docker can build an executable image from a Dockerfile. The image is executable anywhere a Docker runtime is present. containeRit uses harbor for building images and running containers, and sysreqs for installing system dependencies of R packages. Before the planned CRAN release we want to share our work, discuss open challenges such as handling linked libraries (see discussion on geospatial libraries in Rocker), and welcome community feedback. containeRit is developed within the DFG-funded project Opening Reproducible Research to support the creation of Executable Research Compendia (ERC) (Nüst et al. 2017). **References** Boettiger, Carl. 2015. “An Introduction to Docker for Reproducible Research, with Examples from the R Environment.” ACM SIGOPS Operating Systems Review 49 (January): 71–79. doi:10.1145/2723872.2723882. Nüst, Daniel, Markus Konkol, Edzer Pebesma, Christian Kray, Marc Schutzeichel, Holger Przibytzin, and Jörg Lorenz. 2017. “Opening the Publication Process with Executable Research Compendia.” D-Lib Magazine 23 (January). doi:10.1045/january2017-nuest.

Memory, Big Data, NoSQL and Virtualization

In-memory processing has started to become the norm in large scale data handling. This is aclose to the metal analysis of highly important but often neglected aspects of memory accesstimes and how it impacts big data and NoSQL technologies.We cover aspects such as the TLB, the Transparent Huge Pages, the QPI Link, Hyperthreading and the impact of virtualization on high-memory footprint applications. We present benchmarks of various technologies ranging from Cloudera’s Impala to Couchbase and how they are impacted by the underlying hardware.The key takeaway is a better understanding of how to size a cluster, how to choose a cloud provider and an instance type for big data and NoSQL workloads and why not every core or GB of RAM is created equal.

Journey Through The Cloud - Disaster Recovery

Amazon Web Services

Cassandra Performance Benchmark

The Economics of Scaling Cassandra - By Alex Bordei, Techie Product Manager at Bigstep This presentation was made during the "Cassandra Summit 2014" Event, in London. We benchmarked Cassandra on a number of configurations and we show what's the scaling profile. We test Cassandra on Docker as well as Cassandra's In-memory feature. Follow Alex on Twitter: @alexandrubordei Bigstep on Twitter: @BigStepInc If you have any questions, let us know at hello@bigstep.com and we'll do our best to answer. Stay informed: http://blog.bigstep.com/

CPU Optimizations in the CERN Cloud - February 2016

Belmiro Moreira

Linux kernel bug hunting

Andrea Righi

Debugging is an essential part of the Linux kernel development, but without the proper knowledge it can be quite challenging. In this talk, we will cover some techniques that will help you to study how the kernel works “under the hood”. We will see how to crash the kernel in many spectacular ways, how to analyze the crashes and how to hunt down new bugs, fix them and become a better kernel developer.

redis-benchmark with AMD RYZEN 1800X Intel Kaby Lake (i7-7700K) memo

Naoto MATSUMOTO

Usenix04 20040702Todd Deshane

Faster Container Image Distribution on a Variety of Tools with Lazy Pulling

Kohei Tokunaga

High Performance Cloud Computing

Deepak Singh

NUSE (Network Stack in Userspace) at #osio

Hajime Tazaki

Performance Wins with BPF: Getting Started

How to schedule snapdrive space reclamation

Ashwin Pawar

Hâpy eole-gnu-linux-distribution

OpenNebula Project

Le Pôle de Compétences Locigiels Libres a pour mission principale d’éditer la distribution GNU/Linux EOLE en collaboration avec le ministère de l’écologie, du développement durable et de l’environnemet. Après une utilisation intensive d’OpenNebula pour le développement et de qualification de la distribution EOLE, la communauté a souhaité l’intégrer à notre offre de solutions clef en main. Talk given by Daniel Dehennin from Ac-Dijon (ac-dijon.fr) during Paris Techday 2015 http://opennebula.org/community/techdays/techday-paris-2015/

Docker at OpenDNS

OpenDNS

Syslog-ng Performance

Chakrit Phain

Kafka shortTikal Knowledge

Debugging Hung Python Processes With GDB

bmbouter

Humanizing bioinformatics

Energi dan metabolismeshafhandustur

What's hot

Performance Tuning EC2 Instances

PROSEEric Van Hensbergen

containerit at useR!2017 conference, Brussels

Daniel Nüst

Memory, Big Data, NoSQL and Virtualization

Journey Through The Cloud - Disaster Recovery

Amazon Web Services

Cassandra Performance Benchmark

CPU Optimizations in the CERN Cloud - February 2016

Belmiro Moreira

Linux kernel bug hunting

Andrea Righi

redis-benchmark with AMD RYZEN 1800X Intel Kaby Lake (i7-7700K) memo

Naoto MATSUMOTO

Usenix04 20040702Todd Deshane

Faster Container Image Distribution on a Variety of Tools with Lazy Pulling

Kohei Tokunaga

High Performance Cloud Computing

Deepak Singh

NUSE (Network Stack in Userspace) at #osio

Hajime Tazaki

Performance Wins with BPF: Getting Started

How to schedule snapdrive space reclamation

Ashwin Pawar

Hâpy eole-gnu-linux-distribution

OpenNebula Project

Docker at OpenDNS

OpenDNS

Syslog-ng Performance

Chakrit Phain

Kafka shortTikal Knowledge

Debugging Hung Python Processes With GDB

bmbouter

What's hot (20)

Performance Tuning EC2 Instances

PROSE

containerit at useR!2017 conference, Brussels

Memory, Big Data, NoSQL and Virtualization

Journey Through The Cloud - Disaster Recovery

Cassandra Performance Benchmark

CPU Optimizations in the CERN Cloud - February 2016

Linux kernel bug hunting

redis-benchmark with AMD RYZEN 1800X Intel Kaby Lake (i7-7700K) memo

Usenix04 20040702

Faster Container Image Distribution on a Variety of Tools with Lazy Pulling

High Performance Cloud Computing

NUSE (Network Stack in Userspace) at #osio

Performance Wins with BPF: Getting Started

How to schedule snapdrive space reclamation

Hâpy eole-gnu-linux-distribution

Docker at OpenDNS

Syslog-ng Performance

Kafka short

Debugging Hung Python Processes With GDB

Viewers also liked

Humanizing bioinformatics

Energi dan metabolismeshafhandustur

Symmetry and group theoryZuhriyatusSholichah

Tetrahymena genome project 2003 presentation by Jonathan Eisen

Chamberlain PhD Thesis

schamber

Jonathan Eisen talk for #SCS2012 at #ISMB "Networks in genomics and bioinfor...

Bio4j

Pablo Pareja Tobes

J Goecks - The Galaxy Visual Analysis Framework

Jonathan Eisen: Phylogenetic approaches to the analysis of genomes and metage...

Bio::Phylo - phyloinformatic analysis using perl

Rutger Vos

VIZBI 2014 - Visualizing Genomic Variation

20120622 fridayadelboden

Yannick Wurm

Intro to data visualization

Surfacing the deep data of taxonomy

Roderic Page

Procter Vamsas Bosc2009bosc

OBF Address at BOSC 2012

Hilmar Lapp

The role of cost in yeast gene expressionMichael Barton

The neurobiological nature of free will

Björn Brembs

Our own experience of our free will has been classified as either supernatural or an illusion because it is difficult to reconcile with macroscopic determinism as well as with microscopic quantum randomness. The former constituting a prison in which no freedom can exist, the latter signifying destructive chaos rather than creative action. Lost in this dichotomy is the demonstrated constructive combination of chance and necessity in complex systems, such as evolution. Recent converging evidence from neuroscience, ecology and genetics suggests that nervous systems, including human brains, have evolved neural circuits that harness (potentially quantum) chance events by embedding them in the controlling architecture of neuronal rules, in order to carefully inject them as creative components into ongoing goal-directed behavior. This presentation contains evidence that this form of behavioral variability may constitute a necessary neural mechanism for free will to evolve in humans.

Evolution of the RecA Protein: from Systematics to Structure 1995 talk for CA...