Better Bash - Unit and Integration Testing

•

2 likes•2,445 views

Presented at Velocity Santa Clara 2016, June 22 Relevant links: https://github.com/sstephenson/bats https://github.com/concourse/git-resource/tree/master/test https://github.com/rbenv/ruby-build/tree/master/test

@cjcjameson1
Unit and Integration Testing
C.J. Jameson
Better Bash

@cjcjameson
I work at Pivotal, I’m a Toolsmith, I tweet some
2

@cjcjameson3
Customer = Engineer
Reliability & predictable outcomes
Good bug reports
Eager to make tradeoffs

@cjcjameson4
Tradeoffs?
Customization vs. Reuse
User experience vs. Exposing internals
YAGNI

@cjcjameson
How good does this need to be?
The tradeoff with your Bash scripts
5
effort
time
¯_(ツ)_/¯
...sorry dude

@cjcjameson
cover this square with an image (540 x 480 pixels)
7
STOP
When would
anyone write a
test themselves?

@cjcjameson
cover this square with an image (540 x 480 pixels)
8
STOP
When would
anyone write a
test themselves?

@cjcjameson
cover this square with an image (540 x 480 pixels)
9
STOP
When would
anyone run
the tests?

@cjcjameson10
When would anyone run the tests?
STOP

@cjcjameson11
When would anyone run the tests?
P
https://github.com/koalaman/shellcheck

@cjcjameson12
1
https://en.wikipedia.org/wiki/Unit_testing
In general, unit tests...1
Find problems early Make change easy
Drive modular designAre documentationMake integration easy

@cjcjameson13
You’re writing an installer file
Be documentation
Make integration easy
(esp. across distros)
So your unit tests should help:

@cjcjameson14
You’re touching this code for the first and only time
Drive modular designMake change easy
So your unit tests should help:
Be documentation

@cjcjameson15
You’re treating the filesystem as a first-class object
Drive modular designFind problems early
So your unit tests should help:

@cjcjameson16
You’re writing a shared `common.bash` library
Be documentationMake integration easy
So your unit tests should help:

@cjcjameson17
You’re crafting a CLI user experience
Make change easyFind problems early
So your unit tests should help:

@cjcjameson19
Example: Concourse’s `git-resource`
https://ci.concourse.ci/pipelines/resources/jobs/git-resource

@cjcjameson20
Example: “Check the executable bit”

@cjcjameson21
Example: “Check the executable bit”

@cjcjameson22
Example: “Check the executable bit” - integration

@cjcjameson23
Inside the BATS `run` command
https://github.com/sstephenson/bats
- Exit codes are swallowed
- `stdout` and `stderr` are swallowed
- Start by `source`ing in the file under test … so the file runs

When the British were ruling in colonial India, they were concerned about the number of venomous cobra snakes in Delhi. The government therefore offered a bounty for every dead cobra. Initially this was a successful strategy as large numbers of snakes were killed for the reward. Eventually, however, enterprising people began to breed cobras for the income. When the government became aware of this, the reward program was scrapped, causing the cobra breeders to set the now-worthless snakes free. As a result, the wild cobra population further increased. The apparent solution for the problem made the situation even worse. Our tests which we assume to be a safeguard for our code might just make our situation even worse!

Develop your inner tester

Anne-Marie Charrett

Managing a large open source community - OSCON 2016

{code}

Increasing your online presence in the open source community involves more than just writing and committing code. We need to create safe places to collaborate and communicate freely in order to involve more people than just our closest team members. Jonas Rosland and Stephanie Carlson outline a few of the tools they use to tackle this big task and discuss failures, successes, and lessons learned.

Lean Product Management for Web 2.0 Products

Dan Olsen

As digital becomes more central to a book publishers life, the tools must change to keep pace. In this session we look at WordPress as a lightweight digital swiss army knife, a powerful and flexible platform that can be adapted to many publishing needs. We’ll look at using WordPress-based systems for ecommerce, marketing, catalogs and discovery and more. Participants will aslo have a hands-on demo of PressBooks (built on WordPress), a digital book production tool that participants can use to generate an epub and typeset PDFs.

jQuery Mobile: For Fun and Profit

Daniel Cousineau

(Short version) Building a Mobile, Social, Location-Based Game in 5 Weeks

Jennie Lees

Kobo: What Do eBook Customers Really, Really Want? (Tools of Change 2011)

MTamblyn

Smaller, Flatter, SmarterWeb 2.0 Expo

Social Gold: The Design of FarmVille and Other Social Games (Web2Expo 2010)

Amitt Mahajan

Mobilising the world's Natural History - Open Data + Citizen Science

Margaret Gold

my slides for the Ignite Talks at OSCON 2016 in London. Mobilizing the world’s natural history: Open data + citizen science Margaret Gold The Natural History Museum is embarking on an epic journey to digitize 80 million specimens from one of the world’s most important natural history collections. But alongside this, the museum’s citizen science projects invite you to actively contribute to its science research. Margaret explains where the two meet and how they might change the face of natural history.

Web 2.0 Expo Speech: Open Leadership

Charlene Li

Data Science and Smart Systems: Creating the Digital Brain

VMware Tanzu

Big Data technologies enable us to build the digital brain of smart systems. I will illustrate with examples how we build a digital brain by collecting data from a large number of sensors and using the brain to find value in that data. We build a Data Lake using cutting edge technology from Pivotal and use it to store large amounts of sensor and other data. Then we can find patterns in that data by applying the Data Science methodology using sophisticated machine learning and statistical algorithms customized to run on big data within the Data Lake. Armed with these patterns the system can detect anomalies and respond in an appropriate manner. Data Science combined with sensors and actuators can make a system smart!

Hadoop's Impact on the Future of Data Management | Amr Awadallah

Cloudera, Inc.

Tyranny of the SLA

J. Paul Reed

My talk with Jim Kimball on the tyranny of the SLA; in it, we: - Deconstruct the purpose of the service level agreement - Discuss pitfalls of aspects of common SLA clauses, including how current SLAs inhibit the development of resilient systems and the cultivation of a DevOps culture - Explore other potential SLA models that could foster healthier organizational behaviors and dynamics, and ultimately result in better technical outcomes and therefore business outcomes.

Locked Out in London (and tweeting about it) - version with my notes

Sylvain Carle

Did Social Media Hijack My Communications Strategy

Mike Smith

Primum non nocere - Ethical Obligations in Internet Operations

Jan Schaumann

Cracking the code review at SpringIO 2024

Paco van Beckhoven

Code reviews are vital for ensuring good code quality. They serve as one of our last lines of defense against bugs and subpar code reaching production. Yet, they often turn into annoying tasks riddled with frustration, hostility, unclear feedback and lack of standards. How can we improve this crucial process? In this session we will cover: - The Art of Effective Code Reviews - Streamlining the Review Process - Elevating Reviews with Automated Tools By the end of this presentation, you'll have the knowledge on how to organize and improve your code review proces

Multiple Your Crypto Portfolio with the Innovative Features of Advanced Crypt...

Hivelance Technology

Cryptocurrency trading bots are computer programs designed to automate buying, selling, and managing cryptocurrency transactions. These bots utilize advanced algorithms and machine learning techniques to analyze market data, identify trading opportunities, and execute trades on behalf of their users. By automating the decision-making process, crypto trading bots can react to market changes faster than human traders Hivelance, a leading provider of cryptocurrency trading bot development services, stands out as the premier choice for crypto traders and developers. Hivelance boasts a team of seasoned cryptocurrency experts and software engineers who deeply understand the crypto market and the latest trends in automated trading, Hivelance leverages the latest technologies and tools in the industry, including advanced AI and machine learning algorithms, to create highly efficient and adaptable crypto trading bots

Viewers also liked

State of the Internet Operating System: Web2 expo10

Tim O'Reilly

webfonts & @font-face :: in brief

Paul Irish

Where in the publishing world are libraries?

Katie Dunneback

Where 2.0 Perch Product Launch Presentation

perryevans

Epub in the wild

liz_castro

Strata Conference 2014 NYC with Twitter

Taewook Eom

Using WordPress for Digital Workflows and More

Kirk Biglione

jQuery Mobile: For Fun and Profit

Daniel Cousineau

(Short version) Building a Mobile, Social, Location-Based Game in 5 Weeks

Jennie Lees

Kobo: What Do eBook Customers Really, Really Want? (Tools of Change 2011)

MTamblyn

Smaller, Flatter, SmarterWeb 2.0 Expo

Social Gold: The Design of FarmVille and Other Social Games (Web2Expo 2010)

Amitt Mahajan

Mobilising the world's Natural History - Open Data + Citizen Science

Margaret Gold

Web 2.0 Expo Speech: Open Leadership

Charlene Li

Data Science and Smart Systems: Creating the Digital Brain

VMware Tanzu

Hadoop's Impact on the Future of Data Management | Amr Awadallah

Cloudera, Inc.

Tyranny of the SLA

J. Paul Reed

Locked Out in London (and tweeting about it) - version with my notes

Sylvain Carle

Did Social Media Hijack My Communications Strategy

Mike Smith

Primum non nocere - Ethical Obligations in Internet Operations

Jan Schaumann

Viewers also liked (20)

State of the Internet Operating System: Web2 expo10

webfonts & @font-face :: in brief

Where in the publishing world are libraries?

Where 2.0 Perch Product Launch Presentation

Epub in the wild

Strata Conference 2014 NYC with Twitter

Using WordPress for Digital Workflows and More

jQuery Mobile: For Fun and Profit

(Short version) Building a Mobile, Social, Location-Based Game in 5 Weeks

Kobo: What Do eBook Customers Really, Really Want? (Tools of Change 2011)

Smaller, Flatter, Smarter

Social Gold: The Design of FarmVille and Other Social Games (Web2Expo 2010)

Mobilising the world's Natural History - Open Data + Citizen Science

Web 2.0 Expo Speech: Open Leadership

Data Science and Smart Systems: Creating the Digital Brain

Hadoop's Impact on the Future of Data Management | Amr Awadallah

Tyranny of the SLA

Locked Out in London (and tweeting about it) - version with my notes

Did Social Media Hijack My Communications Strategy

Primum non nocere - Ethical Obligations in Internet Operations

Recently uploaded

Cracking the code review at SpringIO 2024

Paco van Beckhoven

Multiple Your Crypto Portfolio with the Innovative Features of Advanced Crypt...

Hivelance Technology

2024 RoOUG Security model for the cloud.pptx

Georgi Kodinov

Developing Distributed High-performance Computing Capabilities of an Open Sci...

Globus

COVID-19 had an unprecedented impact on scientific collaboration. The pandemic and its broad response from the scientific community has forged new relationships among public health practitioners, mathematical modelers, and scientific computing specialists, while revealing critical gaps in exploiting advanced computing systems to support urgent decision making. Informed by our team’s work in applying high-performance computing in support of public health decision makers during the COVID-19 pandemic, we present how Globus technologies are enabling the development of an open science platform for robust epidemic analysis, with the goal of collaborative, secure, distributed, on-demand, and fast time-to-solution analyses to support public health.

Globus Compute Introduction - GlobusWorld 2024

Globus

Accelerate Enterprise Software Engineering with Platformless

WSO2

Key takeaways: Challenges of building platforms and the benefits of platformless. Key principles of platformless, including API-first, cloud-native middleware, platform engineering, and developer experience. How Choreo enables the platformless experience. How key concepts like application architecture, domain-driven design, zero trust, and cell-based architecture are inherently a part of Choreo. Demo of an end-to-end app built and deployed on Choreo.

Quarkus Hidden and Forbidden Extensions

Max Andersen

In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...

Juraj Vysvader

Cyaniclab : Software Development Agency Portfolio.pdf

Cyanic lab

CyanicLab, an offshore custom software development company based in Sweden,India, Finland, is your go-to partner for startup development and innovative web design solutions. Our expert team specializes in crafting cutting-edge software tailored to meet the unique needs of startups and established enterprises alike. From conceptualization to execution, we offer comprehensive services including web and mobile app development, UI/UX design, and ongoing software maintenance. Ready to elevate your business? Contact CyanicLab today and let us propel your vision to success with our top-notch IT solutions.

A Comprehensive Look at Generative AI in Retail App Testing.pdf

kalichargn70th171

Advanced Flow Concepts Every Developer Should Know

Peter Caitens

Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...

Globus

Large Language Models (LLMs) are currently the center of attention in the tech world, particularly for their potential to advance research. In this presentation, we'll explore a straightforward and effective method for quickly initiating inference runs on supercomputers using the vLLM tool with Globus Compute, specifically on the Polaris system at ALCF. We'll begin by briefly discussing the popularity and applications of LLMs in various fields. Following this, we will introduce the vLLM tool, and explain how it integrates with Globus Compute to efficiently manage LLM operations on Polaris. Attendees will learn the practical aspects of setting up and remotely triggering LLMs from local machines, focusing on ease of use and efficiency. This talk is ideal for researchers and practitioners looking to leverage the power of LLMs in their work, offering a clear guide to harnessing supercomputing resources for quick and effective LLM inference.

Explore Modern SharePoint Templates for 2024

Sharepoint Designs

Globus Connect Server Deep Dive - GlobusWorld 2024

Globus

First Steps with Globus Compute Multi-User Endpoints

Globus

In this presentation we will share our experiences around getting started with the Globus Compute multi-user endpoint. Working with the Pharmacology group at the University of Auckland, we have previously written an application using Globus Compute that can offload computationally expensive steps in the researcher's workflows, which they wish to manage from their familiar Windows environments, onto the NeSI (New Zealand eScience Infrastructure) cluster. Some of the challenges we have encountered were that each researcher had to set up and manage their own single-user globus compute endpoint and that the workloads had varying resource requirements (CPUs, memory and wall time) between different runs. We hope that the multi-user endpoint will help to address these challenges and share an update on our progress here.

Enhancing Research Orchestration Capabilities at ORNL.pdf

Globus

Cross-facility research orchestration comes with ever-changing constraints regarding the availability and suitability of various compute and data resources. In short, a flexible data and processing fabric is needed to enable the dynamic redirection of data and compute tasks throughout the lifecycle of an experiment. In this talk, we illustrate how we easily leveraged Globus services to instrument the ACE research testbed at the Oak Ridge Leadership Computing Facility with flexible data and task orchestration capabilities.

TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR

Tier1 app

Even though at surface level ‘java.lang.OutOfMemoryError’ appears as one single error; underlyingly there are 9 types of OutOfMemoryError. Each type of OutOfMemoryError has different causes, diagnosis approaches and solutions. This session equips you with the knowledge, tools, and techniques needed to troubleshoot and conquer OutOfMemoryError in all its forms, ensuring smoother, more efficient Java applications.

Vitthal Shirke Microservices Resume Montevideo

Vitthal Shirke

Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...

Globus

The U.S. Geological Survey (USGS) has made substantial investments in meeting evolving scientific, technical, and policy driven demands on storing, managing, and delivering data. As these demands continue to grow in complexity and scale, the USGS must continue to explore innovative solutions to improve its management, curation, sharing, delivering, and preservation approaches for large-scale research data. Supporting these needs, the USGS has partnered with the University of Chicago-Globus to research and develop advanced repository components and workflows leveraging its current investment in Globus. The primary outcome of this partnership includes the development of a prototype enterprise repository, driven by USGS Data Release requirements, through exploration and implementation of the entire suite of the Globus platform offerings, including Globus Flow, Globus Auth, Globus Transfer, and Globus Search. This presentation will provide insights into this research partnership, introduce the unique requirements and challenges being addressed and provide relevant project progress.

Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...

Globus

The Earth System Grid Federation (ESGF) is a global network of data servers that archives and distributes the planet’s largest collection of Earth system model output for thousands of climate and environmental scientists worldwide. Many of these petabyte-scale data archives are located in proximity to large high-performance computing (HPC) or cloud computing resources, but the primary workflow for data users consists of transferring data, and applying computations on a different system. As a part of the ESGF 2.0 US project (funded by the United States Department of Energy Office of Science), we developed pre-defined data workflows, which can be run on-demand, capable of applying many data reduction and data analysis to the large ESGF data archives, transferring only the resultant analysis (ex. visualizations, smaller data files). In this talk, we will showcase a few of these workflows, highlighting how Globus Flows can be used for petabyte-scale climate analysis.

Recently uploaded (20)

Cracking the code review at SpringIO 2024

Multiple Your Crypto Portfolio with the Innovative Features of Advanced Crypt...

2024 RoOUG Security model for the cloud.pptx

Developing Distributed High-performance Computing Capabilities of an Open Sci...

Globus Compute Introduction - GlobusWorld 2024

Accelerate Enterprise Software Engineering with Platformless

Quarkus Hidden and Forbidden Extensions

In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...

Cyaniclab : Software Development Agency Portfolio.pdf

A Comprehensive Look at Generative AI in Retail App Testing.pdf

Advanced Flow Concepts Every Developer Should Know

Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...

Explore Modern SharePoint Templates for 2024

Globus Connect Server Deep Dive - GlobusWorld 2024

First Steps with Globus Compute Multi-User Endpoints

Enhancing Research Orchestration Capabilities at ORNL.pdf

TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR

Vitthal Shirke Microservices Resume Montevideo

Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...

Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...

Better Bash - Unit and Integration Testing

1. @cjcjameson1 Unit and Integration Testing C.J. Jameson Better Bash

2. @cjcjameson I work at Pivotal, I’m a Toolsmith, I tweet some 2

3. @cjcjameson3 Customer = Engineer Reliability & predictable outcomes Good bug reports Eager to make tradeoffs

4. @cjcjameson4 Tradeoffs? Customization vs. Reuse User experience vs. Exposing internals YAGNI

5. @cjcjameson How good does this need to be? The tradeoff with your Bash scripts 5 effort time ¯_(ツ)_/¯ ...sorry dude

6. @cjcjameson6 Here’s my shiny tool…?

7. @cjcjameson cover this square with an image (540 x 480 pixels) 7 STOP When would anyone write a test themselves?

8. @cjcjameson cover this square with an image (540 x 480 pixels) 8 STOP When would anyone write a test themselves?

9. @cjcjameson cover this square with an image (540 x 480 pixels) 9 STOP When would anyone run the tests?

10. @cjcjameson10 When would anyone run the tests? STOP

11. @cjcjameson11 When would anyone run the tests? P https://github.com/koalaman/shellcheck

12. @cjcjameson12 1 https://en.wikipedia.org/wiki/Unit_testing In general, unit tests...1 Find problems early Make change easy Drive modular designAre documentationMake integration easy

13. @cjcjameson13 You’re writing an installer file Be documentation Make integration easy (esp. across distros) So your unit tests should help:

14. @cjcjameson14 You’re touching this code for the first and only time Drive modular designMake change easy So your unit tests should help: Be documentation

15. @cjcjameson15 You’re treating the filesystem as a first-class object Drive modular designFind problems early So your unit tests should help:

16. @cjcjameson16 You’re writing a shared `common.bash` library Be documentationMake integration easy So your unit tests should help:

17. @cjcjameson17 You’re crafting a CLI user experience Make change easyFind problems early So your unit tests should help:

18. @cjcjameson18 Example: ruby-build

19. @cjcjameson19 Example: Concourse’s `git-resource` https://ci.concourse.ci/pipelines/resources/jobs/git-resource

20. @cjcjameson20 Example: “Check the executable bit”

21. @cjcjameson21 Example: “Check the executable bit”

22. @cjcjameson22 Example: “Check the executable bit” - integration

23. @cjcjameson23 Inside the BATS `run` command https://github.com/sstephenson/bats - Exit codes are swallowed - `stdout` and `stderr` are swallowed - Start by `source`ing in the file under test … so the file runs

24. @cjcjameson24 Thanks!

Better Bash - Unit and Integration Testing

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (20)

Recently uploaded

Recently uploaded (20)

Better Bash - Unit and Integration Testing