This document presents an agenda for becoming a "console cowboy" by learning to be more productive using the terminal and bash shell. It covers the basic terminal tools, bash usage and configuration, utilities like grep, sed and awk, scripting with variables, conditionals and loops, and tools for developers like Homebrew, Git, Xcode and xcpretty. The goal is to stop using the mouse and automate work by writing scripts to harness the power of the Unix command line.
Despite being a slow interpreter, Python is a key component in high-performance computing (HPC). Python is easy to use. C++ is fast. Together they are a beautiful blend. A new tool, pybind11, makes this approach even more attractive to HPC code. It focuses on the niceties C++11 brings in. Beyond the syntactic sugar around the Python C API, it is interesting to see how pybind11 handles the vast difference between the two languages, and what matters to HPC.
A journey through the wonderful world of Node.js C++ addons. This talk was given at the September 8, 2015 NodeMN meetup.
Code: https://github.com/cb1kenobi/nodemn
Wprowadzenie do technologii Big Data / Intro to Big Data EcosystemSages
Introduction to Hadoop Map Reduce, Pig, Hive and Ambari technologies.
Workshop deck prepared and presented on September 5th 2015 by Radosław Stankiewicz.
During that the day participants had also the possibility to go through prepared tutorials and test their analysis on real cluster.
Codepot - Pig i Hive: szybkie wprowadzenie / Pig and Hive crash courseSages
Szybkie wprowadzenie do technologii Pig i Hive z ekosystemu Hadoop. Prezentacja wykonana w ramach warsztatów Codepot w dniu 29.08.2015. Prezentacja wykonana przez Radosława Stankiewicza oraz Bartłomieja Tartanusa.
Cluj Big Data Meetup - Big Data in PracticeSteffen Wenz
At the Cluj Big Data Meetup, we shared some insights into TrustYou's big data tech stack. Also we introduced two tools which we've found useful in our production jobs: Apache Pig and Luigi.
Also check out the code samples on GitHub: https://github.com/trustyou/meetups/tree/master/big-data
Despite being a slow interpreter, Python is a key component in high-performance computing (HPC). Python is easy to use. C++ is fast. Together they are a beautiful blend. A new tool, pybind11, makes this approach even more attractive to HPC code. It focuses on the niceties C++11 brings in. Beyond the syntactic sugar around the Python C API, it is interesting to see how pybind11 handles the vast difference between the two languages, and what matters to HPC.
A journey through the wonderful world of Node.js C++ addons. This talk was given at the September 8, 2015 NodeMN meetup.
Code: https://github.com/cb1kenobi/nodemn
Wprowadzenie do technologii Big Data / Intro to Big Data EcosystemSages
Introduction to Hadoop Map Reduce, Pig, Hive and Ambari technologies.
Workshop deck prepared and presented on September 5th 2015 by Radosław Stankiewicz.
During that the day participants had also the possibility to go through prepared tutorials and test their analysis on real cluster.
Codepot - Pig i Hive: szybkie wprowadzenie / Pig and Hive crash courseSages
Szybkie wprowadzenie do technologii Pig i Hive z ekosystemu Hadoop. Prezentacja wykonana w ramach warsztatów Codepot w dniu 29.08.2015. Prezentacja wykonana przez Radosława Stankiewicza oraz Bartłomieja Tartanusa.
Cluj Big Data Meetup - Big Data in PracticeSteffen Wenz
At the Cluj Big Data Meetup, we shared some insights into TrustYou's big data tech stack. Also we introduced two tools which we've found useful in our production jobs: Apache Pig and Luigi.
Also check out the code samples on GitHub: https://github.com/trustyou/meetups/tree/master/big-data
CONFidence 2015: DTrace + OSX = Fun - Andrzej Dyjak PROIDEA
Speaker: Andrzej Dyjak
Language: English
In recent years security industry started to grow fond of Apple’s iOS and OS X platforms. This talk will cover one of XNU's flagship debugging utilities: DTrace, a dynamic tracing framework for troubleshooting kernel and application problems on production systems in real time. It will be shown how it can be used in order to ease various tasks within the realm of dynamic binary analysis and beyond.
CONFidence: http://confidence.org.pl/
All you need to know about the JavaScript event loopSaša Tatar
Learn the difference between JavaScript Engine, JavaScript Runtime, what is JavaScript event loop and why we should care.
At the end the presentation goes through a couple of examples and implementations of throttle and debounce utility functions.
ASTs are an incredibly powerful tool for understanding and manipulating JavaScript. We'll explore this topic by looking at examples from ESLint, a pluggable static analysis tool, and Browserify, a client-side module bundler. Through these examples we'll see how ASTs can be great for analyzing and even for modifying your JavaScript. This talk should be interesting to anyone that regularly builds apps in JavaScript either on the client-side or on the server-side.
CONFidence 2015: DTrace + OSX = Fun - Andrzej Dyjak PROIDEA
Speaker: Andrzej Dyjak
Language: English
In recent years security industry started to grow fond of Apple’s iOS and OS X platforms. This talk will cover one of XNU's flagship debugging utilities: DTrace, a dynamic tracing framework for troubleshooting kernel and application problems on production systems in real time. It will be shown how it can be used in order to ease various tasks within the realm of dynamic binary analysis and beyond.
CONFidence: http://confidence.org.pl/
All you need to know about the JavaScript event loopSaša Tatar
Learn the difference between JavaScript Engine, JavaScript Runtime, what is JavaScript event loop and why we should care.
At the end the presentation goes through a couple of examples and implementations of throttle and debounce utility functions.
ASTs are an incredibly powerful tool for understanding and manipulating JavaScript. We'll explore this topic by looking at examples from ESLint, a pluggable static analysis tool, and Browserify, a client-side module bundler. Through these examples we'll see how ASTs can be great for analyzing and even for modifying your JavaScript. This talk should be interesting to anyone that regularly builds apps in JavaScript either on the client-side or on the server-side.
Présentation aux Geeks Anonymes Liège par Cyril Soldani, le 13 décembre 2017.
Page des Geeks Anonymes : https://www.recherche.uliege.be/cms/c_9463913/fr/geeks-anonymes
This talks covers some of the basic aspects of the JIP pipeline system (http://pyjip.readthedocs.org) and it's command line interface.
JIP is a system to manage jobs on a Cluster system and simplify the process of building computational pipelines. JIP can interact with Slurm, SGE, PBS/Torque, or LSF clusters and comes with a small local scheduler to run without any remote grid engine.
Whether you call yourself a system administrator, developer, or DevOps sprint mediator, life is too short for sloppy shell scripts! In this talk, we look at how to improve them to stand the test of time. Michael will share how to create a good foundation for your scripts, so they run more reliable now and in the future. Your (future) colleagues might love you for it.
Focus areas of this presentation include error handling, security, style, and best practices. Also, it will cover (many) mistakes made by Michael over the last 20 years.
Geecon 2019 - Taming Code Quality in the Worst Language I Know: BashMichał Kordas
I don't know any other languages with more pitfalls, perils and gotchas than Bash. Still, we use it in almost every larger project for deployment or maintenance scripts, because there is no better, more powerful and more universal choice on Unix platform. However, there is ridiculous amount of things that could go wrong if you don't have deep understanding of shell scripting. Your experience about typical issues with Java or other JVM languages is definitely not enough here. You need to deeply understand Linux ecosystem and its history in order to write correct script... or you don't? I will prove to you that Bash could be tamed and made easy if proper code quality standards and static analysis tools are applied and enforced in your delivery pipelines. I'll share my opinions and experiences from a large banking project and I'll tell you which tools and style guides we use.
A decade ago, the database was assumed to be a solved problem. Relational databases (PostgreSQL, MySQL, SQLite to name a few) were dominating the database market and hierarchical databases (LDAP, DNS) where regarded as niche solutions. The NoSQL revolution surely changed the concept of what a database can be. At the same time, the popularity of mobile devices exploded. This talk will dive into how data structures are persisted and queried on mobile devices today, and try to revive the old question: is the database really a solved problem?
Introduction to JavaScript for Modern Software DevelopmentKenneth Geisshirt
The talk introduces JavaScript for experienced software developer without any knowledge of the language. Moreover, interaction with the browser is presented, and a bit of server-side programming is covered.
My contribution of Dansk Naturvidenskabsfestival 2010. It's a dicussion on being famous or important and how social network analysis can make you understand the difference.
A Study of Variable-Role-based Feature Enrichment in Neural Models of CodeAftab Hussain
Understanding variable roles in code has been found to be helpful by students
in learning programming -- could variable roles help deep neural models in
performing coding tasks? We do an exploratory study.
- These are slides of the talk given at InteNSE'23: The 1st International Workshop on Interpretability and Robustness in Neural Software Engineering, co-located with the 45th International Conference on Software Engineering, ICSE 2023, Melbourne Australia
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...Mind IT Systems
Healthcare providers often struggle with the complexities of chronic conditions and remote patient monitoring, as each patient requires personalized care and ongoing monitoring. Off-the-shelf solutions may not meet these diverse needs, leading to inefficiencies and gaps in care. It’s here, custom healthcare software offers a tailored solution, ensuring improved care and effectiveness.
OpenMetadata Community Meeting - 5th June 2024OpenMetadata
The OpenMetadata Community Meeting was held on June 5th, 2024. In this meeting, we discussed about the data quality capabilities that are integrated with the Incident Manager, providing a complete solution to handle your data observability needs. Watch the end-to-end demo of the data quality features.
* How to run your own data quality framework
* What is the performance impact of running data quality frameworks
* How to run the test cases in your own ETL pipelines
* How the Incident Manager is integrated
* Get notified with alerts when test cases fail
Watch the meeting recording here - https://www.youtube.com/watch?v=UbNOje0kf6E
Utilocate offers a comprehensive solution for locate ticket management by automating and streamlining the entire process. By integrating with Geospatial Information Systems (GIS), it provides accurate mapping and visualization of utility locations, enhancing decision-making and reducing the risk of errors. The system's advanced data analytics tools help identify trends, predict potential issues, and optimize resource allocation, making the locate ticket management process smarter and more efficient. Additionally, automated ticket management ensures consistency and reduces human error, while real-time notifications keep all relevant personnel informed and ready to respond promptly.
The system's ability to streamline workflows and automate ticket routing significantly reduces the time taken to process each ticket, making the process faster and more efficient. Mobile access allows field technicians to update ticket information on the go, ensuring that the latest information is always available and accelerating the locate process. Overall, Utilocate not only enhances the efficiency and accuracy of locate ticket management but also improves safety by minimizing the risk of utility damage through precise and timely locates.
Top Features to Include in Your Winzo Clone App for Business Growth (4).pptxrickgrimesss22
Discover the essential features to incorporate in your Winzo clone app to boost business growth, enhance user engagement, and drive revenue. Learn how to create a compelling gaming experience that stands out in the competitive market.
Navigating the Metaverse: A Journey into Virtual Evolution"Donna Lenk
Join us for an exploration of the Metaverse's evolution, where innovation meets imagination. Discover new dimensions of virtual events, engage with thought-provoking discussions, and witness the transformative power of digital realms."
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...Globus
The Earth System Grid Federation (ESGF) is a global network of data servers that archives and distributes the planet’s largest collection of Earth system model output for thousands of climate and environmental scientists worldwide. Many of these petabyte-scale data archives are located in proximity to large high-performance computing (HPC) or cloud computing resources, but the primary workflow for data users consists of transferring data, and applying computations on a different system. As a part of the ESGF 2.0 US project (funded by the United States Department of Energy Office of Science), we developed pre-defined data workflows, which can be run on-demand, capable of applying many data reduction and data analysis to the large ESGF data archives, transferring only the resultant analysis (ex. visualizations, smaller data files). In this talk, we will showcase a few of these workflows, highlighting how Globus Flows can be used for petabyte-scale climate analysis.
AI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI AppGoogle
AI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI App
👉👉 Click Here To Get More Info 👇👇
https://sumonreview.com/ai-fusion-buddy-review
AI Fusion Buddy Review: Key Features
✅Create Stunning AI App Suite Fully Powered By Google's Latest AI technology, Gemini
✅Use Gemini to Build high-converting Converting Sales Video Scripts, ad copies, Trending Articles, blogs, etc.100% unique!
✅Create Ultra-HD graphics with a single keyword or phrase that commands 10x eyeballs!
✅Fully automated AI articles bulk generation!
✅Auto-post or schedule stunning AI content across all your accounts at once—WordPress, Facebook, LinkedIn, Blogger, and more.
✅With one keyword or URL, generate complete websites, landing pages, and more…
✅Automatically create & sell AI content, graphics, websites, landing pages, & all that gets you paid non-stop 24*7.
✅Pre-built High-Converting 100+ website Templates and 2000+ graphic templates logos, banners, and thumbnail images in Trending Niches.
✅Say goodbye to wasting time logging into multiple Chat GPT & AI Apps once & for all!
✅Save over $5000 per year and kick out dependency on third parties completely!
✅Brand New App: Not available anywhere else!
✅ Beginner-friendly!
✅ZERO upfront cost or any extra expenses
✅Risk-Free: 30-Day Money-Back Guarantee!
✅Commercial License included!
See My Other Reviews Article:
(1) AI Genie Review: https://sumonreview.com/ai-genie-review
(2) SocioWave Review: https://sumonreview.com/sociowave-review
(3) AI Partner & Profit Review: https://sumonreview.com/ai-partner-profit-review
(4) AI Ebook Suite Review: https://sumonreview.com/ai-ebook-suite-review
#AIFusionBuddyReview,
#AIFusionBuddyFeatures,
#AIFusionBuddyPricing,
#AIFusionBuddyProsandCons,
#AIFusionBuddyTutorial,
#AIFusionBuddyUserExperience
#AIFusionBuddyforBeginners,
#AIFusionBuddyBenefits,
#AIFusionBuddyComparison,
#AIFusionBuddyInstallation,
#AIFusionBuddyRefundPolicy,
#AIFusionBuddyDemo,
#AIFusionBuddyMaintenanceFees,
#AIFusionBuddyNewbieFriendly,
#WhatIsAIFusionBuddy?,
#HowDoesAIFusionBuddyWorks
Globus Connect Server Deep Dive - GlobusWorld 2024Globus
We explore the Globus Connect Server (GCS) architecture and experiment with advanced configuration options and use cases. This content is targeted at system administrators who are familiar with GCS and currently operate—or are planning to operate—broader deployments at their institution.
We describe the deployment and use of Globus Compute for remote computation. This content is aimed at researchers who wish to compute on remote resources using a unified programming interface, as well as system administrators who will deploy and operate Globus Compute services on their research computing infrastructure.
First Steps with Globus Compute Multi-User EndpointsGlobus
In this presentation we will share our experiences around getting started with the Globus Compute multi-user endpoint. Working with the Pharmacology group at the University of Auckland, we have previously written an application using Globus Compute that can offload computationally expensive steps in the researcher's workflows, which they wish to manage from their familiar Windows environments, onto the NeSI (New Zealand eScience Infrastructure) cluster. Some of the challenges we have encountered were that each researcher had to set up and manage their own single-user globus compute endpoint and that the workloads had varying resource requirements (CPUs, memory and wall time) between different runs. We hope that the multi-user endpoint will help to address these challenges and share an update on our progress here.
May Marketo Masterclass, London MUG May 22 2024.pdfAdele Miller
Can't make Adobe Summit in Vegas? No sweat because the EMEA Marketo Engage Champions are coming to London to share their Summit sessions, insights and more!
This is a MUG with a twist you don't want to miss.
Quarkus Hidden and Forbidden ExtensionsMax Andersen
Quarkus has a vast extension ecosystem and is known for its subsonic and subatomic feature set. Some of these features are not as well known, and some extensions are less talked about, but that does not make them less interesting - quite the opposite.
Come join this talk to see some tips and tricks for using Quarkus and some of the lesser known features, extensions and development techniques.
AI Genie Review: World’s First Open AI WordPress Website CreatorGoogle
AI Genie Review: World’s First Open AI WordPress Website Creator
👉👉 Click Here To Get More Info 👇👇
https://sumonreview.com/ai-genie-review
AI Genie Review: Key Features
✅Creates Limitless Real-Time Unique Content, auto-publishing Posts, Pages & Images directly from Chat GPT & Open AI on WordPress in any Niche
✅First & Only Google Bard Approved Software That Publishes 100% Original, SEO Friendly Content using Open AI
✅Publish Automated Posts and Pages using AI Genie directly on Your website
✅50 DFY Websites Included Without Adding Any Images, Content Or Doing Anything Yourself
✅Integrated Chat GPT Bot gives Instant Answers on Your Website to Visitors
✅Just Enter the title, and your Content for Pages and Posts will be ready on your website
✅Automatically insert visually appealing images into posts based on keywords and titles.
✅Choose the temperature of the content and control its randomness.
✅Control the length of the content to be generated.
✅Never Worry About Paying Huge Money Monthly To Top Content Creation Platforms
✅100% Easy-to-Use, Newbie-Friendly Technology
✅30-Days Money-Back Guarantee
See My Other Reviews Article:
(1) TubeTrivia AI Review: https://sumonreview.com/tubetrivia-ai-review
(2) SocioWave Review: https://sumonreview.com/sociowave-review
(3) AI Partner & Profit Review: https://sumonreview.com/ai-partner-profit-review
(4) AI Ebook Suite Review: https://sumonreview.com/ai-ebook-suite-review
#AIGenieApp #AIGenieBonus #AIGenieBonuses #AIGenieDemo #AIGenieDownload #AIGenieLegit #AIGenieLiveDemo #AIGenieOTO #AIGeniePreview #AIGenieReview #AIGenieReviewandBonus #AIGenieScamorLegit #AIGenieSoftware #AIGenieUpgrades #AIGenieUpsells #HowDoesAlGenie #HowtoBuyAIGenie #HowtoMakeMoneywithAIGenie #MakeMoneyOnline #MakeMoneywithAIGenie
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...Globus
Large Language Models (LLMs) are currently the center of attention in the tech world, particularly for their potential to advance research. In this presentation, we'll explore a straightforward and effective method for quickly initiating inference runs on supercomputers using the vLLM tool with Globus Compute, specifically on the Polaris system at ALCF. We'll begin by briefly discussing the popularity and applications of LLMs in various fields. Following this, we will introduce the vLLM tool, and explain how it integrates with Globus Compute to efficiently manage LLM operations on Polaris. Attendees will learn the practical aspects of setting up and remotely triggering LLMs from local machines, focusing on ease of use and efficiency. This talk is ideal for researchers and practitioners looking to leverage the power of LLMs in their work, offering a clear guide to harnessing supercomputing resources for quick and effective LLM inference.
Software Engineering, Software Consulting, Tech Lead, Spring Boot, Spring Cloud, Spring Core, Spring JDBC, Spring Transaction, Spring MVC, OpenShift Cloud Platform, Kafka, REST, SOAP, LLD & HLD.
Atelier - Innover avec l’IA Générative et les graphes de connaissancesNeo4j
Atelier - Innover avec l’IA Générative et les graphes de connaissances
Allez au-delà du battage médiatique autour de l’IA et découvrez des techniques pratiques pour utiliser l’IA de manière responsable à travers les données de votre organisation. Explorez comment utiliser les graphes de connaissances pour augmenter la précision, la transparence et la capacité d’explication dans les systèmes d’IA générative. Vous partirez avec une expérience pratique combinant les relations entre les données et les LLM pour apporter du contexte spécifique à votre domaine et améliorer votre raisonnement.
Amenez votre ordinateur portable et nous vous guiderons sur la mise en place de votre propre pile d’IA générative, en vous fournissant des exemples pratiques et codés pour démarrer en quelques minutes.
Atelier - Innover avec l’IA Générative et les graphes de connaissances
Unleash your inner console cowboy
1. Unleash your inner
console cowboy
Kenneth Geisshirt
kg@realm.io
Realm Inc.
@realm
http://github.com/Realm/
http://realm.io/
Ivan Constantin, https://www.flickr.com/photos/ivan70s/
2. Today’s goal
• Present bash as a productivity tool
• stop using the mouse 🐁
• Write scripts to automate your work
• Begin to use advanced tools in your daily work
Become a console cowboy
3. Agenda
• The terminal and the
shell
• Basic usage of bash
• Living on the
command-line
• Useful utilities
• Scripting
• Home brew
• Tools for developers
• git
• Xcode
5. Which terminal?
• iTerm2 is much better
• Easier to change tab (⌘ left
+ right, CTRL+TAB)
• Change Desktop navigation
to ⌥ left + right
• Add CTRL left + right to
iTerm2 preferences
• Keeps SSH connection alive
• http://iterm2.com/
6. Which shell?
OS X comes with many shells
➤ bash, csh, ksh, sh, tcsh, and zsh
Parée, https://www.flickr.com/photos/pareeerica/
Since 10.3, bash has been the default shell
OS X 10.11.1 carries bash 3.2.57 (2014-11-07)
Stephen R. Bourne (Bell lab) introduced the
shell to UNIX in 1977
Home brew has many great bash related packages
7. Redirection
UNIX idioms
• a tool should do one thing
but do it well
• text is the universal data
format
}Output of one utility is input for the next
Bash implements redirection:
• stdout to file: >
• stdin from file <
• append stdout to file: >>
• stderr to stdout: 2>&1
Examples:
echo “Hello” > hello
cat < hello
echo “World” >> hello
clang notfound.m > error 2>&1
8. Pipes
• Introduced to UNIX by Douglas
McIlroy in 1973
• Pipes are the glue in UNIX
component based programming (aka
shell scripting)
• Powerful idiom for stream processing
• The character | is used by all known
shells
Examples
lsof | grep ^Keynote | wc -l
ifconfig | grep -e ^[a-z] | cut -f1 -d:
"Pipeline" by TyIzaeL - Licensed under Public domain via Wikimedia Commons
http://commons.wikimedia.org/wiki/File:Pipeline.svg#mediaviewer/File:Pipeline.svg
9. Configuration
• $HOME/.bash_profile and $HOME/.bashrc
are your personal configuration
• alias - useful for often used options
• Setting prompt (PS1) and search path (PATH)
• Reload configuration: source ~/.bash_profile
10. .bash_profile
# Ignore a few commands in history
export HISTIGNORE="pwd:ls:ls -l:cd"
# don't put duplicate lines in the history. See bash(1) for more options
# don't overwrite GNU Midnight Commander's setting of `ignorespace'.
HISTCONTROL=$HISTCONTROL${HISTCONTROL+:}ignoredups
# Bash completion
if [ -f $(brew --prefix)/etc/bash_completion ]; then
. $(brew --prefix)/etc/bash_completion
fi
# Prompt - including git
PS1='u@h:w$(__git_ps1 " (%s)") $ ‘
# Color ls etc.
alias ls="ls -G"
alias ll="ls -l"
# https://xkcd.com/149/
alias fuck='sudo $(history -p !!)' # rerun as root
11. Keyboard short-cuts
• Bash uses Emacs bindings by default 😄
(help bind for details)
Movement
• CTRL-a beginning of line
• CTRL-e end of line
• CTRL-← One word left
• CTRL-→ One word right
Cut-n-paste
• CTRL-space mark
• 🔙 Delete word
• CTRL-d Delete character
• CTRL-_ undo
• CTRL-k delete until end
of line
• CTRL-y yank from kill-
ring
Remap CTRL-←/→
12. History
• Bash stores your command history
• history - show latest commands
• !! - run last command
• !number - run command number again
• CTRL-r - search in history
• Exclude commands from history:
• export HISTIGNORE=“pwd:ls:ls -l:cd”
• Exclude duplicates from history:
• export HISTCONTROL:ignoredups
13. Completion
• Use TAB to let Bash complete as much as possible
• Use TAB+TAB to show possible completions
• Bash has programmable completion → you can
specify what Bash does
• Large collections of completion recipes exist (home
brew is your friend)
14. Living on the command-line
• cd - - go back to previous folder
• file file - guess file content (magic numbers)
• lsof - list open files
• ps (aux or -ef) and top - show processes
• Simple watch:
while true ; do clear ; command ; sleep
n ; done
15. OS X specific commands
• open file - Starts registered program and open file
• say “Hello world” - speech synthesis (download
extra voices/languages in System preferences)
• ls | pbcopy - copy stdin to paste board
• pbpaste - paste to stdout
• dns-sd -B _ssh._tcp - show Bonjour enabled SSH
hosts
16. Useful utilities
Find files: find . -name ‘*.o’ -delete
Patterns: grep -r list *
Cut field: cut -f1,3 -d: /etc/passwd
Word count: wc -l *.cpp
Transform: tr “ “ “_” < README.org
Sort lines: sort -t: -n -r -k 4 /etc/passwd
Last lines: tail /etc/passwd
First lines: head /etc/passwd
17. sed - the stream editor
• sed is used to edit files non-
interactively
• Option -E gives an editing (regular)
expression
• s/FISH/HORSE/g - substitute
• /FISH/d - delete lines
joinash, https://www.flickr.com/photos/joinash/
Option -i is tricky:
• GNU sed has optional extension
• BSD sed requires extension (‘’ is useful)
18. awk - a processing tool
• awk is a programming
language by itself
• Matching lines are
processed
• line is split in fields
(spaces are default)
Patterns:
BEGIN - before opening file
END - after closing file
A. Aho, P. Weinberger, B. Kernighan
cat foo | awk 'BEGIN {total=0 } END { print total } { total+=$1 }'
20. Bash for programmers
• Bash is a complete programming language
• Shell scripts grow and become ugly 😞
• Execution:
• sh script.sh
• chmod +x script.sh; ./script.sh
• Interpreted language → slow
21. Basic syntax
• White spaces: space and
tab
• Comments: # and to end-
of-line
• Statements: either end-
of-line of ; (semicolon)
• Variables and functions:
Letters, digits and
underscore
#!/bin/bash
# Monte Carlo calculation of pi
NSTEPS=500
NHITS=0
i=0
while [ $i -lt $NSTEPS ]; do
x=$(echo $RANDOM/32767 | bc -l)
y=$(echo $RANDOM/32767 | bc -l)
d=$(echo "sqrt($x*$x+$y*$y) < 1.0" | bc -l)
if [ $d -eq 1 ]; then
NHITS=$(($NHITS + 1))
fi
i=$(($i + 1))
done
PI=$(echo "4.0*$NHITS/$NSTEPS" | bc -l)
echo "PI = $PI"
22. Variables• Case-sensitive names
• No declarations, no types
• Strings: “…” are substituted; ‘…’ are not
• Assignment (=): no spaces!
• $(…) assignment from stdout
including spaces
• I often use awk ‘{print $1}’ to
remove spaces
• $((…)) arithmetic
• $varname - value of variable varname
Built-in variables:
• $# is the number of argument
• $1, $2, … are the arguments
• $$ is the process ID
• $? is exit code of last command
24. Branches
• Simple branching with if then
else fi
• Enclose condition with []
• elif is possible, too
• Use case in esac when you
can many cases and single
condition
String operators:
-z is empty?
-d is directory?
-f is file?
== equal to
!= not equal to
Integer operators:
-eq equal to
-lt less than
-ne not equal to
-gt greater than
25. branches.sh
#!/bin/bash
if [ -z "$1" ]; then
name="Arthur"
else
name="$1"
fi
if [ "$name" != "Arthur" ]; then
echo "Not Arthur"
else
echo "Hello Arthur"
fi
answer=$((6*7))
if [ $answer -eq 42 ]; then
echo "Life, the universe, and everything"
fi
26. branches.sh - con’t
case "$name" in
"Arthur")
echo "Welcome onboard"
;;
"Trillian")
echo "You know Arthur"
;;
*)
echo "Who are you?"
;;
esac
27. Loops
• Simple loops: for … in … ; do …
done
• The seq utility can generate list of
numbers
• Conditional loops: while … ; do …
done
• Line-by-line: while read line ;
do … done
One-liner (similar to watch)
while [ true ]; do
clear;
echo $RANDOM;
sleep 1;
done
28. loops.sh#!/bin/bash
# Multiplication table
for i in $(seq 1 10); do
echo "$i $((3*$i))"
done
# All .sh files
for f in $(ls *.sh); do
echo "$f $(head -1 $f | cut -c3-) $(wc -l $f | awk '{print $1}')"
done
# read self line-by-line
i=1
cat $0 | while read line ; do
nchars=$(echo "$line" | wc -c | awk '{print $1}')
echo "$i $nchars"
i=$(($i+1))
done | sort -n -k 2
29. Functions
• Functions can increase readability of your scripts
• arguments are $1, $2, …
• local variables can be used
• return an integer and get it as $?
• Use global variable to return a string 😒
31. Tips and tricks
• Use set -e to exit early
• or use || exit 1
• set -O pipefail and you can get the exit code of the
first failing program in a pipe
• xcpretty never fails but xcodebuild might
• Use tee to write to stdout and file
• To trace (debugging): set -x or sh -x
32. Tips and tricks
• Always use “$var” when dealing with file names (and strings)
• str=“fish horse”; for i in $str; do echo $i; done
• str=“fish horse”; for i in “$str”; do echo $i; done
• Call mkdir -p when creating folders
• Create temp. files with mktemp /tmp/$$.XXXXXX
• Using variable to modify behaviour of script:
• FLAGS=“-O3 -libc++=stdlibc++” build.sh
• Subshells: (cd foo && rm -f bar)
34. Home brew
• Home brew provides calories for
console cowboys
• You don’t have to be root to install
• Software is installed in /usr/
local/Cellar, and symlinked to /
usr/local/bin
• Brew cask is for binary distribution
• http://brew.sh and http://
caskroom.io
Greg Peverill-Conti, https://www.flickr.com/photos/gregpc/
Examples:
brew search bash
brew info bash
brew install bash
brew update
35. Tools for developers
• Apple provides some basic tools
• nm - display symbol table
• c++filt - Prettify C++ and Java names
• otool -L - display which shared libraries are
required
• libtool - create libraries
• lipo - manipulate fat/universal binaries
zzpza, https://www.flickr.com/photos/zzpza/
Examples:
nm book.o | c++filt
otool -L RealmInspector
37. Xcode
• You can build Xcode projects at the
command-line
xcodebuild -scheme OreDevPlain -
configuration Release -sdk
iphonesimulator
• Targets: clean, build, test
• You can add shell scripts to build phases
38. xcpretty
• The output of xcodebuild can be hard to read
• xcpretty makes it prettier
• Installation:
• sudo gem install xcpretty
• Usage:
• xbuildcode … | xcpretty
40. Further information
• Classical Shell Scripting. R. Arnolds and N.H.F.
Beebe. O’Reilly Media, 2005.
• The sed FAQ: http://sed.sourceforge.net/
sedfaq.html
• Advanced Bash-Scripting Guide: http://
www.tldp.org/LDP/abs/html/
http://realm
.io
W
e
are
hiring