The document provides an overview of using the command line in Linux, highlighting its significance in data science and the Unix philosophy of utilizing simple tools effectively. It covers various command-line operations, shell types, and practical examples for file management and automation tasks. Additionally, it emphasizes the potential of the command line for data manipulation and the automation of backup processes using cron jobs.

1. LEARNTOLOVETHE
COMMANDLINE
Thursday, 19 February, 2015
Google Montreal
Shopify Montreal

2. INLOVEWITH...
THECOMMANDLINE
/ /
http://cli-talk.bitballoon.com
Blanca Mancilla bluciam @Blanckus

4. LinuxCommand.org
You have Linux installed and running. The
GUI is working fine, but you are getting
tired of changing your desktop themes.
You keep seeing this "terminal" thing.

5. WHATISTHECOMMAND
LINE?

6. Anybody with little or no experience?
What do you know about it?
What are you expecting to learn tonight?
WHATISTHECOMMAND
LINE?

7. Specifically, the computer program is the operating
system (UNIX or nowadays Linux) and the interface is
what we called the .
THECOMMANDLINE
ANDTHESHELL
A command-line interface (CLI) is a means of interacting with a
computer program where the user issues commands to the program in
the form of successive lines of text (command lines).
https://en.wikipedia.org/wiki/Command-line_interface
shell

8. TYPESOFSHELLSINLINUX
sh
bash
csh and tcsh
ksh
read more...

9. HOWDOIKNOWWHICH
SHELL?
~ $ w h i c h b a s h
/ b i n / b a s h
~ $ w h i c h s h
/ b i n / s h
~ $ w h i c h c s h
~ $ e c h o $ S H E L L
/ b i n / b a s h
~ $ s h
$ e c h o $ S H E L L
/ b i n / b a s h
$ p s - p $ $
P I D T T Y T I M E C M D
1 7 7 9 3 p t s / 2 8 0 0 : 0 0 : 0 0 s h
$ ^ D
~ $ p s - p $ $
P I D T T Y T I M E C M D
1 5 6 1 5 p t s / 2 8 0 0 : 0 0 : 0 1 b a s h

10. THEINSPIRATION

11. by Jeroen JanssensData Science at the Command Line

12. —Chris H. Wiggins
Associate Professor in the Department of Applied Physics
and Applied Mathematics at Columbia University
and Chief Data Scientist at The New York Times
The Unix philosophy of simple tools, each
doing one job well, then cleverly piped
together, is embodied by the command
line. Jeroen expertly discusses how to
bring that philosophy into your work in
data science, illustrating how the
command line is not only the world of file
input/ output, but also the world of data
manipulation, exploration, and even
modeling.

13. APPENDIX:COMMANDS
alias awk aws bash bc bigmler body cat cd chmod cols
cowsay cp csvcut csvgrep csvjoin csvlook csvsort csvsql
csvstack csvstat curl curlicue cut display drake dseq echo
env export feedgnuplot fieldsplit find for git grep head
header in2csv jq json2csv less ls man mkdir mv parallel
paste pbc pip pwd python R Rio Rio-scatter rm

14. SOMEDETAILS
NOW!

15. Yes! Of course.
FORLOOPS?
f o r i i n * p d f ; d o e v i n c e $ i ; d o n e

16. FORLOOPS
: l y c e u m $ l s * p d f
H o w T o B e A P r o g r a m m e r . p d f S E N G 3 0 2 0 - 0 . 1 . p d f t e a c h i n g - p r g m . p d f
S E N G 3 0 1 0 - B A R T - R e q - v 1 - 1 - 3 d . p d f S E N G 3 0 2 0 _ 0 7 s 2 . p d f
: l y c e u m $ e v i n c e * p d f
: l y c e u m $ f o r i i n * p d f ; d o e v i n c e $ i ; d o n e

17. or even
FORLOOPS
In a script
f o r i i n * p d f
d o
e v i n c e $ i
d o n e
f o r i i n ` l s * p d f `
d o
e v i n c e $ i
d o n e
More info

18. CHECKIFAFILEEXISTSAND
IFNOT...
c l i - t a l k : $ [ - f i n d e x . h t m l ] & & e c h o " Y e s , w e h a v e a p r e s e n t a t i o n ! "
| | e c h o " P A N I C "

19. CHECKIFAFILEEXISTSAND
IFNOT...

20. Let's try a demo
CHECKIFAFILEEXISTSAND
IFNOT...
c l i - t a l k : $ l o c a t e * 9 7 6 8 . J P G

21. GLOBPATTERNS:
REALMAGIC!
Strictly speaking, there are neither regular expressions
nor patterns used in pattern recognition.
They are .glob patterns

22. GLOBPATTERNS:
REALMAGIC!
l s a p p / * . { e r b , s c s s , r b , y m l }
v i p u b l i c _ h t m l / [ i I ] * . h t m *

23. My favorite: Move all image files for the years
2010 to 2012 into .
GLOBPATTERNS:
REALMAGIC!
my_dir
m v { I M G , i m g } * 2 0 1 [ 0 1 2 ] * . { j p g , J P G , p n g , g i f } m y _ d i r

24. FINDARESOURCE
ANYWHEREINYOURHARD
DRIVE

25. versus
FINDARESOURCE...
locate find
L o r i B i z : $ l o c a t e * . e r b
L o r i B i z : $ f i n d . - n a m e * . e r b

26. find . -name *.erb -exec ls {} ;
find . -name *.erb -exec grep mytableauthor {} ;
find . -name *.erb -exec grep -H mytableauthor {} ;
find . -name *.scss -exec grep -H mytableauthor {} ;
find . -name *.scss -exec grep -n -H mytableauthor {} ;
FINDARESOURCE...

27. FINDANDREPLACETRAILING
SPACESINSOURCECODE

28. FIND...TRAILINGSPACESIN
SOURCECODE
c l i - t a l k $ g r e p " s + $ " i n d e x . h t m l

29. ...REPLACETRAILINGSPACES
INSOURCECODE
L o r i B i z $ f i n d . - n a m e * . e r b - e x e c g r e p - H n " s + $ " { } ;
L o r i B i z $ f i n d . - n a m e * . e r b - e x e c g r e p - H n " s + $ " { } ; - e x e c v i { } ;
# O n c e i n v i f i n d a n d r e p l a c e u s i n g t h e c o m m a n d
: % s / s + $ / /

30. MAKINGBACKUPSOF
GITREPOSITORIES
superspreadsheet.wordpress.com

31. MAKINGBACKUPSOF...
1. pullfrom the bare repository
2. commitlocal changes
3. pushchanges to the bare repository

32. Pull and Commit
MAKINGBACKUPSOF...
1 # ! / b i n / s h
2 #
3 # A u t o m a t i c p u l l a n d c o m m i t o n t h e d i r e c t o r i e s l i s t e d i n d i r s _ t o _ p u l l
4
5 C R O N L O G = " / c o m p l e t e / p a t h / t o / p u l l c o m m i t p u s h . l o g "
6 C O M M I T M S G = " / c o m p l e t e / p a t h / t o / m s g A u t o C o m m i t "
7 e c h o > > $ C R O N L O G
8 e c h o " N i g h t l y g i t a u t o - c o m m i t f o r t h e a b s e n t m i n d s : $ ( d a t e ) " > > $ C R O N L O G
9 f o r i i n ` c a t / c o m p l e t e / p a t h / t o / d i r s _ t o _ p u l l `
1 0 d o ( c d $ i ;
1 1 e c h o " P U L L $ i $ ( d a t e ) " > > $ C R O N L O G ;
1 2 g i t p u l l > > $ C R O N L O G 2 > & 1 ;
1 3 e c h o " C O M M I T $ i $ ( d a t e ) " > > $ C R O N L O G ;
1 4 g i t c o m m i t - a - F $ C O M M I T M S G > > $ C R O N L O G 2 > & 1
1 5 e c h o > > $ C R O N L O G )
1 6 d o n e
1 7 e c h o > > $ C R O N L O G ;

33. Push
MAKINGBACKUPSOF...
1 # ! / b i n / s h
2 #
3 # A u t o m a t i c p u s h o n t h e d i r e c t o r i e s l i s t e d i n d i r s _ t o _ p u s h
4
5 C R O N L O G = " / c o m p l e t e / p a t h / t o / p u l l c o m m i t p u s h . l o g "
6 e c h o > > $ C R O N L O G
7 e c h o " N i g h t l y g i t a u t o - p u s h f o r t h e a b s e n t m i n d s : $ ( d a t e ) " > > $ C R O N L O G
8 f o r i i n ` c a t / c o m p l e t e / p a t h / t o / d i r s _ t o _ p u s h `
9 d o ( c d $ i ;
1 0 e c h o " P U S H : $ i $ ( d a t e ) " > > $ C R O N L O G
1 1 g i t p u s h > > $ C R O N L O G 2 > & 1
1 2 e c h o > > $ C R O N L O G
1 3 ) ;
1 4 d o n e
1 5 e c h o > > $ C R O N L O G ;

34. : from the Greek word for time, χρόνος chronos.
MAKINGBACKUPSUSING...
cron

35. https://en.wikipedia.org/wiki/Cron
MAKINGBACKUPSUSING...
Cron is a time-based job scheduler that
Sysadmins use to schedule jobs to run
periodically at fixed times, dates, or
intervals (i.e. download email at regular
intervals).

36. MAKINGBACKUPSUSING...
c r o n t a b - e
minute hour dayOfMonth month dayOfWeek command
3 2 * * * / c o m p l e t e / p a t h / t o / p u l l _ c o m m i t 2 > / c o m p l e t e / p a t h / t o / p u l l _ c o m m i t . e r r
5 2 * * * / c o m p l e t e / p a t h / t o / p u s h 2 > / c o m p l e t e / p a t h / t o / p u s h . e r r

37. Blog post in development!
CREATINGACOPYOFAGIT
REPOSITORYWHICHISNOT
QUITEACOPY

38. THEEND
Are you in love yet?

39. JavaScript source code for the slides taken from
.
I encourage you to try it!
THEEND
reveal.js