GNU Parallel is a command line tool that allows users to execute commands or sets of commands in parallel, enhancing efficiency. The document details installation methods, usage syntax, and examples, including error handling and parameter transmission. Additionally, it discusses GNU Stow as a package manager and provides guidelines for installing and managing packages with examples.

1. GNU Parallel s, i GNU Stow
Ziua Libert˘at, ii Programelor s, i 30 de ani de GNU ˆın B˘alt, i
Radu Dumbr˘aveanu
Universitatea de Stat “A. Russo” din B˘alt, i, Republica Moldova
B˘alt, i, 2013

2. GNU Parallel
GNU parallel este un program cu interfat, ˘a ˆın linia de comand˘a
(CLI) care permite utilizatorului executarea ˆın paralel a unei
comenzi sau a unui set de comenzi (script shell).

3. GNU Parallel
GNU parallel este un program cu interfat, ˘a ˆın linia de comand˘a
(CLI) care permite utilizatorului executarea ˆın paralel a unei
comenzi sau a unui set de comenzi (script shell).
Sintaxa:
parallel [optiuni] [comanda [argumente]] < lista de argumente
sau
parallel [optiuni] [comanda [argumente]] ::: argumente

4. Exemple clasice de utilizare
(echo ls; echo grep root /etc/passwd) | parallel

5. Exemple clasice de utilizare
(echo ls; echo grep root /etc/passwd) | parallel
parallel traceroute ::: foss.org.my debian.org freenetproject.org

6. Exemple clasice de utilizare
(echo ls; echo grep root /etc/passwd) | parallel
parallel traceroute ::: foss.org.my debian.org freenetproject.org
parallel −j0 ping −nc 3 ::: foss.org.my gnu.org freenetproject.org

7. Exemple clasice de utilizare
(echo ls; echo grep root /etc/passwd) | parallel
parallel traceroute ::: foss.org.my debian.org freenetproject.org
parallel −j0 ping −nc 3 ::: foss.org.my gnu.org freenetproject.org
find . −type f | parallel gzip

8. Exemple clasice de utilizare
(echo ls; echo grep root /etc/passwd) | parallel
parallel traceroute ::: foss.org.my debian.org freenetproject.org
parallel −j0 ping −nc 3 ::: foss.org.my gnu.org freenetproject.org
find . −type f | parallel gzip
ls ∗.gz | parallel −X mv {} archive

9. Dificult˘at, i?

10. Dificult˘at, i?
Lansarea oric˘arei comenzi ce cont, ine parallel se sfˆırs, es, te cu
mesajul de eroare:
The program ’parallel’ is currently not installed. You can
install it by typing:
sudo apt−get install moreutils

11. Dificult˘at, i?
Lansarea oric˘arei comenzi ce cont, ine parallel se sfˆırs, es, te cu
mesajul de eroare:
The program ’parallel’ is currently not installed. You can
install it by typing:
sudo apt−get install moreutils
Nu urmat, i acest˘a indicat, ie; comanda parallel din pachetul
moreutils este cu totul altceva.

12. Modalit˘at, i de instalare
Putem instala parallel ˆın felul urm˘ator:
1 sudo apt−get install parallel

13. Modalit˘at, i de instalare
Putem instala parallel ˆın felul urm˘ator:
1 sudo apt−get install parallel
sau
1 wget ftp://ftp.gnu.org/gnu/parallel/parallel−20130922.tar.bz2
2 tar xf parallel−20130922.tar.bz2
3 cd parallel−20130922
4 ./configure
5 make
6 make install

14. Modalit˘at, i de instalare
Putem instala parallel ˆın felul urm˘ator:
1 sudo apt−get install parallel
sau
1 wget ftp://ftp.gnu.org/gnu/parallel/parallel−20130922.tar.bz2
2 tar xf parallel−20130922.tar.bz2
3 cd parallel−20130922
4 ./configure
5 make
6 make install
20130922 = versiunea parallel din 22.09.2013

15. Modalit˘at, i de instalare
Dac˘a nu sˆıntet, i root atunci:
creat, i dosarele ~/bin s, i ~/share apoi
1 wget ftp://ftp.gnu.org/gnu/parallel/parallel−20130922.tar.bz2
2 tar xf parallel−20130922.tar.bz2
3 cd parallel−20130922
4 ./configure −−prefix=$HOME
5 make
6 make install

16. Modalit˘at, i de instalare
Sau dac˘a dorit, i s˘a utilizat, i parallel f˘ar˘a a-l instala (de exemplu
pentru a experimenta):
1 wget http://git.savannah.gnu.org/cgit/parallel.git/plain/src/parallel
2 chmod 755 parallel
3 cp parallel sem
4 mv parallel sem ˜/bin/

17. Modalit˘at, i de instalare
Sau dac˘a dorit, i s˘a utilizat, i parallel f˘ar˘a a-l instala (de exemplu
pentru a experimenta):
1 wget http://git.savannah.gnu.org/cgit/parallel.git/plain/src/parallel
2 chmod 755 parallel
3 cp parallel sem
4 mv parallel sem ˜/bin/
parallel este un script Perl, nu ai ce compila.
Mai mult˘a informat, ie pe
https://github.com/razrichter/gnu_parallel.

18. Dificult˘at, i?

19. Dificult˘at, i?
Lansarea oric˘arei comenzi ce cont, ine parallel se sfˆırs, es, te cu
mesajul de eroare:
1 /bin/bash: {}: command not found
sau
1 /bin/bash: :::: command not found

20. Dificult˘at, i?
Lansarea oric˘arei comenzi ce cont, ine parallel se sfˆırs, es, te cu
mesajul de eroare:
1 /bin/bash: {}: command not found
sau
1 /bin/bash: :::: command not found
Solut, ii:

21. Dificult˘at, i?
Lansarea oric˘arei comenzi ce cont, ine parallel se sfˆırs, es, te cu
mesajul de eroare:
1 /bin/bash: {}: command not found
sau
1 /bin/bash: :::: command not found
Solut, ii:
ad˘augat, i opt, iunea --gnu (parallel --gnu ...);
sau sudo rm /etc/parallael/config;
sau http://stackoverflow.com/questions/16448887/
gnu-parallel-not-working-at-all

22. Exemple de utilizare (divizarea fis, ierelor mari pe blocuri)
cat fisiermare.sql | parallel −k −−pipe grep −i ”text”
cat bigfiledump.sql | parallel −k −−pipe −−block 2M
grep −i ”text”

23. Exemple de utilizare (divizarea fis, ierelor mari pe blocuri)
cat fisiermare.sql | parallel −k −−pipe grep −i ”text”
cat bigfiledump.sql | parallel −k −−pipe −−block 2M
grep −i ”text”
cat bigfiledump.sql | parallel −k −−pipe −−block 2M
−−sshlogin server.example.com,server2.example.net
grep −i ”text”

24. Modalit˘at, i de transmitere a parametrilor
Cum transmitem parametrii la comanda care trebuie executat˘a ˆın
paralel?

25. Modalit˘at, i de transmitere a parametrilor
Cum transmitem parametrii la comanda care trebuie executat˘a ˆın
paralel?
1) Prin intrarea standard:
1 seq 5 | parallel echo
2 1
3 2
4 3
5 4
6 5

26. Modalit˘at, i de transmitere a parametrilor
Accesul la parametri:
1 seq 5 | parallel echo ”parm={}”
2 param=1
3 param=2
4 param=3
5 param=4
6 param=5

27. Modalit˘at, i de transmitere a parametrilor
Mai mult, i parametri deodat˘a:
1 seq 10 | parallel −X echo
2 1 2 3 4 5
3 6 7 8 9 10

28. Modalit˘at, i de transmitere a parametrilor
2) Prin fis, iere:
1 parallel wget < adresse.lst
1 cat adrese.lst
2 foss.org.my
3 debian.org
4 freenetproject.org

29. Modalit˘at, i de transmitere a parametrilor
3) Prin linia de comand˘a:
1 parallel echo ::: 1 2 3 4
1 1
2 2
3 3
4 4

30. Modalit˘at, i de transmitere a parametrilor
3) Prin linia de comand˘a:
1 parallel echo ::: 1 2 3 4
1 1
2 2
3 3
4 4
sau
1 parallel unzip ::: ./∗.zip

31. Specificarea num˘arul de procese paralele
1) Num˘arul de procese = num˘arul de procesoare:
1 seq 10 20 | parallel −j+0 sleep
1 ps f −o pid,command
2 PID COMMAND
3 31379 −bash
4 32270 ps f −o pid,command
5 30463 −bash
6 30579 mc
7 30581 bash −rcfile .bashrc
8 32239 /usr/bin/perl −w /usr/bin/parallel −j+0 sleep
9 32268 sleep 10
10 32269 sleep 11

32. Specificarea num˘arul de procese paralele
2) Num˘arul maximal de procese, explicit:
1 seq 10 20 | parallel −j 5 sleep
1 ps f −o pid,command
2 PID COMMAND
3 31379 −bash
4 32523 ps f −o pid,command
5 30463 −bash
6 30579 mc
7 30581 bash −rcfile .bashrc
8 32487 /usr/bin/perl −w /usr/bin/parallel −j 5 sleep
9 32518 sleep 10
10 32519 sleep 11
11 32520 sleep 12
12 32521 sleep 13
13 32522 sleep 14

33. Specificarea num˘arul de procese paralele
3) Num˘arul de sarcini per procesor, explicit:
1 $ seq 10 20 | parallel −j 150% sleep
1 ps f −o pid,command
2 PID COMMAND
3 31379 −bash
4 32636 ps f −o pid,command
5 30463 −bash
6 30579 mc
7 30581 bash −rcfile .bashrc
8 32602 /usr/bin/perl −w /usr/bin/parallel −j 150% sleep
9 32632 sleep 10
10 32633 sleep 11
11 32634 sleep 12

34. Contraexemplu
1 cd dosarcu66925fisiere
2 /usr/bin/time cp −r ∗ /tmp
3 1.77user 49.62system 6:49.01elapsed 12%CPU
4 7692784inputs+7367600outputs (0major+320minor)

35. Contraexemplu
1 cd dosarcu66925fisiere
2 /usr/bin/time cp −r ∗ /tmp
3 1.77user 49.62system 6:49.01elapsed 12%CPU
4 7692784inputs+7367600outputs (0major+320minor)
1 cd dosarcu66925fisiere
2 ls . | /usr/bin/time parallel −j+0 cp −r {} /tmp
3 2.40user 37.61system 6:30.38elapsed 10%CPU
4 7402904inputs+7367600outputs (21major+27017minor)

36. Contraexemplu
1 cd dosarcu66925fisiere
2 /usr/bin/time cp −r ∗ /tmp
3 1.77user 49.62system 6:49.01elapsed 12%CPU
4 7692784inputs+7367600outputs (0major+320minor)
1 cd dosarcu66925fisiere
2 ls . | /usr/bin/time parallel −j+0 cp −r {} /tmp
3 2.40user 37.61system 6:30.38elapsed 10%CPU
4 7402904inputs+7367600outputs (21major+27017minor)
1 cd dosarcu66925fisiere
2 ls . | /usr/bin/time parallel −j 150% cp −r {} /tmp
3 2.51user 50.74system 12:55.95elapsed 6%CPU
4 7579504inputs+7367608outputs (27major+28524minor)

37. Adrese web
Pagina web
http://www.gnu.org/software/parallel
Manual s, i exemple
http://www.gnu.org/software/parallel/man.html
Sursele
http://savannah.gnu.org/git/?group=parallel

38. Adrese web
Tutoriale
http://savannah.gnu.org/projects/parallel
Video (partea I s, i II)
http://www.youtube.com/watch?v=OpaiGYxkSuQ
http://www.youtube.com/watch?v=P40akGWJ_gY
Miscelaneu
http://static.usenix.org/publications/login/2011-02/pdfs/Tange.pdf
http://wiki.scinethpc.ca/wiki/images/7/7b/Tech-talk-gnu-parallel.pdf
http://www.luga.de/Angebote/Vortraege/GNU_Parallel_LIT_2011/GNU_
Parallel_LIT_2011.pdf
http://www.admin-magazine.com/HPC/Articles/
GNU-Parallel-Multicore-at-the-Command-Line-with-GNU-Parallel

39. Adrese web
Comparat, ie cu alte comenzi (xargs, find -exec, make -j etc)
http://www.gnu.org/software/parallel/man.html#summary_table

40. GNU Stow
GNU Stow este un caz particular de manager de pachete.
Instalare:
1 sudo apt−get install stow

41. Instalarea pachetelor
Cum instal˘am un pachet cu ajutorul lui GNU Stow?
1 sudo make install prefix=/usr/local/stow/numepachet
2 cd /usr/local/stow
3 sudo stow numepachet

42. Instalarea pachetelor
Cum instal˘am un pachet cu ajutorul lui GNU Stow?
1 sudo make install prefix=/usr/local/stow/numepachet
2 cd /usr/local/stow
3 sudo stow numepachet
Pachetele care nu folosesc GNU AutoConf pot s˘a nu accepte
parametrul “prefix”, atunci este nevoie de modific˘ari ˆın fis, ierul
Makefile.

43. Eliminarea pachetelor
1 cd /usr/local/stow/
2 sudo stow −D numepachet

44. Exemplu: mai multe versiuni ale unuia s, i aceluias, i pachet
ˆın aceeas, i distribut, ie
Mai multe versiuni ale unuia s, i aceluias, i pachet ˆımpreun˘a.
1 cd numepachet−3.0
2 sudo make install prefix=/usr/local/stow/numepachet−3.0
3 cd numepachet−3.1−beta
4 sudo make install prefix=/usr/local/stow/numepachet−3.1−beta

45. Exemplu: mai multe versiuni ale unuia s, i aceluias, i pachet
ˆın aceeas, i distribut, ie
Putem accesa fis, iere ambelor pachete prin
1 /usr/local/stow/numepachet−3.0/bin/fisierbinar
s, i
2 /usr/local/stow/numepachet−3.1−beta/bin/fisierbinar
Iar dac˘a execut˘am
1 cd /usr/local/stow
2 sudo stow numepachet−3.0
atunci lansˆınd ˆın linia de comand˘a doar fisierbinar vom accesa
/usr/local/stow/numepachet-3.0/bin/fisierbinar.

46. Adrese web
Pagina web
http://www.gnu.org/software/stow/
Manual s, i exemple
http://www.gnu.org/software/stow/manual/