Perl DBI usage at report building Joe Jiang [email_address]
Perl Language Practical Extraction and ... Reporting Language
Perl Programmer Laziness Impatience Hubris (Boldness)
DBMS Database Management (descide) System
DBI.pm The gate to Management level programming
Back to reporting The man in breau The game they played The suggestion I once made on OLAP
Chemical Perl (Gas-ware) PHP for website Python for architecture Ruby for rails Perl for what?
Reports again DMBS is the container SQL is for query Perl is the formater
Give me Excel Copy HTML table into Excel Or, name HTML as .xls Use some IDE tool?
Toolset basic $ echo 'select SESSION_ID, HIT_ID from hit where USERNAME=? and ACCESS_TIME between to_date(?,?) and to_date...
It worked $ bofc < session.sql user-123 201106122345 YYYYMMDDHH24MI 201106122357 YYYYMMDDHH24MI | wc -l  ↵ SESSION_ID  HIT...
And ... $ bofc < session.sql user-123 201106122345 YYYYMMDDHH24MI 201106122357 YYYYMMDDHH24MI | sort ↵ SESSION_ID  HIT_ID ...
Highlights <ul><li>works with wc -l & sort (Laziness)
works with parameter binding (Boldness)
purify the SQL with limited punctuaction (Impatience) </li></ul>
Camel coding PERL5LIB=/proj/Web/EMIS/Modules perl -MEMIS::Database -le '($s= do{$d=EMIS::Database::gethandle(q(CDB)); $d->...
Hints <ul><li>PERL5LIB=/proj/Web/EMIS/Modules -MEMIS::Database
-le ' ... '
$s = do { $d=...; ...; $d } -> prepare_cached( join q(),<STDIN>)
( $s = ... ) -> excute ( @ARGV );
$, = qq( t );
print STDERR @{$s->{NAME}};
print @r while @r=$s->fetchrow_array; </li></ul>
emp #!/usr/bin/perl -l use  DBI; $s  =  do  { DBI-> connect ( q(DBI:mysql:database=employees) ,  q() ,  q() ); }->prepare_...
Work Horse But I want query more than one DB ... Can you send me another column? Make a comma seperated list in one column...
Report Driven Report <ul><li>binding variables no longer in @ARGV
SQL no longer in STDIN
Here comes loop </li></ul>
Running faster by prepare <ul><li>$s=$d->prepare_cached(do { open(SQL, q(<), shift); join(q(),<SQL>) });
$s->execute(@F);
@r=$s->fetchrow_array;
@x = map {q()} @r unless @x;
IO::Handle::autoflush STDOUT 1; </li></ul>
Upcoming SlideShare
Loading in …5
×

perl usage at database applications

1,057 views
927 views

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,057
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
19
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

perl usage at database applications

  1. 1. Perl DBI usage at report building Joe Jiang [email_address]
  2. 2. Perl Language Practical Extraction and ... Reporting Language
  3. 3. Perl Programmer Laziness Impatience Hubris (Boldness)
  4. 4. DBMS Database Management (descide) System
  5. 5. DBI.pm The gate to Management level programming
  6. 6. Back to reporting The man in breau The game they played The suggestion I once made on OLAP
  7. 7. Chemical Perl (Gas-ware) PHP for website Python for architecture Ruby for rails Perl for what?
  8. 8. Reports again DMBS is the container SQL is for query Perl is the formater
  9. 9. Give me Excel Copy HTML table into Excel Or, name HTML as .xls Use some IDE tool?
  10. 10. Toolset basic $ echo 'select SESSION_ID, HIT_ID from hit where USERNAME=? and ACCESS_TIME between to_date(?,?) and to_date(?,?)' > session.sql ↵
  11. 11. It worked $ bofc < session.sql user-123 201106122345 YYYYMMDDHH24MI 201106122357 YYYYMMDDHH24MI | wc -l ↵ SESSION_ID HIT_ID 12
  12. 12. And ... $ bofc < session.sql user-123 201106122345 YYYYMMDDHH24MI 201106122357 YYYYMMDDHH24MI | sort ↵ SESSION_ID HIT_ID 20110612234046 1772475805 20110612234046 1772475807 20110612234046 1772475809 20110612234046 1772475811 20110612234046 1772475813 ...
  13. 13. Highlights <ul><li>works with wc -l & sort (Laziness)
  14. 14. works with parameter binding (Boldness)
  15. 15. purify the SQL with limited punctuaction (Impatience) </li></ul>
  16. 16. Camel coding PERL5LIB=/proj/Web/EMIS/Modules perl -MEMIS::Database -le '($s= do{$d=EMIS::Database::gethandle(q(CDB)); $d->do(q(begin; nls_format_setting; end;)); $d}->prepare_cached(join q(),<STDIN>))->execute(@ARGV); $,=qq(t); print STDERR @{$s->{NAME}}; print @r while @r=$s->fetchrow_array; $d->disconnect' $*
  17. 17. Hints <ul><li>PERL5LIB=/proj/Web/EMIS/Modules -MEMIS::Database
  18. 18. -le ' ... '
  19. 19. $s = do { $d=...; ...; $d } -> prepare_cached( join q(),<STDIN>)
  20. 20. ( $s = ... ) -> excute ( @ARGV );
  21. 21. $, = qq( t );
  22. 22. print STDERR @{$s->{NAME}};
  23. 23. print @r while @r=$s->fetchrow_array; </li></ul>
  24. 24. emp #!/usr/bin/perl -l use DBI; $s = do { DBI-> connect ( q(DBI:mysql:database=employees) , q() , q() ); }->prepare_cached( join ( q() ,<>) ); $s->execute ( @ARGV ); $, = qq( t ) ; print STDERR @{$s->{ NAME }} ; print @r while @r = $s->fetchrow_array ;
  25. 25. Work Horse But I want query more than one DB ... Can you send me another column? Make a comma seperated list in one column ...
  26. 26. Report Driven Report <ul><li>binding variables no longer in @ARGV
  27. 27. SQL no longer in STDIN
  28. 28. Here comes loop </li></ul>
  29. 29. Running faster by prepare <ul><li>$s=$d->prepare_cached(do { open(SQL, q(<), shift); join(q(),<SQL>) });
  30. 30. $s->execute(@F);
  31. 31. @r=$s->fetchrow_array;
  32. 32. @x = map {q()} @r unless @x;
  33. 33. IO::Handle::autoflush STDOUT 1; </li></ul>
  34. 34. emp-for #!/usr/bin/perl -wlan use DBI; use IO::Handle; BEGIN { $s = do { DBI-> connect ( q(DBI:mysql:database=employees) , q() , q() ); }->prepare_cached( do { open ( SQL , q(<) , shift ); join ( q() , <SQL> ) }); autoflush STDOUT 1 ; $, = qq( t ) ; } END { $s->finish ; $d->disconnect ; close SQL ; } $s->execute ( @F ); if ( @r = $s->fetchrow_array ) { print @r ; @x = map { q() } @r unless @x } else { print @x }
  35. 35. How to list last deptartment? $ echo 'select emp_no from employees limit 3'|emp 2>/dev/null ↵ 10001 10002 10003 $ echo 'select emp_no from employees limit 3'|emp 2>/dev/null| emp-for emp_dept.sql ↵ d005 d007 d004
  36. 36. Highlights <ul><li>First time in Perl script
  37. 37. xargs for columns
  38. 38. Easier than outer join
  39. 39. Update? It's easy </li></ul>
  40. 40. Demo data replace bofc < x.sql | tee x.bak | do-bofc-for.pl y.sql <ul><li>Backup and update in one line
  41. 41. Avoid the &quot;no where&quot; update fault
  42. 42. Save a lot of coding with perl -lan
  43. 43. Or compose the code for debugging purpose </li></ul>
  44. 44. Extensions Merge Rotate Wrap
  45. 45. merge-tab <ul><li>@f= map { IO::File->new($_) } @ARGV;
  46. 46. map { $_->getline } @f
  47. 47. join (qq(t), map { m{(.*)} && $1 } ...
  48. 48. print ... until m{^t+$} </li></ul>
  49. 49. row2col <ul><li>perl -MList::MoreUtils=each_arrayref
  50. 50. push @lines, [@F];
  51. 51. END{ ... }
  52. 52. $ea=each_arrayref(@lines);
  53. 53. print join qq(t), @x while @x=$ea->(); </li></ul>
  54. 54. Demo usage How to describe a wide table? With sample data attached?
  55. 55. Columns $ echo 'select DATA_TYPE, CHARACTER_MAXIMUM_LENGTH from information_schema.columns where table_name=?'| emp employees | tee emp-desc ↵ DATA_TYPE CHARACTER_MAXIMUM_LENGTH int date varchar 14 varchar 16 enum 1 date
  56. 56. Demo Row $ echo 'select * from employees limit 1' | emp 2>&1 | row2col | tee emp-data ↵ emp_no 10001 birth_date 1953-09-02 first_name Georgi last_name Facello gender M hire_date 1986-06-26
  57. 57. And now? Run merge-tab over the two files $ merge-tab emp.data emp.desc ↵ emp_no 10001 int birth_date 1953-09-02 date first_name Georgi varchar 14 last_name Facello varchar 16 gender M enum 1 hire_date 1986-06-26 date
  58. 58. 2excel & 2csv <ul><li>perl -MSpreadsheet::WriteExcel
  59. 59. $s->write_unicode($.-1, $_, encode q(utf16), decode q(utf8), $F[$_])
  60. 60. perl -MText::CSV_XS </li></ul>
  61. 61. 2zip <ul><li>BEGIN { $z=Archive::Zip->new()}
  62. 62. $z->addString($_, $F[0])->desiredCompressionMethod(8);
  63. 63. END { $z->writeToFileHandle(STDOUT) } </li></ul>
  64. 64. Thanks Any Question? http://weibo.com/saved2serve

×