Maxym Kharchenko presented ways to manage Oracle databases with Python. He demonstrated a Python tool to ping multiple Oracle databases concurrently and time the execution. The tool reports the status and timing for each database pinged. Python enforces good coding practices and interfaces well with databases, APIs, and other systems. Learning Python helps develop a more Pythonic way of thinking that can improve code quality and productivity.

1. Maxym Kharchenko
Manage ORACLE databases
with Python

2. Whoami
■ Started as a database kernel developer with C and C++
■ ORACLE DBA for ~ 14 years: + SQL, PL/SQL and Perl
■ Persistence Engineer for the last 4: + Python, R and Scala
■ OCM, ORACLE Ace Associate
■ Blog: http://intermediatesql.com
■ Twitter: @maxymkh

3. Agenda
■ Talk about why Python is awesome
■ Design ORACLE multi db “ping” in Python
■ (hopefully) Demo it

4. The job of an engineer is
to make complex things simple

5. Why script outside the database ?

6. Do I have a good datafilebackup ?
SELECT …
FROM v$backup_datafile
 v$backup_datafile
 Backup files actually exist
 and can be validated
 Backup size is plausible
 On the end storage
 Network transfer as well
 No critical errors in logs
 Etc

7. Some things are
just difficult to do in a database

8. You will, generally, get better Zen
i := i + 1 i += 1

9. You will, generally, get better Zen

10. You will, generally, get better Zen

11. You, probably, have >1 database
ORACLE ORACLE
ORACLE ORACLE ORACLE
ORACLE ORACLE ORACLE
ORACLE MySql MySql
Postgres Cassandra
S3 S3

12. So, why Python ?

13. There should be one way to do it
And it should be obvious

14. Python enforces good coding practices

15. Python enforces good coding practices
foreach my $p (keys %$p1) {if(exists($p1->{$p})
and exists($p2->{$p})) {
if(uc($p1->{$p}) eq uc($p2->{$p})) {$diff-
>{SAME}->{$p} = 1;
} else {$diff->{DIFF}->{$p} = 1;
}}} elsif(! exists($p2->{$p})) {$diff-
>{ONLY_IN_1}->{$p} = 1;}}

16. Python enforces good coding practices
for p in db1_params:
if p in db2_params:
if db1_params[p] == db2_params[p]:
same_parameters.append(p)
else:
diff_parameters.append(p)
else:
only_in_1.append(p)

17. Python should come
with all the batteries included

18. Python interfaces to everything
import urllib
import json
url =
"http://api.openweathermap.org/data/2.5/weather?q=Seattle
,WA"
url_obj = urllib.urlopen(url)
data = url_obj.read().strip()
data = json.loads(data)
print(data['weather'])

19. Python interfaces to everything
import cx_Oracle
sql = "SELECT user FROM dual"
conn = cx_Oracle.connect('scott/tiger@orcl')
cursor = conn.cursor()
cursor.execute(sql)
db_user = cursor.fetchall()

20. Python interfaces to everything
import boto
from boto.s3.key import Key
s3 = boto.connect_s3()
bucket = s3.get_bucket('my_cats')
k.key = 'lazy_cat.jpg'
k.set_contents_from_filename('/tmp/lazy_cat.jpg')

21. Python is pretty popular

22. Python is pretty popular

23. If you know any scripting language
you (almost) know Python

24. Python is similar to other languages
def is_accessible(db_name):
""" Check if database is accessible """
ret = False
db_status = ping_db(db_name)
if "ACTIVE" == db_status:
ret = True
return ret

25. But: Spaces are first class citizens
def print_databases():
""" Print all databases from /etc/oratab """
with open("/etc/oratab") as file:
for line in file:
if line:
print line.strip().split(':')[0]

26. But: Spaces are first class citizens
def print_databases():
""" Print all databases from /etc/oratab ""“
with open("/etc/oratab") as file:
for line in file:
if line:
print line.strip().split(':')[0]
File "./a.py", line 7
print line.strip().split(':')[0]
^
IndentationError:
expected an indented block

27. Compare to Perl
sub print_databases() {
open(my $f, "<", "/etc/oratab")
or die ("Unable to open: $!");
while(my $line = <$f>) {
if ($line =~ /S+/) {
my @aItems = split(':', $line);
print $aItems[0] . "n";
}
}
close($f);
}

28. Compare to Perl
sub print_databases(){open(my $f, "<", "/etc/oratab") or
die ("Unable to open: $!"); while(my $line = <$f>) {
if ($line =~ /S+/) {my @aItems = split(':', $line);
print $aItems[0] . "n";}} close($f);}

29. Functions are first class citizens too

30. Fun with functions
def outer_function(parameter_function):
""" This function accepts function as a parameter """
def inner_function(inner_parameter):
""" This is a nested function """
return inner_parameter
# This returns a function from a function
return inner_function(parameter_function)
# This "calls" function return value as a function
print outer_function(external_function)()

31. Fun with functions: Decorators
def do_stuff():
result = heavy_processing()
def do_stuff():
start = time()
result = heavy_processing()
end = time()
print "Elapsed: %f" % (end-start)
def do_stuff2():
…
def do_stuff3076():

32. Fun with functions: Decorators
def timeit(func):
""" Generic time profiling function """
def time_exec(*args, **kwargs):
start_time = time()
ret = func(*args, **kwargs)
end = time()
print "Elapsed: %f" % (end-start)
return ret
return time_exec

33. Fun with functions: Decorators
do_stuff = timeit(do_stuff)
@timeit
def do_stuff():
…
@timeit
def do_stuff2():
…
@timeit
def do_stuff3076():
…

34. Learn to think Pythonian. It helps!
def print_databases():
file = open('/etc/oratab', 'r')
while True:
line = file.readline()
if len(line) == 0 and not line.endswith('n'):
break
print line.strip().split(':')[0]
file.close()
def print_databases():
with open('/etc/oratab') as file:
for line in file:
print line.strip().split(':')[0]

35. Python and ORACLE

36. Database “multi ping” tool
■ Ping ORACLE database
▪ Report AVAILABLE / NOT AVAILABLE status
■ Option to ping multiple databases
▪ Preferably in parallel
■ Bonus: track execution timing

37. Demo

38. cx_Oracle: Running SQL
import cx_Oracle
def is_db_alive(db_name):
is_alive = False
try:
conn = cx_Oracle.connect("user/password@%s" % db_name)
cursor = conn.cursor()
cursor.execute("SELECT user FROM dual")
except:
is_alive = False
else:
is_alive = True
return is_alive

39. Database ping
> dbping.py c15lv1
PING [c15lv1]: OK
> dbping.py c15lv2
PING [c15lv2]: UNAVAILABLE

40. Database “multi ping”
import dbping
def multi_dbping(db_list, ping_routine):
""" Ping all databases in a list """
for db in db_list:
ping_routine(db)
> dbping_list.py c15lv1 c15lv2 c15lv3
PING [c15lv1]: OK
PING [c15lv2]: UNAVAILABLE
PING [c15lv3]: OK

41. Parallel database “multi ping”
import multiprocessing
def parallel_ping(db_list, target=dbping.print_dbping):
""" Ping db_list databases in parallel """
jobs = []
for d in db_list:
p = multiprocessing.Process(
target=target, args=(d,)
)
jobs.append(p)
p.start()
for p in jobs:
p.join()

42. Parallel database “multi ping”
> dbping_parallel.py c15lv1 c15lv2 c15lv3 c15lv4
PING [c15lv1]: OK
PING [c15lv3]: OK
PING [c15lv4]: OK
PING [c15lv2]: UNAVAILABLE

43. Decorator: Execution Timing
def timeit(func):
""" Generic time profiling function """
def time_exec(*args, **kwargs):
start_time = time()
ret = func(*args, **kwargs)
ela = time() - start_time
print “tElapsed: %.3f seconds" % ela
return ret
return time_exec

44. Execution Timing:
> dbping_parallel_timing.py c15lv1 c15lv2 c15lv3
PING [c15lv3]: OK
Elapsed: 1.186 seconds
PING [c15lv1]: OK
Elapsed: 2.309 seconds
PING [c15lv2]: UNAVAILABLE
Elapsed: 22.511 seconds
@timeit
def print_dbping_with_timing(db_name):
return dbping.print_dbping(db_name)

45. How to start with Python
■ Lots of free resources on the web
▪ Tutorials
▪ Documentation
▪ Stackoverflow.com
▪ “Play” environments
▪ Even books
■ Python self documentation:
▪ dir()
▪ help()

46. Thank you!