The document provides an overview of using Python in conjunction with Oracle databases for data management and manipulation, highlighting features of Python and how to interact with Oracle through installation and coding examples. It discusses libraries like NumPy and Pandas for data analysis and manipulation, as well as methods for querying and calling PL/SQL procedures. Additionally, the document touches upon deploying resources in Oracle Cloud Infrastructure and integrates practical coding demonstrations.

1. Python and Oracle: Allies for Best
of Data Management
Laurent Léturgez – Apr 2019
Oracle Meetup - Lille

2. Whoami
• Oracle Consultant since 2001
• Former developer
• Hadoop, Data Science and other cool topics
• Owner@Premiseo: Data Management on Premises and in the
Cloud
• Blogger since 2004
• http://laurent-leturgez.com
• Twitter : @lleturgez

3. 3 Membership Tiers
• Oracle ACE Director
• Oracle ACE
• Oracle ACE Associate
bit.ly/OracleACEProgram
500+ Technical Experts
Helping Peers Globally
Connect:
Nominate yourself or someone you know: acenomination.oracle.com
@oracleace
Facebook.com/oracleaces
oracle-ace_ww@oracle.com

4. What’s on the menu ?
• Python … short introduction
• How to interact with an Oracle Database ?
• Oracle Data Manipulation inside a Python program
• Numpy arrays
• Pandas datasets
• Python and Oracle … in real life
• Deploying Resource into the Oracle Cloud Infrastructure
• Data Visualization of performance data
• How to inject Oracle data into machine learning program with Python

5. Poll
• Who has already heard about Python ?
• Who has already « played » with Python ?
• Who has already used Python in real-life projects
(Production) ?

6. A short introduction
• Python was created in the late 1980s by Guido van Rossum (NL)
• Python is an
• Interpreted
• High-level
• General purpose
Language processing
• Python’s name comes from Monty Python’s

7. A short introduction
• Python’s features
• Multi-paradigm programming language
• Dynamically typed
• Structured and Object Oriented programming
• Functional programming
• Aspect Oriented programming
• Logic Programming (via extension)
• Code readability oriented (Beautiful is better than Ugly)
• Capable of Exception handling
• BSD Like License

8. A short introduction
• Python’s features
• Multi-paradigm programming language
• Dynamically typed
• Structured and Object Oriented programming
• Functional programming
• Aspect Oriented programming
• Logic Programming (via extension)
• Code readability oriented (Beautiful is better than Ugly)
• Capable of Exception handling
• BSD Like License

9. A short introduction
Text Processing Database DataViz
Machine Learning GUI development
Scientific
Web Servers
Image
processing
Security/Hacking
Cloud / Automation Cluster Computing

10. A short introduction
• Dynamically typed
>>> v=2
>>> print(type(v))
<class 'int'>
>>> v=2.1234
>>> print(type(v))
<class 'float'>
>>> v='hello'
>>> print(type(v))
<class 'str'>
>>> n=[1,2,3,4,5]
>>> print(type(n))
<class 'list'>
>>> n=(1,2,3,4,5)
>>> print(type(n))
<class 'tuple'>
>>> laurent = Dba(name="Laurent",age=40,skills='Good')
>>> print(type(laurent))
<class '__main__.Dba'>

11. A short introduction
• Structured / Object Oriented programming support
class Dba:
def __init__(self, name, age, skills):
# Default parameters are public by default
self.name=name
self.age=age
# Private class parameters (same from methods)
self.__skills=skills
def setAge(self,age):
self.age=age;
def getAge(self):
return self.age
def getSkills(self):
return self.__skills
def setSkills(self,skills):
self.__skills=skills
if __name__ == '__main__':
laurent = Dba(name="Laurent",age=40,skills='Good')
print ("Laurent's age: ",laurent.getAge())
laurent.setAge(41)
print ("After his birthday: " , laurent.getAge ())
laurent.age=42
print ("But wait ... : ", laurent.age)
#print("Laurent's skills:", laurent.__skills) << ERROR
print ("Laurent's skills:", laurent.getSkills())
laurent's type = <class '__main__.Dba'>
Laurent's age: 40
After his birthday: 41
But wait ... : 42
Laurent's skills: Good

12. A short introduction
• Functional Programming Support
• Principle : Code everything as a function
• Example : map a function directly on an array
def addOne(v):
return v+1
if __name__ == '__main__':
lst=[1,2,3,4,5]
#crap
for i in range(len(lst)):
lst[i]=addOne(lst[i])
print(l)
# Clean ... we map addOne function on each value of the array
print(list(map(addOne, lst)))
# Better ... same with a lambda (anonymous function)
print(list(map(lambda x:x+1,lst)))
[2, 3, 4, 5, 6]
[3, 4, 5, 6, 7]
[3, 4, 5, 6, 7]

13. A short introduction
• Code Readability
• No semi-colon
• No brackets
• Indentation usage
• Same indentation means same logical block / function or class definition
/* C – C++ - Java */
int square(int x) {
return x*x;
}
int result = 0;
for(int i=0; i<100; i++){
result += square(i);
}
# Python code
def square(x):
return x*x
result = 0
for i in range(100):
result += square(i)

14. A short introduction
• Exception Handling def divide(x, y):
try:
result = x / y
except ZeroDivisionError:
print("division by zero!")
else:
print("result is", result)
finally:
print("Bye bye ...")
if __name__ == '__main__':
divide(4,2)
divide(4,0)
result is 2.0
Bye bye ...
division by zero!
Bye bye ...
• Using user Exception is possible by :
• Creating your own exception Class
• Your Class should be derived from the Exception class
>>> print(2/0)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ZeroDivisionError: division by zero

15. A short introduction
• Installation and usage
• Download and Install Python 2 or 3 distribution
• https://www.python.org
• https://conda.io
• Create virtual environment
• Virtual Environment allows you to manage separate packages installation
• Multiple projects enablement

16. A short introduction
• Multiple environment with virtualenv
$ pip install --user virtualenv # installs virtualenv is the local user context
$ python -m virtualenv testenv # creates a new virtual environment named testenv
New python executable in /Users/leturgezl/testenv/bin/python2.7
Also creating executable in /Users/leturgezl/testenv/bin/python
Installing setuptools, pip, wheel...
done.
$ source testenv/bin/activate # Activates testenv’s virtual environment
(testenv) $ which python
/Users/leturgezl/testenv/bin/python
(testenv) $ pip install numpy pandas requests # Installs packages into testenv environment
(testenv) $ deactivate # deactivate testenv environment
$ which python
/usr/local/bin/python

17. A short introduction
• Multiple environment with anaconda (MiniConda)
• Install MiniConda : https://conda.io/projects/conda/en/latest/user-guide/install/index.html#regular-installation
$ conda env list # lists environments
# conda environments:
#
base * /Users/leturgezl/miniconda3
jupyter /Users/leturgezl/miniconda3/envs/jupyter
$ conda create --name testenv # creates a new virtual environment in Conda
$ conda env list
# conda environments:
#
base * /Users/leturgezl/miniconda3
jupyter /Users/leturgezl/miniconda3/envs/jupytermachinelearning
testenv /Users/leturgezl/miniconda3/envs/testenv
$ conda activate testenv # activates a virtual environment
(testenv)$ conda install numpy pandas # Installs packages and dependencies into the activated environment
(testenv)$ conda list # Lists all installed packages
(testenv)$ which python
/Users/leturgezl/miniconda3/envs/testenv/bin/python
(testenv)$ conda deactivate # deactivates the current environment
$

18. A short introduction
• Available IDEs
• General Editor / IDE with Python’s support
• Eclipse + PyDev
• Atom
• Vi / Vim / GNU Emacs
• Visual Studio
• Python Specific Editor / IDE
• PyCharm (JetBrain)
• Spyder

19. How to interact with an Oracle
Database ?

20. How to interact with an Oracle Database ?
1. Install an Oracle Instant Client
2. Install cx_oracle (pip or conda)
3. Code in Python

21. How to interact with an Oracle Database ?
• Create a connection to an Oracle DB
import cx_Oracle as cx
import os
oh="/Users/leturgezl/Applications/instantclient_12_2"
os.environ["ORACLE_HOME"]=oh
os.environ["PATH"]=oh+os.pathsep+os.environ["PATH"]
os.environ["NLS_LANG"]="AMERICAN_AMERICA.AL32UTF8"
def connectToOracle(url, username, password, mode=None):
if mode is not None:
connection = cx.Connection (user=username,
password=password,
dsn=url,
mode=mode)
else:
connection = cx.Connection (user=username,
password=password,
dsn=url)
return connection
# main
if __name__ == '__main__':
c=cx.Connection
try:
c=connectToOracle(url="192.168.99.3:1521/orcl",
username="sys",
password="oracle",
mode=cx.SYSDBA)
except cx.DatabaseError as ex:
err, =ex.args
print("Error code = ",err.code)
print("Error Message = ",err.message)
os.exit(1)
else:
print("Data Source Name = ", c.dsn)
a = "true" if c.autocommit == 1 else "False"
print("Autocommit = ", a)
print("Session Edition = ", c.edition)
print("Encoding = ", c.encoding)
print("National Encoding = ", c.nencoding)
print("Logical Tx Id = ", c.ltxid)
print("Server version = ", c.version)
finally:
c.close()

22. How to interact with an Oracle Database ?
• Querying an Oracle DB
type(r) = <class 'list'> -- r= [('log_archive_dest_1', None)]
type(n) = <class 'numpy.ndarray'> -- n= [['log_archive_dest_1' None]]
def executeStmt(conn, stmt, parameters):
if conn is not None and isinstance (conn, cx_Oracle.Connection):
cur = conn.cursor()
if parameters is None:
cur.execute (stmt)
else:
cur.execute(stmt,parameters)
return cur
# main
if __name__ == '__main__':
c=cx_Oracle.Connection
stmt="select name,value from v$parameter where name = :param"
try:
c=connectToOracle(url="192.168.99.3:1521/orcl" username="sys", password="oracle",mode=cx.SYSDBA)
p = {'param': "log_archive_dest_1"}
curs=executeStmt(conn=c,stmt=stmt,parameters=p)
r = curs.fetchall()
print("type(r) = ",type(r)," -- r=",r)
n = np.array (r)
print("type(n) = ",type(n)," -- n=", n)
…/…

23. How to interact with an Oracle Database ?
• Calling PL/SQL
try:
c=connectToOracle("192.168.99.3:1521/orcl","laurent","laurent")
# CREATE OR REPLACE PROCEDURE GENERAL_SAL_INCREASE (pct in number default 0)
curs=c.cursor();
curs.callproc("GENERAL_SAL_INCREASE",[0])
# CREATE OR REPLACE PROCEDURE GENERAL_SAL_INCREASE_OUT (pct in number default 0, avg_sal out number) is
curs = c.cursor ();
avgsal = curs.var(cx_Oracle.NUMBER)
curs.callproc ("GENERAL_SAL_INCREASE_OUT", [ 5, avgsal ])
# CREATE OR REPLACE FUNCTION GET_SAL (fname in varchar2, lname in varchar2) return number
curs = c.cursor ();
janette_king_sal = curs.callfunc("GET_SAL", cx_Oracle.NUMBER, ["Janette","King"]);
print("janette king sal", janette_king_sal)
curs.close()
except cx_Oracle.DatabaseError as ex:
err, = ex.args
print("Error code = ",err.code)
print("Error Message = ",err.message)
os.exit(1)
finally:
c.close()

24. How to interact with an Oracle Database ?
• And much more …
• Connection Pooling
• LOBs
• Queues
• Types
• Exceptions
https://cx-oracle.readthedocs.io/en/latest/

25. Oracle Data Manipulation inside a
Python program

26. Oracle Data Manipulation inside a Python program
• Numpy
• Fundamental package for scientific computing / analysis
• Support for large, multi-dimensional arrays and matrices
• Collection of high level mathematical functions
• Usually, numpy ndarrays are the input of all python’s image processing
• With an Oracle Database
• Suitable for processing numerical data (Statistics, transformation etc.)
• Limitations
• Not very suitable for appending/inserting new data / new columns
• No relational operators nor filtering operations
• No label for columns
https://docs.scipy.org/doc/

27. Oracle Data Manipulation inside a Python program
• Pandas
• Fundamental package for time series and Data Analysis
• Provides Data Structure close to a relational table (DataFrame)
• Read/Write data from memory to data structures / file format
• Relational operation possible
• Data merging, joining, and filtering possible
https://pandas.pydata.org/pandas-docs/stable/

28. Oracle Data Manipulation inside a Python program
• Numpy to pandas conversion possible
import pandas as pd
import numpy as np
nparray = np.array([['Pieter', 1, 2, 3, 4], ['Philip', 5.1, 5.2, 5.3, 5.4], ['Laurent', 6, 7, 8, 9]])
print(nparray)
df = pd.DataFrame(nparray, index=['ix1', 'ix2', 'ix3'], columns=['A', 'B', 'C', 'D', 'E'])
print(df)
[['Pieter' '1' '2' '3' '4']
['Philip' '5.1' '5.2' '5.3' '5.4']
['Laurent' '6' '7' '8' '9']]
A B C D E
ix1 Pieter 1 2 3 4
ix2 Philip 5.1 5.2 5.3 5.4
ix3 Laurent 6 7 8 9

29. Oracle Data Manipulation inside a Python program
• Pandas to numpy conversion possible
df = pd.DataFrame(np.random.randn(6, 4),index=list('abcdef'),columns=list('ABCD'))
print(df)
print(type(df.values))
print(df.values)
print(df['B'].values)
A B C D
a -0.204867 0.984730 0.244504 -0.095993
b 0.766063 -1.310598 0.576282 -2.035271
c -1.823711 1.653540 -0.255029 -0.321602
d 0.123362 -0.568470 -0.065062 -0.154888
e 0.104442 0.994248 -0.501259 0.951386
f 1.704871 0.875581 -0.421238 1.126849
<class 'numpy.ndarray'>
[[-0.20486668 0.98472986 0.24450356 -0.09599315]
[ 0.76606293 -1.31059791 0.57628166 -2.03527139]
[-1.82371056 1.6535402 -0.25502909 -0.32160212]
[ 0.12336172 -0.56846958 -0.06506246 -0.15488848]
[ 0.10444187 0.99424804 -0.50125872 0.95138638]
[ 1.70487111 0.87558073 -0.42123793 1.12684927]]
[ 0.98472986 -1.31059791 1.6535402 -0.56846958 0.99424804 0.87558073]

30. Oracle Data Manipulation inside a Python program
• Reading from Oracle to numpy array
• Get a list from a fetch (or fetchall) on a cx_Oracle cursor object
• Convert the list object to a numpy ndarray object
def executeStmt(conn, stmt, parameters):
if conn is not None and isinstance (conn, cx_Oracle.Connection):
cur = conn.cursor()
if parameters is None:
cur.execute (stmt)
else:
cur.execute(stmt,parameters)
return cur
# main
if __name__ == '__main__':
c=cx_Oracle.Connection
stmt="select name,value from v$parameter where name = :param"
try:
c=connectToOracle(url="192.168.99.3:1521/orcl" username="sys",
password="oracle",mode=cx.SYSDBA)
p = {'param': "log_archive_dest_1"}
curs=executeStmt(conn=c,stmt=stmt,parameters=p)
r = curs.fetchall()
n = np.array(r)
…/…

31. Oracle Data Manipulation inside a Python program
• Reading from Oracle to pandas dataframe
• Open a connection to Oracle by using a cx_Oracle Connection object
• Use read_sql method from a pandas Object
import pandas as pd
import cx_Oracle
try:
connection = cx_Oracle.Connection (user="sys",
password="oracle",
dsn="192.168.99.3:1521/orcl",
mode=cx_Oracle.SYSDBA)
p = {'param': "log_archive_dest_1"}
dataframe=pd.read_sql(stmt,con=c,params=p)
print(dataframe)
except cx_Oracle.DatabaseError as ex:
…/…

32. Python and Oracle … in real life
Deploying Resource into the Oracle Cloud Infrastructure

33. Deploying Resource into the Oracle Cloud Infrastructure
• Required package
• oci
• Local configuration
• User OCID: this can be found
in the User’s Page in OCI
• Tenancy OCID: this can be
found in the Tenancy’s page in
OCI
• Your OCI region
• Keys: A private key file, its
public key, and the related
fingerprint.
import oci

34. Deploying Resource into the Oracle Cloud Infrastructure
• A configuration file
• Loaded with Python
$ cat ~/.oci/config
[DEFAULT]
user=ocid1.user.oc1..aaaaaaaamcel7xygkvhe....aaaaaaaaaaaaaaaaaaaaa
fingerprint=35:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa
key_file=~/.oci/oci_api_key.pem
tenancy=ocid1.tenancy.oc1..aaaaaaaahgagkf7xygkvhe....aaaaaaaaaaaaaaaaaaaaa
region=eu-frankfurt-1
>>> import oci
>>> import pandas as pd
>>> config=oci.config.from_file()
>>> df=pd.DataFrame.from_dict(config, orient='index')
>>> df
0
log_requests False
additional_user_agent
pass_phrase None
user ocid1.user.oc1..aaaaaaaamcel7xygkvhe...
fingerprint 35:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa
key_file ~/.oci/oci_api_key.pem
tenancy ocid1.tenancy.oc1..aaaaaaaahgagkf7xygkvhe...
region eu-frankfurt-1
config=oci.config.from_file(profile_name="laurent")
config=oci.config.from_file(file_location="~/OCI_cfg.uat")
• Alternatives

35. Deploying Resource into the Oracle Cloud Infrastructure
• A configuration embedded in the code
config = {
"user": "ocid1.user.oc1..aaaaaaaamcel7xygkvhe....aaaaaaaaaaaaaaaaaaaaa" ,
"key_file": "~/.oci/oci_api_key.pem",
"fingerprint": "35:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa",
"tenancy": "ocid1.tenancy.oc1..aaaaaaaahgagkf7xygkvhe....aaaaaaaaaaaaaaaaaaaaa",
"region": "eu-frankfurt-1"
}

36. Deploying Resource into the Oracle Cloud Infrastructure
• Example : Object Storage bucket creation
• Full Python SDK API available
https://oracle-cloud-infrastructure-python-sdk.readthedocs.io
import oci
from oci.object_storage.models import CreateBucketDetails
object_storage = oci.object_storage.ObjectStorageClient(config)
compartment_id = config["tenancy"]
namespace = object_storage.get_namespace().data
bucket_name = "python-bucket"
print("Creating a new bucket {!r} in compartment {!r}".format(bucket_name, compartment_id))
request = CreateBucketDetails()
request.compartment_id = compartment_id
request.name = bucket_name
bucket = object_storage.create_bucket(namespace, request)

37. Deploying Resource into the Oracle Cloud Infrastructure
• Example : Object Storage bucket creation

38. Deploying Resource into the Oracle Cloud Infrastructure
• Example : Put a binary file inside the previously created bucket
object_name = "python_file"
with open("images/myimage.png", mode='rb') as file:
my_data = file.read()
obj = object_storage.put_object(
namespace,
bucket_name,
object_name,
my_data)

39. Python and Oracle … in real life
Visualization of performance data

40. Visualization of performance data
• What is Data Visualization (aka. Dataviz) ?

41. Visualization of performance data
• Available libraries
• matplotlib : https://matplotlib.org/index.html
• seaborn : https://seaborn.pydata.org/
• based on matplotlib and statistics oriented
• ggplot : https://pypi.org/project/ggplot/
• bokeh : https://bokeh.pydata.org/en/latest/
• D3.js style without Javascript
• Interactive
• For visualization in browser
• plotly : https://plot.ly/python/
• Online platform for data visualization
…/…

42. Visualization of performance data
• Real life case …
• Small Business Customer
• No Full Time DBA
• Need of a performance tool to quickly identify performance bottleneck
• Target slow performance
• Punctually
• Vertically ie. during a period of time : every morning between 5AM & 6AM
• Horizontally ie. during X hours on a day.
• Solution : An AWR-Based HeatMap

43. Visualization of performance data
• Extract Active Average Sessions per hour
(Thanks Marcin Przepiórowski)
• Divide AAS per number of CPU Cores 
Database Load
• DB Load ~= 0: Database is idle
• DB Load < 1, no bottleneck
• DB Load ~ 1, database uses all system
resources (If one DB / system)
• DB Load > 1, Database loaded
• DB Load >> 2 , Database is overloaded
SELECT mtime,
ROUND(SUM(load),2) LOAD
FROM
(SELECT TO_CHAR(sample_time,'YYYY-MM-DD HH24') mtime,
DECODE(session_state,'WAITING',COUNT(*),0)/360 c1,
DECODE(session_state,'ON CPU',COUNT( *),0) /360 c2,
COUNT(*)/360 cnt,
COUNT(*)/360/cpu.core_nb load
FROM dba_hist_active_sess_history,
(SELECT value AS core_nb
FROM v$osstat
WHERE stat_name='NUM_CPU_CORES’
) cpu
GROUP BY TO_CHAR(sample_time,'YYYY-MM-DD HH24'),
session_state,
cpu.core_nb
)
GROUP BY mtime
ORDER BY mtime;

44. Visualization of performance data
• Next Step … transform those ugly numbers into a nice plotting
• program / script
• Webserver
• Django
• Flask (Demo … if we have enough time ;) )
• But wait … do you know Jupyter to do that stuff ?
• JuPyTer stands for Julia, Python, and R : Not specific to Python
• Based on notebooks
• A notebook
• Integrates code and its output into a single document
• Combines visualisations, narrative text, mathematical equations etc.
• Step by Step execution
• Perfect for rapid and reusable development
• More here : https://jupyter.org/

45. Visualization of performance data
• Oracle, Python, Jupyter and DataViz … nothing worth a demo !!!

46. Python and Oracle … in real life
How to inject Oracle data into machine learning program with Python

47. Inject Oracle data into machine learning program with
Python
• The problem
• A typical competitive intelligence problem
• Competitor’s product line analysis
• Is my product line is wide enough compared to competitors?
• Specifically for « online competitors »
• The Non-Oracle stuff
• How to analyze competitor’s product line ?
• Scrap the competitor’s website to identify product of interest.
• Classify the products of interest per features.
• Running Shoes
• Per size
• Per color
• Use Convolutional Neural Networks and Clustering to classify pictures from
the data scraping process

48. Inject Oracle data into machine learning program with
Python Description: Adult shoes
Color: N/A
Size: 39-45
Description: Junior ski helmet
Color: white
Size: XS
Description: Adult running shoes
Color: N/A
Size: 42-46
Description: Wave Ultima
Color: yellow
Size: 37-46
Description: Tennis shoes
Color: N/A
Size: 42-43
Description: women swimming suit
Color: dark blue
Size: M
Description: Timer
Color: N/A
Size: N/A
DATA
SCRAPING
Etc …

49. Inject Oracle data into machine learning program with
Python
Convolutional Neural Network
(CNN)
Running shoes probability: 0.96
Running shoes probability: 0.97
Running shoes probability: 0.94
Description: Adult shoes
Color: N/A
Size: 39-45
Description: Junior ski helmet
Color: white
Size: XS
Description: Adult running shoes
Color: N/A
Size: 42-46
Description: Wave Ultima
Color: yellow
Size: 37-46
Description: Tennis shoes
Color: N/A
Size: 42-43
Description: women swimming suit
Color: dark blue
Size: M
Description: Timer
Color: N/A
Size: N/A
Running shoes probability: 0.01
Running shoes probability: 0.57
Running shoes probability: 0.02
Running shoes probability: 0.12

50. Inject Oracle data into machine learning program with
Python
Convolutional Neural Network
(CNN)
+
Color: red
Size: 39-45
Color: blue
Size: 42-46
Color: yellow
Size: 37-46
Description: Adult shoes
Color: N/A
Size: 39-45
Description: Junior ski helmet
Color: white
Size: XS
Description: Adult running shoes
Color: N/A
Size: 42-46
Description: Wave Ultima
Color: black
Size: 37-46
Description: Tennis shoes
Color: N/A
Size: 42-43
Description: women swimming suit
Color: dark blue
Size: M
Description: Timer
Color: N/A
Size: N/A Kmeans Clustering (Colors detection)

51. Inject Oracle data into machine learning program with
Python
• Competitor’s
product line
• Type : running shoes
• Represented by
color and size

52. Inject Oracle data into machine learning program with
Python
• Black, violet, blue, and
red are well
represented (magenta,
green, yellow and white
less)
• Intermediate sizes are
not as much
represented as others
• Big sizes are not really
represented

53. Inject Oracle data into machine learning program with
Python
• The Oracle stuff
• My products are stored in an Oracle DB
• Want to compare my product line to competitor
• I will use the same classifier for convinience (CNN)
• Input : Blob (Image) and varchars in DB
• CNN Input : image representation given in a numpy multidimension array (Tensor)
• Job to do
• How to convert unstructured data (Byte Stream) to structured data (numpy
multidimensional array)
• How to convert BLOB Data into numpy representation ?

54. Inject Oracle data into machine learning program with
Python
• How to convert BLOB Data into numpy representation ?
• Use of PIL package (Python Image Library)
• PIL is now deprecated
• Replaced by pillow
• Need to install pillow (pip/conda install pillow)
• But import is still based on PIL (compatibility)

55. Inject Oracle data into machine learning program with
Python
• How to convert BLOB Data into numpy representation ?
import numpy as np
from PIL import Image
from io import BytesIO
c = cx_Oracle.Connection (user="laurent",
password="laurent",
dsn="192.168.99.3:1521/orcl")
cur = c.cursor()
cur.execute('select b from bin_file where rownum=1')
c1=cur.fetchone()[0]
s=c1.read()
im=Image.open(BytesIO(s))
# imgsize=32,32
# im.thumbnail(imgsize)
A=np.array(im)
print("A=",A)
print("A.shape=",A.shape)
cur.close()
Get a connection
Fetch the first row and
get the first element of
the row
Read the cx_Oracle.BLOB
(c1) as a bytes stream
Image (PIL) object
creation from the Bytes
Stream
Optionally resize the
image
Convert the image object
to a numpy Object
A= [[[22. 15. 6. ... 62. 59. 53.]
[18. 13. 4. ... 65. 63. 58.]
[18. 14. 6. ... 59. 62. 61.]
...
[20. 19. 19. ... 66. 65. 65.]
[29. 25. 21. ... 66. 68. 69.]
[40. 29. 19. ... 66. 70. 72.]],
…/…
[243. 243. 243. ... 243. 243. 243.]]]
A.shape= (64, 64, 3)

56. Inject Oracle data into machine learning
program with Python
Competitor’s product line Your product line
No product above 46
Few product in violet
line
Few product in the
green line
No product for small and
intermediates sizes
Competitor’s PL is
more dense

57. Questions ?