Distributed Queries in IDS: New features in IDS. Keshava Murthy, Architect, Informix Software. IBM, [email_address] Session Number 2988

Introduction to distributed query Configuring IDS for distributed queries What’s new in IDS v10 and v11 Enhancements in IDS v10 Enhancements in IDS v11 Future directions and feedback

Introduction to distributed query

Configuring IDS for distributed queries

What’s new in IDS v10 and v11

Enhancements in IDS v10

Enhancements in IDS v11

Future directions and feedback

inventorydb Application Application IDS Server: order_server IDS Server: eduserver Application orderdb payrolldb edudb CROSS-Database SELECT o.ordid, i.invdesc FROM orderdb:orders o , inventorydb:inventory i Where o.customerid = 1234 and o.ordinvid = i.invid; CROSS-Database SELECT p.empid, c.course FROM payrolldb:employees e, edudb:course c WHERE p.empid = 1234 And p.courseid = c.coursid

inventorydb Application Application order_server orderdb emp_server empdb Application orderdb payrolldb edudb CROSS-SERVER SELECT p.empid, o.orderid, o.orderdesc FROM payrolldb@emp_server:employees e, orderdb@order_server:course c WHERE e.empid = 1234 And e.courseid = c.coursid

Scenarios to use distributed queries. Need to access the latest data. Centralized control Data is in different systems – be in same or different location. Data can be in multiple version of IDS or XPS Data integration Heterogeneous data base systems.

Need to access the latest data.

Centralized control

Data is in different systems – be in same or different location.

Data can be in multiple version of IDS or XPS

Data integration

Heterogeneous data base systems.

When to use distributed queries. Real time data access and modification. Reliable data access and modifications. Need full 2-phase commit support Used by the transaction manager at the co-ordinator Commit phase goes thru 2 phases 1 st phase: PREPARE to COMMIT 2 nd phase: commit if all the polled servers are OK.

Real time data access and modification.

Reliable data access and modifications.

Need full 2-phase commit support

Used by the transaction manager at the co-ordinator

Commit phase goes thru 2 phases

1 st phase: PREPARE to COMMIT

2 nd phase: commit if all the polled servers are OK.

CROSS-SERVER SELECT p.empid, o.orderid, o.orderdesc FROM payrolldb@emp_server:employees e, orderdb@order_server:course c WHERE e.empid = 1234 And e.courseid = c.coursid inventorydb order_server:orderdb orderdb server2 server3 server5 server4

sqlhosts Inventorysrv onsoctcp myinv.domaon.com 5463 orderserver onsoctcp 9.48.52.345 4526 Informix server name Network protocol IP address Port number

Inventorysrv onsoctcp myinv.domaon.com 5463

orderserver onsoctcp 9.48.52.345 4526

QUERY: ------ SELECT t.col1 FROM orderby@server1:tab1 t ORDER BY t.col2 Estimated Cost: 1 Estimated # of Rows Returned: 1 Temporary Files Required For: Order By 1) sqlqa.t: SEQUENTIAL SCAN QUERY: ------ SELECT rt1.col1 FROM orderby@server2:tab1 rt1 ORDER BY rt1.col2 Estimated Cost: 1 Estimated # of Rows Returned: 1 1) sqlqa.rt1: REMOTE PATH Remote SQL Request: select x0.col1 from orderby:"sqlqa".tab1 x0 order by x0.col2

QUERY:

------

SELECT t.col1 FROM orderby@server1:tab1 t ORDER BY t.col2

Estimated Cost: 1

Estimated # of Rows Returned: 1

Temporary Files Required For: Order By

1) sqlqa.t: SEQUENTIAL SCAN

QUERY:

------

SELECT rt1.col1 FROM orderby@server2:tab1 rt1 ORDER BY rt1.col2

Estimated Cost: 1

Estimated # of Rows Returned: 1

1) sqlqa.rt1: REMOTE PATH

Remote SQL Request:

select x0.col1 from orderby:"sqlqa".tab1 x0 order by x0.col2

QUERY: ------ select a1.t1c2, a2.t2c2, a3.t3c2, a4.t4c2 from (rdb2@server2:t1 a1 left join rdb2@server2:t2 a2 on a1.t1c1 = a2.t2c1) cross join (rdb2@server2:t3 a3 left join rdb2@server2:t4 a4 on a3.t3c1 = a4.t4c1) Estimated Cost: 7 Estimated # of Rows Returned: 11 1) sqlqa.a3, sqlqa.a4: REMOTE PATH Remote SQL Request: select x0.t3c1 ,x0.t3c2 ,x1.t4c1 ,x1.t4c2 from (rdb2:"sqlqa".t3 x0 left join rdb2:"sqlqa".t4 x1 on (x0.t3c1 = x1.t4c1 ) ) 2) sqlqa.a1, sqlqa.a2: REMOTE PATH Remote SQL Request: select x2.t1c1 ,x2.t1c2 ,x3.t2c1 ,x3.t2c2 from ((rdb2:"sqlqa".t1 x2 left join rdb2:"sqlqa".t2 x3 on (x2.t1c1 = x3.t2c1 ) )) NESTED LOOP JOIN

QUERY:

------

select a1.t1c2, a2.t2c2, a3.t3c2, a4.t4c2

from (rdb2@server2:t1 a1 left join rdb2@server2:t2 a2

on a1.t1c1 = a2.t2c1)

cross join (rdb2@server2:t3 a3 left join rdb2@server2:t4 a4

on a3.t3c1 = a4.t4c1)

Estimated Cost: 7

Estimated # of Rows Returned: 11

1) sqlqa.a3, sqlqa.a4: REMOTE PATH

Remote SQL Request:

select x0.t3c1 ,x0.t3c2 ,x1.t4c1 ,x1.t4c2 from (rdb2:"sqlqa".t3

x0 left join rdb2:"sqlqa".t4 x1 on (x0.t3c1 = x1.t4c1 ) )

2) sqlqa.a1, sqlqa.a2: REMOTE PATH

Remote SQL Request:

select x2.t1c1 ,x2.t1c2 ,x3.t2c1 ,x3.t2c2 from ((rdb2:"sqlqa".t1

x2 left join rdb2:"sqlqa".t2 x3 on (x2.t1c1 = x3.t2c1 ) ))

NESTED LOOP JOIN

Informix Dynamic Server Datatypes SET Complex Datatypes Basic SQL Types Numeric Character Time Large Objects Extended Data types Collection Row Types Multi-Set List Distinct Builtin-extended Boolean, lvarchar, blob, clob opaque User Defined Datatypes

Built-in SQL Datatypes Character Numeric LargeObjects Time Char Varchar Nchar Nvarchar Int Int8 float Decimal Numeric Money Smallint Smallfloat Serial Serial8 Bigint Bigserial datetime Interval date Text Byte

Char

Varchar

Nchar

Nvarchar

Int

Int8

float

Decimal

Numeric

Money

Smallint

Smallfloat

Serial

Serial8

Bigint

Bigserial

datetime

Interval

date

Text

Byte

Built-in Extended types Boolean LVARCHAR BLOB CLOB

Boolean

LVARCHAR

BLOB

CLOB

Cross-Database Queries – one server IDS v10 and above New Extended Data types support Built-in UDTS across databases, single server instance BOOLEAN LVARCHAR BLOB CLOB Non built-in UDTS across databases in a single server instance, with explicit cast to LVARCHAR Boolean, LVARCHAR, BLOB and CLOB are commonly used in applications.

IDS v10 and above

New Extended Data types support

Built-in UDTS across databases, single server instance

BOOLEAN

LVARCHAR

BLOB

CLOB

Non built-in UDTS across databases in a single server instance, with explicit cast to LVARCHAR

Boolean, LVARCHAR, BLOB and CLOB are commonly used in applications.

Cross-Database Queries – one server IDS v10 and above New types can be used in Direct DML Views Synonyms Stored procedure parameters and return types C and Java UDRs

IDS v10 and above

New types can be used in

Direct DML

Views

Synonyms

Stored procedure parameters and return types

C and Java UDRs

Cross Database example: SELECT DATABASE orderdb; SELECT item_num,item_desc,item_specs,item_pic,item_life FROM inventorydb:itemtab WHERE item_num = 1; SELECT item_sm_pk::LVARCHAR,item_lg_pk::varchar FROM inventorydb:itemtab WHERE item_num = 2;

DATABASE orderdb;

SELECT item_num,item_desc,item_specs,item_pic,item_life

FROM inventorydb:itemtab

WHERE item_num = 1;

SELECT item_sm_pk::LVARCHAR,item_lg_pk::varchar

FROM inventorydb:itemtab

WHERE item_num = 2;

Cross Database example: INSERT Insert on Remote table having built-in udts INSERT INTO remotedb:remtab (boolean_col, lvarchar_col, blob_col, clob_col) VALUES (‘t’, “lvarchar_data”, Filetoblob(‘blobfile’, ‘client’), Filetoclob(‘clobfile’, ‘client’)) Insert into remote table Select from local table built-in udts INSERT INTO remotedb:remtab ( boolean_col, lvarchar_col, blob_col, clob_col) SELECT bool_col, lvar_col, b_col, c_col FROM local_tab WHERE bool_col = “t”; Insert on Remote table non built-in UDTs INSERT INTO remotedb:remtab (udt1_col) SELECT udt_col::lvarchar FROM local_tab WHERE udt2_col::lvarchar = “ldata2row1”;

Insert on Remote table having built-in udts

INSERT INTO remotedb:remtab

(boolean_col, lvarchar_col, blob_col, clob_col)

VALUES (‘t’, “lvarchar_data”, Filetoblob(‘blobfile’, ‘client’),

Filetoclob(‘clobfile’, ‘client’))

Insert into remote table Select from local table built-in udts

INSERT INTO remotedb:remtab

( boolean_col, lvarchar_col, blob_col, clob_col)

SELECT bool_col, lvar_col, b_col, c_col

FROM local_tab

WHERE bool_col = “t”;

Insert on Remote table non built-in UDTs

INSERT INTO remotedb:remtab (udt1_col)

SELECT udt_col::lvarchar

FROM local_tab

WHERE udt2_col::lvarchar = “ldata2row1”;

Cross Database example: UPDATE/DELETE DATABASE orderdb; UPDATE inventorydb:itemtab SET (item_lg_pk) = ((SELECT order_lg_pk::lvarcharFROM ordertab WHERE order_sm_pk::lvarchar = "box(2,2,2,2)")) WHERE order_shelf_life = "t";

DATABASE orderdb;

UPDATE inventorydb:itemtab

SET (item_lg_pk) =

((SELECT order_lg_pk::lvarcharFROM ordertab

WHERE order_sm_pk::lvarchar = "box(2,2,2,2)"))

WHERE order_shelf_life = "t";

Cross Database example: CREATE VIEW DATABASE orderdb; CREATE VIEW item_pk_view AS SELECT item_lg_pk::lvarchar, item_sm_pk::varchar FROM inventorydb:itemtab WHERE item_sm_pk::char matches "box(2,2,2,2)*"

DATABASE orderdb;

CREATE VIEW item_pk_view AS

SELECT item_lg_pk::lvarchar,

item_sm_pk::varchar

FROM inventorydb:itemtab

WHERE item_sm_pk::char matches "box(2,2,2,2)*"

CROSS-SERVER SELECT p.empid, o.orderid, o.orderdesc FROM payrolldb@emp_server:employees e, orderdb@order_server:course c WHERE e.empid = 1234 And e.courseid = c.coursid Cross Server inventorydb order_server:orderdb orderdb server2 server3 server5 server4

Enhancements in IDS v11.10 Cross-Server queries now support Boolean Lvarchar Distinct types of Boolean and Lvarchar Distinct types of built-in SQL types Numeric (int, decimal, etc) character (char, varchar, etc) Time (datetime) No support for blob and clob yet

Cross-Server queries now support

Boolean

Lvarchar

Distinct types of Boolean and Lvarchar

Distinct types of built-in SQL types

Numeric (int, decimal, etc)

character (char, varchar, etc)

Time (datetime)

No support for blob and clob yet

Builtin-extended Boolean, lvarchar, blob, clob Informix Dynamic Server Datatypes SET Complex Datatypes Basic SQL Types Numeric Character Time Large Objects Extended Data types User Defined Datatypes Collection Row Types Multi-Set List Distinct Opaque

Application IDS Server: lenexa Database: geardb Table: winter_gear with Boolean, lvarchar and distinct types C, SPL, Java functions IDS Server: portland Database:geardb Table: rain_gear with Boolean, lvarchar, and distinct types C, SPL, Java functions IDS Server: menlo Database: geardb Table: winter_gear with Boolean, lvarchar and distinct types C, SPL, Java functions ISTAR: Distributed query connections SQL and result set exchange Multiple IDS server setup IDS Server: toronto Database:geardb C, SPL, Java functions

Connect to Menlo : CREATE TABLE sunny_gear(partner int, active Boolean, desc lvarchar(4096)); -- Select active partners at (Menlo, Lenexa and Portland) SELECT x.partner, x.desc, y.desc, z.desc FROM geardb@menlo:sunny_gear x, geardb@portland:rain_gear y, geardb@lenexa:winter_gear z WHERE x.partner = y.partner and x.partner = z.partner and x.active = ‘T’ and y.active = ‘T’ and z.active = ‘T’ ORDER BY x.partner;

Connect to Menlo :

CREATE TABLE sunny_gear(partner int, active Boolean, desc lvarchar(4096));

-- Select active partners at (Menlo, Lenexa and Portland)

SELECT x.partner, x.desc, y.desc, z.desc

FROM geardb@menlo:sunny_gear x,

geardb@portland:rain_gear y,

geardb@lenexa:winter_gear z

WHERE x.partner = y.partner and

x.partner = z.partner and

x.active = ‘T’ and

y.active = ‘T’ and

z.active = ‘T’

ORDER BY x.partner;

Enhancements to distributed Explain. Explain file explains the query plan chosen by optimizer The explain file at co-ordinator provides the session ID at the sub-ordinator If the explain is ON at the co-ordinator, subordinator automatically generates the explain file. Can be disabled by DISTR_QUERY_FLAGS 0x01 enable or disable the distributed query plan 0x02 enable the statistics in the explain file. See the online.log for the exact location of the explain file.

Explain file explains the query plan chosen by optimizer

The explain file at co-ordinator provides the session ID at the sub-ordinator

If the explain is ON at the co-ordinator, subordinator automatically generates the explain file.

Can be disabled by DISTR_QUERY_FLAGS

0x01 enable or disable the distributed query plan

0x02 enable the statistics in the explain file.

See the online.log for the exact location of the explain file.

On the coordinator QUERY: ------ SELECT rt1.col1 FROM orderby@server2:tab1 rt1 ORDER BY rt1.col2 Estimated Cost: 1 Estimated # of Rows Returned: 1 1) sqlqa.rt1: REMOTE PATH Remote Server name: server2 Remote Session id: 6 Remote SQL Request: select x0.col1 from orderby:"sqlqa".tab1 x0 order by x0.col2 On the subordinator: 09:19:34 Explain file for session 6 : usr/appusr1/sqexplain.out.6

On the coordinator

QUERY:

------

SELECT rt1.col1 FROM orderby@server2:tab1 rt1 ORDER BY rt1.col2

Estimated Cost: 1

Estimated # of Rows Returned: 1

1) sqlqa.rt1: REMOTE PATH

Remote Server name: server2

Remote Session id: 6

Remote SQL Request:

select x0.col1 from orderby:"sqlqa".tab1 x0 order by x0.col2

On the subordinator:

09:19:34 Explain file for session 6 : usr/appusr1/sqexplain.out.6

Thank You [email_address] Session: 2988A

Thank You

[email_address]

Session: 2988A