The document discusses secrets and best practices for optimizing the performance of an OLTP system. It describes how the speaker's team was able to reduce response times by 50% through focused tuning of the application to database interface. Some techniques that helped include identifying redundant database calls, reducing round trips by passing data in arrays, processing data in bulk using INSERT statements, and returning less unused data. The document provides recommendations for locking strategies, using JDBC features like arrays and batching, and setting the optimal row prefetch.

1. Tarik R. Essawi
VeriSign
Session 116
Secrets of Highly Scalable OLTP
Architectures

2. Introduction
• Provides real-world best practices for a mission-critical
OLTP system.
• Focused tuning of the application to database interface
can yield a 50% reduction in response time.
• Secrets and best practices that every developer can
use for tuning an already tuned system.

3. Background
• New SLA’s required that
we tune the system.
• Project is very large
• Waterfall development
lifecycle.
• One application engineer
and one database
engineer performed the
majority of the work.
• Originally tuned using
traditional methods.
• 75% of the time was spent
in the database.
Command SLA Response
Times
CHECK-
DOMAIN
25 milliseconds
ADD-
DOMAIN
50 milliseconds
MODIFY-
DOMAIN
100
milliseconds
DELETE-
DOMAIN
100
milliseconds

4. First Secret Discovered
Challenge: Identify where time is actually being spent.
How:
1.) Traced the database sessions.
2.) Identified what database calls were being made
and how many milliseconds each individual call took.
3.) Used Trace Analyzer from Oracle to parse the
trace files.
Result: For example, we found two business rules calling
the same operation to retrieve customer data. By
removing the redundant call we saved 8 milliseconds.
Lesson Learned: Every database call counts.

5. Second Secret Discovered
Challenge: Reduce round trips between the database and application.
• Most important item we found.
• Every database round trip involves additional overhead due to the
network latency.

6. How: Pass data in an object form using arrays.
Use Arrays
• Consider the sample Orders schema above.
• An order for a computer is a good example. A single order
contains order items such as the computer, monitor, keyboard,
and mouse.

7. Traditional Versus Array
• Traditional Method
–Call a procedure to insert an Order into the Orders table.
–Loop through the Order Items, passing them individually to a
procedure to be inserted into the Order Items table.
In the previous example a total of five database calls occur.
• Array Method
–Call a procedure to add the Order and an array of Order Items
in one procedure call.
• Internally insert the Order into the Orders table and its
associated Order Items into the Order Items table.
Using this method, five database calls are replaced by one.

8. Using Arrays
Below is an example of how to start using arrays.
1.) A schema level type needs to be created:
CREATE TYPE OrderItemsType AS object (Order_Id NUMBER(12),
Line_Item_Id NUMBER(3), ……..);
2.) Create a table of the type:
CREATE TYPE OrderItemsTab AS table OF OrderItemsType;
3.) Create a procedure that includes both the order and an array of the
order items:
CREATE PROCEDURE Add_Order_and_Items(
pOrder_Id IN Orders.Order_id%TYPE,
pOrder_Date IN Orders.Order_Date%TYPE,
……..
pOrder_Items IN OrderItemsTab);
Lesson Learned: The overhead of making database calls is expensive.

9. Third Secret Discovered
Challenge: Process Data in Bulk.
How: Forall, Bulk Collect, or the example below:
Result: Using one insert statement we were able to insert
multiple records without a looping control structure.
INSERT INTO order_items (order_id, line_item_id,….)
SELECT order_id, line_item_id,…….
FROM TABLE (CAST (pOrder_Items AS OrderItemsTab));
Lesson Learned: Processing in Bulk increases performance.

10. Fourth Secret Discovered
Challenge: Return less data.
How: Identify data not being used by the application.
Result: In our case a system generated sequence value was
returned but never used. We were able to stop returning
the value and saw a performance increase.
Lesson Learned: The process of returning even a single
value can make a difference.

11. Best Practices

12. Fastest Way to Pass Data
Identify the fastest way to pass data to the application.
• Three mainstream ways are:
– Primitive Data Types
– In/Out Parameters in PL/SQL
– Result Sets
• Primitive Data Types: fastest way to pass back a
single data value.
• In/Out Parameters: fastest way to pass back multiple
single data values.
• Result Sets: fastest way to pass back record sets but
are not best suited to returning single records.

13. NOCOPY
• By default Oracle creates temporary copies of
IN/OUT and OUT variable data within stored
procedures.
• The NOCOPY hint passes only a reference to the
object instead of the entire object.
• NOCOPY hint is added to a prodecures signature for
each IN/OUT and OUT variable.
Pro: Speeds up execution.
Con: Partial data can be returned back to the client
during error conditions.

14. Locking Strategies
• The locking strategies concentrate on updating or
deleting records.
• Inserts do not require locking because the object never
exists until the insert.
• Two main ways to lock data:
For short user or API transactions in B2B environments:
1. Lock and get the data.
2. Update or delete the data.
3. Commit the transaction.

15. Locking Strategies Cont’d
Longer user interface type transactions:
1. Get the data.
2. When the data is updated use a version identifier to
ensure that the record has not been updated by
someone else.
• Include the version identifier in the where clause of the
update statement.
3. Verify the transaction.
a. If the statement updates zero rows then the
record was already modified and the transaction
should be rolled back.
b. If the statement updates the proper number of
rows then the record updated successfully.

16. JDBC
JDBC features are constantly evolving, so become
aware of the latest features.
• JDBC Arrays.
– Tighter integration.
– Can send objects to pl/sql.
• JDBC Batching.
– Greatly increases performance.
– Reduces Round Trips.
– Client sends a group of statements in a batch.

17. JDBC Cont’d
Setting the row prefetch
• When retrieving large amounts of data setting the
setDefaultRowPrefetch() attribute in the application
code can move more data with each call.
Use the latest JDBC driver provided by the database
vendor.
• Upgraded to the Oracle 10g database driver and saw
a 10 millisecond improvement.
• New driver also had new features.

18. Lessons Learned
• Writing code the right way takes the same
amount of effort as writing it the wrong way.
• It should not be assumed that the
communication between the application and
database is as efficient as possible.
• In our experience tuning the application to
database interface yielded a 50% reduction in
response time.

19. Lessons Learned Cont’d
• When passing data use primitive data types
whenever possible. Use result sets only when
required.
• Use JDBC Array and Batching features if
possible.
• Use the fastest JDBC driver available, don’t
assume you have the fastest.

20. Final Thoughts
• Knowing the secrets and best practices listed
today will make any application more scalable
by virtue of using fewer scarce resources.
• These best practices should be incorporated
in a standard development lifecycle to ensure
your system is always operating as efficiently
as possible.

21. Questions?

22. Thank You
Please complete your evaluation forms.
Speaker Name: Tarik R. Essawi
Session name: Secrets of Highly Scalable OLTP
Architectures
Session#: 116
Contact Info: tessawi@verisign.com