ABL Code Performance Optimization Techniques

ABL Code Performance
Peter Judge
pjudge@progress.com

© 2016 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved.2
$ whoami
pjudge@progress.com
Software Architect*
@ Progress since 2003
Integration-y stuff – Authentication Gateway, HTTP-Out,
Corticon et al
OE Best Practices / OERA / AutoEdge / CCS
4GL since 1996
* Aka programmer who knows PowerPoint

compile-listing ain’t enough
 Intro
 Network effects: you can only fit so much into a pipe of a given size
 Indexes: it's easier to find things quickly when you know where you left them
 Unnecessary work: it's better to work "smarter"
 Conclusion

network effects
Shared memory DB connections make programmers look like rock stars
Network (client/server) connections are more like the
morning after – not so pretty

network effects - basics
MTU
-prefetch*
-Mm
-Mm
-Mm
-Mm
-Mm
-Mm
-Mm

network effects - basics
 Field lists: fields of record that are sent to the client
 Prefetch: multiple records per message (NO-LOCK queries)
 Network message buffer size (-Mm db startup param)
 MTU (maximum transmission unit): the largest packet size that can be transmitted over
a network not an OpenEdge parameter
FOR EACH Customer FIELDS (Name Balance)
FOR EACH Customer EXCEPT (Comments)

field lists (1)
for each order where order-num > x:
Type: FOR Statement
Client Sort: N
Scrolling: N
Table: wshop.Order
Indexes: Order-Num
Query Statistics: Bad1 logmgr.p line 23
QueryId: 101299360
DB Blocks accessed:
wshop : 15599
DB Reads:
Table: wshop.Order : 4557
Index: Order.Order-Num : UNAVAILABLE
wshop.Order Table:
4GL Records: 3399
Records from server: 3399
Useful: 3399
Failed: 0
Select By Client: N
16K DB reads
4557 fragments
3399 records

field lists (2)
for each order fields(order-num) where order-num > x:
Type: FOR Statement
Client Sort: N
Scrolling: N
Table: wshop.Order
Indexes: Order-Num
Query Statistics: Bad1 logmgr.p line 23
QueryId: 101299360
DB Blocks accessed:
wshop : 11333
DB Reads:
Table: wshop.Order : 3400
Index: Order.Order-Num : UNAVAILABLE
wshop.Order Table:
4GL Records: 3399
Records from server: 3399
Useful: 3399
Failed: 0
Select By Client: N
11K DB reads
3400 fragments
3399 records

network effects - lobs
Thank <insert favourite deity> Progress does NOT send the LOB across the wire unless
you ask for it
 The LOB field is really a separate entity to the record
 The real record only contains a pointer to the LOB
 The LOB may be in another storage area (and should be)
 When you access the LOB, the client requests it from the server

 Message network buffer size : -Mm
• Database startup parameter
– From 11.6.0+ you only need it on the db server
– Previous needed matching values on client and db server
• ABL clients only
• Ignored for non-network connections
network effects – legacy parameters
32600
350
1024
8192
-Mm (bytes)

network effects – prefetch parameters
 Used on NO-LOCK queries
• Forward only or scrolling
 10.2B06 + and 11.1+
-prefetchPriority Server defers poll for other requests while filling message
Current suggested value 100 records added before next poll
-prefetchDelay fills first message. By default first message contains one record
In theory this is better. In practice the ms difference is not
significant
-prefetchNumRecs how many records are stuffed in a message
100 records is a good start (default is 16)
-prefetchFactor How full (%-wise) to fill a message.
Aim for 90-100%

maximum transmission unit (MTU)
 A network parameter set at the NIC level
 Enable on the routing infrastructure
 Default is 1500
 “Jumbo Frames” is typically 9000 bytes
 The advantage lies in the relative size of network header data
• 1500 byte MTU: 1460 byte payload / 1538 byte total = 95% efficient
• 9000 byte MTU: 8960 byte payload / 9038 byte total = 99% efficient

monitoring server messages
 _ActServer VST
• Key: _server-ID
• Interesting fields: _Server-ByteSent, _ServerMsgSent, _ServerRecSent
• Calculate send size (Bytes sent / messages sent) and compare to –Mm
 Bug: When a record is larger than –Mm, only the first msg is counted
• I.e. if you send a 4K record and –Mm is 1024, only 1 msg and 1024 bytes sent recorded
• Blobs sent in 32,000 byte chunks – each chunk increments msgSent by 1 and byteSent by -
Mm

indexes are fun
 We are going to cover the rules without going into all the dirty details
 There are a number of excellent presentations on index rules
• Mike Lonski’s numerous “Pick an Index” PPTs are my favourites
 Come to PUG Challenge in Manchester NH (USA) in June or Prague CZ in November
to see Mike’s presentation

but before we get to the rules…
 This only applies to 4GL and not SQL
 Rules are applied in hierarchical order to filter indexes
• This is important: Each rule is applied and the result is one or more remaining indexes
 The first 5 rules only apply to a subset of indexes
• Compiler scans all fields in the query and selects all indexes that have leading components
with those fields
 Field match rules must be contiguous
• “Equality” on fields 1 and 3 of the index counts as 1, not 2
Rules continue to be applied until there is at least one index left

the rules
For the first-pass set of indexes, select one or more from
1. If CONTAINS use word-index
2. Unique index with all components involved in the equality matches
3. Most active equality matches
• Sorta kinda…full matches trump partial matches
• But only if more than 1 field (sometimes)
4. Most active range matches
5. Most active sort matches
If no "good" index is found, select one from
6. The primary index
7. First index alphabetically by name

order table indexes
Table Records Size Size Count LOBs Size Mean
PUB.Order 727285 63.9M 92 727285 -- -- --
Indexes
Flags Index Name Cnt Field Name
u CustOrder 2 + CustNum + Ordernum
OrderDate 1 + OrderDate
pu OrderNum 1 + Ordernum
OrderStatus 1 + OrderStatus
SalesRep 1 + SalesRep
w SRepW 1 + SalesRep
SRepDate 2 + SalesRep + OrderDate
DateSRep 2 + OrderDate + SalesRep
SDateOstat 2 + ShipDate + OrderStatus

for each order where
Index Selection Rule
U
CustOrder OrderDate
PU
OrderNum OrderStatus SalesRep
W
sRepW SRepDate DateSRep SDateOstatCarrier
If “CONTAINS”, use word-index
Unique index with all components
involved in the equality matches
Most active equality matches (Full
matches trump partial matches)
Most active range matches
Most active sort matches
The primary index
First index alphabetically by
name

where salesrep = "BBB"
U
CustOrder OrderDate
PU
W
If “CONTAINS”, use word-index X X X
involved in the equality matches X X X
matches trump partial matches) 1 / 1 X 1 / 2
The primary index
name
1
3

where salesrep = "DKP" and orderdate = 09/05/2011
U
CustOrder OrderDate
PU
W
If “CONTAINS”, use word-index X X X X X
involved in the equality matches X X X X X
matches trump partial matches) X X X 2 / 2 2 / 2
Most active range matches X X
Most active sort matches X X
The primary index X X
name X
2
5
1

where salesrep = "DKP" and orderdate = 09/05/2011 by salesrep
U
CustOrder OrderDate
PU
W
matches trump partial matches) X X X 2 / 2 2 / 2
The primary index X X
name X
2
5
1

where salesrep >= "DKP" and orderdate >= 09/05/2011 by salesrep
U
CustOrder OrderDate
PU
W
matches trump partial matches) X X X X X
Most active range matches 2 / 2 2 /2
Most active sort matches 1 / 2 X
The primary index
name
2
5
1

multiple index use
 Where clause includes “AND”
• ALL components of each index are involved in equality matches
• No unique indexes are involved
 Where clause includes “OR”
• Both sides of OR contain at least the lead component of an index
• Either equality or range match
 CAREFUL: return order not guaranteed

where salesrep = "BBB" and orderStatus = "Shipped"
U
CustOrder OrderDate
PU
W
If “CONTAINS”, use word-index X X X X
involved in the equality matches X X X X
matches trump partial matches) 1 / 1 1/ 1 X 1 / 2
The primary index
name
2
4

gotchas
 Expressions break bracketing
for each order no-lock where month(orderDate) = 1 ...
 BEGINS does NOT break bracketing
• Considered a range bracket
for each order no-lock where salesRep begins "D"
• Uses the order.salesRep index
 Temp-table rules are subtly different
 When in doubt, test and verify
• LOG-MANAGER is your friend

where month(orderDate) > 6
U
CustOrder OrderDate
PU
W
If “CONTAINS”, use word-index
involved in the equality matches
matches trump partial matches)
The primary index *
name
0

test and verify? how !?!
Using the for each order no-lock where salesRep begins "D" example
 VST data (shown here via ProTop)
 Query object handle q:index-information(i)
 LOG-MANAGER

log-manager
 A terribly underused but awesomely amazing tool
 Allows you to leave debug messages in your code
• No more /* Message here vValue. */
 Create some secret hotkey sequence to activate
• I.e. you can turn it on in production for one user
 Writes detailed info to a log file
assign log-manager:logfile-name = "c:tempwshop.log"
log-manager:logging-level = 3 // There are more levels than you think
log-manager:log-entry-types = "4GLTrace,4GLTrans,QryInfo".

test and verify with log-manager
 Still using the for each order no-lock where salesRep begins "D" example
Type: Dynamically Opened Query
PREPARE-STRING: for each order no-lock where salesrep begins 'd'
Prepared at Runtime
Client Sort: N
Scrolling: Y
Table: sports2000.Order
Indexes: SalesRep
Times prepared: 1
Time to prepare (ms): 0
DB Blocks accessed:
sports2000 : 366074
DB Reads:
Table: sports2000.Order : 182913
Index: Order.SalesRep : UNAVAILABLE

unnecessary work
1. Too many calls
2. Too much stuff per call
3. Too many copies of data

make fewer roundtrips
getLoginLanguages()
getLoginCompanies()
loginWindow.w
run getLoginLanguages()
run getLoginCompanies()
run buildUI().
1 2 3 4 5 6
1
3
2
4
6
5
time elapsed
A
7
7

make fewer roundtrips
getUiData()
getLoginLanguages()
getLoginCompanies()
loginWindow.w
run getUiData()
run buildUI().
1 2 5 6
1
6
2
B
7
7
5
1 2 3 4 5 6
time elapsed
A 7

no deep copies
define temp-table ttData ...
run populateData (output table ttData).
run showData (input table ttData).
run getChanges (output table ttData).
run saveChanges (input table ttData).
h = buffer ttData:handle.
run populateData (output h).
run showData (input h).
run getChanges (output h).
run saveChanges (input h).

no deep copies
run populateData (output table ttData).
run showData (input table ttData).
run getChanges (output table ttData).
run saveChanges (input table ttData).
run populateData (output table ttData by-reference).
run showData (input table ttData by-reference).
run getChanges (output table ttData by-reference).
run saveChanges (input table ttData by-reference).

finding deep copies: find it
 Make sure you have a problem
• LOG-MANAGER:LOG-ENTRY-TYPES = 'temp-tables'
• Data structure reference counts – datasets, buffers, objects, procedures, etc
define variable hDS as handle no-undo.
define variable iLoop as integer no-undo.
hDS = session:first-dataset.
do while valid-handle(hDS):
assign iCnt = iCnt + 1.
hDS = hDS:next-sibling.
end.

sidebar: dataset-handles are not really handles
 Yes, you define HANDLE variables to work with them
 But you can pass / call them as datasets
• To a parameter defined as DATASET-HANDLE
• To a parameter defined as DATASET
• To a parameter defined as HANDLE
 Nice thing with DATASET-HANDLE is that if you receive them you can use static code
against them as if you got a 'real' dataset

dataset-handles are not really handles
procedure fetch_data:
def output param poLotsaData as JsonObject.
def var hDataset as handle.
def var oDataObject as Object.
oDataObject = new CustomerData().
oDataObject:GetData(
output dataset-handle hDataset).
poLotsaData:Read(hDataset).
/* now has a property called dsCustomer */
oDataObject = new EmployeeData().
oDataObject:GetData(
poLotsaData:Read(hDataset).
/* now has a property called dsCustomer
AND one called dsEmployee */
end procedure.
class CustomerData:
define dataset dsCustomer for ttCustomer, ...
method public void GetData(
output dataset dsCustomer).
end class.
class EmployeeData:
define dataset dsEmployee for ttEmployee,
ttDepartment, ...
method public void GetData(
output dataset dsEmployee).
end class.

finding deep copies: fix it
 Make value passing of tables/dataset the
exception
• Look for ANY and ALL temp-table, table-
handle, dataset, dataset-handle calls without
BY-REFERENCE
• Know who "owns" the data
 When you have to make a deep copy, clean
up after yourself
 Desperate measures may be needed
run get_data.p (
finally:
delete object hDataset.
end finally.
run get_data.p (
output dataset-handle hDataset
by-reference).
do while valid-handle(hDS):
delete object hDS.
end.

conclusion
 The aim today was to show you areas you can look at for improving the performance of
your application
 From a code perspective, by
• coding appropriately to the indexes you have
• retrieving only data that's appropriate
• not leaving data "lying around"
 From At a design/architecture perspective, by
• appropriately balancing the size and quantity of network requests
 From an infrastructure perspective, by
• making sure that you optimise the data transfers from the DB server

ABL Code Performance Optimization Techniques

ABL Code Performance Optimization Techniques

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to ABL Code Performance Optimization Techniques

Similar to ABL Code Performance Optimization Techniques (20)

More from Alen Leit

More from Alen Leit (8)

Recently uploaded

Recently uploaded (20)

ABL Code Performance Optimization Techniques