Performance Improvements

Niklas Nebel Sun Microsystems PERFORMANCE IMPROVEMENTS IN CALC

Agenda
Introduction and context
Local optimizations
Handling sheets separately
DataPilot performance
Load & save outlook

Introduction and Context

Performance work in all of OOo
Performance project
Big improvements from 3.0 to 3.2
Start-up: Cold start of Writer 20% faster
Writer load performance
Comparable with MS Word 2007
Impress load performance
Comparable with MS PowerPoint 2007
Calc performance
Load and save: Up to twice as fast
Recalculation: Up to 20 times faster (extreme case)

Local Optimizations

API Usage When Saving Text Cells
Filter uses getFormula API method
Single quote character added if text can be parsed as a number
Unnecessary parsing step
Can take up to 17% of CPU time

Querying the Document Null Date
Internal representation: Days since the null date
File format: XML Schema dates ( ≈ ISO 8601)
Utility method for conversion
Queries the null date from the document
Several UNO calls
Querying once is enough
10% of CPU time if only date cells are used

Collecting Formatted Cell Ranges
Collect cell ranges with equal cell formats
For generating automatic styles
Keep a list of ranges for each set of formats
Try to join adjacent ranges
Formats are kept and iterated column-wise
Can use this information when trying to join
Prevents pathological cases

Formula Optimizations
String handling when formuas are parsed
Functions, references, names are case-insensitive
Operators, separators, parentheses are not
Reduce case conversion calls
5% of CPU time saved
Sorting of values for MEDIAN etc.
Not necessary to completely sort the array
Use std::nth_element STL method instead
Faster calculation after loading

Formula Recalculation (1)
Detection of duplicate notifications
When a cell range is modified
Parameter range can contain several changed cells
Notify each range only once
Also useful for single-cell change
Parameter range can contain several changed results
Extreme case: Issue 95967 – 20x faster

Handling Sheets Separately

Updating Row Heights
Optimal row height depends on local conditions
Especially fonts
Core structures need concrete height values
Positioning of shapes: Whole file
File format: relative to cell position
Internally: absolute positions
Screen output: Only single sheet
Update row heights
After loading: Visible sheet and sheets with shapes
Others as needed (display, printing, …)

Updating Row Heights: Comments
Cell comments (formerly: notes) are shapes
Often used in large sheets
Usually not shown
Create shape only when comment is shown
Saves time if there are many hidden comments
Row heights can be updated later

Updating Row Heights: Results
No effect for single sheet
Little improvement for text and numbers
30% CPU time with date cells on many sheets
Formula results don't have to be calculated

Partial Saving
Don't generate XML elements for whole file
Copy unchanged parts on stream level
Could copy from temporary storage
Storage layer creates copy of the unpacked file
Access the original file
Uncompress on the fly
Cost
File access: Read the compressed file
CPU: Uncompress

Experiment: Incremental Saving
Generate XML elements only for changed cells
Proof of concept: Only single-cell changes
No additional information kept after loading
Minimal parsing to find affected cells in stream
Takes extra time
Less if affected cells near start of file
Results (compared to 3.0):
40 – 70% improvement in CPU time
30 – 50% improvement in total time

Sheet-Wise Saving
Handle sheets instead of individual cells
Fewer sheets than cells
Additional information can be kept in memory
Easier to find modified sheets than modified cells
One obvious limitation:
Only useful with several sheets

Finding Modified Sheets
Few code changes for most types of changes
Formula notification for cell contents
Formula calculation for changed results
Cell format changes
Column widths or row heights
Handled separately: Print ranges, etc.
Currently no handling of drawing layer changes
All sheets are considered modified

Automatic Styles
Direct formats are collected in automatic styles
Referenced by name
Generated name (“ce1” etc.)
One list for the whole document
Have to be created with the same names again
Implemented for cell contents (incl. comments)
Keep a mapping of names to cell/text positions
Collect styles for unchanged sheets first
Include in existing duplicate detection for other sheets
Sheets with shapes always saved normally

Putting the Parts Together
When loading a file
Compatibility checks: Namespaces, encoding
Keep stream positions and style information
Steps to save a spreadsheet document
meta.xml, styles.xml, embedded objects: as usual
content.xml
Generate common content and modified sheets
For each sheet: Generate or copy stream portion
For “Save” and “Save As” update stream positions

Results
Influencing factors
Unchanged sheets
Type of sheet content
CPU time / file access
Example
Text, numbers, dates
16 sheets
Single sheet modified
Twice as fast
On top of other changes

Formula Recalculation (2)
Sheet area is divided into “slots”
16 columns by 128 rows
Range dependency registered in all affected slots
Needs attention when row limit is changed
Change: Use hash_set instead of set
Faster modification of dependency structures
Loading time
Change: Separate structures per sheet
Faster recalculation if several sheets are used

DataPilot Performance

DataPilot Memory Usage
Issue 55266: Several fields with many items
Fix now under way from IBM Symphony team
Don't allocate results for all child items
New cache table
CWS datapilotperf
Planned for OOo 3.3
Combination of large fields no longer a limitation

Load & Save Outlook

DOM Usage
Prototype by Christian Lippka for Impress
Use fast SAX to fill a compact DOM representation
Import from DOM, possibly parallel to parsing
Results for Impress
Only 2% improvement for typical presentation
Filling DOM tree uses 2% of CPU time
Not worth the effort
Calc may be different
Larger number of XML elements
But: Memory usage twice the XML stream size

Further Separation of Sheets
Load only the visible sheet
Load other sheets as needed, or in background
Parse XML fragment from stream, or use DOM
Formulas, charts may depend on changed cells
Dependencies must be known before saving
Parse formulas only as needed
Per sheet or individually
Already a separate step (but for all formulas)
Handle several sheets in parallel
More fine-grained locking needed

PERFORMANCE IMPROVEMENTS IN CALC Niklas Nebel [email_address]

Performance Improvements

by Alexandro Colorado on Nov 21, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

Statistics

Performance Improvements — Presentation Transcript