Your SlideShare is downloading. ×
Achievements
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Achievements

375
views

Published on


0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
375
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Achievement and challenges in Calc Summary of my activities in 2009, and plans going forward. OpenOffice.org Developer Novell, Inc. 9:00 AM – 9:45 AM November 5, 2009
  • 2. Agenda
    • Highlight of 2009
      • External reference re-work
      • 3. DataPilot improvements
      • 4. Row limit increase to 1 million rows
      • 5. Other on-going work worth mentioning
    • Where do we go from here?
    • 6. Working remotely from home
  • 7. External Reference Re-Work
  • 8. External range name support
    • Feature inspired by Excel interop. (i3740)
    • 9. Requires caching of referenced data.
      • Range names.
      • 10. Cell values used by range names.
    • ScExternalRefManager was born.
      • Caches referenced data only.
      • 11. Originally for caching range names only.
  • 12. Anatomy of ScExternalRefManager
  • 13. Old implementation
    • Emulated cache tables with existing features
      • Hidden sheets with special name.
      • 14. Cached entire sheet including non-referenced data.
    Reference a single cell... Whole sheet gets cached & still a regular sheet.
  • 15. External reference re-work Let's use ScExternalRefManager for both external range names and external references.
    • PROS
      • Cleaner implementation
        • Single caching back-end instead of two.
        • 16. No more “is this table an external cache?” query.
      • Saves precious table indexes (0 – 255).
      • 17. Efficient storage.
      • 18. Works better with .xlsx import.
  • 19. External reference re-work (cont'd)
    • CONS
      • Lots of changes necessary in many areas
        • Formula parser – add new token types to handle external data separately.
        • 20. ODF import/export – populating cache from, and dumping cache to, <table:table> structures.
        • 21. Excel import/export – modifies the way cached data are loaded and written. SUPBOOK, EXTERNNAME, XCT, and CRN records.
  • 22. Work started in mooxlsc CWS
    • Developers involved
    • Lots of hard work, lots of grounds to cover. (~2 months).
    • 25. Integrated into DEV300_m38. A big load off...
  • 26. The Unexpected (!)
    • External references in charts broke!
      • Data sequence providers to be re-implemented.
      • 27. External reference listeners added.
      • 28. Large re-factoring after feature freeze (3000 LOC).
    • Cell functions broke!
      • More functions than anticipated.
      • 29. Had to be re-implemented.
      • 30. Some are yet to be fixed (database functions).
  • 31. DataPilot Improvements
  • 32. Field popup window Before After
  • 33. Take 1 (Go-OO 3.1)
    • To provide visual feedback on fields with hidden members.
    • 34. Easy access to field members' visibility.
    • 35. Pop-up window; not a dialog.
    • 36. Bare minimum; just enough functionality.
  • 37. Take 2 (Go-OO 3.1.1)
    • To support sorting of field members via custom sort list.
    • 38. Larger font size.
    • 39. Submenu-capable menu control, dynamically populated (to be used for autofilter later).
  • 40. Take 3 (upstream build)
    • Toggle All check box.
    • 41. Single item buttons.
    • 42. Keyboard access.
    • 43. Accessibility support.
    • 44. Scheduled for 3.3 inclusion.
    • 45. In progress in koheidatapilot03 cws
  • 46. Custom field names
    • Assign names different from the original, simply by typing into the cell.
    • Requires ODF file format change. ODF-Next ?
  • 50. Row Limit Increase (1 million Rows)
  • 51. Lots of users want this...
    • Issue 30215, with 147 votes (!)
    • 52. Natural progression of how people use spreadsheet app; quasi database.
    • 53. Interop with Excel 2007.
  • 54. So, what's so hard about this ?
    • Performance, performance, performance! - lots of 0 to MAXROW loop all over the place.
    • 55. Ordinary-sized documents should not be affected.
    • 56. More rows, more data.
    Wait and see approach; let's increase the row limit, and see what happens.
  • 57. Row flag storage
    • Grouped bit-fields over row segments: hidden, filtered, page breaks (automatic and manual).
    • 58. Separate attributes of different distribution characteristics.
      • Hidden and filtered – range-oriented, top-skewed.
      • 59. Automatic page breaks – single rows, distributed semi-evenly from 0 to MAXROW.
      • 60. Manual page breaks – single rows, very few, random.
    • ::std::set for page breaks.
    • 61. Flat segment tree for hidden and filtered attributes.
  • 62. Flat segment tree
  • 63. Flat segment tree (cont'd)
    • Modified from segment tree.
    • 64. Segments don't overlap.
    • 65. Neighboring segments have different values.
    • 66. A query for a key also provides range info.
      • Shrinks the number of loops by skipping hidden or filtered rows.
      • 67. Avoid querying for row flags for each and every row.
    • Independent from OOo code base.
  • 68. Pagenation performance
    • Calculating page break positions took ~40 seconds.
      • Problem: sub-optimal data structure for storing page break positions.
      • 69. Solution: store page break positions in ::std::set separately.
      • 70. Result: pagenation now ~0.1 second.
  • 71. Print preview performance
    • Page preview was horrendously slow, with constant 100% CPU usage.
      • Problem: pagenation was performed at every pixel movement of the mouse cursor.
      • 72. Solution: Perform pagenation only when necessary.
      • 73. Result: CPU usage down to ~idle during print preview.
  • 74. Range selection performance
    • Calc became sluggish while selecting columns.
      • Problem: the position of the auto-fill mark became expensive to calculate & it was done on every change of range selection mark.
      • 75. Solution: Reduce the number of auto-fill position calculation by:
        • calculating it only when it's visible, and
        • 76. cache the calculated position to avoid redundant calculation.
      • Still room for improvement.
  • 77. Other minor performance tweaks
    • Sorting on large data.
      • Remove redundant broadcasting.
      • 78. Use ::std::vector instead of Table to store sorted data to avoid excessive array re-allocation. Don't forget to call reserve().
      • 79. Sort time down from 64 seconds to 8 seconds.
    • Selective use of (GTK's) progress bar.
      • Set lower boundary on use of progress bar.
      • 80. GTK's progress bar quite expensive to draw.
  • 81. Outstanding issues
    • Other performance issues lurking somewhere (such as Issue 106249).
    • 82. Note positioning at higher rows.
    • 83. Maybe other stuff I haven't thought of.
  • 84. Other On-going Work Worth Mentioning.
  • 85. Colored tab (by Daniel Watson)
    • Blessed by the UX.
    • 86. Needs ODF file format change.
    • 87. In progress in calctabcolor cws.
  • 88. Text import filter enhancement
  • 89. Copied range border
    • Paste by ENTER (by Jon Pryor).
    • 90. On-going in koheicopyborder cws.
  • 91. PHONETIC function
    • Original patch by Hiroyuki Ikezoe.
    • 92. Important for Japanese users. Perhaps usable for Chinese users (Pinyin)?
    • 93. In calcphonetic cws.
    • 94. Long way to go?
      • Editing UI
      • 95. Display in cell
      • 96. IM interaction
  • 97. Automatic decimal adjustment
  • 98. Cursor positioning in selection
    • Cursor stays at origin cell.
    • 99. Improved usability for keyboard users.
  • 100. Where do we go from here?
    • Realistically (every day work)
      • Feature parity with Excel.
      • 101. Better round-trip with xls.
    • Probably (still important)
      • Performance
        • Calculation – multi-threading?
        • 102. DataPilot
    • Hopefully (personal pet peeve, ITO?)
      • Eye candy
      • 103. More visual feedback
  • 104. Working remotely from home
    • European-centric development – hard on those of us in remote time zones.
    • 105. Become your own expert – code analysis, class diagram, symbol database.
    • 106. No face-to-face talk, limited to emails, IRC – occasional voice communication and periodic face-to-face meetings helpful. Build trust.
    • 107. Unable to share hardware – get your own, invest wisely.
  • 108.