FMK2018- FileMaker 17 Performance lab HOnza Koudelka

9. FileMaker Konferenz | Liechtenstein | 17.-20. Oktober 2018
www.filemaker-konferenz.com
Performance Questions & Test Results
HOnza Koudelka
FileMaker 17 Performance Lab

FileMaker 17 Performance Lab | HOnza Koudelka
HOnza Koudelka
• Co-founder and co-owner of 24U Software,
FBA Platinum Member
• Developing FileMaker solutions since 1991
• FileMaker 8, 10, 11, 12, 13, 14, 15, 16, and 17
Certified Developer
• Leading team of 5 other certified developers
• Winner of the FileMaker DevCon 2015 Developer Cup
• FileMaker Mad Dog Award 2011 winner
• Mad Optimizer and achiever of the impossible…
• Vendor session speaker at FileMaker DevCon, spoke at Pause[x]London and
Pause[x]Berlin, and remotely at PauseOnError Portland
• FileMaker Konferenz 2016 & 2017 speaker

Agenda
• FileMaker 17 Automated Test Results
• Recent Discoveries
• Benchmarking Techniques
• Optimization Techniques
• Questions & Answers

Why Performance Lab?
• Perceived speed defines what’s important
• Technical speed makes things possible

Faster is not always better

Test Equipment
• Mac Pro (early 2008)
• 2 x 2,8 GHz Intel Xeon
• 8 GB RAM
• Separate hard drive for data
• macOS 10.13 High Sierra
• FileMaker Server versions 15, 16, 17
• FileMaker Pro Advanced 12, 13, 14, 15, 16, 17

Automated tests performed
• Export & Import Records as tab delimited text
• Creating records with a lot of text
• Sort Records by indexed/unindexed field
• Perform Find
• indexed vs unindexed field
• small vs large result found set
• number vs text
• unique vs common values
• Position vs PatternCount
• Replace Field Contents vs Looped Set Field
• Summary fields vs Aggregate functions vs ExecuteSQL
• Delete All Records, Truncate Table, Delete All Found Records (leaving one record)
159
858
tests

Manually tested
• Spider vs Anchor/Buoy vs Selector-Connector
• Container vs text
• Structure impact on Set Field performance
• Slow network impact
• Layout rendering
• Conditional formatting
• Relationship sorting
• Data API vs PHP API
NEW

What was NOT tested
• Multiple users
• Multiple concurrent tasks
• Working with container fields
• Exact amounts of data transferred between client and server
• FileMaker Go
• FileMaker WebDirect
• FileMaker Cloud
• XML / PHP / ODBC / JDBC connections
• Two-machine deployment
• FileMaker Server with enabled SSL (except for Data API vs PHP API)
• Server-side schedules, including backup schedules
• Different cache sizes and other configuration differences

FileMaker 17 Test Results
12 13 14 15 16
? ?

Sorting records

Performing Find in a simple table

Batch-modifying records

FMSE shares resources with FMS
Indexed Unindexed

Recent Discoveries
• Spider vs Anchor/Buoy vs Selector-Connector
• Container vs text (danger of Base64)
• Structure impact on Set Field performance
• Slow network impact
• Layout rendering
• Expensive objects
• Conditional formatting
• Relationship sorting
• Data API vs PHP API

Spider vs Anchor/Buoy vs Selector-Connector

Under the Hood: Opening a File
• Seven-step opening process
• Step 3: Create or reuse temp file (no reuse for Perform Script On Server)
• Step 5: DBEngine processing
• List of tables
• Relationships
• Master map of all table occurrences
• Step 6: FMEngine processing
• Window
• Layout, Value lists, Font mappings, Custom menus
• Script triggers

Why is this important?
• User opens database only once or twice per day
• Every CWP session opens a file
• Every server-side script creates a new client session
• Every Perform Script On Server creates a new client session
• Perform Script On Server can be run asynchronously (without waiting)
• One user can start multiple server-side scripts within a single second
1 second to start a script = TOO SLOW
How long does your solution take to open?

Real solution test

Conversion to Anchor/Buoy
Before:
• 2359 total table occurrences
• 799 table occurrences in largest TOG
• 13 relationships longest path between TOs
After:
• Largest TOG split to 21 smaller TOGs
• 2349 total table occurrences
• 282 table occurrences in largest TOG
• 9 relationships longest path between TOs

After conversion to Anchor/Buoy

After adding Selector-Connector

Time to Perform Script on Server
Spider Anchor/Buoy Selector
Connector

Beware Base64
• Real experience from a customer
• Huge containers can be stored as text
• Doing so can have fatal impact on performance

Insanely slow Set Field
• Posted in FileMaker Community
• Blank layout
• Isolated table occurrence
• No records in any table
• Open locally in FileMaker Pro Advanced
• Creating 100 records in a loop takes…
30 seconds

• 1037 relationships
• 1004 in this TOG
• 690 around this TO
• 7 hops longest path

1037
1116 1194
7516
9701
Total number
of relationships

• large number of relationships all depend on the contents of one field
• FileMaker looks for all the possible combinations of relationships that can
be invalidated because that field changes
• That information is not in the join graph but in the field dependencies stored as
part of the field definitions.
• Roughly 27,000 possible joins that could be affected
• Higher level view code looks at all affected TOs and checks to see if any base
tables of those TOs are currently in any window and if there are any portals
• => Potential for optimization in future versions
• Thanks to Clay Maeckel for this under-the-hood info

Slow network impact

Notifications are sent…
• from server to client as part of reply to next client’s request
• whenever a change is committed
• to every client having the file open (even hidden)
• about every record (unless entire table is changed)
• in about 32K chunks
• without actual record data

Top Factors
• Bandwidth - amount of data transferred
• Latency - number of calls
• Packet Loss / Stability - persistency, online vs synchronization
• Examine with Network Link Conditioner & Top Call Stats

Things to avoid with connected users
• Modifying huge amounts
of records (in open files)
• Changing data in records
containing keys for portals
• Changing relationship
graph (in any file)

Recent optimization - 24U Collector

Layout rendering

Layout rendering
• Toolbox_GetTimestamp
• Benchmarks in global variable
• OnTimer script trigger

Layout rendering
• 8 conditional formatting rules
• 31 columns
• 29 records
• 8 x 31 x 29 = 7192
• Little chance to optimize time
per calc, it is necessary to
optimize their count
• Almost 7 seconds before
starting to evaluate conditional
formatting due to button bars &
popovers

Layout rendering
• Repeating text field replaced button bar (86.8 % saved)
• TextColor replaced conditional formatting
• Optimized condition calculation

Data API vs PHP API
• PHP API is based on Tomcat
• Data API is based on Node.js
• Data API is supposed to be faster and more stable
• PHP API is deprecated and disabled by default
• Data API requires HTTPS only, PHP API allows HTTP

Data API vs PHP API

WPE after trying to fetch 300k records

Benchmarking Techniques
• Scripts
• 24U FM Bench
• Get ( CurrentTimeUTCMilliseconds )
• Calculations
• 24U FM Bench
• Toolbox_GetTimestamp (microseconds)
• Layouts & other
• Unstored calculations + OnTimer script trigger
• Binary Splitting
• Top Call Stats
• Network Link Conditioner

Top Optimization Techniques
• Remove it if you don’t need it
• Script layouts
• Virtual List
• Perform Script On Server (or server-side queue)
• Pre-calculated summaries
• Simpler relationship graph
• Calculated text color instead of conditional formatting
• Card window instead of popover
• Avoid too many button bars
• Sort portal instead of relationship
• New table with fewer fields
• Conditional Set Field

More resources
• FileMaker Community > Discussions > Performance
• FileMaker DevCon Session Recordings (available on YouTube)
• Performance related sessions
• Under the Hood sessions
• FM Academy & Skeleton Key webinars (FMAcademy.com)
• “Designing for WAN Performance”
• FileMaker Optimizers LinkedIn Group (FMOptimizers.com)
• 24U FM Bench (FMBench.com)
• Big-O Cheat Sheet (bigocheatsheet.com)
• A Gentle Introduction to Algorithm Complexity Analysis (discrete.gr/complexity/)
• Enabling HTTPS Without Sacrificing Your Web Performance by Billy Hoffmann
(https://zoompf.com/blog/2014/12/optimizing-tls-handshake/)
• Ask for consultation at honza@24uSoftware.com or 24usw.com/consult

Vielen Dank unseren Sponsoren
Danke für das Bewerten dieses Vortrages

FMK2018- FileMaker 17 Performance lab HOnza Koudelka

Recommended

Recommended

More Related Content

Similar to FMK2018- FileMaker 17 Performance lab HOnza Koudelka

Similar to FMK2018- FileMaker 17 Performance lab HOnza Koudelka (20)

More from Verein FM Konferenz

More from Verein FM Konferenz (20)

Recently uploaded

Recently uploaded (20)

FMK2018- FileMaker 17 Performance lab HOnza Koudelka