Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
High Performance
Core Data
HighPerformanceCoreData.com
MatthewMorey.com | @xzolian
BuoyExplorer.com
Agenda
Managed Object
Context
Managed
Object
Managed
Object
Persistent Store
Coordinator
Persistent Store
(SQLite, XML, ...)
Mode...
Memory
Speed
Less More
Slow Faster
Memory
Speed
Less More
Slow Faster
Memory
Speed
Less More
Slow Faster
Tools
#define TICK NSDate *startTime = [NSDate date]
#define TOCK NSLog(@"Elapsed Time: %f",
-[startTime timeIntervalSinceNow])
#define TICK NSDate *startTime = [NSDate date]
#define TOCK NSLog(@"Elapsed Time: %f",
-[startTime timeIntervalSinceNow])
...
-com.apple.CoreData.SQLDebug [1,2,3]
-com.apple.CoreData.SQLDebug [1,2,3]
-com.apple.CoreData.SQLDebug [1,2,3]
-com.apple.CoreData.SQLDebug [1,2,3]
-com.apple.CoreData.SQLDebug [1,2,3]
-com.apple.CoreData.SyntaxColoredLogging 1
-com.apple.CoreData.MigrationDebug 1
-com....
$ sqlite3
$ sqlite3 UFO.sqlite
sqlite> select * from sqlite_master;
table|ZUFOSIGHTING|ZUFOSIGHTING|3|CREATE TABLE
ZUFOSIGHTING ( Z_...
sqlite> SELECT t0.ZSHAPE, COUNT( t0.ZSHAPE )
FROM ZUFOSIGHTING t0 GROUP BY t0.ZSHAPE;
changed|1
changing|1546
chevron|760
...
Pony Debugger
SQLite
Professional
Tools
•Macros
•Launch Arguments
•SQL
•Instruments
•Measure, Measure, Measure
Fetching
NSFetchRequest *fetchRequest =
[NSFetchRequest fetchRequestWithEntityName:@"UFOSighting"];
[fetchRequest setFetchBatchSize...
NSFetchRequest *fetchRequest =
[NSFetchRequest fetchRequestWithEntityName:@"UFOSighting"];
NSArray *UFOSightingsArray =
[s...
NSFetchRequest *fetchRequest =
[NSFetchRequest fetchRequestWithEntityName:@"UFOSighting"];
[fetchRequest setResultType:NSD...
NSFetchRequest *fetchRequest =
[NSFetchRequest fetchRequestWithEntityName:@"UFOSighting"];
NSExpressionDescription *expres...
Prefetch Required Relationships
[fetchRequest
setRelationshipKeyPathsForPrefetching:
@[@"photo"]];
Fetching
•Don’t fetch more than you need
•Set a batch size
•Let SQLite do the calculations
•Prefetch required relationships
Predicates
Light Comparisons First
[fetchRequest setPredicate:
[NSPredicate predicateWithFormat:
@"shape == %@ AND duration > %i", @"sphere", 30]];
[fetchRequest setPredicate:
[NSPredicate predicateWithFormat:
@"shape == %@ AND duration > %i", @"sphere", 30]];
[fetchReq...
Be Cautions with Strings
Predicate
Costs
Less More
Beginswith
Endswith
Equality
==
Contains
Matches
Predicate
Costs
Less More
Beginswith
Endswith
Equality
==
Contains
Matches
[cd] cost you even more
Predicates
•Do light (numerical) comparisons first
•Beginswith/Endswith instead of Contains/Matches
•Don’t use [cd]
Searching
Canonicalize String Properties
Canonicalize String Properties
Description canonicalizedDesc
Green Mën green men
BEAM beam
Prόbë probe
ABDUCTION abduction
- (void)setDesc:(NSString *)desc {
if (_desc != desc) {
_desc = desc;
[self updateCanonicalizedDesc:desc];
}
}
- (void)upd...
- (void)setDesc:(NSString *)desc {
if (_desc != desc) {
_desc = desc;
[self updateCanonicalizedDesc:desc];
}
}
- (void)upd...
Canonicalized Tokens
componentsSeperatedByCharactersInSet
Separate Stack
Managed Object
Context
Persistent Store
Coordinator
Persistent Store
(SQLite, XML, ...)
Model
Primary
Managed Object
Conte...
Managed Object
Context
Persistent Store
Coordinator
Persistent Store
(SQLite, XML, ...)
Model
Primary
Managed Object
Conte...
Hash of Tokens
Searching
•Canonicalize text properties
•Use tokens
•Separate persistence stack
•Hash of tokens
Data Model
Don’t Overnormalize
•Duplication isn’t always bad
•Let UI drive data model
Use External Storage
Data Model
•Don’t overnormalize
•Use external storage for large attributes
•Large blobs as separate entity
•Less data take...
App Launch
dispatch_queue_t queue =
dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
dispatch_async(queue, ^{
// Create...
“Lots of Things” Problem
- (void)applicationDidEnterBackground:(UIApplication *)application {
UIApplication *app = [UIApplication sharedApplication...
Importing Data
Main Queue vs Private Queue
self.managedObjectContext = self.persistenceStack.managedObjectContext;
self.managedObjectCont...
Main Queue vs Private Queue
self.managedObjectContext = self.persistenceStack.managedObjectContext;
self.managedObjectCont...
Typical Import Algorithm
1) JSON to NSDictionary
2) Find existing NSManagedObject
3) Optionally, create new NSManagedObjec...
Typical Import Algorithm
1) JSON to NSDictionary
2) Find existing NSManagedObject
3) Optionally, create new NSManagedObjec...
Inefficient Find-Or-Create Algorithm
+ (instancetype)findOrCreateWithIdentifier:(id)identifier
inContext:(NSManagedObjectCo...
Inefficient Find-Or-Create Algorithm
+ (instancetype)findOrCreateWithIdentifier:(id)identifier
inContext:(NSManagedObjectCo...
1
2
3
4
...
New Data
1
4
...
Old Data
1
2
3
4
...
New Data
1
4
...
Old Data
Update
1
2
3
4
...
New Data
1
4
...
Old Data
Insert
1
2
3
4
...
New Data
1
4
...
Old Data
2
1
2
3
4
...
New Data
1
4
...
Old Data
2
1
2
3
4
...
New Data
1
4
...
Old Data
2
Insert
1
2
3
4
...
New Data
1
4
...
Old Data
2
3
1
2
3
4
...
New Data
1
4
...
Old Data
2
3
Update
Inefficient Find-Or-Create Algorithm
NSEnumerator *jsonEnumerator = [sortedArray objectEnumerator];
NSEnumerator *fetchResu...
Inefficient Find-Or-Create Algorithm
NSEnumerator *jsonEnumerator = [sortedArray objectEnumerator];
NSEnumerator *fetchResu...
Inefficient Find-Or-Create Algorithm
NSEnumerator *jsonEnumerator = [sortedArray objectEnumerator];
NSEnumerator *fetchResu...
Inefficient Find-Or-Create Algorithm
NSEnumerator *jsonEnumerator = [sortedArray objectEnumerator];
NSEnumerator *fetchResu...
NSUInteger totalUFOSightings = [UFOSightingsArray count];
NSInteger totalBatches = totalUFOSightings / MDM_BATCH_SIZE_IMPO...
NSUInteger totalUFOSightings = [UFOSightingsArray count];
NSInteger totalBatches = totalUFOSightings / MDM_BATCH_SIZE_IMPO...
NSUInteger totalUFOSightings = [UFOSightingsArray count];
NSInteger totalBatches = totalUFOSightings / MDM_BATCH_SIZE_IMPO...
Pre-Populated SQLite
File in App Bundle
Download Pre-Populated
SQLite File
Importing Data
•Do import work on private queues
•Use efficient find-or-create algorithm
•Work in batches to keep memory l...
Memory
[context refreshObject:UFOSightingManagedObject mergeChanges:NO];
[context reset];
Concurrency
Models
Managed Object
Context
Managed
Object
Managed
Object
Persistent Store
Coordinator
Persistent Store
(SQLite, XML, ...)
Mode...
Managed Object
Context
Managed
Object
Managed
Object
Persistent Store
Coordinator
Persistent Store
(SQLite, XML, ...)
Mode...
Managed Object
Context
Persistent Store
Coordinator
Persistent Store
(SQLite, XML, ...)
Model
Managed Object
Context
Main ...
Managed Object
Context
Persistent Store
Coordinator
Persistent Store
(SQLite, XML, ...)
Model
Managed Object
Context
Main ...
Managed Object
Context
Persistent Store
Coordinator
Persistent Store
(SQLite, XML, ...)
Model
Managed Object
Context
Main ...
Managed Object
Context
Persistent Store
Coordinator
Persistent Store
(SQLite, XML, ...)
Model
Managed Object
Context
Main ...
Persistent Store
Coordinator
Persistent Store
Model
Main Queue
Managed Object
Context
Managed Object
Context
Private Queue
Persistent Store
Coordinator
Persistent Store
Model
Main Queue
Managed Object
Context
Managed Object
Context
Private Queue...
Persistent Store
Coordinator
Persistent Store
Model
Main Queue
Managed Object
Context
Managed Object
Context
Private Queue...
Persistent Store
Coordinator
Persistent Store
Model
Main Queue
Managed Object
Context
Managed Object
Context
Private Queue...
Persistent Store
Coordinator
Persistent Store
Model
Main Queue
Managed Object
Context
Managed Object
Context
Private Queue...
Persistent Store
Coordinator
Persistent Store
Model
Main Queue
Managed Object
Context
Private Queue
Managed Object
Context...
Persistent Store
Coordinator
Persistent Store
Model
Main Queue
Managed Object
Context
Private Queue
Managed Object
Context...
Persistent Store
Coordinator
Persistent Store
Model
Main Queue
Managed Object
Context
Private Queue
Managed Object
Context...
Persistent Store
Coordinator
Persistent Store
Model
Main Queue
Managed Object
Context
Private Queue
Managed Object
Context...
Persistent Store
Coordinator
Persistent Store
Model
Main Queue
Managed Object
Context
Private Queue
Managed Object
Context...
Persistent Store
Coordinator
Persistent Store
Model
Main Queue
Managed Object
Context
Private Queue
Managed Object
Context...
Managed Object
Context
Persistent Store
Coordinator
Persistent Store
(SQLite, XML, ...)
Model
Managed Object
Context
Main ...
Managed Object
Context
Persistent Store
Coordinator
Persistent Store
(SQLite, XML, ...)
Model
Managed Object
Context
Main ...
Managed Object
Context
Persistent Store
Coordinator
Persistent Store
(SQLite, XML, ...)
Model
Managed Object
Context
Main ...
Managed Object
Context
Persistent Store
Coordinator
Persistent Store
(SQLite, XML, ...)
Model
Managed Object
Context
Main ...
Managed Object
Context
Persistent Store
Coordinator
Persistent Store
(SQLite, XML, ...)
Model
Managed Object
Context
Main ...
Managed Object
Context
Persistent Store
Coordinator
Persistent Store
(SQLite, XML, ...)
Model
Managed Object
Context
Main ...
Managed Object
Context
Persistent Store
Coordinator
Persistent Store
(SQLite, XML, ...)
Model
Managed Object
Context
Main ...
TLDR
Managed Object
Context
Persistent Store
Coordinator
Persistent Store
(SQLite, XML, ...)
Model
Managed Object
Context
Main ...
Concurrency Models
•Use the simplest model that meets your needs
•Update main context after large imports have completed
Bang
Head
Here
In general...
•Use the tools, and measure, measure, measure
•Don’t load more than you need to
•Don’t use [cd]
•Be smart wi...
Questions?
HighPerformanceCoreData.com
MatthewMorey.com | @xzolian
High Performance Core Data
High Performance Core Data
High Performance Core Data
High Performance Core Data
High Performance Core Data
High Performance Core Data
High Performance Core Data
High Performance Core Data
High Performance Core Data
High Performance Core Data
High Performance Core Data
High Performance Core Data
High Performance Core Data
High Performance Core Data
High Performance Core Data
High Performance Core Data
High Performance Core Data
High Performance Core Data
High Performance Core Data
High Performance Core Data
High Performance Core Data
High Performance Core Data
High Performance Core Data
High Performance Core Data
Upcoming SlideShare
Loading in …5
×

High Performance Core Data

4,369 views

Published on

http://highperformancecoredata.com/

Learn how to analyze, debug, and squeeze every last bit of performance out of Core Data. The standard Core Data implementation is very powerful and flexible but lacks performance. In this advanced session we will cover various performance analysis tools, model optimization, and various high performance concurrency models. This is an advanced discussion that assumes you have used Core Data before.

Published in: Technology, Education

High Performance Core Data

  1. 1. High Performance Core Data HighPerformanceCoreData.com MatthewMorey.com | @xzolian
  2. 2. BuoyExplorer.com
  3. 3. Agenda
  4. 4. Managed Object Context Managed Object Managed Object Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Main Queue
  5. 5. Memory Speed Less More Slow Faster
  6. 6. Memory Speed Less More Slow Faster
  7. 7. Memory Speed Less More Slow Faster
  8. 8. Tools
  9. 9. #define TICK NSDate *startTime = [NSDate date] #define TOCK NSLog(@"Elapsed Time: %f", -[startTime timeIntervalSinceNow])
  10. 10. #define TICK NSDate *startTime = [NSDate date] #define TOCK NSLog(@"Elapsed Time: %f", -[startTime timeIntervalSinceNow]) ... TICK; [self methodYouWantToMeasure]; TOCK;
  11. 11. -com.apple.CoreData.SQLDebug [1,2,3]
  12. 12. -com.apple.CoreData.SQLDebug [1,2,3]
  13. 13. -com.apple.CoreData.SQLDebug [1,2,3]
  14. 14. -com.apple.CoreData.SQLDebug [1,2,3]
  15. 15. -com.apple.CoreData.SQLDebug [1,2,3] -com.apple.CoreData.SyntaxColoredLogging 1 -com.apple.CoreData.MigrationDebug 1 -com.apple.CoreData.SQLiteDebugSynchronous [0,1,2] -com.apple.CoreData.SQLiteIntegrityCheck [1] -com.apple.CoreData.ThreadingDebug [1,2,3]
  16. 16. $ sqlite3
  17. 17. $ sqlite3 UFO.sqlite sqlite> select * from sqlite_master; table|ZUFOSIGHTING|ZUFOSIGHTING|3|CREATE TABLE ZUFOSIGHTING ( Z_PK...VARCHAR ) table|Z_PRIMARYKEY|Z_PRIMARYKEY|4|CREATE TABLE Z_PRIMARYKEY (Z_ENT...INTEGER) table|Z_METADATA|Z_METADATA|5|CREATE TABLE Z_METADATA (Z_VERSION...BLOB)
  18. 18. sqlite> SELECT t0.ZSHAPE, COUNT( t0.ZSHAPE ) FROM ZUFOSIGHTING t0 GROUP BY t0.ZSHAPE; changed|1 changing|1546 chevron|760 cigar|1782 circle|5271 cone|265 ... teardrop|595 triangle|6082 unknown|4490
  19. 19. Pony Debugger
  20. 20. SQLite Professional
  21. 21. Tools •Macros •Launch Arguments •SQL •Instruments •Measure, Measure, Measure
  22. 22. Fetching
  23. 23. NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:@"UFOSighting"]; [fetchRequest setFetchBatchSize:20];
  24. 24. NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:@"UFOSighting"]; NSArray *UFOSightingsArray = [self.managedObjectContext executeFetchRequest:fetchRequest error:NULL]; NSMutableDictionary *uniqueShapesDictionary = [NSMutableDictionary dictionary]; for (UFOSighting *sighting in UFOSightingsArray) { ... // Count unique shape items ... } NSManagedObject
  25. 25. NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:@"UFOSighting"]; [fetchRequest setResultType:NSDictionaryResultType]; [fetchRequest setPropertiesToFetch:@"shape"]; NSDictionaryResultType
  26. 26. NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:@"UFOSighting"]; NSExpressionDescription *expressionDescription = [[NSExpressionDescription alloc] init]; [expressionDescription setName:@"count"]; [expressionDescription setExpression: [NSExpression expressionForFunction:@"count:" arguments: @[[NSExpression expressionForKeyPath:@"shape"]]]]; [fetchRequest setPropertiesToFetch:@[@"shape", expressionDescription]]; [fetchRequest setPropertiesToGroupBy:@[@"shape"]]; [fetchRequest setResultType:NSDictionaryResultType]; NSExpression
  27. 27. Prefetch Required Relationships
  28. 28. [fetchRequest setRelationshipKeyPathsForPrefetching: @[@"photo"]];
  29. 29. Fetching •Don’t fetch more than you need •Set a batch size •Let SQLite do the calculations •Prefetch required relationships
  30. 30. Predicates
  31. 31. Light Comparisons First
  32. 32. [fetchRequest setPredicate: [NSPredicate predicateWithFormat: @"shape == %@ AND duration > %i", @"sphere", 30]];
  33. 33. [fetchRequest setPredicate: [NSPredicate predicateWithFormat: @"shape == %@ AND duration > %i", @"sphere", 30]]; [fetchRequest setPredicate: [NSPredicate predicateWithFormat: @"duration > %i AND shape == %@", 30, @"sphere"]];
  34. 34. Be Cautions with Strings
  35. 35. Predicate Costs Less More Beginswith Endswith Equality == Contains Matches
  36. 36. Predicate Costs Less More Beginswith Endswith Equality == Contains Matches [cd] cost you even more
  37. 37. Predicates •Do light (numerical) comparisons first •Beginswith/Endswith instead of Contains/Matches •Don’t use [cd]
  38. 38. Searching
  39. 39. Canonicalize String Properties
  40. 40. Canonicalize String Properties Description canonicalizedDesc Green Mën green men BEAM beam Prόbë probe ABDUCTION abduction
  41. 41. - (void)setDesc:(NSString *)desc { if (_desc != desc) { _desc = desc; [self updateCanonicalizedDesc:desc]; } } - (void)updateCanonicalizedDesc:(NSString *)desc { NSMutableString *result = [NSMutableString stringWithString:desc]; CFStringNormalize((CFMutableStringRef)result, kCFStringNormalizationFormD); CFStringFold((CFMutableStringRef)result, kCFCompareCaseInsensitive | kCFCompareDiacriticInsensitive | kCFCompareWidthInsensitive, NULL); self.canonicalizedDesc = [NSString stringWithString:result]; }
  42. 42. - (void)setDesc:(NSString *)desc { if (_desc != desc) { _desc = desc; [self updateCanonicalizedDesc:desc]; } } - (void)updateCanonicalizedDesc:(NSString *)desc { NSMutableString *result = [NSMutableString stringWithString:desc]; CFStringNormalize((CFMutableStringRef)result, kCFStringNormalizationFormD); CFStringFold((CFMutableStringRef)result, kCFCompareCaseInsensitive | kCFCompareDiacriticInsensitive | kCFCompareWidthInsensitive, NULL); self.canonicalizedDesc = [NSString stringWithString:result]; }
  43. 43. Canonicalized Tokens
  44. 44. componentsSeperatedByCharactersInSet
  45. 45. Separate Stack
  46. 46. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Primary Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Search Tokens
  47. 47. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Primary Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Search Tokens URIRepresentation
  48. 48. Hash of Tokens
  49. 49. Searching •Canonicalize text properties •Use tokens •Separate persistence stack •Hash of tokens
  50. 50. Data Model
  51. 51. Don’t Overnormalize •Duplication isn’t always bad •Let UI drive data model
  52. 52. Use External Storage
  53. 53. Data Model •Don’t overnormalize •Use external storage for large attributes •Large blobs as separate entity •Less data takes less time to fetch
  54. 54. App Launch
  55. 55. dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0); dispatch_async(queue, ^{ // Create persistent store coordinator with model // Add persistent store to store coordinator // Create managed object context // Context is fully initialized, notify view controllers dispatch_sync(dispatch_get_main_queue(), ^{ [self persistentStackInitialized]; }); });
  56. 56. “Lots of Things” Problem
  57. 57. - (void)applicationDidEnterBackground:(UIApplication *)application { UIApplication *app = [UIApplication sharedApplication]; __block UIBackgroundTaskIdentifier backgroundTask = 0; backgroundTask = [app beginBackgroundTaskWithExpirationHandler:^{ [app endBackgroundTask:backgroundTask]; backgroundTask = UIBackgroundTaskInvalid; }]; dispatch_async( dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{ // Delete or update a bunch of NSManagedObjects // If success [app endBackgroundTask:backgroundTask]; backgroundTask = UIBackgroundTaskInvalid; // If failure [app endBackgroundTask:backgroundTask]; backgroundTask = UIBackgroundTaskInvalid; }); }
  58. 58. Importing Data
  59. 59. Main Queue vs Private Queue self.managedObjectContext = self.persistenceStack.managedObjectContext; self.managedObjectContext.undoManager = nil; [self.managedObjectContext performBlockAndWait:^{ [self import]; }];
  60. 60. Main Queue vs Private Queue self.managedObjectContext = self.persistenceStack.managedObjectContext; self.managedObjectContext.undoManager = nil; [self.managedObjectContext performBlockAndWait:^{ [self import]; }]; NSManagedObjectContext *privateManagedObjectContext = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSPrivateQueueConcurrencyType]; [privateManagedObjectContext setPersistentStoreCoordinator: self.managedObjectContext.persistentStoreCoordinator]; privateManagedObjectContext.undoManager = nil; [privateManagedObjectContext performBlockAndWait:^{ [self import]; }];
  61. 61. Typical Import Algorithm 1) JSON to NSDictionary 2) Find existing NSManagedObject 3) Optionally, create new NSManagedObject 4) Set attributes of new/updated NSManagedObject
  62. 62. Typical Import Algorithm 1) JSON to NSDictionary 2) Find existing NSManagedObject 3) Optionally, create new NSManagedObject 4) Set attributes of new/updated NSManagedObject
  63. 63. Inefficient Find-Or-Create Algorithm + (instancetype)findOrCreateWithIdentifier:(id)identifier inContext:(NSManagedObjectContext *)context { NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:@"UFOSighting”]; fetchRequest.predicate = [NSPredicate predicateWithFormat:@"%K = %@", @"GUID”, identifier]; fetchRequest.fetchLimit = 1; id object = [[context executeFetchRequest:fetchRequest error:NULL] lastObject]; if (object == nil) { object = [UFOSighting insertNewObjectIntoContext:context]; } return object; }
  64. 64. Inefficient Find-Or-Create Algorithm + (instancetype)findOrCreateWithIdentifier:(id)identifier inContext:(NSManagedObjectContext *)context { NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:@"UFOSighting”]; fetchRequest.predicate = [NSPredicate predicateWithFormat:@"%K = %@", @"GUID”, identifier]; fetchRequest.fetchLimit = 1; id object = [[context executeFetchRequest:fetchRequest error:NULL] lastObject]; if (object == nil) { object = [UFOSighting insertNewObjectIntoContext:context]; } return object; }
  65. 65. 1 2 3 4 ... New Data 1 4 ... Old Data
  66. 66. 1 2 3 4 ... New Data 1 4 ... Old Data Update
  67. 67. 1 2 3 4 ... New Data 1 4 ... Old Data Insert
  68. 68. 1 2 3 4 ... New Data 1 4 ... Old Data 2
  69. 69. 1 2 3 4 ... New Data 1 4 ... Old Data 2
  70. 70. 1 2 3 4 ... New Data 1 4 ... Old Data 2 Insert
  71. 71. 1 2 3 4 ... New Data 1 4 ... Old Data 2 3
  72. 72. 1 2 3 4 ... New Data 1 4 ... Old Data 2 3 Update
  73. 73. Inefficient Find-Or-Create Algorithm NSEnumerator *jsonEnumerator = [sortedArray objectEnumerator]; NSEnumerator *fetchResultsEnumerator = [fetchResults objectEnumerator]; NSDictionary *UFOSightingDictionary = [jsonEnumerator nextObject]; UFOSighting *UFOSightingManagedObject = [fetchResultsEnumerator nextObject]; while (UFOSightingDictionary) { // Check if managed object already exist // Not an update, create new managed object // Set new attributes if (isUpdate) { UFOSightingDictionary = [jsonEnumerator nextObject]; UFOSightingManagedObject = [fetchResultsEnumerator nextObject]; } else { UFOSightingDictionary = [jsonEnumerator nextObject]; } }
  74. 74. Inefficient Find-Or-Create Algorithm NSEnumerator *jsonEnumerator = [sortedArray objectEnumerator]; NSEnumerator *fetchResultsEnumerator = [fetchResults objectEnumerator]; NSDictionary *UFOSightingDictionary = [jsonEnumerator nextObject]; UFOSighting *UFOSightingManagedObject = [fetchResultsEnumerator nextObject]; while (UFOSightingDictionary) { // Check if managed object already exist // Not an update, create new managed object // Set new attributes if (isUpdate) { UFOSightingDictionary = [jsonEnumerator nextObject]; UFOSightingManagedObject = [fetchResultsEnumerator nextObject]; } else { UFOSightingDictionary = [jsonEnumerator nextObject]; } }
  75. 75. Inefficient Find-Or-Create Algorithm NSEnumerator *jsonEnumerator = [sortedArray objectEnumerator]; NSEnumerator *fetchResultsEnumerator = [fetchResults objectEnumerator]; NSDictionary *UFOSightingDictionary = [jsonEnumerator nextObject]; UFOSighting *UFOSightingManagedObject = [fetchResultsEnumerator nextObject]; while (UFOSightingDictionary) { // Check if managed object already exist // Not an update, create new managed object // Set new attributes if (isUpdate) { UFOSightingDictionary = [jsonEnumerator nextObject]; UFOSightingManagedObject = [fetchResultsEnumerator nextObject]; } else { UFOSightingDictionary = [jsonEnumerator nextObject]; } }
  76. 76. Inefficient Find-Or-Create Algorithm NSEnumerator *jsonEnumerator = [sortedArray objectEnumerator]; NSEnumerator *fetchResultsEnumerator = [fetchResults objectEnumerator]; NSDictionary *UFOSightingDictionary = [jsonEnumerator nextObject]; UFOSighting *UFOSightingManagedObject = [fetchResultsEnumerator nextObject]; while (UFOSightingDictionary) { // Check if managed object already exist // Not an update, create new managed object // Set new attributes if (isUpdate) { UFOSightingDictionary = [jsonEnumerator nextObject]; UFOSightingManagedObject = [fetchResultsEnumerator nextObject]; } else { UFOSightingDictionary = [jsonEnumerator nextObject]; } }
  77. 77. NSUInteger totalUFOSightings = [UFOSightingsArray count]; NSInteger totalBatches = totalUFOSightings / MDM_BATCH_SIZE_IMPORT; // Create array with just the unique keys NSArray *jsonGUIDArray = [sortedArray valueForKey:UFO_KEY_JSON_GUID]; for (NSInteger batchCounter = 0; batchCounter <= totalBatches; batchCounter++) { // Create batch range based on batch size NSRange range = NSMakeRange(batchCounter*MDM_BATCH_SIZE_IMPORT,MDM_BATCH_SIZE_IMPORT); NSArray *jsonBatchGUIDArray = [jsonGUIDArray subarrayWithRange:range]; // Grab sorted persisted managed objects, based on the batch NSFetchRequest *fetchRequest = [[NSFetchRequest alloc]initWithEntityName:@"UFOSighting"]; NSPredicate *fetchPredicate = [NSPredicate predicateWithFormat:@"%K IN %@", @"GUID", jsonBatchGUIDArray]; // ... while (UFOSightingDictionary) { // ... // Efficient find-or-create algorithm // ... } }
  78. 78. NSUInteger totalUFOSightings = [UFOSightingsArray count]; NSInteger totalBatches = totalUFOSightings / MDM_BATCH_SIZE_IMPORT; // Create array with just the unique keys NSArray *jsonGUIDArray = [sortedArray valueForKey:UFO_KEY_JSON_GUID]; for (NSInteger batchCounter = 0; batchCounter <= totalBatches; batchCounter++) { // Create batch range based on batch size NSRange range = NSMakeRange(batchCounter*MDM_BATCH_SIZE_IMPORT,MDM_BATCH_SIZE_IMPORT); NSArray *jsonBatchGUIDArray = [jsonGUIDArray subarrayWithRange:range]; // Grab sorted persisted managed objects, based on the batch NSFetchRequest *fetchRequest = [[NSFetchRequest alloc]initWithEntityName:@"UFOSighting"]; NSPredicate *fetchPredicate = [NSPredicate predicateWithFormat:@"%K IN %@", @"GUID", jsonBatchGUIDArray]; // ... while (UFOSightingDictionary) { // ... // Efficient find-or-create algorithm // ... } }
  79. 79. NSUInteger totalUFOSightings = [UFOSightingsArray count]; NSInteger totalBatches = totalUFOSightings / MDM_BATCH_SIZE_IMPORT; // Create array with just the unique keys NSArray *jsonGUIDArray = [sortedArray valueForKey:UFO_KEY_JSON_GUID]; for (NSInteger batchCounter = 0; batchCounter <= totalBatches; batchCounter++) { // Create batch range based on batch size NSRange range = NSMakeRange(batchCounter*MDM_BATCH_SIZE_IMPORT,MDM_BATCH_SIZE_IMPORT); NSArray *jsonBatchGUIDArray = [jsonGUIDArray subarrayWithRange:range]; // Grab sorted persisted managed objects, based on the batch NSFetchRequest *fetchRequest = [[NSFetchRequest alloc]initWithEntityName:@"UFOSighting"]; NSPredicate *fetchPredicate = [NSPredicate predicateWithFormat:@"%K IN %@", @"GUID", jsonBatchGUIDArray]; // ... while (UFOSightingDictionary) { // ... // Efficient find-or-create algorithm // ... } }
  80. 80. Pre-Populated SQLite File in App Bundle
  81. 81. Download Pre-Populated SQLite File
  82. 82. Importing Data •Do import work on private queues •Use efficient find-or-create algorithm •Work in batches to keep memory low •Use pre-populated SQLite file
  83. 83. Memory
  84. 84. [context refreshObject:UFOSightingManagedObject mergeChanges:NO]; [context reset];
  85. 85. Concurrency Models
  86. 86. Managed Object Context Managed Object Managed Object Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Main Queue
  87. 87. Managed Object Context Managed Object Managed Object Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Main Queue
  88. 88. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Managed Object Context Main Queue Private Queue
  89. 89. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Managed Object Context Main Queue Private Queue
  90. 90. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Managed Object Context Main Queue Private Queue Context Did Save Notification Merge Changes From Notification
  91. 91. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Managed Object Context Main Queue Private Queue Context Did Save Notification Refetch and Reload
  92. 92. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object Context Managed Object Context Private Queue
  93. 93. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object Context Managed Object Context Private Queue UI Work
  94. 94. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object Context Managed Object Context Private Queue UI Work Persisting to Disk
  95. 95. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object Context Managed Object Context Private Queue UI Work Persisting to Disk
  96. 96. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object Context Managed Object Context Private Queue UI Work Persisting to Disk
  97. 97. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object Context Private Queue Managed Object Context Managed Object Context Private Queue
  98. 98. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object Context Private Queue Managed Object Context Managed Object Context Private Queue Background Updates
  99. 99. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object Context Private Queue Managed Object Context Managed Object Context Private Queue Background Updates UI Work
  100. 100. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object Context Private Queue Managed Object Context Managed Object Context Private Queue Background Updates UI Work Persisting to Disk
  101. 101. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object Context Private Queue Managed Object Context Managed Object Context Private Queue Background Updates UI Work Persisting to Disk
  102. 102. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object Context Private Queue Managed Object Context Managed Object Context Private Queue Background Updates UI Work Persisting to Disk __block NSManagedObjectContext *temporaryContext = self.workerObjectContext; __block NSManagedObjectContext *managedObjectContext = self.managedObjectContext; __block NSManagedObjectContext *writerObjectContext = self.writerManagedObjectContext; [temporaryContext performBlock:^{ NSError *error = nil; if (![temporaryContext save:&error]) { // TODO: Handle error } [managedObjectContext performBlock:^{ NSError *error = nil; if (![managedObjectContext save:&error]) { // TODO: Handle error } [writerObjectContext performBlock:^{ NSError *error = nil; if (![writerObjectContext save:&error]) { // TODO: Handle error } // Success!!! }]; // writerObjectContext }]; // managedObjectContext }]; // temporaryContext
  103. 103. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Managed Object Context Main Queue Private Queue Persistent Store Coordinator Model
  104. 104. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Managed Object Context Main Queue Private Queue Persistent Store Coordinator Model
  105. 105. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Managed Object Context Main Queue Private Queue Persistent Store Coordinator Model
  106. 106. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Managed Object Context Main Queue Private Queue Persistent Store Coordinator Model Context Did Save Notification
  107. 107. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Managed Object Context Main Queue Private Queue Persistent Store Coordinator Model Context Did Save Notification Merge Changes From Notification
  108. 108. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Managed Object Context Main Queue Private Queue Persistent Store Coordinator Model Context Did Save Notification Refetch and Reload
  109. 109. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Managed Object Context Main Queue Private Queue Persistent Store Coordinator Model Context Did Save Notification Refetch and Reload •Cannot pass objects between persistent store coordinators •Cannot pass objectIDs between persistent store coordinators •You can use URIRepresentation •You can use mergeChangesFromContextDidSaveNotification
  110. 110. TLDR
  111. 111. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Managed Object Context Main Queue Private Queue Context Did Save Notification Refetch and Reload
  112. 112. Concurrency Models •Use the simplest model that meets your needs •Update main context after large imports have completed
  113. 113. Bang Head Here
  114. 114. In general... •Use the tools, and measure, measure, measure •Don’t load more than you need to •Don’t use [cd] •Be smart with your data model •Import on a private queue in batches •Pick the correct (but simplest) concurrency model
  115. 115. Questions? HighPerformanceCoreData.com MatthewMorey.com | @xzolian

×