Synchronize IOS core data with web service? - web-services

Question

Here is my problem:
I want to use core data - speed and connectivity issues to build my IOS app. The data stored in core data is coming from a SQLServer database which I can access through a yet-to-be-defined web service.
Any changes to the data stored in core data needs to be synchronized with the SQLServer via a web service. In addition, I need to buffer changes that don't get synchronized because of connectivity issues.
I also need to update core data with any changes that have occured on the server. This could happen on a schedule set in user preferences.
Solutions I've Explored:
Using NSIncrementalStore class (new in IOS 5). I'm very confused on what this does exactly but it sounds promising. From what I can tell, you subclass NSIncrementalStore which allows you to intercept the regular core data API calls. I could then pass on the the information to core data as well as sync it with the external database via a web service. I could be completely wrong. But assuming I'm right, how would I sync deltas if the connection to the internet is down?
AFIncrementalStore - This is a subclass off of NSIncrementalStore using AFNetworking to do the web services piece.
RestKit - I'm a little concerned on how active this API is and it seems to be going through a transition to block functionality. Has anyone used this extensively?
I'm leaning towards AFIncrementalStore since this is using (what seems to be) a more standard approach. The problem is, I could be completely off on what NSIncrementalStore really is.
A link to some sample code or tutorial would be great!

Answer 1

My solution to this was to store two copies of the data set in a CoreData database. One represents the last-known server state and is immutable. The other is edited by the user.
When it is time to sync changes, the app creates a diff between the edited and immutable copies of the data. The app sends the diff to a web service which applies the diff to its own copy of the data. It replies with a full copy of the data set, which the app overwrites onto both of its copies of the data.
The advantages are:
If there is no network connectivity, no changes are lost: the diff is calculated each time the data set needs to be sent, and the immutable copy is only changed on a successful sync.
Only the minimum amount of information that needs to be sent is transmitted.
Multiple people can edit the same data at the same time without using locking strategies with a minimum opportunity for data loss via overwrites.
The disadvantages are:
Writing the diffing code is complex.
Writing the merging service is complex.
Unless you are a metaprogramming guru, you'll find that your diff/merge code is brittle and has to change whenever you change your object model.
Here are some of the considerations I had when coming up with the strategy:
If you allow changes to be made offline, checkin/checkout locking won't work (how can you establish a lock with no connection?).
What happens if two people edit the same data at the same time?
What happens if one person edits data on one iOS device when connectionless, switches it off, edits on another device and then turns the original device back on?
Multithreading with CoreData is an entire problem class in itself.
The closest thing I've heard of to out-of-the-box support to do anything remotely like this is the new iCloud/CoreData syncing system in iOS6, which automatically transmits entities from a CoreData database to iCloud when they change. However, that means you have to use iCloud.
EDIT: This is very late, I know, but here's a class that is capable of producing a diff between two NSManagedObject instances.
// SZManagedObjectDiff.h
#interface SZManagedObjectDiff
- (NSDictionary *)diffNewObject:(NSManagedObject *)newObject withOldObject:(NSManagedObject *)oldObject
#end
// SZManagedObjectDiff.m
#import "SZManagedObjectDiff.h"
#implementation SZManagedObjectDiff
- (NSDictionary *)diffNewObject:(NSManagedObject *)newObject withOldObject:(NSManagedObject *)oldObject {
NSDictionary *attributeDiff = [self diffAttributesOfNewObject:newObject withOldObject:oldObject];
NSDictionary *relationshipsDiff = [self diffRelationshipsOfNewObject:newObject withOldObject:oldObject];
NSMutableDictionary *diff = [NSMutableDictionary dictionary];
if (attributeDiff.count > 0) {
diff[#"attributes"] = attributeDiff;
}
if (relationshipsDiff.count > 0) {
diff[#"relationships"] = relationshipsDiff;
}
if (diff.count > 0) {
diff[#"entityName"] = newObject ? newObject.entity.name : oldObject.entity.name;
NSString *idAttributeName = newObject ? newObject.entity.userInfo[#"id"] : oldObject.entity.userInfo[#"id"];
if (idAttributeName) {
id itemId = newObject ? [newObject valueForKey:idAttributeName] : [oldObject valueForKey:idAttributeName];
if (itemId) {
diff[idAttributeName] = itemId;
}
}
}
return diff;
}
- (NSDictionary *)diffRelationshipsOfNewObject:(NSManagedObject *)newObject withOldObject:(NSManagedObject *)oldObject {
NSMutableDictionary *diff = [NSMutableDictionary dictionary];
NSDictionary *relationships = newObject == nil ? [[oldObject entity] relationshipsByName] : [[newObject entity] relationshipsByName];
for (NSString *name in relationships) {
NSRelationshipDescription *relationship = relationships[name];
if (relationship.deleteRule != NSCascadeDeleteRule) continue;
SEL selector = NSSelectorFromString(name);
id newValue = nil;
id oldValue = nil;
if (newObject != nil && [newObject respondsToSelector:selector]) newValue = [newObject performSelector:selector];
if (oldObject != nil && [oldObject respondsToSelector:selector]) oldValue = [oldObject performSelector:selector];
if (relationship.isToMany) {
NSArray *changes = [self diffNewSet:newValue withOldSet:oldValue];
if (changes.count > 0) {
diff[name] = changes;
}
} else {
NSDictionary *relationshipDiff = [self diffNewObject:newValue withOldObject:oldValue];
if (relationshipDiff.count > 0) {
diff[name] = relationshipDiff;
}
}
}
return diff;
}
- (NSDictionary *)diffAttributesOfNewObject:(NSManagedObject *)newObject withOldObject:(NSManagedObject *)oldObject {
NSMutableDictionary *diff = [NSMutableDictionary dictionary];
NSArray *attributeNames = newObject == nil ? [[[oldObject entity] attributesByName] allKeys] : [[[newObject entity] attributesByName] allKeys];
for (NSString *name in attributeNames) {
SEL selector = NSSelectorFromString(name);
id newValue = nil;
id oldValue = nil;
if (newObject != nil && [newObject respondsToSelector:selector]) newValue = [newObject performSelector:selector];
if (oldObject != nil && [oldObject respondsToSelector:selector]) oldValue = [oldObject performSelector:selector];
newValue = newValue ? newValue : [NSNull null];
oldValue = oldValue ? oldValue : [NSNull null];
if (![newValue isEqual:oldValue]) {
diff[name] = #{ #"new": newValue, #"old": oldValue };
}
}
return diff;
}
- (NSArray *)diffNewSet:(NSSet *)newSet withOldSet:(NSSet *)oldSet {
NSMutableArray *changes = [NSMutableArray array];
// Find all items that have been newly created or updated.
for (NSManagedObject *newItem in newSet) {
NSString *idAttributeName = newItem.entity.userInfo[#"id"];
NSAssert(idAttributeName, #"Entities must have an id property set in their user info.");
id newItemId = [newItem valueForKey:idAttributeName];
NSManagedObject *oldItem = nil;
for (NSManagedObject *setItem in oldSet) {
id setItemId = [setItem valueForKey:idAttributeName];
if ([setItemId isEqual:newItemId]) {
oldItem = setItem;
break;
}
}
NSDictionary *diff = [self diffNewObject:newItem withOldObject:oldItem];
if (diff.count > 0) {
[changes addObject:diff];
}
}
// Find all items that have been deleted.
for (NSManagedObject *oldItem in oldSet) {
NSString *idAttributeName = oldItem.entity.userInfo[#"id"];
NSAssert(idAttributeName, #"Entities must have an id property set in their user info.");
id oldItemId = [oldItem valueForKey:idAttributeName];
NSManagedObject *newItem = nil;
for (NSManagedObject *setItem in newSet) {
id setItemId = [setItem valueForKey:idAttributeName];
if ([setItemId isEqual:oldItemId]) {
newItem = setItem;
break;
}
}
if (!newItem) {
NSDictionary *diff = [self diffNewObject:newItem withOldObject:oldItem];
if (diff.count > 0) {
[changes addObject:diff];
}
}
}
return changes;
}
#end
There's more information about what it does, how it does it and its limitations/assumptions here:
http://simianzombie.com/?p=2379

Answer 2

Use the Parse platform and its IOS SDK to structure and store info. It can cache data locally so you can retrieve it quickly and when there is no connectivity.