Importing 100MB XML file into iOS Core Data Model

In my app I need to import a 100MB xml file into a Core Data Model.

So far, I've imported a 100KB xml file and everything works fine. However, not sure how long it takes to import a 100MB xml. I will run it tonite, in the while.. do you think this approach is good ?

thanks

开发者_如何学CNSManagedObjectContext * context = [self managedObjectContext];

    // Delete all documents
    NSFetchRequest * fetch = [[[NSFetchRequest alloc] init] autorelease];
    [fetch setEntity:[NSEntityDescription entityForName:@"Document" inManagedObjectContext:context]];
    NSArray * result = [context executeFetchRequest:fetch error:nil];
    for (id basket in result)
        [context deleteObject:basket];



    //Insert documents
    TBXML * tbxml = [[TBXML tbxmlWithXMLFile:@"categ_small.xml"] retain];
    TBXMLElement * root = tbxml.rootXMLElement;
    TBXMLElement * doc = [TBXML childElementNamed:@"doc" parentElement:root];

    do {
        TBXMLElement * idDoc = [TBXML childElementNamed:@"id" parentElement:doc];
        TBXMLElement * titleDoc = [TBXML childElementNamed:@"title" parentElement:doc];
        TBXMLElement * descriptionDoc = [TBXML childElementNamed:@"description" parentElement:doc];
        TBXMLElement * time = [TBXML childElementNamed:@"time" parentElement:doc];
        TBXMLElement * tags = [TBXML childElementNamed:@"tags" parentElement:doc];
        TBXMLElement * geo = [TBXML childElementNamed:@"geo" parentElement:doc];
        TBXMLElement * event = [TBXML childElementNamed:@"event" parentElement:doc];
        TBXMLElement * user = [TBXML childElementNamed:@"user" parentElement:doc];
        TBXMLElement * categ = [TBXML childElementNamed:@"categ" parentElement:doc];

        NSManagedObject *newDocument = [NSEntityDescription
                                        insertNewObjectForEntityForName:@"Document"
                                        inManagedObjectContext:context];

        [newDocument setValue:[TBXML textForElement:idDoc] forKey:@"idDoc"];
        [newDocument setValue:[TBXML textForElement:titleDoc] forKey:@"titleDoc"];
        [newDocument setValue:[TBXML textForElement:descriptionDoc] forKey:@"descriptionDoc"];
        [newDocument setValue:[TBXML textForElement:time] forKey:@"time"];
        [newDocument setValue:[TBXML textForElement:tags] forKey:@"tags"];
        [newDocument setValue:[TBXML textForElement:geo] forKey:@"geo"];
        [newDocument setValue:[TBXML textForElement:event] forKey:@"event"];
        [newDocument setValue:[TBXML textForElement:user] forKey:@"user"];
        [newDocument setValue:[TBXML textForElement:categ] forKey:@"categ"];

    } while ((doc = doc->nextSibling));  

UPDATE This is a one-time operation which runs in the Simulator only, and it won't be deployed with the final application.

I'd do the following steps:

• Write a converter XML->SQLite with ruby or php, or if you are not a friend of scripting languages then have a look at SQLite Manager it's a Firefox Plugin to manage SQLite database and it's capable of importing XML.

• Prepopulate core-data for shipping following this tutorial: How To Preload/Import Existing Data

Putting a document that size in resident memory is a quick way to get your app killed by the system.

To import an XML document that large on a system with limited resources you'll want streamed parsing, which will keep the resident memory footprint small. There are plenty of options here. Apple's NSXMLParser will do the trick, although it's not as fast as some of the alternatives. (A good overview of those alternatives is here.)

For the implementation, I suggest parsing in an NSOperation subclass, periodically notifying the main thread with progress updates. This will prevent the UI from blocking as you parse arbitrarily large documents.

Using NSEntityDescription the way you are will create individual autoreleased objects for every doc sibling in the XML file, which is another quick way to run short of memory. Take a look at Apple's Efficiently Importing Data documentation for the correct way to do this.

If this is a one-off process used to build an CoreData or SQLite database to distribute with you app, It will likely work, but only on the simulator. However, if you need to perform this on the device, you'll need to switch to a streaming parser.