finding duplicates between my list and what's in a database
I would like some advice on how to implement the solution to following I have a list 开发者_如何学Goof objects. (hundreds of elements, like 500-1000, or more). I have a table in the database of records for such objects. Database has million of records. I need to send a list of object to the database, and report back with the list of the duplicate if found. Initial solution, load everything from database to Java, then compare lists - is bad solution. We have out-of-memory issue, trying to load all the millions of records from database.
Is there some identifier in the object by which you can look it up in the database? If yes, you can do the following:
Get the identifiers for your list of objects
Put them into a SELECT statement to see which are already in the database
Put the objects that are not yet in the table into an INSERT statement
If the list you get in 1 is too big for a SELECT, you can also put them into a temporary table and do a JOIN statement with the table of objects.
Cheers
精彩评论