Magento: Rebuilding Index after self-written product import/update throws exception

we have a module to import products into magento. The module was written by a Magento programmer. It uses the Magento API and is written in PHP. It reads a CSV-like file (the actual delimiter used is '#'), translates a line into a product, then writes back the product to Magento. So far, so good.

I've duplicated and changed the module a few times. The last change was to update products, not import whole products. The module worked for about 950 of these products, then threw an exception:

magento:/srv/www/vhosts/www.mysite.com/shell # php import_colors.php ../var/import/colors.csv.out.csv
PHP Fatal error:  Uncaught exception 'Zend_Db_Statement_Exception' with message 'SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '12804-176-1-563' for key 'PRIMARY'' in /srv/www/vhosts/www.mysite.com/lib/Zend/Db/Statement/Pdo.php:234
Stack trace:
#0 /srv/www/vhosts/www.mysite.com/lib/Zend/Db/Statement.php(300): Zend_Db_Statement_Pdo->_execute(Array)
#1 /srv/www/vhosts/www.mysite.com/lib/Zend/Db/Adapter/Abstract.php(468): Zend_Db_Statement->execute(Array)
#2 /srv/www/vhosts/www.mysite.com/lib/Zend/Db/Adapter/Pdo/Abstract.php(238): Zend_Db_Adapter_Abstract->query('INSERT INTO `ca...', Array)
#3 /srv/www/vhosts/www.mysite.com/lib/Varien/Db/Adapter/Pdo/Mysql.php(333): Zend_Db_Adapter_Pdo_Abstract->query('INSERT INTO `ca...', Array)
#4 /srv/www/vhosts/www.mysite.com/lib/Varien/Db/Adapter/Pdo/Mysql.php(1537): Varien_Db_Adapter_Pdo_Mysql->query('INSERT INTO `ca...', Array)
#5 /srv/www/vhosts/www.mysite.com/app/code/core/Mage/Catalog/Model/Resource/Eav/Mysql4/Product/Indexer/Eav in /srv/www/vhosts/www.mysite.com/lib/Zend/Db/Statement/Pdo.php on line 234

Fatal error: Uncaught exception 'Zend_Db_Statement_Exception' with message 'SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '12804-176-1-563' for key 'PRIMARY'' in /srv/www/vhosts/www.mysite.com/lib/Zend/Db/Statement/Pdo.php:234

Stack trace:
#0 /srv/www/vhosts/www.mysite.com/lib/Zend/Db/Statement.php(300): Zend_Db_Statement_Pdo->_execute(Array)
#1 /srv/www/vhosts/www.mysite.com/lib/Zend/Db/Adapter/Abstract.php(468): Zend_Db_Statement->execute(Array)
#2 /srv/www/vhosts/www.mysite.com/lib/Zend/Db/Adapter/Pdo/Abstract.php(238): Zend_Db_Adapter_Abstract->query('INSERT INTO `ca...', Array)
#3 /srv/www/vhosts/www.mysite.com/lib/Varien/Db/Adapter/Pdo/Mysql.php(333): Zend_Db_Adapter_Pdo_Abstract->query('INSERT INTO `ca...', Array)
#4 /srv/www/vhosts/www.mysite.com/lib/Varien/Db/Adapter/Pdo/Mysql.php(1537): Varien_Db_Adapter_Pdo_Mysql->query('INSERT INTO `ca...', Array)
#5 /srv/www/vhosts/www.mysite.com/app/code/core/Mage/Catalog/Model/Resource/Eav/Mysql4/Product/Indexer/Eav in /srv/www/vhosts/www.mysite.com/lib/Zend/Db/Statement/Pdo.php on line 234

I thought nothing of it. The customer was happily changing the last 50 or so products on his own, and the error message was forgotten.

However, today the customer called that his product attribute index would not build, the index in the backoffice remained on orange "Processing". So I've investigated and found out that the index can be built by using shell/index开发者_如何学Cer.php. So I've tried and, and lo and behold, the exact same error message I almost forgot.

I've dumped the database and tried to find out which table actually contained the data listed in the error message (12804-176-1-563), but I had no luck. It seems that these 4 dates are nowhere to be found in this order.

Now, there are two options on how to proceed. One would be to find out which table actually contains the problem, so that it might be fixed using Magento itself. The other way might be to clear the corresponding index table and rebuild the index from scratch. I'm not certain digging around in a database schema I do not fully understand is a good idea.

Can anybody help me identify the problem?

Please check that this is not something simple such as some duplicated products in the database that have been 'aborted' to leave a product entry with no SKU value.

I have the same issue with a self written PHP import script. I am still not sure how to fix this, but I found that in 12804-176-1-563, 12804 refers to the id of the product in the DB. So if you delete/recreate or disable that product, then the indexer will work fine will work fine. If you have any tip on why there are some integrity constrain violations please share! :-)