App engine bulk loader download warning "No descending index on __key__, performing serial download"
I'm using the following to download all instances of one of my kinds:
appcfg.py download_data --config_file=bulkloader.yaml --kind=ModelName --filename=ModelName.csv --url=http://appid.appspot.com/remote_api
If the kind has more instances than the batch size, then I get this warning:
No descending index on __k开发者_JAVA技巧ey__, performing serial download
I don't have any custom indexes, or any properties with indexes disabled.
Do I 'need' to do something to resolve this warning, or is it just a warning I can safely ignore? Does it effect the speed of the download?
This article on the bulkloader includes the warning message in the sample output, but makes no mention of it.
This post on the app engine group says that I need to create an index. However adding more indexes would slow down writes to my entities - which I'd rather not do as I am going to be writing entities more often than I will be doing bulk data downloads.
Thanks.
As the error describes, without a descending index on __key__
for the model you're downloading, the bulkloader has to download serially. If you add the index as described, it will be able to download in parallel. If you don't, it will work fine, but will be slower to download, as it operates serially.
Note that an additional index has only a small impact on latency, as index rows are written in parallel to the entity write, meaning the write only takes as long as the slowest update.
I've solved this problem by add this code to index.yaml
kind: books
- properties:
name: __key__
- direction: desc
kind: books
- properties:
name: another_indexes_here
If your using JAVA and the datastore-indexes.xml file.
Add this (Assuming the name of the kind is "Books") to the datastore-indexes.xml file:
<datastore-index kind="Books" ancestor="false" source="auto">
<property name="__key__" direction="desc"/>
</datastore-index>
Then redeploy your app. Ensure you check the datastore index tab to see that the __key__
is serving.
Then you can try your download again.
精彩评论