Using eventlet with SQLAlchemy
I'm using eventlet to build a simple website crawler starting from this example. I would like to use SQLAl开发者_运维百科chemy to store web pages' content and metadata for further processing. It's possible to use SQLAlchemy along with eventlet? Could anyone provide a simple example?
The OpenStack Compute (Nova) project uses SQLAlchemy and eventlet, you might be interested in their approach.
Sorry for late reply.
It would really depend on database connection library you use.
- For C extension library, like MySQLdb, Eventlet has db_pool module as used in OpenStack example. Basically, it creates a pool of OS threads to execute blocking DB operations. So you can't really have a lot of parallel queries, but most probably your database would not survive them anyway, so that's not an issue.
- For pure Python library, like myconnpy, pg8000, then just call
eventlet.monkey_patch()
. - As a special case, Eventlet supports monkey patching for psycopg2. It is a C extension but it is awesome enough to provide IO hooks, so you get best of both worlds. Again, simple call to
monkey_patch()
would do the trick.
精彩评论