Python: Lazy DB from CSV data?

I've got a relativ开发者_如何学Goely small (<100K) numerical CSV dataset that I want to process and graph with some numpy and pylab utilities, and it occurred to me that there's probably a better way of processing the data than ridiculous custom if-ladders for siphoning out the relevent experimental scenarios and comparisons.

If this data were in a DB rather than a CSV this wouldn't be a problem, but throwing together a 'real' db instance for the sake of this seems to be overkill. Is there a pythonic solution to what I'm looking for?

TL;DR Want to query CSV files like a DB / move CSV's into a mini-db.

Without knowing any specific details (at all) of your case, I'll expect that you'll find eventually one of the following ladders as a dominant one for your case:

• Simply just use the built in Python sqlite3.
  • However, if the relational model is not a necessity then pytables may be the way to go on.
    • Perhaps still, structured arrays can provide the necessary functionality.
      • But plenty can be still achieved with just proper usage of plain logic functions.
        • After all, get acquainted to live with your ridiculous custom if-ladders.

Obviously, any of the ladders sketched above will posses its specific pros and cons, depending on the actual case. Thus a really careful mix of them may eventually yield to best 'overall' result.

Perhaps pandas could help you out. In particular the query function.

Pandas can also do joins but at that point I would switch to SQL. A tiny database wrapper is dataset.

I once started to write a library of utilities called wavemol. One subpackage I developed was wavemol.fileaccess, which contains a CSV parsing class, which allows to access the file in a more practical way. Check here the methods provided.

check here the source

you may need to install wavemol.core first. I am not actively developing this code anymore, but if you are interested and this stuff does the trick for you, I may find some time to refocus on it a bit and put it back on track (of course help is welcome, but not necessary to make it a little better). I sort of lost interest into it because I changed job and I didn't need this stuff anymore.