开发者

ETL framework for loading data into Rails app

I need to load data for my Rails application from multiple providers (REST/SOAP based XML feeds) into the database on a recurring basis. I have written a set of Rake tasks which are kicked off by whenever-generated cron jobs. Each task hits the partner feed endpoint, parses the feed and loads it into the database.

Instead o开发者_StackOverflow中文版f writing Rake tasks, should I use an ETL framework like ActiveWarehouse (http://activewarehouse.rubyforge.org/etl/) instead? Any suggestions on the best way to do this in Rails?


If you are just loading data into a set of tables, and the use case is simple such as just adding new records or updating basic ones, and your load is meeting your requirements, I would stick with that. You could certainly use ActiveWarehouse as well, but it sounds like overkill. If, however, you need to support changing dimensions (ie. preserve history of data changes over time), or other 'data warehouse' features, then something like ActiveWarehouse starts to have more value.

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜