Make OLAP with Hadoop Hive from OLTP Mysql
I bit confuse with Hadoop hive which i read from Wiki used for make OLAP. Now i want to make OLAP on Hive from OLTP database which use Mysql开发者_如何转开发.
How i can solve this? can i use Kettle for make OLAP in Hive? any guidance how to make OLAP on Hive from OLTP mysql ?
Tks.
I would suggest the following approach:
a) To specify the history part of your OLTP process. Usually it is some kind of logs over operations. Lets call it fact table.
b) To have fact table to be partitioned by time
c) Periodically offload oldest partition from the MySQL by exporting it to CSV and deleting from the MySQL.
e) Load this CSV file to Hive
By implementing this schema you will have all but latest data in the hive, and prevent growing of the MySQL OLTP database.
精彩评论