Data mining engines and frameworks? [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments开发者_如何学C, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance. Closed 9 years ago.

What opensource/free data mining engines and frameworks do you know and use for textual data?

Thank you for any advice!

---

Not really sure of what you're looking for. Perhaps something like Lucene?

---

Apache Mahout is an OpenSource Machile Learning library, that can be used with or without MapReduce (Apache Hadoop).

It provides the folloeing algorithms implementation in Java:

• Collaborative Filtering
• User and Item based recommenders
• K-Means, Fuzzy K-Means clustering
• Mean Shift clustering
• Dirichlet process clustering
• Latent Dirichlet Allocation
• Singular value decomposition
• Parallel Frequent Pattern mining
• Complementary Naive Bayes classifier
• Random forest decision tree based classifier

You can read more: http://mahout.apache.org/

http://girlincomputerscience.blogspot.com.br/2010/11/apache-mahout.html

http://www.ibm.com/developerworks/java/library/j-mahout/

---

RapidMiner is free and open source and runs on windows, mac, linux, and is a nice graphical workflow based program. It runs all Weka code, and integrates with R.

---

Weka and Rapidminer aren't that strong on clustering. They mostly do classification and similar predictions, but very little clustering. Have a look at ELKI, which is like WEKA a university project, but has tons of clustering and outlier detection methods.

---

I don't know about engines or frameworks, but I've used this tool called Weka, it has plenty of algorithms implemented in it.

---

And for text processing (rather than numeric data mining and clustering) then the NLTK toolkit is worth a look. This is intended to teach Natural Language Processing techniques in Python. So it is ideal for tinkering with, and you are bound to find many of the component classes and implementations useful if you choose to use Python.

---

RapidMiner is my prefered data mining solution: http://www.RapidMiner.com/

Here is survey of the most popular data mining tools among data mining experts: http://www.kdnuggets.com/2011/05/tools-used-analytics-data-mining.html

KDnuggets Poll 2011: RapidMiner is the most widely used data mining solution among data mining experts world-wide.

---

I'm the author of a Java open-source software for frequent pattern mining. It offers algorithms for mining sequential patterns, association rules, frequent itemsets, etc.

Although it is not specifically designed for text mining, some of the algorithms could be applied in to mine frequent patterns in text. For example, if you want to find some sequences of words that appear often together in several sentences you could apply a sequential pattern mining algorithm. But to do that you would need to to some pre-processing before applying my software so that your text file are in the proper format.

You can check the software here: http://www.philippe-fournier-viger.com/spmf/

---

Apache Mahout offers a bunch of popular algorithms that can also be applied on textual data and is also quite scalable! Apache UIMA doesn't offer data mining algorithms but is a framework that is widely used in natural language processing.