What are the prerequisites to learning natural language processing? [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.

---

Want to improve this question? Update the question so it focuses on one problem only by editing this post.

Closed 5 years ago.

Improve this question

I am planning to learn natural language processing this year.

But when I start reading introductory books on this topic, I found that I miss a lot of points relating mainly to mathematics.

So I'm here searching for what I should le开发者_StackOverflowarn before I can learn nlp, well, more smoothly?

Thanks in advance.

---

There are two main approaches to NLP right now - one is the language-based approach detailed by Jurafsky and Martin (Speech and Language Processing) and the other is a probability and statistics-based approach (Foundations of Statistical Natural Language Processing).

Most people that I've talked to tend to prefer the latter as far as ease of ramping up and useful results. So I would recommend going over probability theory first and then tackling an NLP book (like the second one I linked to, which I am actually using on a project right now with pretty good results).

While I agree with laura that formal language theory is highly useful, I actually think that currently if you just want to get into the actual NL parts of NLP, you can leave formal languages for later as there are enough tools that will do your lexical analysis / parsing / tokenizing / text transformations that you can use those rather than roll your own.

Here is a book describing three such tools - I own it and recommend it as a good introduction to all three. Building Search Applications: Lucene, LingPipe, and Gate

Edit: in response to your question, I would say that the first step would be to get a thorough grounding in the basics of probability (the first 3-5 chapters of any undergrad prob/stats book should be fine), and then from there look up new topics as they come up in the NLP book. For instance, yesterday I had to learn about t-values or something (I'm bad with names) because they happened to be relevant to determining incidence of collocation.

---

There are two excellent NLP classes on Coursera (Dan Jurafsky and Christopher Manning, or Michael Collins)

---

I would also recommend studying up on logic - first-order predicate logic for starters, but also higher-order logics (which are useful for reasoning about beliefs, intentions, knowledge etc. - i.e. consider the statement "The moon is made of green cheese" vs "I think the moon is made of green cheese".

Studying logic is useful for working with meaning representations. Grammars, languages etc. are useful for the parsing etc. but language doesn't fall neatly into a nice easy to parse grammar because, well, we're human :)

The previous poster noted about statistics and probability - very important in current approaches. You might also want to look at Judea Pearl's work on probabilistic inference networks.

You might also want to look at some projects like CYC. It started off as a project for representing common sense knowledge (ultimately language is used to input meaning, and that meaning has to be represented, so knowledge representation is very important). He originally started off with a frames-based approach but by the end it looks like he was basically using a variant of first-order logic.

Some people from the CYC project worked on the semantic web, which is also about meaning representation, and you'll note that semantic web representation is once again an XML equivalent of first-order predicate logic.

---

One of the things you will definitely need is good knowledge in the formal language area (automata, context free grammars, context sensitive grammars). Most of the stuff I've seen relies heavily on this - the best beginner book is, in my opinion, "Introduction to Formal Languages" by Hopcroft, Ullman &co.

I've also seen some techniques pertaining to neural networks, but I do not know much about these.

A third technique (somewhat developed from the formal language area) is based on molecular computing - not sure how extensively it is used. I'd look into A. Paun's books for that.

---

I'm just getting into this field myself but if you're not sure what programming language to start with, I'd recommend Python. It's a simple answer to a lot of research on the topic. Python is easy to pick up, extremely versatile, practical for a wide array of computing tasks, and widely adopted. Perfect for NLP. For an introduction to both Python and how to use it in NLP, I'd recommend the book:

Natural Language Processing with Python --- Analyzing Text with the Natural Language Toolkit by Steven Bird, Ewan Klein, and Edward Loper

http://nltk.org/book/

And Codecademy walks you through learning Python, too. http://www.codecademy.com

Hope it helps.