Splitting a Domain name into constituent words (if possible)?

I want to break a domain name into constituent words and numbers e.g.

iamadomain11.com = ['i', 'am', 'a', 'domain', '11']

How do i do this? I am aware that there may be multiple sets possible, however, i am currently开发者_如何学Go even ok, just getting 1 set of possibilities.

This is actually solved in the O'Reilly Media book, Beautiful Data. In chapter 14, "Natural Language Corpus Data", he creates a splitter to do exactly as you want in Python using a giant freely available token frequency data set.

This is a fun problem! First you would need a dictionary. For performance reasons, store this in an hashset (probably can use the dictionary type in python). You could then iterate over each possible string, ("i", "ia", "iam",..."n11", "1", "11", "1") and check for matches in the dictionary. Then it's a matter of iterating over these matches until you have a contiguous set with no overlaps.

This would be a quick and dirty. There are probably faster ways to do this.

This sounds similar to the problem of tokenising Chinese, where there are no spaces in between words. This paragraph is taken from 'Introduction to Information Retrieval' by Manning, Raghavan & Schütze, available online here:

This phenomenon reaches its limit case with major East Asian Languages (e.g., Chinese, Japanese, Korean, and Thai), where text is written without any spaces between words. [...] One approach here is to perform word segmentation as prior linguistic processing. Methods of word segmentation vary from having a large vocabulary and taking the longest vocabulary match with some heuristics for unknown words to the use of machine learning sequence models, such as hidden Markov models or conditional random fields, trained over hand-segmented words

I would suggest greedy dictionary matching as a first step, then adding heuristics to handle the most common failure cases.