Filtering textual data in Python

I'm having a problem understanding what I'm doing wrong here. I have the code below (fairly straightforward).

def compileWordList(textList, wordDict):
    '''Function to extract words from text lines exc. stops,
        and add associated line nums'''
    i = 0;
    for row in textList:
        i = i + 1
        words = re.split('\W+', row)
        for wordPart in words:
            word = repr(wordPart)
            word = word.lower()
            if not any(word in s for s in stopsList):
                if word not in wordDict:
                    x = wordLineNumsContainer()
                    x.addLineNum(i)
                    wordDict[word] = x
                elif开发者_Python百科 word in wordDict:
                    lineNumValues = wordDict[word]
                    lineNumValues.addLineNum(i)
                    wordDict[word] = lineNumValues
            elif any(word in s for s in stopsList):
                print(word)

The code takes a string (sentence) from a list. It then splits the string for whole words using a re.split() method, returning a list of strings (the words).

I then force the string to lower case. Then I want it to check the word for existence in a list of stop-words I have (words too common in English to bother with). The part that checks if the word is in stopsList never seems to work, as stops words end up in my wordDict every time. Also I added the bottom print(word) statement in order to check it was catching them, but nothing ever gets printed :(

There are hundreds of stop words being used in the strings passing through.

Please can someone enlighten me here? Why are the strings never getting filtered for being stop words?

Many thanks, Alex

What about that?

from collections import defaultdict
import re

stop_words = set(['a', 'is', 'and', 'the', 'i'])
text = [ 'This is the first line in my text'
       , 'and this one is the second line in my text'
       , 'I like texts with three lines, so I added that one'
       ]   
word_line_dict = defaultdict(list)

for line_no, line in enumerate(text, 1): 
    words = set(map(str.lower, re.split('\W+', line)))
    words_ok = words.difference(stop_words)
    for wok in words_ok:
        word_line_dict[wok].append(line_no)

print word_line_dict

many thanks to Gnibbler: better way of writing the for-loop & more pythonic way of handling first insertion into the dict.

which prints (except the formatting of the dict)

{ 'added': [3]
, 'like': [3]
, 'that': [3]
, 'this': [1, 2]
, 'text': [1, 2]
, 'lines': [3]
, 'three': [3]
, 'one': [2, 3]
, 'texts': [3]
, 'second': [2]
, 'so': [3]
, 'in': [1, 2]
, 'line': [1, 2]
, 'my': [1, 2]
, 'with': [3]
, 'first': [1]
}