Attribute Error for strings created from lists

I'm trying to create a data-scraping file for a class, and the data I have to scrape requires that I use while loops to get the right data into separate arrays-- i.e. for states, and SAT averages, etc.

However, once I set up the while loops, my regex that cleared the majority of the html tags from the data broke, and I am getting an error that reads:

Attribute Error: 'NoneType' object has no attribute 'groups'

My Code is:

import re, util
from BeautifulSoup import BeautifulStoneSoup

# create a comma-delineated file
delim = ", "

#base url for sat data
base = "http://www.usatoday.com/news/education/2007-08-28-sat-table_N.htm"

#get webpage object for site
soup = util.mysoupopen(base)

#get column headings
colCols = soup.findAll("td", {"class":"vaTextBold"})

#get data
dataCols = soup.findAll("td", {"class":"vaText"})

#append data to cols
for i in range(len(dataCols)):
    colCols.append(dataCols[i])

#open a csv file to write the data to
fob=open("sat.csv", 'a')

#initiate the 5 arrays
states = []
participate = []
math = []
read = []
write = []

#split into 5 lists for each row
for i in range(len(colCols)):
    if i%5 == 0:
        states.append(colCols[i])
i=1
whi开发者_如何学JAVAle i<=250:
    participate.append(colCols[i])
    i = i+5

i=2
while i<=250:
    math.append(colCols[i])
    i = i+5

i=3
while i<=250:
    read.append(colCols[i])
    i = i+5

i=4
while i<=250:
    write.append(colCols[i])
    i = i+5

#write data to the file
for i in range(len(states)):
    states = str(states[i])
    participate = str(participate[i])
    math = str(math[i])
    read = str(read[i])
    write = str(write[i])

    #regex to remove html from data scraped

    #remove <td> tags
    line = re.search(">(.*)<", states).groups()[0] + delim + re.search(">(.*)<",       participate).groups()[0]+ delim  + re.search(">(.*)<", math).groups()[0] + delim + re.search(">(.*)<", read).groups()[0] + delim  + re.search(">(.*)<", write).groups()[0]

    #append data point to the file
   fob.write(line)

Any ideas regarding why this error suddenly appeared? The regex was working fine until I tried to split the data into different lists. I have already tried printing the various strings inside the final "for" loop to see if any of them were "None" for the first i value (0), but they were all the string that they were supposed to be.

Any help would be greatly appreciated!

It looks like the regex search is failing on (one of) the strings, so it returns None instead of a MatchObject.

Try the following instead of the very long #remove <td> tags line:

out_list = []
for item in (states, participate, math, read, write):
    try:
        out_list.append(re.search(">(.*)<", item).groups()[0])
    except AttributeError:
        print "Regex match failed on", item
        sys.exit()
line = delim.join(out_list)

That way, you can find out where your regex is failing.

Also, I suggest you use .group(1) instead of .groups()[0]. The former is more explicit.