开发者

Ruby - Reading csv file and executing value in loop is skipping over lines in the csv file

I'm sure this is a completely ignorant question but here it goes. The following code's objective is to read a list of id's from a standard csv file, use the value to append to a URL, call the URL and extract a specific attribute via xpath. The problem I'm having is that the loop seems to be skipping some lines.

In example, here is a sample of 10 values:

777961
777972
781033
781044
781055
847066
744187
893908
369009
369010

The code is only reading every other line. The actual file has around 6000 lines, not huge bu开发者_Go百科t I'm only getting about 2500 values returned in the second file.

f = File.open('test.csv', 'r+')

url_f = File.open("url.csv", "w")

for line in f
  f.each_line do |item|
    item = f.gets
    url = "http://test.com/testid=" + item
    client = HTTPClient.new
    resp = client.get_content(url)

    doc = Nokogiri::HTML(resp)
      doc.xpath("//link[@rel='canonical']/@href").each do |attr|
      url_f.puts attr.value
      puts attr.value
      end
    puts item
  end
end


Nevermind, I figured it out.

I had the line item = f.gets which would call the next line every time the loop ran thus skipping every other line. I knew it was a noob question. :P

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜