Ruby - Reading csv file and executing value in loop is skipping over lines in the csv file
I'm sure this is a completely ignorant question but here it goes. The following code's objective is to read a list of id's from a standard csv file, use the value to append to a URL, call the URL and extract a specific attribute via xpath. The problem I'm having is that the loop seems to be skipping some lines.
In example, here is a sample of 10 values:
777961
777972
781033
781044
781055
847066
744187
893908
369009
369010
The code is only reading every other line. The actual file has around 6000 lines, not huge bu开发者_Go百科t I'm only getting about 2500 values returned in the second file.
f = File.open('test.csv', 'r+')
url_f = File.open("url.csv", "w")
for line in f
f.each_line do |item|
item = f.gets
url = "http://test.com/testid=" + item
client = HTTPClient.new
resp = client.get_content(url)
doc = Nokogiri::HTML(resp)
doc.xpath("//link[@rel='canonical']/@href").each do |attr|
url_f.puts attr.value
puts attr.value
end
puts item
end
end
Nevermind, I figured it out.
I had the line item = f.gets
which would call the next line every time the loop ran thus skipping every other line. I knew it was a noob question. :P
精彩评论