Ruby string encoding problem

I've looked at the other ruby/encoding related posts but haven't been able to figure out why the following is not working. Likely just because I'm dense, but here's the situation.

Using Ruby 1.9 on windows. I have a set of CSV files that need some data appended to the end of each line. Whenever I run my script, the appended characters are gibberish. The input text appears to be IBM437 encoding, whereas my string I'm appending starts as US-ASCII. Nothing I've tried with respect to forcing encoding on the input strings or the append string seems to change the resultant output. I'm stumped. The current encoding version is simply the last that I tried.

def append_salesperson(txt, salesperson)
  if txt.length > 2
    return txt.chomp.force_encoding('US-ASCII') + %(, "", "", "#{salesperson}")
  end
end

salespeople = Hash[
    "fname", "Record Manager"]

outfile = File.open("ActData.csv", "w:US-ASCII")

salespeople.each do | filename, recordManager |
  infile = File.open("#{filename}.txt")
  infile.each 开发者_运维百科do |line|
    outfile.puts append_salesperson(line, recordManager)
  end
  infile.close
end
outfile.close

One small note that is related to your question is that you have your csv data as such %(, "", "", "#{salesperson}"). Here you have a space char before your double quotes. This can cause the #{salesperson} to be interpreted as multiple fields if there is a comma in this text. To fix this there can't be white space between the comma and the double quotes. Example: "this is a field","Last, First","and so on". This is one little gotcha that I ran into when creating reports meant to be viewed in Excel.

In Common Format and MIME Type for Comma-Separated Values (CSV) Files they describe the grammar of a csv file for reference.

maybe txt.chomp.force_encoding('US-ASCII') + %(, "", "", "#{salesperson.force_encoding('something')}")

?

It sounds like the CSV data is coming in as UTF-16... hence the puts shows as the printable character (the first byte) plus a space (the second byte).

Have you tried encoding your appended data with .force_encoding(Encoding::UTF-16LE) or .force_encoding(Encoding::UTF-16BE)?