Rails 3 - Heroku - Data Type Binary Issue -- Works locally not on Heroku, suggestions to debug?

I have a rails 3开发者_如何学编程 app on heroku.

I have a table in the app where I store incoming emails. I added a column MESSAGE with a binary data type. That ended up working just fine locally.

I then pushed to heroku and it broke. When I check on heroku it says the datatype is indeed binary but looking at the logs rails is inserted the message(mail) as a string not binary, and I can't say I have a clue why.

Any suggestions here? Any ideas on how to best debug this issue?

ERROR:

ActiveRecord::StatementInvalid (PGError: ERROR:  invalid byte sequence for encoding "UTF8": 0xbb
HINT:  This error can also happen if the byte sequence does not match the encoding expected by the server, which is controlled by "client_encoding".

Thanks for any ideas

So the deal with cloudmailin is that is phrases the message partially for you on their servers and then uses a POST request to provide the data to you, as their documentation says

The message is not sent as a file. Instead it is as if the user pasted the message contents into a textfield.

This means that unless you explicitly convert the message to binary rails will treat it as a string. One way to do this might be to convert the message to a Mail object using the Mail gem. The documentation on cloudmailins website provides an example.

EDIT: Ok I think I misunderstood what was going on here. Assuming you want to save the original email to the database for whatever reason, you need to encode it so the DB won't throw the UTF8 encoding error. To do this convert the data you get from cloudmailin to a Mail object via Mail.new then take your resulting mail object we will call @m and call @em = @m.encoded. @em now contains an ecnoded version of the email which you should be able to store without any issues. When reading the email from the databse simply convert it back by passing the encoded text to Mail.new(@em).

Let me know if that works.

Source: http://docs.cloudmailin.com/post_format

Adding to what some of the other guys have said there are a couple of reasons why this could be happening.

The first is that the message content you are trying to save is not correctly highlighted as UTF-8. All of the parameters we send from CloudMailin should be UTF-8, in which case force_encoding('utf-8') is your friend. If you know the original format you can also use .encode('utf-8')

The other case is that the email headers or content itself is malformed. We have seen this a couple of times especially with subject and from headers. Normally these should be encoded but in many cases the mail client doesn't do this meaning that there's no way to detect the encoding that's being used and convert reliably.

Feel free to contact us at CloudMailin if you're still having issues as it's much easier to see what the issue is by looking at the raw email itself.