Protocol buffers handling very large String message?

I was finally able to write protocol buffers code over REST and did some comparison with XStream which we are currently uses. Everything seems great, only stumble with one thing.

We have very large messages in one particular attributes, say something like this

message Data {
   optional string datavalue=1;
}

Datavalue above are extremely huge text messages. Size is 512kb - 5 Mb.

Protocol buffers deserialize just fine, with superb performance comparing to XStream. However, I notice when I send this message to wire (via REST), it took longer to get response. Always twice longer than XStream. I am thinking this might come from serializing time.

From google documents, it says Protocol buffers is not designed to handle very large messages, al开发者_如何学Pythonthough it can handle very large data set.

I was wondering if anyone has some opinion or maybe solution from my case above?

Thanks

I was benchmarking different serialization tools a while ago and noticed that the Protobuf Java library took about 1.7x as long to serialize strings as java.io.DataOutputStream did. When I looked into it, it seemed to have to do with weird artifact of how the JVM optimizes certain code paths. However, in my benchmarking, XStream was always slower, even with really long strings.

One quick thing to try is the format-compatible Protostuff library in place of Google's Protobuf library.

I remember reading somewhere (trying to locate the article) that protobuf is very good if you have a mix of binary and textual data types. When you are working purely on textual data then you could get better performance and size by compressing it.