Protocol Buffers Invalid embedded descriptor problem

I'm having some problems at runtime with some of my generated protocol buffer classes.

My project layout is as follows:

module/
  protobuf-api/
    proto/
      com/foo/api/Service.proto
      com/foo/shared/Shared.proto
      org/bar/api/Message1.proto
      org/bar/api/Message2.proto

The Service.proto file depends on Shared.proto and some of the Message*.proto files. From the protobuf-api directory, I run the following command to compile:

find . -name *.proto -exec protoc --java_out=java -I=proto {} \;

When I attempt to run my Service, I get the following exception:

java.lang.ExceptionInInitializerError
    at com.linkedin.history.api.protobuf.HistoryServiceProtos$HistoryServiceQuery.(HistoryServiceProtos.java:544)
    at com.linkedin.history.api.serializer.HistoryServiceSerializer.serialize(HistoryServiceSerializer.java:47)
     at test.history.serializer.TestSerializer.testHistoryServiceQuery(TestSerializer.java:38)
    at test.fwk.util.core.BaseTestSuiteCore.r开发者_运维知识库un(BaseTestSuiteCore.java:304)
     at test.fwk.util.core.BaseTestSuiteConf.run(BaseTestSuiteConf.java:186)
    at test.fwk.lispring.BaseTestSuite.run(BaseTestSuite.java:232)
    at test.fwk.lispring.BaseTestSuite.callAppropriateRun(BaseTestSuite.java:265)
    at test.fwk.util.core.BaseTestSuiteCore.run(BaseTestSuiteCore.java:199)
Caused by: java.lang.IllegalArgumentException: Invalid embedded descriptor for "com/linkedin/history/api/protobuf/HistoryService.proto".
     at com.google.protobuf.Descriptors$FileDescriptor.internalBuildGeneratedFileFrom(Descriptors.java:268)
     at com.linkedin.history.api.protobuf.HistoryServiceProtos.(HistoryServiceProtos.java:1794)
Caused by: com.google.protobuf.Descriptors$DescriptorValidationException: com/linkedin/history/api/protobuf/HistoryService.proto: Dependencies passed to FileDescriptor.buildFrom() don't match those listed in the FileDescriptorProto.
     at com.google.protobuf.Descriptors$FileDescriptor.buildFrom(Descriptors.java:221)
     at com.google.protobuf.Descriptors$FileDescriptor.internalBuildGeneratedFileFrom(Descriptors.java:266)

I've read the post here but I think I'm doing everything correctly. Any suggestions on why I'm having the initializer errors? I'm compiling everything with the same -I flag.

I suspect that the problem is that when you're finding the proto file, you've given it the full path, e.g. proto/com/foo/api/Service.proto but when it refers to it via the include directory, it's using com/foo/api/Service.proto

Simple fix - run this from the proto directory:

find . -name *.proto -exec protoc --java_out=../java -I=. {} \;

I must admit I can't remember a lot of the details of protoc (which I really should) but I suspect that will work.

Another alternative which may work:

protoc --java_out=java `find . -name '*.proto'`

i.e. pass all the proto files into a single call to protoc.

I had the same error type in C# and here was my problem: I called the protoc in a pre-build step in my project. There I used Visual Studio built-in macros like $(SolutionDir) and $(ProjectDir) to retrieve necessary paths. Since I referenced *.proto files from other projects, I used two --proto_path options: one for the root path (to resolve import paths) and one for the file itself. My solution file was inside a subdirectory of the root directory, so I used the relative path .. to get to the root. Proto files are always in subdirectory gen of the particular project. All in all, the command was like this:

protoc.exe --proto_path=$(SolutionDir).. --proto_path=$(ProjectDir)gen  $(ProjectDir)gen\DemoFile.proto

It compiled fine, but I got the System.TypeInitializationException at runtime on calling CreateBuilder() method. The problem was that both paths $(SolutionDir).. and $(ProjectDir) (though effectively pointing to the same directory) had different textual representation due to the relative path component ... I solved the problem by consistently using the same path like this:

protoc.exe --proto_path=$(SolutionDir).. $(SolutionDir)..\My\Demo\Project\Directory\gen\DemoFile.proto

It cost me almost 3 days to narrow down and recognize the problem, so I share my solution here in hope that it will save some time for someone.