Addition of new protocol buffer files and recompilation of binaries

Support
1

Hi,

I added a new .proto file in the directory: moby/vendor/github.com/docker/swarmkit/api/ of
the locally cloned moby/moby repository in my PC. I successfully generated the corresponding .pb.go file for this .proto file using the command protoc --go_out=. file.proto.
However the make binary command fails with the following error. (pasting the last few lines of the output of the make binary command below)

Removing bundles/

---> Making bundle: binary (in bundles/binary)
Building: bundles/binary-daemon/dockerd-dev
GOOS="" GOARCH="" GOARM=""
vendor/github.com/docker/swarmkit/agent/exec/controller.go:8:2: found packages api (ca.pb.go) and docker_swarmkit_v1 (firstfile.pb.go) in /go/src/github.com/docker/docker/vendor/github.com/docker/swarmkit/api
Makefile:137: recipe for target 'binary' failed
make: *** [binary] Error 1

It appears that the new files are not being recognized by the build process. How should I go about fixing this? Is there a specific Makefile where they need to be mentioned or any other dependency that needs to be taken care of? Please also let me know if it is even feasible to add new .proto files and generate binaries from moby/moby.

Update: The README file under the same directory (moby/vendor/github.com/docker/swarmkit/api/) mentions using make generate command to regenerate .pb.go files after modifications in .proto files. But, make generate command does not work either and returns the error:
make: *** No rule to make target 'generate'. Stop.

2

Projects in the vendor directory are upstream dependencies. You need to add the protobuf file to swarmkit, build it there, then update the dependency to use it in moby.

3

So, I should ideally do the following:

  1. Clone the swarmkit repository
  2. Add the . proto file, use make generate command to produce the .pb.go file
  3. Push this repo to my own github page
  4. Specify the URL of this repo in moby/vendor.conf precisely like this,

Edit this line in moby/vendor.conf to point to the modified swarmkit repo on my github page instead of the usual swarmkit repo.
Current entry in moby/vendor.conf

github.com/docker/swarmkit 7dded76ec532741c1ad9736cd2bb6d6661f0a386

Edited version
github.com/my-username/my-swarmkit <Reference number>

Is this the right way of updating the dependency? Please point out if I have missed anything or if there is any mistake in this approach.

4

Yep, that should work.

5

Instead of making changes to the fork and the specifying the URL of that fork in vendor.conf, I notice that it is also possible to directly add changes under moby/vendor/github.com/docker/swarmkit/ if one needs to make changes to swarmkit for example. I was able to successfully practice this method and test it. In this test run I added some custom functions in swarmkit modules which executed fine when tested in moby dev container environment and also on the host machine. However, there might be a downside to this approach that I don’t foresee now. Can you comment on whether this approach is valid and will suffice?

6

Add changes to the vendored code is not supported.

I’d recommend reading up a bit on Go development before diving into a project as complex as moby. https://golang.org/doc/code.html is probably a good place to start.

1 Like