Thank you Solomon for the clarifications.
Now let’s tackle the implementation and technical challenges.
Just like for Solomon’s example URLs, the URLs I’ll be using are only for getting specific points across. Please focus on the technical details only.
Version in URL can be inconvenient.
From a Go developer’s perspective, the only version that makes sense in an import path, is the sdk’s major version, which may or may not be the same as the API version, but is definitely not the Docker product’s version.
Here’s why: developer creates a Go library that imports “dockerdevkit.com/v17.06/go/mobycore”, but in v17.07 there is a new feature that they need to make use of, so they now need to change all import paths to v17.07, even in the files where the new feature was not used.
An additional minor annoyance is that their GOPATH will be growing with each client from each Docker release. The only time this is desirable, is if a new SDK is no longer backwards compatible with the previous one, hence why the major version of the SDK does make sense.
If having the SDK version in the URL is not desirable, then we probably should have no version at all in the URL. Instead, we could specify the Docker product’s version as a git tag in the git repo. The fact that Go has shortcomings specifying a git tag in a dependency is being worked on, and the workaround is README + vendoring. Either way, it is a known issue in the Go community.
Dependencies of the client package
The client package has the following dependencies today:
The same Go issue comes up again: how to tell Go developers to use specific versions of these dependencies?
One unconventional way for a Go library, is to use the vendor directory. That’s the behavior the current users of
github.com/docker/docker/client rely on, because that repo also builds binaries.
I suggest, that until Go figures out dependency management, we provide a vendor/ folder in this library repository.
An alternative solution is to ensure that any dependency of the package stays stable, and we have CI in place to guard that assumption. If that’s the preferred solution, we need to evaluate which dependencies from github.com/docker/docker/ can be considered stable in the master branch.
After talking with Tonis, code in moby core can now depend on the session package from moby/buildkit. A PR will be sent to do just that.
Moby Core client package in Devkit vs in Moby Core
Let me call “devkit” the Go package downloadable from a docker URL, used to interact with the Moby Core API, and “client” the Go package that is used in Moby Core to interact with the Moby Core API.
One suggestion is to have devkit have the exact same content as client. The only import paths that may need updating is for dependencies that would be moved to the devkit package, most likely api/types.
Another suggestion is to wrap the client package by inlining the same functions and embedding the types. In that case, the Go developer would still depend on github.com/moby/moby-core/client.
So, do we want to make it a requirement for devkit to not depend on github.com/moby/moby-core ? In that case, what about the pkg/* dependencies? Should we follow up on moving at least those out as detailed in https://github.com/moby/moby/issues/32989 ?
Go get relies solely on VCS
We cannot just put files in an S3 bucket, because go get shells out to git (and other VCSes, but I’ll assume git). Thus we need to serve through whatever transport git supports. The most straightforward answer is a git repo.
One solution is to set up a git proxy that removes the
components/engine folder prefix from the docker-ce repo.
A much simpler solution is to create a git repo that gets updated at each Docker release. Given our current presence on Github, that seems like the most natural place for that git repo. The downside of a Github repo, is we’ll have to have an answer to people who want to contribute there instead of the docker-ce repo.
Should there be one big repo, or one git repo per API? Moby Core, Registry, Cloud, etc.
Either way, we would set up canonical import paths for the Go packages, to use:
dockerdevkit.com/go/mobycore, so that developers who stumble across the git(hub) repo and try to go get it or vendor it directly, have a nice error message prompting them to use the canonical import path.
Solomon suggested we start documenting the Moby Core API and host it on mobyproject.org That’s work to be done: sync with docs team, possibly clone their process, find+replace strings.
There’s also work to be done on the devkit side. Should the Moby Core part of it be a duplicate that’s kept in sync with mobyproject.org or should it simply have the SDK documentations, but link to mobyproject.org for the full HTTP API?
The latter could be interesting, especially if we want some features of the HTTP API to not be exposed by devkit’s mobycore client.
Moby core releases and versioning
I don’t think having Moby Core releases is a blocker for renaming the repo, but it would be great to at least resolve the VERSION issue, because right now it is still
Anybody building moby core should be able to set the version to whatever they like. But for the default, here are the options:
- we set it to
- we decide on an actual version we’d like to release for Moby Core:
This would highlight an evolution
This would highlight a reboot.
I personally don’t have a strong opinion on this.
I’ll update the PR (https://github.com/moby/moby/pull/34226) with changes requested by Akihiro, namely changing the help output to also mention the new binary names.
Please answer with thoughts and concerns, or if you think I forgot to mention something important.