A CLI for managing thrift IDL files.
idl provides a "package manager" for thrift interfaces.
It comes with two CLI commands,
mastercontaining the most recent Thrift IDL file for every service as the source of truth.
The CLI is currently broken down into two commands:
idl- CLI tool meant for end-users, but the publish command can also be run a continuous integration job when interface changes land in production (push-based publishing)
idl-daemon- daemonized process that can be configured to poll all service repositories for interface changes (pull-based publishing).
As a developer I want to be able to talk to other services; To do this I need to find their Thrift interface definitions.
idl CLI solves this need.
idl CLI tool has the following sub-commands:
idl list- List all available services published to the registry.
idl fetch <service-name>- Fetch a particular service's idl and copy it to your project in a standard location.
idl update- Update all previously fetched services to the most recent versions. Note: You cannot pick and choose which services to update. This is intentional.
idl publish- Publish the Thrift IDL file for your service to the idl registry repository. This command should set up to be automatically executed when a change to the service IDL lands on
masteror when that change on
masteris deployed to production.
All commands follow the unix standard of being silent if successful. If
you would like more information about what is happening, run the CLI
This command will create a starter IDL file at the correct filepath in
a new service. You should use it if you've got a new service and want
to quickly scaffold a simple thrift IDL file at the path expected
idl publish command.
$ idl init $
This command will list all available services published to the registry.
$ idl list - github.com:/foo/bar 2015-09-11T23:07:57.610Z - github.com:/foo/baz 2015-09-11T23:07:58.159Z - github.com:/qux/quux 2015-09-11T23:07:58.716Z $
idl fetch <service-name>
This command will fetch a particular service's idl file and copy it to your project in a standard location.
Once you fetch your first service we also write the
meta file that contains the version of the registry as well as the time
it was last changed.
Note: Fetching a new service will result in an implicit update of any
services that have been fetched. For example, if you fetched service
foo a month ago and then fetch service
first update service foo to the most current version before fetching
This command will also update the ./idl/meta.json file in your project.
$ idl fetch github.com:/foo/bar $
This command will update all previously fetched services to the most recent versions. Note: You cannot pick and choose which services to update. This is intentional.
Since the thrift definitions define the interfaces of the services in production, there is only one version for all files. When you update anything, you update everything to the current version.
This command will also update the ./idl/meta.json file in your project.
$ idl update $
This command will publish the Thrift IDL file for your service to the
idl registry repository. This command should set up to be
automatically executed when a change to the service IDL lands on
master or when that change on
master is deployed to production.
$ idl publish $
idl CLI takes the following command line flags. The
--repository, is mandatory until this tool has .rc file
--repository=<git url>- The
idlcommand needs to know the git URL of the registry to be able to run any of the commands above. e.g.
--cacheDir=<path to cache dir>- This is the path to the cache directory that
idlshould use. The default value is
--cwd=<current working directory>- The path to the current working directory in which to execute
--verbose- This tool follows the unix philosophy of being silent on success. Use this flag if you want to see output of what it is doing.
--config=<path to config file>- The path to a JSON config file that specifies any of the above properties.
idl uses dominictarr/rc for
options configuration. This module helps with parsing configuration
from arguments specified at the command line, from environment
variables and from
.rc files. It will probe the following locations:
Given your application name (
appname), rc will look in all the obvious places for configuration.
--config filethen from that file
.idlrcor the first found looking in
./ ../ ../../ ../../../etc.
All services and clients will have a
./idl/ directory at the root of
the repo. All thrift IDL files are contained in this directory.
Service authors need to understand how this directory is organized and should only every edit/modify the thrift IDL files for the service in question. Client authors should never have to edit/modify files in this directory and should only use the files contained therein as references for the interfaces they are consuming.
The idl directory is organized so that every thrift IDL file (for the
service being authored or the services being consumed) is located at a
sub-path that mirrors the git remote
origin URL of a service.
When you execute
git remote -v in your service's repository, you will
see output similar to one of the following:
$ git remote -v origin firstname.lastname@example.org:uber/foo-service.git (fetch) origin email@example.com:uber/foo-service.git (push)
$ git remote -v origin ssh://firstname.lastname@example.org/uber/foo-service.git (fetch) origin ssh://email@example.com/uber/foo-service.git (push)
The idl directory for your service mirrors these two addresses.
Assuming the output above, the idl path for the service
being authoring will be
This directory will contain the thrift IDL files that will be
published to your idl registry repo when
is executed. The IDL files in this particular sub-directory are to be
manually managed by service authors.
All other directories are contain service defitions for services being consumed and should not be edited/modified and should only be consulted as a reference when looking up a type definition for a service or a function definition for a service being consumed.
For service repositories, where the service is also a client of other services, the unmanaged definitions for that service and managed definitions for the services being consumed will live side by side in sibling directories.
The reason for mixing both managed and unmanaged directories together is to support relative filepath includes in thrift files.
For example, if the service firstname.lastname@example.org:foo/bar.git references type definitions from the services email@example.com:foo/baz.git and firstname.lastname@example.org:qux/quux.git, then you might see the following directory and file structure:
$ tree . └── idl ├── github.com │ ├── foo │ │ ├── bar │ │ │ └── bar.thrift │ │ └── baz │ │ └── baz.thrift │ └── qux │ └── quux │ └── quux.thrift └── meta.json 7 directories, 4 files $
... and the
contain the following includes in its header section:
include "../baz/baz.thrift" include "../../qux/quux/quux.thrift"
The complexity of how this directory is organized is a necessary
evil to support relative file includes. If, in the future, the
include directive supports richer semantics, it may be possible
to simplify this directory, but for now it is what is is.
idl-daemon will fetch all the remotes and place
their thrift files in the
upstream repository. You can use
idl fetch to fetch from the upstream repository.
idl-daemon is a command that should be run with
To set up the idl repository you can run the
and it will populate the idl remote repository.
The config file contains the following fields
This project is not done yet:
idl fetchis run.
idl validateso that service authors can locally validate the thrift IDL files for their service before publishing.
idl initto automatically create a boilerplate thrift IDL file using the git URL of the remote origin.
idl config get <property>to get a idl configuration property.
idl config set <property> <value>to set a idl configuration property.
mainfile in config to indicate service entry point.
publishcommand using a regex saved in the .rc file.
npm install idl --global