diff --git a/README.md b/README.md index 7083fca..2adbc6d 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# gokit [![Circle CI](https://circleci.com/gh/go-kit/kit.svg?style=svg)](https://circleci.com/gh/go-kit/kit) [![Drone.io](https://drone.io/github.com/go-kit/kit/status.png)](https://drone.io/github.com/go-kit/kit/latest) [![Travis CI](https://travis-ci.org/go-kit/kit.svg?branch=master)](https://travis-ci.org/go-kit/kit) [![GoDoc](https://godoc.org/github.com/go-kit/kit?status.svg)](https://godoc.org/github.com/go-kit/kit) +# Go kit [![Circle CI](https://circleci.com/gh/go-kit/kit.svg?style=svg)](https://circleci.com/gh/go-kit/kit) [![Drone.io](https://drone.io/github.com/go-kit/kit/status.png)](https://drone.io/github.com/go-kit/kit/latest) [![Travis CI](https://travis-ci.org/go-kit/kit.svg?branch=master)](https://travis-ci.org/go-kit/kit) [![GoDoc](https://godoc.org/github.com/go-kit/kit?status.svg)](https://godoc.org/github.com/go-kit/kit) **Go kit** is a **distributed programming toolkit** for microservices in the modern enterprise. We want to make Go a viable choice for application (business-logic) software in large organizations. @@ -7,18 +7,35 @@ ## Motivation -See [the motivating blog post](http://peter.bourgon.org/go-kit) and [the video of the talk](https://www.youtube.com/watch?v=iFR_7AKkJFU). +Go has emerged as the language of the server, but it remains underrepresented +in large, consumer-focused tech companies like Facebook, Twitter, Netflix, and +SoundCloud. These organizations have largely adopted JVM-based stacks for +their business logic, owing in large part to libraries and ecosystems that +directly support their microservice architectures. + +To reach its next level of success, Go needs more than simple primitives and +idioms. It needs a comprehensive toolkit, for coherent distributed programming +in the large. Go kit is a set of packages and best practices, leveraging years +of production experience, and providing a comprehensive, robust, and trustable +platform for organizations of any size. + +In short, Go kit brings Go to the modern enterprise. + +For more details, see + [the motivating blog post](http://peter.bourgon.org/go-kit) and + [the video of the talk](https://www.youtube.com/watch?v=iFR_7AKkJFU). ## Goals - Operate in a heterogeneous SOA — expect to interact with mostly non-Go-kit services - RPC as the primary messaging pattern - Pluggable serialization and transport — not just JSON over HTTP -- Zipkin-compatible request tracing +- Operate within existing infrastructures — no mandates for specific tools or technologies ## Non-goals -- Supporting messaging patterns other than RPC (in the initial release) — pub/sub, CQRS, etc. +- Supporting messaging patterns other than RPC (in the initial release) — MPI, pub/sub, CQRS, etc. +- Re-implementing functionality that can be provided by wrapping existing packages - Having opinions on deployment, orchestration, process supervision, etc. - Having opinions on configuration passing — flags, env vars, files, etc. @@ -37,22 +54,22 @@ - Service discovery — pending - Example [addsvc](https://github.com/go-kit/kit/tree/master/addsvc) — **implemented** +### Dependency management + +Users who import Go kit into their `package main` are responsible to organize +and maintain all of their dependencies to ensure code compatibility and build +reproducibility. Go kit makes no direct use of dependency management tools like +[Godep](https://github.com/tools/godep). + +We will use a variety of continuous integration providers to find and fix +compatibility problems as soon as they occur. + ## Contributing Please see [CONTRIBUTING.md]. Thank you, [contributors]! [CONTRIBUTING.md]: /CONTRIBUTING.md [contributors]: https://github.com/go-kit/kit/graphs/contributors - -### Dependency management - -Users who import Go kit into their `package main` are responsible to organize -and maintain all of their dependencies to ensure code compatibility and build -reproducibility. Gokit makes no direct use of dependency management tools like -[Godep](https://github.com/tools/godep). - -We will use a variety of continuous integration providers to find and fix -compatibility problems as soon as they occur. ### API stability policy diff --git a/addsvc/README.md b/addsvc/README.md index b8ff324..6ef5263 100644 --- a/addsvc/README.md +++ b/addsvc/README.md @@ -1,15 +1,62 @@ # addsvc -addsvc is an example service, used to illustrate the mechanics of gokit. -It exposes simple functionality on a variety of transports and endpoints. +addsvc is an example service, used to illustrate the mechanics of Go kit. It +exposes a single method to add two integers on a variety of transports and +endpoints. + +## Highlights + +### Configuration via flags + +Go kit has no strong opinions about how to pass configuration to your service. +If your organization has established conventions to pass configuration into +your service, Go kit won't stand in your way. That said, package flag is a +good default: it's simple, well-understood, and provides a self-documenting +configuration surface area. Keeping with + [best practices](http://peter.bourgon.org/go-in-production/#configuration), flags +are defined in func main. + +### Declarative composition + +Go kit strongly favors explicit, declarative composition of interacting +components via a comprehensive func main. Time spent in keystrokes is made up +many, many times over when returning to the code and understanding exactly +what's happening, without having to unravel indirections or abstractions. + +### Multiple transports + +Go kit treats transports — HTTP, Thrift, gRPC, etc. — as pluggable. The same +service can be exposed on any, or multiple, available transports. The addsvc +example demonstrates how to make the same business logic available over +multiple transports simultaneously. + +### Daemonizing + +Go kit has no strong opinions about how to daemonize, supervise, or run your +service. If your organization has established conventions for running +services. Go kit won't stand in your way. Go kit services run equally well as +manually-copied binaries; applications provisioned with configuration +management tools like [Chef][], [Puppet][], or [Ansible][]; in containers like +[Docker][] or [rkt][]; or as part of a comprehensive scheduling platform like +[Kubernetes][], [Mesos][], [OpenStack][], [Deis][], etc. + +[Chef]: https://www.chef.io +[Puppet]: https://puppetlabs.com +[Ansible]: http://www.ansible.com +[Docker]: http://docker.com +[rkt]: https://github.com/coreos/rkt +[Kubernetes]: http://kubernetes.io +[Mesos]: https://mesosphere.com +[OpenStack]: https://www.openstack.org +[Deis]: http://deis.io ## Server To build and run addsvc, ``` -$ go install -$ addsvc +go install +addsvc ``` ## Client diff --git a/addsvc/add.go b/addsvc/add.go index c79f880..be6210b 100644 --- a/addsvc/add.go +++ b/addsvc/add.go @@ -18,9 +18,9 @@ // proxyAdd returns an implementation of Add that invokes a remote Add // service. -func proxyAdd(e endpoint.Endpoint, logger log.Logger) Add { +func proxyAdd(remote endpoint.Endpoint, logger log.Logger) Add { return func(ctx context.Context, a, b int64) int64 { - resp, err := e(ctx, reqrep.AddRequest{A: a, B: b}) + resp, err := remote(ctx, reqrep.AddRequest{A: a, B: b}) if err != nil { logger.Log("err", err) return 0 diff --git a/addsvc/main.go b/addsvc/main.go index 1877716..4bab3be 100644 --- a/addsvc/main.go +++ b/addsvc/main.go @@ -48,7 +48,7 @@ thriftBufferSize = fs.Int("thrift.buffer.size", 0, "0 for unbuffered") thriftFramed = fs.Bool("thrift.framed", false, "true to enable framing") - proxyHTTPAddr = fs.String("proxy.http.url", "", "if set, proxy requests over HTTP to this addsvc") + proxyHTTPURL = fs.String("proxy.http.url", "", "if set, proxy requests over HTTP to this addsvc") zipkinServiceName = fs.String("zipkin.service.name", "addsvc", "Zipkin service name") zipkinCollectorAddr = fs.String("zipkin.collector.addr", "", "Zipkin Scribe collector address (empty will log spans)") @@ -109,9 +109,9 @@ // Our business and operational domain var a Add = pureAdd - if *proxyHTTPAddr != "" { + if *proxyHTTPURL != "" { var e endpoint.Endpoint - e = httpclient.NewClient("GET", *proxyHTTPAddr, zipkin.ToRequest(zipkinSpanFunc)) + e = httpclient.NewClient("GET", *proxyHTTPURL, zipkin.ToRequest(zipkinSpanFunc)) e = zipkin.AnnotateClient(zipkinSpanFunc, zipkinCollector)(e) a = proxyAdd(e, logger) } diff --git a/addsvc/reqrep/request_response.go b/addsvc/reqrep/request_response.go index 59948df..82ab175 100644 --- a/addsvc/reqrep/request_response.go +++ b/addsvc/reqrep/request_response.go @@ -1,6 +1,7 @@ package reqrep -// The request and response types should be annotated sufficiently for all +// The concrete request and response types are defined for each method our +// service implements. Request types should be annotated sufficiently for all // transports we intend to use. // AddRequest is a request for the add method. diff --git a/tracing/README.md b/tracing/README.md index f277844..49f6409 100644 --- a/tracing/README.md +++ b/tracing/README.md @@ -1,21 +1,26 @@ # package tracing -`package tracing` provides [Dapper-style][dapper] request tracing to services. +`package tracing` provides [Dapper][]-style request tracing to services. An implementation exists for [Zipkin][]; [Appdash][] support is planned. -[dapper]: http://research.google.com/pubs/pub36356.html +[Dapper]: http://research.google.com/pubs/pub36356.html [Zipkin]: https://blog.twitter.com/2012/distributed-systems-tracing-with-zipkin [Appdash]: https://sourcegraph.com/blog/117580140734 ## Rationale -TODO +Request tracing is a fundamental building block for large distributed +applications. It's instrumental in understanding request flows, identifying +hot spots, and diagnosing errors. All microservice infrastructures will +benefit from request tracing; sufficiently large infrastructures will require +it. ## Usage -Wrap a [server.Endpoint][] so that it emits traces to a Zipkin collector. +Wrap a server- or client-side [endpoint][] so that it emits traces to a Zipkin +collector. -[server.Endpoint]: http://godoc.org/github.com/go-kit/kit/server#Endpoint +[endpoint]: http://godoc.org/github.com/go-kit/kit/endpoint#Endpoint ```go func main() { @@ -25,16 +30,21 @@ scribeHost = "scribe.internal.net" timeout = 50 * time.Millisecond batchSize = 100 - batchInterval = 3 * time.Second + batchInterval = 5 * time.Second ) - spanFunc := zipkin.NewSpanFunc(myHost, myMethod) collector, _ := zipkin.NewScribeCollector(scribeHost, timeout, batchSize, batchInterval) - var e server.Endpoint - e = makeEndpoint() // for your service - e = zipkin.AnnotateEndpoint(spanFunc, collector) + // Server-side + var server endpoint.Endpoint + server = makeEndpoint() // for your service + server = zipkin.AnnotateServer(spanFunc, collector)(server) + go serveViaHTTP(server) - serve(e) + // Client-side + before := httptransport.ClientBefore(zipkin.ToRequest(spanFunc)) + var client endpoint.Endpoint + client = httptransport.NewClient(addr, codec, factory, before) + client = zipkin.AnnotateClient(spanFunc, collector)(client) } ```