Codebase list golang-github-go-kit-kit / e0ebc18
RFCs have outlived their usefulness Peter Bourgon 8 years ago
10 changed file(s) with 16 addition(s) and 431 deletion(s). Raw diff Collapse all Expand all
+16
-27
README.md less more
126126 of production experience, and providing a comprehensive, robust, and trustable
127127 platform for organizations of any size.
128128
129 In short, Go kit brings Go to the modern enterprise.
129 In short, Go kit makes Go a viable choice for business-domain microservices.
130130
131131 For more details, see
132132 [the motivating blog post](http://peter.bourgon.org/go-kit/) and
143143
144144 ## Non-goals
145145
146 - Supporting messaging patterns other than RPC (in the initial release) — MPI, pub/sub, CQRS, etc.
147 - Re-implementing functionality that can be provided by wrapping existing packages
148 - Having opinions on deployment, orchestration, process supervision, etc.
149 - Having opinions on configuration passing — flags, env vars, files, etc.
146 - Supporting messaging patterns other than RPC (for now) — e.g. MPI, pub/sub, CQRS, etc.
147 - Re-implementing functionality that can be provided by adapting existing software
148 - Having opinions on operational concerns: deployment, configuration, process supervision, orchestration, etc.
150149
151150 ## Contributing
152151
153 Please see [CONTRIBUTING.md]. Thank you, [contributors]!
152 Please see [CONTRIBUTING.md][]. Thank you, [contributors][]!
154153
155154 [CONTRIBUTING.md]: /CONTRIBUTING.md
156155 [contributors]: https://github.com/go-kit/kit/graphs/contributors
157156
158157 ## Dependency management
159158
160 Go kit is a library, designed to be imported into a binary package. Vendoring
161 is currently the best way for binary package authors to ensure reliable,
162 reproducible builds. Therefore, we strongly recommend our users use vendoring
163 for all of their dependencies, including Go kit. To avoid compatibility and
164 availability issues, Go kit doesn't vendor its own dependencies, and doesn't
165 recommend use of third-party import proxies.
166
167 There are several tools which make vendoring easier, including [gb][],
168 [govendor][], and [godep][]. And Go kit uses a variety of continuous
169 integration providers to find and fix compatibility problems as soon as they
170 occur.
159 Go kit is a library, designed to be imported into a binary package.
160 Vendoring is currently the best way for binary package authors to ensure reliable, reproducible builds.
161 Therefore, we strongly recommend our users use vendoring for all of their dependencies, including Go kit.
162 To avoid compatibility and availability issues, Go kit doesn't vendor its own dependencies, and doesn't recommend use of third-party import proxies.
163
164 There are several tools which make vendoring easier, including [gb][], [glide][], [gvt][], [govendor][], and [vendetta][].
165 In addition, Go kit uses a variety of continuous integration providers to find and fix compatibility problems as soon as they occur.
171166
172167 [gb]: http://getgb.io
168 [glide]: https://github.com/Masterminds/glide
169 [gvt]: https://github.com/FiloSottile/gvt
173170 [govendor]: https://github.com/kardianos/govendor
174 [godep]: https://github.com/tools/godep
175
176 ## API stability policy
177
178 The Go kit project depends on code maintained by others.
179 This includes the Go standard library and sub-repositories and other external libraries.
180 The Go language and standard library provide stability guarantees, but the other external libraries typically do not.
181 [The API Stability RFC](https://github.com/go-kit/kit/tree/master/rfc/rfc007-api-stability.md)
182 proposes a standard policy for package authors to advertise API stability.
183 The Go kit project prefers to depend on code that abides the API stability policy.
171 [vendetta]: https://github.com/dpw/vendetta
184172
185173 ## Related projects
186174
229217 - [Architecting for the Cloud](http://fr.slideshare.net/stonse/architecting-for-the-cloud-using-netflixoss-codemash-workshop-29852233) — Netflix
230218 - [Dapper, a Large-Scale Distributed Systems Tracing Infrastructure](http://research.google.com/pubs/pub36356.html) — Google
231219 - [Your Server as a Function](http://monkey.org/~marius/funsrv.pdf) (PDF) — Twitter
220
+0
-23
rfc/README.md less more
0 # RFCs
1
2 This is a place to hold proposals and design discussions for components of the
3 Go kit.
4
5 Let's keep the process lightweight. To propose a new RFC, make a pull request
6 with a new file, roughly in the same format as the others. To make changes to
7 an existing RFC, make a pull request with your changes, and describe your
8 motivation in the comments.
9
10 RFCs follow the [Chef RFC workflow](https://github.com/chef/chef-rfc/blob/master/rfc000-rfc-process.md#rfc-review-and-workflow).
11
12 ![RFC workflow](http://i.imgur.com/qZvqG2g.png "RFC workflow")
13
14 At this stage of the project, RFC ratification will be done by me (Peter
15 Bourgon). Once we reach a usable MVP, i.e. semantic version 1.0.0, I hope we'll
16 transition to a more community-driven governance model.
17
18 For questions and free-form discussion, please use
19
20 - Mailing list: [go-kit](https://groups.google.com/forum/#!forum/go-kit)
21 - Slack: [gophers.slack.com](https://gophers.slack.com) **#go-kit** ([invite](https://gophersinvite.herokuapp.com/))
22
+0
-83
rfc/rfc000-package-server.md less more
0 ---
1 RFC: 000
2 Author: Matt Heath <matt@mattheath.com>
3 Status: Accepted
4 ---
5
6 # Package Server
7
8 ## Motivation
9
10 [http://peter.bourgon.org/go-kit/#package-server](http://peter.bourgon.org/go-kit/#package-server)
11
12 Package server is probably the biggest and most important component of the toolkit. Ideally, we should be able to write our services as implementations of normal, nominal Go interfaces, and delegate integration with the environment to the server package. The package should encode and enforce conventions for server-side concerns, like health checks, system-wide request tracing, connection management, backpressure and throttling, and so on. For each of those topics, it should provide interfaces for different, pluggable strategies. It should integrate with service discovery, and work equally well over multiple transports. Considerable prior art exists in the form of Finagle, Karyon (Netflix's application service library), and likely many more.
13
14 ## Scope
15
16 Sections with a ★ are considered particularly volatile, and may change significantly in the future.
17
18 ### Endpoints
19
20 * An endpoint is defined as a handler interface which receives a request and returns a response or an error to the client.
21 * A server SHALL have one or more endpoints.
22 * An endpoint SHALL accept a request context and propagate this - allowing access to request-specific values such as the identity of the end user, authorization tokens, and the request's deadline.
23 * An endpoint SHALL respect request cancellation if or when the request's deadline expires.
24 * A server MAY expose information about its endpoints, to allow integration with additional tools.
25
26 ### Contexts
27
28 * Requests SHALL be executed within a request [context](https://blog.golang.org/context), which the server will pass through the request chain.
29
30 ### Throttling & Backpressure
31
32 * A server MAY throttle inbound requests and reject requests from clients based on a number of factors.
33 * A server MAY respond with either an Out Of Capacity error, or a Rate Limit Exceeded error when rejecting requests.
34 * A server MAY limit the total number of concurrent requests it can serve.
35 * A server MAY impose rate limits on specific clients.
36 * Rate limit behaviour MAY range from minimum request intervals, to time based, or leaky bucket algorithms.
37 * A server MAY implement a pluggable throttle interface, allowing richer implementations - such as an implementation which shares information across instances of the service.
38
39 ### SLAs & SLIs
40
41 * A server MAY report its contractual SLA per endpoint to a discovery system, allowing clients to estimate response time.
42 * A server MAY expose its actual SLI per endpoint, allowing third-parties to reason about healthiness.
43
44 ### Healthchecks ★
45
46 * A server SHALL accept registration of healthchecks with a defined interface.
47 * A server MAY register default healthchecks to report the health of built in components of the server.
48 * A server SHALL register an endpoint which can be queried to obtain healthcheck information and status.
49 * A server MAY publish these healthcheck statuses via a pluggable transport.
50
51 ### Service Discovery
52
53 * A server SHALL register itself with a service discovery mechanism on startup.
54 * A server SHALL attempt to deregister itself with a service discovery mechanism on shutdown.
55 * The discovery mechanism SHALL be interchangeable, and satisfy a defined interface, however the mechanism itself is beyond the scope of this RFC.
56
57 ### Request Tracing
58
59 * Requests received by the server which are accompanied with tracing information SHALL respect this information and pass this information onto other sub-requests initiated by clients within the server.
60 * The request tracing mechanism SHALL be interchangeable, and satisfy a defined interface, however the mechanism is beyond the scope of this RFC.
61
62 ### Transport
63
64 * A server SHALL receive and respond to requests via a Transport.
65 * The Transport mechanism SHALL be interchangeable, and satisfy a defined interface, however the mechanism of the transport is beyond the scope of this RFC.
66
67 ### Codec ★
68
69 * A server SHALL encode and decode requests and responses via an interchangeable Codec.
70 * A server MAY support multiple encodings, and use the appropriate Codec as indicated by the transport.
71 * A server MAY indicate to the transport the encoding used, allowing clients to easily decode the response.
72
73 ## Implementation
74
75 To be defined.
76
77 ## Further Reading
78
79 * [Your Server as a Function](http://monkey.org/~marius/funsrv.pdf) - Marius Eriksen
80 * [Finagle](https://twitter.github.io/finagle/) - Twitter
81 * [Karyon](https://github.com/Netflix/karyon) - Netflix
82 * [State of the Art in Microservices](https://www.slideshare.net/adriancockcroft/dockercon-state-of-the-art-in-microservices) - Adrian Cockcroft
+0
-19
rfc/rfc001-package-client.md less more
0 ---
1 RFC: 001
2 Author: Peter Bourgon <peter@bourgon.org>
3 Status: Draft
4 ---
5
6 # package client
7
8 ## Motivation
9
10 http://peter.bourgon.org/go-kit/#package-client
11
12 ## Scope
13
14 To be defined.
15
16 ## Implementation
17
18 To be defined.⏎
+0
-19
rfc/rfc002-package-transport.md less more
0 ---
1 RFC: 002
2 Author: Peter Bourgon <peter@bourgon.org>
3 Status: Draft
4 ---
5
6 # package transport
7
8 ## Motivation
9
10 http://peter.bourgon.org/go-kit/#package-transport
11
12 ## Scope
13
14 To be defined.
15
16 ## Implementation
17
18 To be defined.⏎
+0
-66
rfc/rfc003-package-metrics.md less more
0 ---
1 RFC: 003
2 Author: Peter Bourgon <peter@bourgon.org>
3 Status: Draft
4 ---
5
6 # package metrics
7
8 ## Motivation
9
10 http://peter.bourgon.org/go-kit/#package-metrics
11
12 ## Scope
13
14 - Package metrics SHALL implement Gauges, Counters, and Histograms.
15
16 - Each metric type SHALL allow observations with an unlimited number of key/value field pairs,
17 similar to [package log](https://github.com/go-kit/kit/blob/master/rfc/rfc004-package-log.md).
18
19 - Counter SHALL be an increment-only counter of type uint64.
20
21 - Gauge SHALL be an arbitrarily-settable register of type int64.
22
23 - Histogram SHALL collect observations of type int64.
24
25 - These interfaces SHALL be the primary and exclusive API for metrics.
26
27 - We SHALL provide a variety of implementations of each interface that act as a
28 bridge to different backends: expvar, Graphite, statsd, Prometheus, etc.
29
30 - Each metric backend MAY provide additional value-add behaviors. For example,
31 a backend for Histogram may bucket observations according to quantile and
32 calculate additional, derived statistics.
33
34
35 ## Implementation
36
37 https://github.com/go-kit/kit/tree/master/metrics
38
39 ### Gauge
40
41 ```go
42 type Gauge interface {
43 With(Field) Gauge
44 Set(value int64)
45 Add(delta int64)
46 }
47 ```
48
49 ### Counter
50
51 ```go
52 type Counter interface {
53 With(Field) Counter
54 Add(delta uint64)
55 }
56 ```
57
58 ### Histogram
59
60 ```go
61 type Histogram interface {
62 With(Field) Histogram
63 Observe(int64)
64 }
65 ```
+0
-45
rfc/rfc004-package-log.md less more
0 ---
1 RFC: 004
2 Authors: Brian Knox <bknox@digitalocean.com>
3 Status: Accepted
4 ---
5
6 # package log
7
8 ## Motivation
9
10 http://peter.bourgon.org/go-kit/#package-log
11
12 ## Scope
13
14 ### Key / Value Pairs
15 * Log SHALL use key / value pairs for structured data.
16 * Log MAY preserve type safety when expressing logs in a structured format that supports types
17 * Log SHALL provide a way to set a default k/v set per log context
18 * Log MAY provide some pre-canned default keys (level, time, etc) for convenience
19 * Log MAY allow multiple stored logging contexts each with a different set of k/v pairs.
20 * Log MAY allow per log call adhoc k/v pairs (see Logrus as an example)
21
22 ### Transport
23 * Log SHALL be transport agnostic with pluggable transports.
24 * Log MAY implement io.MultiWriters allowing broadcast of logs over multiple transports.
25 * Log MAY use channels instead
26 * Log MAY provide some built in transports such as syslog and logstash.
27 * Log MAY use encoding.* above the transport level
28 * Log MAY provide configurable back pressure handling strategies in the case of blocked Writers
29
30 ### Formats
31 * Log SHALL be format agnostic by providing an interface for log formatting.
32 * Log MAY provide some built in formatters such as RFC3164, JSON, etc.
33
34 ### Levels
35 * Log MAY provide a set of defined severity levels that can be used (perhaps as a wrapper).
36 * Log SHALL include severity as a k/v pair and allow setting it through the same mechanism as any other k/v pair
37 * Log MAY provide wrapper types as a convenience for setting the severity level
38 * Log MUST NOT intrinsically tie severity levels to program actions - e.g., a call to a specific log level should not call a panic.
39 * Log MAY allow tying program actions such as panic to a log level.
40
41 ## Implementation
42
43 * The initial implementation should be a minimal feature set focused only on the scope of the RFC.
44 * Additional features and niceties may be added as there is demonstrable proof that the features solve real world problems.
+0
-19
rfc/rfc005-service-discovery.md less more
0 ---
1 RFC: 005
2 Author: Peter Bourgon <peter@bourgon.org>
3 Status: Draft
4 ---
5
6 # Service discovery
7
8 ## Motivation
9
10 http://peter.bourgon.org/go-kit/#service-discovery
11
12 ## Scope
13
14 To be defined.
15
16 ## Implementation
17
18 To be defined.⏎
+0
-33
rfc/rfc006-request-tracing.md less more
0 ---
1 RFC: 006
2 Author: Peter Bourgon <peter@bourgon.org>
3 Status: Draft
4 ---
5
6 # Request tracing
7
8 ## Motivation
9
10 [Dapper][]-style request tracing is a necessary introspection tool in any large
11 distributed system. Gokit services should support request tracing, including
12 exposition of traces that are compatible with [Zipkin][].
13
14 [Dapper]: http://research.google.com/pubs/pub36356.html
15 [Zipkin]: http://itszero.github.io/blog/2014/03/03/introduction-to-twitters-zipkin
16
17 ## Scope
18
19 - Request tracing SHALL use Dapper terminology: Trace, Span, Tree, etc.
20
21 - Regardless if request tracing is enabled or disabled, if incoming requests
22 contain trace IDs, package server MUST transparently forward them to
23 downstream services.
24
25 - If request tracing is enabled, and incoming requests do not contain trace
26 IDs, package server SHALL generate relevant IDs and forward them to
27 downstream services.
28
29 ## Implementation
30
31 To be defined.
32
+0
-97
rfc/rfc007-api-stability.md less more
0 ---
1 RFC: 007
2 Author: Andrew Gerrand <adg@golang.org>
3 Status: Accepted
4 ---
5
6 # API Stability Policy
7
8 ## Motivation
9
10 The gokit project depends on code maintained by others.
11 This includes the Go standard library and sub-repositories
12 and other external libraries.
13
14 The Go language and standard library provide stability guarantees, but the
15 other external libraries typically do not. This RFC proposes a standard policy
16 for package authors to advertise API stability.
17
18 The intention is that the gokit project will require that its dependencies
19 adhere to the policy, with the greater goal of improving the Go ecosystem
20 as a whole.
21
22 ## Scope
23
24 This policy is for package authors to provide their users with a promise of API
25 stability.
26 This document is similar to and inspired by the [Go 1 compatibility
27 promise](https://golang.org/doc/go1compat).
28
29 An author declaring their package "API Stable" makes the following promise:
30
31 > We will not change the package's exported API in backward incompatible ways.
32 > Future changes to this package will not break dependent code.
33
34 ### Coverage
35
36 The promise of stability includes:
37
38 * The package name,
39 * Exported type declarations and struct fields (names and types),
40 * Exported function and method names, parameters, and return values,
41 * Exported constant names and values,
42 * Exported variable names and values,
43 * The documented behavior of all exported code.
44
45 ### Exceptions
46
47 * Security. A security issue in the package may come to light whose resolution
48 requires breaking compatibility. We reserve the right to address such
49 security issues.
50
51 * Unspecified behavior. Programs that depend on unspecified
52 behavior may break in future releases.
53
54 * Bugs. If the package has a bug, a program that depends on the buggy behavior
55 may break if the bug is fixed. We reserve the right to fix such bugs.
56
57 * Struct literals. For the addition of features it may be necessary to add
58 fields to exported structs in the package API. Code that uses unkeyed struct
59 literals (such as pkg.T{3, "x"}) to create values of these types would fail
60 to compile after such a change. However, code that uses keyed literals
61 (pkg.T{A: 3, B: "x"}) will continue to compile after such a change. We will
62 update such data structures in a way that allows keyed struct literals to
63 remain compatible, although unkeyed literals may fail to compile. (There are
64 also more intricate cases involving nested data structures or interfaces, but
65 they have the same resolution.) We therefore recommend that composite
66 literals whose type is defined in a separate package should use the keyed
67 notation.
68
69 * Dot imports. If a program imports a package using import . "path", additional
70 names later defined in the imported package may conflict with other names
71 defined in the program. We do not recommend the use of import . outside of
72 tests, and using it may cause a program to fail to compile in the future.
73
74 ### Breaking compatibility
75
76 Should the author wish to break compatibility by redesigning the API the author
77 should create a new package with a new import path.
78
79 ### Awareness
80
81 This text should be present in a file named STABILITY in the repository root.
82
83 ### Enforcement
84
85 Tooling may be devised to check the stability of an API as a package evolves,
86 similar to the api tool used by the Go core.
87
88 The [vet](https://godoc.org/golang.org/x/tools/cmd/vet) tool will already
89 detect "untagged" struct literals; that is, struct literals that will break
90 when new fields are added to the struct.
91
92 ## Implementation
93
94 To be defined.
95
96