Codebase list golang-github-farsightsec-go-nmsg / upstream/0.0_git20190917.04d2174
New upstream version 0.0~git20190917.04d2174 Sascha Steinbiss 3 years ago
58 changed file(s) with 6160 addition(s) and 0 deletion(s). Raw diff Collapse all Expand all
+5
-0
COPYRIGHT less more
0 Copyright 2017,2018 Farsight Security, Inc.
1
2 This Source Code Form is subject to the terms of the Mozilla Public
3 License, v. 2.0. If a copy of the MPL was not distributed with this
4 file, You can obtain one at http://mozilla.org/MPL/2.0/.
+373
-0
LICENSE less more
0 Mozilla Public License Version 2.0
1 ==================================
2
3 1. Definitions
4 --------------
5
6 1.1. "Contributor"
7 means each individual or legal entity that creates, contributes to
8 the creation of, or owns Covered Software.
9
10 1.2. "Contributor Version"
11 means the combination of the Contributions of others (if any) used
12 by a Contributor and that particular Contributor's Contribution.
13
14 1.3. "Contribution"
15 means Covered Software of a particular Contributor.
16
17 1.4. "Covered Software"
18 means Source Code Form to which the initial Contributor has attached
19 the notice in Exhibit A, the Executable Form of such Source Code
20 Form, and Modifications of such Source Code Form, in each case
21 including portions thereof.
22
23 1.5. "Incompatible With Secondary Licenses"
24 means
25
26 (a) that the initial Contributor has attached the notice described
27 in Exhibit B to the Covered Software; or
28
29 (b) that the Covered Software was made available under the terms of
30 version 1.1 or earlier of the License, but not also under the
31 terms of a Secondary License.
32
33 1.6. "Executable Form"
34 means any form of the work other than Source Code Form.
35
36 1.7. "Larger Work"
37 means a work that combines Covered Software with other material, in
38 a separate file or files, that is not Covered Software.
39
40 1.8. "License"
41 means this document.
42
43 1.9. "Licensable"
44 means having the right to grant, to the maximum extent possible,
45 whether at the time of the initial grant or subsequently, any and
46 all of the rights conveyed by this License.
47
48 1.10. "Modifications"
49 means any of the following:
50
51 (a) any file in Source Code Form that results from an addition to,
52 deletion from, or modification of the contents of Covered
53 Software; or
54
55 (b) any new file in Source Code Form that contains any Covered
56 Software.
57
58 1.11. "Patent Claims" of a Contributor
59 means any patent claim(s), including without limitation, method,
60 process, and apparatus claims, in any patent Licensable by such
61 Contributor that would be infringed, but for the grant of the
62 License, by the making, using, selling, offering for sale, having
63 made, import, or transfer of either its Contributions or its
64 Contributor Version.
65
66 1.12. "Secondary License"
67 means either the GNU General Public License, Version 2.0, the GNU
68 Lesser General Public License, Version 2.1, the GNU Affero General
69 Public License, Version 3.0, or any later versions of those
70 licenses.
71
72 1.13. "Source Code Form"
73 means the form of the work preferred for making modifications.
74
75 1.14. "You" (or "Your")
76 means an individual or a legal entity exercising rights under this
77 License. For legal entities, "You" includes any entity that
78 controls, is controlled by, or is under common control with You. For
79 purposes of this definition, "control" means (a) the power, direct
80 or indirect, to cause the direction or management of such entity,
81 whether by contract or otherwise, or (b) ownership of more than
82 fifty percent (50%) of the outstanding shares or beneficial
83 ownership of such entity.
84
85 2. License Grants and Conditions
86 --------------------------------
87
88 2.1. Grants
89
90 Each Contributor hereby grants You a world-wide, royalty-free,
91 non-exclusive license:
92
93 (a) under intellectual property rights (other than patent or trademark)
94 Licensable by such Contributor to use, reproduce, make available,
95 modify, display, perform, distribute, and otherwise exploit its
96 Contributions, either on an unmodified basis, with Modifications, or
97 as part of a Larger Work; and
98
99 (b) under Patent Claims of such Contributor to make, use, sell, offer
100 for sale, have made, import, and otherwise transfer either its
101 Contributions or its Contributor Version.
102
103 2.2. Effective Date
104
105 The licenses granted in Section 2.1 with respect to any Contribution
106 become effective for each Contribution on the date the Contributor first
107 distributes such Contribution.
108
109 2.3. Limitations on Grant Scope
110
111 The licenses granted in this Section 2 are the only rights granted under
112 this License. No additional rights or licenses will be implied from the
113 distribution or licensing of Covered Software under this License.
114 Notwithstanding Section 2.1(b) above, no patent license is granted by a
115 Contributor:
116
117 (a) for any code that a Contributor has removed from Covered Software;
118 or
119
120 (b) for infringements caused by: (i) Your and any other third party's
121 modifications of Covered Software, or (ii) the combination of its
122 Contributions with other software (except as part of its Contributor
123 Version); or
124
125 (c) under Patent Claims infringed by Covered Software in the absence of
126 its Contributions.
127
128 This License does not grant any rights in the trademarks, service marks,
129 or logos of any Contributor (except as may be necessary to comply with
130 the notice requirements in Section 3.4).
131
132 2.4. Subsequent Licenses
133
134 No Contributor makes additional grants as a result of Your choice to
135 distribute the Covered Software under a subsequent version of this
136 License (see Section 10.2) or under the terms of a Secondary License (if
137 permitted under the terms of Section 3.3).
138
139 2.5. Representation
140
141 Each Contributor represents that the Contributor believes its
142 Contributions are its original creation(s) or it has sufficient rights
143 to grant the rights to its Contributions conveyed by this License.
144
145 2.6. Fair Use
146
147 This License is not intended to limit any rights You have under
148 applicable copyright doctrines of fair use, fair dealing, or other
149 equivalents.
150
151 2.7. Conditions
152
153 Sections 3.1, 3.2, 3.3, and 3.4 are conditions of the licenses granted
154 in Section 2.1.
155
156 3. Responsibilities
157 -------------------
158
159 3.1. Distribution of Source Form
160
161 All distribution of Covered Software in Source Code Form, including any
162 Modifications that You create or to which You contribute, must be under
163 the terms of this License. You must inform recipients that the Source
164 Code Form of the Covered Software is governed by the terms of this
165 License, and how they can obtain a copy of this License. You may not
166 attempt to alter or restrict the recipients' rights in the Source Code
167 Form.
168
169 3.2. Distribution of Executable Form
170
171 If You distribute Covered Software in Executable Form then:
172
173 (a) such Covered Software must also be made available in Source Code
174 Form, as described in Section 3.1, and You must inform recipients of
175 the Executable Form how they can obtain a copy of such Source Code
176 Form by reasonable means in a timely manner, at a charge no more
177 than the cost of distribution to the recipient; and
178
179 (b) You may distribute such Executable Form under the terms of this
180 License, or sublicense it under different terms, provided that the
181 license for the Executable Form does not attempt to limit or alter
182 the recipients' rights in the Source Code Form under this License.
183
184 3.3. Distribution of a Larger Work
185
186 You may create and distribute a Larger Work under terms of Your choice,
187 provided that You also comply with the requirements of this License for
188 the Covered Software. If the Larger Work is a combination of Covered
189 Software with a work governed by one or more Secondary Licenses, and the
190 Covered Software is not Incompatible With Secondary Licenses, this
191 License permits You to additionally distribute such Covered Software
192 under the terms of such Secondary License(s), so that the recipient of
193 the Larger Work may, at their option, further distribute the Covered
194 Software under the terms of either this License or such Secondary
195 License(s).
196
197 3.4. Notices
198
199 You may not remove or alter the substance of any license notices
200 (including copyright notices, patent notices, disclaimers of warranty,
201 or limitations of liability) contained within the Source Code Form of
202 the Covered Software, except that You may alter any license notices to
203 the extent required to remedy known factual inaccuracies.
204
205 3.5. Application of Additional Terms
206
207 You may choose to offer, and to charge a fee for, warranty, support,
208 indemnity or liability obligations to one or more recipients of Covered
209 Software. However, You may do so only on Your own behalf, and not on
210 behalf of any Contributor. You must make it absolutely clear that any
211 such warranty, support, indemnity, or liability obligation is offered by
212 You alone, and You hereby agree to indemnify every Contributor for any
213 liability incurred by such Contributor as a result of warranty, support,
214 indemnity or liability terms You offer. You may include additional
215 disclaimers of warranty and limitations of liability specific to any
216 jurisdiction.
217
218 4. Inability to Comply Due to Statute or Regulation
219 ---------------------------------------------------
220
221 If it is impossible for You to comply with any of the terms of this
222 License with respect to some or all of the Covered Software due to
223 statute, judicial order, or regulation then You must: (a) comply with
224 the terms of this License to the maximum extent possible; and (b)
225 describe the limitations and the code they affect. Such description must
226 be placed in a text file included with all distributions of the Covered
227 Software under this License. Except to the extent prohibited by statute
228 or regulation, such description must be sufficiently detailed for a
229 recipient of ordinary skill to be able to understand it.
230
231 5. Termination
232 --------------
233
234 5.1. The rights granted under this License will terminate automatically
235 if You fail to comply with any of its terms. However, if You become
236 compliant, then the rights granted under this License from a particular
237 Contributor are reinstated (a) provisionally, unless and until such
238 Contributor explicitly and finally terminates Your grants, and (b) on an
239 ongoing basis, if such Contributor fails to notify You of the
240 non-compliance by some reasonable means prior to 60 days after You have
241 come back into compliance. Moreover, Your grants from a particular
242 Contributor are reinstated on an ongoing basis if such Contributor
243 notifies You of the non-compliance by some reasonable means, this is the
244 first time You have received notice of non-compliance with this License
245 from such Contributor, and You become compliant prior to 30 days after
246 Your receipt of the notice.
247
248 5.2. If You initiate litigation against any entity by asserting a patent
249 infringement claim (excluding declaratory judgment actions,
250 counter-claims, and cross-claims) alleging that a Contributor Version
251 directly or indirectly infringes any patent, then the rights granted to
252 You by any and all Contributors for the Covered Software under Section
253 2.1 of this License shall terminate.
254
255 5.3. In the event of termination under Sections 5.1 or 5.2 above, all
256 end user license agreements (excluding distributors and resellers) which
257 have been validly granted by You or Your distributors under this License
258 prior to termination shall survive termination.
259
260 ************************************************************************
261 * *
262 * 6. Disclaimer of Warranty *
263 * ------------------------- *
264 * *
265 * Covered Software is provided under this License on an "as is" *
266 * basis, without warranty of any kind, either expressed, implied, or *
267 * statutory, including, without limitation, warranties that the *
268 * Covered Software is free of defects, merchantable, fit for a *
269 * particular purpose or non-infringing. The entire risk as to the *
270 * quality and performance of the Covered Software is with You. *
271 * Should any Covered Software prove defective in any respect, You *
272 * (not any Contributor) assume the cost of any necessary servicing, *
273 * repair, or correction. This disclaimer of warranty constitutes an *
274 * essential part of this License. No use of any Covered Software is *
275 * authorized under this License except under this disclaimer. *
276 * *
277 ************************************************************************
278
279 ************************************************************************
280 * *
281 * 7. Limitation of Liability *
282 * -------------------------- *
283 * *
284 * Under no circumstances and under no legal theory, whether tort *
285 * (including negligence), contract, or otherwise, shall any *
286 * Contributor, or anyone who distributes Covered Software as *
287 * permitted above, be liable to You for any direct, indirect, *
288 * special, incidental, or consequential damages of any character *
289 * including, without limitation, damages for lost profits, loss of *
290 * goodwill, work stoppage, computer failure or malfunction, or any *
291 * and all other commercial damages or losses, even if such party *
292 * shall have been informed of the possibility of such damages. This *
293 * limitation of liability shall not apply to liability for death or *
294 * personal injury resulting from such party's negligence to the *
295 * extent applicable law prohibits such limitation. Some *
296 * jurisdictions do not allow the exclusion or limitation of *
297 * incidental or consequential damages, so this exclusion and *
298 * limitation may not apply to You. *
299 * *
300 ************************************************************************
301
302 8. Litigation
303 -------------
304
305 Any litigation relating to this License may be brought only in the
306 courts of a jurisdiction where the defendant maintains its principal
307 place of business and such litigation shall be governed by laws of that
308 jurisdiction, without reference to its conflict-of-law provisions.
309 Nothing in this Section shall prevent a party's ability to bring
310 cross-claims or counter-claims.
311
312 9. Miscellaneous
313 ----------------
314
315 This License represents the complete agreement concerning the subject
316 matter hereof. If any provision of this License is held to be
317 unenforceable, such provision shall be reformed only to the extent
318 necessary to make it enforceable. Any law or regulation which provides
319 that the language of a contract shall be construed against the drafter
320 shall not be used to construe this License against a Contributor.
321
322 10. Versions of the License
323 ---------------------------
324
325 10.1. New Versions
326
327 Mozilla Foundation is the license steward. Except as provided in Section
328 10.3, no one other than the license steward has the right to modify or
329 publish new versions of this License. Each version will be given a
330 distinguishing version number.
331
332 10.2. Effect of New Versions
333
334 You may distribute the Covered Software under the terms of the version
335 of the License under which You originally received the Covered Software,
336 or under the terms of any subsequent version published by the license
337 steward.
338
339 10.3. Modified Versions
340
341 If you create software not governed by this License, and you want to
342 create a new license for such software, you may create and use a
343 modified version of this License if you rename the license and remove
344 any references to the name of the license steward (except to note that
345 such modified license differs from this License).
346
347 10.4. Distributing Source Code Form that is Incompatible With Secondary
348 Licenses
349
350 If You choose to distribute Source Code Form that is Incompatible With
351 Secondary Licenses under the terms of this version of the License, the
352 notice described in Exhibit B of this License must be attached.
353
354 Exhibit A - Source Code Form License Notice
355 -------------------------------------------
356
357 This Source Code Form is subject to the terms of the Mozilla Public
358 License, v. 2.0. If a copy of the MPL was not distributed with this
359 file, You can obtain one at http://mozilla.org/MPL/2.0/.
360
361 If it is not possible or desirable to put the notice in a particular
362 file, then You may include the notice in a location (such as a LICENSE
363 file in a relevant directory) where a recipient would be likely to look
364 for such a notice.
365
366 You may add additional accurate notices of copyright ownership.
367
368 Exhibit B - "Incompatible With Secondary Licenses" Notice
369 ---------------------------------------------------------
370
371 This Source Code Form is "Incompatible With Secondary Licenses", as
372 defined by the Mozilla Public License, v. 2.0.
+61
-0
README.md less more
0 # Pure Golang NMSG Library
1
2 `go-nmsg` is a pure go implementation of the NMSG container and payload
3 format used by the C (nmsg)[https://github.com/farsightsec/nmsg] toolkit
4 and library.
5
6 ## Synopsis
7
8 import "github.com/farsightsec/go-nmsg"
9 import "github.com/farsightsec/go-nmsg/nmsg_base"
10
11 var r io.Reader
12 var w io.Writer
13 ...
14 input := nmsg.NewInput(r, mtu)
15 output := nmsg.BufferedOutput(w)
16 output.SetMaxSize(nmsg.MaxContainerSize, 0)
17
18 for {
19 payload, err := inp.Recv()
20 if err != nil {
21 if nmsg.IsDataError(err) {
22 continue
23 }
24 break
25 }
26 message := payload.Message()
27
28 switch message.(type) {
29 case *nmsg_base.Dnstap:
30 // process dnstap
31 // write copy to output
32 output.Send(payload)
33 }
34 }
35
36 output.Close()
37
38
39 ## Requirements
40
41 `go-nmsg` requires the following open source libraries:
42
43 "github.com/golang/protobuf/proto"
44 "github.com/dnstap/golang-dnstap"
45
46 ## Limitations
47
48 `go-nmsg` can pack and unpack the protobuf structure of an NMSG payload,
49 and the protobuf structure of the data contained in the payload. It does
50 not implement the full functionality of the C libnmsg message
51 modules, such as:
52
53 * Advanced field types (e.g., a protobuf []byte as an IP address)
54 * Generated fields
55 * Formatting of fields for presentation and JSON output
56
57 Applications needing such functionality in go should use the
58 `cgo-nmsg` package included in this distribution under:
59
60 "github.com/farsightsec/go-nmsg/cgo-nmsg"
+10
-0
cgo-nmsg/README.md less more
0 # Golang bindings for NMSG
1
2 `cgo-nmsg` provides Golang bindings to the C libnmsg library.
3
4 The NMSG network message encapsulation library format is an efficient
5 encoding of typed, structured data into payloads which are packed into
6 containers which can be transmitted over the network or stored to disk.
7 For more information, see https://github.com/farsightsec/nmsg/.
8
9 A pure but limited Golang NMSG library is available with `go-nmsg`.
+107
-0
cgo-nmsg/callbacks.go less more
0 /*
1 * Copyright (c) 2017 by Farsight Security, Inc.
2 *
3 * This Source Code Form is subject to the terms of the Mozilla Public
4 * License, v. 2.0. If a copy of the MPL was not distributed with this
5 * file, You can obtain one at http://mozilla.org/MPL/2.0/.
6 */
7
8 package nmsg
9
10 /*
11 #cgo pkg-config: libnmsg
12 #cgo LDFLAGS: -lnmsg
13 #include <nmsg.h>
14 #include <stdlib.h>
15 */
16 import "C"
17 import (
18 "fmt"
19 "runtime"
20 "sync"
21 "unsafe"
22 )
23
24 type outCbEntry struct {
25 index int
26 Output
27 }
28
29 type inCbEntry struct {
30 index int
31 Input
32 }
33
34 var cbm sync.Mutex
35 var outCbTable []Output
36 var inCbTable []Input
37
38 // The C library may not hold a pointer to a Go variable, but we
39 // need to store enough context in the callback user data to find
40 // the go object which registered the callback. We solve this by
41 // allocating memory on the C side (with C.malloc, C.calloc) and
42 // storing a value in that memory which we can use to look up the
43 // Go value on the Go side.
44 //
45 // The approach we take here is to have a package-global list of
46 // Output and Input, and store the index in the list as a C.int
47 // in C-allocated memory. The location of this memory is returned
48 // as an unsafe.Pointer suitable for passing to the (void *user)
49 // argument of libnmsg callback registration functions.
50
51 func registerOutput(o Output) unsafe.Pointer {
52 cbm.Lock()
53 defer cbm.Unlock()
54 idx := len(outCbTable)
55 outCbTable = append(outCbTable, o)
56 idxptr := C.calloc(C.size_t(1), C.size_t(unsafe.Sizeof(C.int(1))))
57 *(*C.int)(idxptr) = C.int(idx)
58 return idxptr
59 }
60
61 func registerInput(i Input) unsafe.Pointer {
62 cbm.Lock()
63 defer cbm.Unlock()
64 idx := len(inCbTable)
65 inCbTable = append(inCbTable, i)
66 idxptr := C.calloc(C.size_t(1), C.size_t(unsafe.Sizeof(C.int(1))))
67 *(*C.int)(idxptr) = C.int(idx)
68 return idxptr
69 }
70
71 //export outputCallback
72 func outputCallback(msg C.nmsg_message_t, user unsafe.Pointer) {
73 idx := int(*(*C.int)(user))
74 if idx < len(outCbTable) {
75 o := outCbTable[idx]
76 o.Write(messageFromC(msg))
77 return
78 }
79 panic(fmt.Sprintf("outputCallback: invalid index %d", idx))
80 }
81
82 //export inputCallback
83 func inputCallback(msg, user unsafe.Pointer) C.nmsg_res {
84 idx := int(*(*C.int)(user))
85 if idx < len(inCbTable) {
86 i := inCbTable[idx]
87 for {
88 m, err := i.Read()
89
90 if ErrorRetry(err) {
91 continue
92 }
93 if err != nil {
94 *(*C.nmsg_message_t)(msg) = nil
95 if e, ok := err.(nmsgResError); ok {
96 return C.nmsg_res(e)
97 }
98 return C.nmsg_res_failure
99 }
100 runtime.SetFinalizer(m, nil)
101 *(*C.nmsg_message_t)(msg) = m.message
102 return C.nmsg_res_success
103 }
104 }
105 panic(fmt.Sprintf("inputCallback: invalid index %d", idx))
106 }
+262
-0
cgo-nmsg/container.go less more
0 /*
1 * Copyright (c) 2017 by Farsight Security, Inc.
2 *
3 * This Source Code Form is subject to the terms of the Mozilla Public
4 * License, v. 2.0. If a copy of the MPL was not distributed with this
5 * file, You can obtain one at http://mozilla.org/MPL/2.0/.
6 */
7
8 package nmsg
9
10 /*
11 #cgo pkg-config: libnmsg
12 #cgo LDFLAGS: -lnmsg
13 #include <nmsg.h>
14 #include <stdlib.h>
15 */
16 import "C"
17 import (
18 "crypto/rand"
19 "encoding/binary"
20 "io"
21 "runtime"
22 "sync"
23 "unsafe"
24 )
25
26 // A Container is a collection of NMSG payloads with a target size.
27 type Container struct {
28 config ContainerConfig
29 sequenceID uint64
30 sequenceNumber uint32
31 container C.nmsg_container_t
32 }
33
34 // ContainerConfig contains
35 type ContainerConfig struct {
36 Compress bool
37 Sequence bool
38 Size int
39 }
40
41 // NewContainer creates a container with the given target size.
42 func NewContainer(conf *ContainerConfig) *Container {
43 c := &Container{config: *conf, container: C.nmsg_container_init(C.size_t(conf.Size))}
44 runtime.SetFinalizer(c, func(c *Container) {
45 C.nmsg_container_destroy(&c.container)
46 })
47 if conf.Sequence {
48 C.nmsg_container_set_sequence(c.container, C.bool(true))
49 binary.Read(rand.Reader, binary.BigEndian, &c.sequenceID)
50 }
51 return c
52 }
53
54 // ErrorFull returns true if the container is full. If the Container Add()
55 // method returns such an error, the message will need to be added to the
56 // next container.
57 func ErrorFull(err error) bool {
58 t, ok := err.(nmsgResError)
59 return ok && t == nmsgResError(C.nmsg_res_container_full)
60 }
61
62 // ErrorOverfull returns true if the container contains a single payload
63 // and its size is greater than the target size.
64 func ErrorOverfull(err error) bool {
65 t, ok := err.(nmsgResError)
66 return ok && t == nmsgResError(C.nmsg_res_container_overfull)
67 }
68
69 // Add adds the supplied Message to the Container.
70 func (c *Container) Add(m *Message) error {
71 return nmsgError(C.nmsg_container_add(c.container, m.message))
72 }
73
74 // Bytes returns the serialized container and resets the container.
75 func (c *Container) Bytes() []byte {
76 var pbuf *C.uint8_t
77 var pbufLen C.size_t
78 res := C.nmsg_container_serialize(c.container,
79 &pbuf, &pbufLen,
80 C.bool(true),
81 C.bool(c.config.Compress),
82 C.uint32_t(c.sequenceNumber),
83 C.uint64_t(c.sequenceID),
84 )
85 defer C.free(unsafe.Pointer(pbuf))
86 if err := nmsgError(res); err != nil {
87 return nil
88 }
89 c.sequenceID++
90 C.nmsg_container_destroy(&c.container)
91 c.container = C.nmsg_container_init(C.size_t(c.config.Size))
92 if c.config.Sequence {
93 C.nmsg_container_set_sequence(c.container, C.bool(true))
94 }
95
96 return C.GoBytes(unsafe.Pointer(pbuf), C.int(pbufLen))
97 }
98
99 // UnpackContainer returns the messages the container contains.
100 func UnpackContainer(b []byte) ([]*Message, error) {
101 var msgarray *C.nmsg_message_t
102 var nmsgs C.size_t
103
104 res := C.nmsg_container_deserialize(
105 (*C.uint8_t)(unsafe.Pointer(&b[0])),
106 C.size_t(len(b)),
107 &msgarray,
108 &nmsgs)
109 if err := nmsgError(res); err != nil {
110 return nil, err
111 }
112 msgs := make([]*Message, 0, int(nmsgs))
113 p := unsafe.Pointer(msgarray)
114 for i := 0; i < int(nmsgs); i++ {
115 mp := unsafe.Pointer(uintptr(p) + uintptr(i)*unsafe.Sizeof(*msgarray))
116 msgs = append(msgs, messageFromC(*(*C.nmsg_message_t)(mp)))
117 }
118
119 C.free(unsafe.Pointer(msgarray))
120 return msgs, nil
121 }
122
123 // A ContainerOutput writes containers to a generic io.Writer. No fragmentation
124 // of oversize containers is performed.
125 type containerOutput struct {
126 mu sync.Mutex
127 w io.Writer
128 c *Container
129 rate *Rate
130 buffered bool
131 empty bool
132 filtervendor uint32
133 filtermsgtype uint32
134 source uint32
135 operator uint32
136 group uint32
137 }
138
139 // NewContainerOutput creates a ContainerOutput writing to the supplied
140 // io.Writer with the given buffer size.
141 func newContainerOutput(w io.Writer, size int) *containerOutput {
142 return &containerOutput{
143 c: NewContainer(&ContainerConfig{
144 Size: size,
145 Sequence: true,
146 }),
147 buffered: true,
148 empty: true,
149 w: w,
150 }
151 }
152
153 func (co *containerOutput) Write(m *Message) error {
154 for {
155 vid, msgtype := m.GetMsgtype()
156 if co.filtervendor > 0 && co.filtervendor != vid {
157 return nil
158 }
159 if co.filtermsgtype > 0 && co.filtermsgtype != msgtype {
160 return nil
161 }
162 if co.source > 0 {
163 m.SetSource(co.source)
164 }
165 if co.operator > 0 {
166 m.SetOperator(co.operator)
167 }
168 if co.group > 0 {
169 m.SetGroup(co.group)
170 }
171
172 co.mu.Lock()
173 err := co.c.Add(m)
174 if co.buffered && err == nil {
175 co.empty = false
176 co.mu.Unlock()
177 return nil
178 }
179 _, werr := co.w.Write(co.c.Bytes())
180 co.empty = true
181 r := co.rate
182 co.mu.Unlock()
183 if r != nil {
184 r.Sleep()
185 }
186 if werr == nil && ErrorFull(err) {
187 continue
188 }
189 return werr
190 }
191 }
192
193 // SetFilterMsgtype instructs the output to only accept Messages
194 // with the given vendor and messagetype, specified by id.
195 func (co *containerOutput) SetFilterMsgtype(vendor, msgtype uint32) {
196 co.filtervendor = vendor
197 co.filtermsgtype = msgtype
198 }
199
200 // SetFilterMsgtypeByname instructs the output to only accept Messages
201 // with the given vendor and messagetype, specified by name.
202 func (co *containerOutput) SetFilterMsgtypeByname(vendor, msgtype string) {
203 cvendor := C.CString(vendor)
204 cmsgtype := C.CString(msgtype)
205 defer C.free(unsafe.Pointer(cvendor))
206 defer C.free(unsafe.Pointer(cmsgtype))
207 cvid := C.nmsg_msgmod_vname_to_vid(cvendor)
208 co.filtervendor = uint32(cvid)
209 co.filtermsgtype = uint32(C.nmsg_msgmod_mname_to_msgtype(cvid, cmsgtype))
210 }
211
212 func (co *containerOutput) SetRate(r *Rate) {
213 co.mu.Lock()
214 co.rate = r
215 co.mu.Unlock()
216 }
217
218 func (co *containerOutput) SetSource(source uint32) {
219 co.source = source
220 }
221
222 func (co *containerOutput) SetOperator(op uint32) {
223 co.operator = op
224 }
225
226 func (co *containerOutput) SetGroup(group uint32) {
227 co.group = group
228 }
229
230 // Flush writes any buffered output to the underlying writer.
231 func (co *containerOutput) Flush() error {
232 co.mu.Lock()
233 written := false
234 defer func() {
235 r := co.rate
236 co.mu.Unlock()
237 if written && r != nil {
238 r.Sleep()
239 }
240 }()
241 if !co.empty {
242 _, werr := co.w.Write(co.c.Bytes())
243 co.empty = true
244 written = true
245 return werr
246 }
247 return nil
248 }
249
250 // SetBuffered controls whether the ContainerOutput collects
251 // multiple messages into a container (buffered == true, the
252 // default), or sends a container per message (buffered == false).
253 func (co *containerOutput) SetBuffered(buffered bool) {
254 co.buffered = buffered
255 }
256
257 // SetCompression controls whether the containers are compressed
258 // before sending.
259 func (co *containerOutput) SetCompression(compress bool) {
260 co.c.config.Compress = compress
261 }
+39
-0
cgo-nmsg/error.go less more
0 /*
1 * Copyright (c) 2017 by Farsight Security, Inc.
2 *
3 * This Source Code Form is subject to the terms of the Mozilla Public
4 * License, v. 2.0. If a copy of the MPL was not distributed with this
5 * file, You can obtain one at http://mozilla.org/MPL/2.0/.
6 */
7
8 package nmsg
9
10 /*
11 #cgo pkg-config: libnmsg
12 #cgo LDFLAGS: -lnmsg
13 #include <nmsg.h>
14 */
15 import "C"
16
17 // NmsgError encapsulates an error condition
18 type nmsgResError C.nmsg_res
19
20 func (n nmsgResError) Error() string {
21 return C.GoString(C.nmsg_res_lookup(uint32(n)))
22 }
23
24 func nmsgError(res C.nmsg_res) error {
25 if res == C.nmsg_res_success {
26 return nil
27 }
28 return nmsgResError(res)
29 }
30
31 // ErrorRetry returns true if the error indicates that the nmsg
32 // operation should be retried.
33 func ErrorRetry(err error) bool {
34 if ne, ok := err.(nmsgResError); ok {
35 return ne == nmsgResError(C.nmsg_res_again)
36 }
37 return false
38 }
+147
-0
cgo-nmsg/input.go less more
0 /*
1 * Copyright (c) 2017 by Farsight Security, Inc.
2 *
3 * This Source Code Form is subject to the terms of the Mozilla Public
4 * License, v. 2.0. If a copy of the MPL was not distributed with this
5 * file, You can obtain one at http://mozilla.org/MPL/2.0/.
6 */
7
8 package nmsg
9
10 /*
11 #cgo pkg-config: libnmsg
12 #cgo LDFLAGS: -lnmsg
13 #include <stdlib.h>
14 #include <nmsg.h>
15
16 extern nmsg_res inputCallback(nmsg_message_t *msg, void *user);
17
18 nmsg_res input_callback(nmsg_message_t *msg, void *user) {
19 return inputCallback(msg, user);
20 }
21 */
22 import "C"
23 import (
24 "io"
25 "net"
26 "os"
27 "unsafe"
28 )
29
30 // An Input is a source of NMSG payloads (Messages).
31 type Input interface {
32 // Read returns a Message or nil, and an error if any.
33 Read() (*Message, error)
34
35 // SetFilterMsgtype instructs the input to discard all Messages
36 // not of the given vendor id and msgtype, specified by number.
37 SetFilterMsgtype(vendor, msgtype uint32)
38
39 // SetFilterMsgtypeByname instructs the input to discard all Messages
40 // not of the given vendor id and msgtype, specified by name.
41 SetFilterMsgtypeByname(vendor, msgtype string)
42
43 // SetFilterSource instructs the input to discard all Messages not
44 // from the supplied source.
45 SetFilterSource(source uint32)
46
47 // SetFilterOperator instructs the input to discard all Messages not
48 // from the supplied operator.
49 SetFilterOperator(operator uint32)
50
51 // SetFilterGroup instructs the input to discard all Messages not
52 // in the supplied group.
53 SetFilterGroup(group uint32)
54 }
55
56 // NmsgInput is an Input managed by libnmsg. It satisfies
57 // the Input interface, and has
58 type nmsgInput struct {
59 file *os.File
60 input C.nmsg_input_t
61 }
62
63 func (i *nmsgInput) Read() (*Message, error) {
64 var msg C.nmsg_message_t
65 res := C.nmsg_input_read(i.input, &msg)
66 if res == C.nmsg_res_success {
67 return messageFromC(msg), nil
68 }
69 return nil, nmsgError(res)
70 }
71
72 func (i *nmsgInput) SetFilterMsgtype(vid, msgtype uint32) {
73 C.nmsg_input_set_filter_msgtype(i.input, C.uint(vid), C.uint(msgtype))
74 }
75
76 func (i *nmsgInput) SetFilterMsgtypeByname(vendor, msgtype string) {
77 cname := C.CString(vendor)
78 ctype := C.CString(msgtype)
79 C.nmsg_input_set_filter_msgtype_byname(i.input, cname, ctype)
80 C.free(unsafe.Pointer(cname))
81 C.free(unsafe.Pointer(ctype))
82 }
83
84 func (i *nmsgInput) SetFilterSource(source uint32) {
85 C.nmsg_input_set_filter_source(i.input, C.uint(source))
86 }
87
88 func (i *nmsgInput) SetFilterOperator(operator uint32) {
89 C.nmsg_input_set_filter_operator(i.input, C.uint(operator))
90 }
91
92 func (i *nmsgInput) SetFilterGroup(group uint32) {
93 C.nmsg_input_set_filter_group(i.input, C.uint(group))
94 }
95
96 // NewInput creates a new Input from an io.Reader.
97 // Currently, the reader must be a *net.UDPConn or a *os.File
98 func NewInput(r io.Reader) Input {
99 switch r := r.(type) {
100 case *net.UDPConn:
101 f, err := r.File()
102 if err != nil {
103 return nil
104 }
105 return &nmsgInput{f, C.nmsg_input_open_sock(C.int(f.Fd()))}
106 case *os.File:
107 return &nmsgInput{r, C.nmsg_input_open_file(C.int(r.Fd()))}
108 default:
109 return nil
110 // return &containerReader{Reader: r}
111 }
112 }
113
114 // NewCallbackInput creates an NmsgInput which calls the supplied InputFunc.
115 func NewCallbackInput(i InputFunc) Input {
116 return &nmsgInput{
117 file: nil,
118 input: C.nmsg_input_open_callback(C.nmsg_cb_message_read(C.input_callback), registerInput(i)),
119 }
120 }
121
122 // An InputFunc is a function with the same signature as Input.Read(), usable
123 // directly as an Input.
124 //
125 // When used directly as an Input, only the Read() method is implemented. All
126 // others are no-ops. If the functionality of the other methods is desired,
127 // the InputFunc can be passed to NewCallbackInput.
128 type InputFunc func() (*Message, error)
129
130 // Read calls the underlying function to return the next message.
131 func (i InputFunc) Read() (*Message, error) { return i() }
132
133 // SetFilterMsgtype satisfies the Input interface with a no-op
134 func (i InputFunc) SetFilterMsgtype(vendor, msgtype uint32) {}
135
136 // SetFilterMsgtypeByname satisfies the Input interface with a no-op
137 func (i InputFunc) SetFilterMsgtypeByname(vendor, msgtype string) {}
138
139 // SetFilterSource satisfies the Input interface with a no-op
140 func (i InputFunc) SetFilterSource(source uint32) {}
141
142 // SetFilterOperator satisfies the Input interface with a no-op
143 func (i InputFunc) SetFilterOperator(operator uint32) {}
144
145 // SetFilterGroup satisfies the Input interface with a no-op
146 func (i InputFunc) SetFilterGroup(group uint32) {}
+97
-0
cgo-nmsg/io.go less more
0 /*
1 * Copyright (c) 2017 by Farsight Security, Inc.
2 *
3 * This Source Code Form is subject to the terms of the Mozilla Public
4 * License, v. 2.0. If a copy of the MPL was not distributed with this
5 * file, You can obtain one at http://mozilla.org/MPL/2.0/.
6 */
7
8 package nmsg
9
10 /*
11 #cgo pkg-config: libnmsg
12 #cgo LDFLAGS: -lnmsg
13 #include <stdlib.h>
14 #include <nmsg.h>
15 */
16 import "C"
17 import "unsafe"
18
19 // IO is a handle to a libnmsg io loop connecting one or more Inputs
20 // with one ore more Outputs.
21 type IO struct {
22 nmsgIO C.nmsg_io_t
23 }
24
25 // NewIO creates and returns a new IO
26 func NewIO() *IO {
27 io := C.nmsg_io_init()
28 if io != nil {
29 return &IO{io}
30 }
31 return nil
32 }
33
34 // AddInputChannel opens an NMSG channel and adds it as an Input to the
35 // IO.
36 func (io *IO) AddInputChannel(channel string) error {
37 cchan := C.CString(channel)
38 res := C.nmsg_io_add_input_channel(io.nmsgIO, cchan, nil)
39 C.free(unsafe.Pointer(cchan))
40 return nmsgError(res)
41 }
42
43 // AddInputSockSpec opens one or more sockets based on the sockspec
44 // (add/port ,or addr/lowport..highport) and adds it to the IO
45 // as an input.
46 func (io *IO) AddInputSockSpec(sockspec string) error {
47 css := C.CString(sockspec)
48 res := C.nmsg_io_add_input_sockspec(io.nmsgIO, css, nil)
49 C.free(unsafe.Pointer(css))
50 return nmsgError(res)
51 }
52
53 // AddInput adds a separately created Input to the IO as an input.
54 func (io *IO) AddInput(i Input) error {
55 ni, ok := i.(*nmsgInput)
56 if !ok {
57 ni = NewCallbackInput(i.Read).(*nmsgInput)
58 }
59 return nmsgError(C.nmsg_io_add_input(io.nmsgIO, ni.input, nil))
60 }
61
62 // AddOutput adds a separately created Output to the IO as an output.
63 func (io *IO) AddOutput(o Output) error {
64 nout, ok := o.(*nmsgOutput)
65 if !ok {
66 nout = NewCallbackOutput(o.Write).(*nmsgOutput)
67 }
68 return nmsgError(C.nmsg_io_add_output(io.nmsgIO, nout.output, nil))
69 }
70
71 // SetMirrored controls whether the IO mirrors output to all outputs
72 // (mirrored = true) or stripes its output across all outputs.
73 func (io *IO) SetMirrored(mirrored bool) {
74 if mirrored {
75 C.nmsg_io_set_output_mode(io.nmsgIO, C.nmsg_io_output_mode_mirror)
76 return
77 }
78 C.nmsg_io_set_output_mode(io.nmsgIO, C.nmsg_io_output_mode_stripe)
79 }
80
81 // SetDebug sets the debug print level of the underlying io.
82 // Larger numbers are more verbose.
83 func (io *IO) SetDebug(debug int) {
84 C.nmsg_io_set_debug(io.nmsgIO, C.int(debug))
85 }
86
87 // Run starts the IO loop, returning when it is finished or broken
88 // with Break()
89 func (io *IO) Run() error {
90 return nmsgError(C.nmsg_io_loop(io.nmsgIO))
91 }
92
93 // Break stops the IO main loop.
94 func (io *IO) Break() {
95 C.nmsg_io_breakloop(io.nmsgIO)
96 }
+380
-0
cgo-nmsg/message.go less more
0 /*
1 * Copyright (c) 2017 by Farsight Security, Inc.
2 *
3 * This Source Code Form is subject to the terms of the Mozilla Public
4 * License, v. 2.0. If a copy of the MPL was not distributed with this
5 * file, You can obtain one at http://mozilla.org/MPL/2.0/.
6 */
7
8 package nmsg
9
10 /*
11 #cgo pkg-config: libnmsg
12 #cgo LDFLAGS: -lnmsg
13 #include <stdlib.h>
14 #include <nmsg.h>
15
16 const char *endline="\n";
17 unsigned flag_repeated = NMSG_MSGMOD_FIELD_REPEATED;
18 */
19 import "C"
20 import (
21 "fmt"
22 "net"
23 "runtime"
24 "unsafe"
25 )
26
27 // MessageMod something something
28 type MessageMod struct {
29 nmsgMsgMod C.nmsg_msgmod_t
30 }
31
32 // MessageModLookupByName something something
33 func MessageModLookupByName(vname, mname string) *MessageMod {
34 vstr := C.CString(vname)
35 mstr := C.CString(mname)
36 defer C.free(unsafe.Pointer(vstr))
37 defer C.free(unsafe.Pointer(mstr))
38 return &MessageMod{C.nmsg_msgmod_lookup_byname(vstr, mstr)}
39 }
40
41 // MessageModLookup something something
42 func MessageModLookup(v, m uint32) *MessageMod {
43 return &MessageMod{C.nmsg_msgmod_lookup(C.uint(v), C.uint(m))}
44 }
45
46 // A Message is a unit of NMSG data.
47 type Message struct {
48 message C.nmsg_message_t
49 }
50
51 // NewMessage initializes a message of a type given by
52 // the supplied MessageMod
53 func NewMessage(mod *MessageMod) *Message {
54 return messageFromC(C.nmsg_message_init(mod.nmsgMsgMod))
55 }
56
57 // NewMessageFromPayload encapsulates a byte buffer in a payload with
58 // the supplied vendor and message type.
59 func NewMessageFromPayload(payload []byte, vendor uint32, msgtype uint32) *Message {
60 Csiz := C.size_t(len(payload))
61 // C.CString allocates a buffer to hold the copy of payload
62 // built by string. This buffer is passed to nmsg_message_from_raw_payload,
63 // which takes ownership of the buffer. It will be freed when
64 // nmsg_message_destroy() is called by the Message finalizer.
65 Cbuf := unsafe.Pointer(C.CString(string(payload)))
66 return messageFromC(C.nmsg_message_from_raw_payload(
67 C.unsigned(vendor), C.unsigned(msgtype),
68 (*C.uint8_t)(Cbuf), Csiz, nil))
69 }
70
71 func messageDestroy(m *Message) {
72 C.nmsg_message_destroy(&m.message)
73 }
74
75 // GetMsgtype returns the vendor and payload type of the message.
76 func (msg *Message) GetMsgtype() (vendor, msgtype uint32) {
77 vendor = uint32(C.nmsg_message_get_vid(msg.message))
78 msgtype = uint32(C.nmsg_message_get_msgtype(msg.message))
79 return
80 }
81
82 // Source returns the source id of the message, or zero if the source id
83 // is not set.
84 func (msg *Message) Source() uint32 {
85 return uint32(C.nmsg_message_get_source(msg.message))
86 }
87
88 // SetSource sets the source id of the message.
89 func (msg *Message) SetSource(source uint32) {
90 C.nmsg_message_set_source(msg.message, C.uint32_t(source))
91 }
92
93 // Operator returns the operator id of the message, or zero if the operator id
94 // is not set.
95 func (msg *Message) Operator() uint32 {
96 return uint32(C.nmsg_message_get_operator(msg.message))
97 }
98
99 // SetOperator sets the operator id of the message.
100 func (msg *Message) SetOperator(operator uint32) {
101 C.nmsg_message_set_operator(msg.message, C.uint32_t(operator))
102 }
103
104 // Group returns the group id of the message, or zero if the group id
105 // is not set.
106 func (msg *Message) Group() uint32 {
107 return uint32(C.nmsg_message_get_group(msg.message))
108 }
109
110 // SetGroup sets the group id of the message.
111 func (msg *Message) SetGroup(group uint32) {
112 C.nmsg_message_set_group(msg.message, C.uint32_t(group))
113 }
114
115 func messageFromC(message C.nmsg_message_t) *Message {
116 msg := &Message{message}
117 runtime.SetFinalizer(msg, messageDestroy)
118 return msg
119 }
120
121 // MarshalJSON formats a JSON representation of the Message
122 func (msg *Message) MarshalJSON() ([]byte, error) {
123 var jsonCstr *C.char
124 err := nmsgError(C.nmsg_message_to_json(msg.message, &jsonCstr))
125 defer C.free(unsafe.Pointer(jsonCstr))
126 if err != nil {
127 return nil, err
128 }
129 return []byte(C.GoString(jsonCstr)), nil
130 }
131
132 // UnmarshalJSON parses a JSON representation of the Message
133 func (msg *Message) UnmarshalJSON(b []byte) error {
134 jsonCstr := C.CString(string(b))
135 defer C.free(unsafe.Pointer(jsonCstr))
136 return nmsgError(C.nmsg_message_from_json(jsonCstr, &msg.message))
137 }
138
139 // MarshalText converts a Message to presentation format.
140 func (msg *Message) MarshalText() ([]byte, error) {
141 var presCstr *C.char
142 err := nmsgError(C.nmsg_message_to_pres(msg.message, &presCstr, C.endline))
143 defer C.free(unsafe.Pointer(presCstr))
144 if err != nil {
145 return nil, err
146 }
147 return []byte(C.GoString(presCstr)), nil
148 }
149
150 // Enum contains both the numeric Value and the string Description of
151 // an enumerated NMSG field value.
152 type Enum struct {
153 Value uint32
154 Description string
155 }
156
157 type fieldValue struct {
158 typ C.nmsg_msgmod_field_type
159 buf unsafe.Pointer
160 size C.int
161 }
162
163 func (msg *Message) getFieldValue(name string, idx int) (fv fieldValue, err error) {
164 var Csize C.size_t
165
166 Cname := C.CString(name)
167 defer C.free(unsafe.Pointer(Cname))
168
169 Cidx := C.uint(uint(idx))
170
171 res := C.nmsg_message_get_field_type(msg.message, Cname, &fv.typ)
172 if err = nmsgError(res); err != nil {
173 return
174 }
175
176 res = C.nmsg_message_get_field(msg.message, Cname, Cidx, &fv.buf, &Csize)
177 if err = nmsgError(res); err != nil {
178 return
179 }
180
181 fv.size = C.int(Csize)
182 return
183 }
184
185 func (msg *Message) setFieldValue(name string, idx int, buf unsafe.Pointer, size int) error {
186 Cname := C.CString(name)
187 defer C.free(unsafe.Pointer(Cname))
188
189 Cidx := C.uint(uint(idx))
190 Csize := C.size_t(size)
191 return nmsgError(C.nmsg_message_set_field(msg.message, Cname, Cidx,
192 (*C.uint8_t)(buf), Csize))
193 }
194
195 // GetUintField retrieves the named field of a unsigned int type from a Message.
196 // If the field has an enumerated type, the numeric value is retrieved.
197 func (msg *Message) GetUintField(name string, idx int) (uint64, error) {
198 fv, err := msg.getFieldValue(name, idx)
199 if err != nil {
200 return 0, err
201 }
202
203 switch fv.typ {
204 case C.nmsg_msgmod_ft_uint16:
205 return uint64(*(*uint16)(fv.buf)), nil
206 case C.nmsg_msgmod_ft_uint32:
207 fallthrough
208 case C.nmsg_msgmod_ft_enum:
209 return uint64(*(*uint32)(fv.buf)), nil
210 case C.nmsg_msgmod_ft_uint64:
211 return *(*uint64)(fv.buf), nil
212 default:
213 return 0, fmt.Errorf("Field %s not of uint type", name)
214 }
215
216 }
217
218 // SetUintField sets the value of a field of type uint16, uint32, or uint64.
219 // The bitsize parameter specifies which type, and must be 16, 32, or 64
220 func (msg *Message) SetUintField(name string, idx, bitsize int, val uint64) error {
221 switch bitsize {
222 case 16:
223 v := uint16(val)
224 return msg.setFieldValue(name, idx, unsafe.Pointer(&v), bitsize)
225 case 32:
226 v := uint32(val)
227 return msg.setFieldValue(name, idx, unsafe.Pointer(&v), bitsize)
228 case 64:
229 v := uint64(val)
230 return msg.setFieldValue(name, idx, unsafe.Pointer(&v), bitsize)
231 default:
232 return fmt.Errorf("Invalid bitsize %d", bitsize)
233 }
234 }
235
236 // GetIntField retrieves the value of a named field of integer type from
237 // a Message.
238 func (msg *Message) GetIntField(name string, idx int) (int64, error) {
239 fv, err := msg.getFieldValue(name, idx)
240 if err != nil {
241 return 0, err
242 }
243
244 switch fv.typ {
245 case C.nmsg_msgmod_ft_int16:
246 return int64(*(*int16)(fv.buf)), nil
247 case C.nmsg_msgmod_ft_int32:
248 return int64(*(*int32)(fv.buf)), nil
249 case C.nmsg_msgmod_ft_int64:
250 return *(*int64)(fv.buf), nil
251 default:
252 return 0, fmt.Errorf("Field %s not of int type", name)
253 }
254 }
255
256 // SetIntField sets the value of an int16, int32, or int64 field in the message.
257 // The bitsize field specifies which size, and must by 16, 32, or 64
258 func (msg *Message) SetIntField(name string, idx, bitsize int, val int64) error {
259 switch bitsize {
260 case 16:
261 v := int16(val)
262 return msg.setFieldValue(name, idx, unsafe.Pointer(&v), bitsize)
263 case 32:
264 v := int32(val)
265 return msg.setFieldValue(name, idx, unsafe.Pointer(&v), bitsize)
266 case 64:
267 v := int64(val)
268 return msg.setFieldValue(name, idx, unsafe.Pointer(&v), bitsize)
269 default:
270 return fmt.Errorf("Invalid bitsize %d", bitsize)
271 }
272 }
273
274 // GetBytesField retrieves a field of type bytes from a Message.
275 func (msg *Message) GetBytesField(name string, idx int) ([]byte, error) {
276 fv, err := msg.getFieldValue(name, idx)
277 if err != nil {
278 return nil, err
279 }
280 if fv.typ != C.nmsg_msgmod_ft_bytes {
281 return nil, fmt.Errorf("Field %s not of bytes type", name)
282 }
283 return C.GoBytes(fv.buf, fv.size), nil
284 }
285
286 // SetBytesField sets the value of a bytes field in a Message
287 func (msg *Message) SetBytesField(name string, idx int, val []byte) error {
288 Cbuf := unsafe.Pointer(&val[0])
289 return msg.setFieldValue(name, idx, Cbuf, len(val))
290 }
291
292 // GetStringField retrieves the value of a string field in a Message
293 func (msg *Message) GetStringField(name string, idx int) (string, error) {
294 fv, err := msg.getFieldValue(name, idx)
295 if err != nil {
296 return "", err
297 }
298 return C.GoStringN((*C.char)(fv.buf), fv.size), nil
299 }
300
301 // SetStringField sets the value of a string field in a Message
302 func (msg *Message) SetStringField(name string, idx int, val string) error {
303 b := []byte(val)
304 Cbuf := unsafe.Pointer(&b[0])
305 return msg.setFieldValue(name, idx, Cbuf, len(val))
306 }
307
308 // GetIPField retrieves the value of an IP field in a Message
309 func (msg *Message) GetIPField(name string, idx int) (net.IP, error) {
310 fv, err := msg.getFieldValue(name, idx)
311 if err != nil {
312 return nil, err
313 }
314 if fv.typ != C.nmsg_msgmod_ft_ip {
315 return nil, fmt.Errorf("Field %s not of iptype", name)
316 }
317 return net.IP(C.GoBytes(fv.buf, fv.size)), nil
318 }
319
320 // SetIPField sets the value of an IP field in a Message
321 func (msg *Message) SetIPField(name string, idx int, val net.IP) error {
322 Cbuf := unsafe.Pointer(&val[0])
323 return msg.setFieldValue(name, idx, Cbuf, len(val))
324 }
325
326 // GetDoubleField retrieves the value of a double field in a Message
327 func (msg *Message) GetDoubleField(name string, idx int) (float64, error) {
328 fv, err := msg.getFieldValue(name, idx)
329 if err != nil {
330 return 0, err
331 }
332 if fv.typ != C.nmsg_msgmod_ft_double {
333 return 0, fmt.Errorf("Field %s is not of double type", name)
334 }
335 return *(*float64)(fv.buf), nil
336 }
337
338 // SetDoubleField sets the value of a double field in a Message
339 func (msg *Message) SetDoubleField(name string, idx int, val float64) error {
340 Cbuf := unsafe.Pointer(&val)
341 return msg.setFieldValue(name, idx, Cbuf, 8)
342 }
343
344 // GetEnumField returns the string description of a Message field
345 // with an enumerated type.
346 func (msg *Message) GetEnumField(name string, idx int) (string, error) {
347 enumValue, err := msg.GetUintField(name, idx)
348 if err != nil {
349 return "", err
350 }
351
352 Cname := C.CString(name)
353 defer C.free(unsafe.Pointer(Cname))
354 var Ename *C.char
355 res := C.nmsg_message_enum_value_to_name(
356 msg.message, Cname, C.unsigned(enumValue),
357 &Ename,
358 )
359 if err = nmsgError(res); err != nil {
360 return "", err
361 }
362 return C.GoString(Ename), nil
363 }
364
365 // SetEnumField sets the value of the named Message field to the value
366 // corresponding to the supplied description.
367 func (msg *Message) SetEnumField(name string, idx int, vname string) error {
368 Cname := C.CString(name)
369 defer C.free(unsafe.Pointer(Cname))
370 Cvname := C.CString(vname)
371 defer C.free(unsafe.Pointer(Cvname))
372
373 var v C.uint
374 res := C.nmsg_message_enum_name_to_value(msg.message, Cname, Cvname, &v)
375 if err := nmsgError(res); err != nil {
376 return err
377 }
378 return msg.SetUintField(name, idx, 32, uint64(v))
379 }
+42
-0
cgo-nmsg/nmsg.go less more
0 /*
1 * Copyright (c) 2017 by Farsight Security, Inc.
2 *
3 * This Source Code Form is subject to the terms of the Mozilla Public
4 * License, v. 2.0. If a copy of the MPL was not distributed with this
5 * file, You can obtain one at http://mozilla.org/MPL/2.0/.
6 */
7
8 package nmsg
9
10 /*
11 #cgo pkg-config: libnmsg
12 #cgo LDFLAGS: -lnmsg
13 #include <nmsg.h>
14
15 int nmsg_wbufsiz_min = NMSG_WBUFSZ_MIN;
16 int nmsg_wbufsiz_max = NMSG_WBUFSZ_MAX;
17 int nmsg_wbufsiz_ether = NMSG_WBUFSZ_ETHER;
18 int nmsg_wbufsiz_jumbo = NMSG_WBUFSZ_JUMBO;
19 */
20 import "C"
21
22 func init() {
23 if C.nmsg_init() != C.nmsg_res_success {
24 panic("failed to initialize nmsg library")
25 }
26 }
27
28 // Buffer Size constants from libnmsg
29 var (
30 BufferSizeMax = int(C.nmsg_wbufsiz_max)
31 BufferSizeMin = int(C.nmsg_wbufsiz_min)
32 BufferSizeEther = int(C.nmsg_wbufsiz_ether)
33 BufferSizeJumbo = int(C.nmsg_wbufsiz_jumbo)
34 )
35
36 // SetDebug sets the debug print level for the nmsg library.
37 // Debugging messages are sent to stderr. Higher debug values
38 // increase verbosity.
39 func SetDebug(debug int) {
40 C.nmsg_set_debug(C.int(debug))
41 }
+193
-0
cgo-nmsg/output.go less more
0 /*
1 * Copyright (c) 2017 by Farsight Security, Inc.
2 *
3 * This Source Code Form is subject to the terms of the Mozilla Public
4 * License, v. 2.0. If a copy of the MPL was not distributed with this
5 * file, You can obtain one at http://mozilla.org/MPL/2.0/.
6 */
7
8 package nmsg
9
10 /*
11 #cgo pkg-config: libnmsg
12 #cgo LDFLAGS: -lnmsg
13 #include <stdlib.h>
14 #include <nmsg.h>
15
16 extern void outputCallback(nmsg_message_t, void *);
17
18 void output_callback(nmsg_message_t msg, void *user) {
19 outputCallback(msg, user);
20 }
21 */
22 import "C"
23 import (
24 "io"
25 "net"
26 "os"
27 "unsafe"
28 )
29
30 // An Output is a destination for NMSG data (Messages)
31 type Output interface {
32 // Write sends the supplied message to the Output.
33 Write(*Message) error
34
35 // SetBuffered controls whether the output buffers Messages into containers
36 // before sending them. NmsgOutputs are buffered by default, but low volume
37 // sources may choose to turn this off to reduce latency.
38 SetBuffered(bool)
39
40 // SetCompression controls whether the output compresses
41 // the container data prior to sending.
42 SetCompression(bool)
43
44 // Flush writes any buffered data to the Output.
45 Flush() error
46
47 // SetFilterMsgtype instructs the output to discard all Messages
48 // not of the supplied vendor and type, specified by number.
49 SetFilterMsgtype(vendor, msgtype uint32)
50
51 // SetFilterMsgtypeByname instructs the output to discard all Messages
52 // not of the supplied vendor and type, specified by name.
53 SetFilterMsgtypeByname(vendor, msgtype string)
54
55 // SetRate sets an output rate limit. The rate is specified
56 // in containers per second, and is checked every freq pauses.
57 // The freq parameter should be about 10-15% of the rate.
58 SetRate(rate *Rate)
59
60 // SetSource instructs the output to set the source parameter
61 // of all outbound messages to the supplied value.
62 SetSource(source uint32)
63
64 // SetOperator instructs the output to set the operator parameter
65 // of all outbound messages to the supplied value.
66 SetOperator(group uint32)
67
68 // SetGroup instructs the output to set the group parameter
69 // of all outbound messages to the supplied value.
70 SetGroup(group uint32)
71 }
72
73 // An NmsgOutput is an output managed by the nmsg library.
74 type nmsgOutput struct {
75 file *os.File
76 rate *Rate
77 output C.nmsg_output_t
78 }
79
80 func (o *nmsgOutput) Write(m *Message) error {
81 return nmsgError(C.nmsg_output_write(o.output, m.message))
82 }
83
84 func (o *nmsgOutput) SetBuffered(buffered bool) {
85 C.nmsg_output_set_buffered(o.output, C.bool(buffered))
86 }
87
88 func (o *nmsgOutput) SetFilterMsgtype(vid, msgtype uint32) {
89 C.nmsg_output_set_filter_msgtype(o.output, C.uint(vid), C.uint(msgtype))
90 }
91
92 func (o *nmsgOutput) SetFilterMsgtypeByname(vendor, msgtype string) {
93 cname := C.CString(vendor)
94 ctype := C.CString(msgtype)
95 C.nmsg_output_set_filter_msgtype_byname(o.output, cname, ctype)
96 C.free(unsafe.Pointer(cname))
97 C.free(unsafe.Pointer(ctype))
98 }
99
100 func (o *nmsgOutput) SetRate(r *Rate) {
101 if r == nil {
102 C.nmsg_output_set_rate(o.output, nil)
103 } else {
104 C.nmsg_output_set_rate(o.output, r.rate)
105 }
106 // keep a reference to avoid calling the finalizer
107 o.rate = r
108 }
109
110 func (o *nmsgOutput) SetSource(source uint32) {
111 C.nmsg_output_set_source(o.output, C.uint(source))
112 }
113
114 func (o *nmsgOutput) SetOperator(operator uint32) {
115 C.nmsg_output_set_operator(o.output, C.uint(operator))
116 }
117
118 func (o *nmsgOutput) SetGroup(group uint32) {
119 C.nmsg_output_set_group(o.output, C.uint(group))
120 }
121
122 func (o *nmsgOutput) SetCompression(compress bool) {
123 C.nmsg_output_set_zlibout(o.output, C.bool(compress))
124 }
125
126 func (o *nmsgOutput) Flush() error {
127 return nmsgError(C.nmsg_output_flush(o.output))
128 }
129
130 // NewOutput creates an output writing to w, with target
131 // container size of bufsiz. The Writer currently must be a
132 // *os.File or *net.UDPConn.
133 func NewOutput(w io.Writer, bufsiz int) Output {
134 switch w := w.(type) {
135 case *net.UDPConn:
136 f, err := w.File()
137 if err != nil {
138 return nil
139 }
140 return &nmsgOutput{f, nil, C.nmsg_output_open_sock(C.int(f.Fd()), C.size_t(bufsiz))}
141 case *os.File:
142 return &nmsgOutput{w, nil, C.nmsg_output_open_file(C.int(w.Fd()), C.size_t(bufsiz))}
143 default:
144 return newContainerOutput(w, bufsiz)
145 }
146 }
147
148 // NewCallbackOutput creates an NmsgOutput which calls o.Send()
149 // on every message.
150 func NewCallbackOutput(o OutputFunc) Output {
151 return &nmsgOutput{
152 file: nil,
153 output: C.nmsg_output_open_callback(C.nmsg_cb_message(C.output_callback), registerOutput(o)),
154 }
155 }
156
157 // An OutputFunc is a function with the same signature as Output.Write, usable
158 // directly as an Output.
159 //
160 // When used directly as an Output, only the Write() method is defined. All others
161 // are no-ops.
162 type OutputFunc func(*Message) error
163
164 // Write calls the underlying function with the supplied message
165 func (o OutputFunc) Write(m *Message) error { return o(m) }
166
167 // Flush satisfies the Output interface with a no-op
168 func (o OutputFunc) Flush() error { return nil }
169
170 // SetBuffered satisfies the Output interface with a no-op
171 func (o OutputFunc) SetBuffered(bool) {}
172
173 // SetCompression satisfies the Output interface with a no-op
174 func (o OutputFunc) SetCompression(bool) {}
175
176 // SetFilterMsgtype satisfies the Output interface with a no-op
177 func (o OutputFunc) SetFilterMsgtype(vendor, msgtype uint32) {}
178
179 // SetFilterMsgtypeByname satisfies the Output interface with a no-op
180 func (o OutputFunc) SetFilterMsgtypeByname(vendor, msgtype string) {}
181
182 // SetRate satisfies the Output interface with a no-op
183 func (o OutputFunc) SetRate(r *Rate) {}
184
185 // SetSource satisfies the Output interface with a no-op
186 func (o OutputFunc) SetSource(source uint32) {}
187
188 // SetOperator satisfies the Output interface with a no-op
189 func (o OutputFunc) SetOperator(group uint32) {}
190
191 // SetGroup satisfies the Output interface with a no-op
192 func (o OutputFunc) SetGroup(group uint32) {}
+39
-0
cgo-nmsg/rate.go less more
0 /*
1 * Copyright (c) 2017 by Farsight Security, Inc.
2 *
3 * This Source Code Form is subject to the terms of the Mozilla Public
4 * License, v. 2.0. If a copy of the MPL was not distributed with this
5 * file, You can obtain one at http://mozilla.org/MPL/2.0/.
6 */
7
8 package nmsg
9
10 /*
11 #cgo pkg-config: libnmsg
12 #cgo LDFLAGS: -lnmsg
13 #include <nmsg.h>
14 #include <stdlib.h>
15 */
16 import "C"
17 import "runtime"
18
19 // A Rate provides Rate limiting across one or more outputs.
20 type Rate struct{ rate C.nmsg_rate_t }
21
22 // NewRate initializes and returns a rate context. The rate parameter
23 // specifies the target rate of packets (containers and fragments) sent
24 // on all outputs using the Rate. The freq parameter specifies how often
25 // (in packets) to check the rate limit.
26 func NewRate(rate, freq uint) *Rate {
27 r := &Rate{C.nmsg_rate_init(C.uint(rate), C.uint(freq))}
28 runtime.SetFinalizer(r, func(r *Rate) {
29 C.nmsg_rate_destroy(&r.rate)
30 })
31 return r
32 }
33
34 // Sleep pauses for an appropriate amount of time to maintain the given
35 // output rate.
36 func (r *Rate) Sleep() {
37 C.nmsg_rate_sleep(r.rate)
38 }
+49
-0
cgo-nmsg/xs.go less more
0 // +build libxs
1
2 /*
3 * Copyright (c) 2017 by Farsight Security, Inc.
4 *
5 * This Source Code Form is subject to the terms of the Mozilla Public
6 * License, v. 2.0. If a copy of the MPL was not distributed with this
7 * file, You can obtain one at http://mozilla.org/MPL/2.0/.
8 */
9
10 package nmsg
11
12 /*
13 #cgo pkg-config: libnmsg libxs
14 #cgo LDFLAGS: -lnmsg -lxs
15 #include <stdlib.h>
16 #include <nmsg.h>
17 #include <xs/xs.h>
18 */
19 import "C"
20 import "unsafe"
21
22 var xsContext unsafe.Pointer
23
24 func init() {
25 xsContext = C.xs_init()
26 }
27
28 // NewXSInput opens an Input reading from the given XS endpoint.
29 func NewXSInput(xep string) Input {
30 cxep := C.CString(xep)
31 defer C.free(unsafe.Pointer(cxep))
32 inp := C.nmsg_input_open_xs_endpoint(xsContext, cxep)
33 if inp == nil {
34 return nil
35 }
36 return &nmsgInput{input: inp}
37 }
38
39 // NewXSOutput creates an output writing to the given XS endpoint.
40 func NewXSOutput(xep string, bufsiz int) Output {
41 cxep := C.CString(xep)
42 defer C.free(unsafe.Pointer(cxep))
43 outp := C.nmsg_output_open_xs_endpoint(xsContext, cxep, C.size_t(bufsiz))
44 if outp == nil {
45 return nil
46 }
47 return &nmsgOutput{output: outp}
48 }
+363
-0
container.go less more
0 /*
1 * Copyright (c) 2017,2018 by Farsight Security, Inc.
2 *
3 * This Source Code Form is subject to the terms of the Mozilla Public
4 * License, v. 2.0. If a copy of the MPL was not distributed with this
5 * file, You can obtain one at http://mozilla.org/MPL/2.0/.
6 */
7
8 package nmsg
9
10 import (
11 "bytes"
12 "encoding/binary"
13 "errors"
14 "fmt"
15 "io"
16 "math/rand"
17
18 "github.com/golang/protobuf/proto"
19 )
20
21 const (
22 nmsgVersion = 2
23 nmsgFlagZlib = 1
24 nmsgFlagFragment = 2
25 headerSize = 10
26 )
27
28 var (
29 nmsgMagic = [4]byte{'N', 'M', 'S', 'G'}
30 errBadMagic = errors.New("Bad NMSG Magic Number")
31 u32max uint32 = (1 << 31)
32 containerOverhead = 10
33 fragmentOverhead = 10 + 4 + proto.Size(
34 &NmsgFragment{
35 Id: &u32max,
36 Current: &u32max,
37 Last: &u32max,
38 Crc: &u32max,
39 })
40 )
41
42 type containerHeader struct {
43 Magic [4]byte
44 Flags, Version byte
45 Length uint32
46 }
47
48 // isCompressed() and isFragmented() are helper functions for readability.
49 func (h *containerHeader) isCompressed() bool {
50 return h.Flags&nmsgFlagZlib != 0
51 }
52
53 func (h *containerHeader) isFragmented() bool {
54 return h.Flags&nmsgFlagFragment != 0
55 }
56
57 // A Container encapsulates an Nmsg envelope, and maintains metadata for
58 // sizing containers as payloads are added.
59 type Container struct {
60 // Maximum size of a container. AddPayload attempts to keep the container
61 // under this size.
62 maxSize int
63 // Maximum size of fragment or container. Any containers larger than this
64 // will be fragmented by WriteTo.
65 writeSize int
66 // If true, compress container contents before writing.
67 compress bool
68 // If true, container was populated from compressed data
69 // This is primarily used in fragment reassembly to detect whether the
70 // fragmented data was compressed prior to fragmentation.
71 isCompressed bool
72 // If nonzero, an estimate of the effectiveness of compression, expressed
73 // as compressedSize / uncompressedSize. Default: 0.5
74 compressionRatio float32
75 // The current estimated size of the serialized data, before compression
76 size int
77 Nmsg
78 *NmsgFragment
79 }
80
81 // NewContainer creates a new empty NMSG container.
82 func NewContainer() *Container {
83 c := &Container{size: containerOverhead}
84 c.SetMaxSize(0, 0)
85 return c
86 }
87
88 // SetMaxSize sets the maximum size (including Marshaling overhead,
89 // container header, and anticipated compression ratio) of a container.
90 // AddPayload attempts to keep the container within this size.
91 //
92 // writeSize specifies the maximum size of containers or fragments.
93 // Containers larger than writeSize will be written as fragments instead
94 // of single containers.
95 //
96 // A writeSize value of 0 is treated as equal to size.
97 func (c *Container) SetMaxSize(size, writeSize int) {
98 if size < MinContainerSize {
99 size = MinContainerSize
100 }
101 if size > MaxContainerSize {
102 size = MaxContainerSize
103 }
104 if writeSize < size {
105 writeSize = size
106 }
107 c.maxSize = size
108 c.writeSize = writeSize
109 }
110
111 // SetCompression instructs WriteTo to write containers with compressed
112 // (if true) or uncompressed (if false) contents.
113 func (c *Container) SetCompression(compress bool) {
114 c.compress = compress
115 }
116
117 // SetCompressionRatio sets an estimated compression ratio for the data.
118 // The default value is 2.0
119 func (c *Container) SetCompressionRatio(ratio float32) {
120 c.compressionRatio = ratio
121 }
122
123 // SetSequenced sets or unsets sequencing on the container stream.
124 // The sequence number is updated every time WriteTo() is called.
125 func (c *Container) SetSequenced(sequenced bool) {
126 if sequenced {
127 seqid := uint64(rand.Uint32()) << 32
128 seqid |= uint64(rand.Uint32())
129 c.Nmsg.SequenceId = proto.Uint64(seqid)
130 c.Nmsg.Sequence = proto.Uint32(0)
131 } else {
132 c.Nmsg.SequenceId = nil
133 c.Nmsg.Sequence = nil
134 }
135 }
136
137 // AddPayload adds the supplied NmsgPayload to the Container if possible.
138 //
139 // The return value 'full' is true if the container is full and needs to
140 // be emptied with WriteTo().
141 //
142 // The return value 'ok' is true if the payload was successfully added to
143 // the container, otherwise, AddPayload() must be called again after WriteTo().
144 //
145 // Both ok and full may be true if the payload is larger than the container's
146 // MaxSize, or if the container is full after adding the payload.
147 func (c *Container) AddPayload(p *NmsgPayload) (ok, full bool) {
148 limit := c.maxSize
149 if c.compress {
150 if c.compressionRatio > 0 {
151 limit = int(float32(limit) * c.compressionRatio)
152 } else {
153 limit *= 2
154 }
155 }
156 ps := p.payloadSize()
157 if c.size+ps >= limit {
158 full = true
159 }
160
161 if !full || c.size == containerOverhead || c.size+ps == limit {
162 ok = true
163 c.size += ps
164 c.Nmsg.Payloads = append(c.Nmsg.Payloads, p)
165 c.Nmsg.PayloadCrcs = append(c.Nmsg.PayloadCrcs, nmsgCRC(p.Payload))
166 }
167
168 return
169 }
170
171 // Reset discards payloads and crcs from the Container
172 func (c *Container) Reset() {
173 c.Nmsg.Payloads = c.Nmsg.Payloads[:0]
174 c.Nmsg.PayloadCrcs = c.Nmsg.PayloadCrcs[:0]
175 c.NmsgFragment = nil
176 }
177
178 // WriteTo writes the Container to Writer w. If the
179 // container requires fragmentation, it will call
180 // w.Write() multiple times.
181 func (c *Container) WriteTo(w io.Writer) (int64, error) {
182 var buf bytes.Buffer
183
184 header := containerHeader{
185 Magic: nmsgMagic,
186 Version: nmsgVersion,
187 }
188
189 defer c.Reset()
190
191 b, err := proto.Marshal(&c.Nmsg)
192 if err != nil {
193 return 0, err
194 }
195
196 if c.compress {
197 b, err = zbufDeflate(b)
198 if err != nil {
199 return 0, err
200 }
201 header.Flags |= nmsgFlagZlib
202 }
203
204 header.Length = uint32(len(b))
205 if c.Nmsg.Sequence != nil {
206 *c.Nmsg.Sequence++
207 }
208 c.size = containerOverhead
209
210 if len(b)+containerOverhead > c.writeSize {
211 return c.writeFragments(w, b)
212 }
213
214 if err = binary.Write(&buf, binary.BigEndian, &header); err != nil {
215 return 0, err
216 }
217
218 if _, err = buf.Write(b); err != nil {
219 return 0, err
220 }
221
222 return buf.WriteTo(w)
223 }
224
225 func (c *Container) writeFragments(w io.Writer, b []byte) (int64, error) {
226 header := containerHeader{
227 Magic: nmsgMagic,
228 Version: nmsgVersion,
229 Flags: nmsgFlagFragment,
230 }
231
232 if c.compress {
233 header.Flags |= nmsgFlagZlib
234 }
235
236 fragSize := c.writeSize - fragmentOverhead
237 lastFrag := len(b) / fragSize
238 fragID := rand.Uint32()
239
240 nf := NmsgFragment{
241 Id: proto.Uint32(fragID),
242 Current: proto.Uint32(uint32(0)),
243 Last: proto.Uint32(uint32(lastFrag)),
244 Crc: proto.Uint32(nmsgCRC(b)),
245 }
246
247 var written int64
248 for i := 0; i <= lastFrag; i++ {
249 var buf bytes.Buffer
250
251 fblen := len(b)
252 if fblen > fragSize {
253 fblen = fragSize
254 }
255
256 *nf.Current = uint32(i)
257 nf.Fragment = b[:fblen]
258 b = b[fblen:]
259
260 fbytes, err := proto.Marshal(&nf)
261 if err != nil {
262 return written, err
263 }
264
265 header.Length = uint32(len(fbytes))
266 if err = binary.Write(&buf, binary.BigEndian, header); err != nil {
267 return written, err
268 }
269
270 if _, err = buf.Write(fbytes); err != nil {
271 return written, err
272 }
273
274 n, err := buf.WriteTo(w)
275 if err != nil {
276 return written, err
277 }
278 written += n
279 }
280 return written, nil
281 }
282
283 // ReadFrom Reads a Container from the given io.Reader. It returns the
284 // number of container bytes read on success.
285 func (c *Container) ReadFrom(r io.Reader) (n int64, err error) {
286 /*
287 * The bytes.Buffer Grow() method may panic with ErrTooLarge.
288 * We catch this panic (and any other error panic()s and return
289 * an error.
290 */
291 defer func() {
292 if r := recover(); r != nil {
293 var ok bool
294 if err, ok = r.(error); !ok {
295 err = fmt.Errorf("nmsg.Container ReadFrom: panic %v", r)
296 }
297 }
298 }()
299 var buf bytes.Buffer
300 var h containerHeader
301 if n, err = io.CopyN(&buf, r, headerSize); err != nil {
302 return n, err
303 }
304
305 err = binary.Read(&buf, binary.BigEndian, &h)
306 if err != nil {
307 return n, &dataError{err}
308 }
309 if h.Magic != nmsgMagic {
310 return 0, &dataError{errBadMagic}
311 }
312
313 buf.Grow(int(h.Length))
314 if n, err = io.CopyN(&buf, r, int64(h.Length)); err != nil {
315 return int64(buf.Len()), err
316 }
317
318 // err = c.fromBytesHeader(buf.Bytes(), &h)
319 err = c.fromNmsgBytes(buf.Bytes(), h.isCompressed(), h.isFragmented())
320 if err != nil {
321 err = &dataError{err}
322 }
323 return int64(buf.Len()), err
324 }
325
326 // FromBytes parses the given buffer as an NMSG container and stores
327 // the result in the receiver *Container.
328 func (c *Container) FromBytes(b []byte) error {
329 var h containerHeader
330 buf := bytes.NewBuffer(b)
331 err := binary.Read(buf, binary.BigEndian, &h)
332 if err != nil {
333 return err
334 }
335 if h.Magic != nmsgMagic {
336 return errBadMagic
337 }
338
339 return c.fromNmsgBytes(buf.Bytes(), h.isCompressed(), h.isFragmented())
340 }
341
342 // fromNmsgBytes parses the contents (b) of an NMSG container, according to
343 // whether the container contents are compressed, fragmented, or both.
344 func (c *Container) fromNmsgBytes(b []byte, compressed, fragmented bool) error {
345 var err error
346 cbytes := b
347 c.isCompressed = compressed
348 if compressed {
349 cbytes, err = zbufInflate(b)
350 if err != nil {
351 return err
352 }
353 }
354
355 if fragmented {
356 c.NmsgFragment = &NmsgFragment{}
357 return proto.Unmarshal(cbytes, c.NmsgFragment)
358 }
359
360 c.NmsgFragment = nil
361 return proto.Unmarshal(cbytes, &c.Nmsg)
362 }
+178
-0
container_test.go less more
0 /*
1 * Copyright (c) 2017 by Farsight Security, Inc.
2 *
3 * This Source Code Form is subject to the terms of the Mozilla Public
4 * License, v. 2.0. If a copy of the MPL was not distributed with this
5 * file, You can obtain one at http://mozilla.org/MPL/2.0/.
6 */
7
8 package nmsg_test
9
10 // These tests verify container compatibility between C libnmsg (wrapped in
11 // cgo-nmsg) and go-nmsg, both with and without compression.
12
13 import (
14 "bytes"
15 "log"
16 "testing"
17
18 cnmsg "github.com/farsightsec/go-nmsg/cgo-nmsg"
19 "github.com/farsightsec/go-nmsg"
20 "github.com/farsightsec/go-nmsg/nmsg_base"
21 )
22
23 func compare(a, b []byte) bool {
24 if len(a) != len(b) {
25 return false
26 }
27 for i := range a {
28 if a[i] != b[i] {
29 return false
30 }
31 }
32 return true
33 }
34
35 func TestContainerGoCgoUnpack(t *testing.T) {
36 b := new(bytes.Buffer)
37 c := nmsg.NewContainer()
38 c.SetMaxSize(nmsg.MinContainerSize, nmsg.MinContainerSize)
39 c.AddPayload(testGoMessage(100))
40 c.WriteTo(b)
41
42 m, err := cnmsg.UnpackContainer(b.Bytes())
43 if err != nil {
44 t.Fatal(err)
45 }
46
47 if len(m) != 1 {
48 t.Fatalf("message count mismatch %d != 1", len(m))
49 }
50
51 if checkCgoMessage(m[0], 100) {
52 return
53 }
54
55 t.Error("payload mismatch")
56 }
57
58 func TestContainerGoCgoUnpackCompress(t *testing.T) {
59 b := new(bytes.Buffer)
60 c := nmsg.NewContainer()
61 c.SetCompression(true)
62 c.SetMaxSize(nmsg.MinContainerSize, nmsg.MinContainerSize)
63 c.AddPayload(testGoMessage(100))
64 c.WriteTo(b)
65
66 byt := b.Bytes()
67
68 m, err := cnmsg.UnpackContainer(byt)
69 if err != nil {
70 t.Fatal(err)
71 }
72
73 if len(m) != 1 {
74 t.Fatalf("message count mismatch %d != 1", len(m))
75 }
76
77 if checkCgoMessage(m[0], 100) {
78 return
79 }
80
81 t.Error("payload mismatch")
82 }
83
84 func testCgoMessage(size int) *cnmsg.Message {
85 mod := cnmsg.MessageModLookupByName("base", "encode")
86 if mod == nil {
87 log.Fatal("module not found")
88 }
89 msg := cnmsg.NewMessage(mod)
90 if err := msg.SetEnumField("type", 0, "TEXT"); err != nil {
91 log.Fatal(err)
92 }
93
94 if err := msg.SetBytesField("payload", 0, make([]byte, size)); err != nil {
95 log.Fatal(err)
96 }
97 return msg
98 }
99
100 func checkCgoMessage(m *cnmsg.Message, size int) bool {
101 b, err := m.GetBytesField("payload", 0)
102 if err != nil {
103 return false
104 }
105 return compare(b, make([]byte, size))
106 }
107
108 func testGoMessage(size int) *nmsg.NmsgPayload {
109 m := new(nmsg_base.Encode)
110 m.Payload = make([]byte, size)
111 m.Type = nmsg_base.EncodeType_TEXT.Enum()
112 p, err := nmsg.Payload(m)
113 if err != nil {
114 log.Fatal(err)
115 }
116 return p
117 }
118
119 func checkGoMessage(m nmsg.Message, size int) bool {
120 enc, ok := m.(*nmsg_base.Encode)
121
122 if !ok {
123 log.Printf("type mismatch: %T != *nmsg_base.Encode", m)
124 return false
125 }
126 return compare(enc.GetPayload(), make([]byte, size))
127 }
128
129 func TestContainerCgoGoUnpack(t *testing.T) {
130 c := cnmsg.NewContainer(&cnmsg.ContainerConfig{
131 Size: cnmsg.BufferSizeMin,
132 })
133 c.Add(testCgoMessage(100))
134
135 i := nmsg.NewInput(bytes.NewReader(c.Bytes()), cnmsg.BufferSizeMin)
136 p, err := i.Recv()
137 if err != nil {
138 t.Fatal(err)
139 }
140
141 m, err := p.Message()
142 if err != nil {
143 t.Fatal(err)
144 }
145
146 if checkGoMessage(m, 100) {
147 return
148 }
149
150 t.Error("payload mismatch")
151 }
152
153 func TestContainerCgoGoUnpackCompress(t *testing.T) {
154 c := cnmsg.NewContainer(&cnmsg.ContainerConfig{
155 Size: cnmsg.BufferSizeMin,
156 Compress: true,
157 })
158 c.Add(testCgoMessage(100))
159
160 byt := c.Bytes()
161 i := nmsg.NewInput(bytes.NewReader(byt), cnmsg.BufferSizeMin)
162 p, err := i.Recv()
163 if err != nil {
164 t.Fatal(err)
165 }
166
167 m, err := p.Message()
168 if err != nil {
169 t.Fatal(err)
170 }
171
172 if checkGoMessage(m, 100) {
173 return
174 }
175
176 t.Error("payload mismatch")
177 }
+44
-0
endian.go less more
0 /*
1 * Copyright (c) 2017 by Farsight Security, Inc.
2 *
3 * This Source Code Form is subject to the terms of the Mozilla Public
4 * License, v. 2.0. If a copy of the MPL was not distributed with this
5 * file, You can obtain one at http://mozilla.org/MPL/2.0/.
6 */
7
8 package nmsg
9
10 // The nmsg C library renders checksums in network byte order before presenting
11 // them to the protobuf-c library as uint32 values. While Go's encoding/binary
12 // library can format and parse uint32 values as BigEndian or LittleEndian byte
13 // arrays, this is not sufficient to calculate an integer that will represent
14 // a BigEndian (network) byte array in the host's native byte order. This
15 // requires determining the host's byte order, a task which Go's type system
16 // makes cumbersome.
17 //
18 // This file uses the "unsafe" package to defeat Go's type system for the
19 // purposes of determining whether the package is running on a BigEndian or
20 // LittleEndian machine, and uses this information to implement htonl.
21
22 import (
23 "encoding/binary"
24 "unsafe"
25 )
26
27 var hostEndian binary.ByteOrder
28
29 func init() {
30 n := uint32(1)
31 b := *(*[4]byte)(unsafe.Pointer(&n))
32 if b[0] == 1 {
33 hostEndian = binary.LittleEndian
34 } else {
35 hostEndian = binary.BigEndian
36 }
37 }
38
39 func htonl(n uint32) uint32 {
40 var buf [4]byte
41 hostEndian.PutUint32(buf[:], n)
42 return binary.BigEndian.Uint32(buf[:])
43 }
+136
-0
input.go less more
0 /*
1 * Copyright (c) 2017,2018 by Farsight Security, Inc.
2 *
3 * This Source Code Form is subject to the terms of the Mozilla Public
4 * License, v. 2.0. If a copy of the MPL was not distributed with this
5 * file, You can obtain one at http://mozilla.org/MPL/2.0/.
6 */
7
8 package nmsg
9
10 import (
11 "bufio"
12 "fmt"
13 "io"
14 "time"
15 )
16
17 // An Input is a source of NMSG Payloads.
18 type Input interface {
19 // Recv() returns the next Nmsg Payload from the input,
20 // blocking if none is available.
21 Recv() (*NmsgPayload, error)
22 // Stats() returns interface statistics
23 Stats() *InputStatistics
24 }
25
26 // InputStatistics holds useful metrics for input performance.
27 type InputStatistics struct {
28 // Count of total container received, including fragments
29 InputContainers uint64
30 // Count of total bytes received and processed
31 InputBytes uint64
32 // Count of containers marked lost by sequence tracking
33 LostContainers uint64
34 // Count of fragment containers received
35 InputFragments uint64
36 // Count of fragments expired from cache
37 ExpiredFragments uint64
38 // Count of containers dropped due to incomplete fragments
39 PartialContainers uint64
40 }
41
42 type dataError struct{ error }
43
44 func (d *dataError) Error() string { return d.error.Error() }
45
46 // IsDataError returns true of the supplied error is an error unpacking
47 // or decoding the NMSG data rather than an I/O error with the input.
48 func IsDataError(err error) bool {
49 _, ok := err.(*dataError)
50 return ok
51 }
52
53 type input struct {
54 r io.Reader
55 n Nmsg
56 fcache *fragCache
57 scache *seqCache
58 stats InputStatistics
59 }
60
61 func (i *input) Stats() *InputStatistics {
62 res := &InputStatistics{}
63 *res = i.stats
64 return res
65 }
66
67 // NewInput constructs an input from the supplied Reader.
68 // The size parameter sizes the input buffer, and should
69 // be greater than the maximum anticipated container size
70 // for datagram inputs.
71 func NewInput(r io.Reader, size int) Input {
72 return &input{
73 r: bufio.NewReaderSize(r, size),
74 n: Nmsg{},
75 fcache: newFragmentCache(2 * time.Minute),
76 scache: newSequenceCache(2 * time.Minute),
77 }
78 }
79
80 type checksumError struct {
81 calc, wire uint32
82 }
83
84 func (c *checksumError) Error() string {
85 return fmt.Sprintf("checksum mismatch: %x != %x", c.calc, c.wire)
86 }
87
88 func (i *input) Recv() (*NmsgPayload, error) {
89 for len(i.n.Payloads) == 0 {
90 var c Container
91 n, err := c.ReadFrom(i.r)
92 if err != nil {
93 return nil, err
94 }
95 if n == 0 {
96 return nil, io.EOF
97 }
98
99 i.stats.InputBytes += uint64(n)
100
101 if c.NmsgFragment != nil {
102 i.stats.InputFragments++
103 var b []byte
104 if b = i.fcache.Insert(c.NmsgFragment); b == nil {
105 continue
106 }
107 err = c.fromNmsgBytes(b, c.isCompressed, false)
108 if err != nil {
109 return nil, &dataError{err}
110 }
111 }
112
113 i.stats.InputContainers++
114 i.stats.LostContainers += uint64(i.scache.Update(&c.Nmsg))
115 i.scache.Expire()
116 i.n = c.Nmsg
117 }
118 ccount, fcount := i.fcache.Expire()
119 i.stats.PartialContainers += uint64(ccount)
120 i.stats.ExpiredFragments += uint64(fcount)
121 p := i.n.Payloads[0]
122 i.n.Payloads = i.n.Payloads[1:]
123
124 var err error
125 if len(i.n.PayloadCrcs) > 0 {
126 wire := i.n.PayloadCrcs[0]
127 calc := nmsgCRC(p.Payload)
128 if wire != calc {
129 err = &dataError{&checksumError{calc, wire}}
130 }
131 i.n.PayloadCrcs = i.n.PayloadCrcs[1:]
132 }
133
134 return p, err
135 }
+104
-0
input_frag.go less more
0 /*
1 * Copyright (c) 2018 by Farsight Security, Inc.
2 *
3 * This Source Code Form is subject to the terms of the Mozilla Public
4 * License, v. 2.0. If a copy of the MPL was not distributed with this
5 * file, You can obtain one at http://mozilla.org/MPL/2.0/.
6 */
7
8 package nmsg
9
10 // NMSG Fragment Cache.
11
12 import (
13 "bytes"
14 "container/list"
15 "sort"
16 "time"
17 )
18
19 type fragCacheEntry struct {
20 lastUsed time.Time
21 id uint32
22 frags fragList
23 }
24
25 // fragList implements sort.Interface to support sorting fragments on
26 // their "Current" field prior to reassembly.
27 type fragList []*NmsgFragment
28
29 func (fl fragList) Len() int { return len(fl) }
30 func (fl fragList) Less(i, j int) bool { return fl[i].GetCurrent() < fl[j].GetCurrent() }
31 func (fl fragList) Swap(i, j int) { fl[i], fl[j] = fl[j], fl[i] }
32
33 type fragCache struct {
34 expiry time.Duration
35 idmap map[uint32]*list.Element
36 lru *list.List
37 }
38
39 func newFragmentCache(expiry time.Duration) *fragCache {
40 return &fragCache{
41 expiry: expiry,
42 idmap: make(map[uint32]*list.Element),
43 lru: list.New(),
44 }
45 }
46
47 // Expire too-old entries from the fragment cache, returning the number
48 // of incomplete containers and fragments dropped.
49 func (fc *fragCache) Expire() (containers, frags int) {
50 for fc.lru.Len() > 0 {
51 lruent := fc.lru.Front()
52 ent := lruent.Value.(*fragCacheEntry)
53 if time.Since(ent.lastUsed) <= fc.expiry {
54 break
55 }
56 containers++
57 frags += len(ent.frags)
58 fc.lru.Remove(lruent)
59 delete(fc.idmap, ent.id)
60 }
61 return
62 }
63
64 // Inserts a fragment into the cache. If the fragment completes a fragmented
65 // container, Insert returns the reassembled container body. Otherwise, returns
66 // nil.
67 func (fc *fragCache) Insert(f *NmsgFragment) []byte {
68 id := f.GetId()
69 lruent, ok := fc.idmap[id]
70 if !ok {
71 fc.idmap[id] = fc.lru.PushBack(
72 &fragCacheEntry{
73 lastUsed: time.Now(),
74 id: id,
75 frags: fragList{f},
76 })
77 return nil
78 }
79
80 ent := lruent.Value.(*fragCacheEntry)
81 for i := range ent.frags {
82 if ent.frags[i].GetCurrent() == f.GetCurrent() {
83 /* duplicate fragment */
84 return nil
85 }
86 }
87 ent.frags = append(ent.frags, f)
88 if ent.frags.Len() <= int(f.GetLast()) {
89 ent.lastUsed = time.Now()
90 fc.lru.MoveToBack(lruent)
91 return nil
92 }
93 fc.lru.Remove(lruent)
94 delete(fc.idmap, id)
95
96 /* sort and reassemble fragments */
97 sort.Sort(ent.frags)
98 var b bytes.Buffer
99 for i := range ent.frags {
100 b.Write(ent.frags[i].GetFragment())
101 }
102 return b.Bytes()
103 }
+91
-0
input_seq.go less more
0 /*
1 * Copyright (c) 2018 by Farsight Security, Inc.
2 *
3 * This Source Code Form is subject to the terms of the Mozilla Public
4 * License, v. 2.0. If a copy of the MPL was not distributed with this
5 * file, You can obtain one at http://mozilla.org/MPL/2.0/.
6 */
7
8 package nmsg
9
10 import (
11 "container/list"
12 "time"
13 )
14
15 type seqCacheEntry struct {
16 lastUsed time.Time
17 seqid uint64
18 nextSeq uint32
19 }
20
21 type seqCache struct {
22 expiry time.Duration
23 idmap map[uint64]*list.Element
24 lru *list.List
25 }
26
27 func newSequenceCache(expiry time.Duration) *seqCache {
28 return &seqCache{
29 expiry: expiry,
30 idmap: make(map[uint64]*list.Element),
31 lru: list.New(),
32 }
33 }
34
35 const maxDrop = 1048576
36
37 func (sc *seqCache) Update(n *Nmsg) (missed int) {
38 if n.Sequence == nil || n.SequenceId == nil {
39 return
40 }
41 seqid := n.GetSequenceId()
42 lruent, ok := sc.idmap[seqid]
43 if !ok {
44 sc.idmap[seqid] = sc.lru.PushBack(
45 &seqCacheEntry{
46 lastUsed: time.Now(),
47 seqid: seqid,
48 nextSeq: n.GetSequence() + 1,
49 })
50 return 0
51 }
52 seq := n.GetSequence()
53 ent := lruent.Value.(*seqCacheEntry)
54
55 ent.lastUsed = time.Now()
56 sc.lru.MoveToBack(lruent)
57
58 if seq == ent.nextSeq {
59 ent.nextSeq++
60 return 0
61 }
62
63 if seq > ent.nextSeq {
64 if seq-ent.nextSeq < maxDrop {
65 missed = int(seq - ent.nextSeq)
66 }
67 ent.nextSeq = seq + 1
68 return missed
69 }
70
71 delta := int64(int64(seq) + (1 << 32) - int64(ent.nextSeq))
72 if delta < maxDrop {
73 missed = int(delta)
74 }
75
76 ent.nextSeq = seq + 1
77 return missed
78 }
79
80 func (sc *seqCache) Expire() {
81 for sc.lru.Len() > 0 {
82 lruent := sc.lru.Front()
83 ent := lruent.Value.(*seqCacheEntry)
84 if time.Since(ent.lastUsed) <= sc.expiry {
85 break
86 }
87 sc.lru.Remove(lruent)
88 delete(sc.idmap, ent.seqid)
89 }
90 }
+153
-0
input_test.go less more
0 /*
1 * Copyright (c) 2017 by Farsight Security, Inc.
2 *
3 * This Source Code Form is subject to the terms of the Mozilla Public
4 * License, v. 2.0. If a copy of the MPL was not distributed with this
5 * file, You can obtain one at http://mozilla.org/MPL/2.0/.
6 */
7
8 package nmsg_test
9
10 import (
11 "bytes"
12 "io"
13 "math"
14 "testing"
15
16 "github.com/farsightsec/go-nmsg"
17 )
18
19 func testReader(t *testing.T, n, size, mtu int) io.Reader {
20 buf := new(bytes.Buffer)
21 // nw := nmsg.NewWriter(w, mtu)
22 o := nmsg.BufferedOutput(buf)
23 o.SetMaxSize(mtu, 0)
24 o.SetSequenced(true)
25
26 p, err := nmsg.Payload(testMessage(size))
27 if err != nil {
28 t.Error(err.Error())
29 return nil
30 }
31
32 for i := 0; i < n; i++ {
33 o.Send(p)
34 }
35
36 o.Close()
37
38 t.Logf("testReader: buf = %d bytes (%d, %d, %d)", buf.Len(), n, size, mtu)
39 return buf
40 }
41
42 func TestInput(t *testing.T) {
43 for _, mtu := range []int{0, 512, 1500} {
44 for _, n := range []int{1, 10, 100} {
45 for _, size := range []int{64, 256, 4096} {
46 i := nmsg.NewInput(testReader(t, n, size, mtu), mtu)
47 if i != nil {
48 c := 0
49 for {
50 _, err := i.Recv()
51 if err != nil {
52 if err != io.EOF {
53 t.Error(err)
54 }
55 break
56 }
57 c++
58 }
59 if c < n {
60 t.Errorf("(%d,%d,%d) expected %d, received %d", n, size, mtu, n, c)
61 }
62 }
63 }
64 }
65 }
66 }
67
68 func TestInputFragExpire(t *testing.T) {
69 // Fragment expiration is not checked here, only in
70 // coverage.
71 var readers []io.Reader
72 npayloads := 10
73 payloadSize := 512
74 mtu := 512
75 for i := 0; i < 1000; i++ {
76 readers = append(readers, testReader(t, npayloads,
77 payloadSize, mtu))
78 }
79 inp := nmsg.NewInput(io.MultiReader(readers...), 512)
80 var count int
81 for ; ; count++ {
82 _, err := inp.Recv()
83 if err != nil {
84 break
85 }
86 }
87 if count != npayloads*1000 {
88 t.Errorf("missed input, received %d payloads", count)
89 }
90 }
91
92 func testLoss(t *testing.T, r io.Reader, loss uint64, title string) {
93 t.Helper()
94 i := nmsg.NewInput(r, nmsg.MaxContainerSize)
95 for {
96 if _, err := i.Recv(); err != nil {
97 break
98 }
99 }
100 stats := i.Stats()
101 if stats.LostContainers != loss {
102 t.Errorf("%s: lost %d (expected %d)", title, stats.LostContainers, loss)
103 }
104 }
105
106 func TestInputSequenceLoss1(t *testing.T) {
107 var buf bytes.Buffer
108 c := nmsg.NewContainer()
109
110 c.SetSequenced(true)
111 c.WriteTo(&buf)
112 c.WriteTo(&buf)
113 *c.Nmsg.Sequence++ // skip one
114 c.WriteTo(&buf)
115
116 testLoss(t, &buf, 1, "drop 1")
117 }
118
119 func TestInputSequenceInterleaveLoss1(t *testing.T) {
120 var buf bytes.Buffer
121
122 c1 := nmsg.NewContainer()
123 c2 := nmsg.NewContainer()
124 c1.SetSequenced(true)
125 c2.SetSequenced(true)
126
127 c1.WriteTo(&buf)
128 c2.WriteTo(&buf)
129 c2.WriteTo(&buf)
130 c1.WriteTo(&buf)
131 c2.WriteTo(&buf)
132 *c1.Nmsg.Sequence++
133 c1.WriteTo(&buf)
134 c2.WriteTo(&buf)
135 testLoss(t, &buf, 1, "interleaved, drop 1")
136 }
137
138 func TestInputSequenceWrap(t *testing.T) {
139 var buf bytes.Buffer
140
141 c := nmsg.NewContainer()
142 c.SetSequenced(true)
143 *c.Nmsg.Sequence = math.MaxUint32 - 1
144 t.Log("sequence", c.Nmsg.GetSequence())
145 c.WriteTo(&buf)
146 t.Log("sequence", c.Nmsg.GetSequence())
147 *c.Nmsg.Sequence++
148 t.Log("sequence", c.Nmsg.GetSequence())
149 c.WriteTo(&buf)
150 t.Log("sequence", c.Nmsg.GetSequence())
151 testLoss(t, &buf, 1, "wrapped, drop 1")
152 }
+49
-0
nmsg.go less more
0 /*
1 * Copyright (c) 2017 by Farsight Security, Inc.
2 *
3 * This Source Code Form is subject to the terms of the Mozilla Public
4 * License, v. 2.0. If a copy of the MPL was not distributed with this
5 * file, You can obtain one at http://mozilla.org/MPL/2.0/.
6 */
7
8 //go:generate protoc --go_out=. nmsg.proto
9
10 package nmsg
11
12 import (
13 "hash/crc32"
14
15 "github.com/golang/protobuf/proto"
16 )
17
18 // Container size limits to avoid silly fragmentation and memory
19 // exhaustion.
20 const (
21 MinContainerSize = 512
22 MaxContainerSize = 1048576
23 EtherContainerSize = 1280
24 invalidContainerSize = MaxContainerSize * 16
25 )
26
27 var crc32c = crc32.MakeTable(crc32.Castagnoli)
28
29 // nmsgCRC calculates a crc32 checksum compatible with that used by
30 // the nmsg C library.
31 //
32 // As in the C library, the checksum is converted to network byte order
33 // before eventually being encoded as a protocol buffers integer. This
34 // defeats the endian neutrality of protocol buffers, but is necessary
35 // for compatibility with the C library operating on little endian machines.
36 func nmsgCRC(b []byte) uint32 {
37 return htonl(crc32.Checksum(b, crc32c))
38 }
39
40 // Message encapsulates a protobuf-encoded payload.
41 //
42 // The values returned by the GetVid() and GetMsgtype() methods return
43 // identify the format of the payload.
44 type Message interface {
45 proto.Message
46 GetVid() uint32
47 GetMsgtype() uint32
48 }
+223
-0
nmsg.pb.go less more
0 // Code generated by protoc-gen-go.
1 // source: nmsg.proto
2 // DO NOT EDIT!
3
4 /*
5 Package nmsg is a generated protocol buffer package.
6
7 It is generated from these files:
8 nmsg.proto
9
10 It has these top-level messages:
11 Nmsg
12 NmsgFragment
13 NmsgPayload
14 */
15 package nmsg
16
17 import proto "github.com/golang/protobuf/proto"
18 import fmt "fmt"
19 import math "math"
20
21 // Reference imports to suppress errors if they are not otherwise used.
22 var _ = proto.Marshal
23 var _ = fmt.Errorf
24 var _ = math.Inf
25
26 // This is a compile-time assertion to ensure that this generated file
27 // is compatible with the proto package it is being compiled against.
28 // A compilation error at this line likely means your copy of the
29 // proto package needs to be updated.
30 const _ = proto.ProtoPackageIsVersion2 // please upgrade the proto package
31
32 type Nmsg struct {
33 Payloads []*NmsgPayload `protobuf:"bytes,1,rep,name=payloads" json:"payloads,omitempty"`
34 PayloadCrcs []uint32 `protobuf:"varint,2,rep,name=payload_crcs" json:"payload_crcs,omitempty"`
35 Sequence *uint32 `protobuf:"varint,3,opt,name=sequence" json:"sequence,omitempty"`
36 SequenceId *uint64 `protobuf:"varint,4,opt,name=sequence_id" json:"sequence_id,omitempty"`
37 XXX_unrecognized []byte `json:"-"`
38 }
39
40 func (m *Nmsg) Reset() { *m = Nmsg{} }
41 func (m *Nmsg) String() string { return proto.CompactTextString(m) }
42 func (*Nmsg) ProtoMessage() {}
43 func (*Nmsg) Descriptor() ([]byte, []int) { return fileDescriptor0, []int{0} }
44
45 func (m *Nmsg) GetPayloads() []*NmsgPayload {
46 if m != nil {
47 return m.Payloads
48 }
49 return nil
50 }
51
52 func (m *Nmsg) GetPayloadCrcs() []uint32 {
53 if m != nil {
54 return m.PayloadCrcs
55 }
56 return nil
57 }
58
59 func (m *Nmsg) GetSequence() uint32 {
60 if m != nil && m.Sequence != nil {
61 return *m.Sequence
62 }
63 return 0
64 }
65
66 func (m *Nmsg) GetSequenceId() uint64 {
67 if m != nil && m.SequenceId != nil {
68 return *m.SequenceId
69 }
70 return 0
71 }
72
73 type NmsgFragment struct {
74 Id *uint32 `protobuf:"varint,1,req,name=id" json:"id,omitempty"`
75 Current *uint32 `protobuf:"varint,2,req,name=current" json:"current,omitempty"`
76 Last *uint32 `protobuf:"varint,3,req,name=last" json:"last,omitempty"`
77 Fragment []byte `protobuf:"bytes,4,req,name=fragment" json:"fragment,omitempty"`
78 Crc *uint32 `protobuf:"varint,5,opt,name=crc" json:"crc,omitempty"`
79 XXX_unrecognized []byte `json:"-"`
80 }
81
82 func (m *NmsgFragment) Reset() { *m = NmsgFragment{} }
83 func (m *NmsgFragment) String() string { return proto.CompactTextString(m) }
84 func (*NmsgFragment) ProtoMessage() {}
85 func (*NmsgFragment) Descriptor() ([]byte, []int) { return fileDescriptor0, []int{1} }
86
87 func (m *NmsgFragment) GetId() uint32 {
88 if m != nil && m.Id != nil {
89 return *m.Id
90 }
91 return 0
92 }
93
94 func (m *NmsgFragment) GetCurrent() uint32 {
95 if m != nil && m.Current != nil {
96 return *m.Current
97 }
98 return 0
99 }
100
101 func (m *NmsgFragment) GetLast() uint32 {
102 if m != nil && m.Last != nil {
103 return *m.Last
104 }
105 return 0
106 }
107
108 func (m *NmsgFragment) GetFragment() []byte {
109 if m != nil {
110 return m.Fragment
111 }
112 return nil
113 }
114
115 func (m *NmsgFragment) GetCrc() uint32 {
116 if m != nil && m.Crc != nil {
117 return *m.Crc
118 }
119 return 0
120 }
121
122 type NmsgPayload struct {
123 Vid *uint32 `protobuf:"varint,1,req,name=vid" json:"vid,omitempty"`
124 Msgtype *uint32 `protobuf:"varint,2,req,name=msgtype" json:"msgtype,omitempty"`
125 TimeSec *int64 `protobuf:"varint,3,req,name=time_sec" json:"time_sec,omitempty"`
126 TimeNsec *uint32 `protobuf:"fixed32,4,req,name=time_nsec" json:"time_nsec,omitempty"`
127 Payload []byte `protobuf:"bytes,5,opt,name=payload" json:"payload,omitempty"`
128 Source *uint32 `protobuf:"varint,7,opt,name=source" json:"source,omitempty"`
129 Operator *uint32 `protobuf:"varint,8,opt,name=operator" json:"operator,omitempty"`
130 Group *uint32 `protobuf:"varint,9,opt,name=group" json:"group,omitempty"`
131 XXX_unrecognized []byte `json:"-"`
132 }
133
134 func (m *NmsgPayload) Reset() { *m = NmsgPayload{} }
135 func (m *NmsgPayload) String() string { return proto.CompactTextString(m) }
136 func (*NmsgPayload) ProtoMessage() {}
137 func (*NmsgPayload) Descriptor() ([]byte, []int) { return fileDescriptor0, []int{2} }
138
139 func (m *NmsgPayload) GetVid() uint32 {
140 if m != nil && m.Vid != nil {
141 return *m.Vid
142 }
143 return 0
144 }
145
146 func (m *NmsgPayload) GetMsgtype() uint32 {
147 if m != nil && m.Msgtype != nil {
148 return *m.Msgtype
149 }
150 return 0
151 }
152
153 func (m *NmsgPayload) GetTimeSec() int64 {
154 if m != nil && m.TimeSec != nil {
155 return *m.TimeSec
156 }
157 return 0
158 }
159
160 func (m *NmsgPayload) GetTimeNsec() uint32 {
161 if m != nil && m.TimeNsec != nil {
162 return *m.TimeNsec
163 }
164 return 0
165 }
166
167 func (m *NmsgPayload) GetPayload() []byte {
168 if m != nil {
169 return m.Payload
170 }
171 return nil
172 }
173
174 func (m *NmsgPayload) GetSource() uint32 {
175 if m != nil && m.Source != nil {
176 return *m.Source
177 }
178 return 0
179 }
180
181 func (m *NmsgPayload) GetOperator() uint32 {
182 if m != nil && m.Operator != nil {
183 return *m.Operator
184 }
185 return 0
186 }
187
188 func (m *NmsgPayload) GetGroup() uint32 {
189 if m != nil && m.Group != nil {
190 return *m.Group
191 }
192 return 0
193 }
194
195 func init() {
196 proto.RegisterType((*Nmsg)(nil), "nmsg.Nmsg")
197 proto.RegisterType((*NmsgFragment)(nil), "nmsg.NmsgFragment")
198 proto.RegisterType((*NmsgPayload)(nil), "nmsg.NmsgPayload")
199 }
200
201 func init() { proto.RegisterFile("nmsg.proto", fileDescriptor0) }
202
203 var fileDescriptor0 = []byte{
204 // 259 bytes of a gzipped FileDescriptorProto
205 0x1f, 0x8b, 0x08, 0x00, 0x00, 0x09, 0x6e, 0x88, 0x02, 0xff, 0x4c, 0x8f, 0xb1, 0x4e, 0xc3, 0x30,
206