Commit 1b707fa14697870e7f76d5fa412d980f4df01532 - nfdump

Update upstream source from tag 'upstream/1.6.20' Update to upstream version '1.6.20' with Debian dir c39ca012d51d36f51aaa0bbfcf3c604bca0218fa Bernhard Schmidt 4 years ago

95 changed file(s) with 7445 addition(s) and 8789 deletion(s). Raw diff Collapse all Expand all

+87

-0

ChangeLog less more

	0	2020-03-29
	1	- Release 1.6.20
	2	- More cleanup on plain number printing
	3
	4	2020-03-28
	5	- Fix plain numbers bug #213
	6	- Fix profiler filer bug
	7
	8	2020-02-22
	9	- Release 1.6.19
	10
	11	2020-02-16
	12	- Add Source security group tag (SGT 34000) issue #192
	13	- Modify heuristic for bidir flows #59
	14	- Add Push for new sgt tag.
	15
	16	2020-02-09
	17	- Cleanup unused nffile records. Add nffileV2.h
	18
	19	2020-02-02
	20	- Fix various compiler warnings and automake issues
	21
	22	2020-02-01
	23	- Remove external global vars
	24	- Set verbose logging off by default in non daemon mode
	25	- Fix make check
	26
	27	2020-01-31
	28	- Remove old legacy records
	29
	30	2020-01-29
	31	- Major code rearrangement in order to prepare for futur versions of nfdump
	32	Move extension definitions into extension files nfx.h
	33	Move exporter definition into extension.h
	34	nffile.h should only contain nffile data block handlich definitions
	35
	36	2020-01-26
	37	- Fix compile error for FreeBSD #203
	38	- Cleanup header code - add new filter.h - free nfdump.h from dust
	39	- More code cleanup - delete nf_common - add output_fmt
	40
	41	2020-01-11
	42	- Fix bug for IPv6 network cidr representation in raw and cvs output mode.
	43
	44	2019-12-05
	45	- Remove unused old code. Fix for #197
	46
	47	2019-12-04
	48	- Fix bidir export issues. Cleanup code. Fix for #195
	49
	50	2019-11-17
	51	- Enable small time intervals < 60s. - #185
	52	- Cleanup old code
	53
	54	2019-11-09
	55	- Fix minor nfpcapd issues
	56	- Fix gcc compile issue
	57
	58	2019-11-08
	59	- Fix corrupt file handling #193
	60
	61	2019-11-03
	62	- More code cleanup. Move code to dedicated files.
	63
	64	2019-10-28
	65	- Restructure code for output into separate files.
	66	- Cleanup old code.
	67	- Fix bug #189 - valid json output
	68	- Fix issue #190. Add compile time option for JunOS.
	69
	70	2019-09-01
	71	- Cleanup configure.ac
	72
	73	2019-08-31
	74	- Fix various c11 compile issues
	75	- Cleanup old compat15 comments
	76
	77	2019-08-24
	78	- Add vlan tags dot1qVlanId, 243, 254 - #182
	79	- Fix sflow issue with Arista switches
	80
	81	2019-08-19
	82	- Fix ft2nfdump next hop and ip router fields
	83
	84	2019-08-16
	85	- Cleanup and fix IPv6 network display in war records.
	86
0	87	2019-08-14
1	88	- Fix compile issues
2	89	- Fix output buffer size for lzo1x_decompress_safe()

+16

-9

README.md less more

0	0	# nfdump
1	1
2		Stable Release v1.6.18
3
4		See the Changelog file for all changes in release 1.6.18
	2	Stable Release v1.6.20
	3
	4	See the Changelog file for all changes in release 1.6.20
5	5
6	6	nfdump is a toolset in order to collect and process netflow and sflow data, sent from netflow/sflow compatible devices.
7	7	The toolset supports netflow __v1__, __v5/v7__,__v9__,__IPFIX__ and __SFLOW__. nfdump supports IPv4 as well as IPv6.
8	8
	9	__Note:__ nfdump 1.6.18 and newer versions __not longer__ support nfdump-1.5.x files. If you have nfdump-1.5.x please convert them
	10	before upgrading.
	11
9	12	nfdump is used as backend toolset for __NfSen__.
10	13
11	14	---

16	19
17	20	__Note:__ The older nfdump-1.5.8-2-NSEL is __not compatible__ with nfdump > 1.6.9 which supports NSEL/NEL.
18	21
19		__Note:__ nfdump 1.6.18 __not longer__ supports nfdump-1.5.x files. If you have nfdump-1.5.x please convert them
20		before upgrading.
	22	__Jun OS NAT Event Logging__ is mostly compatible with CISCO's NAT Event Logging - mostly - it needs another data interpretation.
	23	See __--enable-jnat__ below
21	24
22	25	---
23	26

27	30	does not support the full IPFIX definition.
28	31
29	32	* Supports basically same feature set of elements as netflow_v9 module
30		* Only UDP traffic is accepted no tCP/SCTP
	33	* Only UDP traffic is accepted no TCP/SCTP
31	34	* If you would like to see more IPFIX support, please contact me.
32	35
33	36	---

44	47
45	48	* __--enable-nsel__
46	49	Compile nfdump, to read and process NSEL/NEL event data; default is __NO__
	50	* __--enable-jnat__
	51	compile nfdump, to read and process JunOS NAT event logging __NO__
47	52	* __--enable-ftconv__
48	53	Build the flow-tools to nfdump converter; default is __NO__
49	54	* __--enable-sflow__

52	57	Build nfprofile used by NfSen; default is __NO__
53	58	* __--enable-nftrack__
54	59	Build nftrack used by PortTracker; default is __NO__
55		* __--enable-compat15__
56		Build nfdump, to read nfdump data files created with nfdump 1.5.x; default is __NO__
	60
	61	This code no longer reads nfdump-1.5.x data files. If needed use nfdump up
	62	to v1.6.17
57	63
58	64	Development and beta options
59	65

208	214	proper IP addresses.
209	215
210	216	See the manual pages or use the -h switch for details on using each of
211		the programs. For any questions send email to phaag@users.sourceforge.net
	217	the programs. For any questions send email to peter@people.ops-trust.net
212	218
213	219	Configure your router to export netflow. See the relevant documentation
214	220	for your model.

376	382	----\|---
377	383	NF_N_NAT_EVENT \| 230
378	384	NF_N_INGRESS_VRFID \| 234
	385	NF_N_EGRESS_VRFID \| 235
379	386	NF_N_NAT_INSIDE_GLOBAL_IPV4 \| 225
380	387	NF_N_NAT_OUTSIDE_GLOBAL_IPV4 \| 226
381	388	NF_N_POST_NAPT_SRC_PORT \| 227

-6

bin/Makefile.am less more

32	32	AM_CFLAGS = -ggdb
33	33
34	34	# libnfdump sources
35		output = output_json.c output_json.h
36		common = nf_common.c nf_common.h
	35	output = output_util.h output_util.c output_raw.h output_raw.c
	36	output += output_json.c output_json.h output_csv.c output_csv.h output_pipe.c output_pipe.h
	37	output += output_fmt.c output_fmt.h
37	38	util = util.c util.h
38		filelzo = minilzo.c minilzo.h lzoconf.h lzodefs.h lz4.c lz4.h nffile.c nffile.h nfx.c nfx.h
	39	filelzo = minilzo.c minilzo.h lzoconf.h lzodefs.h lz4.c lz4.h
	40	nffile = nffile.c nffile.h nfx.c nfx.h
39	41	nflist = flist.c flist.h fts_compat.c fts_compat.h
40	42	filter = grammar.y scanner.l nftree.c nftree.h ipconv.c ipconv.h rbtree.h
41	43	exporter = exporter.c exporter.h

59	61	launch = launch.c launch.h
60	62
61	63	lib_LTLIBRARIES = libnfdump.la
62		libnfdump_la_SOURCES = $(output) $(common) $(util) $(filelzo) $(nflist) $(filter) $(exporter)
63		#libnfdump_la_LIBADD = -lz
64		libnfdump_la_LDFLAGS = -release 1.6.18
	64	libnfdump_la_SOURCES = $(output) $(util) $(filelzo) $(nffile) $(nflist) $(filter) $(exporter)
	65	libnfdump_la_LDFLAGS = -release 1.6.20
65	66
66	67
67	68	nfdump_SOURCES = nfdump.c nfdump.h nfstat.c nfstat.h nfexport.c nfexport.h \

bin/anon.flows less more

Binary diff not shown

~~bin/captured~~ less more

Binary diff not shown

+15

-44

bin/collector.c less more

0	0	/*
1		* Copyright (c) 2017, Peter Haag
2		* Copyright (c) 2016, Peter Haag
3		* Copyright (c) 2014, Peter Haag
4		* Copyright (c) 2009, Peter Haag
	1	* Copyright (c) 2009-2020, Peter Haag
5	2	* Copyright (c) 2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
6	3	* All rights reserved.
7	4	*

55	52	#include <stdint.h>
56	53	#endif
57	54
58		#ifndef DEVEL
59		# define dbg_printf(...) /* printf(__VA_ARGS__) */
60		#else
61		# define dbg_printf(...) printf(__VA_ARGS__)
62		#endif
63
64	55	#include "util.h"
65		#include "nf_common.h"
	56	#include "nfdump.h"
66	57	#include "nffile.h"
67	58	#include "bookkeeper.h"
68	59	#include "collector.h"
69	60	#include "nfx.h"
70	61
71	62	#include "nffile_inline.c"
72
73		/* globals */
74		uint32_t default_sampling = 1;
75		uint32_t overwrite_sampling = 0;
76	63
77	64	/* local variables */
78	65	static uint32_t exporter_sysid = 0;

194	181	exit(255);
195	182	}
196	183
	184	char *path = realpath(q, NULL);
	185	if ( !path ) {
	186	fprintf(stderr, "realpath() error %s: %s\n", q, strerror(errno));
	187	return 0;
	188	}
	189
197	190	// check for existing path
198		if ( stat(q, &fstat) ) {
199		fprintf(stderr, "stat() error %s: %s\n", q, strerror(errno));
	191	if ( stat(path, &fstat) ) {
	192	fprintf(stderr, "stat() error %s: %s\n", path, strerror(errno));
200	193	return 0;
201	194	}
202	195	if ( !(fstat.st_mode & S_IFDIR) ) {
203		fprintf(stderr, "No such directory: %s\n", q);
	196	fprintf(stderr, "Not a directory: %s\n", path);
204	197	return 0;
205	198	}
206	199
207	200	// remember path
208		(*source)->datadir = strdup(q);
209		if ( !(*source)->datadir ) {
210		fprintf(stderr, "strdup() error: %s\n", strerror(errno));
211		return 0;
212		}
	201	(*source)->datadir = path;
213	202
214	203	// cache current collector file
215	204	if ( snprintf(s, MAXPATHLEN-1, "%s/%s.%lu", (*source)->datadir , NF_DUMPFILE, (unsigned long)getpid() ) >= (MAXPATHLEN-1)) {

382	371	inet_ntop(ss->ss_family, ptr, ident, sizeof(ident));
383	372	ident[99] = '\0';
384	373	dbg_printf("Dynamic Flow Source IP: %s\n", ident);
385
386		if ( strchr(ident, ':') ) { // condense IPv6 addresses
387		condense_v6(ident);
388		}
389	374
390	375	s = ident;
391	376	while ( *s != '\0' ) {

573	558	} // End of FlushInfoSampler
574	559
575	560	void FlushStdRecords(FlowSource_t *fs) {
576		generic_exporter_t *e = fs->exporter_data;
	561	exporter_t *e = fs->exporter_data;
577	562	int i;
578	563
579	564	while ( e ) {
580		generic_sampler_t *sampler = e->sampler;
	565	sampler_t *sampler = e->sampler;
581	566	AppendToBuffer(fs->nffile, (void *)&(e->info), e->info.header.size);
582	567	while ( sampler ) {
583	568	AppendToBuffer(fs->nffile, (void *)&(sampler->info), sampler->info.header.size);

595	580	} // End of FlushStdRecords
596	581
597	582	void FlushExporterStats(FlowSource_t *fs) {
598		generic_exporter_t *e = fs->exporter_data;
	583	exporter_t *e = fs->exporter_data;
599	584	exporter_stats_record_t *exporter_stats;
600	585	uint32_t i, size;
601	586

653	638
654	639	} // End of FlushExporterStats
655	640
656
657
658		int HasOptionTable(FlowSource_t *fs, uint16_t id ) {
659		option_offset_t *t;
660
661		t = fs->option_offset_table;
662		while ( t && t->id != id )
663		t = t->next;
664
665		dbg_printf("Has option table: %s\n", t == NULL ? "not found" : "found");
666
667		return t != NULL;
668
669		} // End of HasOptionTable

-60

bin/collector.h less more

0	0	/*
1		* Copyright (c) 2017, Peter Haag
2		* Copyright (c) 2014, Peter Haag
3		* Copyright (c) 2009, Peter Haag
	1	* Copyright (c) 2009-2019, Peter Haag
4	2	* Copyright (c) 2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
5	3	* All rights reserved.
6	4	*

41	39	#endif
42	40	#include <sys/socket.h>
43	41
	42	#include "exporter.h"
44	43	#include "bookkeeper.h"
45	44	#include "nffile.h"
46	45
47	46	#define FNAME_SIZE 256
48		#define IDENT_SIZE 32
49	47
50		typedef struct srecord_s {
51		char fname[FNAME_SIZE]; // file name
52		char subdir[FNAME_SIZE]; // subdir name
53		char tstring[16]; // actually 12 needed e.g. 200411011230
54		time_t tstamp; // UNIX time stamp
55		int failed; // in case of an error
56		} srecord_t;
57
58		// common_record_t defines ext_map as uint_8, so max 256 extension maps allowed.
59		// should be enough anyway
60
61		typedef struct option_offset_s {
62		struct option_offset_s *next;
63		uint32_t id; // table id
64		uint32_t flags; // info about this map
65
66		// sampling offsets
67		#define HAS_SAMPLER_DATA 1
68		uint16_t offset_id;
69		uint16_t sampler_id_length;
70		uint16_t offset_mode;
71		uint16_t offset_interval;
72
73		#define HAS_STD_SAMPLER_DATA 2
74		uint16_t offset_std_sampler_interval;
75		uint16_t offset_std_sampler_algorithm;
76
77		} option_offset_t;
78
79		typedef struct generic_sampler_s {
80		struct generic_sampler_s *next;
81		sampler_info_record_t info;
82		} generic_sampler_t;
83
84		typedef struct generic_exporter_s {
85		// link chain
86		struct generic_exporter_s *next;
87
88		// generic exporter information
89		exporter_info_record_t info;
90
91		uint64_t packets; // number of packets sent by this exporter
92		uint64_t flows; // number of flow records sent by this exporter
93		uint32_t sequence_failure; // number of sequence failues
94		uint32_t padding_errors; // number of sequence failues
95
96		generic_sampler_t *sampler;
97
98		} generic_exporter_t;
	48	/* common minimum netflow header for all versions */
	49	typedef struct common_flow_header {
	50	uint16_t version;
	51	uint16_t count;
	52	} common_flow_header_t;
99	53
100	54	typedef struct FlowSource_s {
101	55	// link

120	74	uint64_t last_seen; // in msec
121	75
122	76	// Any exporter specific data
123		generic_exporter_t *exporter_data;
	77	exporter_t *exporter_data;
124	78	uint32_t exporter_count;
125	79	struct timeval received;
126	80

132	86	int num_maps;
133	87	extension_map_t **maps;
134	88	} extension_map_list;
135
136		option_offset_t *option_offset_table;
137	89
138	90	} FlowSource_t;
139	91

165	117
166	118	int FlushInfoSampler(FlowSource_t fs, sampler_info_record_t sampler);
167	119
168		int HasOptionTable(FlowSource_t *fs, uint16_t id );
169
170		void launcher (char commbuff, FlowSource_t FlowSource, char *process, int expire);
171
172	120	/* Default time window in seconds to rotate files */
173	121	#define TIME_WINDOW 300
174	122

-9

bin/content_dns.c less more

0	0	/*
1		* Copyright (c) 2017, Peter Haag
2		* Copyright (c) 2014, Peter Haag
3		* Copyright (c) 2013, Peter Haag
	1	* Copyright (c) 2013-2020, Peter Haag
4	2	* All rights reserved.
5	3	*
6	4	* Redistribution and use in source and binary forms, with or without

73	71	#include <pcap.h>
74	72
75	73	#include "util.h"
	74	#include "nfdump.h"
76	75	#include "nffile.h"
77	76	#include "bookkeeper.h"
78	77	#include "collector.h"
79	78	#include "flowtree.h"
80	79	#include "content_dns.h"
81
82		#ifndef DEVEL
83		# define dbg_printf(...) /* printf(__VA_ARGS__) */
84		#else
85		# define dbg_printf(...) printf(__VA_ARGS__)
86		#endif
87	80
88	81	#include "inline.c"
89	82

+35

-70

bin/expire.c less more

0	0	/*
1		* Copyright (c) 2016, Peter Haag
2		* Copyright (c) 2014, Peter Haag
3		* Copyright (c) 2009, Peter Haag
	1	* Copyright (c) 2009-2019, Peter Haag
4	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
5	3	* All rights reserved.
6	4	*

28	26	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
29	27	* POSSIBILITY OF SUCH DAMAGE.
30	28	*
31		* $Author: haag $
32		*
33		* $Id: expire.c 39 2009-11-25 08:11:15Z haag $
34		*
35		* $LastChangedRevision: 39 $
36		*
37	29	*/
38	30
39	31	#include "config.h"

77	69
78	70	static int compare(const FTSENT f1, const FTSENT f2);
79	71
80		#if 0
81		#define unlink unlink_debug
82
83		static int unlink_debug (const char *path) {
84		printf("Unlink %s\n", path);
85		return 0;
86		} // End of unlink_debug
87		#endif
88
89	72	static void IntHandler(int signal) {
90	73
91	74	switch (signal) {

143	126	p++;
144	127	}
145	128	if ( p == s ) {
146		fprintf(stderr, "Missing number in '%s'\n", s);
	129	LogError("Missing number in '%s'", s);
147	130	return 0;
148	131	}
149	132

171	154	fac = 1024LL * 1024LL * 1024LL * 1024LL;
172	155	break;
173	156	default:
174		fprintf(stderr, "Garbage character(s) '%s' in '%s'\n", p, s);
	157	LogError("Garbage character(s) '%s' in '%s'", p, s);
175	158	return 0;
176	159	}
177	160

183	166
184	167	if ( *p ) {
185	168	// extra garbage after factor
186		fprintf(stderr, "Garbage character(s) '%s''in '%s'\n", p, s);
	169	LogError("Garbage character(s) '%s''in '%s'", p, s);
187	170	return 0;
188	171	}
189	172	*r = '\0';

221	204	p++;
222	205	}
223	206	if ( p == q ) {
224		fprintf(stderr, "Missing number before '%s'\n", q);
	207	LogError("Missing number before '%s'", q);
225	208	return 0;
226	209	}
227	210	switch (*p) {
228	211	case 'w':
229	212	*p++ = '\0';
230	213	if ( weeks ) {
231		fprintf(stderr, "Ambiguous weeks %sw\n", q);
	214	LogError("Ambiguous weeks %sw", q);
232	215	return 0;
233	216	}
234	217	weeks = strtoll(q, NULL, 10);

236	219	case 'd':
237	220	*p++ = '\0';
238	221	if ( days ) {
239		fprintf(stderr, "Ambiguous days %sD\n", q);
	222	LogError("Ambiguous days %sD", q);
240	223	return 0;
241	224	}
242	225	days = strtoll(q, NULL, 10);

245	228	case 'H':
246	229	if ( p ) p++ = '\0';
247	230	if ( hours ) {
248		fprintf(stderr, "Ambiguous hours %sH\n", q);
	231	LogError("Ambiguous hours %sH", q);
249	232	return 0;
250	233	}
251	234	hours = strtoll(q, NULL, 10);

253	236	case 'M':
254	237	*p++ = '\0';
255	238	if ( minutes ) {
256		fprintf(stderr, "Ambiguous minutes %sM\n", q);
	239	LogError("Ambiguous minutes %sM", q);
257	240	return 0;
258	241	}
259	242	minutes = strtoll(q, NULL, 10);
260	243	break;
261	244	default:
262		fprintf(stderr, "Unknown time unit '%s'\n", q);
	245	LogError("Unknown time unit '%s'", q);
263	246	return 0;
264	247
265	248	}

284	267	dirstat->filesize = dirstat->numfiles = 0;
285	268	dirstat->first = 0;
286	269	dirstat->last = 0;
287		strncpy(first_timestring, "999999999999", 15);
288		strncpy(last_timestring, "000000000000", 15);
	270	strncpy(first_timestring, "99999999999999", 15);
	271	strncpy(last_timestring, "00000000000000", 15);
289	272
290	273	fts = fts_open(path, FTS_LOGICAL, compare);
291	274	if ( !fts ) {

293	276	return;
294	277	}
295	278	while ( (ftsent = fts_read(fts)) != NULL) {
296		if ( ftsent->fts_info == FTS_F && ftsent->fts_namelen == 19 ) {
297		// nfcapd.200604301200 strlen = 19
	279	if ( ftsent->fts_info == FTS_F &&
	280	((ftsent->fts_namelen == 19) \|\| (ftsent->fts_namelen == 21)) ) {
	281	// nfcapd.200604301200 strlen = 19
	282	// nfcapd.20190430120010 strlen = 21
298	283	if ( strncmp(ftsent->fts_name, "nfcapd.", 7) == 0 ) {
299	284	char s, p = &(ftsent->fts_name[7]);
300	285

338	323	}
339	324	fts_close(fts);
340	325
	326
341	327	// no files means do rebuild next time, otherwise the stat record may not be accurate
342	328	if ( dirstat->numfiles == 0 ) {
343	329	dirstat->first = dirstat->last = time(NULL);

347	333	dirstat->last = ISO2UNIX(last_timestring);
348	334	dirstat->status = STATFILE_OK;
349	335	}
	336	dbg_printf("Rescan dir - first: %s, last: %s\n", first_timestring, last_timestring);
350	337
351	338	} // End of RescanDir
352	339

374	361
375	362	time_t t_watermark = now - (time_t)((maxlife * dirstat->low_water)/100);
376	363
377		// printf("Expire files before %s", ctime(&t_expire));
	364	dbg_printf("Expire files before %s", ctime(&t_expire));
378	365	expire_timelimit = strdup(UNIX2ISO(t_watermark));
379		// printf("down to %s", ctime(&t_watermark));
380		// printf("Diff: %i\n", t_watermark - t_expire );
	366	dbg_printf("down to %s", ctime(&t_watermark));
	367	dbg_printf("Diff: %li\n", t_watermark - t_expire );
381	368
382	369	if ( dirstat->last < t_expire && (isatty(STDIN_FILENO) ) ) {
383	370	// this means all files will get expired - are you sure ?

418	405	while ( !done && ((ftsent = fts_read(fts)) != NULL) ) {
419	406	if ( ftsent->fts_info == FTS_F ) {
420	407	dir_files++; // count files in directories
421		if ( ftsent->fts_namelen == 19 && strncmp(ftsent->fts_name, "nfcapd.", 7) == 0 ) {
422		// nfcapd.200604301200 strlen = 19
	408	if ( (ftsent->fts_namelen == 19 \|\| ftsent->fts_namelen == 21) &&
	409	strncmp(ftsent->fts_name, "nfcapd.", 7) == 0 ) {
	410	// nfcapd.200604301200 strlen = 19
	411	// nfcapd.20190430120010 strlen = 21
423	412	char s, p = &(ftsent->fts_name[7]);
424	413
425	414	// process only nfcapd. files

488	477	// do not delete base data directory ( level == 0 )
489	478	if ( dir_files == 0 && ftsent->fts_level > 0 ) {
490	479	// directory is empty and can be deleted
491		// printf("Will remove directory %s\n", ftsent->fts_path);
	480	dbg_printf("Will remove directory %s\n", ftsent->fts_path);
492	481	if ( rmdir(ftsent->fts_path) != 0 ) {
493	482	LogError( "rmdir() error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
494	483	}

546	535
547	536	while ( current_channel->ftsent ) {
548	537
549		/*
550		FTSENT *ftsent = current_channel->ftsent;
551		char *finfo;
552		switch (ftsent->fts_info) {
553		case FTS_ERR:
554		case FTS_NS:
555		LogError( "fts_read() %s error in %s line %d: %s\n",
556		current_channel->ftsent->fts_path, __FILE__, __LINE__, strerror(current_channel->ftsent->fts_errno) );
557		break;
558		case FTS_D:
559		finfo = "DIR pre ";
560		break;
561		case FTS_DP:
562		finfo = "DIR post";
563		break;
564		case FTS_F:
565		finfo = "FILE ";
566		break;
567		default:
568		finfo = "<undef> ";
569		}
570		printf("%u %i %s %s %s\n", ftsent->fts_info, ftsent->fts_level, finfo, ftsent->fts_path, ftsent->fts_name);
571		*/
572	538	if ( current_channel->ftsent->fts_info == FTS_ERR \|\|
573	539	current_channel->ftsent->fts_info == FTS_NS) {
574	540	LogError( "fts_read() %s error in %s line %d: %s\n",

583	549	current_channel->ftsent->fts_number++;
584	550
585	551	// if ftsent points to first valid file, break
586		if ( current_channel->ftsent->fts_namelen == 19 && strncmp(current_channel->ftsent->fts_name, "nfcapd.", 7) == 0 )
	552	if ( (current_channel->ftsent->fts_namelen == 19 \|\| current_channel->ftsent->fts_namelen == 21) &&
	553	strncmp(current_channel->ftsent->fts_name, "nfcapd.", 7) == 0 )
587	554	break;
588	555
589	556	// otherwise loop

607	574	SetupSignalHandler();
608	575
609	576	if ( maxlife ) {
610		// time_t t_expire = now - maxlife;
	577	// time_t t_expire = now - maxlife;
611	578	// build an appropriate string for comparing
612	579	time_t t_watermark = now - (time_t)((maxlife * current_stat->low_water)/100);
613	580
614		// printf("Expire files before %s", ctime(&t_expire));
615	581	expire_timelimit = strdup(UNIX2ISO(t_watermark));
616		// printf("down to %s", ctime(&t_watermark));
617		// printf("Diff: %i\n", t_watermark - t_expire );
	582	dbg_printf("down to %s", ctime(&t_watermark));
618	583
619	584	}
620	585

661	626	// expire_channel now points to the channel with oldest file
662	627	// do expire
663	628	p = &(expire_channel->ftsent->fts_name[7]);
664		// printf("File: %s\n", expire_channel->ftsent->fts_path);
	629	dbg_printf("File: %s\n", expire_channel->ftsent->fts_path);
665	630
666	631	if ( !size_done ) {
667	632	// expire size-wise if needed
668		// printf(" Size expire %llu %llu\n", current_stat->filesize, sizelimit);
	633	dbg_printf(" Size expire %llu %llu\n", current_stat->filesize, sizelimit);
669	634	if ( current_stat->filesize > sizelimit ) {
670	635	// need to delete this file
671	636	if ( unlink(expire_channel->ftsent->fts_path) == 0 ) {

692	657	size_done = 1;
693	658	}
694	659	} else if ( !lifetime_done ) {
695		// printf(" Time expire \n");
	660	dbg_printf(" Time expire \n");
696	661	// expire time-wise if needed
697	662	// this part of the code is executed only when size-wise is already fullfilled
698	663	if ( strcmp(p, expire_timelimit) < 0 ) {

732	697	while ( expire_channel->ftsent ) {
733	698	if ( expire_channel->ftsent->fts_info == FTS_F ) { // entry is a file
734	699	expire_channel->ftsent->fts_number++;
735		if ( expire_channel->ftsent->fts_namelen == 19 &&
	700	if ( (expire_channel->ftsent->fts_namelen == 19 \|\| expire_channel->ftsent->fts_namelen == 21) &&
736	701	strncmp(expire_channel->ftsent->fts_name, "nfcapd.", 7) == 0 ) {
737	702	// if ftsent points to next valid file
738	703	char *p = &(expire_channel->ftsent->fts_name[7]);

757	722	// do not delete base data directory ( level == 0 )
758	723	if ( expire_channel->ftsent->fts_number == 0 && expire_channel->ftsent->fts_level > 0 ) {
759	724	// directory is empty and can be deleted
760		// printf("Will remove directory %s\n", expire_channel->ftsent->fts_path);
	725	dbg_printf("Will remove directory %s\n", expire_channel->ftsent->fts_path);
761	726	if ( rmdir(expire_channel->ftsent->fts_path) != 0 ) {
762	727	LogError( "rmdir() error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
763	728	}

+14

-53

bin/exporter.c less more

0	0	/*
1		* Copyright (c) 2017, Peter Haag
2		* Copyright (c) 2016, Peter Haag
3		* Copyright (c) 2014, Peter Haag
4		* Copyright (c) 2012, Peter Haag
	1	* Copyright (c) 2012-2020, Peter Haag
5	2	*
6	3	* Redistribution and use in source and binary forms, with or without
7	4	* modification, are permitted provided that the following conditions are met:

50	47	#include <stdint.h>
51	48	#endif
52	49
53		#ifndef DEVEL
54		# define dbg_printf(...) /* printf(__VA_ARGS__) */
55		#else
56		# define dbg_printf(...) printf(__VA_ARGS__)
57		#endif
58
59	50	#include "util.h"
	51	#include "nfdump.h"
60	52	#include "nffile.h"
61	53	#include "nfx.h"
62	54	#include "nfnet.h"
63	55	#include "bookkeeper.h"
64	56	#include "collector.h"
65	57	#include "exporter.h"
66		#include "nf_common.h"
67	58	#include "netflow_v1.h"
68	59	#include "netflow_v5_v7.h"
69	60	#include "netflow_v9.h"
70	61	#include "ipfix.h"
71	62
72	63	/* global */
73		generic_exporter_t **exporter_list;
	64	exporter_t **exporter_list;
74	65
75	66	/* local variables */
76	67	#define MAX_EXPORTERS 65536
77		static generic_exporter_t *exporter_root;
	68	static exporter_t *exporter_root;
78	69
79	70	#include "nffile_inline.c"
80	71

83	74	/* functions */
84	75	int InitExporterList(void) {
85	76
86		exporter_list = calloc(MAX_EXPORTERS, sizeof(generic_exporter_t *));
	77	exporter_list = calloc(MAX_EXPORTERS, sizeof(exporter_t *));
87	78	if ( !exporter_list ) {
88	79	LogError("malloc() error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
89	80	return 0;

135	126	}
136	127
137	128	// slot[id] is now available
138		exporter_list[id] = (generic_exporter_t *)calloc(1, sizeof(generic_exporter_t));
	129	exporter_list[id] = (exporter_t *)calloc(1, sizeof(exporter_t));
139	130	if ( !exporter_list[id] ) {
140	131	LogError("malloc() error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
141	132	return 0;

184	175
185	176	int AddSamplerInfo(sampler_info_record_t *sampler_record) {
186	177	uint32_t id;
187		generic_sampler_t **sampler;
	178	sampler_t **sampler;
188	179
189	180	if ( sampler_record->header.size != sizeof(sampler_info_record_t) ) {
190	181	LogError("Corrupt sampler record in %s line %d\n", __FILE__, __LINE__);

212	203	sampler = &((*sampler)->next);
213	204	}
214	205
215		sampler = (generic_sampler_t )malloc(sizeof(generic_sampler_t));
	206	sampler = (sampler_t )malloc(sizeof(sampler_t));
216	207	if ( !*sampler ) {
217	208	LogError("malloc() error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
218	209	return 0;

299	290	i = 1;
300	291	while ( i < MAX_EXPORTERS && exporter_list[i] != NULL ) {
301	292	exporter_info_record_t *exporter;
302		generic_sampler_t *sampler;
	293	sampler_t *sampler;
303	294
304	295	exporter = &exporter_list[i]->info;
305	296	AppendToBuffer(nffile, (void *)exporter, exporter->header.size);

371	362
372	363	for ( i=0; i < nffile->block_header->NumRecords; i++ ) {
373	364	switch ( record->type ) {
374		// ExporterRecordType and SamplerRecordype tc versions only
375		case ExporterRecordType: {
376		#define IP_STRING_LEN 40
377		char ipstr[IP_STRING_LEN];
378		exporter_record_t exporter_record = (exporter_record_t )record ;
379		found = 1;
380		printf("\n");
381		if ( exporter_record->sa_family == AF_INET ) {
382		uint32_t _ip = htonl(exporter_record->ip.V4);
383		inet_ntop(AF_INET, &_ip, ipstr, sizeof(ipstr));
384		printf("SysID: %u, IP: %16s, version: %u, ID: %2u, Sequence Failures: %u\n", exporter_record->sysid,
385		ipstr, exporter_record->version, exporter_record->exporter_id, exporter_record->sequence_failure);
386		} else if ( exporter_record->sa_family == AF_INET6 ) {
387		uint64_t _ip[2];
388		_ip[0] = htonll(exporter_record->ip.V6[0]);
389		_ip[1] = htonll(exporter_record->ip.V6[1]);
390		inet_ntop(AF_INET6, &_ip, ipstr, sizeof(ipstr));
391		printf("SysID: %u, IP: %40s, version: %u, ID: %2u, Sequence Failures: %u\n", exporter_record->sysid,
392		ipstr, exporter_record->version, exporter_record->exporter_id, exporter_record->sequence_failure);
393		} else {
394		strncpy(ipstr, "<unknown>", IP_STRING_LEN);
395		printf("** Exporter IP version unknown **\n");
396		}
397		} break;
398		case SamplerRecordype: {
399		sampler_record_t sampler_record = (sampler_record_t )record;;
400		if ( sampler_record->id < 0 ) {
401		printf(" Generic Sampler: mode: %u, interval: %u\n",
402		sampler_record->mode, sampler_record->interval);
403		} else {
404		printf(" Sampler: id: %i, mode: %u, interval: %u\n",
405		sampler_record->id, sampler_record->mode, sampler_record->interval);
406		}
407		} break;
	365	case LegacyRecordType1:
	366	case LegacyRecordType2:
	367	LogError("Legacy record type: %i no longer supported\n", record->type);
	368	break;
408	369	case ExporterInfoRecordType:
409	370	found = 1;
410	371	if ( !AddExporterInfo((exporter_info_record_t *)record) ) {

437	398	char ipstr[IP_STRING_LEN];
438	399
439	400	exporter_info_record_t *exporter;
440		generic_sampler_t *sampler;
	401	sampler_t *sampler;
441	402
442	403	printf("\n");
443	404	exporter = &exporter_list[i]->info;

+133

-3

bin/exporter.h less more

0	0	/*
1		* Copyright (c) 2017, Peter Haag
2		* Copyright (c) 2014, Peter Haag
3		* Copyright (c) 2012, Peter Haag
	1	* Copyright (c) 2012-2020, Peter Haag
4	2	*
5	3	* Redistribution and use in source and binary forms, with or without
6	4	* modification, are permitted provided that the following conditions are met:

40	38
41	39	#include "nffile.h"
42	40
	41	typedef struct optionTag_s {
	42	uint16_t offset;
	43	uint16_t length;
	44	} optionTag_t;
	45
	46	/*
	47	*
	48	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	49	* \| - \| 0 \| 1 \| 2 \| 3 \| 4 \| 5 \| 6 \| 7 \|
	50	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	51	* \| 0 \| record type == 7 \| size \| version \|
	52	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	53	* \| 1 \| \|
	54	* +----+--------------+--------------+--------------+---------- ip ------------+--------------+--------------+--------------+
	55	* \| 2 \| \|
	56	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	57	* \| 3 \| sa_family \| sysid \| id \|
	58	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	59	*/
	60	typedef struct exporter_info_record_s {
	61	record_header_t header;
	62
	63	// exporter version
	64	uint32_t version;
	65	#define SFLOW_VERSION 9999
	66
	67	// IP address
	68	ip_addr_t ip;
	69	uint16_t sa_family;
	70
	71	// internal assigned ID
	72	uint16_t sysid;
	73
	74	// exporter ID/Domain ID/Observation Domain ID assigned by the device
	75	uint32_t id;
	76
	77	} exporter_info_record_t;
	78
	79	/*
	80	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	81	* \| - \| 0 \| 1 \| 2 \| 3 \| 4 \| 5 \| 6 \| 7 \|
	82	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	83	* \| 0 \| record type == 8 \| size \| stat_count \|
	84	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	85	* \| 1 \| sysid[0] \| sequence_failure[0] \|
	86	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	87	* \| 2 \| packets[0] \|
	88	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	89	* \| 3 \| flows[0] \|
	90	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	91	* ... more stat records [x], one for each exporter
	92	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	93	*/
	94	typedef struct exporter_stats_record_s {
	95	record_header_t header;
	96
	97	uint32_t stat_count; // number of stat records
	98
	99	struct exporter_stat_s {
	100	uint32_t sysid; // identifies the exporter
	101	uint32_t sequence_failure; // number of sequence failues
	102	uint64_t packets; // number of packets sent by this exporter
	103	uint64_t flows; // number of flow records sent by this exporter
	104	} stat[1];
	105
	106	} exporter_stats_record_t;
	107
	108	/*
	109	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	110	* \| - \| 0 \| 1 \| 2 \| 3 \| 4 \| 5 \| 6 \| 7 \|
	111	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	112	* \| 0 \| record type == 9 \| size \| id \|
	113	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	114	* \| 1 \| interval \| mode \| exporter_sysid \|
	115	* +----+--------------+--------------+--------------+-----------------------------+--------------+--------------+--------------+
	116	*/
	117	typedef struct sampler_info_record_s {
	118	record_header_t header;
	119
	120	// sampler data
	121	int32_t id; // id assigned by the exporting device
	122	uint32_t interval; // sampling interval
	123	uint16_t mode; // sampling mode
	124	uint16_t exporter_sysid; // internal reference to exporter
	125
	126	} sampler_info_record_t;
	127
	128	typedef struct sampler_s {
	129	struct sampler_s *next;
	130	sampler_info_record_t info; // sampler record nffile
	131	} sampler_t;
	132
	133	typedef struct exporter_s {
	134	// linked chain
	135	struct exporter_s *next;
	136
	137	// exporter information
	138	exporter_info_record_t info; // exporter record nffile
	139
	140	uint64_t packets; // number of packets sent by this exporter
	141	uint64_t flows; // number of flow records sent by this exporter
	142	uint32_t sequence_failure; // number of sequence failues
	143	uint32_t padding_errors; // number of sequence failues
	144
	145	sampler_t *sampler; // list of samplers associated with this exporter
	146
	147	} exporter_t;
	148
	149	typedef struct samplerOption_s {
	150	struct samplerOption_s *next;
	151	uint32_t tableID; // table id
	152	#define STDSAMPLING34 1
	153	#define STDSAMPLING35 2
	154	#define STDMASK 0x3
	155	#define STDFLAGS 0x3
	156
	157	#define SAMPLER302 4
	158	#define SAMPLER304 8
	159	#define SAMPLER305 16
	160	#define SAMPLERMASK 0x1C
	161	#define SAMPLERFLAGS 0x1C
	162
	163	uint32_t flags; // info about this map
	164
	165	// sampling offset/length values
	166	optionTag_t id;
	167	optionTag_t mode;
	168	optionTag_t interval;
	169
	170	} samplerOption_t;
	171
	172
43	173	int InitExporterList(void);
44	174
45	175	int AddExporterInfo(exporter_info_record_t *exporter_record);

+151

-0

bin/filter.h less more

	0	/*
	1	* Copyright (c) 2020, Peter Haag
	2	* All rights reserved.
	3	*
	4	* Redistribution and use in source and binary forms, with or without
	5	* modification, are permitted provided that the following conditions are met:
	6	*
	7	* * Redistributions of source code must retain the above copyright notice,
	8	* this list of conditions and the following disclaimer.
	9	* * Redistributions in binary form must reproduce the above copyright notice,
	10	* this list of conditions and the following disclaimer in the documentation
	11	* and/or other materials provided with the distribution.
	12	* * Neither the name of the author nor the names of its contributors may be
	13	* used to endorse or promote products derived from this software without
	14	* specific prior written permission.
	15	*
	16	* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
	17	* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	18	* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	19	* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
	20	* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
	21	* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
	22	* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
	23	* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
	24	* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
	25	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
	26	* POSSIBILITY OF SUCH DAMAGE.
	27	*
	28	*/
	29
	30	#ifndef _FILTER_H
	31	#define _FILTER_H 1
	32
	33	#include "config.h"
	34
	35	#include <sys/types.h>
	36	#ifdef HAVE_STDINT_H
	37	#include <stdint.h>
	38	#endif
	39
	40	#include "rbtree.h"
	41
	42	#define NSEL_EVENT_IGNORE 0LL
	43	#define NSEL_EVENT_CREATE 1LL
	44	#define NSEL_EVENT_DELETE 2LL
	45	#define NSEL_EVENT_DENIED 3LL
	46	#define NSEL_EVENT_ALERT 4LL
	47	#define NSEL_EVENT_UPDATE 5LL
	48
	49	#define NEL_EVENT_INVALID 0LL
	50	#define NEL_EVENT_ADD 1LL
	51	#define NEL_EVENT_DELETE 2LL
	52
	53	/*
	54	* Definitions
	55	*/
	56	enum { CMP_EQ = 0, CMP_GT, CMP_LT, CMP_IDENT, CMP_FLAGS, CMP_IPLIST, CMP_ULLIST };
	57
	58	/*
	59	* filter functions:
	60	* For some filter functions, netflow records need to be processed first in order to filter them
	61	* This involves all data not directly available in the netflow record, such as packets per second etc.
	62	* Filter speed is a bit slower due to extra netflow processsing
	63	* The sequence of the enum values must correspond with the entries in the flow_procs array
	64	*/
	65
	66	enum { FUNC_NONE = 0, /* no function - just plain filtering - just to be complete here */
	67	FUNC_PPS, /* function code for pps ( packet per second ) filter function */
	68	FUNC_BPS, /* function code for bps ( bits per second ) filter function */
	69	FUNC_BPP, /* function code for bpp ( bytes per packet ) filter function */
	70	FUNC_DURATION, /* function code for duration ( in miliseconds ) filter function */
	71	FUNC_MPLS_EOS, /* function code for matching End of MPLS Stack label */
	72	FUNC_MPLS_ANY, /* function code for matching any MPLS label */
	73	FUNC_PBLOCK /* function code for matching ports against pblock start */
	74	};
	75
	76	typedef struct FilterParam {
	77	uint16_t comp;
	78	uint16_t direction;
	79	uint32_t data;
	80	uint32_t inout;
	81	uint32_t acl;
	82	uint32_t self;
	83	} FilterParam_t;
	84
	85	/* Definition of the IP list node */
	86	struct IPListNode {
	87	RB_ENTRY(IPListNode) entry;
	88	uint64_t ip[2];
	89	uint64_t mask[2];
	90	};
	91
	92	/* Definition of the port/AS list node */
	93	struct ULongListNode {
	94	RB_ENTRY(ULongListNode) entry;
	95	uint64_t value;
	96	};
	97
	98	/* IP tree type */
	99	typedef RB_HEAD(IPtree, IPListNode) IPlist_t;
	100
	101	/* Port/AS tree type */
	102	typedef RB_HEAD(ULongtree, ULongListNode) ULongtree_t;
	103
	104	// Insert the RB prototypes here
	105	RB_PROTOTYPE(IPtree, IPListNode, entry, IPNodeCMP);
	106
	107	RB_PROTOTYPE(ULongtree, ULongListNode, entry, ULNodeCMP);
	108
	109	/* parser/scanner prototypes */
	110	int yyparse(void);
	111
	112	int yylex(void);
	113
	114	void lex_cleanup(void);
	115
	116	void lex_init(char *buf);
	117
	118	int ScreenIdentString(char *string);
	119
	120	/*
	121	* Returns next free slot in blocklist
	122	*/
	123	uint32_t NewBlock(uint32_t offset, uint64_t mask, uint64_t value, uint16_t comp, uint32_t function, void *data);
	124
	125	/*
	126	* Connects the to blocks b1 and b2 ( AND ) and returns index of superblock
	127	*/
	128	uint32_t Connect_AND(uint32_t b1, uint32_t b2);
	129
	130	/*
	131	* Connects the to blocks b1 and b2 ( OR ) and returns index of superblock
	132	*/
	133	uint32_t Connect_OR(uint32_t b1, uint32_t b2);
	134
	135	/*
	136	* Inverts OnTrue and OnFalse
	137	*/
	138	uint32_t Invert(uint32_t a );
	139
	140	/*
	141	* Add label to filter index
	142	*/
	143	void AddLabel(uint32_t index, char *label);
	144
	145	/*
	146	* Add Ident to Identlist
	147	*/
	148	uint32_t AddIdent(char *Ident);
	149
	150	#endif //_FILTER_H

-10

bin/flist.c less more

0	0	/*
1		* Copyright (c) 2014, Peter Haag
2		* Copyright (c) 2009, Peter Haag
	1	* Copyright (c) 2009-2020, Peter Haag
3	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
4	3	* All rights reserved.
5	4	*

27	26	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
28	27	* POSSIBILITY OF SUCH DAMAGE.
29	28	*
30		* $Author: haag $
31		*
32		* $Id: flist.c 39 2009-11-25 08:11:15Z haag $
33		*
34		* $LastChangedRevision: 39 $
35		*
36	29	*/
37	30
38	31	#include "config.h"

67	60	#define fts_set fts_set_compat
68	61	#endif
69	62
	63	#include "util.h"
	64	#include "nfdump.h"
70	65	#include "nffile.h"
71		#include "util.h"
72	66	#include "flist.h"
73	67
74	68	/*

1111	1105	struct tm *t_tm;
1112	1106	int i;
1113	1107
1114		if ( strlen(filename) == 19 && (strncmp(filename, "nfcapd.", 7) == 0) ) {
	1108	size_t len = strlen(filename);
	1109	if ( (len == 19 \|\| len == 21) && (strncmp(filename, "nfcapd.", 7) == 0) ) {
1115	1110	char *p = &filename[7];
1116	1111	time_t t = ISO2UNIX(p);
1117	1112	t_tm = localtime(&t);

-3

bin/flowtree.c less more

0	0	/*
1		* Copyright (c) 2011-2019, Peter Haag
	1	* Copyright (c) 2011-2020, Peter Haag
2	2	* All rights reserved.
3	3	*
4	4	* Redistribution and use in source and binary forms, with or without

44	44	#include <pthread.h>
45	45	#include <assert.h>
46	46
47		#include "rbtree.h"
	47	#include "util.h"
	48	#include "nfdump.h"
48	49	#include "nffile.h"
49	50	#include "bookkeeper.h"
50	51	#include "collector.h"
51	52	#include "netflow_pcap.h"
52		#include "util.h"
53	53	#include "flowtree.h"
54	54
55	55	static void spin_lock(int *p);

+12

-14

bin/ft2nfdump.c less more

0	0	/*
1	1	* All rights reserved.
2		* Copyright (c) 2017, Peter Haag
3		* Copyright (c) 2016, Peter Haag
4		* Copyright (c) 2014, Peter Haag
5		* Copyright (c) 2009, Peter Haag
	2	* Copyright (c) 2009-2020, Peter Haag
6	3	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
7	4	* Copyright (c) 2001 Mark Fullmer and The Ohio State University
8	5	* All rights reserved.

57	54	#include <stdint.h>
58	55	#endif
59	56
60		#include "ftlib.h"
61		#include "nf_common.h"
	57	#include "util.h"
	58	#include "nfdump.h"
62	59	#include "nffile.h"
63	60	#include "nfx.h"
64		#include "launch.h"
65		#include "util.h"
	61	#include "exporter.h"
	62	#include "ftlib.h"
	63	#include "output_raw.h"
66	64
67	65	/* Global defines */
68	66	#define MAXRECORDS 30

155	153	extension_info->map->ex_id[i++] = EX_MULIPLE;
156	154	}
157	155
158		if (!ftio_check_xfield(ftio, FT_XFIELD_NEXTHOP )) {
	156	if (!ftio_check_xfield(ftio, FT_XFIELD_PEER_NEXTHOP )) {
159	157	extension_info->map->ex_id[i++] = EX_NEXT_HOP_v4;
160	158	}
161	159

267	265	record.dir = 0;
268	266	record.dst_tos = 0;
269	267	break;
	268	case EX_NEXT_HOP_v4:
	269	record.ip_nexthop.V4 = ((uint32_t)(rec+fo.peer_nexthop));
	270	break;
270	271	case EX_ROUTER_IP_v4:
271		record.ip_nexthop.V4 = ((uint32_t)(rec+fo.peer_nexthop));
272		break;
273		case EX_NEXT_HOP_v4:
274		record.ip_router.V4 = ((uint32_t)(rec+fo.router_sc));
	272	record.ip_router.V4 = ((uint32_t)(rec+fo.exaddr));
275	273	break;
276	274	case EX_ROUTER_ID:
277	275	record.engine_type = ((uint8_t)(rec+fo.engine_type));

286	284
287	285	if ( extended ) {
288	286	char *string;
289		format_file_block_record(&record, &string, 0);
	287	flow_record_to_raw(&record, &string, 0);
290	288	fprintf(stderr, "%s\n", string);
291	289	}
292	290

+83

-5

bin/grammar.y less more

0	0	/*
1		* Copyright (c) 2017
2		* Copyright (c) 2016
	1	* Copyright (c) 2016-2020, Peter Haag
3	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
4	3	* All rights reserved.
5	4	*

48	47	#include <stdint.h>
49	48	#endif
50	49
51		#include "nf_common.h"
	50	#include "util.h"
	51	#include "output_util.h"
52	52	#include "rbtree.h"
	53	#include "filter.h"
53	54	#include "nfdump.h"
54	55	#include "nffile.h"
55	56	#include "nftree.h"
56	57	#include "ipconv.h"
57		#include "util.h"
	58
	59	#define AnyMask 0xffffffffffffffffLL
58	60
59	61	/*
60	62	* function prototypes

64	66	static uint32_t ChainHosts(uint64_t offsets, uint64_t hostlist, int num_records, int type);
65	67
66	68	static uint64_t VerifyMac(char *s);
	69
	70	static int InitSymbols(void);
	71
	72	static uint32_t Get_fwd_status_id(char *status);
67	73
68	74	enum { DIR_UNSPEC = 1,
69	75	SOURCE, DESTINATION, SOURCE_AND_DESTINATION, SOURCE_OR_DESTINATION,

83	89	extern char *FilterFilename;
84	90
85	91	static uint32_t num_ip;
	92
	93	static struct fwd_status_def_s {
	94	uint32_t id;
	95	char *name;
	96	} fwd_status_def_list[] = {
	97	{ 0, "Ukwn"}, // Unknown
	98	{ 1, "Forw"}, // Normal forwarding
	99	{ 2, "Frag"}, // Fragmented
	100	{ 16, "Drop"}, // Drop
	101	{ 17, "DaclD"}, // Drop ACL deny
	102	{ 18, "Daclp"}, // Drop ACL drop
	103	{ 19, "Noroute"}, // Unroutable
	104	{ 20, "Dadj"}, // Drop Adjacency
	105	{ 21, "Dfrag"}, // Drop Fragmentation & DF set
	106	{ 22, "Dbadh"}, // Drop Bad header checksum
	107	{ 23, "Dbadtlen"}, // Drop Bad total Length
	108	{ 24, "Dbadhlen"}, // Drop Bad Header Length
	109	{ 25, "DbadTTL"}, // Drop bad TTL
	110	{ 26, "Dpolicy"}, // Drop Policer
	111	{ 27, "Dwred"}, // Drop WRED
	112	{ 28, "Drpf"}, // Drop RPF
	113	{ 29, "Dforus"}, // Drop For us
	114	{ 30, "DbadOf"}, // Drop Bad output interface
	115	{ 31, "Dhw"}, // Drop Hardware
	116	{ 128, "Term"}, // Terminate
	117	{ 129, "Tadj"}, // Terminate Punt Adjacency
	118	{ 130, "TincAdj"}, // Terminate Incomplete Adjacency
	119	{ 131, "Tforus"}, // Terminate For us
	120	{ 0, NULL} // Last entry
	121	};
	122
	123	static char **fwd_status = NULL;
86	124
87	125	char yyerror_buff[256];
88	126

162	200
163	201	\| PROTO STRING {
164	202	int64_t proto;
165		proto = Proto_num($2);
	203	proto = ProtoNum($2);
166	204
167	205	if ( proto > 255 ) {
168	206	yyerror("Protocol number > 255");

2213	2251	return mac;
2214	2252
2215	2253	} // End of VerifyMac
	2254
	2255	static int InitSymbols(void) {
	2256	int i;
	2257
	2258	// already initialised?
	2259	if ( fwd_status )
	2260	return 1;
	2261
	2262	// fill fwd status cache table
	2263	fwd_status = ( char *)calloc(256, sizeof(char ));
	2264	if ( !fwd_status ) {
	2265	fprintf(stderr, "malloc(): %s line %d: %s", __FILE__, __LINE__, strerror (errno));
	2266	return 0;
	2267	}
	2268	i=0;
	2269	while ( fwd_status_def_list[i].name ) {
	2270	uint32_t j = fwd_status_def_list[i].id;
	2271	fwd_status[j] = fwd_status_def_list[i].name;
	2272	i++;
	2273	}
	2274	return 1;
	2275
	2276	} // End of InitSymbols
	2277
	2278	static uint32_t Get_fwd_status_id(char *status) {
	2279	int i;
	2280
	2281	if ( !fwd_status && !InitSymbols() )
	2282	yyerror("malloc() error");
	2283
	2284	i = 0;
	2285	while ( i < 256 ) {
	2286	if ( fwd_status[i] && strcasecmp(fwd_status[i], status) == 0 )
	2287	return i;
	2288	i++;
	2289	}
	2290	return 256;
	2291
	2292	} // End of Get_fwd_status_id
	2293

-1732

~~bin/grammar.y.mine~~ less more

0		/*
1		* This file is part of the nfdump project.
2		*
3		* Copyright (c) 2004, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
4		* All rights reserved.
5		*
6		* Redistribution and use in source and binary forms, with or without
7		* modification, are permitted provided that the following conditions are met:
8		*
9		* * Redistributions of source code must retain the above copyright notice,
10		* this list of conditions and the following disclaimer.
11		* * Redistributions in binary form must reproduce the above copyright notice,
12		* this list of conditions and the following disclaimer in the documentation
13		* and/or other materials provided with the distribution.
14		* * Neither the name of SWITCH nor the names of its contributors may be
15		* used to endorse or promote products derived from this software without
16		* specific prior written permission.
17		*
18		* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
19		* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
20		* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
21		* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
22		* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
23		* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
24		* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
25		* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
26		* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
27		* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
28		* POSSIBILITY OF SUCH DAMAGE.
29		*
30		* $Author: peter $
31		*
32		* $Id: grammar.y 100 2008-08-15 11:36:21Z peter $
33		*
34		* $LastChangedRevision: 100 $
35		*
36		*
37		*
38		*/
39
40		%{
41
42		#include "config.h"
43
44		#include <stdio.h>
45		#include <sys/types.h>
46		#include <sys/socket.h>
47		#include <netinet/in.h>
48		#include <string.h>
49		#include <stdlib.h>
50		#include <ctype.h>
51
52		#ifdef HAVE_STDINT_H
53		#include <stdint.h>
54		#endif
55
56		#include "nf_common.h"
57		#include "rbtree.h"
58		#include "nfdump.h"
59		#include "nffile.h"
60		#include "nftree.h"
61		#include "ipconv.h"
62		#include "util.h"
63
64		/*
65		* function prototypes
66		*/
67		static void yyerror(char *msg);
68
69		static uint32_t ChainHosts(uint64_t *hostlist, int num_records, int type);
70
71		static uint64_t VerifyMac(char *s);
72
73		enum { DIR_UNSPEC = 1, SOURCE, DESTINATION, SOURCE_AND_DESTINATION, SOURCE_OR_DESTINATION, DIR_IN, DIR_OUT, IN_SRC, IN_DST, OUT_SRC, OUT_DST };
74
75		/* var defs */
76		extern int lineno;
77		extern char *yytext;
78		extern uint64_t *IPstack;
79		extern uint32_t StartNode;
80		extern uint16_t Extended;
81		extern int (FilterEngine)(uint32_t );
82		extern char *FilterFilename;
83
84		static uint32_t num_ip;
85
86		#define MPLSMAX 0x00ffffff
87		%}
88
89		%union {
90		uint64_t value;
91		char *s;
92		FilterParam_t param;
93		void *list;
94		}
95
96		%token ANY IP IF MAC MPLS TOS DIR FLAGS PROTO MASK HOSTNAME NET PORT FWDSTAT IN OUT SRC DST EQ LT GT
97		%token NUMBER STRING IDENT ALPHA_FLAGS PROTOSTR PORTNUM ICMP_TYPE ICMP_CODE ENGINE_TYPE ENGINE_ID AS PACKETS BYTES FLOWS
98		%token PPS BPS BPP DURATION
99		%token IPV4 IPV6 NEXTHOP BGPNEXTHOP ROUTER VLAN
100		%token NOT END
101		%type <value> expr NUMBER PORTNUM ICMP_TYPE ICMP_CODE
102		%type <s> STRING IDENT ALPHA_FLAGS PROTOSTR
103		%type <param> dqual term comp
104		%type <list> iplist ullist
105
106		%left '+' OR
107		%left '*' AND
108		%left NEGATE
109
110		%%
111		prog: /* empty */
112		\| expr {
113		StartNode = $1;
114		}
115		;
116
117		term: ANY { /* this is an unconditionally true expression, as a filter applies in any case */
118		$$.self = NewBlock(OffsetProto, 0, 0, CMP_EQ, FUNC_NONE, NULL );
119		}
120
121		\| IDENT STRING {
122		if ( !ScreenIdentString($2) ) {
123		yyerror("Illegal ident string");
124		YYABORT;
125		}
126
127		uint32_t index = AddIdent($2);
128		$$.self = NewBlock(0, 0, index, CMP_IDENT, FUNC_NONE, NULL );
129		}
130
131		\| IPV4 {
132		$$.self = NewBlock(OffsetRecordFlags, (1LL << ShiftRecordFlags) & MaskRecordFlags,
133		(0LL << ShiftRecordFlags) & MaskRecordFlags, CMP_EQ, FUNC_NONE, NULL);
134		}
135
136		\| IPV6 {
137		$$.self = NewBlock(OffsetRecordFlags, (1LL << ShiftRecordFlags) & MaskRecordFlags,
138		(1LL << ShiftRecordFlags) & MaskRecordFlags, CMP_EQ, FUNC_NONE, NULL);
139		}
140
141		\| PROTO NUMBER {
142		int64_t proto;
143		proto = $2;
144
145		if ( proto > 255 ) {
146		yyerror("Protocol number > 255");
147		YYABORT;
148		}
149		if ( proto < 0 ) {
150		yyerror("Unknown protocol");
151		YYABORT;
152		}
153		$$.self = NewBlock(OffsetProto, MaskProto, (proto << ShiftProto) & MaskProto, CMP_EQ, FUNC_NONE, NULL);
154
155		}
156
157		\| PROTO STRING {
158		int64_t proto;
159		proto = Proto_num($2);
160
161		if ( proto > 255 ) {
162		yyerror("Protocol number > 255");
163		YYABORT;
164		}
165		if ( proto < 0 ) {
166		yyerror("Unknown protocol");
167		YYABORT;
168		}
169		$$.self = NewBlock(OffsetProto, MaskProto, (proto << ShiftProto) & MaskProto, CMP_EQ, FUNC_NONE, NULL);
170		}
171
172		\| dqual PACKETS comp NUMBER {
173
174		switch ( $$.direction ) {
175		case DIR_UNSPEC:
176		case DIR_IN:
177		$$.self = NewBlock(OffsetPackets, MaskPackets, $4, $3.comp, FUNC_NONE, NULL);
178		break;
179		case DIR_OUT:
180		$$.self = NewBlock(OffsetOutPackets, MaskPackets, $4, $3.comp, FUNC_NONE, NULL);
181		break;
182		case SOURCE:
183		case DESTINATION:
184		case SOURCE_OR_DESTINATION:
185		case SOURCE_AND_DESTINATION:
186		case IN_SRC:
187		case IN_DST:
188		case OUT_SRC:
189		case OUT_DST:
190		yyerror("This token is not expected here!");
191		YYABORT;
192		break;
193		default:
194		/* should never happen */
195		yyerror("Internal parser error");
196		YYABORT;
197		} // End of switch
198
199		}
200
201		\| dqual BYTES comp NUMBER {
202
203		switch ( $$.direction ) {
204		case DIR_UNSPEC:
205		case DIR_IN:
206		$$.self = NewBlock(OffsetBytes, MaskBytes, $4, $3.comp, FUNC_NONE, NULL);
207		break;
208		case DIR_OUT:
209		$$.self = NewBlock(OffsetOutBytes, MaskBytes, $4, $3.comp, FUNC_NONE, NULL);
210		break;
211		case SOURCE:
212		case DESTINATION:
213		case SOURCE_OR_DESTINATION:
214		case SOURCE_AND_DESTINATION:
215		case IN_SRC:
216		case IN_DST:
217		case OUT_SRC:
218		case OUT_DST:
219		yyerror("This token is not expected here!");
220		YYABORT;
221		break;
222		default:
223		/* should never happen */
224		yyerror("Internal parser error");
225		YYABORT;
226		} // End of switch
227
228		}
229
230		\| FLOWS comp NUMBER {
231		$$.self = NewBlock(OffsetAggrFlows, MaskFlows, $3, $2.comp, FUNC_NONE, NULL);
232		}
233
234		\| PPS comp NUMBER {
235		$$.self = NewBlock(0, AnyMask, $3, $2.comp, FUNC_PPS, NULL);
236		}
237
238		\| BPS comp NUMBER {
239		$$.self = NewBlock(0, AnyMask, $3, $2.comp, FUNC_BPS, NULL);
240		}
241
242		\| BPP comp NUMBER {
243		$$.self = NewBlock(0, AnyMask, $3, $2.comp, FUNC_BPP, NULL);
244		}
245
246		\| DURATION comp NUMBER {
247		$$.self = NewBlock(0, AnyMask, $3, $2.comp, FUNC_DURATION, NULL);
248		}
249
250		\| dqual TOS comp NUMBER {
251		if ( $4 > 255 ) {
252		yyerror("TOS must be 0..255");
253		YYABORT;
254		}
255
256		switch ( $$.direction ) {
257		case DIR_UNSPEC:
258		case SOURCE:
259		$$.self = NewBlock(OffsetTos, MaskTos, ($4 << ShiftTos) & MaskTos, $3.comp, FUNC_NONE, NULL);
260		break;
261		case DESTINATION:
262		$$.self = NewBlock(OffsetDstTos, MaskDstTos, ($4 << ShiftDstTos) & MaskDstTos, $3.comp, FUNC_NONE, NULL);
263		break;
264		case SOURCE_OR_DESTINATION:
265		$$.self = Connect_OR(
266		NewBlock(OffsetTos, MaskTos, ($4 << ShiftTos) & MaskTos, $3.comp, FUNC_NONE, NULL),
267		NewBlock(OffsetDstTos, MaskDstTos, ($4 << ShiftDstTos) & MaskDstTos, $3.comp, FUNC_NONE, NULL)
268		);
269		break;
270		case SOURCE_AND_DESTINATION:
271		$$.self = Connect_AND(
272		NewBlock(OffsetTos, MaskTos, ($4 << ShiftTos) & MaskTos, $3.comp, FUNC_NONE, NULL),
273		NewBlock(OffsetDstTos, MaskDstTos, ($4 << ShiftDstTos) & MaskDstTos, $3.comp, FUNC_NONE, NULL)
274		);
275		break;
276		case DIR_IN:
277		case DIR_OUT:
278		case IN_SRC:
279		case IN_DST:
280		case OUT_SRC:
281		case OUT_DST:
282		yyerror("This token is not expected here!");
283		YYABORT;
284		break;
285		default:
286		/* should never happen */
287		yyerror("Internal parser error");
288		YYABORT;
289		}
290		}
291
292		\| FLAGS comp NUMBER {
293		if ( $3 > 63 ) {
294		yyerror("Flags must be 0..63");
295		YYABORT;
296		}
297		$$.self = NewBlock(OffsetFlags, MaskFlags, ($3 << ShiftFlags) & MaskFlags, $2.comp, FUNC_NONE, NULL);
298		}
299
300		\| FLAGS STRING {
301		uint64_t fl = 0;
302		int cnt = 0;
303		size_t len = strlen($2);
304
305		if ( len > 7 ) {
306		yyerror("Too many flags");
307		YYABORT;
308		}
309
310		if ( strchr($2, 'F') ) { fl \|= 1; cnt++; }
311		if ( strchr($2, 'S') ) { fl \|= 2; cnt++; }
312		if ( strchr($2, 'R') ) { fl \|= 4; cnt++; }
313		if ( strchr($2, 'P') ) { fl \|= 8; cnt++; }
314		if ( strchr($2, 'A') ) { fl \|= 16; cnt++; }
315		if ( strchr($2, 'U') ) { fl \|= 32; cnt++; }
316		if ( strchr($2, 'X') ) { fl = 63; cnt++; }
317
318		if ( cnt != len ) {
319		yyerror("Too many flags");
320		YYABORT;
321		}
322
323		$$.self = NewBlock(OffsetFlags, (fl << ShiftFlags) & MaskFlags,
324		(fl << ShiftFlags) & MaskFlags, CMP_FLAGS, FUNC_NONE, NULL);
325		}
326
327		\| dqual IP STRING {
328		int af, bytes, ret;
329
330		ret = parse_ip(&af, $3, IPstack, &bytes, ALLOW_LOOKUP, &num_ip);
331
332		if ( ret == 0 ) {
333		yyerror("Error parsing IP address.");
334		YYABORT;
335		}
336
337		// ret == -1 will never happen here, as ALLOW_LOOKUP is set
338		if ( ret == -2 ) {
339		// could not resolv host => 'not any'
340		$$.self = Invert(NewBlock(OffsetProto, 0, 0, CMP_EQ, FUNC_NONE, NULL ));
341		} else {
342		if ( af && (( af == PF_INET && bytes != 4 ) \|\| ( af == PF_INET6 && bytes != 16 ))) {
343		yyerror("incomplete IP address");
344		YYABORT;
345		}
346
347		switch ( $$.direction ) {
348		case SOURCE:
349		case DESTINATION:
350		$$.self = ChainHosts(IPstack, num_ip, $$.direction);
351		break;
352		case DIR_UNSPEC:
353		case SOURCE_OR_DESTINATION: {
354		uint32_t src = ChainHosts(IPstack, num_ip, SOURCE);
355		uint32_t dst = ChainHosts(IPstack, num_ip, DESTINATION);
356		$$.self = Connect_OR(src, dst);
357		} break;
358		case SOURCE_AND_DESTINATION: {
359		uint32_t src = ChainHosts(IPstack, num_ip, SOURCE);
360		uint32_t dst = ChainHosts(IPstack, num_ip, DESTINATION);
361		$$.self = Connect_AND(src, dst);
362		} break;
363		case DIR_IN:
364		case DIR_OUT:
365		case IN_SRC:
366		case IN_DST:
367		case OUT_SRC:
368		case OUT_DST:
369		yyerror("This token is not expected here!");
370		YYABORT;
371		break;
372		default:
373		/* should never happen */
374		yyerror("Internal parser error");
375		YYABORT;
376
377		} // End of switch
378
379		}
380		}
381
382		\| dqual IP IN '[' iplist ']' {
383
384		$$.direction = $1.direction;
385
386		switch ( $$.direction ) {
387		case SOURCE:
388		$$.self = NewBlock(OffsetSrcIPv6a, MaskIPv6, 0 , CMP_IPLIST, FUNC_NONE, (void *)$5 );
389		break;
390		case DESTINATION:
391		$$.self = NewBlock(OffsetDstIPv6a, MaskIPv6, 0 , CMP_IPLIST, FUNC_NONE, (void *)$5 );
392		break;
393		case DIR_UNSPEC:
394		case SOURCE_OR_DESTINATION:
395		$$.self = Connect_OR(
396		NewBlock(OffsetSrcIPv6a, MaskIPv6, 0 , CMP_IPLIST, FUNC_NONE, (void *)$5 ),
397		NewBlock(OffsetDstIPv6a, MaskIPv6, 0 , CMP_IPLIST, FUNC_NONE, (void *)$5 )
398		);
399		break;
400		case SOURCE_AND_DESTINATION:
401		$$.self = Connect_AND(
402		NewBlock(OffsetSrcIPv6a, MaskIPv6, 0 , CMP_IPLIST, FUNC_NONE, (void *)$5 ),
403		NewBlock(OffsetDstIPv6a, MaskIPv6, 0 , CMP_IPLIST, FUNC_NONE, (void *)$5 )
404		);
405		break;
406		case DIR_IN:
407		case DIR_OUT:
408		case IN_SRC:
409		case IN_DST:
410		case OUT_SRC:
411		case OUT_DST:
412		yyerror("This token is not expected here!");
413		YYABORT;
414		break;
415		default:
416		/* should never happen */
417		yyerror("Internal parser error");
418		YYABORT;
419		}
420		}
421
422		\| NEXTHOP IP STRING {
423		int af, bytes, ret;
424
425		ret = parse_ip(&af, $3, IPstack, &bytes, STRICT_IP, &num_ip);
426
427		if ( ret == 0 ) {
428		yyerror("Error parsing IP address.");
429		YYABORT;
430		}
431
432		if ( ret == -1 ) {
433		yyerror("IP address required - hostname not allowed here.");
434		YYABORT;
435		}
436		// ret == -2 will never happen here, as STRICT_IP is set
437
438		if ( af && (( af == PF_INET && bytes != 4 ) \|\| ( af == PF_INET6 && bytes != 16 ))) {
439		yyerror("incomplete IP address");
440		YYABORT;
441		}
442
443		$$.self = Connect_AND(
444		NewBlock(OffsetNexthopv6b, MaskIPv6, IPstack[1] , CMP_EQ, FUNC_NONE, NULL ),
445		NewBlock(OffsetNexthopv6a, MaskIPv6, IPstack[0] , CMP_EQ, FUNC_NONE, NULL )
446		);
447		}
448
449		\| BGPNEXTHOP IP STRING {
450		int af, bytes, ret;
451
452		ret = parse_ip(&af, $3, IPstack, &bytes, STRICT_IP, &num_ip);
453
454		if ( ret == 0 ) {
455		yyerror("Error parsing IP address.");
456		YYABORT;
457		}
458
459		if ( ret == -1 ) {
460		yyerror("IP address required - hostname not allowed here.");
461		YYABORT;
462		}
463		// ret == -2 will never happen here, as STRICT_IP is set
464
465		if ( af && (( af == PF_INET && bytes != 4 ) \|\| ( af == PF_INET6 && bytes != 16 ))) {
466		yyerror("incomplete IP address");
467		YYABORT;
468		}
469
470		$$.self = Connect_AND(
471		NewBlock(OffsetBGPNexthopv6b, MaskIPv6, IPstack[1] , CMP_EQ, FUNC_NONE, NULL ),
472		NewBlock(OffsetBGPNexthopv6a, MaskIPv6, IPstack[0] , CMP_EQ, FUNC_NONE, NULL )
473		);
474		}
475
476		\| ROUTER IP STRING {
477		int af, bytes, ret;
478
479		ret = parse_ip(&af, $3, IPstack, &bytes, STRICT_IP, &num_ip);
480
481		if ( ret == 0 ) {
482		yyerror("Error parsing IP address.");
483		YYABORT;
484		}
485
486		if ( ret == -1 ) {
487		yyerror("IP address required - hostname not allowed here.");
488		YYABORT;
489		}
490		// ret == -2 will never happen here, as STRICT_IP is set
491
492		if ( af && (( af == PF_INET && bytes != 4 ) \|\| ( af == PF_INET6 && bytes != 16 ))) {
493		yyerror("incomplete IP address");
494		YYABORT;
495		}
496
497		$$.self = Connect_AND(
498		NewBlock(OffsetRouterv6b, MaskIPv6, IPstack[1] , CMP_EQ, FUNC_NONE, NULL ),
499		NewBlock(OffsetRouterv6a, MaskIPv6, IPstack[0] , CMP_EQ, FUNC_NONE, NULL )
500		);
501		}
502
503		\| dqual PORT comp NUMBER {
504		$$.direction = $1.direction;
505		if ( $4 > 65535 ) {
506		yyerror("Port outside of range 0..65535");
507		YYABORT;
508		}
509
510		switch ( $$.direction ) {
511		case SOURCE:
512		$$.self = NewBlock(OffsetPort, MaskSrcPort, ($4 << ShiftSrcPort) & MaskSrcPort, $3.comp, FUNC_NONE, NULL );
513		break;
514		case DESTINATION:
515		$$.self = NewBlock(OffsetPort, MaskDstPort, ($4 << ShiftDstPort) & MaskDstPort, $3.comp, FUNC_NONE, NULL );
516		break;
517		case DIR_UNSPEC:
518		case SOURCE_OR_DESTINATION:
519		$$.self = Connect_OR(
520		NewBlock(OffsetPort, MaskSrcPort, ($4 << ShiftSrcPort) & MaskSrcPort, $3.comp, FUNC_NONE, NULL ),
521		NewBlock(OffsetPort, MaskDstPort, ($4 << ShiftDstPort) & MaskDstPort, $3.comp, FUNC_NONE, NULL )
522		);
523		break;
524		case SOURCE_AND_DESTINATION:
525		$$.self = Connect_AND(
526		NewBlock(OffsetPort, MaskSrcPort, ($4 << ShiftSrcPort) & MaskSrcPort, $3.comp, FUNC_NONE, NULL ),
527		NewBlock(OffsetPort, MaskDstPort, ($4 << ShiftDstPort) & MaskDstPort, $3.comp, FUNC_NONE, NULL )
528		);
529		break;
530		case DIR_IN:
531		case DIR_OUT:
532		case IN_SRC:
533		case IN_DST:
534		case OUT_SRC:
535		case OUT_DST:
536		yyerror("This token is not expected here!");
537		YYABORT;
538		break;
539		default:
540		/* should never happen */
541		yyerror("Internal parser error");
542		YYABORT;
543		} // End switch
544
545		}
546
547		\| dqual PORT IN '[' ullist ']' {
548		struct ULongListNode *node;
549		ULongtree_t *root = NULL;
550
551		$$.direction = $1.direction;
552		if ( $$.direction == DIR_UNSPEC \|\| $$.direction == SOURCE_OR_DESTINATION \|\| $$.direction == SOURCE_AND_DESTINATION ) {
553		// src and/or dst port
554		// we need a second rbtree due to different shifts for src and dst ports
555		root = malloc(sizeof(ULongtree_t));
556
557		struct ULongListNode *n;
558		if ( root == NULL) {
559		yyerror("malloc() error");
560		YYABORT;
561		}
562		RB_INIT(root);
563
564		RB_FOREACH(node, ULongtree, (ULongtree_t *)$5) {
565		if ((n = malloc(sizeof(struct ULongListNode))) == NULL) {
566		yyerror("malloc() error");
567		YYABORT;
568		}
569		n->value = (node->value << ShiftDstPort) & MaskDstPort;
570		node->value = (node->value << ShiftSrcPort) & MaskSrcPort;
571		RB_INSERT(ULongtree, root, n);
572		}
573		}
574
575		switch ( $$.direction ) {
576		case SOURCE:
577		RB_FOREACH(node, ULongtree, (ULongtree_t *)$5) {
578		node->value = (node->value << ShiftSrcPort) & MaskSrcPort;
579		}
580		$$.self = NewBlock(OffsetPort, MaskSrcPort, 0, CMP_ULLIST, FUNC_NONE, (void *)$5 );
581		break;
582		case DESTINATION:
583		RB_FOREACH(node, ULongtree, (ULongtree_t *)$5) {
584		node->value = (node->value << ShiftDstPort) & MaskDstPort;
585		}
586		$$.self = NewBlock(OffsetPort, MaskDstPort, 0, CMP_ULLIST, FUNC_NONE, (void *)$5 );
587		break;
588		case DIR_UNSPEC:
589		case SOURCE_OR_DESTINATION:
590		$$.self = Connect_OR(
591		NewBlock(OffsetPort, MaskSrcPort, 0, CMP_ULLIST, FUNC_NONE, (void *)$5 ),
592		NewBlock(OffsetPort, MaskDstPort, 0, CMP_ULLIST, FUNC_NONE, (void *)root )
593		);
594		break;
595		case SOURCE_AND_DESTINATION:
596		$$.self = Connect_AND(
597		NewBlock(OffsetPort, MaskSrcPort, 0, CMP_ULLIST, FUNC_NONE, (void *)$5 ),
598		NewBlock(OffsetPort, MaskDstPort, 0, CMP_ULLIST, FUNC_NONE, (void *)root )
599		);
600		break;
601		case DIR_IN:
602		case DIR_OUT:
603		case IN_SRC:
604		case IN_DST:
605		case OUT_SRC:
606		case OUT_DST:
607		yyerror("This token is not expected here!");
608		YYABORT;
609		break;
610		default:
611		/* should never happen */
612		yyerror("Internal parser error");
613		YYABORT;
614		} // End of switch
615
616		}
617
618		\| ICMP_TYPE NUMBER {
619		if ( $2 > 255 ) {
620		yyerror("ICMP tpye of range 0..15");
621		YYABORT;
622		}
623		$$.self = Connect_AND(
624		// imply proto ICMP with a proto ICMP block
625		Connect_OR (
626		NewBlock(OffsetProto, MaskProto, ((uint64_t)IPPROTO_ICMP << ShiftProto) & MaskProto, CMP_EQ, FUNC_NONE, NULL),
627		NewBlock(OffsetProto, MaskProto, ((uint64_t)IPPROTO_ICMPV6 << ShiftProto) & MaskProto, CMP_EQ, FUNC_NONE, NULL)
628		),
629		NewBlock(OffsetPort, MaskICMPtype, ($2 << ShiftICMPtype) & MaskICMPtype, CMP_EQ, FUNC_NONE, NULL )
630		);
631
632		}
633
634		\| ICMP_CODE NUMBER {
635		if ( $2 > 255 ) {
636		yyerror("ICMP code of range 0..15");
637		YYABORT;
638		}
639		$$.self = Connect_AND(
640		// imply proto ICMP with a proto ICMP block
641		Connect_OR (
642		NewBlock(OffsetProto, MaskProto, ((uint64_t)IPPROTO_ICMP << ShiftProto) & MaskProto, CMP_EQ, FUNC_NONE, NULL),
643		NewBlock(OffsetProto, MaskProto, ((uint64_t)IPPROTO_ICMPV6 << ShiftProto) & MaskProto, CMP_EQ, FUNC_NONE, NULL)
644		),
645		NewBlock(OffsetPort, MaskICMPcode, ($2 << ShiftICMPcode) & MaskICMPcode, CMP_EQ, FUNC_NONE, NULL )
646		);
647
648		}
649
650		\| ENGINE_TYPE comp NUMBER {
651		if ( $3 > 255 ) {
652		yyerror("Engine type of range 0..255");
653		YYABORT;
654		}
655		$$.self = NewBlock(OffsetRouterID, MaskEngineType, ($3 << ShiftEngineType) & MaskEngineType, $2.comp, FUNC_NONE, NULL);
656
657		}
658
659		\| ENGINE_ID comp NUMBER {
660		if ( $3 > 255 ) {
661		yyerror("Engine ID of range 0..255");
662		YYABORT;
663		}
664		$$.self = NewBlock(OffsetRouterID, MaskEngineID, ($3 << ShiftEngineID) & MaskEngineID, $2.comp, FUNC_NONE, NULL);
665
666		}
667
668		\| dqual AS comp NUMBER {
669		$$.direction = $1.direction;
670		if ( $4 > 0x7FFFFFFF \|\| $4 < 0 ) {
671		yyerror("AS number of range");
672		YYABORT;
673		}
674
675		switch ( $$.direction ) {
676		case SOURCE:
677		$$.self = NewBlock(OffsetAS, MaskSrcAS, ($4 << ShiftSrcAS) & MaskSrcAS, $3.comp, FUNC_NONE, NULL );
678		break;
679		case DESTINATION:
680		$$.self = NewBlock(OffsetAS, MaskDstAS, ($4 << ShiftDstAS) & MaskDstAS, $3.comp, FUNC_NONE, NULL);
681		break;
682		case DIR_UNSPEC:
683		case SOURCE_OR_DESTINATION:
684		$$.self = Connect_OR(
685		NewBlock(OffsetAS, MaskSrcAS, ($4 << ShiftSrcAS) & MaskSrcAS, $3.comp, FUNC_NONE, NULL ),
686		NewBlock(OffsetAS, MaskDstAS, ($4 << ShiftDstAS) & MaskDstAS, $3.comp, FUNC_NONE, NULL)
687		);
688		break;
689		case SOURCE_AND_DESTINATION:
690		$$.self = Connect_AND(
691		NewBlock(OffsetAS, MaskSrcAS, ($4 << ShiftSrcAS) & MaskSrcAS, $3.comp, FUNC_NONE, NULL ),
692		NewBlock(OffsetAS, MaskDstAS, ($4 << ShiftDstAS) & MaskDstAS, $3.comp, FUNC_NONE, NULL)
693		);
694		break;
695		case DIR_IN:
696		case DIR_OUT:
697		case IN_SRC:
698		case IN_DST:
699		case OUT_SRC:
700		case OUT_DST:
701		yyerror("This token is not expected here!");
702		YYABORT;
703		break;
704		default:
705		/* should never happen */
706		yyerror("Internal parser error");
707		YYABORT;
708		} // End of switch
709
710		}
711
712		\| dqual AS IN '[' ullist ']' {
713		struct ULongListNode *node;
714		ULongtree_t *root = NULL;
715
716		$$.direction = $1.direction;
717		if ( $$.direction == DIR_UNSPEC \|\| $$.direction == SOURCE_OR_DESTINATION \|\| $$.direction == SOURCE_AND_DESTINATION ) {
718		// src and/or dst AS
719		// we need a second rbtree due to different shifts for src and dst AS
720		root = malloc(sizeof(ULongtree_t));
721
722		struct ULongListNode *n;
723		if ( root == NULL) {
724		yyerror("malloc() error");
725		YYABORT;
726		}
727		RB_INIT(root);
728
729		RB_FOREACH(node, ULongtree, (ULongtree_t *)$5) {
730
731		if ((n = malloc(sizeof(struct ULongListNode))) == NULL) {
732		yyerror("malloc() error");
733		YYABORT;
734		}
735		n->value = (node->value << ShiftDstAS) & MaskDstAS;
736		node->value = (node->value << ShiftSrcAS) & MaskSrcAS;
737		RB_INSERT(ULongtree, root, n);
738		}
739		}
740
741		switch ( $$.direction ) {
742		case SOURCE:
743		RB_FOREACH(node, ULongtree, (ULongtree_t *)$5) {
744		node->value = (node->value << ShiftSrcAS) & MaskSrcAS;
745		}
746		$$.self = NewBlock(OffsetAS, MaskSrcAS, 0, CMP_ULLIST, FUNC_NONE, (void *)$5 );
747		break;
748		case DESTINATION:
749		RB_FOREACH(node, ULongtree, (ULongtree_t *)$5) {
750		node->value = (node->value << ShiftDstAS) & MaskDstAS;
751		}
752		$$.self = NewBlock(OffsetAS, MaskDstAS, 0, CMP_ULLIST, FUNC_NONE, (void *)$5 );
753		break;
754		case DIR_UNSPEC:
755		case SOURCE_OR_DESTINATION:
756		$$.self = Connect_OR(
757		NewBlock(OffsetAS, MaskSrcAS, 0, CMP_ULLIST, FUNC_NONE, (void *)$5 ),
758		NewBlock(OffsetAS, MaskDstAS, 0, CMP_ULLIST, FUNC_NONE, (void *)root )
759		);
760		break;
761		case SOURCE_AND_DESTINATION:
762		$$.self = Connect_AND(
763		NewBlock(OffsetAS, MaskSrcAS, 0, CMP_ULLIST, FUNC_NONE, (void *)$5 ),
764		NewBlock(OffsetAS, MaskDstAS, 0, CMP_ULLIST, FUNC_NONE, (void *)root )
765		);
766		break;
767		case DIR_IN:
768		case DIR_OUT:
769		case IN_SRC:
770		case IN_DST:
771		case OUT_SRC:
772		case OUT_DST:
773		yyerror("This token is not expected here!");
774		YYABORT;
775		break;
776		default:
777		/* should never happen */
778		yyerror("Internal parser error");
779		YYABORT;
780		}
781
782		}
783
784		\| dqual MASK NUMBER {
785		$$.direction = $1.direction;
786		if ( $3 > 255 ) {
787		yyerror("Mask outside of range 0..255");
788		YYABORT;
789		}
790
791		switch ( $$.direction ) {
792		case SOURCE:
793		$$.self = NewBlock(OffsetMask, MaskSrcMask, ($3 << ShiftSrcMask) & MaskSrcMask, CMP_EQ, FUNC_NONE, NULL );
794		break;
795		case DESTINATION:
796		$$.self = NewBlock(OffsetMask, MaskDstMask, ($3 << ShiftDstMask) & MaskDstMask, CMP_EQ, FUNC_NONE, NULL );
797		break;
798		case DIR_UNSPEC:
799		case SOURCE_OR_DESTINATION:
800		$$.self = Connect_OR(
801		NewBlock(OffsetMask, MaskSrcMask, ($3 << ShiftSrcMask) & MaskSrcMask, CMP_EQ, FUNC_NONE, NULL ),
802		NewBlock(OffsetMask, MaskDstMask, ($3 << ShiftDstMask) & MaskDstMask, CMP_EQ, FUNC_NONE, NULL )
803		);
804		break;
805		case SOURCE_AND_DESTINATION:
806		$$.self = Connect_AND(
807		NewBlock(OffsetMask, MaskSrcMask, ($3 << ShiftSrcMask) & MaskSrcMask, CMP_EQ, FUNC_NONE, NULL ),
808		NewBlock(OffsetMask, MaskDstMask, ($3 << ShiftDstMask) & MaskDstMask, CMP_EQ, FUNC_NONE, NULL )
809		);
810		break;
811		case DIR_IN:
812		case DIR_OUT:
813		case IN_SRC:
814		case IN_DST:
815		case OUT_SRC:
816		case OUT_DST:
817		yyerror("This token is not expected here!");
818		YYABORT;
819		break;
820		default:
821		/* should never happen */
822		yyerror("Internal parser error");
823		YYABORT;
824		} // End switch
825
826		}
827
828		\| dqual NET STRING STRING {
829		int af, bytes, ret;
830		uint64_t mask[2];
831		ret = parse_ip(&af, $3, IPstack, &bytes, STRICT_IP, &num_ip);
832
833		if ( ret == 0 ) {
834		yyerror("Invalid IP address");
835		YYABORT;
836		}
837
838		if ( ret == -1 ) {
839		yyerror("IP address required - hostname not allowed here.");
840		YYABORT;
841		}
842		// ret == -2 will never happen here, as STRICT_IP is set
843
844		if ( af != PF_INET ) {
845		yyerror("IP netmask syntax valid only for IPv4");
846		YYABORT;
847		}
848		if ( bytes != 4 ) {
849		yyerror("Need complete IP address");
850		YYABORT;
851		}
852
853		ret = parse_ip(&af, $4, mask, &bytes, STRICT_IP, &num_ip);
854		if ( ret == 0 ) {
855		yyerror("Invalid IP address");
856		YYABORT;
857		}
858		if ( ret == -1 ) {
859		yyerror("IP address required - hostname not allowed here.");
860		YYABORT;
861		}
862		// ret == -2 will never happen here, as STRICT_IP is set
863
864		if ( af != PF_INET \|\| bytes != 4 ) {
865		yyerror("Invalid netmask for IPv4 address");
866		YYABORT;
867		}
868
869		IPstack[0] &= mask[0];
870		IPstack[1] &= mask[1];
871
872		$$.direction = $1.direction;
873
874		switch ( $$.direction ) {
875		case SOURCE:
876		$$.self = Connect_AND(
877		NewBlock(OffsetSrcIPv6b, mask[1], IPstack[1] , CMP_EQ, FUNC_NONE, NULL ),
878		NewBlock(OffsetSrcIPv6a, mask[0], IPstack[0] , CMP_EQ, FUNC_NONE, NULL )
879		);
880		break;
881		case DESTINATION:
882		$$.self = Connect_AND(
883		NewBlock(OffsetDstIPv6b, mask[1], IPstack[1] , CMP_EQ, FUNC_NONE, NULL ),
884		NewBlock(OffsetDstIPv6a, mask[0], IPstack[0] , CMP_EQ, FUNC_NONE, NULL )
885		);
886		break;
887		case DIR_UNSPEC:
888		case SOURCE_OR_DESTINATION:
889		$$.self = Connect_OR(
890		Connect_AND(
891		NewBlock(OffsetSrcIPv6b, mask[1], IPstack[1] , CMP_EQ, FUNC_NONE, NULL ),
892		NewBlock(OffsetSrcIPv6a, mask[0], IPstack[0] , CMP_EQ, FUNC_NONE, NULL )
893		),
894		Connect_AND(
895		NewBlock(OffsetDstIPv6b, mask[1], IPstack[1] , CMP_EQ, FUNC_NONE, NULL ),
896		NewBlock(OffsetDstIPv6a, mask[0], IPstack[0] , CMP_EQ, FUNC_NONE, NULL )
897		)
898		);
899		break;
900		case SOURCE_AND_DESTINATION:
901		$$.self = Connect_AND(
902		Connect_AND(
903		NewBlock(OffsetSrcIPv6b, mask[1], IPstack[1] , CMP_EQ, FUNC_NONE, NULL ),
904		NewBlock(OffsetSrcIPv6a, mask[0], IPstack[0] , CMP_EQ, FUNC_NONE, NULL )
905		),
906		Connect_AND(
907		NewBlock(OffsetDstIPv6b, mask[1], IPstack[1] , CMP_EQ, FUNC_NONE, NULL ),
908		NewBlock(OffsetDstIPv6a, mask[0], IPstack[0] , CMP_EQ, FUNC_NONE, NULL )
909		)
910		);
911		break;
912		case DIR_IN:
913		case DIR_OUT:
914		case IN_SRC:
915		case IN_DST:
916		case OUT_SRC:
917		case OUT_DST:
918		yyerror("This token is not expected here!");
919		YYABORT;
920		break;
921		default:
922		/* should never happen */
923		yyerror("Internal parser error");
924		YYABORT;
925		} // End of switch
926
927		}
928
929		\| dqual NET STRING '/' NUMBER {
930		int af, bytes, ret;
931		uint64_t mask[2];
932
933		ret = parse_ip(&af, $3, IPstack, &bytes, STRICT_IP, &num_ip);
934		if ( ret == 0 ) {
935		yyerror("Invalid IP address");
936		YYABORT;
937		}
938		if ( ret == -1 ) {
939		yyerror("IP address required - hostname not allowed here.");
940		YYABORT;
941		}
942		// ret == -2 will never happen here, as STRICT_IP is set
943
944
945		if ( $5 > (bytes*8) ) {
946		yyerror("Too many netbits for this IP address");
947		YYABORT;
948		}
949
950		if ( af == PF_INET ) {
951		mask[0] = 0xffffffffffffffffLL;
952		mask[1] = 0xffffffffffffffffLL << ( 32 - $5 );
953		} else { // PF_INET6
954		if ( $5 > 64 ) {
955		mask[0] = 0xffffffffffffffffLL;
956		mask[1] = 0xffffffffffffffffLL << ( 128 - $5 );
957		} else {
958		mask[0] = 0xffffffffffffffffLL << ( 64 - $5 );
959		mask[1] = 0;
960		}
961		}
962		// IP aadresses are stored in network representation
963		mask[0] = mask[0];
964		mask[1] = mask[1];
965
966		IPstack[0] &= mask[0];
967		IPstack[1] &= mask[1];
968
969		$$.direction = $1.direction;
970		switch ( $$.direction ) {
971		case SOURCE:
972		$$.self = Connect_AND(
973		NewBlock(OffsetSrcIPv6b, mask[1], IPstack[1] , CMP_EQ, FUNC_NONE, NULL ),
974		NewBlock(OffsetSrcIPv6a, mask[0], IPstack[0] , CMP_EQ, FUNC_NONE, NULL )
975		);
976		break;
977		case DESTINATION:
978		$$.self = Connect_AND(
979		NewBlock(OffsetDstIPv6b, mask[1], IPstack[1] , CMP_EQ, FUNC_NONE, NULL ),
980		NewBlock(OffsetDstIPv6a, mask[0], IPstack[0] , CMP_EQ, FUNC_NONE, NULL )
981		);
982		break;
983		case DIR_UNSPEC:
984		case SOURCE_OR_DESTINATION:
985		$$.self = Connect_OR(
986		Connect_AND(
987		NewBlock(OffsetSrcIPv6b, mask[1], IPstack[1] , CMP_EQ, FUNC_NONE, NULL ),
988		NewBlock(OffsetSrcIPv6a, mask[0], IPstack[0] , CMP_EQ, FUNC_NONE, NULL )
989		),
990		Connect_AND(
991		NewBlock(OffsetDstIPv6b, mask[1], IPstack[1] , CMP_EQ, FUNC_NONE, NULL ),
992		NewBlock(OffsetDstIPv6a, mask[0], IPstack[0] , CMP_EQ, FUNC_NONE, NULL )
993		)
994		);
995		break;
996		case SOURCE_AND_DESTINATION:
997		$$.self = Connect_AND(
998		Connect_AND(
999		NewBlock(OffsetSrcIPv6b, mask[1], IPstack[1] , CMP_EQ, FUNC_NONE, NULL ),
1000		NewBlock(OffsetSrcIPv6a, mask[0], IPstack[0] , CMP_EQ, FUNC_NONE, NULL )
1001		),
1002		Connect_AND(
1003		NewBlock(OffsetDstIPv6b, mask[1], IPstack[1] , CMP_EQ, FUNC_NONE, NULL ),
1004		NewBlock(OffsetDstIPv6a, mask[0], IPstack[0] , CMP_EQ, FUNC_NONE, NULL )
1005		)
1006		);
1007		break;
1008		case DIR_IN:
1009		case DIR_OUT:
1010		case IN_SRC:
1011		case IN_DST:
1012		case OUT_SRC:
1013		case OUT_DST:
1014		yyerror("This token is not expected here!");
1015		YYABORT;
1016		break;
1017		default:
1018		/* should never happen */
1019		yyerror("Internal parser error");
1020		YYABORT;
1021		} // End of switch
1022
1023		}
1024
1025		\| dqual IF NUMBER {
1026		if ( $3 > 65535 ) {
1027		yyerror("Input interface number must be 0..65535");
1028		YYABORT;
1029		}
1030
1031		switch ( $$.direction ) {
1032		case DIR_UNSPEC:
1033		$$.self = Connect_OR(
1034		NewBlock(OffsetInOut, MaskInput, ($3 << ShiftInput) & MaskInput, CMP_EQ, FUNC_NONE, NULL),
1035		NewBlock(OffsetInOut, MaskOutput, ($3 << ShiftOutput) & MaskOutput, CMP_EQ, FUNC_NONE, NULL)
1036		);
1037		break;
1038		case DIR_IN:
1039		$$.self = NewBlock(OffsetInOut, MaskInput, ($3 << ShiftInput) & MaskInput, CMP_EQ, FUNC_NONE, NULL);
1040		break;
1041		case DIR_OUT:
1042		$$.self = NewBlock(OffsetInOut, MaskOutput, ($3 << ShiftOutput) & MaskOutput, CMP_EQ, FUNC_NONE, NULL);
1043		break;
1044		case SOURCE:
1045		case DESTINATION:
1046		case SOURCE_OR_DESTINATION:
1047		case SOURCE_AND_DESTINATION:
1048		case IN_SRC:
1049		case IN_DST:
1050		case OUT_SRC:
1051		case OUT_DST:
1052		yyerror("This token is not expected here!");
1053		YYABORT;
1054		break;
1055		default:
1056		/* should never happen */
1057		yyerror("Internal parser error");
1058		YYABORT;
1059		} // End of switch
1060
1061		}
1062
1063		\| dqual VLAN NUMBER {
1064		$$.direction = $1.direction;
1065		if ( $3 > 65535 \|\| $3 < 0 ) {
1066		yyerror("VLAN number of range 0..65535");
1067		YYABORT;
1068		}
1069
1070		switch ( $$.direction ) {
1071		case SOURCE:
1072		$$.self = NewBlock(OffsetVlan, MaskSrcVlan, ($3 << ShiftSrcVlan) & MaskSrcVlan, CMP_EQ, FUNC_NONE, NULL );
1073		break;
1074		case DESTINATION:
1075		$$.self = NewBlock(OffsetVlan, MaskDstVlan, ($3 << ShiftDstVlan) & MaskDstVlan, CMP_EQ, FUNC_NONE, NULL);
1076		break;
1077		case DIR_UNSPEC:
1078		case SOURCE_OR_DESTINATION:
1079		$$.self = Connect_OR(
1080		NewBlock(OffsetVlan, MaskSrcVlan, ($3 << ShiftSrcVlan) & MaskSrcVlan, CMP_EQ, FUNC_NONE, NULL ),
1081		NewBlock(OffsetVlan, MaskDstVlan, ($3 << ShiftDstVlan) & MaskDstVlan, CMP_EQ, FUNC_NONE, NULL)
1082		);
1083		break;
1084		case SOURCE_AND_DESTINATION:
1085		$$.self = Connect_AND(
1086		NewBlock(OffsetVlan, MaskSrcVlan, ($3 << ShiftSrcVlan) & MaskSrcVlan, CMP_EQ, FUNC_NONE, NULL ),
1087		NewBlock(OffsetVlan, MaskDstVlan, ($3 << ShiftDstVlan) & MaskDstVlan, CMP_EQ, FUNC_NONE, NULL)
1088		);
1089		break;
1090		case DIR_IN:
1091		case DIR_OUT:
1092		case IN_SRC:
1093		case IN_DST:
1094		case OUT_SRC:
1095		case OUT_DST:
1096		yyerror("This token is not expected here!");
1097		YYABORT;
1098		break;
1099		default:
1100		/* should never happen */
1101		yyerror("Internal parser error");
1102		YYABORT;
1103		} // End of switch
1104
1105		}
1106
1107		\| dqual MAC STRING {
1108		uint64_t mac = VerifyMac($3);
1109		if ( mac == 0 ) {
1110		yyerror("Invalid MAC address format");
1111		YYABORT;
1112		}
1113		switch ( $$.direction ) {
1114		case DIR_UNSPEC: {
1115		uint32_t in, out;
1116		in = Connect_OR(
1117		NewBlock(OffsetInSrcMAC, MaskMac, mac, CMP_EQ, FUNC_NONE, NULL ),
1118		NewBlock(OffsetInDstMAC, MaskMac, mac, CMP_EQ, FUNC_NONE, NULL )
1119		);
1120		out = Connect_OR(
1121		NewBlock(OffsetOutSrcMAC, MaskMac, mac, CMP_EQ, FUNC_NONE, NULL ),
1122		NewBlock(OffsetOutDstMAC, MaskMac, mac, CMP_EQ, FUNC_NONE, NULL )
1123		);
1124		$$.self = Connect_OR(in, out);
1125		} break;
1126		case DIR_IN:
1127		$$.self = Connect_OR(
1128		NewBlock(OffsetInSrcMAC, MaskMac, mac, CMP_EQ, FUNC_NONE, NULL ),
1129		NewBlock(OffsetInDstMAC, MaskMac, mac, CMP_EQ, FUNC_NONE, NULL )
1130		);
1131		break;
1132		case DIR_OUT:
1133		$$.self = Connect_OR(
1134		NewBlock(OffsetOutSrcMAC, MaskMac, mac, CMP_EQ, FUNC_NONE, NULL ),
1135		NewBlock(OffsetOutDstMAC, MaskMac, mac, CMP_EQ, FUNC_NONE, NULL )
1136		);
1137		break;
1138		case SOURCE:
1139		$$.self = Connect_OR(
1140		NewBlock(OffsetInSrcMAC, MaskMac, mac, CMP_EQ, FUNC_NONE, NULL ),
1141		NewBlock(OffsetOutSrcMAC, MaskMac, mac, CMP_EQ, FUNC_NONE, NULL )
1142		);
1143		break;
1144		case DESTINATION:
1145		$$.self = Connect_OR(
1146		NewBlock(OffsetInDstMAC, MaskMac, mac, CMP_EQ, FUNC_NONE, NULL ),
1147		NewBlock(OffsetOutDstMAC, MaskMac, mac, CMP_EQ, FUNC_NONE, NULL )
1148		);
1149		break;
1150		case IN_SRC:
1151		$$.self = NewBlock(OffsetInSrcMAC, MaskMac, mac, CMP_EQ, FUNC_NONE, NULL );
1152		break;
1153		case IN_DST:
1154		$$.self = NewBlock(OffsetInDstMAC, MaskMac, mac, CMP_EQ, FUNC_NONE, NULL );
1155		break;
1156		case OUT_SRC:
1157		$$.self = NewBlock(OffsetOutSrcMAC, MaskMac, mac, CMP_EQ, FUNC_NONE, NULL );
1158		break;
1159		case OUT_DST:
1160		$$.self = NewBlock(OffsetOutDstMAC, MaskMac, mac, CMP_EQ, FUNC_NONE, NULL );
1161		break;
1162		case SOURCE_OR_DESTINATION:
1163		case SOURCE_AND_DESTINATION:
1164		yyerror("This token is not expected here!");
1165		YYABORT;
1166		break;
1167		default:
1168		/* should never happen */
1169		yyerror("Internal parser error");
1170		YYABORT;
1171		} // End of switch
1172		}
1173
1174		\| MPLS STRING comp NUMBER {
1175		if ( $4 > MPLSMAX ) {
1176		yyerror("MPLS value out of range");
1177		YYABORT;
1178		}
1179
1180		// search for label1 - label10
1181		if ( strncasecmp($2, "label", 5) == 0 ) {
1182		uint64_t mask;
1183		uint32_t offset, shift;
1184		char *s = &$2[5];
1185		if ( s == '\0' ) {
1186		yyerror("Missing label number");
1187		YYABORT;
1188		}
1189		int i = (int)strtol(s, (char **)NULL, 10);
1190
1191		switch (i) {
1192		case 1:
1193		offset = OffsetMPLS12;
1194		mask = MaskMPLSlabelOdd;
1195		shift = ShiftMPLSlabelOdd;
1196		break;
1197		case 2:
1198		offset = OffsetMPLS12;
1199		mask = MaskMPLSlabelEven;
1200		shift = ShiftMPLSlabelEven;
1201		break;
1202		case 3:
1203		offset = OffsetMPLS34;
1204		mask = MaskMPLSlabelOdd;
1205		shift = ShiftMPLSlabelOdd;
1206		break;
1207		case 4:
1208		offset = OffsetMPLS34;
1209		mask = MaskMPLSlabelEven;
1210		shift = ShiftMPLSlabelEven;
1211		break;
1212		case 5:
1213		offset = OffsetMPLS56;
1214		mask = MaskMPLSlabelOdd;
1215		shift = ShiftMPLSlabelOdd;
1216		break;
1217		case 6:
1218		offset = OffsetMPLS56;
1219		mask = MaskMPLSlabelEven;
1220		shift = ShiftMPLSlabelEven;
1221		break;
1222		case 7:
1223		offset = OffsetMPLS78;
1224		mask = MaskMPLSlabelOdd;
1225		shift = ShiftMPLSlabelOdd;
1226		break;
1227		case 8:
1228		offset = OffsetMPLS78;
1229		mask = MaskMPLSlabelEven;
1230		shift = ShiftMPLSlabelEven;
1231		break;
1232		case 9:
1233		offset = OffsetMPLS910;
1234		mask = MaskMPLSlabelOdd;
1235		shift = ShiftMPLSlabelOdd;
1236		break;
1237		case 10:
1238		offset = OffsetMPLS910;
1239		mask = MaskMPLSlabelEven;
1240		shift = ShiftMPLSlabelEven;
1241		break;
1242		default:
1243		yyerror("MPLS label out of range 1..10");
1244		YYABORT;
1245		}
1246		$$.self = NewBlock(offset, mask, ($4 << shift) & mask, $3.comp, FUNC_NONE, NULL );
1247
1248		} else if ( strcasecmp($2, "eos") == 0 ) {
1249		// match End of Stack label
1250		$$.self = NewBlock(0, AnyMask, $4 << 4, $3.comp, FUNC_MPLS_EOS, NULL );
1251
1252		} else if ( strncasecmp($2, "exp", 3) == 0 ) {
1253		uint64_t mask;
1254		uint32_t offset, shift;
1255		char *s = &$2[3];
1256		if ( s == '\0' ) {
1257		yyerror("Missing label number");
1258		YYABORT;
1259		}
1260		int i = (int)strtol(s, (char **)NULL, 10);
1261
1262		if ( $4 < 0 \|\| $4 > 7 ) {
1263		yyerror("MPLS exp value out of range");
1264		YYABORT;
1265		}
1266
1267		switch (i) {
1268		case 1:
1269		offset = OffsetMPLS12;
1270		mask = MaskMPLSexpOdd;
1271		shift = ShiftMPLSexpOdd;
1272		break;
1273		case 2:
1274		offset = OffsetMPLS12;
1275		mask = MaskMPLSexpEven;
1276		shift = ShiftMPLSexpEven;
1277		break;
1278		case 3:
1279		offset = OffsetMPLS34;
1280		mask = MaskMPLSexpOdd;
1281		shift = ShiftMPLSexpOdd;
1282		break;
1283		case 4:
1284		offset = OffsetMPLS34;
1285		mask = MaskMPLSexpEven;
1286		shift = ShiftMPLSexpEven;
1287		break;
1288		case 5:
1289		offset = OffsetMPLS56;
1290		mask = MaskMPLSexpOdd;
1291		shift = ShiftMPLSexpOdd;
1292		break;
1293		case 6:
1294		offset = OffsetMPLS56;
1295		mask = MaskMPLSexpEven;
1296		shift = ShiftMPLSexpEven;
1297		break;
1298		case 7:
1299		offset = OffsetMPLS78;
1300		mask = MaskMPLSexpOdd;
1301		shift = ShiftMPLSexpOdd;
1302		break;
1303		case 8:
1304		offset = OffsetMPLS78;
1305		mask = MaskMPLSexpEven;
1306		shift = ShiftMPLSexpEven;
1307		break;
1308		case 9:
1309		offset = OffsetMPLS910;
1310		mask = MaskMPLSexpOdd;
1311		shift = ShiftMPLSexpOdd;
1312		break;
1313		case 10:
1314		offset = OffsetMPLS910;
1315		mask = MaskMPLSexpEven;
1316		shift = ShiftMPLSexpEven;
1317		break;
1318		default:
1319		yyerror("MPLS label out of range 1..10");
1320		YYABORT;
1321		}
1322		$$.self = NewBlock(offset, mask, $4 << shift, $3.comp, FUNC_NONE, NULL );
1323
1324		} else {
1325		yyerror("Unknown MPLS option");
1326		YYABORT;
1327		}
1328		}
1329		\| MPLS ANY NUMBER {
1330		uint32_t *opt = malloc(sizeof(uint32_t));
1331		if ( $3 > MPLSMAX ) {
1332		yyerror("MPLS value out of range");
1333		YYABORT;
1334		}
1335		if ( opt == NULL) {
1336		yyerror("malloc() error");
1337		YYABORT;
1338		}
1339		*opt = $3 << 4;
1340		$$.self = NewBlock(0, AnyMask, $3 << 4, CMP_EQ, FUNC_MPLS_ANY, opt );
1341
1342		}
1343		\| FWDSTAT NUMBER {
1344		if ( $2 > 255 ) {
1345		yyerror("Forwarding status of range 0..255");
1346		YYABORT;
1347		}
1348		$$.self = NewBlock(OffsetStatus, MaskStatus, ($2 << ShiftStatus) & MaskStatus, CMP_EQ, FUNC_NONE, NULL);
1349		}
1350
1351		\| FWDSTAT STRING {
1352		uint64_t id = Get_fwd_status_id($2);
1353		if (id == 256 ) {
1354		yyerror("Unknown forwarding status");
1355		YYABORT;
1356		}
1357
1358		$$.self = NewBlock(OffsetStatus, MaskStatus, (id << ShiftStatus) & MaskStatus, CMP_EQ, FUNC_NONE, NULL);
1359
1360		}
1361
1362		\| DIR NUMBER {
1363		if ( $2 > 2 ) {
1364		yyerror("Flow direction status of range 0, 1");
1365		YYABORT;
1366		}
1367		$$.self = NewBlock(OffsetDir, MaskDir, ($2 << ShiftDir) & MaskDir, CMP_EQ, FUNC_NONE, NULL);
1368
1369		}
1370
1371		\| DIR STRING {
1372		uint64_t dir = 0xFF;
1373		if ( strcasecmp($2, "ingress") == 0 )
1374		dir = 0;
1375		else if ( strcasecmp($2, "egress") == 0 )
1376		dir = 1;
1377		else {
1378		yyerror("Flow direction status of range ingress, egress");
1379		YYABORT;
1380		}
1381
1382		$$.self = NewBlock(OffsetDir, MaskDir, (dir << ShiftDir) & MaskDir, CMP_EQ, FUNC_NONE, NULL);
1383
1384		}
1385
1386		/* iplist definition */
1387		iplist: STRING {
1388		int i, af, bytes, ret;
1389		struct IPListNode *node;
1390
1391		IPlist_t *root = malloc(sizeof(IPlist_t));
1392
1393		if ( root == NULL) {
1394		yyerror("malloc() error");
1395		YYABORT;
1396		}
1397		RB_INIT(root);
1398
1399		ret = parse_ip(&af, $1, IPstack, &bytes, ALLOW_LOOKUP, &num_ip);
1400
1401		if ( ret == 0 ) {
1402		yyerror("Invalid IP address");
1403		YYABORT;
1404		}
1405		// ret == -1 will never happen here, as ALLOW_LOOKUP is set
1406
1407		if ( ret != -2 ) {
1408		if ( af && (( af == PF_INET && bytes != 4 ) \|\| ( af == PF_INET6 && bytes != 16 ))) {
1409		yyerror("incomplete IP address");
1410		YYABORT;
1411		}
1412
1413		for ( i=0; i<num_ip; i++ ) {
1414		if ((node = malloc(sizeof(struct IPListNode))) == NULL) {
1415		yyerror("malloc() error");
1416		YYABORT;
1417		}
1418		node->ip[0] = IPstack[2*i];
1419		node->ip[1] = IPstack[2*i+1];
1420		node->mask[0] = 0xffffffffffffffffLL;
1421		node->mask[1] = 0xffffffffffffffffLL;
1422		RB_INSERT(IPtree, root, node);
1423		}
1424
1425		}
1426		$$ = (void *)root;
1427
1428		}
1429
1430		iplist: STRING '/' NUMBER {
1431		int af, bytes, ret;
1432		struct IPListNode *node;
1433
1434		IPlist_t *root = malloc(sizeof(IPlist_t));
1435
1436		if ( root == NULL) {
1437		yyerror("malloc() error");
1438		YYABORT;
1439		}
1440		RB_INIT(root);
1441
1442		ret = parse_ip(&af, $1, IPstack, &bytes, STRICT_IP, &num_ip);
1443
1444		if ( ret == 0 ) {
1445		yyerror("Invalid IP address");
1446		YYABORT;
1447		}
1448		// ret == -1 will never happen here, as ALLOW_LOOKUP is set
1449
1450		if ( ret != -2 ) {
1451		if ( af && (( af == PF_INET && bytes != 4 ) \|\| ( af == PF_INET6 && bytes != 16 ))) {
1452		yyerror("incomplete IP address");
1453		YYABORT;
1454		}
1455
1456		if ((node = malloc(sizeof(struct IPListNode))) == NULL) {
1457		yyerror("malloc() error");
1458		YYABORT;
1459		}
1460
1461		if ( af == PF_INET ) {
1462		node->mask[0] = 0xffffffffffffffffLL;
1463		node->mask[1] = 0xffffffffffffffffLL << ( 32 - $3 );
1464		} else { // PF_INET6
1465		if ( $3 > 64 ) {
1466		node->mask[0] = 0xffffffffffffffffLL;
1467		node->mask[1] = 0xffffffffffffffffLL << ( 128 - $3 );
1468		} else {
1469		node->mask[0] = 0xffffffffffffffffLL << ( 64 - $3 );
1470		node->mask[1] = 0;
1471		}
1472		}
1473
1474		node->ip[0] = IPstack[0] & node->mask[0];
1475		node->ip[1] = IPstack[1] & node->mask[1];
1476
1477		RB_INSERT(IPtree, root, node);
1478
1479		}
1480		$$ = (void *)root;
1481
1482		}
1483
1484		\| iplist STRING {
1485		int i, af, bytes, ret;
1486		struct IPListNode *node;
1487
1488		ret = parse_ip(&af, $2, IPstack, &bytes, ALLOW_LOOKUP, &num_ip);
1489
1490		if ( ret == 0 ) {
1491		yyerror("Invalid IP address");
1492		YYABORT;
1493		}
1494		if ( af && (( af == PF_INET && bytes != 4 ) \|\| ( af == PF_INET6 && bytes != 16 ))) {
1495		yyerror("incomplete IP address");
1496		YYABORT;
1497		}
1498
1499		// ret == - 2 means lookup failure
1500		if ( ret != -2 ) {
1501		for ( i=0; i<num_ip; i++ ) {
1502		if ((node = malloc(sizeof(struct IPListNode))) == NULL) {
1503		yyerror("malloc() error");
1504		YYABORT;
1505		}
1506		node->ip[0] = IPstack[2*i];
1507		node->ip[1] = IPstack[2*i+1];
1508		node->mask[0] = 0xffffffffffffffffLL;
1509		node->mask[1] = 0xffffffffffffffffLL;
1510
1511		RB_INSERT(IPtree, (IPlist_t *)$$, node);
1512		}
1513		}
1514		}
1515
1516		\| iplist STRING '/' NUMBER {
1517		int af, bytes, ret;
1518		struct IPListNode *node;
1519
1520		ret = parse_ip(&af, $2, IPstack, &bytes, STRICT_IP, &num_ip);
1521
1522		if ( ret == 0 ) {
1523		yyerror("Invalid IP address");
1524		YYABORT;
1525		}
1526		if ( af && (( af == PF_INET && bytes != 4 ) \|\| ( af == PF_INET6 && bytes != 16 ))) {
1527		yyerror("incomplete IP address");
1528		YYABORT;
1529		}
1530
1531		// ret == - 2 means lookup failure
1532		if ( ret != -2 ) {
1533		if ((node = malloc(sizeof(struct IPListNode))) == NULL) {
1534		yyerror("malloc() error");
1535		YYABORT;
1536		}
1537		if ( af == PF_INET ) {
1538		node->mask[0] = 0xffffffffffffffffLL;
1539		node->mask[1] = 0xffffffffffffffffLL << ( 32 - $4 );
1540		} else { // PF_INET6
1541		if ( $4 > 64 ) {
1542		node->mask[0] = 0xffffffffffffffffLL;
1543		node->mask[1] = 0xffffffffffffffffLL << ( 128 - $4 );
1544		} else {
1545		node->mask[0] = 0xffffffffffffffffLL << ( 64 - $4 );
1546		node->mask[1] = 0;
1547		}
1548		}
1549
1550		node->ip[0] = IPstack[0] & node->mask[0];
1551		node->ip[1] = IPstack[1] & node->mask[1];
1552
1553		RB_INSERT(IPtree, (IPlist_t *)$$, node);
1554		}
1555		}
1556
1557		;
1558
1559		/* ULlist definition */
1560		ullist: NUMBER {
1561		struct ULongListNode *node;
1562
1563		if ( $1 > 65535 ) {
1564		yyerror("Value outside of range 0..65535");
1565		YYABORT;
1566		}
1567		ULongtree_t *root = malloc(sizeof(ULongtree_t));
1568
1569		if ( root == NULL) {
1570		yyerror("malloc() error");
1571		YYABORT;
1572		}
1573		RB_INIT(root);
1574
1575		if ((node = malloc(sizeof(struct ULongListNode))) == NULL) {
1576		yyerror("malloc() error");
1577		YYABORT;
1578		}
1579		node->value = $1;
1580
1581		RB_INSERT(ULongtree, root, node);
1582		$$ = (void *)root;
1583		}
1584		\| ullist NUMBER {
1585		struct ULongListNode *node;
1586
1587		if ( $2 > 65535 ) {
1588		yyerror("Value outside of range 0..65535");
1589		YYABORT;
1590		}
1591		if ((node = malloc(sizeof(struct ULongListNode))) == NULL) {
1592		yyerror("malloc() error");
1593		YYABORT;
1594		}
1595		node->value = $2;
1596		RB_INSERT(ULongtree, (ULongtree_t *)$$, node);
1597		}
1598		;
1599
1600		/* scaling qualifiers */
1601
1602		/* comparator qualifiers */
1603		comp: { $$.comp = CMP_EQ; }
1604		\| EQ { $$.comp = CMP_EQ; }
1605		\| LT { $$.comp = CMP_LT; }
1606		\| GT { $$.comp = CMP_GT; }
1607		;
1608
1609		/* 'direction' qualifiers */
1610		dqual: { $$.direction = DIR_UNSPEC; }
1611		\| SRC { $$.direction = SOURCE; }
1612		\| DST { $$.direction = DESTINATION; }
1613		\| SRC OR DST { $$.direction = SOURCE_OR_DESTINATION; }
1614		\| DST OR SRC { $$.direction = SOURCE_OR_DESTINATION; }
1615		\| SRC AND DST { $$.direction = SOURCE_AND_DESTINATION; }
1616		\| DST AND SRC { $$.direction = SOURCE_AND_DESTINATION; }
1617		\| IN { $$.direction = DIR_IN; }
1618		\| OUT { $$.direction = DIR_OUT; }
1619		\| IN SRC { $$.direction = IN_SRC; }
1620		\| IN DST { $$.direction = IN_DST; }
1621		\| OUT SRC { $$.direction = OUT_SRC; }
1622		\| OUT DST { $$.direction = OUT_DST; }
1623		;
1624
1625		expr: term { $$ = $1.self; }
1626		\| expr OR expr { $$ = Connect_OR($1, $3); }
1627		\| expr AND expr { $$ = Connect_AND($1, $3); }
1628		\| NOT expr %prec NEGATE { $$ = Invert($2); }
1629		\| '(' expr ')' { $$ = $2; }
1630		;
1631
1632		%%
1633
1634		static void yyerror(char *msg) {
1635		if ( FilterFilename )
1636		fprintf(stderr,"File '%s' line %d: %s at '%s'\n", FilterFilename, lineno, msg, yytext);
1637		else
1638		fprintf(stderr,"Line %d: %s at '%s'\n", lineno, msg, yytext);
1639		} /* End of yyerror */
1640
1641		static uint32_t ChainHosts(uint64_t *hostlist, int num_records, int type) {
1642		uint32_t offset_a, offset_b, i, j, block;
1643
1644		if ( type == SOURCE ) {
1645		offset_a = OffsetSrcIPv6a;
1646		offset_b = OffsetSrcIPv6b;
1647		} else {
1648		offset_a = OffsetDstIPv6a;
1649		offset_b = OffsetDstIPv6b;
1650		}
1651
1652		i = 0;
1653		block = Connect_AND(
1654		NewBlock(offset_b, MaskIPv6, hostlist[i+1] , CMP_EQ, FUNC_NONE, NULL ),
1655		NewBlock(offset_a, MaskIPv6, hostlist[i] , CMP_EQ, FUNC_NONE, NULL )
1656		);
1657		i += 2;
1658		for ( j=1; j<num_records; j++ ) {
1659		uint32_t b = Connect_AND(
1660		NewBlock(offset_b, MaskIPv6, hostlist[i+1] , CMP_EQ, FUNC_NONE, NULL ),
1661		NewBlock(offset_a, MaskIPv6, hostlist[i] , CMP_EQ, FUNC_NONE, NULL )
1662		);
1663		block = Connect_OR(block, b);
1664		i += 2;
1665		}
1666
1667		return block;
1668
1669		} // End of ChainHosts
1670
1671		uint64_t VerifyMac(char *s) {
1672		uint64_t mac;
1673		size_t slen = strlen(s);
1674		long l;
1675		char p, q, *r;
1676		int i;
1677
1678		if ( slen > 17 )
1679		return 0;
1680
1681		for (i=0; i<slen; i++ ) {
1682		if ( !isxdigit(s[i]) && s[i] != ':' )
1683		return 0;
1684		}
1685
1686		p = strdup(s);
1687		if ( !p ) {
1688		yyerror("malloc() error");
1689		return 0;
1690		}
1691
1692		mac = 0;
1693		i = 0; // number of MAC octets must be 6
1694		r = p;
1695		q = strchr(r, ':');
1696		while ( r && i < 6 ) {
1697		if ( q )
1698		*q = '\0';
1699		l = strtol(r, NULL, 16);
1700		if ( l > 255 ) {
1701		free(p);
1702		return 0;
1703		}
1704
1705		mac = ( mac << 8 ) \| (l & 0xFF );
1706		i++;
1707
1708		if ( q ) {
1709		r = ++q;
1710		q = strchr(r, ':');
1711		} else
1712		r = NULL;
1713		}
1714
1715		if ( i != 6 )
1716		return 0;
1717
1718		return mac;
1719
1720		} // End of VerifyMac
1721
1722		/*
1723
1724		mpls 1 == 3
1725		mpls label1 == 3
1726		mpls any == 4
1727
1728
1729
1730
1731		*/

+36

-0

bin/inline.c less more

49	49
50	50	static inline uint64_t Get_val64(void *p);
51	51
	52	static inline uint64_t Get_val(void *p, uint32_t index, uint32_t length);
	53
52	54	static inline void Put_val16(uint16_t v, void *p);
53	55
54	56	static inline void Put_val24(uint32_t v, void *p);

228	230	return mask.val.val64;
229	231
230	232	} // End of Get_val64
	233
	234	static inline uint64_t Get_val(void *p, uint32_t index, uint32_t length) {
	235
	236	switch (length) {
	237	case 1:
	238	return ((uint8_t )(p + index));
	239	break;
	240	case 2:
	241	return Get_val16(p + index);
	242	break;
	243	case 3:
	244	return Get_val24(p + index);
	245	break;
	246	case 4:
	247	return Get_val32(p + index);
	248	break;
	249	case 5:
	250	return Get_val40(p + index);
	251	break;
	252	case 6:
	253	return Get_val48(p + index);
	254	break;
	255	case 7:
	256	return Get_val56(p + index);
	257	break;
	258	case 8:
	259	return Get_val64(p + index);
	260	break;
	261	default:
	262	return 0;
	263	}
	264	return 0;
	265
	266	} // End of Get_val
231	267
232	268	static inline void Put_val16(uint16_t v, void *p) {
233	269	uint8_t out = (uint8_t )p;

-10

bin/ipconv.c less more

0	0	/*
1		* Copyright (c) 2014, Peter Haag
2		* Copyright (c) 2009, Peter Haag
	1	* Copyright (c) 2009-2020, Peter Haag
3	2	* Copyright (c) 2008-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
4	3	* All rights reserved.
5	4	*

42	41	* ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS
43	42	* SOFTWARE.
44	43	*
45		*
46		* $Author: haag $
47		*
48		* $Id: ipconv.c 39 2009-11-25 08:11:15Z haag $
49		*
50		* $LastChangedRevision: 39 $
51		*
52	44	*/
53	45
54	46	#include "config.h"

72	64	#include <stdint.h>
73	65	#endif
74	66
	67	#include "util.h"
	68	#include "nfdump.h"
75	69	#include "nffile.h"
76		#include "util.h"
77	70	#include "ipconv.h"
78	71
79	72

+226

-198

bin/ipfix.c less more

0	0	/*
1		* Copyright (c) 2012-2019, Peter Haag
	1	* Copyright (c) 2012-2020, Peter Haag
2	2	* All rights reserved.
3	3	*
4	4	* Redistribution and use in source and binary forms, with or without

44	44	#include <stdint.h>
45	45	#endif
46	46
	47	#include "util.h"
	48	#include "nfdump.h"
47	49	#include "nffile.h"
48	50	#include "nfx.h"
49	51	#include "nfnet.h"
50		#include "nf_common.h"
51		#include "util.h"
	52	#include "output_raw.h"
52	53	#include "bookkeeper.h"
53	54	#include "collector.h"
54	55	#include "exporter.h"
55	56	#include "ipfix.h"
56	57
57		#ifndef DEVEL
58		# define dbg_printf(...) /* printf(__VA_ARGS__) */
59		#else
60		# define dbg_printf(...) printf(__VA_ARGS__)
61		#endif
62
63		// a few handy macros
64	58	#define GET_FLOWSET_ID(p) (Get_val16(p))
65	59	#define GET_FLOWSET_LENGTH(p) (Get_val16((void *)((p) + 2)))
66	60
67	61	#define GET_TEMPLATE_ID(p) (Get_val16(p))
68	62	#define GET_TEMPLATE_COUNT(p) (Get_val16((void *)((p) + 2)))
69	63
70		#define GET_OPTION_TEMPLATE_ID(p) (Get_val16(p))
71		#define GET_OPTION_TEMPLATE_FIELD_COUNT(p) (Get_val16((void *)((p) + 2)))
72		#define GET_OPTION_TEMPLATE_SCOPE_FIELD_COUNT(p) (Get_val16((void *)((p) + 4)))
	64	#define GET_OPTION_TEMPLATE_ID(p) (Get_val16(p))
	65	#define GET_OPTION_TEMPLATE_FIELD_COUNT(p) (Get_val16((void *)((p) + 2)))
	66	#define GET_OPTION_TEMPLATE_SCOPE_FIELD_COUNT(p) (Get_val16((void *)((p) + 4)))
73	67
74	68	#define DYN_FIELD_LENGTH 65535
75	69

158	152	* All Obervation Domains from all exporter are stored in a linked list
159	153	* which uniquely can identify each exporter/Observation Domain
160	154	*/
161		typedef struct exporter_ipfix_domain_s {
162		struct exporter_ipfix_domain_s *next; // linkes list to next exporter
163
164		// generic exporter information
	155	typedef struct exporterDomain_s {
	156	struct exporterDomain_s *next; // linkes list to next exporter
	157
	158	// exporter information
165	159	exporter_info_record_t info;
166	160
167	161	uint64_t packets; // number of packets sent by this exporter

169	163	uint32_t sequence_failure; // number of sequence failues
170	164	uint32_t padding_errors; // number of sequence failues
171	165
172		// generic sampler
173		generic_sampler_t *sampler;
	166	// sampler
	167	sampler_t *sampler; // sampler info
	168	samplerOption_t *samplerOption; // sampler options table info
174	169
175	170	// exporter parameters
176	171	uint32_t ExportTime;

189	184	// the last template we processed as a cache
190	185	input_translation_t *current_table;
191	186
192		} exporter_ipfix_domain_t;
	187	} exporterDomain_t;
193	188
194	189
195	190	static struct ipfix_element_map_s {

289	284	} cache;
290	285
291	286	// module limited globals
	287	static int verbose;
	288	static uint32_t default_sampling;
	289	static uint32_t overwrite_sampling;
292	290	static uint32_t processed_records;
293	291
294	292	// externals
295		extern int verbose;
296	293	extern uint32_t Max_num_extensions;
297	294	extern extension_descriptor_t extension_descriptor[];
298		extern uint32_t default_sampling;
299		extern uint32_t overwrite_sampling;
300	295
301	296	// prototypes
302		static void InsertStdSamplerOffset(FlowSource_t *fs, uint16_t id, uint16_t offset_std_sampler_interval,
303		uint16_t offset_std_sampler_algorithm);
304
305		static void InsertSampler(FlowSource_t fs, exporter_ipfix_domain_t exporter, int32_t id, uint16_t mode, uint32_t interval);
306
307		static input_translation_t add_translation_table(exporter_ipfix_domain_t exporter, uint16_t id);
308
309		static void remove_translation_table(FlowSource_t fs, exporter_ipfix_domain_t exporter, uint16_t id);
310
311		static void remove_all_translation_tables(exporter_ipfix_domain_t *exporter);
312
313		static exporter_ipfix_domain_t GetExporter(FlowSource_t fs, ipfix_header_t *ipfix_header);
	297	static int HasOptionTable(exporterDomain_t *exporter, uint16_t tableID );
	298
	299	static void InsertSamplerOption(exporterDomain_t exporter, samplerOption_t samplerOption);
	300
	301	static void InsertSampler(FlowSource_t fs, exporterDomain_t exporter, int32_t id, uint16_t mode, uint32_t interval);
	302
	303	static input_translation_t add_translation_table(exporterDomain_t exporter, uint16_t id);
	304
	305	static void remove_translation_table(FlowSource_t fs, exporterDomain_t exporter, uint16_t id);
	306
	307	static void remove_all_translation_tables(exporterDomain_t *exporter);
	308
	309	static exporterDomain_t GetExporter(FlowSource_t fs, ipfix_header_t *ipfix_header);
314	310
315	311	static uint32_t MapElement(uint16_t Type, uint16_t Length, uint32_t order);
316	312

320	316
321	317	static int reorder_sequencer(input_translation_t *table);
322	318
323		static void Process_ipfix_templates(exporter_ipfix_domain_t exporter, void flowset_header, uint32_t size_left, FlowSource_t *fs);
324
325		static void Process_ipfix_template_add(exporter_ipfix_domain_t exporter, void DataPtr, uint32_t size_left, FlowSource_t *fs);
326
327		static void Process_ipfix_template_withdraw(exporter_ipfix_domain_t exporter, void DataPtr, uint32_t size_left, FlowSource_t *fs);
328
329		static void Process_ipfix_option_data(exporter_ipfix_domain_t exporter, void data_flowset, FlowSource_t *fs);
330
331		static void Process_ipfix_data(exporter_ipfix_domain_t exporter, uint32_t ExportTime, void data_flowset, FlowSource_t fs, input_translation_t table );
	319	static void Process_ipfix_templates(exporterDomain_t exporter, void flowset_header, uint32_t size_left, FlowSource_t *fs);
	320
	321	static void Process_ipfix_template_add(exporterDomain_t exporter, void DataPtr, uint32_t size_left, FlowSource_t *fs);
	322
	323	static void Process_ipfix_template_withdraw(exporterDomain_t exporter, void DataPtr, uint32_t size_left, FlowSource_t *fs);
	324
	325	static void Process_ipfix_option_data(exporterDomain_t exporter, void data_flowset, FlowSource_t *fs);
	326
	327	static void Process_ipfix_data(exporterDomain_t exporter, uint32_t ExportTime, void data_flowset, FlowSource_t fs, input_translation_t table );
332	328
333	329	#include "inline.c"
334	330	#include "nffile_inline.c"
335	331
336		int Init_IPFIX(void) {
	332	int Init_IPFIX(int v, uint32_t sampling, uint32_t overwrite) {
337	333	int i;
	334
	335	verbose = v;
	336	default_sampling = sampling;
	337	overwrite_sampling = overwrite;
338	338
339	339	cache.lookup_info = (struct element_param_s *)calloc(65536, sizeof(struct element_param_s));
340	340	cache.common_extensions = (uint32_t )malloc((Max_num_extensions+1)sizeof(uint32_t));

360	360
361	361	} // End of Init_IPFIX
362	362
363		static exporter_ipfix_domain_t GetExporter(FlowSource_t fs, ipfix_header_t *ipfix_header) {
	363	static int HasOptionTable(exporterDomain_t *exporter, uint16_t tableID ) {
	364	samplerOption_t *s;
	365
	366	s = exporter->samplerOption;
	367	while ( s && s->tableID != tableID )
	368	s = s->next;
	369
	370	dbg_printf("Has option table: %s\n", s == NULL ? "not found" : "found");
	371
	372	return s != NULL;
	373
	374	} // End of HasOptionTable
	375
	376	static exporterDomain_t GetExporter(FlowSource_t fs, ipfix_header_t *ipfix_header) {
364	377	#define IP_STRING_LEN 40
365	378	char ipstr[IP_STRING_LEN];
366		exporter_ipfix_domain_t e = (exporter_ipfix_domain_t )&(fs->exporter_data);
	379	exporterDomain_t e = (exporterDomain_t )&(fs->exporter_data);
367	380	uint32_t ObservationDomain = ntohl(ipfix_header->ObservationDomain);
368	381
369	382	while ( *e ) {

386	399	}
387	400
388	401	// nothing found
389		e = (exporter_ipfix_domain_t )malloc(sizeof(exporter_ipfix_domain_t));
	402	e = (exporterDomain_t )malloc(sizeof(exporterDomain_t));
390	403	if ( !(*e)) {
391	404	LogError("Process_ipfix: Panic! malloc() %s line %d: %s", __FILE__, __LINE__, strerror (errno));
392	405	return NULL;
393	406	}
394		memset((void )(e), 0, sizeof(exporter_ipfix_domain_t));
	407	memset((void )(e), 0, sizeof(exporterDomain_t));
395	408	(*e)->info.header.type = ExporterInfoRecordType;
396	409	(*e)->info.header.size = sizeof(exporter_info_record_t);
397	410	(*e)->info.id = ObservationDomain;

449	462
450	463	} // End of MapElement
451	464
452		static input_translation_t GetTranslationTable(exporter_ipfix_domain_t exporter, uint16_t id) {
	465	static input_translation_t GetTranslationTable(exporterDomain_t exporter, uint16_t id) {
453	466	input_translation_t *table;
454	467
455	468	if ( exporter->current_table && ( exporter->current_table->id == id ) )

472	485
473	486	} // End of GetTranslationTable
474	487
475		static input_translation_t add_translation_table(exporter_ipfix_domain_t exporter, uint16_t id) {
	488	static input_translation_t add_translation_table(exporterDomain_t exporter, uint16_t id) {
476	489	input_translation_t **table;
477	490
478	491	table = &(exporter->input_translation_table);

499	512
500	513	} // End of add_translation_table
501	514
502		static void remove_translation_table(FlowSource_t fs, exporter_ipfix_domain_t exporter, uint16_t id) {
	515	static void remove_translation_table(FlowSource_t fs, exporterDomain_t exporter, uint16_t id) {
503	516	input_translation_t table, parent;
504	517
505	518	LogInfo("Process_ipfix: [%u] Withdraw template id: %i",

539	552
540	553	} // End of remove_translation_table
541	554
542		static void remove_all_translation_tables(exporter_ipfix_domain_t *exporter) {
	555	static void remove_all_translation_tables(exporterDomain_t *exporter) {
543	556	input_translation_t table, next;
544	557
545	558	LogInfo("Process_ipfix: Withdraw all templates from observation domain %u\n",

764	777
765	778	} // End of reorder_sequencer
766	779
767		static input_translation_t setup_translation_table (exporter_ipfix_domain_t exporter, uint16_t id) {
	780	static input_translation_t setup_translation_table (exporterDomain_t exporter, uint16_t id) {
768	781	input_translation_t *table;
769	782	extension_map_t *extension_map;
770	783	uint32_t i, ipv6, offset, next_extension;

1092	1105
1093	1106	} // End of setup_translation_table
1094	1107
1095		static void InsertStdSamplerOffset( FlowSource_t *fs, uint16_t id, uint16_t offset_std_sampler_interval, uint16_t offset_std_sampler_algorithm) {
1096		option_offset_t **t;
1097
1098		t = &(fs->option_offset_table);
1099		while ( *t ) {
1100		if ( (*t)->id == id ) { // table already known to us - update data
1101		dbg_printf("Found existing std sampling info in template %i\n", id);
1102		break;
1103		}
1104
1105		t = &((*t)->next);
1106		}
1107
1108		if ( *t == NULL ) { // new table
1109		dbg_printf("Allocate new std sampling info from template %i\n", id);
1110		t = (option_offset_t )calloc(1, sizeof(option_offset_t));
1111		if ( !*t ) {
1112		LogError("malloc() allocation error at %s line %u: %s" , __FILE__, __LINE__, strerror (errno));
1113		return ;
1114		}
1115		LogInfo("Process_v9: New std sampler: interval: %i, algorithm: %i",
1116		offset_std_sampler_interval, offset_std_sampler_algorithm);
1117		} // else existing table
1118
1119		dbg_printf("Insert/Update sampling info from template %i\n", id);
1120		SetFlag((*t)->flags, HAS_STD_SAMPLER_DATA);
1121		(*t)->id = id;
1122		(*t)->offset_id = 0;
1123		(*t)->offset_mode = 0;
1124		(*t)->offset_interval = 0;
1125		(*t)->offset_std_sampler_interval = offset_std_sampler_interval;
1126		(*t)->offset_std_sampler_algorithm = offset_std_sampler_algorithm;
1127
1128		} // End of InsertStdSamplerOffset
1129
1130		static void InsertSampler(FlowSource_t fs, exporter_ipfix_domain_t exporter, int32_t id, uint16_t mode, uint32_t interval) {
1131		generic_sampler_t *sampler;
	1108	static void InsertSampler(FlowSource_t fs, exporterDomain_t exporter, int32_t id, uint16_t mode, uint32_t interval) {
	1109	sampler_t *sampler;
1132	1110
1133	1111	dbg_printf("[%u] Insert Sampler: Exporter is 0x%llu\n", exporter->info.id, (long long unsigned)exporter);
1134	1112	if ( !exporter->sampler ) {
1135	1113	// no samplers so far
1136		sampler = (generic_sampler_t *)malloc(sizeof(generic_sampler_t));
	1114	sampler = (sampler_t *)malloc(sizeof(sampler_t));
1137	1115	if ( !sampler ) {
1138	1116	LogError( "Process_v9: Panic! malloc(): %s line %d: %s", __FILE__, __LINE__, strerror (errno));
1139	1117	return;

1180	1158	// test for end of chain
1181	1159	if ( sampler->next == NULL ) {
1182	1160	// end of sampler chain - insert new sampler
1183		sampler->next = (generic_sampler_t *)malloc(sizeof(generic_sampler_t));
	1161	sampler->next = (sampler_t *)malloc(sizeof(sampler_t));
1184	1162	if ( !sampler->next ) {
1185	1163	LogError( "Process_v9: Panic! malloc(): %s line %d: %s", __FILE__, __LINE__, strerror (errno));
1186	1164	return;

1211	1189
1212	1190	} // End of InsertSampler
1213	1191
1214		static void Process_ipfix_templates(exporter_ipfix_domain_t exporter, void flowset_header, uint32_t size_left, FlowSource_t *fs) {
	1192	static void InsertSamplerOption(exporterDomain_t exporter, samplerOption_t samplerOption) {
	1193	samplerOption_t s, parent;
	1194
	1195	parent = NULL;
	1196	s = exporter->samplerOption;
	1197	while (s) {
	1198	if ( s->tableID == samplerOption->tableID ) { // table already known to us - update data
	1199	dbg_printf("Found existing sampling info in template %i\n", samplerOption->tableID);
	1200	break;
	1201	}
	1202	parent = s;
	1203	s = s->next;
	1204	}
	1205
	1206	if ( s != NULL ) { // existing entry
	1207	// replace existing table
	1208	dbg_printf("Replace existing sampler table ID %i\n", samplerOption->tableID);
	1209	if ( parent ) {
	1210	parent->next = samplerOption;
	1211	} else {
	1212	exporter->samplerOption = samplerOption;
	1213	}
	1214	samplerOption->next = s->next;
	1215	free(s);
	1216	s = NULL;
	1217	} else { // new entry
	1218	dbg_printf("New sampling table ID %i\n", samplerOption->tableID);
	1219	// push new sampling table
	1220	samplerOption->next = exporter->samplerOption;
	1221	exporter->samplerOption = samplerOption;
	1222	}
	1223
	1224	dbg_printf("Update/Insert sampler table id: %u flags: 0x%x - sampler ID: %u/%u, mode: %u/%u, interval: %u/%u\n",
	1225	samplerOption->tableID, samplerOption->flags,
	1226	samplerOption->id.offset, samplerOption->id.length,
	1227	samplerOption->mode.offset, samplerOption->mode.length,
	1228	samplerOption->interval.offset, samplerOption->interval.length);
	1229
	1230	} // End of InsertSamplerOption
	1231
	1232	static void Process_ipfix_templates(exporterDomain_t exporter, void flowset_header, uint32_t size_left, FlowSource_t *fs) {
1215	1233	ipfix_template_record_t *ipfix_template_record;
1216	1234	void *DataPtr;
1217	1235	uint32_t count;

1234	1252
1235	1253	} // End of Process_ipfix_templates
1236	1254
1237		static void Process_ipfix_template_add(exporter_ipfix_domain_t exporter, void DataPtr, uint32_t size_left, FlowSource_t *fs) {
	1255	static void Process_ipfix_template_add(exporterDomain_t exporter, void DataPtr, uint32_t size_left, FlowSource_t *fs) {
1238	1256	input_translation_t *translation_table;
1239	1257	ipfix_template_record_t *ipfix_template_record;
1240	1258	ipfix_template_elements_std_t *NextElement;

1417	1435
1418	1436	} // End of Process_ipfix_template_add
1419	1437
1420		static void Process_ipfix_template_withdraw(exporter_ipfix_domain_t exporter, void DataPtr, uint32_t size_left, FlowSource_t *fs) {
	1438	static void Process_ipfix_template_withdraw(exporterDomain_t exporter, void DataPtr, uint32_t size_left, FlowSource_t *fs) {
1421	1439	ipfix_template_record_t *ipfix_template_record;
1422	1440
1423	1441	// a template flowset can contain multiple records ( templates )

1457	1475
1458	1476	} // End of Process_ipfix_template_withdraw
1459	1477
1460		static void Process_ipfix_option_templates(exporter_ipfix_domain_t exporter, void option_template_flowset, FlowSource_t *fs) {
1461		uint8_t *DataPtr;
1462		uint32_t size_left, size_required, i;
	1478	static void Process_ipfix_option_templates(exporterDomain_t exporter, void option_template_flowset, FlowSource_t *fs) {
	1479	uint8_t *option_template;
	1480	uint32_t size_left, size_required;
1463	1481	// uint32_t nr_scopes, nr_options;
1464		uint16_t id, field_count, scope_field_count, offset;
1465		uint16_t offset_std_sampler_interval, offset_std_sampler_algorithm, found_std_sampling;
1466
1467		i = 0; // keep compiler happy
1468		size_left = GET_FLOWSET_LENGTH(option_template_flowset) - 4; // -4 for flowset header -> id and length
	1482	uint16_t tableID, field_count, scope_field_count, offset;
	1483	samplerOption_t *samplerOption;
	1484
	1485	size_left = GET_FLOWSET_LENGTH(option_template_flowset) - 4; // -4 for flowset header -> id and length
1469	1486	if ( size_left < 6 ) {
1470	1487	LogError("Process_ipfix: [%u] option template length error: size left %u too small for an options template",
1471	1488	exporter->info.id, size_left);
1472	1489	return;
1473	1490	}
1474	1491
1475		DataPtr = option_template_flowset + 4;
1476		id = GET_OPTION_TEMPLATE_ID(DataPtr);
1477		field_count = GET_OPTION_TEMPLATE_FIELD_COUNT(DataPtr);
1478		scope_field_count = GET_OPTION_TEMPLATE_SCOPE_FIELD_COUNT(DataPtr);
1479		DataPtr += 6;
	1492	option_template = option_template_flowset + 4;
	1493	tableID = GET_OPTION_TEMPLATE_ID(option_template);
	1494	field_count = GET_OPTION_TEMPLATE_FIELD_COUNT(option_template);
	1495	scope_field_count = GET_OPTION_TEMPLATE_SCOPE_FIELD_COUNT(option_template);
	1496	option_template += 6;
1480	1497	size_left -= 6;
1481	1498
1482		dbg_printf("Decode Option Template. id: %u, field count: %u, scope field count: %u\n",
1483		id, field_count, scope_field_count);
	1499	dbg_printf("Decode Option Template. tableID: %u, field count: %u, scope field count: %u\n",
	1500	tableID, field_count, scope_field_count);
1484	1501
1485	1502	if ( scope_field_count == 0 ) {
1486	1503	LogError("Process_ipfx: [%u] scope field count error: length must not be zero",

1505	1522	return;
1506	1523	}
1507	1524
1508		offset_std_sampler_interval = 0;
1509		offset_std_sampler_algorithm = 0;
1510		found_std_sampling = 0;
	1525	samplerOption = (samplerOption_t *)malloc(sizeof(samplerOption_t));
	1526	if ( !samplerOption ) {
	1527	LogError("Error malloc(): %s in %s:%d", strerror (errno), __FILE__, __LINE__);
	1528	return;
	1529	}
	1530	memset((void *)samplerOption, 0, sizeof(samplerOption_t));
	1531
	1532	samplerOption->tableID = tableID;
	1533
	1534	int i;
1511	1535	offset = 0;
1512
1513	1536	for ( i=0; i<scope_field_count; i++ ) {
1514	1537	uint16_t id, length;
1515	1538	int Enterprise;

1519	1542	exporter->info.id, __FILE__, __LINE__, strerror (errno));
1520	1543	return;
1521	1544	}
1522		id = Get_val16(DataPtr); DataPtr += 2;
1523		length = Get_val16(DataPtr); DataPtr += 2;
	1545	id = Get_val16(option_template); option_template += 2;
	1546	length = Get_val16(option_template); option_template += 2;
1524	1547	size_left -= 4;
1525	1548	Enterprise = id & 0x8000 ? 1 : 0;
1526	1549	if ( Enterprise ) {

1531	1554	dbg_printf("option template length error: size left %u too small\n", size_left);
1532	1555	return;
1533	1556	}
1534		DataPtr += 4;
	1557	option_template += 4;
1535	1558	size_left -= 4;
1536	1559	dbg_printf(" [%i] Enterprise: 1, scope id: %u, scope length %u enterprise value: %u\n",
1537		i, id, length, Get_val32(DataPtr));
	1560	i, id, length, Get_val32(option_template));
1538	1561	} else {
1539	1562	dbg_printf(" [%i] Enterprise: 0, scope id: %u, scope length %u\n", i, id, length);
1540	1563	}
1541
1542	1564	offset += length;
1543	1565	}
1544	1566
1545	1567	for ( ;i<field_count; i++ ) {
1546	1568	uint32_t enterprise_value;
1547		uint16_t id, length;
	1569	uint16_t type, length;
1548	1570	int Enterprise;
1549	1571
1550	1572	// keep compiler happy
1551	1573	UNUSED(enterprise_value);
1552		id = Get_val16(DataPtr); DataPtr += 2;
1553		length = Get_val16(DataPtr); DataPtr += 2;
	1574	type = Get_val16(option_template); option_template += 2;
	1575	length = Get_val16(option_template); option_template += 2;
1554	1576	size_left -= 4;
1555		Enterprise = id & 0x8000 ? 1 : 0;
	1577	Enterprise = type & 0x8000 ? 1 : 0;
1556	1578	if ( Enterprise ) {
1557	1579	size_required += 4;
1558	1580	if ( size_left < 4 ) {

1561	1583	dbg_printf("option template length error: size left %u too small\n", size_left);
1562	1584	return;
1563	1585	}
1564		enterprise_value = Get_val32(DataPtr);
1565		DataPtr += 4;
	1586	enterprise_value = Get_val32(option_template);
	1587	option_template += 4;
1566	1588	size_left -= 4;
1567		dbg_printf(" [%i] Enterprise: 1, option id: %u, option length %u enterprise value: %u\n",
1568		i, id, length, enterprise_value);
	1589	dbg_printf(" [%i] Enterprise: 1, option type: %u, option length %u enterprise value: %u\n",
	1590	i, type, length, enterprise_value);
1569	1591	} else {
1570		dbg_printf(" [%i] Enterprise: 0, option id: %u, option length %u\n", i, id, length);
1571		}
1572
1573		switch (id) {
	1592	dbg_printf(" [%i] Enterprise: 0, option type: %u, option length %u\n", i, type, length);
	1593	}
	1594
	1595	switch (type) {
1574	1596	// general sampling
1575		case IPFIX_samplingInterval: // legacy #34
1576		case IPFIX_samplingPacketInterval: // #305
1577		if ( length == 4 ) {
1578		offset_std_sampler_interval = offset;
1579		found_std_sampling++;
1580		dbg_printf(" 4 byte sampling interval option at offset: %u\n", offset);
1581		} else {
1582		LogError("Process_ipfix: [%u] option template error: sampling option lenth != 4 bytes: %u",
1583		exporter->info.id, length);
1584		}
1585		break;
1586		case IPFIX_samplingAlgorithm: // legacy #35
1587		case IPFIX_selectorAlgorithm: // #304
1588		if ( length == 1 ) {
1589		offset_std_sampler_algorithm = offset;
1590		dbg_printf(" 1 byte sampling algorithm option at offset: %u\n", offset);
1591		found_std_sampling++;
1592		} else {
1593		LogError("Process_ipfix: [%u] option template error: algorithm option lenth != 1 byte: %u",
1594		exporter->info.id, length);
1595		}
1596		break;
1597		}
1598
	1597	case IPFIX_samplingInterval: // #34
	1598	samplerOption->interval.length = length;
	1599	samplerOption->interval.offset = offset;
	1600	SetFlag(samplerOption->flags, STDSAMPLING34);
	1601	break;
	1602	case IPFIX_samplingAlgorithm: // #35
	1603	samplerOption->mode.length = length;
	1604	samplerOption->mode.offset = offset;
	1605	SetFlag(samplerOption->flags, STDSAMPLING35);
	1606	break;
	1607
	1608	// individual samplers
	1609	case IPFIX_samplerId: // #48 depricated - fall through
	1610	case IPFIX_selectorId: // #302
	1611	samplerOption->id.length = length;
	1612	samplerOption->id.offset = offset;
	1613	SetFlag(samplerOption->flags, SAMPLER302);
	1614	break;
	1615	case IPFIX_samplerMode: // #49 depricated - fall through
	1616	case IPFIX_selectorAlgorithm: // #304
	1617	samplerOption->mode.length = length;
	1618	samplerOption->mode.offset = offset;
	1619	SetFlag(samplerOption->flags, SAMPLER304);
	1620	break;
	1621	case IPFIX_samplerRandomInterval: // #50 depricated - fall through
	1622	case IPFIX_samplingPacketInterval: // #305
	1623	samplerOption->interval.length = length;
	1624	samplerOption->interval.offset = offset;
	1625	SetFlag(samplerOption->flags, SAMPLER305);
	1626	break;
	1627	}
1599	1628	offset += length;
1600	1629	}
1601	1630
1602		if ( offset_std_sampler_interval ) {
1603		dbg_printf("[%u] Std sampling interval found. offset: %u\n",
1604		exporter->info.id, offset_std_sampler_interval);
1605		if ( offset_std_sampler_algorithm )
1606		dbg_printf("[%u] Std sampling algorithm found. offset: %u\n",
1607		exporter->info.id, offset_std_sampler_algorithm);
1608		InsertStdSamplerOffset(fs, id, offset_std_sampler_interval, offset_std_sampler_algorithm);
1609		dbg_printf("\n");
1610		}
1611
	1631	if ( (samplerOption->flags & SAMPLERMASK ) == SAMPLERFLAGS) {
	1632	dbg_printf("[%u] Sampler information found\n", exporter->info.id);
	1633	InsertSamplerOption(exporter, samplerOption);
	1634	} else if ( (samplerOption->flags & STDMASK ) == STDFLAGS) {
	1635	dbg_printf("[%u] Std sampling information found\n", exporter->info.id);
	1636	InsertSamplerOption(exporter, samplerOption);
	1637	} else {
	1638	free(samplerOption);
	1639	dbg_printf("[%u] No Sampling information found\n", exporter->info.id);
	1640	}
1612	1641	processed_records++;
1613	1642
1614	1643	} // End of Process_ipfix_option_templates
1615	1644
1616
1617		static void Process_ipfix_data(exporter_ipfix_domain_t exporter, uint32_t ExportTime, void data_flowset, FlowSource_t fs, input_translation_t table ){
	1645	static void Process_ipfix_data(exporterDomain_t exporter, uint32_t ExportTime, void data_flowset, FlowSource_t fs, input_translation_t table ){
1618	1646	uint64_t sampling_rate;
1619	1647	uint32_t size_left;
1620	1648	uint8_t in, out;

1632	1660	// Check if sampling is announced
1633	1661	sampling_rate = 1;
1634	1662
1635		generic_sampler_t *sampler = exporter->sampler;
	1663	sampler_t *sampler = exporter->sampler;
1636	1664	while ( sampler && sampler->info.id != -1 )
1637	1665	sampler = sampler->next;
1638	1666

1944	1972	master_record_t master_record;
1945	1973	memset((void *)&master_record, 0, sizeof(master_record_t));
1946	1974	ExpandRecord_v2((common_record_t *)data_record, &(table->extension_info), &(exporter->info), &master_record);
1947		format_file_block_record(&master_record, &string, 0);
	1975	flow_record_to_raw(&master_record, &string, 0);
1948	1976	printf("%s\n", string);
1949	1977	}
1950	1978

1980	2008
1981	2009	} // End of Process_ipfix_data
1982	2010
1983		static void Process_ipfix_option_data(exporter_ipfix_domain_t exporter, void data_flowset, FlowSource_t *fs) {
1984		option_offset_t *offset_table;
1985		uint32_t id;
	2011	static void Process_ipfix_option_data(exporterDomain_t exporter, void data_flowset, FlowSource_t *fs) {
	2012	samplerOption_t *samplerOption;
	2013	uint32_t tableID;
1986	2014	uint8_t *in;
1987	2015
1988		id = GET_FLOWSET_ID(data_flowset);
1989
1990		offset_table = fs->option_offset_table;
1991		while ( offset_table && offset_table->id != id )
1992		offset_table = offset_table->next;
1993
1994		if ( !offset_table ) {
	2016	tableID = GET_FLOWSET_ID(data_flowset);
	2017
	2018	samplerOption = exporter->samplerOption;
	2019	while ( samplerOption && samplerOption->tableID != tableID )
	2020	samplerOption = samplerOption->next;
	2021
	2022	if ( !samplerOption ) {
1995	2023	// should never happen - catch it anyway
1996	2024	LogError( "Process_ipfix: Panic! - No Offset table found! : %s line %d", __FILE__, __LINE__);
1997	2025	return;

2005	2033	// map input buffer as a byte array
2006	2034	in = (uint8_t *)(data_flowset + 4); // skip flowset header
2007	2035
2008		if ( TestFlag(offset_table->flags, HAS_SAMPLER_DATA) ) {
	2036	if ( (samplerOption->flags & SAMPLERMASK ) == SAMPLERFLAGS) {
2009	2037	int32_t id;
2010	2038	uint16_t mode;
2011	2039	uint32_t interval;
2012		if (offset_table->sampler_id_length == 2) {
2013		id = Get_val16((void *)&in[offset_table->offset_id]);
2014		} else {
2015		id = in[offset_table->offset_id];
2016		}
2017
2018		mode = offset_table->offset_mode ? in[offset_table->offset_mode] : 0;
2019		interval = Get_val32((void *)&in[offset_table->offset_interval]);
2020
	2040
	2041	id = Get_val(in, samplerOption->id.offset, samplerOption->id.length);
	2042	mode = Get_val(in, samplerOption->mode.offset, samplerOption->mode.length);
	2043	interval = Get_val(in, samplerOption->interval.offset, samplerOption->interval.length);
	2044
2021	2045	InsertSampler(fs, exporter, id, mode, interval);
2022	2046
2023	2047	dbg_printf("Extracted Sampler data:\n");
2024		dbg_printf("Sampler ID : %u\n", id);
	2048	dbg_printf("Sampler ID : %u\n", id);
2025	2049	dbg_printf("Sampler mode : %u\n", mode);
2026	2050	dbg_printf("Sampler interval: %u\n", interval);
2027	2051	}
2028	2052
2029		if ( TestFlag(offset_table->flags, HAS_STD_SAMPLER_DATA) ) {
2030		int32_t id = -1;
2031		uint16_t mode = offset_table->offset_std_sampler_algorithm ? in[offset_table->offset_std_sampler_algorithm] : 0;
2032		uint32_t interval = Get_val32((void *)&in[offset_table->offset_std_sampler_interval]);
	2053	if ( (samplerOption->flags & STDMASK ) == STDFLAGS) {
	2054	int32_t id;
	2055	uint16_t mode;
	2056	uint32_t interval;
	2057
	2058	id = -1;
	2059	mode = Get_val(in, samplerOption->mode.offset, samplerOption->mode.length);
	2060	interval = Get_val(in, samplerOption->interval.offset, samplerOption->interval.length);
2033	2061
2034	2062	InsertSampler(fs, exporter, id, mode, interval);
2035	2063
2036	2064	dbg_printf("Extracted Std Sampler data:\n");
2037		dbg_printf("Sampler ID : %i\n", id);
	2065	dbg_printf("Sampler ID : %i\n", id);
2038	2066	dbg_printf("Sampler algorithm: %u\n", mode);
2039	2067	dbg_printf("Sampler interval : %u\n", interval);
2040	2068

2046	2074	} // End of Process_ipfix_option_data
2047	2075
2048	2076	void Process_IPFIX(void in_buff, ssize_t in_buff_cnt, FlowSource_t fs) {
2049		exporter_ipfix_domain_t *exporter;
	2077	exporterDomain_t *exporter;
2050	2078	ssize_t size_left;
2051	2079	uint32_t ExportTime, Sequence, flowset_length;
2052	2080	ipfix_header_t *ipfix_header;

2176	2204	if ( table ) {
2177	2205	Process_ipfix_data(exporter, ExportTime, flowset_header, fs, table);
2178	2206	exporter->DataRecords++;
2179		} else if ( HasOptionTable(fs, flowset_id) ) {
	2207	} else if ( HasOptionTable(exporter, flowset_id) ) {
2180	2208	Process_ipfix_option_data(exporter, flowset_header, fs);
2181	2209	} else {
2182	2210	// maybe a flowset with option data

+15

-10

bin/ipfix.h less more

0	0	/*
1		* Copyright (c) 2009-2019, Peter Haag
	1	* Copyright (c) 2009-2020, Peter Haag
2	2	* All rights reserved.
3	3	*
4	4	* Redistribution and use in source and binary forms, with or without

230	230	// deprecated elements for sampling
231	231	#define IPFIX_samplingInterval 34
232	232	#define IPFIX_samplingAlgorithm 35
233		// 1 - Deterministic Sampling,
234		// 2 - Random Sampling.
235
236		#define IPFIX_samplingPacketInterval 305
237		#define IPFIX_selectorAlgorithm 304
238
239		// reserved 38, 39
240		// reserved 48, 49, 50, 51
	233
	234	// depricated 38, 39
	235
	236	#define IPFIX_samplerId 48
	237	#define IPFIX_samplerMode 49
	238	#define IPFIX_samplerRandomInterval 50
241	239
242	240	// #define IPFIX_MIN_TTL 52
243	241	// #define IPFIX_MAX_TTL 53

274	272	#define IPFIX_flowEndMilliseconds 153
275	273	#define IPFIX_flowStartDeltaMicroseconds 158
276	274	#define IPFIX_flowEndDeltaMicroseconds 159
	275	#define IPFIX_postOctetTotalCount 171
	276	#define IPFIX_postPacketTotalCount 172
	277
	278	// sampling
	279	#define IPFIX_selectorId 302
	280	#define IPFIX_selectorAlgorithm 304
	281	#define IPFIX_samplingPacketInterval 305
277	282
278	283	// Private Enterprise Numbers
279	284	#define IPFIX_ReverseInformationElement 29305
280	285
281	286	/* prototypes */
282		int Init_IPFIX(void);
	287	int Init_IPFIX(int v, uint32_t sampling, uint32_t overwrite);
283	288
284	289	void Process_IPFIX(void in_buff, ssize_t in_buff_cnt, FlowSource_t fs);
285	290

-4

bin/ipfrag.c less more

0	0	/*
1		* Copyright (c) 2014-2019, Peter Haag
	1	* Copyright (c) 2014-2020, Peter Haag
2	2	* All rights reserved.
3	3	*
4	4	* Redistribution and use in source and binary forms, with or without

44	44	#include <netinet/in.h>
45	45	#endif
46	46
	47	#include <sys/socket.h>
47	48	#include <netinet/in.h>
48	49	#include <netinet/ip.h>
49	50	#include <arpa/inet.h>

193	194	struct IPFragNode FindNode, *n;
194	195	hole_t hole, h, *hole_parent;
195	196	uint16_t more_fragments, first, last, max;
196		int found_hole;
	197	dbg(int found_hole;)
197	198	char src_s[16], dst_s[16];
198	199
199	200	if ( (when - lastExpire ) > 10 ) {

239	240	}
240	241
241	242	// last fragment - sets max offset
242		found_hole = 0;
	243	dbg(found_hole = 0;)
243	244	max = more_fragments == 0 ? last : 0;
244	245	dbg_printf("Fragment assembly: first: %u, last: %u, MF: %u, ID: %x\n", first, last, more_fragments, ident);
245	246	while (hole) {

291	292	return NULL;
292	293	}
293	294	// fragment fits into hole
294		found_hole = 1;
	295	dbg(found_hole = 1;)
295	296	if ( last > n->data_size )
296	297	n->data_size = last;
297	298

+25

-31

bin/launch.c less more

0	0	/*
1		* Copyright (c) 2017, Peter Haag
2		* Copyright (c) 2016, Peter Haag
3		* Copyright (c) 2014, Peter Haag
4		* Copyright (c) 2009, Peter Haag
	1	* Copyright (c) 2009-2020, Peter Haag
5	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
6	3	* All rights reserved.
7	4	*

66	63	#define fts_set fts_set_compat
67	64	#endif
68	65
	66	#include "util.h"
	67	#include "nfdump.h"
	68	#include "nffile.h"
69	69	#include "expire.h"
70		#include "nffile.h"
71	70	#include "collector.h"
72		#include "util.h"
	71	#include "launch.h"
	72
	73	#define DEVEL 1
73	74
74	75	static int done, launch, child_exit;
75	76

117	118
118	119	q = strdup(process);
119	120	if ( !q ) {
120		perror("Process cmdline");
	121	LogError("strdup() error in %s:%i: %s", __FILE__, __LINE__ , strerror(errno));
121	122	return NULL;
122	123	}
123	124	i = 0;

136	137	s = InfoRecord->tstring;
137	138	break;
138	139	case 'u' :
139		#if defined __OpenBSD__ \|\| defined __FreeBSD__
140		snprintf(tmp, 16, "%i", InfoRecord->tstamp);
141		#else
142		snprintf(tmp, 16, "%li", InfoRecord->tstamp);
143		#endif
	140	snprintf(tmp, 16, "%lli", (long long)InfoRecord->tstamp);
144	141	tmp[15] = 0;
145	142	s = tmp;
146	143	break;

154	151	if ( s ) {
155	152	q = realloc(q, strlen(q) + strlen(s));
156	153	if ( !q ) {
157		perror("Process cmdline");
	154	LogError("realloc() error in %s:%i: %s", __FILE__, __LINE__ , strerror(errno));
158	155	return NULL;
159	156	}
160		// be a bit paranoid and prevent endless expansion
	157	// sanity check
161	158	if ( strlen(q) > MAXCMDLEN ) {
162		// this is fishy
163		LogError("Error: cmdline too long!");
	159	LogError("command expand error in %s:%i: cmd line too long", __FILE__, __LINE__);
164	160	return NULL;
165	161	}
166	162	memmove(&q[i] + strlen(s), &q[i+2], strlen(&q[i+2]) + 1); // include trailing '0' in memmove

319	315
320	316	} // End of do_expire
321	317
322		void launcher (char commbuff, FlowSource_t FlowSource, char *process, int expire) {
	318	void launcher (void commbuff, FlowSource_t FlowSource, char *process, int expire) {
323	319	FlowSource_t *fs;
324	320	struct sigaction act;
325	321	char *args[MAXARGS];

336	332	char *cmd = NULL;
337	333	srecord_t TestRecord;
338	334	// check for valid command expansion
339		strncpy(TestRecord.fname, "test", FNAME_SIZE-1);
340		TestRecord.fname[FNAME_SIZE-1] = 0;
341		strncpy(TestRecord.tstring, "200407110845", 15);
342		TestRecord.tstring[15] = 0;
	335	strncpy(TestRecord.fname, "test", MAXPATHLEN-1);
	336	TestRecord.fname[MAXPATHLEN-1] = 0;
	337	strncpy(TestRecord.tstring, "20190711084500", MAXTIMESTRING-1);
	338	TestRecord.tstring[MAXTIMESTRING-1] = 0;
343	339	TestRecord.tstamp = 1;
344	340
	341	// checkk valid command expansion
345	342	fs = FlowSource;
346		while ( fs ) {
347		cmd = cmd_expand(&TestRecord, fs->Ident, fs->datadir, process);
348		if ( cmd == NULL ) {
349		LogError("Launcher: ident: %s, Unable to expand command: '%s'", fs->Ident, process);
350		exit(255);
351		}
352
353		fs = fs->next;
354		}
	343	cmd = cmd_expand(&TestRecord, fs->Ident, fs->datadir, process);
	344	if ( cmd == NULL ) {
	345	LogError("Launcher: ident: %s, Unable to expand command: '%s'", fs->Ident, process);
	346	exit(255);
	347	}
	348	free(cmd);
355	349	}
356	350
357	351	/* Signal handling */

373	367	launch = 0;
374	368
375	369	if ( process ) {
376		char *cmd = NULL;
	370	char *cmd = NULL;
377	371
378	372	fs = FlowSource;
379	373	while ( fs ) {

-10

bin/launch.h less more

0	0	/*
1		* Copyright (c) 2017, Peter Haag
2		* Copyright (c) 2014, Peter Haag
3		* Copyright (c) 2009, Peter Haag
	1	* Copyright (c) 2009-2019, Peter Haag
4	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
5	3	* All rights reserved.
6	4	*

36	34	#include "config.h"
37	35
38	36	#include <time.h>
	37	#include <sys/param.h>
	38	#include "collector.h"
	39	#include "nffile.h"
39	40
40		#define FNAME_SIZE 256
41		#define IDENT_SIZE 32
	41	#define MAXTIMESTRING 64
42	42
43	43	typedef struct srecord_s {
44		char fname[FNAME_SIZE]; // file name
45		char subdir[FNAME_SIZE]; // subdir name
46		char ident[IDENT_SIZE]; // -I ident string
47		char tstring[16]; // actually 12 needed e.g. 200411011230
	44	char fname[MAXPATHLEN]; // %f file name
	45	char tstring[MAXTIMESTRING]; // %t 14 needed for YYYYmmddHHMMSS + opt. timezone
48	46	time_t tstamp; // UNIX time stamp
	47	int failed; // in case of an error
49	48	} srecord_t;
50	49
51		void launcher (char commbuff, char datadir, char *process, int expire);
	50	void launcher (void commbuff, FlowSource_t FlowSource, char *process, int expire);
52	51
53	52	#endif //_LAUNCH_H

-9

bin/netflow_pcap.c less more

0	0	/*
1		* Copyright (c) 2013-2019, Peter Haag
	1	* Copyright (c) 2013-2020, Peter Haag
2	2	* All rights reserved.
3	3	*
4	4	* Redistribution and use in source and binary forms, with or without

45	45	#endif
46	46
47	47	#include "util.h"
	48	#include "nfdump.h"
48	49	#include "nffile.h"
49	50	#include "nfx.h"
50	51	#include "nfnet.h"
51		#include "nf_common.h"
	52	#include "output_raw.h"
52	53	#include "bookkeeper.h"
53	54	#include "collector.h"
54	55	#include "exporter.h"
55	56
56	57	#include "flowtree.h"
57	58	#include "netflow_pcap.h"
58
59		#ifndef DEVEL
60		# define dbg_printf(...) /* printf(__VA_ARGS__) */
61		#else
62		# define dbg_printf(...) printf(__VA_ARGS__)
63		#endif
64	59
65	60	extern int verbose;
66	61	extern extension_descriptor_t extension_descriptor[];

311	306	if ( verbose ) {
312	307	master_record_t master_record;
313	308	ExpandRecord_v2((common_record_t *)common_record, &pcap_extension_info, NULL, &master_record);
314		format_file_block_record(&master_record, &string, 0);
	309	flow_record_to_raw(&master_record, &string, 0);
315	310	printf("%s\n", string);
316	311	}
317	312

+13

-14

bin/netflow_v1.c less more

0	0	/*
1		* Copyright (c) 2017, Peter Haag
2		* Copyright (c) 2016, Peter Haag
3		* Copyright (c) 2014, Peter Haag
4		* Copyright (c) 2009, Peter Haag
	1	* Copyright (c) 2009-2020, Peter Haag
5	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
6	3	* All rights reserved.
7	4	*

50	47	#endif
51	48
52	49	#include "util.h"
	50	#include "nfdump.h"
53	51	#include "nffile.h"
54	52	#include "nfx.h"
55	53	#include "nfnet.h"
56		#include "nf_common.h"
	54	#include "output_raw.h"
57	55	#include "bookkeeper.h"
58	56	#include "collector.h"
59	57	#include "exporter.h"
60	58	#include "netflow_v1.h"
61	59
62		extern int verbose;
63	60	extern extension_descriptor_t extension_descriptor[];
64	61
65	62	/* module limited globals */
	63	static int verbose;
66	64	static extension_info_t v1_extension_info; // common for all v1 records
67	65	static uint16_t v1_output_record_size;
68	66

79	77	#define V1_BLOCK_DATA_SIZE (sizeof(v1_block_t) - sizeof(uint32_t))
80	78
81	79	typedef struct exporter_v1_s {
82		// identical to generic_exporter_t
	80	// identical to exporter_t
83	81	struct exporter_v1_s *next;
84	82
85		// generic exporter information
	83	// exporter information
86	84	exporter_info_record_t info;
87	85
88	86	uint64_t packets; // number of packets sent by this exporter

90	88	uint32_t sequence_failure; // number of sequence failues
91	89	uint32_t padding_errors; // number of sequence failues
92	90
93		generic_sampler_t *sampler;
94		// End of generic_exporter_t
	91	sampler_t *sampler;
	92	// End of exporter_t
95	93
96	94	// extension map
97	95	extension_map_t *extension_map;

104	102
105	103	#include "nffile_inline.c"
106	104
107		int Init_v1(void) {
	105	int Init_v1(int v) {
108	106	int i, id, map_index;
109	107	int extension_size;
110	108	uint16_t map_size;
111	109
	110	verbose = v;
112	111	// prepare v1 extension map
113	112	v1_extension_info.map = NULL;
114	113	v1_extension_info.next = NULL;

138	137	if ( ( map_size & 0x3 ) != 0 )
139	138	map_size += 2;
140	139
141		// Create a generic netflow v1 extension map
	140	// Create a netflow v1 extension map
142	141	v1_extension_info.map = (extension_map_t *)malloc((size_t)map_size);
143	142	if ( !v1_extension_info.map ) {
144	143	LogError("Process_v1: malloc() error in %s line %d: %s\n", __FILE__, __LINE__, strerror (errno));

202	201	(*e)->padding_errors = 0;
203	202	(*e)->sampler = NULL;
204	203
205		// copy the v1 generic extension map
	204	// copy the v1 extension map
206	205	(e)->extension_map = (extension_map_t )malloc(v1_extension_info.map->size);
207	206	if ( !(*e)->extension_map ) {
208	207	LogError("Process_v1: malloc() error in %s line %d: %s\n", __FILE__, __LINE__, strerror (errno));

454	453	master_record_t master_record;
455	454	memset((void *)&master_record, 0, sizeof(master_record_t));
456	455	ExpandRecord_v2((common_record_t *)common_record, &v1_extension_info, &(exporter->info), &master_record);
457		format_file_block_record(&master_record, &string, 0);
	456	flow_record_to_raw(&master_record, &string, 0);
458	457	printf("%s\n", string);
459	458	}
460	459

-5

bin/netflow_v1.h less more

0	0	/*
1		* Copyright (c) 2017, Peter Haag
2		* Copyright (c) 2016, Peter Haag
3		* Copyright (c) 2014, Peter Haag
4		* Copyright (c) 2009, Peter Haag
	1	* Copyright (c) 2009-2020, Peter Haag
5	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
6	3	* All rights reserved.
7	4	*

77	74
78	75
79	76	/* prototypes */
80		int Init_v1(void);
	77	int Init_v1(int v);
81	78
82	79	void Process_v1(void in_buff, ssize_t in_buff_cnt, FlowSource_t fs);
83	80

+23

-26

bin/netflow_v5_v7.c less more

0	0	/*
1		* Copyright (c) 2017, Peter Haag
2		* Copyright (c) 2016, Peter Haag
3		* Copyright (c) 2014, Peter Haag
4		* Copyright (c) 2009, Peter Haag
	1	* Copyright (c) 2009-2020, Peter Haag
5	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
6	3	* All rights reserved.
7	4	*

50	47	#endif
51	48
52	49	#include "util.h"
	50	#include "nfdump.h"
53	51	#include "nffile.h"
54	52	#include "nfx.h"
55	53	#include "nfnet.h"
56		#include "nf_common.h"
	54	#include "output_raw.h"
57	55	#include "bookkeeper.h"
58	56	#include "collector.h"
59	57	#include "exporter.h"
60	58	#include "netflow_v5_v7.h"
61	59
62		#ifndef DEVEL
63		# define dbg_printf(...) /* printf(__VA_ARGS__) */
64		#else
65		# define dbg_printf(...) printf(__VA_ARGS__)
66		#endif
67
68		extern int verbose;
69	60	extern extension_descriptor_t extension_descriptor[];
70		extern uint32_t default_sampling;
71		extern uint32_t overwrite_sampling;
72	61
73	62	/* module limited globals */
	63	static int verbose;
	64	static uint32_t default_sampling;
	65	static uint32_t overwrite_sampling;
	66
74	67	static extension_info_t v5_extension_info; // common for all v5 records
75	68	static uint16_t v5_output_record_size, v5_output_record_base_size;
76	69

82	75	#define V5_BLOCK_DATA_SIZE (sizeof(ipv4_block_t) - sizeof(uint32_t) + 2 * sizeof(uint64_t))
83	76
84	77	typedef struct exporter_v5_s {
85		// identical to generic_exporter_t
	78	// identical to exporter_t
86	79	struct exporter_v5_s *next;
87	80
88		// generic exporter information
	81	// exporter information
89	82	exporter_info_record_t info;
90	83
91	84	uint64_t packets; // number of packets sent by this exporter

93	86	uint32_t sequence_failure; // number of sequence failues
94	87	uint32_t padding_errors; // number of sequence failues
95	88
96		// generic sampler
97		generic_sampler_t *sampler;
98		// end of generic_exporter_t
	89	// sampler
	90	sampler_t *sampler;
	91	// end of exporter_t
99	92
100	93	// sequence vars
101	94	int64_t last_sequence;

122	115
123	116	#include "nffile_inline.c"
124	117
125		int Init_v5_v7_input(void) {
	118	int Init_v5_v7_input(int v, uint32_t sampling, uint32_t overwrite) {
126	119	int i, id, map_index;
127	120	int extension_size;
128	121	uint16_t map_size;
129	122
130		extension_size = 0;
	123	verbose = v;
	124	default_sampling = sampling;
	125	overwrite_sampling = overwrite;
	126	extension_size = 0;
	127
131	128	// prepare v5 extension map
132	129	v5_extension_info.map = NULL;
133	130	v5_extension_info.next = NULL;

152	149	if ( ( map_size & 0x3 ) != 0 )
153	150	map_size += 2;
154	151
155		// Create a generic v5 extension map
	152	// Create a v5 extension map
156	153	v5_extension_info.map = (extension_map_t *)malloc((size_t)map_size);
157	154	if ( !v5_extension_info.map ) {
158	155	LogError("Process_v5: malloc() error in %s line %d: %s\n", __FILE__, __LINE__, strerror (errno));

183	180
184	181	static inline exporter_v5_t GetExporter(FlowSource_t fs, netflow_v5_header_t *header) {
185	182	exporter_v5_t e = (exporter_v5_t )&(fs->exporter_data);
186		generic_sampler_t *sampler;
	183	sampler_t *sampler;
187	184	uint16_t engine_tag = ntohs(header->engine_tag);
188	185	uint16_t version = ntohs(header->version);
189	186	#define IP_STRING_LEN 40

217	214	(*e)->flows = 0;
218	215	(*e)->first = 1;
219	216
220		sampler = (generic_sampler_t *)malloc(sizeof(generic_sampler_t));
	217	sampler = (sampler_t *)malloc(sizeof(sampler_t));
221	218	if ( !sampler ) {
222	219	LogError("Process_v5: malloc() error in %s line %d: %s\n", __FILE__, __LINE__, strerror (errno));
223	220	return NULL;

235	232	if ( sampler->info.interval == 0 )
236	233	sampler->info.interval = default_sampling;
237	234
238		// copy the v5 generic extension map
	235	// copy the v5 extension map
239	236	(e)->extension_map = (extension_map_t )malloc(v5_extension_info.map->size);
240	237	if ( !(*e)->extension_map ) {
241	238	LogError("Process_v5: malloc() error in %s line %d: %s\n", __FILE__, __LINE__, strerror (errno));

607	604	master_record_t master_record;
608	605	memset((void *)&master_record, 0, sizeof(master_record_t));
609	606	ExpandRecord_v2((common_record_t *)common_record, &v5_extension_info, &(exporter->info), &master_record);
610		format_file_block_record(&master_record, &string, 0);
	607	flow_record_to_raw(&master_record, &string, 0);
611	608	printf("%s\n", string);
612	609	}
613	610

-4

bin/netflow_v5_v7.h less more

0	0	/*
1		* Copyright (c) 2017, Peter Haag
2		* Copyright (c) 2014, Peter Haag
3		* Copyright (c) 2009, Peter Haag
	1	* Copyright (c) 2009-2020, Peter Haag
4	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
5	3	* All rights reserved.
6	4	*

124	122	} netflow_v7_record_t;
125	123
126	124	/* prototypes */
127		int Init_v5_v7_input(void);
	125	int Init_v5_v7_input(int v, uint32_t sampling, uint32_t overwrite);
128	126
129	127	void Process_v5_v7(void in_buff, ssize_t in_buff_cnt, FlowSource_t fs);
130	128

+204

-236

bin/netflow_v9.c less more

0	0	/*
1		* Copyright (c) 2017, Peter Haag
2		* Copyright (c) 2014, Peter Haag
3		* Copyright (c) 2009, Peter Haag
	1	* Copyright (c) 2009-2020, Peter Haag
4	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
5	3	* All rights reserved.
6	4	*

47	45	#include <stdint.h>
48	46	#endif
49	47
	48	#include "util.h"
	49	#include "nfdump.h"
50	50	#include "nffile.h"
51	51	#include "nfx.h"
52	52	#include "nfnet.h"
53		#include "nf_common.h"
54		#include "util.h"
	53	#include "output_raw.h"
55	54	#include "bookkeeper.h"
56	55	#include "collector.h"
57	56	#include "exporter.h"
58	57	#include "netflow_v9.h"
59	58
60		#ifndef DEVEL
61		# define dbg_printf(...) /* printf(__VA_ARGS__) */
62		#else
63		# define dbg_printf(...) printf(__VA_ARGS__)
64		#endif
65
66		// a few handy macros
67	59	#define GET_FLOWSET_ID(p) (Get_val16(p))
68	60	#define GET_FLOWSET_LENGTH(p) (Get_val16((void *)((p) + 2)))
69	61

76	68
77	69	#include "inline.c"
78	70
79		extern int verbose;
80	71	extern extension_descriptor_t extension_descriptor[];
81	72	extern uint32_t Max_num_extensions;
82		extern uint32_t default_sampling;
83		extern uint32_t overwrite_sampling;
	73
	74	static int verbose;
	75	static uint32_t default_sampling;
	76	static uint32_t overwrite_sampling;
84	77
85	78	typedef struct sequence_map_s {
86	79	/* sequence definition:

157	150
158	151	} input_translation_t;
159	152
160		typedef struct exporter_v9_domain_s {
161		// identical to generic_exporter_t
162		struct exporter_v9_domain_s *next;
163
164		// generic exporter information
	153	typedef struct exporter_domain_s {
	154	// identical to exporter_t
	155	struct exporter_domain_s *next;
	156
	157	// exporter information
165	158	exporter_info_record_t info;
166	159
167	160	uint64_t packets; // number of packets sent by this exporter

169	162	uint32_t sequence_failure; // number of sequence failues
170	163	uint32_t padding_errors; // number of padding errors
171	164
172		// generic sampler
173		generic_sampler_t *sampler;
174		// end of generic_exporter_t
	165	// sampler
	166	sampler_t *sampler;
	167	samplerOption_t *samplerOption; // sampler options table info
175	168
176	169	// exporter parameters
177	170	uint64_t boot_time;

191	184	// translation table
192	185	input_translation_t *input_translation_table;
193	186	input_translation_t *current_table;
194		} exporter_v9_domain_t;
	187	} exporterDomain_t;
195	188
196	189
197	190	/* module limited globals */

273	266
274	267	{ NF9_SRC_VLAN, "src vlan", _2bytes, _2bytes, move16, zero16, EX_VLAN},
275	268	{ NF9_DST_VLAN, "dst vlan", _2bytes, _2bytes, move16, zero16, EX_VLAN},
	269	{ NF9_dot1qVlanId, "src vlan", _2bytes, _2bytes, move16, zero16, EX_VLAN},
	270	{ NF9_postDot1qVlanId, "dst vlan", _2bytes, _2bytes, move16, zero16, EX_VLAN},
276	271
277	272	{ NF9_DIRECTION, "direction", _1byte, _1byte, move8, zero8, EX_MULIPLE },
278	273

309	304	{ NF_F_ICMP_CODE, "FNF ICMP code", _1byte, _1byte, move8, zero8, EX_NSEL_COMMON },
310	305	{ NF_F_ICMP_TYPE_IPV6, "ASA ICMP type V6", _1byte, _1byte, move8, zero8, EX_NSEL_COMMON },
311	306	{ NF_F_ICMP_CODE_IPV6, "ASA ICMP code V6", _1byte, _1byte, move8, zero8, EX_NSEL_COMMON },
	307	// zone base firewall
	308	{ NF_FW_CTS_SRC_SGT, "Source security group tag", _2bytes, _2bytes, move16, zero16, EX_NSEL_COMMON },
	309
312	310	// XlATE extensions
313	311	{ NF_F_XLATE_SRC_ADDR_IPV4, "ASA V4 xsrc addr", _4bytes, _4bytes, move32, zero32, EX_NSEL_XLATE_IP_v4 },
314	312	{ NF_F_XLATE_DST_ADDR_IPV4, "ASA V4 xdst addr", _4bytes, _4bytes, move32, zero32, EX_NSEL_XLATE_IP_v4 },

397	395	static uint32_t processed_records;
398	396
399	397	/* local function prototypes */
400		static void InsertSamplerOffset( FlowSource_t *fs, uint16_t id, uint16_t offset_sampler_id, uint16_t sampler_id_length,
401		uint16_t offset_sampler_mode, uint16_t offset_sampler_interval);
402
403		static void InsertStdSamplerOffset( FlowSource_t *fs, uint16_t id, uint16_t offset_std_sampler_interval,
404		uint16_t offset_std_sampler_algorithm);
405
406		static void InsertSampler( FlowSource_t fs, exporter_v9_domain_t exporter, int32_t id, uint16_t mode, uint32_t interval);
407
408		static inline void Process_v9_templates(exporter_v9_domain_t exporter, void template_flowset, FlowSource_t *fs);
409
410		static inline void Process_v9_option_templates(exporter_v9_domain_t exporter, void option_template_flowset, FlowSource_t *fs);
411
412		static inline void Process_v9_data(exporter_v9_domain_t exporter, void data_flowset, FlowSource_t fs, input_translation_t table );
413
414		static inline void Process_v9_option_data(exporter_v9_domain_t exporter, void data_flowset, FlowSource_t *fs);
415
416		static inline exporter_v9_domain_t GetExporter(FlowSource_t fs, uint32_t exporter_id);
417
418		static inline input_translation_t GetTranslationTable(exporter_v9_domain_t exporter, uint16_t id);
419
420		static input_translation_t setup_translation_table (exporter_v9_domain_t exporter, uint16_t id, uint16_t input_record_size);
421
422		static input_translation_t add_translation_table(exporter_v9_domain_t exporter, uint16_t id);
	398	static int HasOptionTable(exporterDomain_t *exporter, uint16_t tableID );
	399
	400	static void InsertSamplerOption(exporterDomain_t exporter, samplerOption_t samplerOption);
	401
	402	static void InsertSampler( FlowSource_t fs, exporterDomain_t exporter, int32_t id, uint16_t mode, uint32_t interval);
	403
	404	static inline void Process_v9_templates(exporterDomain_t exporter, void template_flowset, FlowSource_t *fs);
	405
	406	static inline void Process_v9_option_templates(exporterDomain_t exporter, void option_template_flowset, FlowSource_t *fs);
	407
	408	static inline void Process_v9_data(exporterDomain_t exporter, void data_flowset, FlowSource_t fs, input_translation_t table );
	409
	410	static inline void Process_v9_option_data(exporterDomain_t exporter, void data_flowset, FlowSource_t *fs);
	411
	412	static inline exporterDomain_t GetExporter(FlowSource_t fs, uint32_t exporter_id);
	413
	414	static inline input_translation_t GetTranslationTable(exporterDomain_t exporter, uint16_t id);
	415
	416	static input_translation_t setup_translation_table (exporterDomain_t exporter, uint16_t id, uint16_t input_record_size);
	417
	418	static input_translation_t add_translation_table(exporterDomain_t exporter, uint16_t id);
423	419
424	420	static output_template_t GetOutputTemplate(uint32_t flags, extension_map_t extension_map);
425	421

441	437
442	438	#include "nffile_inline.c"
443	439
444		int Init_v9(void) {
	440	int Init_v9(int v, uint32_t sampling, uint32_t overwrite) {
445	441	int i;
446	442
	443	verbose = v;
	444	default_sampling = sampling;
	445	overwrite_sampling = overwrite;
447	446	output_templates = NULL;
448	447
449	448	cache.lookup_info = (struct element_param_s *)calloc(65536, sizeof(struct element_param_s));

470	469
471	470	} // End of Init_v9
472	471
473		static inline exporter_v9_domain_t GetExporter(FlowSource_t fs, uint32_t exporter_id) {
	472	static int HasOptionTable(exporterDomain_t *exporter, uint16_t tableID ) {
	473	samplerOption_t *s;
	474
	475	s = exporter->samplerOption;
	476	while ( s && s->tableID != tableID )
	477	s = s->next;
	478
	479	dbg_printf("Has option table: %s\n", s == NULL ? "not found" : "found");
	480
	481	return s != NULL;
	482
	483	} // End of HasOptionTable
	484
	485	static inline exporterDomain_t GetExporter(FlowSource_t fs, uint32_t exporter_id) {
474	486	#define IP_STRING_LEN 40
475	487	char ipstr[IP_STRING_LEN];
476		exporter_v9_domain_t e = (exporter_v9_domain_t )&(fs->exporter_data);
	488	exporterDomain_t e = (exporterDomain_t )&(fs->exporter_data);
477	489
478	490	while ( *e ) {
479	491	if ( (e)->info.id == exporter_id && (e)->info.version == 9 &&

495	507	}
496	508
497	509	// nothing found
498		e = (exporter_v9_domain_t )malloc(sizeof(exporter_v9_domain_t));
	510	e = (exporterDomain_t )malloc(sizeof(exporterDomain_t));
499	511	if ( !(*e)) {
500	512	LogError( "Process_v9: Panic! malloc() %s line %d: %s", __FILE__, __LINE__, strerror (errno));
501	513	return NULL;
502	514	}
503		memset((void )(e), 0, sizeof(exporter_v9_domain_t));
	515	memset((void )(e), 0, sizeof(exporterDomain_t));
504	516	(*e)->info.header.type = ExporterInfoRecordType;
505	517	(*e)->info.header.size = sizeof(exporter_info_record_t);
506	518	(*e)->info.version = 9;

560	572
561	573	} // End of MapElement
562	574
563		static inline input_translation_t GetTranslationTable(exporter_v9_domain_t exporter, uint16_t id) {
	575	static inline input_translation_t GetTranslationTable(exporterDomain_t exporter, uint16_t id) {
564	576	input_translation_t *table;
565	577
566	578	if ( exporter->current_table && ( exporter->current_table->id == id ) )

584	596
585	597	} // End of GetTranslationTable
586	598
587		static input_translation_t add_translation_table(exporter_v9_domain_t exporter, uint16_t id) {
	599	static input_translation_t add_translation_table(exporterDomain_t exporter, uint16_t id) {
588	600	input_translation_t **table;
589	601
590	602	table = &(exporter->input_translation_table);

651	663	} // End of PushSequence
652	664
653	665
654		static input_translation_t setup_translation_table (exporter_v9_domain_t exporter, uint16_t id, uint16_t input_record_size) {
	666	static input_translation_t setup_translation_table (exporterDomain_t exporter, uint16_t id, uint16_t input_record_size) {
655	667	input_translation_t *table;
656	668	extension_map_t *extension_map;
657	669	uint32_t i, ipv6, offset, next_extension;

867	879	SetFlag(table->flags, FLAG_IPV6_NHB);
868	880	break;
869	881	case EX_VLAN:
870		PushSequence( table, NF9_SRC_VLAN, &offset, NULL, 0);
871		PushSequence( table, NF9_DST_VLAN, &offset, NULL, 0);
	882	if ( cache.lookup_info[NF9_dot1qVlanId].found ) {
	883	PushSequence( table, NF9_dot1qVlanId, &offset, NULL, 0);
	884	PushSequence( table, NF9_postDot1qVlanId, &offset, NULL, 0);
	885	} else {
	886	PushSequence( table, NF9_SRC_VLAN, &offset, NULL, 0);
	887	PushSequence( table, NF9_DST_VLAN, &offset, NULL, 0);
	888	}
872	889	break;
873	890	case EX_OUT_PKG_4:
874	891	PushSequence( table, NF9_OUT_PKTS, &offset, &table->out_packets, 0);

1023	1040	PushSequence( table, NF_F_FW_EVENT, &offset, NULL, 0);
1024	1041	offset += 1;
1025	1042	PushSequence( table, NF_F_FW_EXT_EVENT, &offset, NULL, 0);
1026		offset += 2;
	1043	PushSequence( table, NF_FW_CTS_SRC_SGT, &offset, NULL, 0);
1027	1044	break;
1028	1045	case EX_NSEL_XLATE_PORTS:
1029	1046	if ( cache.lookup_info[NF_F_XLATE_SRC_ADDR_84].found ) {

1149	1166
1150	1167	} // End of setup_translation_table
1151	1168
1152		static void InsertSamplerOffset( FlowSource_t *fs, uint16_t id, uint16_t offset_sampler_id, uint16_t sampler_id_length,
1153		uint16_t offset_sampler_mode, uint16_t offset_sampler_interval) {
1154		option_offset_t **t;
1155
1156		t = &(fs->option_offset_table);
1157		while ( *t ) {
1158		if ( (*t)->id == id ) { // table already known to us - update data
1159		dbg_printf("Found existing sampling info in template %i\n", id);
	1169	static void InsertSamplerOption(exporterDomain_t exporter, samplerOption_t samplerOption) {
	1170	samplerOption_t s, parent;
	1171
	1172	parent = NULL;
	1173	s = exporter->samplerOption;
	1174	while (s) {
	1175	if ( s->tableID == samplerOption->tableID ) { // table already known to us - update data
	1176	dbg_printf("Found existing sampling info in template %i\n", samplerOption->tableID);
1160	1177	break;
1161	1178	}
1162
1163		t = &((*t)->next);
1164		}
1165
1166		if ( *t == NULL ) { // new table
1167		dbg_printf("Allocate new sampling info from template %i\n", id);
1168		t = (option_offset_t )calloc(1, sizeof(option_offset_t));
1169		if ( !*t ) {
1170		fprintf(stderr, "malloc() allocation error: %s\n", strerror(errno));
1171		return ;
1172		}
1173		dbg_printf("Process_v9: New sampler: ID %i, mode: %i, interval: %i\n",
1174		offset_sampler_id, offset_sampler_mode, offset_sampler_interval);
1175		} // else existing table
1176
1177		dbg_printf("Insert/Update sampling info from template %i\n", id);
1178		SetFlag((*t)->flags, HAS_SAMPLER_DATA);
1179		(*t)->id = id;
1180		(*t)->offset_id = offset_sampler_id;
1181		(*t)->sampler_id_length = sampler_id_length;
1182		(*t)->offset_mode = offset_sampler_mode;
1183		(*t)->offset_interval = offset_sampler_interval;
1184
1185		} // End of InsertSamplerOffset
1186
1187		static void InsertStdSamplerOffset( FlowSource_t *fs, uint16_t id, uint16_t offset_std_sampler_interval, uint16_t offset_std_sampler_algorithm) {
1188		option_offset_t **t;
1189
1190		t = &(fs->option_offset_table);
1191		while ( *t ) {
1192		if ( (*t)->id == id ) { // table already known to us - update data
1193		dbg_printf("Found existing std sampling info in template %i\n", id);
1194		break;
1195		}
1196
1197		t = &((*t)->next);
1198		}
1199
1200		if ( *t == NULL ) { // new table
1201		dbg_printf("Allocate new std sampling info from template %i\n", id);
1202		t = (option_offset_t )calloc(1, sizeof(option_offset_t));
1203		if ( !*t ) {
1204		fprintf(stderr, "malloc() allocation error: %s\n", strerror(errno));
1205		return ;
1206		}
1207		LogError( "Process_v9: New std sampler: interval: %i, algorithm: %i",
1208		offset_std_sampler_interval, offset_std_sampler_algorithm);
1209		} // else existing table
1210
1211		dbg_printf("Insert/Update sampling info from template %i\n", id);
1212		SetFlag((*t)->flags, HAS_STD_SAMPLER_DATA);
1213		(*t)->id = id;
1214		(*t)->offset_id = 0;
1215		(*t)->offset_mode = 0;
1216		(*t)->offset_interval = 0;
1217		(*t)->offset_std_sampler_interval = offset_std_sampler_interval;
1218		(*t)->offset_std_sampler_algorithm = offset_std_sampler_algorithm;
1219
1220		} // End of InsertStdSamplerOffset
1221
1222		static inline void Process_v9_templates(exporter_v9_domain_t exporter, void template_flowset, FlowSource_t *fs) {
	1179	parent = s;
	1180	s = s->next;
	1181	}
	1182
	1183	if ( s != NULL ) { // existing entry
	1184	// replace existing table
	1185	dbg_printf("Replace existing sampler table ID %i\n", samplerOption->tableID);
	1186	if ( parent ) {
	1187	parent->next = samplerOption;
	1188	} else {
	1189	exporter->samplerOption = samplerOption;
	1190	}
	1191	samplerOption->next = s->next;
	1192	free(s);
	1193	s = NULL;
	1194	} else { // new entry
	1195	dbg_printf("New sampling table ID %i\n", samplerOption->tableID);
	1196	// push new sampling table
	1197	samplerOption->next = exporter->samplerOption;
	1198	exporter->samplerOption = samplerOption;
	1199	}
	1200
	1201	dbg_printf("Update/Insert sampler table id: %u flags: 0x%x - sampler ID: %u/%u, mode: %u/%u, interval: %u/%u\n",
	1202	samplerOption->tableID, samplerOption->flags,
	1203	samplerOption->id.offset, samplerOption->id.length,
	1204	samplerOption->mode.offset, samplerOption->mode.length,
	1205	samplerOption->interval.offset, samplerOption->interval.length);
	1206
	1207	} // End of InsertSamplerOption
	1208
	1209	static inline void Process_v9_templates(exporterDomain_t exporter, void template_flowset, FlowSource_t *fs) {
1223	1210	void *template;
1224	1211	input_translation_t *translation_table;
1225	1212	uint16_t id, count, Offset;

1244	1231	if ( v9_element_map[i].id == v9_element_map[i-1].id )
1245	1232	continue;
1246	1233	cache.lookup_info[Type].index = i;
1247		// other elements cleard be memset
	1234	// other elements cleared by memset
1248	1235	}
1249	1236
1250	1237	id = GET_TEMPLATE_ID(template);

1348	1335
1349	1336	} // End of Process_v9_templates
1350	1337
1351		static inline void Process_v9_option_templates(exporter_v9_domain_t exporter, void option_template_flowset, FlowSource_t *fs) {
	1338	static inline void Process_v9_option_templates(exporterDomain_t exporter, void option_template_flowset, FlowSource_t *fs) {
1352	1339	uint8_t option_template, p;
1353		uint32_t size_left, nr_scopes, nr_options, i;
1354		uint16_t id, scope_length, option_length, offset, sampler_id_length;
1355		uint16_t offset_sampler_id, offset_sampler_mode, offset_sampler_interval, found_sampler;
1356		uint16_t offset_std_sampler_interval, offset_std_sampler_algorithm, found_std_sampling;
1357
1358		i = 0; // keep compiler happy
	1340	uint32_t size_left, nr_scopes, nr_options;
	1341	uint16_t tableID, scope_length, option_length;
	1342	samplerOption_t *samplerOption;
	1343
1359	1344	size_left = GET_FLOWSET_LENGTH(option_template_flowset) - 4; // -4 for flowset header -> id and length
1360	1345	option_template = option_template_flowset + 4;
1361		id = GET_OPTION_TEMPLATE_ID(option_template);
	1346	tableID = GET_OPTION_TEMPLATE_ID(option_template);
1362	1347	scope_length = GET_OPTION_TEMPLATE_OPTION_SCOPE_LENGTH(option_template);
1363	1348	option_length = GET_OPTION_TEMPLATE_OPTION_LENGTH(option_template);
1364	1349

1383	1368	nr_scopes = scope_length >> 2;
1384	1369	nr_options = option_length >> 2;
1385	1370
1386		dbg_printf("\n[%u] Option Template ID: %u\n", exporter->info.id, id);
	1371	dbg_printf("\n[%u] Option Template ID: %u\n", exporter->info.id, tableID);
1387	1372	dbg_printf("Scope length: %u Option length: %u\n", scope_length, option_length);
1388	1373
1389		sampler_id_length = 0;
1390		offset_sampler_id = 0;
1391		offset_sampler_mode = 0;
1392		offset_sampler_interval = 0;
1393		offset_std_sampler_interval = 0;
1394		offset_std_sampler_algorithm = 0;
1395		found_sampler = 0;
1396		found_std_sampling = 0;
1397		offset = 0;
1398
	1374	samplerOption = (samplerOption_t *)malloc(sizeof(samplerOption_t));
	1375	if ( !samplerOption ) {
	1376	LogError("Error malloc(): %s in %s:%d", strerror (errno), __FILE__, __LINE__);
	1377	return;
	1378	}
	1379	memset((void *)samplerOption, 0, sizeof(samplerOption_t));
	1380
	1381	samplerOption->tableID = tableID;
	1382
	1383	int i;
	1384	uint16_t offset = 0;
1399	1385	p = option_template + 6; // start of length/type data
1400	1386	for ( i=0; i<nr_scopes; i++ ) {
1401	1387	#ifdef DEVEL

1434	1420	for ( ; i<(nr_scopes+nr_options); i++ ) {
1435	1421	uint16_t type = Get_val16(p); p = p + 2;
1436	1422	uint16_t length = Get_val16(p); p = p + 2;
1437		uint32_t index = cache.lookup_info[type].index;
1438	1423	dbg_printf("Option field Type: %u, length %u\n", type, length);
1439		if ( !index ) {
1440		dbg_printf("Unsupported: Option field Type: %u, length %u\n", type, length);
1441		offset += length;
1442		continue;
1443		}
1444		while ( index && v9_element_map[index].id == type ) {
1445		if ( length == v9_element_map[index].length ) {
1446		break;
1447		}
1448		index++;
1449		}
1450
1451		if ( index && v9_element_map[index].length != length ) {
1452		LogError("Process_v9: Option field Type: %u, length %u not supported\n", type, length);
1453		dbg_printf("Process_v9: Option field Type: %u, length %u not supported\n", type, length);
1454		offset += length;
1455		continue;
1456		}
	1424
1457	1425	switch (type) {
1458	1426	// general sampling
1459		case NF9_SAMPLING_INTERVAL:
1460		offset_std_sampler_interval = offset;
1461		found_std_sampling++;
1462		break;
1463		case NF9_SAMPLING_ALGORITHM:
1464		offset_std_sampler_algorithm = offset;
1465		found_std_sampling++;
	1427	case NF9_SAMPLING_INTERVAL: // #34
	1428	samplerOption->interval.length = length;
	1429	samplerOption->interval.offset = offset;
	1430	SetFlag(samplerOption->flags, STDSAMPLING34);
	1431	break;
	1432	case NF9_SAMPLING_ALGORITHM: // #35
	1433	samplerOption->mode.length = length;
	1434	samplerOption->mode.offset = offset;
	1435	SetFlag(samplerOption->flags, STDSAMPLING35);
1466	1436	break;
1467	1437
1468	1438	// individual samplers
1469		case NF9_FLOW_SAMPLER_ID: // depricated
1470		case NF_SELECTOR_ID:
1471		offset_sampler_id = offset;
1472		sampler_id_length = length;
1473		found_sampler++;
1474		break;
1475		case FLOW_SAMPLER_MODE: // // depricated
1476		case NF_SELECTOR_ALGORITHM:
1477		offset_sampler_mode = offset;
1478		found_sampler++;
1479		break;
1480		case NF9_FLOW_SAMPLER_RANDOM_INTERVAL: // depricated
1481		case NF_SAMPLING_INTERVAL:
1482		offset_sampler_interval = offset;
1483		offset_std_sampler_interval = offset;
1484		found_sampler++;
1485		found_std_sampling++;
	1439	case NF9_FLOW_SAMPLER_ID: // #48 depricated - fall through
	1440	case NF_SELECTOR_ID: // #302
	1441	samplerOption->id.length = length;
	1442	samplerOption->id.offset = offset;
	1443	SetFlag(samplerOption->flags, SAMPLER302);
	1444	break;
	1445	case FLOW_SAMPLER_MODE: // #49 depricated - fall through
	1446	case NF_SELECTOR_ALGORITHM: // #304
	1447	samplerOption->mode.length = length;
	1448	samplerOption->mode.offset = offset;
	1449	SetFlag(samplerOption->flags, SAMPLER304);
	1450	break;
	1451	case NF9_FLOW_SAMPLER_RANDOM_INTERVAL: // #50 depricated - fall through
	1452	case NF_SAMPLING_INTERVAL: // #305
	1453	samplerOption->interval.length = length;
	1454	samplerOption->interval.offset = offset;
	1455	SetFlag(samplerOption->flags, SAMPLER305);
1486	1456	break;
1487	1457	}
1488	1458	offset += length;
1489	1459	}
1490	1460
1491		if ( found_sampler == 3 ) { // need all three tags
1492		dbg_printf("[%u] Sampling information found\n", exporter->info.id);
1493		InsertSamplerOffset(fs, id, offset_sampler_id, sampler_id_length, offset_sampler_mode, offset_sampler_interval);
1494		} else if ( found_std_sampling == 2 ) { // need all two tags
1495		dbg_printf("[%u] Std sampling information found. offset intervall: %u, offset algo: %u\n",
1496		exporter->info.id, offset_std_sampler_interval, offset_std_sampler_algorithm);
1497		InsertStdSamplerOffset(fs, id, offset_std_sampler_interval, offset_std_sampler_algorithm);
	1461	if ( (samplerOption->flags & SAMPLERMASK ) == SAMPLERFLAGS) {
	1462	dbg_printf("[%u] Sampler information found\n", exporter->info.id);
	1463	InsertSamplerOption(exporter, samplerOption);
	1464	} else if ( (samplerOption->flags & STDMASK ) == STDFLAGS) {
	1465	dbg_printf("[%u] Std sampling information found\n", exporter->info.id);
	1466	InsertSamplerOption(exporter, samplerOption);
1498	1467	} else {
	1468	free(samplerOption);
1499	1469	dbg_printf("[%u] No Sampling information found\n", exporter->info.id);
1500	1470	}
	1471	processed_records++;
1501	1472	dbg_printf("\n");
1502		processed_records++;
1503	1473
1504	1474	} // End of Process_v9_option_templates
1505	1475
1506
1507		static inline void Process_v9_data(exporter_v9_domain_t exporter, void data_flowset, FlowSource_t fs, input_translation_t table ){
	1476	static inline void Process_v9_data(exporterDomain_t exporter, void data_flowset, FlowSource_t fs, input_translation_t table ){
1508	1477	uint64_t start_time, end_time, sampling_rate;
1509	1478	uint32_t size_left;
1510	1479	uint8_t in, out;

1525	1494
1526	1495	// Check if sampling is announced
1527	1496	if ( table->sampler_offset && exporter->sampler ) {
1528		generic_sampler_t *sampler = exporter->sampler;
	1497	sampler_t *sampler = exporter->sampler;
1529	1498	uint32_t sampler_id;
1530	1499	if ( table->sampler_size == 2 ) {
1531	1500	sampler_id = Get_val16((void *)&in[table->sampler_offset]);

1549	1518	}
1550	1519
1551	1520	} else {
1552		generic_sampler_t *sampler = exporter->sampler;
	1521	sampler_t *sampler = exporter->sampler;
1553	1522	while ( sampler && sampler->info.id != -1 )
1554	1523	sampler = sampler->next;
1555	1524

1955	1924	master_record_t master_record;
1956	1925	memset((void *)&master_record, 0, sizeof(master_record_t));
1957	1926	ExpandRecord_v2((common_record_t *)data_record, &(table->extension_info), &(exporter->info), &master_record);
1958		format_file_block_record(&master_record, &string, 0);
	1927	flow_record_to_raw(&master_record, &string, 0);
1959	1928	printf("%s\n", string);
1960	1929	}
1961	1930

1984	1953
1985	1954	} // End of Process_v9_data
1986	1955
1987		static inline void Process_v9_option_data(exporter_v9_domain_t exporter, void data_flowset, FlowSource_t *fs) {
1988		option_offset_t *offset_table;
1989		uint32_t id;
	1956	static inline void Process_v9_option_data(exporterDomain_t exporter, void data_flowset, FlowSource_t *fs) {
	1957	samplerOption_t *samplerOption;
	1958	uint32_t tableID;
1990	1959	uint8_t *in;
1991	1960
1992		id = GET_FLOWSET_ID(data_flowset);
1993
1994		offset_table = fs->option_offset_table;
1995		while ( offset_table && offset_table->id != id )
1996		offset_table = offset_table->next;
1997
1998		if ( !offset_table ) {
	1961	tableID = GET_FLOWSET_ID(data_flowset);
	1962
	1963	samplerOption = exporter->samplerOption;
	1964	while ( samplerOption && samplerOption->tableID != tableID )
	1965	samplerOption = samplerOption->next;
	1966
	1967	if ( !samplerOption ) {
1999	1968	// should never happen - catch it anyway
2000	1969	LogError( "Process_v9: Panic! - No Offset table found! : %s line %d", __FILE__, __LINE__);
2001	1970	return;

2009	1978	// map input buffer as a byte array
2010	1979	in = (uint8_t *)(data_flowset + 4); // skip flowset header
2011	1980
2012		if ( TestFlag(offset_table->flags, HAS_SAMPLER_DATA) ) {
	1981	if ( (samplerOption->flags & SAMPLERMASK ) == SAMPLERFLAGS) {
2013	1982	int32_t id;
2014	1983	uint16_t mode;
2015	1984	uint32_t interval;
2016		if (offset_table->sampler_id_length == 2) {
2017		id = Get_val16((void *)&in[offset_table->offset_id]);
2018		} else {
2019		id = in[offset_table->offset_id];
2020		}
2021		mode = in[offset_table->offset_mode];
2022		interval = Get_val32((void *)&in[offset_table->offset_interval]);
	1985
	1986	id = Get_val(in, samplerOption->id.offset, samplerOption->id.length);
	1987	mode = Get_val(in, samplerOption->mode.offset, samplerOption->mode.length);
	1988	interval = Get_val(in, samplerOption->interval.offset, samplerOption->interval.length);
2023	1989
2024	1990	dbg_printf("Extracted Sampler data:\n");
2025	1991	dbg_printf("Sampler ID : %u\n", id);

2029	1995	InsertSampler(fs, exporter, id, mode, interval);
2030	1996	}
2031	1997
2032		if ( TestFlag(offset_table->flags, HAS_STD_SAMPLER_DATA) ) {
2033		int32_t id = -1;
2034		uint16_t mode = in[offset_table->offset_std_sampler_algorithm];
2035		uint32_t interval = Get_val32((void *)&in[offset_table->offset_std_sampler_interval]);
	1998	if ( (samplerOption->flags & STDMASK ) == STDFLAGS) {
	1999	int32_t id;
	2000	uint16_t mode;
	2001	uint32_t interval;
	2002
	2003	id = -1;
	2004	mode = Get_val(in, samplerOption->mode.offset, samplerOption->mode.length);
	2005	interval = Get_val(in, samplerOption->interval.offset, samplerOption->interval.length);
2036	2006
2037	2007	InsertSampler(fs, exporter, id, mode, interval);
2038	2008

2041	2011	dbg_printf("Sampler algorithm: %u\n", mode);
2042	2012	dbg_printf("Sampler interval : %u\n", interval);
2043	2013
2044		LogInfo( "Set std sampler: algorithm: %u, interval: %u\n",
2045		mode, interval);
2046	2014	dbg_printf("Set std sampler: algorithm: %u, interval: %u\n",
2047	2015	mode, interval);
2048	2016	}

2051	2019	} // End of Process_v9_option_data
2052	2020
2053	2021	void Process_v9(void in_buff, ssize_t in_buff_cnt, FlowSource_t fs) {
2054		exporter_v9_domain_t *exporter;
	2022	exporterDomain_t *exporter;
2055	2023	void *flowset_header;
2056	2024	netflow_v9_header_t *v9_header;
2057	2025	int64_t distance;

2190	2158	table = GetTranslationTable(exporter, flowset_id);
2191	2159	if ( table ) {
2192	2160	Process_v9_data(exporter, flowset_header, fs, table);
2193		} else if ( HasOptionTable(fs, flowset_id) ) {
	2161	} else if ( HasOptionTable(exporter, flowset_id) ) {
2194	2162	Process_v9_option_data(exporter, flowset_header, fs);
2195	2163	} else {
2196	2164	// maybe a flowset with option data

2863	2831	time_t now = time(NULL);
2864	2832
2865	2833	#ifdef DEVEL
2866		// char *string;
2867		// format_file_block_record(master_record, 1, &string, 0);
2868		// dbg_printf("%s\n", string);
	2834	char *string;
	2835	flow_record_to_raw(master_record, &string, 0);
	2836	printf("%s\n", string);
2869	2837	#endif
2870	2838
2871	2839	if ( !v9_output_header->unix_secs ) { // first time a record is added

3025	2993	} // End of Add_v9_output_record
3026	2994
3027	2995
3028		static void InsertSampler( FlowSource_t fs, exporter_v9_domain_t exporter, int32_t id, uint16_t mode, uint32_t interval) {
3029		generic_sampler_t *sampler;
	2996	static void InsertSampler( FlowSource_t fs, exporterDomain_t exporter, int32_t id, uint16_t mode, uint32_t interval) {
	2997	sampler_t *sampler;
3030	2998
3031	2999	dbg_printf("[%u] Insert Sampler: Exporter is 0x%llu\n", exporter->info.id, (long long unsigned)exporter);
3032	3000	if ( !exporter->sampler ) {
3033	3001	// no samplers so far
3034		sampler = (generic_sampler_t *)malloc(sizeof(generic_sampler_t));
	3002	sampler = (sampler_t *)malloc(sizeof(sampler_t));
3035	3003	if ( !sampler ) {
3036	3004	LogError( "Process_v9: Panic! malloc(): %s line %d: %s", __FILE__, __LINE__, strerror (errno));
3037	3005	return;

3078	3046	// test for end of chain
3079	3047	if ( sampler->next == NULL ) {
3080	3048	// end of sampler chain - insert new sampler
3081		sampler->next = (generic_sampler_t *)malloc(sizeof(generic_sampler_t));
	3049	sampler->next = (sampler_t *)malloc(sizeof(sampler_t));
3082	3050	if ( !sampler->next ) {
3083	3051	LogError( "Process_v9: Panic! malloc(): %s line %d: %s", __FILE__, __LINE__, strerror (errno));
3084	3052	return;

-4

bin/netflow_v9.h less more

0	0	/*
1		* Copyright (c) 2017, Peter Haag
2		* Copyright (c) 2014, Peter Haag
3		* Copyright (c) 2009, Peter Haag
	1	* Copyright (c) 2009-2020, Peter Haag
4	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
5	3	* All rights reserved.
6	4	*

256	254
257	255	#define NF9_BGP_ADJ_NEXT_AS 128
258	256	#define NF9_BGP_ADJ_PREV_AS 129
	257	#define NF9_dot1qVlanId 243
	258	#define NF9_postDot1qVlanId 254
259	259
260	260	// CISCO ASA NSEL extension - Network Security Event Logging
261	261	#define NF_F_FLOW_BYTES 85

294	294	#define NF_F_INITIATORPACKETS 298
295	295	#define NF_F_RESPONDERPACKETS 299
296	296
	297	// Zone-Based Firewall Logging
	298	#define NF_FW_CTS_SRC_SGT 34000
	299
297	300	// Cisco ASR 1000 series NEL extension - Nat Event Logging
298	301	#define NF_N_NAT_EVENT 230
299	302	#define NF_N_INGRESS_VRFID 234
300	303	#define NF_N_EGRESS_VRFID 235
	304
301	305	#define NF_F_XLATE_PORT_BLOCK_START 361
302	306	#define NF_F_XLATE_PORT_BLOCK_END 362
303	307	#define NF_F_XLATE_PORT_BLOCK_STEP 363

312	316	#define NF9_NPROBE_APPL_LATENCY_USEC 57559
313	317
314	318	/* prototypes */
315		int Init_v9(void);
	319	int Init_v9(int v, uint32_t sampling, uint32_t overwrite);
316	320
317	321	void Process_v9(void in_buff, ssize_t in_buff_cnt, FlowSource_t fs);
318	322

-2925

~~bin/nf_common.c~~ less more

0		/*
1		* Copyright (c) 2009-2018, Peter Haag
2		* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
3		* All rights reserved.
4		*
5		* Redistribution and use in source and binary forms, with or without
6		* modification, are permitted provided that the following conditions are met:
7		*
8		* * Redistributions of source code must retain the above copyright notice,
9		* this list of conditions and the following disclaimer.
10		* * Redistributions in binary form must reproduce the above copyright notice,
11		* this list of conditions and the following disclaimer in the documentation
12		* and/or other materials provided with the distribution.
13		* * Neither the name of the author nor the names of its contributors may be
14		* used to endorse or promote products derived from this software without
15		* specific prior written permission.
16		*
17		* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
18		* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
19		* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
20		* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
21		* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
22		* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
23		* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
24		* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
25		* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
26		* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
27		* POSSIBILITY OF SUCH DAMAGE.
28		*
29		*/
30
31		#include "config.h"
32
33		#include <stdio.h>
34		#include <stddef.h>
35		#include <sys/types.h>
36		#include <sys/socket.h>
37		#include <netinet/in.h>
38		#include <arpa/inet.h>
39		#include <time.h>
40		#include <string.h>
41		#include <ctype.h>
42		#include <stdlib.h>
43		#include <errno.h>
44
45		#ifdef HAVE_STDINT_H
46		#include <stdint.h>
47		#endif
48
49		#include "nffile.h"
50		#include "nf_common.h"
51		#include "util.h"
52
53		typedef void (string_function_t)(master_record_t , char *);
54
55		static struct token_list_s {
56		string_function_t string_function; // function generation output string
57		char *string_buffer; // buffer for output string
58		} *token_list;
59
60		static int max_token_index = 0;
61		static int token_index = 0;
62
63		#define BLOCK_SIZE 32
64
65		static char **format_list; // ordered list of all individual strings formating the output line
66		static int max_format_index = 0;
67		static int format_index = 0;
68
69		static int do_tag = 0;
70		static int long_v6 = 0;
71		static int scale = 1;
72		static double duration;
73
74		#ifdef NSEL
75		static char *NSEL_event_string[6] = {
76		"IGNORE", "CREATE", "DELETE", "DENIED", "ALERT", "UPDATE"
77		};
78
79		static char *NEL_event_string[3] = {
80		"INVALID", "ADD", "DELETE"
81		};
82		#endif
83
84		#define STRINGSIZE 10240
85		#define IP_STRING_LEN (INET6_ADDRSTRLEN)
86
87		static char header_string[STRINGSIZE];
88		static char data_string[STRINGSIZE];
89
90		// tag
91		static char tag_string[2];
92
93		/* prototypes */
94		static inline void ICMP_Port_decode(master_record_t r, char string);
95
96		static void InitFormatParser(void);
97
98		static void AddToken(int index);
99
100		static void AddString(char *string);
101
102		static void String_FlowFlags(master_record_t r, char string);
103
104		static void String_FirstSeen(master_record_t r, char string);
105
106		static void String_LastSeen(master_record_t r, char string);
107
108		static void String_Received(master_record_t r, char string);
109
110		static void String_FirstSeenRaw(master_record_t r, char string);
111
112		static void String_LastSeenRaw(master_record_t r, char string);
113
114		static void String_ReceivedRaw(master_record_t r, char string);
115
116		static void String_Duration(master_record_t r, char string);
117
118		static void String_Protocol(master_record_t r, char string);
119
120		static void String_SrcAddr(master_record_t r, char string);
121
122		static void String_DstAddr(master_record_t r, char string);
123
124		static void String_SrcAddrPort(master_record_t r, char string);
125
126		static void String_DstAddrPort(master_record_t r, char string);
127
128		static void String_SrcNet(master_record_t r, char string);
129
130		static void String_DstNet(master_record_t r, char string);
131
132		static void String_NextHop(master_record_t r, char string);
133
134		static void String_BGPNextHop(master_record_t r, char string);
135
136		static void String_RouterIP(master_record_t r, char string);
137
138		static void String_SrcPort(master_record_t r, char string);
139
140		static void String_DstPort(master_record_t r, char string);
141
142		static void String_ICMP_code(master_record_t r, char string);
143
144		static void String_ICMP_type(master_record_t r, char string);
145
146		static void String_SrcAS(master_record_t r, char string);
147
148		static void String_DstAS(master_record_t r, char string);
149
150		static void String_NextAS(master_record_t r, char string);
151
152		static void String_PrevAS(master_record_t r, char string);
153
154		static void String_Input(master_record_t r, char string);
155
156		static void String_Output(master_record_t r, char string);
157
158		static void String_InPackets(master_record_t r, char string);
159
160		static void String_OutPackets(master_record_t r, char string);
161
162		static void String_InBytes(master_record_t r, char string);
163
164		static void String_OutBytes(master_record_t r, char string);
165
166		static void String_Flows(master_record_t r, char string);
167
168		static void String_Tos(master_record_t r, char string);
169
170		static void String_Dir(master_record_t r, char string);
171
172		static void String_SrcTos(master_record_t r, char string);
173
174		static void String_DstTos(master_record_t r, char string);
175
176		static void String_SrcMask(master_record_t r, char string);
177
178		static void String_DstMask(master_record_t r, char string);
179
180		static void String_SrcVlan(master_record_t r, char string);
181
182		static void String_DstVlan(master_record_t r, char string);
183
184		static void String_FwdStatus(master_record_t r, char string);
185
186		static void String_Flags(master_record_t r, char string);
187
188		static void String_InSrcMac(master_record_t r, char string);
189
190		static void String_OutDstMac(master_record_t r, char string);
191
192		static void String_InDstMac(master_record_t r, char string);
193
194		static void String_OutSrcMac(master_record_t r, char string);
195
196		static void String_MPLS_1(master_record_t r, char string);
197
198		static void String_MPLS_2(master_record_t r, char string);
199
200		static void String_MPLS_3(master_record_t r, char string);
201
202		static void String_MPLS_4(master_record_t r, char string);
203
204		static void String_MPLS_5(master_record_t r, char string);
205
206		static void String_MPLS_6(master_record_t r, char string);
207
208		static void String_MPLS_7(master_record_t r, char string);
209
210		static void String_MPLS_8(master_record_t r, char string);
211
212		static void String_MPLS_9(master_record_t r, char string);
213
214		static void String_MPLS_10(master_record_t r, char string);
215
216		static void String_MPLSs(master_record_t r, char string);
217
218		static void String_Engine(master_record_t r, char string);
219
220		static void String_Label(master_record_t r, char string);
221
222		static void String_ClientLatency(master_record_t r, char string);
223
224		static void String_ServerLatency(master_record_t r, char string);
225
226		static void String_AppLatency(master_record_t r, char string);
227
228		static void String_bps(master_record_t r, char string);
229
230		static void String_pps(master_record_t r, char string);
231
232		static void String_bpp(master_record_t r, char string);
233
234		static void String_ExpSysID(master_record_t r, char string);
235
236		#ifdef NSEL
237		static void String_EventTime(master_record_t r, char string);
238
239		static void String_nfc(master_record_t r, char string);
240
241		static void String_evt(master_record_t r, char string);
242
243		static void String_xevt(master_record_t r, char string);
244
245		static void String_msec(master_record_t r, char string);
246
247		static void String_iacl(master_record_t r, char string);
248
249		static void String_eacl(master_record_t r, char string);
250
251		static void String_xlateSrcAddr(master_record_t r, char string);
252
253		static void String_xlateDstAddr(master_record_t r, char string);
254
255		static void String_xlateSrcPort(master_record_t r, char string);
256
257		static void String_xlateDstPort(master_record_t r, char string);
258
259		static void String_xlateSrcAddrPort(master_record_t r, char string);
260
261		static void String_xlateDstAddrPort(master_record_t r, char string);
262
263		static void String_userName(master_record_t r, char string);
264
265		static void String_ivrf(master_record_t r, char string);
266
267		static void String_evrf(master_record_t r, char string);
268
269		static void String_PortBlockStart(master_record_t r, char string);
270
271		static void String_PortBlockEnd(master_record_t r, char string);
272
273		static void String_PortBlockStep(master_record_t r, char string);
274
275		static void String_PortBlockSize(master_record_t r, char string);
276
277		#endif
278
279		static struct format_token_list_s {
280		char *token; // token
281		int is_address; // is an IP address
282		char *header; // header line description
283		string_function_t string_function; // function generation output string
284		} format_token_list[] = {
285		{ "%ff", 0, "Flow Flags", String_FlowFlags }, // flow flags in hex
286		{ "%tfs", 0, "Date first seen ", String_FirstSeen }, // Start Time - first seen
287		{ "%ts", 0, "Date first seen ", String_FirstSeen }, // Start Time - first seen
288		{ "%tsr", 0, "Date first seen (raw) ", String_FirstSeenRaw }, // Start Time - first seen, seconds
289		{ "%te", 0, "Date last seen ", String_LastSeen }, // End Time - last seen
290		{ "%ter", 0, "Date last seen (raw) ", String_LastSeenRaw }, // End Time - first seen, seconds
291		{ "%tr", 0, "Date flow received ", String_Received }, // Received Time
292		{ "%trr", 0, "Date flow received (raw) ", String_ReceivedRaw }, // Received Time, seconds
293		{ "%td", 0, " Duration", String_Duration }, // Duration
294		{ "%exp", 0, "Exp ID", String_ExpSysID }, // Exporter SysID
295		{ "%pr", 0, "Proto", String_Protocol }, // Protocol
296		{ "%sa", 1, " Src IP Addr", String_SrcAddr }, // Source Address
297		{ "%da", 1, " Dst IP Addr", String_DstAddr }, // Destination Address
298		{ "%sn", 1, " Src Network", String_SrcNet }, // Source Address applied source netmask
299		{ "%dn", 1, " Dst Network", String_DstNet }, // Destination Address applied source netmask
300		{ "%nh", 1, " Next-hop IP", String_NextHop }, // Next-hop IP Address
301		{ "%nhb", 1, " BGP next-hop IP", String_BGPNextHop }, // BGP Next-hop IP Address
302		{ "%ra", 1, " Router IP", String_RouterIP }, // Router IP Address
303		{ "%sap", 1, " Src IP Addr:Port ", String_SrcAddrPort }, // Source Address:Port
304		{ "%dap", 1, " Dst IP Addr:Port ", String_DstAddrPort }, // Destination Address:Port
305		{ "%sp", 0, "Src Pt", String_SrcPort }, // Source Port
306		{ "%dp", 0, "Dst Pt", String_DstPort }, // Destination Port
307		{ "%it", 0, "ICMP-T", String_ICMP_type }, // ICMP type
308		{ "%ic", 0, "ICMP-C", String_ICMP_code }, // ICMP code
309		{ "%sas", 0, "Src AS", String_SrcAS }, // Source AS
310		{ "%das", 0, "Dst AS", String_DstAS }, // Destination AS
311		{ "%nas", 0, "Next AS", String_NextAS }, // Next AS
312		{ "%pas", 0, "Prev AS", String_PrevAS }, // Previous AS
313		{ "%in", 0, " Input", String_Input }, // Input Interface num
314		{ "%out", 0, "Output", String_Output }, // Output Interface num
315		{ "%pkt", 0, " Packets", String_InPackets }, // Packets - default input - compat
316		{ "%ipkt", 0, " In Pkt", String_InPackets }, // In Packets
317		{ "%opkt", 0, " Out Pkt", String_OutPackets }, // Out Packets
318		{ "%byt", 0, " Bytes", String_InBytes }, // Bytes - default input - compat
319		{ "%ibyt", 0, " In Byte", String_InBytes }, // In Bytes
320		{ "%obyt", 0, "Out Byte", String_OutBytes }, // In Bytes
321		{ "%fl", 0, "Flows", String_Flows }, // Flows
322		{ "%flg", 0, " Flags", String_Flags }, // TCP Flags
323		{ "%tos", 0, "Tos", String_Tos }, // Tos - compat
324		{ "%stos", 0, "STos", String_SrcTos }, // Tos - Src tos
325		{ "%dtos", 0, "DTos", String_DstTos }, // Tos - Dst tos
326		{ "%dir", 0, "Dir", String_Dir }, // Direction: ingress, egress
327		{ "%smk", 0, "SMask", String_SrcMask }, // Src mask
328		{ "%dmk", 0, "DMask", String_DstMask }, // Dst mask
329		{ "%fwd", 0, "Fwd", String_FwdStatus }, // Forwarding Status
330		{ "%svln", 0, "SVlan", String_SrcVlan }, // Src Vlan
331		{ "%dvln", 0, "DVlan", String_DstVlan }, // Dst Vlan
332		{ "%ismc", 0, " In src MAC Addr", String_InSrcMac }, // Input Src Mac Addr
333		{ "%odmc", 0, " Out dst MAC Addr", String_OutDstMac }, // Output Dst Mac Addr
334		{ "%idmc", 0, " In dst MAC Addr", String_InDstMac }, // Input Dst Mac Addr
335		{ "%osmc", 0, " Out src MAC Addr", String_OutSrcMac }, // Output Src Mac Addr
336		{ "%mpls1", 0, " MPLS lbl 1 ", String_MPLS_1 }, // MPLS Label 1
337		{ "%mpls2", 0, " MPLS lbl 2 ", String_MPLS_2 }, // MPLS Label 2
338		{ "%mpls3", 0, " MPLS lbl 3 ", String_MPLS_3 }, // MPLS Label 3
339		{ "%mpls4", 0, " MPLS lbl 4 ", String_MPLS_4 }, // MPLS Label 4
340		{ "%mpls5", 0, " MPLS lbl 5 ", String_MPLS_5 }, // MPLS Label 5
341		{ "%mpls6", 0, " MPLS lbl 6 ", String_MPLS_6 }, // MPLS Label 6
342		{ "%mpls7", 0, " MPLS lbl 7 ", String_MPLS_7 }, // MPLS Label 7
343		{ "%mpls8", 0, " MPLS lbl 8 ", String_MPLS_8 }, // MPLS Label 8
344		{ "%mpls9", 0, " MPLS lbl 9 ", String_MPLS_9 }, // MPLS Label 9
345		{ "%mpls10", 0, " MPLS lbl 10", String_MPLS_10 }, // MPLS Label 10
346		{ "%mpls", 0, " MPLS labels 1-10 ", String_MPLSs }, // All MPLS labels
347		//
348		{ "%bps", 0, " bps", String_bps }, // bps - bits per second
349		{ "%pps", 0, " pps", String_pps }, // pps - packets per second
350		{ "%bpp", 0, " Bpp", String_bpp }, // bpp - Bytes per package
351		{ "%eng", 0, " engine", String_Engine }, // Engine Type/ID
352		{ "%lbl", 0, " label", String_Label }, // Flow Label
353
354		#ifdef NSEL
355		// NSEL specifics
356		{ "%nfc", 0, " Conn-ID", String_nfc }, // NSEL connection ID
357		{ "%tevt", 0, "Event time ",String_EventTime }, // NSEL Flow start time
358		{ "%evt", 0, " Event", String_evt }, // NSEL event
359		{ "%xevt", 0, " XEvent", String_xevt }, // NSEL xevent
360		{ "%msec", 0, " Event Time", String_msec}, // NSEL event time in msec
361		{ "%iacl", 0, "Ingress ACL ", String_iacl}, // NSEL ingress ACL
362		{ "%eacl", 0, "Egress ACL ", String_eacl}, // NSEL egress ACL
363		{ "%xsa", 0, " X-late Src IP", String_xlateSrcAddr}, // NSEL XLATE src IP
364		{ "%xda", 0, " X-late Dst IP", String_xlateDstAddr}, // NSEL XLATE dst IP
365		{ "%xsp", 0, "XsPort", String_xlateSrcPort}, // NSEL XLATE src port
366		{ "%xdp", 0, "XdPort", String_xlateDstPort}, // NSEL SLATE dst port
367		{ "%xsap", 1, " X-Src IP Addr:Port ", String_xlateSrcAddrPort }, // Xlate Source Address:Port
368		{ "%xdap", 1, " X-Dst IP Addr:Port ", String_xlateDstAddrPort }, // Xlate Destination Address:Port
369		{ "%uname", 0, "UserName", String_userName}, // NSEL user name
370
371		// NEL
372		// for v.1.6.10 compatibility, keep NEL specific addr/port format tokens
373		{ "%nevt", 0, " Event", String_evt }, // NAT event
374		{ "%vrf", 0, " I-VRF-ID", String_ivrf }, // NAT ivrf ID - compatible
375		{ "%ivrf", 0, " I-VRF-ID", String_ivrf }, // NAT ivrf ID
376		{ "%evrf", 0, " E-VRF-ID", String_evrf }, // NAT ivrf ID
377		{ "%nsa", 0, " X-late Src IP", String_xlateSrcAddr}, // NAT XLATE src IP
378		{ "%nda", 0, " X-late Dst IP", String_xlateDstAddr}, // NAT XLATE dst IP
379		{ "%nsp", 0, "XsPort", String_xlateSrcPort}, // NAT XLATE src port
380		{ "%ndp", 0, "XdPort", String_xlateDstPort}, // NAT SLATE dst port
381		{ "%nsap", 1, " X-Src IP Addr:Port ", String_xlateSrcAddrPort },// NAT Xlate Source Address:Port
382		{ "%ndap", 1, " X-Dst IP Addr:Port ", String_xlateDstAddrPort },// NAT Xlate Destination Address:Port
383
384		// Port block allocation
385		{ "%pbstart", 0, "Pb-Start", String_PortBlockStart}, // Port block start
386		{ "%pbend", 0, "Pb-End", String_PortBlockEnd}, // Port block end
387		{ "%pbstep", 0, "Pb-Step", String_PortBlockStep}, // Port block step
388		{ "%pbsize", 0, "Pb-Size", String_PortBlockSize}, // Port block size
389		#endif
390
391		// latency extension for nfpcapd and nprobe
392		{ "%cl", 0, "C Latency", String_ClientLatency }, // client latency
393		{ "%sl", 0, "S latency", String_ServerLatency }, // server latency
394		{ "%al", 0, "A latency", String_AppLatency }, // app latency
395
396		{ NULL, 0, NULL, NULL }
397		};
398
399		/* each of the tokens above must not generate output strings larger than this */
400		#define MAX_STRING_LENGTH 256
401
402		#define NumProtos 138
403		#define MAX_PROTO_STR 8
404		char protolist[NumProtos][MAX_PROTO_STR] = {
405		" 0", // 0 masked out - no protocol info - set to '0'
406		"ICMP ", // 1 Internet Control Message
407		"IGMP ", // 2 Internet Group Management
408		"GGP ", // 3 Gateway-to-Gateway
409		"IPIP ", // 4 IP in IP (encapsulation)
410		"ST ", // 5 Stream
411		"TCP ", // 6 Transmission Control
412		"CBT ", // 7 CBT
413		"EGP ", // 8 Exterior Gateway Protocol
414		"IGP ", // 9 any private interior gateway (used by Cisco for their IGRP)
415		"BBN ", // 10 BBN RCC Monitoring
416		"NVPII", // 11 Network Voice Protocol
417		"PUP ", // 12 PUP
418		"ARGUS", // 13 ARGUS
419		"ENCOM", // 14 EMCON
420		"XNET ", // 15 Cross Net Debugger
421		"CHAOS", // 16 Chaos
422		"UDP ", // 17 User Datagram
423		"MUX ", // 18 Multiplexing
424		"DCN ", // 19 DCN Measurement Subsystems
425		"HMP ", // 20 Host Monitoring
426		"PRM ", // 21 Packet Radio Measurement
427		"XNS ", // 22 XEROX NS IDP
428		"Trnk1", // 23 Trunk-1
429		"Trnk2", // 24 Trunk-2
430		"Leaf1", // 25 Leaf-1
431		"Leaf2", // 26 Leaf-2
432		"RDP ", // 27 Reliable Data Protocol
433		"IRTP ", // 28 Internet Reliable Transaction
434		"ISO-4", // 29 ISO Transport Protocol Class 4
435		"NETBK", // 30 Bulk Data Transfer Protocol
436		"MFESP", // 31 MFE Network Services Protocol
437		"MEINP", // 32 MERIT Internodal Protocol
438		"DCCP ", // 33 Datagram Congestion Control Protocol
439		"3PC ", // 34 Third Party Connect Protocol
440		"IDPR ", // 35 Inter-Domain Policy Routing Protocol
441		"XTP ", // 36 XTP
442		"DDP ", // 37 Datagram Delivery Protocol
443		"IDPR ", // 38 IDPR Control Message Transport Proto
444		"TP++ ", // 39 TP++ Transport Protocol
445		"IL ", // 40 IL Transport Protocol
446		"IPv6 ", // 41 IPv6
447		"SDRP ", // 42 Source Demand Routing Protocol
448		"Rte6 ", // 43 Routing Header for IPv6
449		"Frag6", // 44 Fragment Header for IPv6
450		"IDRP ", // 45 Inter-Domain Routing Protocol
451		"RSVP ", // 46 Reservation Protocol
452		"GRE ", // 47 General Routing Encapsulation
453		"MHRP ", // 48 Mobile Host Routing Protocol
454		"BNA ", // 49 BNA
455		"ESP ", // 50 Encap Security Payload
456		"AH ", // 51 Authentication Header
457		"INLSP", // 52 Integrated Net Layer Security TUBA
458		"SWIPE", // 53 IP with Encryption
459		"NARP ", // 54 NBMA Address Resolution Protocol
460		"MOBIL", // 55 IP Mobility
461		"TLSP ", // 56 Transport Layer Security Protocol
462		"SKIP ", // 57 SKIP
463		"ICMP6", // 58 ICMP for IPv6
464		"NOHE6", // 59 No Next Header for IPv6
465		"OPTS6", // 60 Destination Options for IPv6
466		"HOST ", // 61 any host internal protocol
467		"CFTP ", // 62 CFTP
468		"NET ", // 63 any local network
469		"SATNT", // 64 SATNET and Backroom EXPAK
470		"KLAN ", // 65 Kryptolan
471		"RVD ", // 66 MIT Remote Virtual Disk Protocol
472		"IPPC ", // 67 Internet Pluribus Packet Core
473		"FS ", // 68 any distributed file system
474		"SATM ", // 69 SATNET Monitoring
475		"VISA ", // 70 VISA Protocol
476		"IPCV ", // 71 Internet Packet Core Utility
477		"CPNX ", // 72 Computer Protocol Network Executive
478		"CPHB ", // 73 Computer Protocol Heart Beat
479		"WSN ", // 74 Wang Span Network
480		"PVP ", // 75 Packet Video Protocol
481		"BSATM", // 76 Backroom SATNET Monitoring
482		"SUNND", // 77 SUN ND PROTOCOL-Temporary
483		"WBMON", // 78 WIDEBAND Monitoring
484		"WBEXP", // 79 WIDEBAND EXPAK
485		"ISOIP", // 80 ISO Internet Protocol
486		"VMTP ", // 81 VMTP
487		"SVMTP", // 82 SECURE-VMTP
488		"VINES", // 83 VINES
489		"TTP ", // 84 TTP
490		"NSIGP", // 85 NSFNET-IGP
491		"DGP ", // 86 Dissimilar Gateway Protocol
492		"TCF ", // 87 TCF
493		"EIGRP", // 88 EIGRP
494		"OSPF ", // 89 OSPFIGP
495		"S-RPC", // 90 Sprite RPC Protocol
496		"LARP ", // 91 Locus Address Resolution Protocol
497		"MTP ", // 92 Multicast Transport Protocol
498		"AX.25", // 93 AX.25 Frames
499		"IPIP ", // 94 IP-within-IP Encapsulation Protocol
500		"MICP ", // 95 Mobile Internetworking Control Protocol
501		"SCCSP", // 96 Semaphore Communications Sec. Protocol
502		"ETHIP", // 97 Ethernet-within-IP Encapsulation
503		"ENCAP", // 98 Encapsulation Header
504		"99 ", // 99 any private encryption scheme
505		"GMTP ", // 100 GMTP
506		"IFMP ", // 101 Ipsilon Flow Management Protocol
507		"PNNI ", // 102 PNNI over IP
508		"PIM ", // 103 Protocol Independent Multicast
509		"ARIS ", // 104 ARIS
510		"SCPS ", // 105 SCPS
511		"QNX ", // 106 QNX
512		"A/N ", // 107 Active Networks
513		"IPcmp", // 108 IP Payload Compression Protocol
514		"SNP ", // 109 Sitara Networks Protocol
515		"CpqPP", // 110 Compaq Peer Protocol
516		"IPXIP", // 111 IPX in IP
517		"VRRP ", // 112 Virtual Router Redundancy Protocol
518		"PGM ", // 113 PGM Reliable Transport Protocol
519		"0hop ", // 114 any 0-hop protocol
520		"L2TP ", // 115 Layer Two Tunneling Protocol
521		"DDX ", // 116 D-II Data Exchange (DDX)
522		"IATP ", // 117 Interactive Agent Transfer Protocol
523		"STP ", // 118 Schedule Transfer Protocol
524		"SRP ", // 119 SpectraLink Radio Protocol
525		"UTI ", // 120 UTI
526		"SMP ", // 121 Simple Message Protocol
527		"SM ", // 122 SM
528		"PTP ", // 123 Performance Transparency Protocol
529		"ISIS4", // 124 ISIS over IPv4
530		"FIRE ", // 125 FIRE
531		"CRTP ", // 126 Combat Radio Transport Protocol
532		"CRUDP", // 127 Combat Radio User Datagram
533		"128 ", // 128 SSCOPMCE
534		"IPLT ", // 129 IPLP
535		"SPS ", // 130 Secure Packet Shield
536		"PIPE ", // 131 Private IP Encapsulation within IP
537		"SCTP ", // 132 Stream Control Transmission Protocol
538		"FC ", // 133 Fibre Channel
539		"134 ", // 134 RSVP-E2E-IGNORE
540		"MHEAD", // 135 Mobility Header
541		"UDP-L", // 136 UDPLite
542		"MPLS " // 137 MPLS-in-IP
543		};
544
545		static struct fwd_status_def_s {
546		uint32_t id;
547		char *name;
548		} fwd_status_def_list[] = {
549		{ 0, "Ukwn"}, // Unknown
550		{ 1, "Forw"}, // Normal forwarding
551		{ 2, "Frag"}, // Fragmented
552		{ 16, "Drop"}, // Drop
553		{ 17, "DaclD"}, // Drop ACL deny
554		{ 18, "Daclp"}, // Drop ACL drop
555		{ 19, "Noroute"}, // Unroutable
556		{ 20, "Dadj"}, // Drop Adjacency
557		{ 21, "Dfrag"}, // Drop Fragmentation & DF set
558		{ 22, "Dbadh"}, // Drop Bad header checksum
559		{ 23, "Dbadtlen"}, // Drop Bad total Length
560		{ 24, "Dbadhlen"}, // Drop Bad Header Length
561		{ 25, "DbadTTL"}, // Drop bad TTL
562		{ 26, "Dpolicy"}, // Drop Policer
563		{ 27, "Dwred"}, // Drop WRED
564		{ 28, "Drpf"}, // Drop RPF
565		{ 29, "Dforus"}, // Drop For us
566		{ 30, "DbadOf"}, // Drop Bad output interface
567		{ 31, "Dhw"}, // Drop Hardware
568		{ 128, "Term"}, // Terminate
569		{ 129, "Tadj"}, // Terminate Punt Adjacency
570		{ 130, "TincAdj"}, // Terminate Incomplete Adjacency
571		{ 131, "Tforus"}, // Terminate For us
572		{ 0, NULL} // Last entry
573		};
574
575		static char **fwd_status = NULL;
576
577		#include "applybits_inline.c"
578
579		/* functions */
580
581		int InitSymbols(void) {
582		int i;
583
584		// already initialised?
585		if ( fwd_status )
586		return 1;
587
588		// fill fwd status cache table
589		fwd_status = ( char *)calloc(256, sizeof(char ));
590		if ( !fwd_status ) {
591		fprintf(stderr, "Process_v9: Panic! malloc(): %s line %d: %s", __FILE__, __LINE__, strerror (errno));
592		return 0;
593		}
594		i=0;
595		while ( fwd_status_def_list[i].name ) {
596		uint32_t j = fwd_status_def_list[i].id;
597		fwd_status[j] = fwd_status_def_list[i].name;
598		i++;
599		}
600		return 1;
601
602		} // End of InitSymbols
603
604		void Setv6Mode(int mode) {
605		long_v6 += mode;
606		}
607
608		int Getv6Mode(void) {
609		return long_v6;
610		}
611
612		void Proto_string(uint8_t protonum, char *protostr) {
613
614		if ( protonum >= NumProtos \|\| !scale ) {
615		snprintf(protostr,16,"%-5i", protonum );
616		} else {
617		strncpy(protostr, protolist[protonum], 16);
618		}
619
620		} // End of Proto_string
621
622		int Proto_num(char *protostr) {
623		int i, len;
624
625		if ( (len = strlen(protostr)) >= 6 )
626		return -1;
627
628		for ( i=0; i<NumProtos; i++ ) {
629		if ( strncasecmp(protostr,protolist[i], len) == 0 &&
630		( protolist[i][len] == 0 \|\| protolist[i][len] == ' ') )
631		return i;
632		}
633
634		return -1;
635
636		} // End of Proto_num
637
638		uint32_t Get_fwd_status_id(char *status) {
639		int i;
640
641		i = 0;
642		while ( i < 256 ) {
643		if ( fwd_status[i] && strcasecmp(fwd_status[i], status) == 0 )
644		return i;
645		i++;
646		}
647		return 256;
648
649		} // End of Get_fwd_status_id
650
651		char *Get_fwd_status_name(uint32_t id) {
652
653		return id < 256 ? fwd_status[id] : NULL;
654
655		} // End of Get_fwd_status_name
656
657		void format_file_block_header(void header, char *s, int tag) {
658		data_block_header_t h = (data_block_header_t )header;
659
660		snprintf(data_string,STRINGSIZE-1 ,""
661		"File Block Header: \n"
662		" NumBlocks = %10u\n"
663		" Size = %10u\n"
664		" id = %10u\n",
665		h->NumRecords,
666		h->size,
667		h->id);
668		*s = data_string;
669
670		} // End of format_file_block_header
671
672		void format_file_block_record(void record, char *s, int tag) {
673		char *_s, as[IP_STRING_LEN], ds[IP_STRING_LEN], datestr1[64], datestr2[64], datestr3[64], flags_str[16];
674		char s_snet[IP_STRING_LEN], s_dnet[IP_STRING_LEN], s_proto[32];
675		int i, id;
676		ssize_t slen, _slen;
677		time_t when;
678		struct tm *ts;
679		master_record_t r = (master_record_t )record;
680		extension_map_t *extension_map = r->map_ref;
681
682		as[0] = 0;
683		ds[0] = 0;
684		if ( TestFlag(r->flags,FLAG_IPV6_ADDR ) != 0 ) { // IPv6
685		uint64_t snet[2];
686		uint64_t dnet[2];
687
688		// remember IPs for network
689		snet[0] = r->V6.srcaddr[0];
690		snet[1] = r->V6.srcaddr[1];
691		dnet[0] = r->V6.dstaddr[0];
692		dnet[1] = r->V6.dstaddr[1];
693		r->V6.srcaddr[0] = htonll(r->V6.srcaddr[0]);
694		r->V6.srcaddr[1] = htonll(r->V6.srcaddr[1]);
695		r->V6.dstaddr[0] = htonll(r->V6.dstaddr[0]);
696		r->V6.dstaddr[1] = htonll(r->V6.dstaddr[1]);
697		inet_ntop(AF_INET6, r->V6.srcaddr, as, sizeof(as));
698		inet_ntop(AF_INET6, r->V6.dstaddr, ds, sizeof(ds));
699		if ( ! long_v6 ) {
700		condense_v6(as);
701		condense_v6(ds);
702		}
703		if ( r->src_mask \|\| r->dst_mask) {
704		if ( r->src_mask > 64 )
705		snet[1] &= 0xffffffffffffffffLL << ( 128 - r->src_mask );
706		else {
707		snet[1] &= 0xffffffffffffffffLL << ( 64 - r->src_mask );
708		snet[1] = 0;
709		}
710		snet[0] = htonll(snet[0]);
711		snet[1] = htonll(snet[1]);
712		inet_ntop(AF_INET6, &snet, s_snet, sizeof(s_snet));
713
714		if ( r->dst_mask > 64 )
715		dnet[1] &= 0xffffffffffffffffLL << ( 128 - r->dst_mask );
716		else {
717		dnet[1] &= 0xffffffffffffffffLL << ( 64 - r->dst_mask );
718		dnet[1] = 0;
719		}
720		dnet[0] = htonll(dnet[0]);
721		dnet[1] = htonll(dnet[1]);
722		inet_ntop(AF_INET6, &dnet, s_dnet, sizeof(s_dnet));
723		if ( ! long_v6 ) {
724		condense_v6(s_snet);
725		condense_v6(s_dnet);
726		}
727
728		} else {
729		s_snet[0] = '\0';
730		s_dnet[0] = '\0';
731		}
732
733		} else { // IPv4
734		uint32_t snet, dnet;
735		snet = r->V4.srcaddr;
736		dnet = r->V4.dstaddr;
737		r->V4.srcaddr = htonl(r->V4.srcaddr);
738		r->V4.dstaddr = htonl(r->V4.dstaddr);
739		inet_ntop(AF_INET, &r->V4.srcaddr, as, sizeof(as));
740		inet_ntop(AF_INET, &r->V4.dstaddr, ds, sizeof(ds));
741		if ( r->src_mask \|\| r->dst_mask) {
742		snet &= 0xffffffffL << ( 32 - r->src_mask );
743		snet = htonl(snet);
744		inet_ntop(AF_INET, &snet, s_snet, sizeof(s_snet));
745
746		dnet &= 0xffffffffL << ( 32 - r->dst_mask );
747		dnet = htonl(dnet);
748		inet_ntop(AF_INET, &dnet, s_dnet, sizeof(s_dnet));
749		} else {
750		s_snet[0] = '\0';
751		s_dnet[0] = '\0';
752		}
753		}
754		as[IP_STRING_LEN-1] = 0;
755		ds[IP_STRING_LEN-1] = 0;
756
757		when = r->first;
758		ts = localtime(&when);
759		strftime(datestr1, 63, "%Y-%m-%d %H:%M:%S", ts);
760
761		when = r->last;
762		ts = localtime(&when);
763		strftime(datestr2, 63, "%Y-%m-%d %H:%M:%S", ts);
764
765		String_Flags(record, flags_str);
766
767		_s = data_string;
768		slen = STRINGSIZE;
769		snprintf(_s, slen-1, "\n"
770		"Flow Record: \n"
771		" Flags = 0x%.2x %s, %s\n"
772		" label = %16s\n"
773		" export sysid = %5u\n"
774		" size = %5u\n"
775		" first = %10u [%s]\n"
776		" last = %10u [%s]\n"
777		" msec_first = %5u\n"
778		" msec_last = %5u\n"
779		" src addr = %16s\n"
780		" dst addr = %16s\n"
781		,
782		r->flags, TestFlag(r->flags, FLAG_EVENT) ? "EVENT" : "FLOW",
783		TestFlag(r->flags, FLAG_SAMPLED) ? "Sampled" : "Unsampled",
784		r->label ? r->label : "<none>",
785		r->exporter_sysid, r->size, r->first,
786		datestr1, r->last, datestr2, r->msec_first, r->msec_last,
787		as, ds );
788
789		_slen = strlen(data_string);
790		_s = data_string + _slen;
791		slen = STRINGSIZE - _slen;
792
793		if ( r->prot == IPPROTO_ICMP \|\| r->prot == IPPROTO_ICMPV6 ) { // ICMP
794		snprintf(_s, slen-1,
795		" ICMP = %2u.%-2u type.code\n",
796		r->icmp_type, r->icmp_code);
797		} else {
798		snprintf(_s, slen-1,
799		" src port = %5u\n"
800		" dst port = %5u\n",
801		r->srcport, r->dstport);
802		}
803
804		_slen = strlen(data_string);
805		_s = data_string + _slen;
806		slen = STRINGSIZE - _slen;
807
808		Proto_string(r->prot, s_proto);
809
810		snprintf(_s, slen-1,
811		" fwd status = %3u\n"
812		" tcp flags = 0x%.2x %s\n"
813		" proto = %3u %s\n"
814		" (src)tos = %3u\n"
815		" (in)packets = %10llu\n"
816		" (in)bytes = %10llu\n",
817		r->fwd_status, r->tcp_flags, flags_str, r->prot, s_proto, r->tos,
818		(unsigned long long)r->dPkts, (unsigned long long)r->dOctets);
819
820		_slen = strlen(data_string);
821		_s = data_string + _slen;
822		slen = STRINGSIZE - _slen;
823
824		i = 0;
825		while ( (id = extension_map->ex_id[i]) != 0 ) {
826		if ( slen <= 20 ) {
827		fprintf(stderr, "String too short! Missing record data!\n");
828		data_string[STRINGSIZE-1] = 0;
829		*s = data_string;
830		}
831		switch(id) {
832		case EX_IO_SNMP_2:
833		case EX_IO_SNMP_4:
834		snprintf(_s, slen-1,
835		" input = %5u\n"
836		" output = %5u\n"
837		, r->input, r->output);
838		_slen = strlen(data_string);
839		_s = data_string + _slen;
840		slen = STRINGSIZE - _slen;
841		break;
842		case EX_AS_2:
843		case EX_AS_4:
844		snprintf(_s, slen-1,
845		" src as = %5u\n"
846		" dst as = %5u\n"
847		, r->srcas, r->dstas);
848		_slen = strlen(data_string);
849		_s = data_string + _slen;
850		slen = STRINGSIZE - _slen;
851		break;
852		case EX_BGPADJ:
853		snprintf(_s, slen-1,
854		" next as = %5u\n"
855		" prev as = %5u\n"
856		, r->bgpNextAdjacentAS, r->bgpPrevAdjacentAS);
857		_slen = strlen(data_string);
858		_s = data_string + _slen;
859		slen = STRINGSIZE - _slen;
860		break;
861		case EX_MULIPLE:
862		snprintf(_s, slen-1,
863		" src mask = %5u %s/%u\n"
864		" dst mask = %5u %s/%u\n"
865		" dst tos = %3u\n"
866		" direction = %3u\n"
867		, r->src_mask, s_snet, r->src_mask, r->dst_mask, s_dnet, r->dst_mask, r->dst_tos, r->dir );
868		_slen = strlen(data_string);
869		_s = data_string + _slen;
870		slen = STRINGSIZE - _slen;
871		break;
872		case EX_NEXT_HOP_v4:
873		case EX_NEXT_HOP_v6:
874		if ( (r->flags & FLAG_IPV6_NH ) != 0 ) { // IPv6
875		as[0] = 0;
876		r->ip_nexthop.V6[0] = htonll(r->ip_nexthop.V6[0]);
877		r->ip_nexthop.V6[1] = htonll(r->ip_nexthop.V6[1]);
878		inet_ntop(AF_INET6, r->ip_nexthop.V6, as, sizeof(as));
879		if ( ! long_v6 ) {
880		condense_v6(as);
881		condense_v6(ds);
882		}
883		as[IP_STRING_LEN-1] = 0;
884		} else {
885		as[0] = 0;
886		r->ip_nexthop.V4 = htonl(r->ip_nexthop.V4);
887		inet_ntop(AF_INET, &r->ip_nexthop.V4, as, sizeof(as));
888		as[IP_STRING_LEN-1] = 0;
889		}
890
891		snprintf(_s, slen-1,
892		" ip next hop = %16s\n"
893		, as);
894		_slen = strlen(data_string);
895		_s = data_string + _slen;
896		slen = STRINGSIZE - _slen;
897		break;
898		case EX_NEXT_HOP_BGP_v4:
899		case EX_NEXT_HOP_BGP_v6:
900		if ( (r->flags & FLAG_IPV6_NHB ) != 0 ) { // IPv6
901		as[0] = 0;
902		r->bgp_nexthop.V6[0] = htonll(r->bgp_nexthop.V6[0]);
903		r->bgp_nexthop.V6[1] = htonll(r->bgp_nexthop.V6[1]);
904		inet_ntop(AF_INET6, r->ip_nexthop.V6, as, sizeof(as));
905		if ( ! long_v6 ) {
906		condense_v6(as);
907		condense_v6(ds);
908		}
909		as[IP_STRING_LEN-1] = 0;
910		} else {
911		as[0] = 0;
912		r->bgp_nexthop.V4 = htonl(r->bgp_nexthop.V4);
913		inet_ntop(AF_INET, &r->bgp_nexthop.V4, as, sizeof(as));
914		as[IP_STRING_LEN-1] = 0;
915		}
916		snprintf(_s, slen-1,
917		" bgp next hop = %16s\n"
918		, as);
919		_slen = strlen(data_string);
920		_s = data_string + _slen;
921		slen = STRINGSIZE - _slen;
922		break;
923		case EX_VLAN:
924		snprintf(_s, slen-1,
925		" src vlan = %5u\n"
926		" dst vlan = %5u\n"
927		, r->src_vlan, r->dst_vlan);
928		_slen = strlen(data_string);
929		_s = data_string + _slen;
930		slen = STRINGSIZE - _slen;
931		break;
932		case EX_OUT_PKG_4:
933		case EX_OUT_PKG_8:
934		snprintf(_s, slen-1,
935		" out packets = %10llu\n"
936		, (long long unsigned)r->out_pkts);
937		_slen = strlen(data_string);
938		_s = data_string + _slen;
939		slen = STRINGSIZE - _slen;
940		break;
941		case EX_OUT_BYTES_4:
942		case EX_OUT_BYTES_8:
943		snprintf(_s, slen-1,
944		" out bytes = %10llu\n"
945		, (long long unsigned)r->out_bytes);
946		_slen = strlen(data_string);
947		_s = data_string + _slen;
948		slen = STRINGSIZE - _slen;
949		break;
950		case EX_AGGR_FLOWS_4:
951		case EX_AGGR_FLOWS_8:
952		snprintf(_s, slen-1,
953		" aggr flows = %10llu\n"
954		, (long long unsigned)r->aggr_flows);
955		_slen = strlen(data_string);
956		_s = data_string + _slen;
957		slen = STRINGSIZE - _slen;
958		break;
959		case EX_MAC_1: {
960		int i;
961		uint8_t mac1[6], mac2[6];
962
963		for ( i=0; i<6; i++ ) {
964		mac1[i] = (r->in_src_mac >> ( i*8 )) & 0xFF;
965		}
966		for ( i=0; i<6; i++ ) {
967		mac2[i] = (r->out_dst_mac >> ( i*8 )) & 0xFF;
968		}
969
970		snprintf(_s, slen-1,
971		" in src mac = %.2x:%.2x:%.2x:%.2x:%.2x:%.2x\n"
972		" out dst mac = %.2x:%.2x:%.2x:%.2x:%.2x:%.2x\n"
973		, mac1[5], mac1[4], mac1[3], mac1[2], mac1[1], mac1[0], mac2[5], mac2[4], mac2[3], mac2[2], mac2[1], mac2[0] );
974		_slen = strlen(data_string);
975		_s = data_string + _slen;
976		slen = STRINGSIZE - _slen;
977		} break;
978		case EX_MAC_2: {
979		int i;
980		uint8_t mac1[6], mac2[6];
981
982		for ( i=0; i<6; i++ ) {
983		mac1[i] = (r->in_dst_mac >> ( i*8 )) & 0xFF;
984		}
985		for ( i=0; i<6; i++ ) {
986		mac2[i] = (r->out_src_mac >> ( i*8 )) & 0xFF;
987		}
988
989		snprintf(_s, slen-1,
990		" in dst mac = %.2x:%.2x:%.2x:%.2x:%.2x:%.2x\n"
991		" out src mac = %.2x:%.2x:%.2x:%.2x:%.2x:%.2x\n"
992		, mac1[5], mac1[4], mac1[3], mac1[2], mac1[1], mac1[0], mac2[5], mac2[4], mac2[3], mac2[2], mac2[1], mac2[0] );
993		_slen = strlen(data_string);
994		_s = data_string + _slen;
995		slen = STRINGSIZE - _slen;
996		} break;
997		case EX_MPLS: {
998		unsigned int i;
999		for ( i=0; i<10; i++ ) {
1000		snprintf(_s, slen-1,
1001		" MPLS Lbl %2u = %8u-%1u-%1u\n", i+1
1002		, r->mpls_label[i] >> 4 , (r->mpls_label[i] & 0xF ) >> 1, r->mpls_label[i] & 1 );
1003		_slen = strlen(data_string);
1004		_s = data_string + _slen;
1005		slen = STRINGSIZE - _slen;
1006		}
1007		} break;
1008		case EX_ROUTER_IP_v4:
1009		as[0] = 0;
1010		r->ip_router.V4 = htonl(r->ip_router.V4);
1011		inet_ntop(AF_INET, &r->ip_router.V4, as, sizeof(as));
1012		as[IP_STRING_LEN-1] = 0;
1013
1014		snprintf(_s, slen-1,
1015		" ip router = %16s\n"
1016		, as);
1017		_slen = strlen(data_string);
1018		_s = data_string + _slen;
1019		slen = STRINGSIZE - _slen;
1020
1021		break;
1022		case EX_ROUTER_IP_v6:
1023		as[0] = 0;
1024		r->ip_router.V6[0] = htonll(r->ip_router.V6[0]);
1025		r->ip_router.V6[1] = htonll(r->ip_router.V6[1]);
1026		inet_ntop(AF_INET6, &r->ip_router.V6, as, sizeof(as));
1027		if ( ! long_v6 ) {
1028		condense_v6(as);
1029		}
1030		as[IP_STRING_LEN-1] = 0;
1031
1032		snprintf(_s, slen-1,
1033		" ip router = %16s\n"
1034		, as);
1035		_slen = strlen(data_string);
1036		_s = data_string + _slen;
1037		slen = STRINGSIZE - _slen;
1038		break;
1039		case EX_LATENCY: {
1040		double f1, f2, f3;
1041		f1 = (double)r->client_nw_delay_usec / 1000.0;
1042		f2 = (double)r->server_nw_delay_usec / 1000.0;
1043		f3 = (double)r->appl_latency_usec / 1000.0;
1044
1045		snprintf(_s, slen-1,
1046		" cli latency = %9.3f ms\n"
1047		" srv latency = %9.3f ms\n"
1048		" app latency = %9.3f ms\n"
1049		, f1, f2, f3);
1050
1051		_slen = strlen(data_string);
1052		_s = data_string + _slen;
1053		slen = STRINGSIZE - _slen;
1054
1055		} break;
1056		case EX_ROUTER_ID:
1057		snprintf(_s, slen-1,
1058		" engine type = %5u\n"
1059		" engine ID = %5u\n"
1060		, r->engine_type, r->engine_id);
1061		_slen = strlen(data_string);
1062		_s = data_string + _slen;
1063		slen = STRINGSIZE - _slen;
1064		break;
1065		case EX_RECEIVED:
1066		when = r->received / 1000LL;
1067		ts = localtime(&when);
1068		strftime(datestr3, 63, "%Y-%m-%d %H:%M:%S", ts);
1069
1070		snprintf(_s, slen-1,
1071		" received at = %13llu [%s.%03llu]\n"
1072		, (long long unsigned)r->received, datestr3, (long long unsigned)(r->received % 1000L));
1073		_slen = strlen(data_string);
1074		_s = data_string + _slen;
1075		slen = STRINGSIZE - _slen;
1076		break;
1077		#ifdef NSEL
1078		case EX_NSEL_COMMON: {
1079		char *event = "UNKNOWN";
1080		if ( r->event <= 5 ) {
1081		event = NSEL_event_string[r->event];
1082		}
1083		when = r->event_time / 1000LL;
1084		ts = localtime(&when);
1085		strftime(datestr3, 63, "%Y-%m-%d %H:%M:%S", ts);
1086		snprintf(_s, slen-1,
1087		" connect ID = %10u\n"
1088		" fw event = %5u: %s\n"
1089		" fw ext event = %5u\n"
1090		" Event time = %13llu [%s.%03llu]\n"
1091		, r->conn_id, r->event, event, r->fw_xevent
1092		, (long long unsigned)r->event_time, datestr3, (long long unsigned)(r->event_time % 1000L));
1093		_slen = strlen(data_string);
1094		_s = data_string + _slen;
1095		slen = STRINGSIZE - _slen;
1096		} break;
1097		case EX_NEL_COMMON: {
1098		char *event = "UNKNOWN";
1099		if ( r->event <= 2 ) {
1100		event = NEL_event_string[r->event];
1101		}
1102		snprintf(_s, slen-1,
1103		" nat event = %5u: %s\n"
1104		" ingress VRF = %10u\n"
1105		" egress VRF = %10u\n"
1106		, r->event, event, r->ingress_vrfid, r->egress_vrfid);
1107		_slen = strlen(data_string);
1108		_s = data_string + _slen;
1109		slen = STRINGSIZE - _slen;
1110		} break;
1111		case EX_NSEL_XLATE_PORTS: {
1112		snprintf(_s, slen-1,
1113		" src xlt port = %5u\n"
1114		" dst xlt port = %5u\n"
1115		, r->xlate_src_port, r->xlate_dst_port );
1116		_slen = strlen(data_string);
1117		_s = data_string + _slen;
1118		slen = STRINGSIZE - _slen;
1119		} break;
1120		case EX_PORT_BLOCK_ALLOC: {
1121		snprintf(_s, slen-1,
1122		" pblock start = %5u\n"
1123		" pblock end = %5u\n"
1124		" pblock step = %5u\n"
1125		" pblock size = %5u\n"
1126		, r->block_start, r->block_end, r->block_step, r->block_size );
1127		_slen = strlen(data_string);
1128		_s = data_string + _slen;
1129		slen = STRINGSIZE - _slen;
1130		} break;
1131		case EX_NSEL_XLATE_IP_v4:
1132		as[0] = 0;
1133		ds[0] = 0;
1134		r->xlate_src_ip.V4 = htonl(r->xlate_src_ip.V4);
1135		r->xlate_dst_ip.V4 = htonl(r->xlate_dst_ip.V4);
1136		inet_ntop(AF_INET, &r->xlate_src_ip.V4, as, sizeof(as));
1137		inet_ntop(AF_INET, &r->xlate_dst_ip.V4, ds, sizeof(ds));
1138		as[IP_STRING_LEN-1] = 0;
1139		ds[IP_STRING_LEN-1] = 0;
1140
1141		snprintf(_s, slen-1,
1142		" src xlt ip = %16s\n"
1143		" dst xlt ip = %16s\n"
1144		, as, ds);
1145		_slen = strlen(data_string);
1146		_s = data_string + _slen;
1147		slen = STRINGSIZE - _slen;
1148		break;
1149		case EX_NSEL_XLATE_IP_v6:
1150		as[0] = 0;
1151		ds[0] = 0;
1152		r->xlate_src_ip.V6[0] = htonll(r->xlate_src_ip.V6[0]);
1153		r->xlate_src_ip.V6[1] = htonll(r->xlate_src_ip.V6[1]);
1154		r->xlate_dst_ip.V6[0] = htonll(r->xlate_dst_ip.V6[0]);
1155		r->xlate_dst_ip.V6[1] = htonll(r->xlate_dst_ip.V6[1]);
1156		inet_ntop(AF_INET6, &r->xlate_src_ip.V6, as, sizeof(as));
1157		inet_ntop(AF_INET6, &r->xlate_dst_ip.V6, ds, sizeof(ds));
1158		if ( ! long_v6 ) {
1159		condense_v6(as);
1160		condense_v6(ds);
1161		}
1162		as[IP_STRING_LEN-1] = 0;
1163		ds[IP_STRING_LEN-1] = 0;
1164
1165		snprintf(_s, slen-1,
1166		" src xlate ip = %16s\n"
1167		" dst xlate ip = %16s\n"
1168		, as, ds);
1169		_slen = strlen(data_string);
1170		_s = data_string + _slen;
1171		slen = STRINGSIZE - _slen;
1172		break;
1173		case EX_NSEL_ACL:
1174		snprintf(_s, slen-1,
1175		" Ingress ACL = 0x%x/0x%x/0x%x\n"
1176		" Egress ACL = 0x%x/0x%x/0x%x\n"
1177		, r->ingress_acl_id[0], r->ingress_acl_id[1], r->ingress_acl_id[2],
1178		r->egress_acl_id[0], r->egress_acl_id[1], r->egress_acl_id[2]);
1179		_slen = strlen(data_string);
1180		_s = data_string + _slen;
1181		slen = STRINGSIZE - _slen;
1182		break;
1183		case EX_NSEL_USER:
1184		case EX_NSEL_USER_MAX:
1185		snprintf(_s, slen-1,
1186		" User name = %s\n"
1187		, r->username[0] ? r->username : " <empty>");
1188		_slen = strlen(data_string);
1189		_s = data_string + _slen;
1190		slen = STRINGSIZE - _slen;
1191		break;
1192		#endif
1193		default:
1194		snprintf(_s, slen-1, "Type %u not implemented\n", id);
1195
1196		}
1197		i++;
1198		}
1199
1200		data_string[STRINGSIZE-1] = 0;
1201		*s = data_string;
1202
1203
1204		} // End of format_file_block_record
1205
1206		void flow_record_to_pipe(void record, char * s, int tag) {
1207		uint32_t sa[4], da[4];
1208		int af;
1209		master_record_t r = (master_record_t )record;
1210
1211		if ( (r->flags & FLAG_IPV6_ADDR ) != 0 ) { // IPv6
1212		af = PF_INET6;
1213		} else { // IPv4
1214		af = PF_INET;
1215		}
1216
1217		// Make sure Endian does not screw us up
1218		sa[0] = ( r->V6.srcaddr[0] >> 32 ) & 0xffffffffLL;
1219		sa[1] = r->V6.srcaddr[0] & 0xffffffffLL;
1220		sa[2] = ( r->V6.srcaddr[1] >> 32 ) & 0xffffffffLL;
1221		sa[3] = r->V6.srcaddr[1] & 0xffffffffLL;
1222
1223		da[0] = ( r->V6.dstaddr[0] >> 32 ) & 0xffffffffLL;
1224		da[1] = r->V6.dstaddr[0] & 0xffffffffLL;
1225		da[2] = ( r->V6.dstaddr[1] >> 32 ) & 0xffffffffLL;
1226		da[3] = r->V6.dstaddr[1] & 0xffffffffLL;
1227
1228		snprintf(data_string, STRINGSIZE-1 ,"%i\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%llu\|%llu",
1229		af, r->first, r->msec_first ,r->last, r->msec_last, r->prot,
1230		sa[0], sa[1], sa[2], sa[3], r->srcport, da[0], da[1], da[2], da[3], r->dstport,
1231		r->srcas, r->dstas, r->input, r->output,
1232		r->tcp_flags, r->tos, (unsigned long long)r->dPkts, (unsigned long long)r->dOctets);
1233
1234		data_string[STRINGSIZE-1] = 0;
1235
1236		*s = data_string;
1237
1238		} // End of flow_record_pipe
1239
1240		void flow_record_to_csv(void record, char * s, int tag) {
1241		char *_s, as[IP_STRING_LEN], ds[IP_STRING_LEN];
1242		char proto_str[MAX_PROTO_STR], datestr1[64], datestr2[64], datestr3[64], flags_str[16];
1243		char s_snet[IP_STRING_LEN], s_dnet[IP_STRING_LEN];
1244		ssize_t slen, _slen;
1245		time_t when;
1246		struct tm *ts;
1247		master_record_t r = (master_record_t )record;
1248
1249		as[0] = 0;
1250		ds[0] = 0;
1251		if ( (r->flags & FLAG_IPV6_ADDR ) != 0 ) { // IPv6
1252		uint64_t snet[2];
1253		uint64_t dnet[2];
1254
1255		// remember IPs for network
1256		snet[0] = r->V6.srcaddr[0];
1257		snet[1] = r->V6.srcaddr[1];
1258		dnet[0] = r->V6.dstaddr[0];
1259		dnet[1] = r->V6.dstaddr[1];
1260		r->V6.srcaddr[0] = htonll(r->V6.srcaddr[0]);
1261		r->V6.srcaddr[1] = htonll(r->V6.srcaddr[1]);
1262		r->V6.dstaddr[0] = htonll(r->V6.dstaddr[0]);
1263		r->V6.dstaddr[1] = htonll(r->V6.dstaddr[1]);
1264		inet_ntop(AF_INET6, r->V6.srcaddr, as, sizeof(as));
1265		inet_ntop(AF_INET6, r->V6.dstaddr, ds, sizeof(ds));
1266
1267		if ( r->src_mask \|\| r->dst_mask) {
1268		if ( r->src_mask > 64 )
1269		snet[1] &= 0xffffffffffffffffLL << ( 128 - r->src_mask );
1270		else {
1271		snet[1] &= 0xffffffffffffffffLL << ( 64 - r->src_mask );
1272		snet[1] = 0;
1273		}
1274		snet[0] = htonll(snet[0]);
1275		snet[1] = htonll(snet[1]);
1276		inet_ntop(AF_INET6, &snet, s_snet, sizeof(s_snet));
1277
1278		if ( r->dst_mask > 64 )
1279		dnet[1] &= 0xffffffffffffffffLL << ( 128 - r->dst_mask );
1280		else {
1281		dnet[1] &= 0xffffffffffffffffLL << ( 64 - r->dst_mask );
1282		dnet[1] = 0;
1283		}
1284		dnet[0] = htonll(dnet[0]);
1285		dnet[1] = htonll(dnet[1]);
1286		inet_ntop(AF_INET6, &dnet, s_dnet, sizeof(s_dnet));
1287
1288		} else {
1289		s_snet[0] = '\0';
1290		s_dnet[0] = '\0';
1291		}
1292
1293		} else { // IPv4
1294		uint32_t snet, dnet;
1295		snet = r->V4.srcaddr;
1296		dnet = r->V4.dstaddr;
1297		r->V4.srcaddr = htonl(r->V4.srcaddr);
1298		r->V4.dstaddr = htonl(r->V4.dstaddr);
1299		inet_ntop(AF_INET, &r->V4.srcaddr, as, sizeof(as));
1300		inet_ntop(AF_INET, &r->V4.dstaddr, ds, sizeof(ds));
1301		if ( r->src_mask \|\| r->dst_mask) {
1302		snet &= 0xffffffffL << ( 32 - r->src_mask );
1303		snet = htonl(snet);
1304		inet_ntop(AF_INET, &snet, s_snet, sizeof(s_snet));
1305
1306		dnet &= 0xffffffffL << ( 32 - r->dst_mask );
1307		dnet = htonl(dnet);
1308		inet_ntop(AF_INET, &dnet, s_dnet, sizeof(s_dnet));
1309		} else {
1310		s_snet[0] = '\0';
1311		s_dnet[0] = '\0';
1312		}
1313		}
1314		as[IP_STRING_LEN-1] = 0;
1315		ds[IP_STRING_LEN-1] = 0;
1316
1317		when = r->first;
1318		ts = localtime(&when);
1319		strftime(datestr1, 63, "%Y-%m-%d %H:%M:%S", ts);
1320
1321		when = r->last;
1322		ts = localtime(&when);
1323		strftime(datestr2, 63, "%Y-%m-%d %H:%M:%S", ts);
1324
1325		duration = r->last - r->first;
1326		duration += ((double)r->msec_last - (double)r->msec_first) / 1000.0;
1327
1328		String_Flags(record, flags_str);
1329
1330		if ( r->prot >= NumProtos ) {
1331		snprintf(proto_str,MAX_PROTO_STR-1,"%u", r->prot );
1332		proto_str[MAX_PROTO_STR-1] = '\0';
1333		} else {
1334		int i = 0;;
1335		strncpy(proto_str, protolist[r->prot], MAX_PROTO_STR);
1336		// remove white spaces for csv
1337		while ( proto_str[i] ) {
1338		if ( proto_str[i] == ' ' )
1339		proto_str[i] = '\0';
1340		i++;
1341		}
1342		}
1343
1344		_s = data_string;
1345		slen = STRINGSIZE;
1346		snprintf(_s, slen-1, "%s,%s,%.3f,%s,%s,%u,%u,%s,%s,%u,%u,%llu,%llu,%llu,%llu",
1347		datestr1, datestr2, duration, as,ds,r->srcport, r->dstport, proto_str, flags_str,
1348		r->fwd_status, r->tos, (unsigned long long)r->dPkts, (unsigned long long)r->dOctets,
1349		(long long unsigned)r->out_pkts, (long long unsigned)r->out_bytes
1350		);
1351
1352		_slen = strlen(data_string);
1353		_s += _slen;
1354		slen -= _slen;
1355
1356		// EX_IO_SNMP_2:
1357		// EX_IO_SNMP_4:
1358		snprintf(_s, slen-1, ",%u,%u" , r->input, r->output);
1359		_slen = strlen(data_string);
1360		_s = data_string + _slen;
1361		slen = STRINGSIZE - _slen;
1362
1363		// EX_AS_2:
1364		// EX_AS_4:
1365		snprintf(_s, slen-1, ",%u,%u", r->srcas, r->dstas);
1366		_slen = strlen(data_string);
1367		_s = data_string + _slen;
1368		slen = STRINGSIZE - _slen;
1369
1370		// EX_MULIPLE:
1371		snprintf(_s, slen-1, ",%u,%u,%u,%u" , r->src_mask, r->dst_mask, r->dst_tos, r->dir );
1372		_slen = strlen(data_string);
1373		_s = data_string + _slen;
1374		slen = STRINGSIZE - _slen;
1375
1376		if ( (r->flags & FLAG_IPV6_NH ) != 0 ) { // IPv6
1377		// EX_NEXT_HOP_v6:
1378		as[0] = 0;
1379		r->ip_nexthop.V6[0] = htonll(r->ip_nexthop.V6[0]);
1380		r->ip_nexthop.V6[1] = htonll(r->ip_nexthop.V6[1]);
1381		inet_ntop(AF_INET6, r->ip_nexthop.V6, as, sizeof(as));
1382		as[IP_STRING_LEN-1] = 0;
1383
1384		snprintf(_s, slen-1, ",%s", as);
1385		_slen = strlen(data_string);
1386		_s = data_string + _slen;
1387		slen = STRINGSIZE - _slen;
1388
1389		} else {
1390		// EX_NEXT_HOP_v4:
1391		as[0] = 0;
1392		r->ip_nexthop.V4 = htonl(r->ip_nexthop.V4);
1393		inet_ntop(AF_INET, &r->ip_nexthop.V4, as, sizeof(as));
1394		as[IP_STRING_LEN-1] = 0;
1395
1396		snprintf(_s, slen-1, ",%s", as);
1397		_slen = strlen(data_string);
1398		_s = data_string + _slen;
1399		slen = STRINGSIZE - _slen;
1400		}
1401
1402		if ( (r->flags & FLAG_IPV6_NHB ) != 0 ) { // IPv6
1403		// EX_NEXT_HOP_BGP_v6:
1404		as[0] = 0;
1405		r->bgp_nexthop.V6[0] = htonll(r->bgp_nexthop.V6[0]);
1406		r->bgp_nexthop.V6[1] = htonll(r->bgp_nexthop.V6[1]);
1407		inet_ntop(AF_INET6, r->ip_nexthop.V6, as, sizeof(as));
1408		as[IP_STRING_LEN-1] = 0;
1409
1410		snprintf(_s, slen-1, ",%s", as);
1411		_slen = strlen(data_string);
1412		_s = data_string + _slen;
1413		slen = STRINGSIZE - _slen;
1414
1415		} else {
1416		// EX_NEXT_HOP_BGP_v4:
1417		as[0] = 0;
1418		r->bgp_nexthop.V4 = htonl(r->bgp_nexthop.V4);
1419		inet_ntop(AF_INET, &r->bgp_nexthop.V4, as, sizeof(as));
1420		as[IP_STRING_LEN-1] = 0;
1421
1422		snprintf(_s, slen-1, ",%s", as);
1423		_slen = strlen(data_string);
1424		_s = data_string + _slen;
1425		slen = STRINGSIZE - _slen;
1426
1427		}
1428
1429		// EX_VLAN:
1430		snprintf(_s, slen-1, ",%u,%u", r->src_vlan, r->dst_vlan);
1431		_slen = strlen(data_string);
1432		_s = data_string + _slen;
1433		slen = STRINGSIZE - _slen;
1434
1435
1436		/* already in default output:
1437		EX_OUT_PKG_4:
1438		EX_OUT_PKG_8:
1439		EX_OUT_BYTES_4:
1440		EX_OUT_BYTES_8:
1441		*/
1442
1443		// case EX_MAC_1:
1444		{
1445		int i;
1446		uint8_t mac1[6], mac2[6];
1447
1448		for ( i=0; i<6; i++ ) {
1449		mac1[i] = (r->in_src_mac >> ( i*8 )) & 0xFF;
1450		}
1451		for ( i=0; i<6; i++ ) {
1452		mac2[i] = (r->out_dst_mac >> ( i*8 )) & 0xFF;
1453		}
1454
1455		snprintf(_s, slen-1, ",%.2x:%.2x:%.2x:%.2x:%.2x:%.2x,%.2x:%.2x:%.2x:%.2x:%.2x:%.2x",
1456		mac1[5], mac1[4], mac1[3], mac1[2], mac1[1], mac1[0],
1457		mac2[5], mac2[4], mac2[3], mac2[2], mac2[1], mac2[0] );
1458		_slen = strlen(data_string);
1459		_s = data_string + _slen;
1460		slen = STRINGSIZE - _slen;
1461		}
1462
1463		// EX_MAC_2:
1464		{
1465		int i;
1466		uint8_t mac1[6], mac2[6];
1467
1468		for ( i=0; i<6; i++ ) {
1469		mac1[i] = (r->in_dst_mac >> ( i*8 )) & 0xFF;
1470		}
1471		for ( i=0; i<6; i++ ) {
1472		mac2[i] = (r->out_src_mac >> ( i*8 )) & 0xFF;
1473		}
1474
1475		snprintf(_s, slen-1, ",%.2x:%.2x:%.2x:%.2x:%.2x:%.2x,%.2x:%.2x:%.2x:%.2x:%.2x:%.2x",
1476		mac1[5], mac1[4], mac1[3], mac1[2], mac1[1], mac1[0],
1477		mac2[5], mac2[4], mac2[3], mac2[2], mac2[1], mac2[0] );
1478		_slen = strlen(data_string);
1479		_s = data_string + _slen;
1480		slen = STRINGSIZE - _slen;
1481		}
1482
1483		// EX_MPLS:
1484		{
1485		unsigned int i;
1486		for ( i=0; i<10; i++ ) {
1487		snprintf(_s, slen-1, ",%u-%1u-%1u",
1488		r->mpls_label[i] >> 4 , (r->mpls_label[i] & 0xF ) >> 1, r->mpls_label[i] & 1 );
1489		_slen = strlen(data_string);
1490		_s = data_string + _slen;
1491		slen = STRINGSIZE - _slen;
1492		}
1493		}
1494
1495		{
1496		double f1, f2, f3;
1497		f1 = (double)r->client_nw_delay_usec / 1000.0;
1498		f2 = (double)r->server_nw_delay_usec / 1000.0;
1499		f3 = (double)r->appl_latency_usec / 1000.0;
1500
1501		snprintf(_s, slen-1,
1502		",%9.3f,%9.3f,%9.3f", f1, f2, f3);
1503
1504		_slen = strlen(data_string);
1505		_s = data_string + _slen;
1506		slen = STRINGSIZE - _slen;
1507		}
1508
1509
1510		// EX_ROUTER_IP_v4:
1511		if ( (r->flags & FLAG_IPV6_EXP ) != 0 ) { // IPv6
1512		// EX_NEXT_HOP_v6:
1513		as[0] = 0;
1514		r->ip_router.V6[0] = htonll(r->ip_router.V6[0]);
1515		r->ip_router.V6[1] = htonll(r->ip_router.V6[1]);
1516		inet_ntop(AF_INET6, r->ip_router.V6, as, sizeof(as));
1517		as[IP_STRING_LEN-1] = 0;
1518
1519		snprintf(_s, slen-1, ",%s", as);
1520		_slen = strlen(data_string);
1521		_s = data_string + _slen;
1522		slen = STRINGSIZE - _slen;
1523
1524		} else {
1525		// EX_NEXT_HOP_v4:
1526		as[0] = 0;
1527		r->ip_router.V4 = htonl(r->ip_router.V4);
1528		inet_ntop(AF_INET, &r->ip_router.V4, as, sizeof(as));
1529		as[IP_STRING_LEN-1] = 0;
1530
1531		snprintf(_s, slen-1, ",%s", as);
1532		_slen = strlen(data_string);
1533		_s = data_string + _slen;
1534		slen = STRINGSIZE - _slen;
1535		}
1536
1537		// EX_ROUTER_ID
1538		snprintf(_s, slen-1, ",%u/%u", r->engine_type, r->engine_id);
1539		_slen = strlen(data_string);
1540		_s = data_string + _slen;
1541		slen = STRINGSIZE - _slen;
1542
1543		// Exporter SysID
1544		snprintf(_s, slen-1, ",%u", r->exporter_sysid);
1545		_slen = strlen(data_string);
1546		_s = data_string + _slen;
1547		slen = STRINGSIZE - _slen;
1548
1549		// Date flow received
1550		when = r->received / 1000LL;
1551		ts = localtime(&when);
1552		strftime(datestr3, 63, ",%Y-%m-%d %H:%M:%S", ts);
1553
1554		snprintf(_s, slen-1, "%s.%03llu", datestr3, (long long unsigned)r->received % 1000LL);
1555		_slen = strlen(data_string);
1556		_s = data_string + _slen;
1557		slen = STRINGSIZE - _slen;
1558
1559		// snprintf(_s, slen-1, "\n");
1560		data_string[STRINGSIZE-1] = 0;
1561		*s = data_string;
1562
1563
1564		} // End of flow_record_to_csv
1565
1566		void flow_record_to_null(void record, char * s, int tag) {
1567		// empty - do not list any flows
1568		} // End of flow_record_to_null
1569
1570		void format_special(void record, char * s, int tag) {
1571		master_record_t r = (master_record_t )record;
1572		int i, index;
1573
1574		do_tag = tag;
1575		tag_string[0] = do_tag ? TAG_CHAR : '\0';
1576		tag_string[1] = '\0';
1577
1578		duration = r->last - r->first;
1579		duration += ((double)r->msec_last - (double)r->msec_first) / 1000.0;
1580		for ( i=0; i<token_index; i++ ) {
1581		token_list[i].string_function(r, token_list[i].string_buffer);
1582		}
1583
1584		// concat all strings together for the output line
1585		i = 0;
1586		for ( index=0; index<format_index; index++ ) {
1587		int j = 0;
1588		while ( format_list[index][j] && i < STRINGSIZE )
1589		data_string[i++] = format_list[index][j++];
1590		}
1591		if ( i < STRINGSIZE )
1592		data_string[i] = '\0';
1593
1594		data_string[STRINGSIZE-1] = '\0';
1595		*s = data_string;
1596
1597		} // End of format_special
1598
1599		char *get_record_header(void) {
1600		return header_string;
1601		} // End of get_record_header
1602
1603		void set_record_header(void) {
1604
1605		snprintf(header_string, STRINGSIZE-1, "ts,te,td,sa,da,sp,dp,pr,flg,fwd,stos,ipkt,ibyt,opkt,obyt,in,out,sas,das,smk,dmk,dtos,dir,nh,nhb,svln,dvln,ismc,odmc,idmc,osmc,mpls1,mpls2,mpls3,mpls4,mpls5,mpls6,mpls7,mpls8,mpls9,mpls10,cl,sl,al,ra,eng,exid,tr");
1606		header_string[STRINGSIZE-1] = '\0';
1607
1608		} // End of format_csv_header
1609
1610		static void InitFormatParser(void) {
1611
1612		max_format_index = max_token_index = BLOCK_SIZE;
1613		format_list = (char *)malloc(max_format_index sizeof(char *));
1614		token_list = (struct token_list_s )malloc(max_token_index sizeof(struct token_list_s));
1615		if ( !format_list \|\| !token_list ) {
1616		fprintf(stderr, "Memory allocation error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
1617		exit(255);
1618		}
1619
1620
1621		} // End of InitFormatParser
1622
1623		static void AddToken(int index) {
1624
1625		if ( token_index >= max_token_index ) { // no slot available - expand table
1626		max_token_index += BLOCK_SIZE;
1627		token_list = (struct token_list_s )realloc(token_list, max_token_index sizeof(struct token_list_s));
1628		if ( !token_list ) {
1629		fprintf(stderr, "Memory allocation error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
1630		exit(255);
1631		}
1632		}
1633		token_list[token_index].string_function = format_token_list[index].string_function;
1634		token_list[token_index].string_buffer = malloc(MAX_STRING_LENGTH);
1635		if ( !token_list[token_index].string_buffer ) {
1636		fprintf(stderr, "Memory allocation error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
1637		exit(255);
1638		}
1639
1640		AddString(token_list[token_index].string_buffer);
1641		token_index++;
1642
1643		} // End of AddToken
1644
1645		/* Add either a static string or the memory for a variable string from a token to the list */
1646		static void AddString(char *string) {
1647
1648		if ( !string ) {
1649		fprintf(stderr, "Panic! NULL string in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
1650		exit(255);
1651		}
1652
1653		if ( format_index >= max_format_index ) { // no slot available - expand table
1654		max_format_index += BLOCK_SIZE;
1655		format_list = (char *)realloc(format_list, max_format_index sizeof(char *));
1656		if ( !format_list ) {
1657		fprintf(stderr, "Memory allocation error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
1658		exit(255);
1659		}
1660		}
1661		format_list[format_index++] = string;
1662
1663		} // End of AddString
1664
1665		static char* RecursiveReplace(char format, printmap_t printmap) {
1666		int i = 0;
1667
1668		while ( printmap[i].printmode ) {
1669		char s, r;
1670		// check for printmode string
1671		s = strstr(format, printmap[i].printmode);
1672		if ( s && printmap[i].Format && s != format ) {
1673		int len = strlen(printmap[i].printmode);
1674		if ( !isalpha((int)s[len]) ) {
1675		s--;
1676		if ( s[0] == '%' ) {
1677		int newlen = strlen(format) + strlen(printmap[i].Format);
1678		r = malloc(newlen);
1679		if ( !r ) {
1680		LogError("malloc() allocation error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
1681		exit(255);
1682		}
1683		s[0] = '\0';
1684		snprintf(r, newlen, "%s%s%s", format, printmap[i].Format, &(s[len+1]) );
1685		r[newlen-1] = '\0';
1686		free(format);
1687		format = r;
1688		}
1689		}
1690		}
1691		i++;
1692		}
1693
1694		return format;
1695
1696		} // End of RecursiveReplace
1697
1698		int ParseOutputFormat(char format, int plain_numbers, printmap_t printmap) {
1699		char c, s, *h;
1700		int i, remaining;
1701
1702		scale = plain_numbers == 0;
1703
1704		InitFormatParser();
1705
1706		s = strdup(format);
1707		if ( !s ) {
1708		fprintf(stderr, "Memory allocation error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
1709		exit(255);
1710		}
1711		s = RecursiveReplace(s, printmap);
1712		c = s;
1713
1714		h = header_string;
1715		*h = '\0';
1716		while ( *c ) {
1717		if ( *c == '%' ) { // it's a token from format_token_list
1718		i = 0;
1719		remaining = strlen(c);
1720		while ( format_token_list[i].token ) { // sweep through the list
1721		int len = strlen(format_token_list[i].token);
1722
1723		// a token is separated by either a space, another token, or end of string
1724		if ( remaining >= len && !isalpha((int)c[len]) ) {
1725		// separator found a expected position
1726		char p = c[len]; // save separator;
1727		c[len] = '\0';
1728		if ( strncmp(format_token_list[i].token, c, len) == 0 ) { // token found
1729		AddToken(i);
1730		if ( long_v6 && format_token_list[i].is_address )
1731		snprintf(h, STRINGSIZE-1-strlen(h), "%23s%s", "", format_token_list[i].header);
1732		else
1733		snprintf(h, STRINGSIZE-1-strlen(h), "%s", format_token_list[i].header);
1734		h += strlen(h);
1735		c[len] = p;
1736		c += len;
1737		break;
1738		} else {
1739		c[len] = p;
1740		}
1741		}
1742		i++;
1743		}
1744		if ( format_token_list[i].token == NULL ) {
1745		fprintf(stderr, "Output format parse error at: %s\n", c);
1746		free(s);
1747		return 0;
1748		}
1749		} else { // it's a static string
1750		/* a static string goes up to next '%' or end of string */
1751		char *p = strchr(c, '%');
1752		char format[16];
1753		if ( p ) {
1754		// p points to next '%' token
1755		*p = '\0';
1756		AddString(strdup(c));
1757		snprintf(format, 15, "%%%zus", strlen(c));
1758		format[15] = '\0';
1759		snprintf(h, STRINGSIZE-1-strlen(h), format, "");
1760		h += strlen(h);
1761		*p = '%';
1762		c = p;
1763		} else {
1764		// static string up to end of format string
1765		AddString(strdup(c));
1766		snprintf(format, 15, "%%%zus", strlen(c));
1767		format[15] = '\0';
1768		snprintf(h, STRINGSIZE-1-strlen(h), format, "");
1769		h += strlen(h);
1770		*c = '\0';
1771		}
1772		}
1773		}
1774
1775		free(s);
1776		return 1;
1777
1778		} // End of ParseOutputFormat
1779
1780		void condense_v6(char *s) {
1781		size_t len = strlen(s);
1782		char p, q;
1783
1784		if ( len <= 16 )
1785		return;
1786
1787		// orig: 2001:620:1000:cafe:20e:35ff:fec0:fed5 len = 37
1788		// condensed: 2001:62..e0:fed5
1789		p = s + 7;
1790		*p++ = '.';
1791		*p++ = '.';
1792		q = s + len - 7;
1793		while ( *q ) {
1794		p++ = q++;
1795		}
1796		*p = 0;
1797
1798		} // End of condense_v6
1799
1800		static inline void ICMP_Port_decode(master_record_t r, char string) {
1801
1802		if ( r->prot == IPPROTO_ICMP \|\| r->prot == IPPROTO_ICMPV6 ) { // ICMP
1803		snprintf(string, MAX_STRING_LENGTH-1, "%u.%u", r->icmp_type, r->icmp_code);
1804		} else { // dst port
1805		snprintf(string, MAX_STRING_LENGTH-1, "%u", r->dstport);
1806		}
1807		string[MAX_STRING_LENGTH-1] = '\0';
1808
1809		} // End of ICMP_Port_decode
1810
1811		/* functions, which create the individual strings for the output line */
1812		static void String_FlowFlags(master_record_t r, char string) {
1813
1814		snprintf(string, MAX_STRING_LENGTH-1, "0x%.2x", r->flags);
1815		string[MAX_STRING_LENGTH-1] = '\0';
1816
1817		} // End of String_FlowFlags
1818
1819		static void String_FirstSeen(master_record_t r, char string) {
1820		time_t tt;
1821		struct tm * ts;
1822		char *s;
1823
1824		tt = r->first;
1825		ts = localtime(&tt);
1826		strftime(string, MAX_STRING_LENGTH-1, "%Y-%m-%d %H:%M:%S", ts);
1827		s = string + strlen(string);
1828		snprintf(s, MAX_STRING_LENGTH-strlen(string)-1,".%03u", r->msec_first);
1829		string[MAX_STRING_LENGTH-1] = '\0';
1830
1831		} // End of String_FirstSeen
1832
1833		static void String_LastSeen(master_record_t r, char string) {
1834		time_t tt;
1835		struct tm * ts;
1836		char *s;
1837
1838		tt = r->last;
1839		ts = localtime(&tt);
1840		strftime(string, MAX_STRING_LENGTH-1, "%Y-%m-%d %H:%M:%S", ts);
1841		s = string + strlen(string);
1842		snprintf(s, MAX_STRING_LENGTH-strlen(string)-1,".%03u", r->msec_last);
1843		string[MAX_STRING_LENGTH-1] = '\0';
1844
1845		} // End of String_LastSeen
1846
1847		static void String_Received(master_record_t r, char string) {
1848		time_t tt;
1849		struct tm * ts;
1850		char *s;
1851
1852		tt = r->received / 1000LL;
1853		ts = localtime(&tt);
1854		strftime(string, MAX_STRING_LENGTH-1, "%Y-%m-%d %H:%M:%S", ts);
1855		s = string + strlen(string);
1856		snprintf(s, MAX_STRING_LENGTH-strlen(string)-1,".%03llu", r->received % 1000LL);
1857		string[MAX_STRING_LENGTH-1] = '\0';
1858
1859		} // End of String_Received
1860
1861		static void String_ReceivedRaw(master_record_t r, char string) {
1862
1863		/* snprintf does write \0, and the max is INCL the terminating \0 */
1864		snprintf(string, MAX_STRING_LENGTH, "%.3f", r->received/1000.0);
1865
1866		} // End of String_ReceivedRaw
1867
1868		static void String_FirstSeenRaw(master_record_t r, char string) {
1869
1870		/* snprintf does write \0, and the max is INCL the terminating \0 */
1871		snprintf(string, MAX_STRING_LENGTH, "%u.%03u", r->first, r->msec_first);
1872
1873		} // End of String_FirstSeenRaw
1874
1875		static void String_LastSeenRaw(master_record_t r, char string) {
1876
1877		/* snprintf does write \0, and the max is INCL the terminating \0 */
1878		snprintf(string, MAX_STRING_LENGTH, "%u.%03u", r->last, r->msec_last);
1879
1880		} // End of String_LastSeenRaw
1881
1882
1883		#ifdef NSEL
1884		static void String_EventTime(master_record_t r, char string) {
1885		time_t tt;
1886		struct tm * ts;
1887		char *s;
1888
1889		tt = r->event_time / 1000LL;
1890		ts = localtime(&tt);
1891		strftime(string, MAX_STRING_LENGTH-1, "%Y-%m-%d %H:%M:%S", ts);
1892		s = string + strlen(string);
1893		snprintf(s, MAX_STRING_LENGTH-strlen(string)-1,".%03llu", r->event_time % 1000LL);
1894		string[MAX_STRING_LENGTH-1] = '\0';
1895
1896		} // End of String_EventTime
1897		#endif
1898
1899		static void String_Duration(master_record_t r, char string) {
1900
1901		snprintf(string, MAX_STRING_LENGTH-1 ,"%9.3f", duration);
1902		string[MAX_STRING_LENGTH-1] = '\0';
1903
1904		} // End of String_Duration
1905
1906		static void String_Protocol(master_record_t r, char string) {
1907		char s[16];
1908
1909		Proto_string(r->prot, s);
1910		snprintf(string, MAX_STRING_LENGTH-1 ,"%s", s);
1911		string[MAX_STRING_LENGTH-1] = '\0';
1912
1913		} // End of String_Protocol
1914
1915		static void String_SrcAddr(master_record_t r, char string) {
1916		char tmp_str[IP_STRING_LEN];
1917
1918		tmp_str[0] = 0;
1919		if ( (r->flags & FLAG_IPV6_ADDR ) != 0 ) { // IPv6
1920		uint64_t ip[2];
1921
1922		ip[0] = htonll(r->V6.srcaddr[0]);
1923		ip[1] = htonll(r->V6.srcaddr[1]);
1924		inet_ntop(AF_INET6, ip, tmp_str, sizeof(tmp_str));
1925		if ( ! long_v6 ) {
1926		condense_v6(tmp_str);
1927		}
1928		} else { // IPv4
1929		uint32_t ip;
1930		ip = htonl(r->V4.srcaddr);
1931		inet_ntop(AF_INET, &ip, tmp_str, sizeof(tmp_str));
1932		}
1933		tmp_str[IP_STRING_LEN-1] = 0;
1934		if ( long_v6 )
1935		snprintf(string, MAX_STRING_LENGTH-1, "%s%39s", tag_string, tmp_str);
1936		else
1937		snprintf(string, MAX_STRING_LENGTH-1, "%s%16s", tag_string, tmp_str);
1938
1939		string[MAX_STRING_LENGTH-1] = 0;
1940
1941
1942		} // End of String_SrcAddr
1943
1944		static void String_SrcAddrPort(master_record_t r, char string) {
1945		char tmp_str[IP_STRING_LEN], portchar;
1946
1947		tmp_str[0] = 0;
1948		if ( (r->flags & FLAG_IPV6_ADDR ) != 0 ) { // IPv6
1949		uint64_t ip[2];
1950
1951		ip[0] = htonll(r->V6.srcaddr[0]);
1952		ip[1] = htonll(r->V6.srcaddr[1]);
1953		inet_ntop(AF_INET6, ip, tmp_str, sizeof(tmp_str));
1954		if ( ! long_v6 ) {
1955		condense_v6(tmp_str);
1956		}
1957		portchar = '.';
1958		} else { // IPv4
1959		uint32_t ip;
1960		ip = htonl(r->V4.srcaddr);
1961		inet_ntop(AF_INET, &ip, tmp_str, sizeof(tmp_str));
1962		portchar = ':';
1963		}
1964		tmp_str[IP_STRING_LEN-1] = 0;
1965
1966		if ( long_v6 )
1967		snprintf(string, MAX_STRING_LENGTH-1, "%s%39s%c%-5i", tag_string, tmp_str, portchar, r->srcport);
1968		else
1969		snprintf(string, MAX_STRING_LENGTH-1, "%s%16s%c%-5i", tag_string, tmp_str, portchar, r->srcport);
1970
1971		string[MAX_STRING_LENGTH-1] = 0;
1972
1973		} // End of String_SrcAddrPort
1974
1975		static void String_DstAddr(master_record_t r, char string) {
1976		char tmp_str[IP_STRING_LEN];
1977
1978		tmp_str[0] = 0;
1979		if ( (r->flags & FLAG_IPV6_ADDR ) != 0 ) { // IPv6
1980		uint64_t ip[2];
1981
1982		ip[0] = htonll(r->V6.dstaddr[0]);
1983		ip[1] = htonll(r->V6.dstaddr[1]);
1984		inet_ntop(AF_INET6, ip, tmp_str, sizeof(tmp_str));
1985		if ( ! long_v6 ) {
1986		condense_v6(tmp_str);
1987		}
1988		} else { // IPv4
1989		uint32_t ip;
1990		ip = htonl(r->V4.dstaddr);
1991		inet_ntop(AF_INET, &ip, tmp_str, sizeof(tmp_str));
1992		}
1993		tmp_str[IP_STRING_LEN-1] = 0;
1994		if ( long_v6 )
1995		snprintf(string, MAX_STRING_LENGTH-1, "%s%39s", tag_string, tmp_str);
1996		else
1997		snprintf(string, MAX_STRING_LENGTH-1, "%s%16s", tag_string, tmp_str);
1998
1999		string[MAX_STRING_LENGTH-1] = 0;
2000
2001
2002		} // End of String_DstAddr
2003
2004
2005		static void String_NextHop(master_record_t r, char string) {
2006		char tmp_str[IP_STRING_LEN];
2007
2008		tmp_str[0] = 0;
2009		if ( (r->flags & FLAG_IPV6_NH ) != 0 ) { // IPv6
2010		uint64_t ip[2];
2011
2012		ip[0] = htonll(r->ip_nexthop.V6[0]);
2013		ip[1] = htonll(r->ip_nexthop.V6[1]);
2014		inet_ntop(AF_INET6, ip, tmp_str, sizeof(tmp_str));
2015		if ( ! long_v6 ) {
2016		condense_v6(tmp_str);
2017		}
2018		} else { // IPv4
2019		uint32_t ip;
2020		ip = htonl(r->ip_nexthop.V4);
2021		inet_ntop(AF_INET, &ip, tmp_str, sizeof(tmp_str));
2022		}
2023		tmp_str[IP_STRING_LEN-1] = 0;
2024		if ( long_v6 )
2025		snprintf(string, MAX_STRING_LENGTH-1, "%s%39s", tag_string, tmp_str);
2026		else
2027		snprintf(string, MAX_STRING_LENGTH-1, "%s%16s", tag_string, tmp_str);
2028
2029		string[MAX_STRING_LENGTH-1] = 0;
2030
2031
2032		} // End of String_NextHop
2033
2034		static void String_BGPNextHop(master_record_t r, char string) {
2035		char tmp_str[IP_STRING_LEN];
2036
2037		tmp_str[0] = 0;
2038		if ( (r->flags & FLAG_IPV6_NHB ) != 0 ) { // IPv6
2039		uint64_t ip[2];
2040
2041		ip[0] = htonll(r->bgp_nexthop.V6[0]);
2042		ip[1] = htonll(r->bgp_nexthop.V6[1]);
2043		inet_ntop(AF_INET6, ip, tmp_str, sizeof(tmp_str));
2044		if ( ! long_v6 ) {
2045		condense_v6(tmp_str);
2046		}
2047		} else { // IPv4
2048		uint32_t ip;
2049		ip = htonl(r->bgp_nexthop.V4);
2050		inet_ntop(AF_INET, &ip, tmp_str, sizeof(tmp_str));
2051		}
2052		tmp_str[IP_STRING_LEN-1] = 0;
2053		if ( long_v6 )
2054		snprintf(string, MAX_STRING_LENGTH-1, "%s%39s", tag_string, tmp_str);
2055		else
2056		snprintf(string, MAX_STRING_LENGTH-1, "%s%16s", tag_string, tmp_str);
2057
2058		string[MAX_STRING_LENGTH-1] = 0;
2059
2060
2061		} // End of String_NextHop
2062
2063		static void String_RouterIP(master_record_t r, char string) {
2064		char tmp_str[IP_STRING_LEN];
2065
2066		tmp_str[0] = 0;
2067		if ( (r->flags & FLAG_IPV6_EXP ) != 0 ) { // IPv6
2068		uint64_t ip[2];
2069
2070		ip[0] = htonll(r->ip_router.V6[0]);
2071		ip[1] = htonll(r->ip_router.V6[1]);
2072		inet_ntop(AF_INET6, ip, tmp_str, sizeof(tmp_str));
2073		if ( ! long_v6 ) {
2074		condense_v6(tmp_str);
2075		}
2076		} else { // IPv4
2077		uint32_t ip;
2078		ip = htonl(r->ip_router.V4);
2079		inet_ntop(AF_INET, &ip, tmp_str, sizeof(tmp_str));
2080		}
2081		tmp_str[IP_STRING_LEN-1] = 0;
2082		if ( long_v6 )
2083		snprintf(string, MAX_STRING_LENGTH-1, "%s%39s", tag_string, tmp_str);
2084		else
2085		snprintf(string, MAX_STRING_LENGTH-1, "%s%16s", tag_string, tmp_str);
2086
2087		string[MAX_STRING_LENGTH-1] = 0;
2088
2089
2090		} // End of String_RouterIP
2091
2092
2093		static void String_DstAddrPort(master_record_t r, char string) {
2094		char tmp_str[IP_STRING_LEN], portchar;
2095		char icmp_port[MAX_STRING_LENGTH];
2096
2097		tmp_str[0] = 0;
2098		if ( (r->flags & FLAG_IPV6_ADDR ) != 0 ) { // IPv6
2099		uint64_t ip[2];
2100
2101		ip[0] = htonll(r->V6.dstaddr[0]);
2102		ip[1] = htonll(r->V6.dstaddr[1]);
2103		inet_ntop(AF_INET6, ip, tmp_str, sizeof(tmp_str));
2104		if ( ! long_v6 ) {
2105		condense_v6(tmp_str);
2106		}
2107		portchar = '.';
2108		} else { // IPv4
2109		uint32_t ip;
2110		ip = htonl(r->V4.dstaddr);
2111		inet_ntop(AF_INET, &ip, tmp_str, sizeof(tmp_str));
2112		portchar = ':';
2113		}
2114		tmp_str[IP_STRING_LEN-1] = 0;
2115		ICMP_Port_decode(r, icmp_port);
2116
2117		if ( long_v6 )
2118		snprintf(string, MAX_STRING_LENGTH-1, "%s%39s%c%-5s", tag_string, tmp_str, portchar, icmp_port);
2119		else
2120		snprintf(string, MAX_STRING_LENGTH-1, "%s%16s%c%-5s", tag_string, tmp_str, portchar, icmp_port);
2121
2122		string[MAX_STRING_LENGTH-1] = 0;
2123
2124		} // End of String_DstAddrPort
2125
2126		static void String_SrcNet(master_record_t r, char string) {
2127		char tmp_str[IP_STRING_LEN];
2128
2129		ApplyNetMaskBits(r, 1);
2130
2131		tmp_str[0] = 0;
2132		if ( (r->flags & FLAG_IPV6_ADDR ) != 0 ) { // IPv6
2133		uint64_t ip[2];
2134
2135		ip[0] = htonll(r->V6.srcaddr[0]);
2136		ip[1] = htonll(r->V6.srcaddr[1]);
2137		inet_ntop(AF_INET6, ip, tmp_str, sizeof(tmp_str));
2138		if ( ! long_v6 ) {
2139		condense_v6(tmp_str);
2140		}
2141		} else { // IPv4
2142		uint32_t ip;
2143		ip = htonl(r->V4.srcaddr);
2144		inet_ntop(AF_INET, &ip, tmp_str, sizeof(tmp_str));
2145		}
2146		tmp_str[IP_STRING_LEN-1] = 0;
2147		if ( long_v6 )
2148		snprintf(string, MAX_STRING_LENGTH-1, "%s%39s/%-2u", tag_string, tmp_str, r->src_mask );
2149		else
2150		snprintf(string, MAX_STRING_LENGTH-1, "%s%16s/%-2u", tag_string, tmp_str, r->src_mask );
2151
2152		string[MAX_STRING_LENGTH-1] = 0;
2153
2154
2155		} // End of String_SrcNet
2156
2157		static void String_DstNet(master_record_t r, char string) {
2158		char tmp_str[IP_STRING_LEN];
2159
2160		ApplyNetMaskBits(r, 2);
2161
2162		tmp_str[0] = 0;
2163		if ( (r->flags & FLAG_IPV6_ADDR ) != 0 ) { // IPv6
2164		uint64_t ip[2];
2165
2166		ip[0] = htonll(r->V6.dstaddr[0]);
2167		ip[1] = htonll(r->V6.dstaddr[1]);
2168		inet_ntop(AF_INET6, ip, tmp_str, sizeof(tmp_str));
2169		if ( ! long_v6 ) {
2170		condense_v6(tmp_str);
2171		}
2172		} else { // IPv4
2173		uint32_t ip;
2174		ip = htonl(r->V4.dstaddr);
2175		inet_ntop(AF_INET, &ip, tmp_str, sizeof(tmp_str));
2176		}
2177		tmp_str[IP_STRING_LEN-1] = 0;
2178		if ( long_v6 )
2179		snprintf(string, MAX_STRING_LENGTH-1, "%s%39s/%-2u", tag_string, tmp_str, r->dst_mask );
2180		else
2181		snprintf(string, MAX_STRING_LENGTH-1, "%s%16s/%-2u", tag_string, tmp_str, r->dst_mask );
2182
2183		string[MAX_STRING_LENGTH-1] = 0;
2184
2185
2186		} // End of String_DstNet
2187
2188		static void String_SrcPort(master_record_t r, char string) {
2189
2190		snprintf(string, MAX_STRING_LENGTH-1 ,"%6u", r->srcport);
2191		string[MAX_STRING_LENGTH-1] = '\0';
2192
2193		} // End of String_SrcPort
2194
2195		static void String_DstPort(master_record_t r, char string) {
2196		char tmp[MAX_STRING_LENGTH];
2197
2198		ICMP_Port_decode(r, tmp);
2199		snprintf(string, MAX_STRING_LENGTH-1 ,"%6s", tmp);
2200		string[MAX_STRING_LENGTH-1] = '\0';
2201
2202		} // End of String_DstPort
2203
2204		static void String_ICMP_type(master_record_t r, char string) {
2205		uint8_t type;
2206
2207		type = ( r->prot == IPPROTO_ICMP \|\| r->prot == IPPROTO_ICMPV6 ) ? r->icmp_type : 0;
2208		snprintf(string, MAX_STRING_LENGTH-1, "%6u", type);
2209		string[MAX_STRING_LENGTH-1] = 0;
2210
2211		} // End of String_ICMP_type
2212
2213		static void String_ICMP_code(master_record_t r, char string) {
2214		uint8_t code;
2215
2216		code = ( r->prot == IPPROTO_ICMP \|\| r->prot == IPPROTO_ICMPV6 ) ? r->icmp_code : 0;
2217		snprintf(string, MAX_STRING_LENGTH-1, "%6u", code);
2218		string[MAX_STRING_LENGTH-1] = 0;
2219
2220		} // End of String_ICMP_code
2221
2222		static void String_SrcAS(master_record_t r, char string) {
2223
2224		snprintf(string, MAX_STRING_LENGTH-1 ,"%6u", r->srcas);
2225		string[MAX_STRING_LENGTH-1] = '\0';
2226
2227		} // End of String_SrcAS
2228
2229		static void String_DstAS(master_record_t r, char string) {
2230
2231		snprintf(string, MAX_STRING_LENGTH-1 ,"%6u", r->dstas);
2232		string[MAX_STRING_LENGTH-1] = '\0';
2233
2234		} // End of String_DstAS
2235
2236		static void String_NextAS(master_record_t r, char string) {
2237
2238		snprintf(string, MAX_STRING_LENGTH-1 ," %6u", r->bgpNextAdjacentAS);
2239		string[MAX_STRING_LENGTH-1] = '\0';
2240
2241		} // End of String_NextAS
2242
2243		static void String_PrevAS(master_record_t r, char string) {
2244
2245		snprintf(string, MAX_STRING_LENGTH-1 ," %6u", r->bgpPrevAdjacentAS);
2246		string[MAX_STRING_LENGTH-1] = '\0';
2247
2248		} // End of String_PrevAS
2249
2250		static void String_Input(master_record_t r, char string) {
2251
2252		snprintf(string, MAX_STRING_LENGTH-1 ,"%6u", r->input);
2253		string[MAX_STRING_LENGTH-1] = '\0';
2254
2255		} // End of String_Input
2256
2257		static void String_Output(master_record_t r, char string) {
2258
2259		snprintf(string, MAX_STRING_LENGTH-1 ,"%6u", r->output);
2260		string[MAX_STRING_LENGTH-1] = '\0';
2261
2262		} // End of String_Output
2263
2264		static void String_InPackets(master_record_t r, char string) {
2265		char s[NUMBER_STRING_SIZE];
2266
2267		format_number(r->dPkts, s, scale, FIXED_WIDTH);
2268		snprintf(string, MAX_STRING_LENGTH-1 ,"%8s", s);
2269		string[MAX_STRING_LENGTH-1] = '\0';
2270
2271		} // End of String_InPackets
2272
2273		static void String_OutPackets(master_record_t r, char string) {
2274		char s[NUMBER_STRING_SIZE];
2275
2276		format_number(r->out_pkts, s, scale, FIXED_WIDTH);
2277		snprintf(string, MAX_STRING_LENGTH-1 ,"%8s", s);
2278		string[MAX_STRING_LENGTH-1] = '\0';
2279
2280		} // End of String_OutPackets
2281
2282		static void String_InBytes(master_record_t r, char string) {
2283		char s[NUMBER_STRING_SIZE];
2284
2285		format_number(r->dOctets, s, scale, FIXED_WIDTH);
2286		snprintf(string, MAX_STRING_LENGTH-1 ,"%8s", s);
2287		string[MAX_STRING_LENGTH-1] = '\0';
2288
2289		} // End of String_InBytes
2290
2291		static void String_OutBytes(master_record_t r, char string) {
2292		char s[NUMBER_STRING_SIZE];
2293
2294		format_number(r->out_bytes, s, scale, FIXED_WIDTH);
2295		snprintf(string, MAX_STRING_LENGTH-1 ,"%8s", s);
2296		string[MAX_STRING_LENGTH-1] = '\0';
2297
2298		} // End of String_OutBytes
2299
2300		static void String_Flows(master_record_t r, char string) {
2301
2302		// snprintf(string, MAX_STRING_LENGTH-1 ,"%5llu", r->aggr_flows ? (unsigned long long)r->aggr_flows : 1 );
2303		snprintf(string, MAX_STRING_LENGTH-1 ,"%5llu", (unsigned long long)r->aggr_flows );
2304		string[MAX_STRING_LENGTH-1] = '\0';
2305
2306		} // End of String_Flows
2307
2308		static void String_Tos(master_record_t r, char string) {
2309
2310		snprintf(string, MAX_STRING_LENGTH-1 ,"%3u", r->tos);
2311		string[MAX_STRING_LENGTH-1] = '\0';
2312
2313		} // End of String_Tos
2314
2315		static void String_SrcTos(master_record_t r, char string) {
2316
2317		snprintf(string, MAX_STRING_LENGTH-1 ,"%4u", r->tos);
2318		string[MAX_STRING_LENGTH-1] = '\0';
2319
2320		} // End of String_SrcTos
2321
2322		static void String_DstTos(master_record_t r, char string) {
2323
2324		snprintf(string, MAX_STRING_LENGTH-1 ,"%4u", r->dst_tos);
2325		string[MAX_STRING_LENGTH-1] = '\0';
2326
2327		} // End of String_DstTos
2328
2329		static void String_SrcMask(master_record_t r, char string) {
2330
2331		snprintf(string, MAX_STRING_LENGTH-1 ,"%5u", r->src_mask);
2332		string[MAX_STRING_LENGTH-1] = '\0';
2333
2334		} // End of String_SrcMask
2335
2336		static void String_DstMask(master_record_t r, char string) {
2337
2338		snprintf(string, MAX_STRING_LENGTH-1 ,"%5u", r->dst_mask);
2339		string[MAX_STRING_LENGTH-1] = '\0';
2340
2341		} // End of String_DstMask
2342
2343		static void String_SrcVlan(master_record_t r, char string) {
2344
2345		snprintf(string, MAX_STRING_LENGTH-1 ,"%5u", r->src_vlan);
2346		string[MAX_STRING_LENGTH-1] = '\0';
2347
2348		} // End of String_SrcVlan
2349
2350		static void String_DstVlan(master_record_t r, char string) {
2351
2352		snprintf(string, MAX_STRING_LENGTH-1 ,"%5u", r->dst_vlan);
2353		string[MAX_STRING_LENGTH-1] = '\0';
2354
2355		} // End of String_DstVlan
2356
2357		static void String_Dir(master_record_t r, char string) {
2358
2359		snprintf(string, MAX_STRING_LENGTH-1 ,"%3c", r->dir ? 'E' : 'I' );
2360		string[MAX_STRING_LENGTH-1] = '\0';
2361
2362		} // End of String_Dir
2363
2364		static void String_FwdStatus(master_record_t r, char string) {
2365
2366		snprintf(string, MAX_STRING_LENGTH-1 ,"%3u", r->fwd_status);
2367		string[MAX_STRING_LENGTH-1] = '\0';
2368
2369		} // End of String_FwdStatus
2370
2371		static void String_Flags(master_record_t r, char string) {
2372
2373		string[0] = r->tcp_flags & 128 ? 'C' : '.'; // Congestion window reduced - CWR
2374		string[1] = r->tcp_flags & 64 ? 'E' : '.'; // ECN-Echo
2375		string[2] = r->tcp_flags & 32 ? 'U' : '.'; // Urgent
2376		string[3] = r->tcp_flags & 16 ? 'A' : '.'; // Ack
2377		string[4] = r->tcp_flags & 8 ? 'P' : '.'; // Push
2378		string[5] = r->tcp_flags & 4 ? 'R' : '.'; // Reset
2379		string[6] = r->tcp_flags & 2 ? 'S' : '.'; // Syn
2380		string[7] = r->tcp_flags & 1 ? 'F' : '.'; // Fin
2381		string[8] = '\0';
2382
2383		} // End of String_Flags
2384
2385		static void String_InSrcMac(master_record_t r, char string) {
2386		int i;
2387		uint8_t mac[6];
2388
2389		for ( i=0; i<6; i++ ) {
2390		mac[i] = (r->in_src_mac >> ( i*8 )) & 0xFF;
2391		}
2392		snprintf(string, MAX_STRING_LENGTH-1 ,"%.2x:%.2x:%.2x:%.2x:%.2x:%.2x", mac[5], mac[4], mac[3], mac[2], mac[1], mac[0]);
2393		string[MAX_STRING_LENGTH-1] = '\0';
2394
2395		} // End of String_InSrcMac
2396
2397		static void String_OutDstMac(master_record_t r, char string) {
2398		int i;
2399		uint8_t mac[6];
2400
2401		for ( i=0; i<6; i++ ) {
2402		mac[i] = (r->out_dst_mac >> ( i*8 )) & 0xFF;
2403		}
2404		snprintf(string, MAX_STRING_LENGTH-1 ,"%.2x:%.2x:%.2x:%.2x:%.2x:%.2x", mac[5], mac[4], mac[3], mac[2], mac[1], mac[0]);
2405		string[MAX_STRING_LENGTH-1] = '\0';
2406
2407		} // End of String_OutDstMac
2408
2409		static void String_InDstMac(master_record_t r, char string) {
2410		int i;
2411		uint8_t mac[6];
2412
2413		for ( i=0; i<6; i++ ) {
2414		mac[i] = (r->in_dst_mac >> ( i*8 )) & 0xFF;
2415		}
2416		snprintf(string, MAX_STRING_LENGTH-1 ,"%.2x:%.2x:%.2x:%.2x:%.2x:%.2x", mac[5], mac[4], mac[3], mac[2], mac[1], mac[0]);
2417		string[MAX_STRING_LENGTH-1] = '\0';
2418
2419		} // End of String_InDstMac
2420
2421		static void String_OutSrcMac(master_record_t r, char string) {
2422		int i;
2423		uint8_t mac[6];
2424
2425		for ( i=0; i<6; i++ ) {
2426		mac[i] = (r->out_src_mac >> ( i*8 )) & 0xFF;
2427		}
2428		snprintf(string, MAX_STRING_LENGTH-1 ,"%.2x:%.2x:%.2x:%.2x:%.2x:%.2x", mac[5], mac[4], mac[3], mac[2], mac[1], mac[0]);
2429		string[MAX_STRING_LENGTH-1] = '\0';
2430
2431		} // End of String_OutSrcMac
2432
2433		static void String_MPLS_1(master_record_t r, char string) {
2434
2435		snprintf(string, MAX_STRING_LENGTH-1 ,"%8u-%1u-%1u",
2436		r->mpls_label[0] >> 4 , (r->mpls_label[0] & 0xF ) >> 1, r->mpls_label[0] & 1);
2437		string[MAX_STRING_LENGTH-1] = '\0';
2438
2439		} // End of String_MPLS
2440
2441		static void String_MPLS_2(master_record_t r, char string) {
2442
2443		snprintf(string, MAX_STRING_LENGTH-1 ,"%8u-%1u-%1u",
2444		r->mpls_label[1] >> 4 , (r->mpls_label[1] & 0xF ) >> 1, r->mpls_label[1] & 1);
2445		string[MAX_STRING_LENGTH-1] = '\0';
2446
2447		} // End of String_MPLS
2448
2449		static void String_MPLS_3(master_record_t r, char string) {
2450
2451		snprintf(string, MAX_STRING_LENGTH-1 ,"%8u-%1u-%1u",
2452		r->mpls_label[2] >> 4 , (r->mpls_label[2] & 0xF ) >> 1, r->mpls_label[2] & 1);
2453		string[MAX_STRING_LENGTH-1] = '\0';
2454
2455		} // End of String_MPLS
2456
2457		static void String_MPLS_4(master_record_t r, char string) {
2458
2459		snprintf(string, MAX_STRING_LENGTH-1 ,"%8u-%1u-%1u",
2460		r->mpls_label[3] >> 4 , (r->mpls_label[3] & 0xF ) >> 1, r->mpls_label[3] & 1);
2461		string[MAX_STRING_LENGTH-1] = '\0';
2462
2463		} // End of String_MPLS
2464
2465		static void String_MPLS_5(master_record_t r, char string) {
2466
2467		snprintf(string, MAX_STRING_LENGTH-1 ,"%8u-%1u-%1u",
2468		r->mpls_label[4] >> 4 , (r->mpls_label[4] & 0xF ) >> 1, r->mpls_label[4] & 1);
2469		string[MAX_STRING_LENGTH-1] = '\0';
2470
2471		} // End of String_MPLS
2472
2473		static void String_MPLS_6(master_record_t r, char string) {
2474
2475		snprintf(string, MAX_STRING_LENGTH-1 ,"%8u-%1u-%1u",
2476		r->mpls_label[5] >> 4 , (r->mpls_label[5] & 0xF ) >> 1, r->mpls_label[5] & 1);
2477		string[MAX_STRING_LENGTH-1] = '\0';
2478
2479		} // End of String_MPLS
2480
2481		static void String_MPLS_7(master_record_t r, char string) {
2482
2483		snprintf(string, MAX_STRING_LENGTH-1 ,"%8u-%1u-%1u",
2484		r->mpls_label[6] >> 4 , (r->mpls_label[6] & 0xF ) >> 1, r->mpls_label[6] & 1);
2485		string[MAX_STRING_LENGTH-1] = '\0';
2486
2487		} // End of String_MPLS
2488
2489		static void String_MPLS_8(master_record_t r, char string) {
2490
2491		snprintf(string, MAX_STRING_LENGTH-1 ,"%8u-%1u-%1u",
2492		r->mpls_label[7] >> 4 , (r->mpls_label[7] & 0xF ) >> 1, r->mpls_label[7] & 1);
2493		string[MAX_STRING_LENGTH-1] = '\0';
2494
2495		} // End of String_MPLS
2496
2497		static void String_MPLS_9(master_record_t r, char string) {
2498
2499		snprintf(string, MAX_STRING_LENGTH-1 ,"%8u-%1u-%1u",
2500		r->mpls_label[8] >> 4 , (r->mpls_label[8] & 0xF ) >> 1, r->mpls_label[8] & 1);
2501		string[MAX_STRING_LENGTH-1] = '\0';
2502
2503		} // End of String_MPLS
2504
2505		static void String_MPLS_10(master_record_t r, char string) {
2506
2507		snprintf(string, MAX_STRING_LENGTH-1 ,"%8u-%1u-%1u",
2508		r->mpls_label[9] >> 4 , (r->mpls_label[9] & 0xF ) >> 1, r->mpls_label[9] & 1);
2509		string[MAX_STRING_LENGTH-1] = '\0';
2510
2511		} // End of String_MPLS
2512
2513		static void String_MPLSs(master_record_t r, char string) {
2514
2515		snprintf(string, MAX_STRING_LENGTH-1 ,"%8u-%1u-%1u %8u-%1u-%1u %8u-%1u-%1u %8u-%1u-%1u %8u-%1u-%1u %8u-%1u-%1u %8u-%1u-%1u %8u-%1u-%1u %8u-%1u-%1u %8u-%1u-%1u ",
2516		r->mpls_label[0] >> 4 , (r->mpls_label[0] & 0xF ) >> 1, r->mpls_label[0] & 1,
2517		r->mpls_label[1] >> 4 , (r->mpls_label[1] & 0xF ) >> 1, r->mpls_label[1] & 1,
2518		r->mpls_label[2] >> 4 , (r->mpls_label[2] & 0xF ) >> 1, r->mpls_label[2] & 1,
2519		r->mpls_label[3] >> 4 , (r->mpls_label[3] & 0xF ) >> 1, r->mpls_label[3] & 1,
2520		r->mpls_label[4] >> 4 , (r->mpls_label[4] & 0xF ) >> 1, r->mpls_label[4] & 1,
2521		r->mpls_label[5] >> 4 , (r->mpls_label[5] & 0xF ) >> 1, r->mpls_label[5] & 1,
2522		r->mpls_label[6] >> 4 , (r->mpls_label[6] & 0xF ) >> 1, r->mpls_label[6] & 1,
2523		r->mpls_label[7] >> 4 , (r->mpls_label[7] & 0xF ) >> 1, r->mpls_label[7] & 1,
2524		r->mpls_label[8] >> 4 , (r->mpls_label[8] & 0xF ) >> 1, r->mpls_label[8] & 1,
2525		r->mpls_label[9] >> 4 , (r->mpls_label[9] & 0xF ) >> 1, r->mpls_label[9] & 1
2526		);
2527		string[MAX_STRING_LENGTH-1] = '\0';
2528
2529		} // End of String_MPLSs
2530
2531		static void String_Engine(master_record_t r, char string) {
2532
2533		snprintf(string, MAX_STRING_LENGTH-1 ,"%3u/%-3u", r->engine_type, r->engine_id);
2534		string[MAX_STRING_LENGTH-1] = '\0';
2535
2536		} // End of String_Engine
2537
2538		static void String_Label(master_record_t r, char string) {
2539
2540		if ( r->label )
2541		snprintf(string, MAX_STRING_LENGTH-1 ,"%16s", r->label);
2542		else
2543		snprintf(string, MAX_STRING_LENGTH-1 ,"<none>");
2544
2545		string[MAX_STRING_LENGTH-1] = '\0';
2546
2547		} // End of String_Label
2548
2549		static void String_ClientLatency(master_record_t r, char string) {
2550		double latency;
2551
2552		latency = (double)r->client_nw_delay_usec / 1000.0;
2553		snprintf(string, MAX_STRING_LENGTH-1 ,"%9.3f", latency);
2554		string[MAX_STRING_LENGTH-1] = '\0';
2555
2556		} // End of String_ClientLatency
2557
2558		static void String_ServerLatency(master_record_t r, char string) {
2559		double latency;
2560
2561		latency = (double)r->server_nw_delay_usec / 1000.0;
2562		snprintf(string, MAX_STRING_LENGTH-1 ,"%9.3f", latency);
2563		string[MAX_STRING_LENGTH-1] = '\0';
2564
2565		} // End of String_ServerLatency
2566
2567		static void String_AppLatency(master_record_t r, char string) {
2568		double latency;
2569
2570		latency = (double)r->appl_latency_usec / 1000.0;
2571		snprintf(string, MAX_STRING_LENGTH-1 ,"%9.3f", latency);
2572		string[MAX_STRING_LENGTH-1] = '\0';
2573
2574		} // End of String_AppLatency
2575
2576		static void String_bps(master_record_t r, char string) {
2577		uint64_t bps;
2578		char s[NUMBER_STRING_SIZE];
2579
2580		if ( duration ) {
2581		bps = (( r->dOctets << 3 ) / duration); // bits per second. ( >> 3 ) -> * 8 to convert octets into bits
2582		} else {
2583		bps = 0;
2584		}
2585		format_number(bps, s, scale, FIXED_WIDTH);
2586		snprintf(string, MAX_STRING_LENGTH-1 ,"%8s", s);
2587		string[MAX_STRING_LENGTH-1] = '\0';
2588
2589		} // End of String_bps
2590
2591		static void String_pps(master_record_t r, char string) {
2592		uint64_t pps;
2593		char s[NUMBER_STRING_SIZE];
2594
2595		if ( duration ) {
2596		pps = r->dPkts / duration; // packets per second
2597		} else {
2598		pps = 0;
2599		}
2600		format_number(pps, s, scale, FIXED_WIDTH);
2601		snprintf(string, MAX_STRING_LENGTH-1 ,"%8s", s);
2602		string[MAX_STRING_LENGTH-1] = '\0';
2603
2604		} // End of String_Duration
2605
2606		static void String_bpp(master_record_t r, char string) {
2607		uint32_t Bpp;
2608
2609		string[MAX_STRING_LENGTH-1] = '\0';
2610
2611		if ( r->dPkts )
2612		Bpp = r->dOctets / r->dPkts; // Bytes per Packet
2613		else
2614		Bpp = 0;
2615		snprintf(string, MAX_STRING_LENGTH-1 ,"%6u", Bpp);
2616		string[MAX_STRING_LENGTH-1] = '\0';
2617
2618		} // End of String_bpp
2619
2620		static void String_ExpSysID(master_record_t r, char string) {
2621
2622		string[MAX_STRING_LENGTH-1] = '\0';
2623
2624		snprintf(string, MAX_STRING_LENGTH-1 ,"%6u", r->exporter_sysid);
2625		string[MAX_STRING_LENGTH-1] = '\0';
2626
2627		} // End of String_ExpSysID
2628
2629		#ifdef NSEL
2630		static void String_nfc(master_record_t r, char string) {
2631
2632		snprintf(string, MAX_STRING_LENGTH-1, "%10u", r->conn_id);
2633		string[MAX_STRING_LENGTH-1] = '\0';
2634
2635		} // End of String_nfc
2636
2637		static void String_evt(master_record_t r, char string) {
2638
2639		if ( r->event_flag == FW_EVENT ) {
2640		switch(r->event) {
2641		case 0:
2642		snprintf(string, MAX_STRING_LENGTH-1 ,"%3s", "IGNORE");
2643		break;
2644		case 1:
2645		snprintf(string, MAX_STRING_LENGTH-1 ,"%6s", "CREATE");
2646		break;
2647		case 2:
2648		snprintf(string, MAX_STRING_LENGTH-1 ,"%6s", "DELETE");
2649		break;
2650		case 3:
2651		snprintf(string, MAX_STRING_LENGTH-1 ,"%6s", "DENIED");
2652		break;
2653		case 4:
2654		snprintf(string, MAX_STRING_LENGTH-1 ,"%6s", "ALERT");
2655		break;
2656		case 5:
2657		snprintf(string, MAX_STRING_LENGTH-1 ,"%6s", "UPDATE");
2658		break;
2659		default:
2660		snprintf(string, MAX_STRING_LENGTH-1 ,"%6s", "UNKNOW");
2661		}
2662		} else {
2663		switch(r->event) {
2664		case 0:
2665		snprintf(string, MAX_STRING_LENGTH-1 ,"%3s", "INVALID");
2666		break;
2667		case 1:
2668		snprintf(string, MAX_STRING_LENGTH-1 ,"%6s", "ADD");
2669		break;
2670		case 2:
2671		snprintf(string, MAX_STRING_LENGTH-1 ,"%6s", "DELETE");
2672		break;
2673		default:
2674		snprintf(string, MAX_STRING_LENGTH-1 ,"%6s", "UNKNOW");
2675		}
2676		}
2677		string[MAX_STRING_LENGTH-1] = '\0';
2678
2679		} // End of String_evt
2680
2681
2682		static void String_xevt(master_record_t r, char string) {
2683
2684		switch( r->fw_xevent) {
2685		case 0:
2686		snprintf(string, MAX_STRING_LENGTH-1 ,"%7s", "Ignore");
2687		break;
2688		case 1001:
2689		snprintf(string,MAX_STRING_LENGTH-1,"%7s","I-ACL");
2690		break;
2691		case 1002:
2692		snprintf(string,MAX_STRING_LENGTH-1,"%7s","E-ACL");
2693		break;
2694		case 1003:
2695		snprintf(string,MAX_STRING_LENGTH-1,"%7s","Adap");
2696		break;
2697		case 1004:
2698		snprintf(string,MAX_STRING_LENGTH-1,"%7s","No Syn");
2699		break;
2700		default:
2701		snprintf(string,MAX_STRING_LENGTH-1,"%7u",r->fw_xevent);
2702		}
2703		string[MAX_STRING_LENGTH-1] = '\0';
2704
2705		} // End of String_xevt
2706
2707		static void String_msec(master_record_t r, char string) {
2708		unsigned long long etime;
2709
2710		etime = 1000LL * (unsigned long long)r->first + (unsigned long long)r->msec_first;
2711		snprintf(string, MAX_STRING_LENGTH-1,"%13llu", etime);
2712		string[MAX_STRING_LENGTH-1] = '\0';
2713
2714		} // End of String_msec
2715
2716		static void String_iacl(master_record_t r, char string) {
2717
2718		snprintf(string, MAX_STRING_LENGTH-1, "0x%-8x 0x%-8x 0x%-8x",
2719		r->ingress_acl_id[0], r->ingress_acl_id[1], r->ingress_acl_id[2]);
2720		string[MAX_STRING_LENGTH-1] = 0;
2721
2722		} // End of String_iacl
2723
2724		static void String_eacl(master_record_t r, char string) {
2725
2726		snprintf(string, MAX_STRING_LENGTH-1, "%10u %10u %10u",
2727		r->egress_acl_id[0], r->egress_acl_id[1], r->egress_acl_id[2]);
2728
2729		string[MAX_STRING_LENGTH-1] = 0;
2730
2731		} // End of String_eacl
2732
2733		static void String_xlateSrcAddr(master_record_t r, char string) {
2734		char tmp_str[IP_STRING_LEN];
2735
2736		tmp_str[0] = 0;
2737		if ( (r->xlate_flags & 1 ) != 0 ) { // IPv6
2738		uint64_t ip[2];
2739
2740		ip[0] = htonll(r->xlate_src_ip.V6[0]);
2741		ip[1] = htonll(r->xlate_src_ip.V6[1]);
2742		inet_ntop(AF_INET6, ip, tmp_str, sizeof(tmp_str));
2743		if ( ! long_v6 ) {
2744		condense_v6(tmp_str);
2745		}
2746		} else { // IPv4
2747		uint32_t ip;
2748		ip = htonl(r->xlate_src_ip.V4);
2749		inet_ntop(AF_INET, &ip, tmp_str, sizeof(tmp_str));
2750		}
2751		tmp_str[IP_STRING_LEN-1] = 0;
2752		if ( long_v6 )
2753		snprintf(string, MAX_STRING_LENGTH-1, "%s%39s", tag_string, tmp_str);
2754		else
2755		snprintf(string, MAX_STRING_LENGTH-1, "%s%16s", tag_string, tmp_str);
2756
2757		string[MAX_STRING_LENGTH-1] = 0;
2758
2759		} // End of String_xlateSrcAddr
2760
2761		static void String_xlateDstAddr(master_record_t r, char string) {
2762		char tmp_str[IP_STRING_LEN];
2763
2764		tmp_str[0] = 0;
2765		if ( (r->xlate_flags & 1 ) != 0 ) { // IPv6
2766		uint64_t ip[2];
2767
2768		ip[0] = htonll(r->xlate_dst_ip.V6[0]);
2769		ip[1] = htonll(r->xlate_dst_ip.V6[1]);
2770		inet_ntop(AF_INET6, ip, tmp_str, sizeof(tmp_str));
2771		if ( ! long_v6 ) {
2772		condense_v6(tmp_str);
2773		}
2774		} else { // IPv4
2775		uint32_t ip;
2776		ip = htonl(r->xlate_dst_ip.V4);
2777		inet_ntop(AF_INET, &ip, tmp_str, sizeof(tmp_str));
2778		}
2779		tmp_str[IP_STRING_LEN-1] = 0;
2780		if ( long_v6 )
2781		snprintf(string, MAX_STRING_LENGTH-1, "%s%39s", tag_string, tmp_str);
2782		else
2783		snprintf(string, MAX_STRING_LENGTH-1, "%s%16s", tag_string, tmp_str);
2784
2785		string[MAX_STRING_LENGTH-1] = 0;
2786
2787		} // End of String_xlateDstAddr
2788
2789		static void String_xlateSrcPort(master_record_t r, char string) {
2790
2791		snprintf(string, MAX_STRING_LENGTH-1 ,"%6u", r->xlate_src_port);
2792		string[MAX_STRING_LENGTH-1] = '\0';
2793
2794		} // End of String_xlateSrcPort
2795
2796		static void String_xlateDstPort(master_record_t r, char string) {
2797
2798		snprintf(string, MAX_STRING_LENGTH-1 ,"%6u", r->xlate_dst_port);
2799		string[MAX_STRING_LENGTH-1] = '\0';
2800
2801		} // End of String_xlateDstPort
2802
2803		static void String_xlateSrcAddrPort(master_record_t r, char string) {
2804		char tmp_str[IP_STRING_LEN], portchar;
2805
2806		tmp_str[0] = 0;
2807		if ( (r->xlate_flags & 1 ) != 0 ) { // IPv6
2808		uint64_t ip[2];
2809
2810		ip[0] = htonll(r->xlate_src_ip.V6[0]);
2811		ip[1] = htonll(r->xlate_src_ip.V6[1]);
2812		inet_ntop(AF_INET6, ip, tmp_str, sizeof(tmp_str));
2813		if ( ! long_v6 ) {
2814		condense_v6(tmp_str);
2815		}
2816
2817		portchar = '.';
2818		} else { // IPv4
2819		uint32_t ip;
2820		ip = htonl(r->xlate_src_ip.V4);
2821		inet_ntop(AF_INET, &ip, tmp_str, sizeof(tmp_str));
2822
2823		portchar = ':';
2824		}
2825		tmp_str[IP_STRING_LEN-1] = 0;
2826
2827		if ( long_v6 )
2828		snprintf(string, MAX_STRING_LENGTH-1, "%s%39s%c%-5i", tag_string, tmp_str, portchar, r->xlate_src_port);
2829		else
2830		snprintf(string, MAX_STRING_LENGTH-1, "%s%16s%c%-5i", tag_string, tmp_str, portchar, r->xlate_src_port);
2831
2832		string[MAX_STRING_LENGTH-1] = 0;
2833
2834		} // End of String_xlateSrcAddrPort
2835
2836		static void String_xlateDstAddrPort(master_record_t r, char string) {
2837		char tmp_str[IP_STRING_LEN], portchar;
2838
2839		tmp_str[0] = 0;
2840		if ( (r->xlate_flags & 1 ) != 0 ) { // IPv6
2841		uint64_t ip[2];
2842
2843		ip[0] = htonll(r->xlate_dst_ip.V6[0]);
2844		ip[1] = htonll(r->xlate_dst_ip.V6[1]);
2845		inet_ntop(AF_INET6, ip, tmp_str, sizeof(tmp_str));
2846		if ( ! long_v6 ) {
2847		condense_v6(tmp_str);
2848		}
2849
2850		portchar = '.';
2851		} else { // IPv4
2852		uint32_t ip;
2853		ip = htonl(r->xlate_dst_ip.V4);
2854		inet_ntop(AF_INET, &ip, tmp_str, sizeof(tmp_str));
2855
2856		portchar = ':';
2857		}
2858		tmp_str[IP_STRING_LEN-1] = 0;
2859
2860		if ( long_v6 )
2861		snprintf(string, MAX_STRING_LENGTH-1, "%s%39s%c%-5i", tag_string, tmp_str, portchar, r->xlate_dst_port);
2862		else
2863		snprintf(string, MAX_STRING_LENGTH-1, "%s%16s%c%-5i", tag_string, tmp_str, portchar, r->xlate_dst_port);
2864
2865		string[MAX_STRING_LENGTH-1] = 0;
2866
2867
2868		} // End of String_xlateDstAddrPort
2869
2870		static void String_userName(master_record_t r, char string) {
2871
2872		if ( r->username[0] == '\0' )
2873		snprintf(string, MAX_STRING_LENGTH-1 ,"%s", "<empty>");
2874		else
2875		snprintf(string, MAX_STRING_LENGTH-1 ,"%s", r->username);
2876
2877		string[MAX_STRING_LENGTH-1] = '\0';
2878
2879		} // End of String_userName
2880
2881		static void String_ivrf(master_record_t r, char string) {
2882
2883		snprintf(string, MAX_STRING_LENGTH-1, "%10u", r->ingress_vrfid);
2884		string[MAX_STRING_LENGTH-1] = '\0';
2885
2886		} // End of String_ivrf
2887
2888		static void String_evrf(master_record_t r, char string) {
2889
2890		snprintf(string, MAX_STRING_LENGTH-1, "%10u", r->egress_vrfid);
2891		string[MAX_STRING_LENGTH-1] = '\0';
2892
2893		} // End of String_evrf
2894
2895		static void String_PortBlockStart(master_record_t r, char string) {
2896
2897		snprintf(string, MAX_STRING_LENGTH-1 ,"%7u", r->block_start);
2898		string[MAX_STRING_LENGTH-1] = '\0';
2899
2900		} // End of String_PortBlockStart
2901
2902		static void String_PortBlockEnd(master_record_t r, char string) {
2903
2904		snprintf(string, MAX_STRING_LENGTH-1 ,"%7u", r->block_end);
2905		string[MAX_STRING_LENGTH-1] = '\0';
2906
2907		} // End of String_PortBlockEnd
2908
2909		static void String_PortBlockStep(master_record_t r, char string) {
2910
2911		snprintf(string, MAX_STRING_LENGTH-1 ,"%7u", r->block_step);
2912		string[MAX_STRING_LENGTH-1] = '\0';
2913
2914		} // End of String_PortBlockStep
2915
2916		static void String_PortBlockSize(master_record_t r, char string) {
2917
2918		snprintf(string, MAX_STRING_LENGTH-1 ,"%7u", r->block_size);
2919		string[MAX_STRING_LENGTH-1] = '\0';
2920
2921		} // End of String_PortBlockSize
2922
2923
2924		#endif

-123

~~bin/nf_common.h~~ less more

0		/*
1		* Copyright (c) 2017, Peter Haag
2		* Copyright (c) 2014, Peter Haag
3		* Copyright (c) 2009, Peter Haag
4		* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
5		* All rights reserved.
6		*
7		* Redistribution and use in source and binary forms, with or without
8		* modification, are permitted provided that the following conditions are met:
9		*
10		* * Redistributions of source code must retain the above copyright notice,
11		* this list of conditions and the following disclaimer.
12		* * Redistributions in binary form must reproduce the above copyright notice,
13		* this list of conditions and the following disclaimer in the documentation
14		* and/or other materials provided with the distribution.
15		* * Neither the name of the author nor the names of its contributors may be
16		* used to endorse or promote products derived from this software without
17		* specific prior written permission.
18		*
19		* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
20		* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
21		* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
22		* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
23		* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
24		* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
25		* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
26		* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
27		* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
28		* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
29		* POSSIBILITY OF SUCH DAMAGE.
30		*
31		*/
32
33		#ifndef _NF_COMMON_H
34		#define _NF_COMMON_H 1
35
36		#include "config.h"
37
38		#include <sys/types.h>
39		#ifdef HAVE_STDINT_H
40		#include <stdint.h>
41		#endif
42		#include <time.h>
43
44		typedef void (printer_t)(void , char **, int);
45
46		#if ( SIZEOF_VOID_P == 8 )
47		typedef uint64_t pointer_addr_t;
48		#else
49		typedef uint32_t pointer_addr_t;
50		#endif
51
52		typedef struct msec_time_s {
53		time_t sec;
54		uint16_t msec;
55		} msec_time_tt;
56
57		/* common minimum netflow header for all versions */
58		typedef struct common_flow_header {
59		uint16_t version;
60		uint16_t count;
61		} common_flow_header_t;
62
63		typedef struct printmap_s {
64		char *printmode; // name of the output format
65		printer_t func; // name of the function, which prints the record
66		char *Format; // output format definition
67		} printmap_t;
68
69		#define NSEL_EVENT_IGNORE 0LL
70		#define NSEL_EVENT_CREATE 1LL
71		#define NSEL_EVENT_DELETE 2LL
72		#define NSEL_EVENT_DENIED 3LL
73		#define NSEL_EVENT_ALERT 4LL
74		#define NSEL_EVENT_UPDATE 5LL
75
76		#define NEL_EVENT_INVALID 0LL
77		#define NEL_EVENT_ADD 1LL
78		#define NEL_EVENT_DELETE 2LL
79
80		/* prototypes */
81
82		int InitSymbols(void);
83
84		void Setv6Mode(int mode);
85
86		int Getv6Mode(void);
87
88		int Proto_num(char *protostr);
89
90		void format_file_block_header(void header, char *s, int tag);
91
92		char *format_csv_header(void);
93
94		char *get_record_header(void);
95
96		void set_record_header(void);
97
98		void format_file_block_record(void record, char *s, int tag);
99
100		void flow_record_to_pipe(void record, char * s, int tag);
101
102		void flow_record_to_csv(void record, char * s, int tag);
103
104		void flow_record_to_null(void record, char * s, int tag);
105
106		int ParseOutputFormat(char format, int plain_numbers, printmap_t printmap);
107
108		void format_special(void record, char * s, int tag);
109
110
111		uint32_t Get_fwd_status_id(char *status);
112
113		char *Get_fwd_status_name(uint32_t id);
114
115		void Proto_string(uint8_t protonum, char *protostr);
116
117		void condense_v6(char *s);
118
119		#define TAG_CHAR ''
120
121		#endif //_NF_COMMON_H
122

-16

bin/nfanon.c less more

0	0	/*
1		* Copyright (c) 2009-2019, Peter Haag
	1	* Copyright (c) 2009-2020, Peter Haag
2	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
3	3	* All rights reserved.
4	4	*

50	50	#include <stdint.h>
51	51	#endif
52	52
	53	#include "util.h"
	54	#include "nfdump.h"
53	55	#include "nffile.h"
54	56	#include "nfx.h"
55		#include "util.h"
	57	#include "exporter.h"
56	58	#include "flist.h"
57	59	#include "panonymizer.h"
58
59		#if ( SIZEOF_VOID_P == 8 )
60		typedef uint64_t pointer_addr_t;
61		#else
62		typedef uint32_t pointer_addr_t;
63		#endif
64	60
65	61	// module limited globals
66	62	extension_map_list_t *extension_map_list;

297	293	} break; // not really needed
298	294	}
299	295
300		if ( nffile_r->block_header->id == Large_BLOCK_Type ) {
301		// skip
302		continue;
303		}
304
305	296	if ( nffile_r->block_header->id != DATA_BLOCK_TYPE_2 ) {
306	297	fprintf(stderr, "Can't process block type %u. Skip block.\n", nffile_r->block_header->id);
307	298	continue;

347	338	exit(255);
348	339	}
349	340	} break;
350		case ExporterRecordType:
351		case SamplerRecordype:
	341	case LegacyRecordType1:
	342	case LegacyRecordType2:
352	343	case ExporterInfoRecordType:
353	344	case ExporterStatRecordType:
354	345	case SamplerInfoRecordype:

404	395	PAnonymizer_Init((uint8_t *)CryptoPAnKey);
405	396	break;
406	397	case 'L':
407		if ( !InitLog("argv[0]", optarg) )
	398	if ( !InitLog(0, "argv[0]", optarg, 0) )
408	399	exit(255);
409	400	break;
410	401	case 'r':

+81

-84

bin/nfcapd.c less more

0	0	/*
1		* Copyright (c) 2009-2017, Peter Haag
	1	* Copyright (c) 2009-2020, Peter Haag
2	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
3	3	* All rights reserved.
4	4	*

28	28	*
29	29	*/
30	30
31		/*
32		* Because NetFlow export uses UDP to send export datagrams, it is possible
33		* for datagrams to be lost. To determine whether flow export information has
34		* been lost, Version 5, Version 7, and Version 8 headers contain a flow
35		* sequence number. The sequence number is equal to the sequence number of the
36		* previous datagram plus the number of flows in the previous datagram. After
37		* receiving a new datagram, the receiving application can subtract the expected
38		* sequence number from the sequence number in the header to derive the number
39		* of missed flows.
40		*/
41
42	31	#include "config.h"
43	32
44	33	#include <stdio.h>

74	63	#endif
75	64
76	65	#include "util.h"
	66	#include "nfdump.h"
77	67	#include "nffile.h"
78	68	#include "nfx.h"
79		#include "nf_common.h"
	69	#include "exporter.h"
80	70	#include "nfnet.h"
81	71	#include "flist.h"
82	72	#include "nfstatfile.h"
83	73	#include "bookkeeper.h"
	74	#include "launch.h"
84	75	#include "collector.h"
85		#include "exporter.h"
86	76	#include "netflow_v1.h"
87	77	#include "netflow_v5_v7.h"
88	78	#include "netflow_v9.h"

105	95	#define DEFAULTHOSTNAME "127.0.0.1"
106	96	#define SENDSOCK_BUFFSIZE 200000
107	97
108		#ifndef DEVEL
109		# define dbg_printf(...) /* printf(__VA_ARGS__) */
110		#else
111		# define dbg_printf(...) printf(__VA_ARGS__)
112		#endif
113
114		/* globals */
115		caddr_t shmem;
116		int verbose = 0;
	98	static void *shmem = NULL;
	99	static int verbose = 0;
	100	static uint32_t default_sampling = 1;
	101	static uint32_t overwrite_sampling = 0;
117	102
118	103	extern uint32_t default_sampling; // the default sampling rate when nothing else applies. set by -S
119	104	extern uint32_t overwrite_sampling; // unconditionally overwrite sampling rate with given sampling rate -S

173	158	"-B bufflen\tSet socket buffer to bufflen bytes\n"
174	159	"-e\t\tExpire data at each cycle.\n"
175	160	"-D\t\tFork to background\n"
176		"-E\t\tPrint extended format of netflow data. for debugging purpose only.\n"
	161	"-E\t\tPrint extended format of netflow data. For debugging purpose only.\n"
177	162	"-T\t\tInclude extension tags in records.\n"
178	163	"-4\t\tListen on IPv4 (default).\n"
179	164	"-6\t\tListen on IPv6.\n"
180	165	"-V\t\tPrint version and exit.\n"
181		"-Z\t\tAdd timezone offset to filenamet.\n"
	166	"-Z\t\tAdd timezone offset to filename.\n"
182	167	, name);
183	168	} // End of usage
184	169

353	338
354	339	} // End of SetPriv
355	340
	341	static void format_file_block_header(data_block_header_t *header) {
	342
	343	printf("\n"
	344	"File Block Header: \n"
	345	" NumBlocks = %10u\n"
	346	" Size = %10u\n"
	347	" id = %10u\n",
	348	header->NumRecords,
	349	header->size,
	350	header->id);
	351
	352	} // End of format_file_block_header
	353
356	354	#include "nffile_inline.c"
357	355	#include "collector_inline.c"
358	356

369	367	ssize_t cnt;
370	368	void *in_buff;
371	369	int err;
372		char *string;
373	370	srecord_t *commbuff;
374	371
375		if ( !Init_v1() \|\| !Init_v5_v7_input() \|\| !Init_v9() \|\| !Init_IPFIX() )
	372	if ( !Init_v1(verbose) \|\| !Init_v5_v7_input(verbose, default_sampling, overwrite_sampling) \|\|
	373	!Init_v9(verbose, default_sampling, overwrite_sampling) \|\| !Init_IPFIX(verbose, default_sampling, overwrite_sampling) )
376	374	return;
377	375
378	376	in_buff = malloc(NETWORK_INPUT_BUFF_SIZE);

410	408	periodic_trigger = 0;
411	409	ignored_packets = 0;
412	410
413		// wake up at least at next time slot (twin) + some Overdue time
414		alarm(t_start + twin + OVERDUE_TIME - time(NULL));
	411	// wake up at least at next time slot (twin) + 1s
	412	alarm(t_start + twin + 1 - time(NULL));
415	413	/*
416	414	* Main processing loop:
417	415	* this loop, continues until done = 1, set by the signal handler

460	458	t_now = tv.tv_sec;
461	459
462	460	if ( ((t_now - t_start) >= twin) \|\| done ) {
463		char subfilename[64];
464	461	struct tm *now;
465		char *subdir, fmt[64];
	462	char *subdir, fmt[MAXTIMESTRING];
466	463
467	464	alarm(0);
468	465	now = localtime(&t_start);
469		strftime(fmt, sizeof fmt, time_extension, now);
	466	strftime(fmt, sizeof(fmt), time_extension, now);
470	467
471	468	// prepare sub dir hierarchy
472	469	if ( use_subdirs ) {

477	474
478	475	// failed to generate subdir path - put flows into base directory
479	476	subdir = NULL;
480		snprintf(subfilename, 63, "nfcapd.%s", fmt);
481		} else {
482		snprintf(subfilename, 63, "%s/nfcapd.%s", subdir, fmt);
483		}
	477	}
484	478	} else {
485	479	subdir = NULL;
486		snprintf(subfilename, 63, "nfcapd.%s", fmt);
487	480	}
488		subfilename[63] = '\0';
489	481
490	482	// for each flow source update the stats, close the file and re-initialize the new file
491	483	fs = FlowSource;

496	488
497	489	if ( verbose ) {
498	490	// Dump to stdout
499		format_file_block_header(nffile->block_header, &string, 0);
500		printf("%s\n", string);
	491	format_file_block_header(nffile->block_header);
501	492	}
502	493
503	494	if ( nffile->block_header->NumRecords ) {
504	495	// flush current buffer to disc
505	496	if ( WriteBlock(nffile) <= 0 )
506		LogError("Ident: %s, failed to write output buffer to disk: '%s'" , fs->Ident, strerror(errno));
	497	LogError("Ident: %s, failed to write output buffer to disk: '%s'" ,
	498	fs->Ident, strerror(errno));
507	499	} // else - no new records in current block
508	500
509	501
510	502	// prepare filename
511		snprintf(nfcapd_filename, MAXPATHLEN-1, "%s/%s", fs->datadir, subfilename);
	503	if ( subdir ) {
	504	if ( SetupSubDir(fs->datadir, subdir, error, 255) ) {
	505	snprintf(nfcapd_filename, MAXPATHLEN-1, "%s/%s/nfcapd.%s", fs->datadir, subdir, fmt);
	506	} else {
	507	LogError("Ident: %s, Failed to create sub hier directories: %s", fs->Ident, error );
	508	// skip subdir - put flows directly into current directory
	509	snprintf(nfcapd_filename, MAXPATHLEN-1, "%s/nfcapd.%s", fs->datadir, fmt);
	510	}
	511	} else {
	512	snprintf(nfcapd_filename, MAXPATHLEN-1, "%s/nfcapd.%s", fs->datadir, fmt);
	513	}
512	514	nfcapd_filename[MAXPATHLEN-1] = '\0';
513	515
514	516	// update stat record

528	530	// Close file
529	531	CloseUpdateFile(nffile, fs->Ident);
530	532
531		if ( subdir && !SetupSubDir(fs->datadir, subdir, error, 255) ) {
532		// in this case the flows get lost! - the rename will fail
533		// but this should not happen anyway, unless i/o problems, inode problems etc.
534		LogError("Ident: %s, Failed to create sub hier directories: %s", fs->Ident, error );
535		}
536
537	533	// if rename fails, we are in big trouble, as we need to get rid of the old .current file
538	534	// otherwise, we will loose flows and can not continue collecting new flows
539	535	err = rename(fs->current, nfcapd_filename);

557	553
558	554	// log stats
559	555	LogInfo("Ident: '%s' Flows: %llu, Packets: %llu, Bytes: %llu, Sequence Errors: %u, Bad Packets: %u",
560		fs->Ident, (unsigned long long)nffile->stat_record->numflows, (unsigned long long)nffile->stat_record->numpackets,
	556	fs->Ident, (unsigned long long)nffile->stat_record->numflows,
	557	(unsigned long long)nffile->stat_record->numpackets,
561	558	(unsigned long long)nffile->stat_record->numbytes, nffile->stat_record->sequence_failure, fs->bad_packets);
562	559
563	560	// reset stats

578	575
579	576	// next flow source
580	577	fs = fs->next;
	578
581	579	} // end of while (fs)
582	580
583		// All flow sources updated - signal launcher if required
	581	// trigger launcher if required
584	582	if ( launcher_pid ) {
585	583	// Signal launcher
586
587		// prepare filename for %f expansion
588		strncpy(commbuff->fname, subfilename, FNAME_SIZE-1);
589		commbuff->fname[FNAME_SIZE-1] = 0;
590		snprintf(commbuff->tstring, 16, "%i%02i%02i%02i%02i",
591		now->tm_year + 1900, now->tm_mon + 1, now->tm_mday, now->tm_hour, now->tm_min);
592		commbuff->tstring[15] = 0;
	584
	585	strncpy(commbuff->tstring, fmt, MAXTIMESTRING);
	586	commbuff->tstring[MAXTIMESTRING-1] = '\0';
	587
593	588	commbuff->tstamp = t_start;
594		if ( subdir )
595		strncpy(commbuff->subdir, subdir, FNAME_SIZE);
596		else
597		commbuff->subdir[0] = '\0';
	589	if ( subdir ) {
	590	snprintf(commbuff->fname, MAXPATHLEN-1, "%s/nfcapd.%s", subdir, fmt);
	591	} else {
	592	snprintf(commbuff->fname, MAXPATHLEN-1, "nfcapd.%s", fmt);
	593	}
	594	commbuff->fname[MAXPATHLEN-1] = '\0';
598	595
599	596	if ( launcher_alive ) {
600	597	LogInfo("Signal launcher");

614	611	t_start += twin;
615	612	/* t_start = filename time stamp: begin of slot
616	613	* + twin = end of next time interval
617		* + OVERDUE_TIME = if no data is collected, this is at latest to act
	614	* + 1 = act at least 1s after time window expired
618	615	* - t_now = difference value to now
619	616	*/
620		alarm(t_start + twin + OVERDUE_TIME - t_now);
	617	alarm(t_start + twin + 1 - t_now);
621	618
622	619	}
623	620

752	749	struct sigaction act;
753	750	int family, bufflen;
754	751	time_t twin, t_start;
755		int sock, synctime, do_daemonize, expire, spec_time_extension, report_sequence;
	752	int sock, do_daemonize, expire, spec_time_extension, report_sequence;
756	753	int subdir_index, sampling_rate, compress;
757	754	int c, i;
758	755	#ifdef PCAP
759		char *pcap_file;
760
761		pcap_file = NULL;
	756	char *pcap_file = NULL;
762	757	#endif
763	758
764	759	receive_packet = recvfrom;
765		verbose = synctime = do_daemonize = 0;
	760	verbose = do_daemonize = 0;
766	761	bufflen = 0;
767	762	family = AF_UNSPEC;
768	763	launcher_pid = 0;

857	852	exit(255);
858	853	break;
859	854	case 'w':
860		synctime = 1;
	855	// allow for compatibility - always sync timeslot
861	856	break;
862	857	case 'B':
863	858	bufflen = strtol(optarg, &checkptr, 10);

884	879	exit(255);
885	880	}
886	881	tmp[MAXPATHLEN-1] = 0;
887		snprintf(pidfile, MAXPATHLEN - 1 - strlen(tmp), "%s/%s", tmp, optarg);
	882	if ( (strlen(tmp) + strlen(optarg) + 3) < MAXPATHLEN ) {
	883	snprintf(pidfile, MAXPATHLEN - 3 - strlen(tmp), "%s/%s", tmp, optarg);
	884	} else {
	885	fprintf(stderr, "pidfile MAXPATHLEN error:\n");
	886	exit(255);
	887	}
888	888	}
889	889	// pidfile now absolute path
890	890	pidfile[MAXPATHLEN-1] = 0;

936	936	fprintf(stderr, "ERROR: Path too long!\n");
937	937	exit(255);
938	938	}
	939	datadir = realpath(datadir, NULL);
939	940	if ( stat(datadir, &fstat) < 0 ) {
940	941	fprintf(stderr, "stat() failed on %s: %s\n", datadir, strerror(errno));
941	942	exit(255);

950	951	break;
951	952	case 't':
952	953	twin = atoi(optarg);
953		if ( twin <= 0 ) {
954		fprintf(stderr, "ERROR: time frame <= 0\n");
	954	if ( twin < 2 ) {
	955	LogError("time interval <= 2s not allowed");
955	956	exit(255);
956	957	}
957	958	if (twin < 60) {
958		fprintf(stderr, "WARNING, Very small time frame - < 60s!\n");
	959	time_extension = "%Y%m%d%H%M%S";
959	960	}
960	961	break;
961	962	case 'x':

1022	1023	exit(255);
1023	1024	}
1024	1025
1025		if ( do_daemonize && !InitLog(argv[0], SYSLOG_FACILITY)) {
	1026	if ( !InitLog(do_daemonize, argv[0], SYSLOG_FACILITY, verbose) ) {
1026	1027	exit(255);
1027	1028	}
1028	1029

1123	1124	}
1124	1125
1125	1126	t_start = time(NULL);
1126		if ( synctime )
1127		t_start = t_start - ( t_start % twin);
	1127	t_start = t_start - ( t_start % twin);
1128	1128
1129	1129	if ( do_daemonize ) {
1130	1130	verbose = 0;

1147	1147
1148	1148	done = 0;
1149	1149	if ( launch_process \|\| expire ) {
1150		// for efficiency reason, the process collecting the data
1151		// and the process launching processes, when a new file becomes
1152		// available are separated. Communication is done using signals
1153		// as well as shared memory
1154		// prepare shared memory
	1150
	1151	// create laucher comm memory struct
1155	1152	shmem = mmap(0, sizeof(srecord_t), PROT_READ \| PROT_WRITE, MAP_ANON \| MAP_SHARED, -1, 0);
1156		if ( shmem == (caddr_t)-1 ) {
1157		LogError("mmap() error: %s", strerror(errno));
	1153	if ( shmem == MAP_FAILED) {
	1154	LogError("mmap() error in %s:%i: %s", __FILE__, __LINE__ , strerror(errno));
1158	1155	close(sock);
1159	1156	exit(255);
1160	1157	}

1164	1161	case 0:
1165	1162	// child
1166	1163	close(sock);
1167		launcher((char *)shmem, FlowSource, launch_process, expire);
	1164	launcher(shmem, FlowSource, launch_process, expire);
1168	1165	_exit(0);
1169	1166	break;
1170	1167	case -1:

+107

-119

bin/nfdump.c less more

0	0	/*
1		* Copyright (c) 2009-2019, Peter Haag
	1	* Copyright (c) 2009-2020, Peter Haag
2	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
3	3	* All rights reserved.
4	4	*

50	50	#include <stdint.h>
51	51	#endif
52	52
	53	#include "util.h"
	54	#include "nfdump.h"
53	55	#include "nffile.h"
54	56	#include "nfx.h"
	57	#include "flist.h"
55	58	#include "nfnet.h"
56	59	#include "bookkeeper.h"
57	60	#include "collector.h"
58	61	#include "exporter.h"
59		#include "nf_common.h"
	62	#include "output_util.h"
	63	#include "output_raw.h"
	64	#include "output_pipe.h"
	65	#include "output_csv.h"
60	66	#include "output_json.h"
61	67	#include "netflow_v5_v7.h"
62	68	#include "netflow_v9.h"
63		#include "rbtree.h"
64	69	#include "nftree.h"
65	70	#include "nfprof.h"
66		#include "nfdump.h"
67	71	#include "nflowcache.h"
68	72	#include "nfstat.h"
69	73	#include "nfexport.h"
70	74	#include "ipconv.h"
71		#include "util.h"
72		#include "flist.h"
73
74		#ifndef DEVEL
75		# define dbg_printf(...) /* printf(__VA_ARGS__) */
76		#else
77		# define dbg_printf(...) printf(__VA_ARGS__)
78		#endif
79	75
80	76	/* hash parameters */
81	77	#define NumPrealloc 128000

83	79	#define AGGR_SIZE 7
84	80
85	81	/* Global Variables */
86		FilterEngine_data_t *Engine;
	82	FilterEngine_t *Engine;
87	83
88	84	extern char *FilterFilename;
89	85	extern uint32_t loopcnt;

92	88	const char *nfdump_version = VERSION;
93	89
94	90	static uint64_t total_bytes;
95		static uint32_t total_flows;
	91	static uint32_t recordCount;
96	92	static uint32_t skipped_blocks;
97	93	static uint32_t is_anonymized;
98	94	static time_t t_first_flow, t_last_flow;

105	101
106	102	extension_map_list_t *extension_map_list;
107	103
108		extern generic_exporter_t **exporter_list;
	104	extern exporter_t **exporter_list;
109	105	/*
110	106	* Output Formats:
111	107	* User defined output formats can be compiled into nfdump, for easy access

209	205	* 5. Recompile nfdump
210	206	*/
211	207
	208	static void flow_record_to_null(void record, char * s, int tag);
	209
212	210	// Assign print functions for all output options -o
213	211	// Teminated with a NULL record
214	212	printmap_t printmap[] = {
215		{ "raw", format_file_block_record, NULL },
216		{ "line", format_special, FORMAT_line },
217		{ "long", format_special, FORMAT_long },
218		{ "extended", format_special, FORMAT_extended },
219		{ "biline", format_special, FORMAT_biline },
220		{ "bilong", format_special, FORMAT_bilong },
221		{ "pipe", flow_record_to_pipe, NULL },
222		{ "json", flow_record_to_json, NULL },
223		{ "csv", flow_record_to_csv, NULL },
224		{ "null", flow_record_to_null, NULL },
	213	{ "raw", flow_record_to_raw, raw_prolog, raw_epilog, NULL },
	214	{ "line", format_special, text_prolog, text_epilog, FORMAT_line },
	215	{ "long", format_special, text_prolog, text_epilog, FORMAT_long },
	216	{ "extended", format_special, text_prolog, text_epilog, FORMAT_extended },
	217	{ "biline", format_special, text_prolog, text_epilog, FORMAT_biline },
	218	{ "bilong", format_special, text_prolog, text_epilog, FORMAT_bilong },
	219	{ "pipe", flow_record_to_pipe, pipe_prolog, pipe_epilog, NULL },
	220	{ "json", flow_record_to_json, json_prolog, json_epilog, NULL },
	221	{ "csv", flow_record_to_csv, csv_prolog, csv_epilog, NULL },
	222	{ "null", flow_record_to_null, text_prolog, text_epilog, NULL },
225	223	#ifdef NSEL
226		{ "nsel", format_special, FORMAT_nsel },
227		{ "nel", format_special, FORMAT_nel },
	224	{ "nsel", format_special, text_prolog, text_epilog, FORMAT_nsel },
	225	{ "nel", format_special, text_prolog, text_epilog, FORMAT_nel },
228	226	#endif
229	227
230	228	// add your formats here
231	229
232	230	// This is always the last line
233		{ NULL, NULL, NULL }
	231	{ NULL, NULL, NULL, NULL, "" }
234	232	};
235	233
236	234	// For automatic output format generation in case of custom aggregation

243	241	/* Function Prototypes */
244	242	static void usage(char *name);
245	243
246		static void PrintSummary(stat_record_t *stat_record, int plain_numbers, int csv_output);
	244	static void PrintSummary(stat_record_t *stat_record, int printPlain, int csv_output);
247	245
248	246	static stat_record_t process_data(char *wfile, int element_stat, int flow_stat, int sort_flows,
249		printer_t print_header, printer_t print_record, time_t twin_start, time_t twin_end,
250		uint64_t limitflows, int tag, int compress);
	247	printer_t print_record, time_t twin_start, time_t twin_end,
	248	uint64_t limitRecords, int tag, int compress);
251	249
252	250	/* Functions */
253	251

299	297	"\t\t csv ',' separated, machine parseable output format.\n"
300	298	"\t\t json json output format.\n"
301	299	"\t\t pipe '\|' separated legacy machine parseable output format.\n"
	300	"\t\t null no flow records, but statistics output.\n"
302	301	"\t\t\tmode may be extended by '6' for full IPv6 listing. e.g.long6, extended6.\n"
303	302	"-E <file>\tPrint exporter and sampling info for collected flows.\n"
304	303	"-v <file>\tverify netflow data file. Print version and blocks.\n"

309	308	"\t\tyyyy/MM/dd.hh:mm:ss[-yyyy/MM/dd.hh:mm:ss]\n", name);
310	309	} /* usage */
311	310
312
313		static void PrintSummary(stat_record_t *stat_record, int plain_numbers, int csv_output) {
	311	static void flow_record_to_null(void record, char * s, int tag) {
	312	// empty - do not list any flows
	313	} // End of flow_record_to_null
	314
	315	static void PrintSummary(stat_record_t *stat_record, int printPlain, int csv_output) {
314	316	static double duration;
315	317	uint64_t bps, pps, bpp;
316	318	char byte_str[NUMBER_STRING_SIZE], packet_str[NUMBER_STRING_SIZE];

336	338	(long long unsigned)stat_record->numflows, (long long unsigned)stat_record->numbytes,
337	339	(long long unsigned)stat_record->numpackets, (long long unsigned)bps,
338	340	(long long unsigned)pps, (long long unsigned)bpp );
339		} else if ( plain_numbers ) {
	341	/*
	342	} else if ( printPlain ) {
340	343	printf("Summary: total flows: %llu, total bytes: %llu, total packets: %llu, avg bps: %llu, avg pps: %llu, avg bpp: %llu\n",
341	344	(long long unsigned)stat_record->numflows, (long long unsigned)stat_record->numbytes,
342	345	(long long unsigned)stat_record->numpackets, (long long unsigned)bps,
343	346	(long long unsigned)pps, (long long unsigned)bpp );
	347	*/
344	348	} else {
345		format_number(stat_record->numbytes, byte_str, DO_SCALE_NUMBER, VAR_LENGTH);
346		format_number(stat_record->numpackets, packet_str, DO_SCALE_NUMBER, VAR_LENGTH);
347		format_number(bps, bps_str, DO_SCALE_NUMBER, VAR_LENGTH);
348		format_number(pps, pps_str, DO_SCALE_NUMBER, VAR_LENGTH);
349		format_number(bpp, bpp_str, DO_SCALE_NUMBER, VAR_LENGTH);
	349	format_number(stat_record->numbytes, byte_str, printPlain, VAR_LENGTH);
	350	format_number(stat_record->numpackets, packet_str, printPlain, VAR_LENGTH);
	351	format_number(bps, bps_str, printPlain, VAR_LENGTH);
	352	format_number(pps, pps_str, printPlain, VAR_LENGTH);
	353	format_number(bpp, bpp_str, printPlain, VAR_LENGTH);
350	354	printf("Summary: total flows: %llu, total bytes: %s, total packets: %s, avg bps: %s, avg pps: %s, avg bpp: %s\n",
351	355	(unsigned long long)stat_record->numflows, byte_str, packet_str, bps_str, pps_str, bpp_str );
352	356	}

354	358	} // End of PrintSummary
355	359
356	360	stat_record_t process_data(char *wfile, int element_stat, int flow_stat, int sort_flows,
357		printer_t print_header, printer_t print_record, time_t twin_start, time_t twin_end,
358		uint64_t limitflows, int tag, int compress) {
	361	printer_t print_record, time_t twin_start, time_t twin_end,
	362	uint64_t limitRecords, int tag, int compress) {
359	363	common_record_t flow_record, record_ptr;
360	364	master_record_t *master_record;
361	365	nffile_t nffile_w, nffile_r;

455	459	total_bytes += ret;
456	460	}
457	461
458		if ( nffile_r->block_header->id == Large_BLOCK_Type ) {
459		// skip
460		printf("Xstat block skipped ...\n");
461		continue;
462		}
463
464	462	if ( nffile_r->block_header->id != DATA_BLOCK_TYPE_2 ) {
465	463	if ( nffile_r->block_header->id == DATA_BLOCK_TYPE_1 ) {
466		LogError("Can't process nfdump 1.5.x block type 1. Add --enable-compat15 to compile compatibility code. Skip block.\n");
	464	LogError("nfdump 1.5.x block type 1 no longer supported. Skip block.\n");
467	465	} else {
468	466	LogError("Can't process block type %u. Skip block.\n", nffile_r->block_header->id);
469	467	}

487	485	case CommonRecordType: {
488	486	int match;
489	487	uint32_t map_id;
490		generic_exporter_t *exp_info;
	488	exporter_t *exp_info;
491	489
492	490	// valid flow_record converted if needed
493	491	map_id = flow_record->ext_map;

503	501	continue;
504	502	}
505	503
506		total_flows++;
	504	recordCount++;
507	505	master_record = &(extension_map_list->slot[map_id]->master_record);
508	506	Engine->nfrecord = (uint64_t *)master_record;
509	507	ExpandRecord_v2( flow_record, extension_map_list->slot[map_id],

512	510	// Time based filter
513	511	// if no time filter is given, the result is always true
514	512	match = twin_start && (master_record->first < twin_start \|\| master_record->last > twin_end) ? 0 : 1;
515		match &= limitflows ? stat_record.numflows < limitflows : 1;
	513	match &= limitRecords ? recordCount <= limitRecords : 1;
516	514
517	515	// filter netflow record with user supplied filter
518	516	if ( match )

554	552	// if we need to print out this record
555	553	print_record(master_record, &string, tag);
556	554	if ( string ) {
557		if ( limitflows ) {
558		if ( (stat_record.numflows <= limitflows) )
559		printf("%s\n", string);
560		} else
561		printf("%s\n", string);
	555	printf("%s\n", string);
562	556	}
563	557	} else {
564	558	// mutually exclusive conditions should prevent executing this code

583	577	exit(255);
584	578	}
585	579	} break;
586		case ExporterRecordType:
587		case SamplerRecordype:
588		// Silently skip exporter records
	580	case LegacyRecordType1:
	581	case LegacyRecordType2:
	582	// Silently skip legacy records
589	583	break;
590	584	case ExporterInfoRecordType: {
591	585	int ret = AddExporterInfo((exporter_info_record_t *)record_ptr);

620	614	} // for all records
621	615
622	616	// check if we are done, due to -c option
623		if ( limitflows )
624		done = stat_record.numflows >= limitflows;
	617	if ( limitRecords )
	618	done = recordCount >= limitRecords;
625	619
626	620	} // while
627	621

656	650	int main( int argc, char **argv ) {
657	651	struct stat stat_buff;
658	652	stat_record_t sum_stat;
659		printer_t print_header, print_record;
	653	printer_t print_record;
	654	func_prolog_t print_prolog;
	655	func_epilog_t print_epilog;
660	656	nfprof_t profile_data;
661	657	char rfile, Rfile, Mdirs, wfile, ffile, filter, tstring, stat_type;
662		char byte_limit_string, packet_limit_string, print_format, record_header;
	658	char byte_limit_string, packet_limit_string, *print_format;
663	659	char print_order, query_file, nameserver, aggr_fmt;
664	660	int c, ffd, ret, element_stat, fdump;
665		int i, user_format, quiet, flow_stat, topN, aggregate, aggregate_mask, bidir;
	661	int i, quiet, flow_stat, topN, aggregate, aggregate_mask, bidir;
666	662	int print_stat, syntax_only, date_sorted, do_tag, compress;
667		int plain_numbers, GuessDir, pipe_output, csv_output, ModifyCompress;
	663	int printPlain, GuessDir, pipe_output, csv_output, json_output, ModifyCompress;
668	664	time_t t_start, t_end;
669		uint32_t limitflows;
	665	uint32_t limitRecords;
670	666	char Ident[IDENTLEN];
671	667
672	668	rfile = Rfile = Mdirs = wfile = ffile = filter = tstring = stat_type = NULL;

680	676	flow_stat = 0;
681	677	print_stat = 0;
682	678	element_stat = 0;
683		limitflows = 0;
	679	limitRecords = 0;
684	680	date_sorted = 0;
685	681	total_bytes = 0;
686		total_flows = 0;
	682	recordCount = 0;
687	683	skipped_blocks = 0;
688	684	do_tag = 0;
689	685	quiet = 0;
690		user_format = 0;
691	686	compress = NOT_COMPRESSED;
692		plain_numbers = 0;
	687	printPlain = 0;
693	688	pipe_output = 0;
694	689	csv_output = 0;
	690	json_output = 0;
695	691	is_anonymized = 0;
696	692	GuessDir = 0;
697	693	nameserver = NULL;
698	694
699	695	print_format = NULL;
700		print_header = NULL;
701	696	print_record = NULL;
	697	print_prolog = NULL;
	698	print_epilog = NULL;
702	699	print_order = NULL;
703	700	query_file = NULL;
704	701	ModifyCompress = -1;
705	702	aggr_fmt = NULL;
706		record_header = NULL;
707	703
708	704	Ident[0] = '\0';
709	705

777	773	compress = LZO_COMPRESSED;
778	774	break;
779	775	case 'c':
780		limitflows = atoi(optarg);
781		if ( !limitflows ) {
	776	limitRecords = atoi(optarg);
	777	if ( !limitRecords ) {
782	778	LogError("Option -c needs a number > 0\n");
783	779	exit(255);
784	780	}

810	806	byte_limit_string = optarg;
811	807	break;
812	808	case 'N':
813		plain_numbers = 1;
	809	printPlain = 1;
814	810	break;
815	811	case 'f':
816	812	ffile = optarg;

837	833	break;
838	834	case 'o': // output mode
839	835	print_format = optarg;
	836	// limit input chars
	837	if ( strlen(print_format) > 512 ) {
	838	LogError("Length of ouput format string too big - > 512\n");
	839	exit(255);
	840	}
840	841	break;
841	842	case 'O': { // stat order by
842	843	int ret;

865	866	do_tag = 1;
866	867	break;
867	868	case 'i':
868		strncpy(Ident, optarg, IDENT_SIZE);
869		Ident[IDENT_SIZE - 1] = 0;
	869	strncpy(Ident, optarg, IDENTLEN);
	870	Ident[IDENTLEN - 1] = 0;
870	871	if ( strchr(Ident, ' ') ) {
871	872	LogError("Ident must not contain spaces\n");
872	873	exit(255);

996	997	print_format = DefaultMode;
997	998	}
998	999
999		// limit input chars
1000		if ( strlen(print_format) > 512 ) {
1001		LogError("Length of ouput format string too big - > 512\n");
1002		exit(255);
1003		}
1004	1000	if ( strncasecmp(print_format, "fmt:", 4) == 0 ) {
1005	1001	// special user defined output format
1006	1002	char *format = &print_format[4];
1007	1003	if ( strlen(format) ) {
1008		if ( !ParseOutputFormat(format, plain_numbers, printmap) )
	1004	if ( !ParseOutputFormat(format, printPlain, printmap) )
1009	1005	exit(255);
1010	1006	print_record = format_special;
1011		record_header = get_record_header();
1012		user_format = 1;
	1007	print_prolog = text_prolog;
1013	1008	} else {
1014	1009	LogError("Missing format description for user defined output format!\n");
1015	1010	exit(255);

1031	1026	while ( printmap[i].printmode ) {
1032	1027	if ( strncasecmp(print_format, printmap[i].printmode, MAXMODELEN) == 0 ) {
1033	1028	if ( printmap[i].Format ) {
1034		if ( !ParseOutputFormat(printmap[i].Format, plain_numbers, printmap) )
	1029	if ( !ParseOutputFormat(printmap[i].Format, printPlain, printmap) )
1035	1030	exit(255);
1036	1031	// predefined custom format
1037		print_record = printmap[i].func;
1038		record_header = get_record_header();
1039		user_format = 1;
	1032	print_record = printmap[i].func_record;
	1033	print_prolog = printmap[i].func_prolog;
	1034	print_epilog = printmap[i].func_epilog;
1040	1035	} else {
1041	1036	// To support the pipe output format for element stats - check for pipe, and remember this
1042	1037	if ( strncasecmp(print_format, "pipe", MAXMODELEN) == 0 ) {

1044	1039	}
1045	1040	if ( strncasecmp(print_format, "csv", MAXMODELEN) == 0 ) {
1046	1041	csv_output = 1;
1047		set_record_header();
1048		record_header = get_record_header();
	1042	}
	1043	if ( strncasecmp(print_format, "json", MAXMODELEN) == 0 ) {
	1044	json_output = 1;
1049	1045	}
1050	1046	// predefined static format
1051		print_record = printmap[i].func;
1052		user_format = 0;
	1047	print_record = printmap[i].func_record;
	1048	print_prolog = printmap[i].func_prolog;
	1049	print_epilog = printmap[i].func_epilog;
1053	1050	}
1054	1051	break;
1055	1052	}

1062	1059	exit(255);
1063	1060	}
1064	1061
1065		// this is the only case, where headers are printed.
1066		if ( strncasecmp(print_format, "raw", 16) == 0 )
1067		print_header = format_file_block_header;
1068
1069	1062	if ( aggregate && (flow_stat \|\| element_stat) ) {
1070	1063	aggregate = 0;
1071	1064	LogError("Command line switch -s overwrites -a\n");

1109	1102
1110	1103	if ( fdump ) {
1111	1104	printf("StartNode: %i Engine: %s\n", Engine->StartNode, Engine->Extended ? "Extended" : "Fast");
1112		DumpList(Engine);
	1105	DumpEngine(Engine);
1113	1106	exit(0);
1114	1107	}
1115	1108

1135	1128	}
1136	1129
1137	1130
1138		if ( !(flow_stat \|\| element_stat \|\| wfile \|\| quiet ) && record_header ) {
1139		if ( user_format ) {
1140		printf("%s\n", record_header);
1141		} else {
1142		// static format - no static format with header any more, but keep code anyway
1143		if ( Getv6Mode() ) {
1144		printf("%s\n", record_header);
1145		} else
1146		printf("%s\n", record_header);
1147		}
	1131	if ( !(flow_stat \|\| element_stat \|\| wfile \|\| quiet ) && print_prolog ) {
	1132	print_prolog();
1148	1133	}
1149	1134
1150	1135	nfprof_start(&profile_data);
1151	1136	sum_stat = process_data(wfile, element_stat, aggregate \|\| flow_stat, print_order != NULL,
1152		print_header, print_record, t_start, t_end,
1153		limitflows, do_tag, compress);
1154		nfprof_end(&profile_data, total_flows);
1155
	1137	print_record, t_start, t_end,
	1138	limitRecords, do_tag, compress);
	1139	nfprof_end(&profile_data, recordCount);
	1140
1156	1141	if ( total_bytes == 0 ) {
1157	1142	printf("No matched flows\n");
1158	1143	exit(0);

1163	1148	nffile_t *nffile = OpenNewFile(wfile, NULL, compress, is_anonymized, NULL);
1164	1149	if ( !nffile )
1165	1150	exit(255);
1166		if ( ExportFlowTable(nffile, aggregate, bidir, date_sorted, extension_map_list) ) {
	1151	if ( ExportFlowTable(nffile, aggregate, bidir, GuessDir, date_sorted, extension_map_list) ) {
1167	1152	CloseUpdateFile(nffile, Ident );
1168	1153	} else {
1169	1154	CloseFile(nffile);

1176	1161	}
1177	1162
1178	1163	if (flow_stat) {
1179		PrintFlowStat(record_header, print_record, topN, do_tag, quiet, csv_output, extension_map_list);
	1164	PrintFlowStat(print_prolog, print_record, topN, do_tag, quiet, csv_output, extension_map_list);
1180	1165	#ifdef DEVEL
1181	1166	printf("Loopcnt: %u\n", loopcnt);
1182	1167	#endif
1183	1168	}
1184	1169
1185	1170	if (element_stat) {
1186		PrintElementStat(&sum_stat, plain_numbers, record_header, print_record, topN, do_tag, quiet, pipe_output, csv_output);
	1171	PrintElementStat(&sum_stat, printPlain, print_record, topN, do_tag, quiet, pipe_output, csv_output);
1187	1172	}
1188	1173
1189		if ( !quiet ) {
	1174	if ( print_epilog ) {
	1175	print_epilog();
	1176	}
	1177	if ( !quiet && !json_output ) {
1190	1178	if ( csv_output ) {
1191		PrintSummary(&sum_stat, plain_numbers, csv_output);
	1179	PrintSummary(&sum_stat, printPlain, csv_output);
1192	1180	} else if ( !wfile ) {
1193	1181	if (is_anonymized)
1194	1182	printf("IP addresses anonymised\n");
1195		PrintSummary(&sum_stat, plain_numbers, csv_output);
	1183	PrintSummary(&sum_stat, printPlain, csv_output);
1196	1184	if ( t_last_flow == 0 ) {
1197	1185	// in case of a pre 1.6.6 collected and empty flow file
1198	1186	printf("Time window: <unknown>\n");

1200	1188	printf("Time window: %s\n", TimeString(t_first_flow, t_last_flow));
1201	1189	}
1202	1190	printf("Total flows processed: %u, Blocks skipped: %u, Bytes read: %llu\n",
1203		total_flows, skipped_blocks, (unsigned long long)total_bytes);
	1191	recordCount, skipped_blocks, (unsigned long long)total_bytes);
1204	1192	nfprof_print(&profile_data, stdout);
1205	1193	}
1206	1194	}

+724

-45

bin/nfdump.h less more

0	0	/*
1		* Copyright (c) 2017, Peter Haag
2		* Copyright (c) 2014, Peter Haag
3		* Copyright (c) 2009, Peter Haag
4		* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
	1	* Copyright (c) 2009-2020, Peter Haag
5	2	* All rights reserved.
6	3	*
7	4	* Redistribution and use in source and binary forms, with or without

40	37	#include <stdint.h>
41	38	#endif
42	39
43		#include "rbtree.h"
44
45		#define BuffNumRecords 1024
46
47		/*
48		* Offset definitions for filter engine. Offsets must agree with the defined
49		* flow record definition data_block_record_t in nffile.h
	40	#define V4 ip_union._v4
	41	#define V6 ip_union._v6
	42
	43	// single IP addr for next hop and bgp next hop
	44	typedef struct ip_addr_s {
	45	union {
	46	struct {
	47	#ifdef WORDS_BIGENDIAN
	48	uint32_t fill[3];
	49	uint32_t _v4;
	50	#else
	51	uint32_t fill1[2];
	52	uint32_t _v4;
	53	uint32_t fill2;
	54	#endif
	55	};
	56	uint64_t _v6[2];
	57	} ip_union;
	58	#define IP_ADDR_T
	59	} ip_addr_t;
	60
	61	// forward declaration
	62	typedef struct exporter_info_record_s exporter_info_record_t;
	63	typedef struct extension_map_s extension_map_t;
	64
	65	/* the master record contains all possible records unpacked */
	66	typedef struct master_record_s {
	67	// common information from all netflow versions
	68	// // interpreted as uint64_t[]
	69	// #ifdef WORDS_BIGENDIAN
	70
	71	uint16_t type; // index 0 0xffff 0000 0000 0000
	72	uint16_t size; // index 0 0x0000'ffff'0000 0000
	73	uint16_t flags; // index 0 0x0000'0000'ffff'0000
	74	uint16_t ext_map; // index 0 0x0000'0000'0000'ffff
	75	# define OffsetRecordFlags 0
	76	#ifdef WORDS_BIGENDIAN
	77	# define MaskRecordFlags 0x00000000ff000000LL
	78	# define ShiftRecordFlags 24
	79	#else
	80	# define MaskRecordFlags 0x000000ff00000000LL
	81	# define ShiftRecordFlags 32
	82	#endif
	83
	84	//
	85	uint16_t msec_first; // index 1 0xffff'0000'0000'0000
	86	uint16_t msec_last; // index 1 0x0000'ffff'0000'0000
	87
	88	// 12 bytes offset in master record to first
	89	#define BYTE_OFFSET_first 12
	90
	91	uint32_t first; // index 1 0x0000'0000'ffff'ffff
	92
	93	//
	94	uint32_t last; // index 2 0xffff'ffff'0000'0000
	95	uint8_t fwd_status; // index 2 0x0000'0000'ff00'0000
	96	uint8_t tcp_flags; // index 2 0x0000'0000'00ff'0000
	97	uint8_t prot; // index 2 0x0000'0000'0000'ff00
	98	uint8_t tos; // index 2 0x0000'0000'0000'00ff
	99	#ifdef WORDS_BIGENDIAN
	100	# define OffsetStatus 2
	101	# define MaskStatus 0x00000000ff000000LL
	102	# define ShiftStatus 24
	103
	104	# define OffsetFlags 2
	105	# define MaskFlags 0x0000000000ff0000LL
	106	#define ShiftFlags 16
	107
	108	# define OffsetProto 2
	109	# define MaskProto 0x000000000000ff00LL
	110	# define ShiftProto 8
	111
	112	# define OffsetTos 2
	113	# define MaskTos 0x00000000000000ffLL
	114	# define ShiftTos 0
	115
	116	#else
	117	# define OffsetStatus 2
	118	# define MaskStatus 0x000000ff00000000LL
	119	# define ShiftStatus 32
	120
	121	# define OffsetFlags 2
	122	# define MaskFlags 0x0000ff0000000000LL
	123	# define ShiftFlags 40
	124
	125	# define OffsetProto 2
	126	# define MaskProto 0x00ff000000000000LL
	127	# define ShiftProto 48
	128
	129	# define OffsetTos 2
	130	# define MaskTos 0xff00000000000000LL
	131	# define ShiftTos 56
	132	#endif
	133
	134	uint16_t srcport; // index 3 0xffff'0000'0000'0000
	135	uint16_t dstport; // index 3 0x0000'ffff'0000'0000
	136	uint16_t exporter_sysid; // index 3 0x0000'0000'ffff'0000
	137
	138	union {
	139	struct {
	140	#ifdef WORDS_BIGENDIAN
	141	uint8_t icmp_type; // index 3 0x0000'0000'0000'ff00
	142	uint8_t icmp_code; // index 3 0x0000'0000'0000'00ff
	143	#else
	144	// little endian confusion ...
	145	uint8_t icmp_code;
	146	uint8_t icmp_type;
	147	#endif
	148	};
	149	uint16_t icmp;
	150	};
	151
	152	#ifdef WORDS_BIGENDIAN
	153	# define OffsetPort 3
	154	# define OffsetExporterSysID 3
	155	# define MaskSrcPort 0xffff000000000000LL
	156	# define ShiftSrcPort 48
	157
	158	# define MaskDstPort 0x0000ffff00000000LL
	159	# define ShiftDstPort 32
	160
	161	# define MaskExporterSysID 0x00000000ffff0000LL
	162	# define ShiftExporterSysID 16
	163
	164	# define MaskICMPtype 0x000000000000ff00LL
	165	# define ShiftICMPtype 8
	166	# define MaskICMPcode 0x00000000000000ffLL
	167	# define ShiftICMPcode 0
	168
	169	#else
	170	# define OffsetPort 3
	171	# define OffsetExporterSysID 3
	172	# define MaskSrcPort 0x000000000000ffffLL
	173	# define ShiftSrcPort 0
	174
	175	# define MaskDstPort 0x00000000ffff0000LL
	176	# define ShiftDstPort 16
	177
	178	# define MaskExporterSysID 0x0000ffff00000000LL
	179	# define ShiftExporterSysID 32
	180
	181	# define MaskICMPtype 0xff00000000000000LL
	182	# define ShiftICMPtype 56
	183	# define MaskICMPcode 0x00ff000000000000LL
	184	# define ShiftICMPcode 48
	185	#endif
	186
	187	// extension 4 / 5
	188	uint32_t input; // index 4 0xffff'ffff'0000'0000
	189	uint32_t output; // index 4 0x0000'0000'ffff'ffff
	190	#ifdef WORDS_BIGENDIAN
	191	# define OffsetInOut 4
	192	# define MaskInput 0xffffffff00000000LL
	193	# define ShiftInput 32
	194	# define MaskOutput 0x00000000ffffffffLL
	195	# define ShiftOutput 0
	196
	197	#else
	198	# define OffsetInOut 4
	199	# define MaskInput 0x00000000ffffffffLL
	200	# define ShiftInput 0
	201	# define MaskOutput 0xffffffff00000000LL
	202	# define ShiftOutput 32
	203	#endif
	204
	205	// extension 6 / 7
	206	uint32_t srcas; // index 5 0xffff'ffff'0000'0000
	207	uint32_t dstas; // index 5 0x0000'0000'ffff'ffff
	208	#ifdef WORDS_BIGENDIAN
	209	# define OffsetAS 5
	210	# define MaskSrcAS 0xffffffff00000000LL
	211	# define ShiftSrcAS 32
	212	# define MaskDstAS 0x00000000ffffffffLL
	213	# define ShiftDstAS 0
	214
	215	#else
	216	# define OffsetAS 5
	217	# define MaskSrcAS 0x00000000ffffffffLL
	218	# define ShiftSrcAS 0
	219	# define MaskDstAS 0xffffffff00000000LL
	220	# define ShiftDstAS 32
	221	#endif
	222
	223
	224	// IP address block
	225	union {
	226	struct _ipv4_s {
	227	#ifdef WORDS_BIGENDIAN
	228	uint32_t fill1[3]; // <empty> index 6 0xffff'ffff'ffff'ffff
	229	// <empty> index 7 0xffff'ffff'0000'0000
	230	uint32_t srcaddr; // srcaddr index 7 0x0000'0000'ffff'ffff
	231	uint32_t fill2[3]; // <empty> index 8 0xffff'ffff'ffff'ffff
	232	// <empty> index 9 0xffff'ffff'0000'0000
	233	uint32_t dstaddr; // dstaddr index 9 0x0000'0000'ffff'ffff
	234	#else
	235	uint32_t fill1[2]; // <empty> index 6 0xffff'ffff'ffff'ffff
	236	uint32_t srcaddr; // srcaddr index 7 0xffff'ffff'0000'0000
	237	uint32_t fill2; // <empty> index 7 0x0000'0000'ffff'ffff
	238	uint32_t fill3[2]; // <empty> index 8 0xffff'ffff'ffff'ffff
	239	uint32_t dstaddr; // dstaddr index 9 0xffff'ffff'0000'0000
	240	uint32_t fill4; // <empty> index 9 0xffff'ffff'0000'0000
	241	#endif
	242	} _v4;
	243	struct _ipv6_s {
	244	uint64_t srcaddr[2]; // srcaddr[0-1] index 6 0xffff'ffff'ffff'ffff
	245	// srcaddr[2-3] index 7 0xffff'ffff'ffff'ffff
	246	uint64_t dstaddr[2]; // dstaddr[0-1] index 8 0xffff'ffff'ffff'ffff
	247	// dstaddr[2-3] index 9 0xffff'ffff'ffff'ffff
	248	} _v6;
	249	struct _ip64_s {
	250	uint64_t addr[4];
	251	} _ip_64;
	252	} ip_union;
	253
	254	#ifdef WORDS_BIGENDIAN
	255	# define OffsetSrcIPv4 7
	256	# define MaskSrcIPv4 0x00000000ffffffffLL
	257	# define ShiftSrcIPv4 0
	258
	259	# define OffsetDstIPv4 9
	260	# define MaskDstIPv4 0x00000000ffffffffLL
	261	# define ShiftDstIPv4 0
	262
	263	# define OffsetSrcIPv6a 6
	264	# define OffsetSrcIPv6b 7
	265	# define OffsetDstIPv6a 8
	266	# define OffsetDstIPv6b 9
	267	# define MaskIPv6 0xffffffffffffffffLL
	268	# define ShiftIPv6 0
	269
	270	#else
	271	# define OffsetSrcIPv4 6
	272	# define MaskSrcIPv4 0xffffffff00000000LL
	273	# define ShiftSrcIPv4 32
	274
	275	# define OffsetDstIPv4 8
	276	# define MaskDstIPv4 0xffffffff00000000LL
	277	# define ShiftDstIPv4 32
	278
	279	# define OffsetSrcIPv6a 6
	280	# define OffsetSrcIPv6b 7
	281	# define OffsetDstIPv6a 8
	282	# define OffsetDstIPv6b 9
	283	# define MaskIPv6 0xffffffffffffffffLL
	284	# define ShiftIPv6 0
	285	#endif
	286
	287
	288	// counter block - expanded to 8 bytes
	289	uint64_t dPkts; // index 10 0xffff'ffff'ffff'ffff
	290	# define OffsetPackets 10
	291	# define MaskPackets 0xffffffffffffffffLL
	292	# define ShiftPackets 0
	293
	294	uint64_t dOctets; // index 11 0xffff'ffff'ffff'ffff
	295	# define OffsetBytes 11
	296	# define MaskBytes 0xffffffffffffffffLL
	297	# define ShiftBytes 0
	298
	299	// extension 9 / 10
	300	ip_addr_t ip_nexthop; // ipv4 index 13 0x0000'0000'ffff'ffff
	301	// ipv6 index 12 0xffff'ffff'ffff'ffff
	302	// ipv6 index 13 0xffff'ffff'ffff'ffff
	303
	304	#ifdef WORDS_BIGENDIAN
	305	# define OffsetNexthopv4 13
	306	# define MaskNexthopv4 0x00000000ffffffffLL
	307	# define ShiftNexthopv4 0
	308
	309	# define OffsetNexthopv6a 12
	310	# define OffsetNexthopv6b 13
	311	// MaskIPv6 and ShiftIPv6 already defined
	312
	313	#else
	314	# define OffsetNexthopv4 13
	315	# define MaskNexthopv4 0xffffffff00000000LL
	316	# define ShiftNexthopv4 0
	317
	318	# define OffsetNexthopv6a 12
	319	# define OffsetNexthopv6b 13
	320	#endif
	321
	322	// extension 11 / 12
	323	ip_addr_t bgp_nexthop; // ipv4 index 15 0x0000'0000'ffff'ffff
	324	// ipv6 index 14 0xffff'ffff'ffff'ffff
	325	// ipv6 index 15 0xffff'ffff'ffff'ffff
	326
	327	#ifdef WORDS_BIGENDIAN
	328	# define OffsetBGPNexthopv4 15
	329	# define MaskBGPNexthopv4 0x00000000ffffffffLL
	330	# define ShiftBGPNexthopv4 0
	331
	332	# define OffsetBGPNexthopv6a 14
	333	# define OffsetBGPNexthopv6b 15
	334	// MaskIPv6 and ShiftIPv6 already defined
	335
	336	#else
	337	# define OffsetBGPNexthopv4 15
	338	# define MaskBGPNexthopv4 0xffffffff00000000LL
	339	# define ShiftBGPNexthopv4 0
	340
	341	# define OffsetBGPNexthopv6a 14
	342	# define OffsetBGPNexthopv6b 15
	343	#endif
	344
	345	// extension 8
	346	union {
	347	struct {
	348	uint8_t dst_tos; // index 16 0xff00'0000'0000'0000
	349	uint8_t dir; // index 16 0x00ff'0000'0000'0000
	350	uint8_t src_mask; // index 16 0x0000'ff00'0000'0000
	351	uint8_t dst_mask; // index 16 0x0000'00ff'0000'0000
	352	};
	353	uint32_t any;
	354	};
	355
	356	// extension 13
	357	uint16_t src_vlan; // index 16 0x0000'0000'ffff'0000
	358	uint16_t dst_vlan; // index 16 0x0000'0000'0000'ffff
	359
	360	#ifdef WORDS_BIGENDIAN
	361	# define OffsetDstTos 16
	362	# define MaskDstTos 0xff00000000000000LL
	363	# define ShiftDstTos 56
	364
	365	# define OffsetDir 16
	366	# define MaskDir 0x00ff000000000000LL
	367	# define ShiftDir 48
	368
	369	# define OffsetMask 16
	370	# define MaskSrcMask 0x0000ff0000000000LL
	371	# define ShiftSrcMask 40
	372
	373	# define MaskDstMask 0x000000ff00000000LL
	374	# define ShiftDstMask 32
	375
	376	# define OffsetVlan 16
	377	# define MaskSrcVlan 0x00000000ffff0000LL
	378	# define ShiftSrcVlan 16
	379
	380	# define MaskDstVlan 0x000000000000ffffLL
	381	# define ShiftDstVlan 0
	382
	383	#else
	384	# define OffsetDstTos 16
	385	# define MaskDstTos 0x00000000000000ffLL
	386	# define ShiftDstTos 0
	387
	388	# define OffsetDir 16
	389	# define MaskDir 0x000000000000ff00LL
	390	# define ShiftDir 8
	391
	392	# define OffsetMask 16
	393	# define MaskSrcMask 0x0000000000ff0000LL
	394	# define ShiftSrcMask 16
	395
	396	# define MaskDstMask 0x00000000ff000000LL
	397	# define ShiftDstMask 24
	398
	399	# define OffsetVlan 16
	400	# define MaskSrcVlan 0x0000ffff00000000LL
	401	# define ShiftSrcVlan 32
	402
	403	# define MaskDstVlan 0xffff000000000000LL
	404	# define ShiftDstVlan 48
	405
	406	#endif
	407
	408	// extension 14 / 15
	409	uint64_t out_pkts; // index 17 0xffff'ffff'ffff'ffff
	410	# define OffsetOutPackets 17
	411	// MaskPackets and ShiftPackets already defined
	412
	413	// extension 16 / 17
	414	uint64_t out_bytes; // index 18 0xffff'ffff'ffff'ffff
	415	# define OffsetOutBytes 18
	416
	417	// extension 18 / 19
	418	uint64_t aggr_flows; // index 19 0xffff'ffff'ffff'ffff
	419	# define OffsetAggrFlows 19
	420	# define MaskFlows 0xffffffffffffffffLL
	421
	422	// extension 20
	423	uint64_t in_src_mac; // index 20 0xffff'ffff'ffff'ffff
	424	# define OffsetInSrcMAC 20
	425	# define MaskMac 0xffffffffffffffffLL
	426
	427	// extension 20
	428	uint64_t out_dst_mac; // index 21 0xffff'ffff'ffff'ffff
	429	# define OffsetOutDstMAC 21
	430
	431	// extension 21
	432	uint64_t in_dst_mac; // index 22 0xffff'ffff'ffff'ffff
	433	# define OffsetInDstMAC 22
	434
	435	// extension 21
	436	uint64_t out_src_mac; // index 23 0xffff'ffff'ffff'ffff
	437	# define OffsetOutSrcMAC 23
	438
	439	// extension 22
	440	uint32_t mpls_label[10];
	441	# define OffsetMPLS12 24
	442	# define OffsetMPLS34 25
	443	# define OffsetMPLS56 26
	444	# define OffsetMPLS78 27
	445	# define OffsetMPLS910 28
	446
	447	#ifdef WORDS_BIGENDIAN
	448	# define MaskMPLSlabelOdd 0x00fffff000000000LL
	449	# define ShiftMPLSlabelOdd 36
	450	# define MaskMPLSexpOdd 0x0000000e00000000LL
	451	# define ShiftMPLSexpOdd 33
	452
	453	# define MaskMPLSlabelEven 0x0000000000fffff0LL
	454	# define ShiftMPLSlabelEven 4
	455	# define MaskMPLSexpEven 0x000000000000000eLL
	456	# define ShiftMPLSexpEven 1
	457	#else
	458	# define MaskMPLSlabelOdd 0x000000000000fff0LL
	459	# define ShiftMPLSlabelOdd 4
	460	# define MaskMPLSexpOdd 0x000000000000000eLL
	461	# define ShiftMPLSexpOdd 1
	462
	463	# define MaskMPLSlabelEven 0x00fffff000000000LL
	464	# define ShiftMPLSlabelEven 36
	465	# define MaskMPLSexpEven 0x0000000e00000000LL
	466	# define ShiftMPLSexpEven 33
	467
	468	#endif
	469
	470	// extension 23 / 24
	471	ip_addr_t ip_router; // ipv4 index 30 0x0000'0000'ffff'ffff
	472	// ipv6 index 29 0xffff'ffff'ffff'ffff
	473	// ipv6 index 30 0xffff'ffff'ffff'ffff
	474
	475	#ifdef WORDS_BIGENDIAN
	476	# define OffsetRouterv4 30
	477	# define MaskRouterv4 0x00000000ffffffffLL
	478	# define ShiftRouterv4 0
	479
	480	# define OffsetRouterv6a 29
	481	# define OffsetRouterv6b 30
	482	// MaskIPv6 and ShiftIPv6 already defined
	483
	484	#else
	485	# define OffsetRouterv4 30
	486	# define MaskRouterv4 0xffffffff00000000LL
	487	# define ShiftRouterv4 0
	488
	489	# define OffsetRouterv6a 29
	490	# define OffsetRouterv6b 30
	491	#endif
	492
	493	uint16_t sec_group_tag; // sec group tag index 31 0xffff'0000'0000'0000
	494	uint8_t engine_type; // type index 31 0x0000'ff00'0000'0000
	495	uint8_t engine_id; // ID index 31 0x0000'00ff'0000'0000
	496	uint32_t fill2;
	497
	498	# define OffsetRouterID 31
	499	#ifdef WORDS_BIGENDIAN
	500	# define MaskEngineType 0x0000FF0000000000LL
	501	# define ShiftEngineType 40
	502	# define MaskEngineID 0x000000FF00000000LL
	503	# define ShiftEngineID 32
	504
	505	#else
	506	# define MaskEngineType 0x0000000000FF0000LL
	507	# define ShiftEngineType 16
	508	# define MaskEngineID 0x00000000FF000000LL
	509	# define ShiftEngineID 24
	510	#endif
	511
	512	// IPFIX extensions in v9
	513	// BGP next/prev AS
	514	uint32_t bgpNextAdjacentAS; // index 32 0xffff'ffff'0000'0000
	515	uint32_t bgpPrevAdjacentAS; // index 32 0x0000'0000'ffff'ffff
	516
	517	// extension 18
	518	# define OffsetBGPadj 32
	519	#ifdef WORDS_BIGENDIAN
	520	# define MaskBGPadjNext 0xFFFFFFFF00000000LL
	521	# define ShiftBGPadjNext 32
	522	# define MaskBGPadjPrev 0x00000000FFFFFFFFLL
	523	# define ShiftBGPadjPrev 0
	524
	525	#else
	526	# define MaskBGPadjNext 0x00000000FFFFFFFFLL
	527	# define ShiftBGPadjNext 0
	528	# define MaskBGPadjPrev 0xFFFFFFFF00000000LL
	529	# define ShiftBGPadjPrev 32
	530	#endif
	531
	532	// NSEL extensions
	533	#ifdef NSEL
	534	#define NSEL_BASE_OFFSET (offsetof(master_record_t, conn_id) >> 3)
	535
	536	// common block
	537	# define OffsetConnID NSEL_BASE_OFFSET
	538	# define OffsetNATevent NSEL_BASE_OFFSET
	539	uint32_t conn_id; // index OffsetConnID 0xffff'ffff'0000'0000
	540	uint8_t event; // index OffsetConnID 0x0000'0000'ff00'0000
	541	#define FW_EVENT 1
	542	#define NAT_EVENT 2
	543	uint8_t event_flag; // index OffsetConnID 0x0000'0000'00ff'0000
	544	uint16_t fw_xevent; // index OffsetConnID 0x0000'0000'0000'ffff
	545	uint64_t event_time; // index OffsetConnID +1 0x1111'1111'1111'1111
	546	#ifdef WORDS_BIGENDIAN
	547	# define MaskConnID 0xFFFFFFFF00000000LL
	548	# define ShiftConnID 32
	549	# define MaskFWevent 0x00000000FF000000LL
	550	# define ShiftFWevent 24
	551	# define MasNATevent 0x00000000FF000000LL
	552	# define ShiftNATevent 24
	553	# define MaskFWXevent 0x000000000000FFFFLL
	554	# define ShiftFWXevent 0
	555	#else
	556	# define MaskConnID 0x00000000FFFFFFFFLL
	557	# define ShiftConnID 0
	558	# define MaskFWevent 0x000000FF00000000LL
	559	# define ShiftFWevent 32
	560	# define MasNATevent 0x000000FF00000000LL
	561	# define ShiftNATevent 32
	562	# define MaskFWXevent 0xFFFF000000000000LL
	563	# define ShiftFWXevent 48
	564
	565	#endif
	566
	567	// xlate ip/port
	568	# define OffsetXLATEPort NSEL_BASE_OFFSET+2
	569	uint16_t xlate_src_port; // index OffsetXLATEPort 0xffff'0000'0000'0000
	570	uint16_t xlate_dst_port; // index OffsetXLATEPort 0x0000'ffff'0000'0000
	571	uint32_t xlate_flags;
	572	# define OffsetXLATESRCIP NSEL_BASE_OFFSET+3
	573	ip_addr_t xlate_src_ip; // ipv4 OffsetXLATESRCIP +1 0x0000'0000'ffff'ffff
	574	// ipv6 OffsetXLATESRCIP 0xffff'ffff'ffff'ffff
	575	// ipv6 OffsetXLATESRCIP 0xffff'ffff'ffff'ffff
	576
	577	ip_addr_t xlate_dst_ip; // ipv4 OffsetXLATEDSTIP +1 0x0000'0000'ffff'ffff
	578	// ipv6 OffsetXLATEDSTIP 0xffff'ffff'ffff'ffff
	579	// ipv6 OffsetXLATEDSTIP 0xffff'ffff'ffff'ffff
	580	#ifdef WORDS_BIGENDIAN
	581	# define MaskXLATESRCPORT 0xFFFF000000000000LL
	582	# define ShiftXLATESRCPORT 48
	583	# define MaskXLATEDSTPORT 0x0000FFFF00000000LL
	584	# define ShiftXLATEDSTPORT 32
	585
	586	# define OffsetXLATESRCv4 OffsetXLATESRCIP+1
	587	# define MaskXLATEIPv4 0x00000000fFFFFFFFLL
	588	# define ShiftXLATEIPv4 0
	589
	590	# define OffsetXLATESRCv6a OffsetXLATESRCIP
	591	# define OffsetXLATESRCv6b OffsetXLATESRCIP+1
	592
	593	# define OffsetXLATEDSTv6a OffsetXLATESRCIP+2
	594	# define OffsetXLATEDSTv6b OffsetXLATESRCIP+3
	595
	596	#else
	597	# define MaskXLATESRCPORT 0x000000000000FFFFLL
	598	# define ShiftXLATESRCPORT 0
	599	# define MaskXLATEDSTPORT 0x00000000FFFF0000LL
	600	# define ShiftXLATEDSTPORT 16
	601
	602	# define OffsetXLATESRCv4 OffsetXLATESRCIP+1
	603	# define MaskXLATEIPv4 0xFFFFFFFF00000000LL
	604	# define ShiftXLATEIPv4 32
	605
	606	# define OffsetXLATESRCv6a OffsetXLATESRCIP
	607	# define OffsetXLATESRCv6b OffsetXLATESRCIP+1
	608
	609	# define OffsetXLATEDSTv6a OffsetXLATESRCIP+2
	610	# define OffsetXLATEDSTv6b OffsetXLATESRCIP+3
	611
	612	#endif
	613
	614
	615	// ingress/egress ACL id
	616	# define OffsetIngressAclId NSEL_BASE_OFFSET+7
	617	# define OffsetIngressAceId NSEL_BASE_OFFSET+7
	618	# define OffsetIngressGrpId NSEL_BASE_OFFSET+8
	619	# define OffsetEgressAclId NSEL_BASE_OFFSET+8
	620	# define OffsetEgressAceId NSEL_BASE_OFFSET+9
	621	# define OffsetEgressGrpId NSEL_BASE_OFFSET+9
	622	uint32_t ingress_acl_id[3]; // index OffsetIngressAclId 0xffff'ffff'0000'0000
	623	// index OffsetIngressAceId 0x0000'0000'ffff'ffff
	624	// index OffsetIngressGrpId 0xffff'ffff'0000'0000
	625	uint32_t egress_acl_id[3]; // index OffsetEgressAclId 0x0000'0000'ffff'ffff
	626	// index OffsetEgressAceId 0xffff'ffff'0000'0000
	627	// index OffsetEgressGrpId 0x0000'0000'ffff'ffff
	628	#ifdef WORDS_BIGENDIAN
	629	#define MaskIngressAclId 0xffffffff00000000LL
	630	#define ShiftIngressAclId 32
	631	#define MaskIngressAceId 0x00000000ffffffffLL
	632	#define ShiftIngressAceId 0
	633	#define MaskIngressGrpId 0xffffffff00000000LL
	634	#define ShiftIngressGrpId 32
	635	#define MaskEgressAclId 0x00000000ffffffffLL
	636	#define ShiftEgressAclId 0
	637	#define MaskEgressAceId 0xffffffff00000000LL
	638	#define ShiftEgressAceId 32
	639	#define MaskEgressGrpId 0x00000000ffffffffLL
	640	#define ShiftEgressGrpId 0
	641	#else
	642	#define MaskIngressAclId 0x00000000ffffffffLL
	643	#define ShiftIngressAclId 0
	644	#define MaskIngressAceId 0xffffffff00000000LL
	645	#define ShiftIngressAceId 32
	646	#define MaskIngressGrpId 0x00000000ffffffffLL
	647	#define ShiftIngressGrpId 0
	648	#define MaskEgressAclId 0xffffffff00000000LL
	649	#define ShiftEgressAclId 32
	650	#define MaskEgressAceId 0x00000000ffffffffLL
	651	#define ShiftEgressAceId 0
	652	#define MaskEgressGrpId 0xffffffff00000000LL
	653	#define ShiftEgressGrpId 32
	654	#endif
	655
	656	// username
	657	# define OffsetUsername NSEL_BASE_OFFSET+10
	658	char username[72];
	659
	660	// NAT extensions
	661	// NAT event is mapped into ASA event
	662	#define NAT_BASE_OFFSET (offsetof(master_record_t, ingress_vrfid) >> 3)
	663	// common block
	664	# define OffsetNELcommon NEL_BASE_OFFSET
	665	# define OffsetIVRFID NAT_BASE_OFFSET
	666	# define OffsetEVRFID NAT_BASE_OFFSET
	667	# define OffsetPortBlock NAT_BASE_OFFSET+1
	668	uint32_t ingress_vrfid; // OffsetIVRFID 0xffff'ffff'0000'0000
	669	uint32_t egress_vrfid; // OffsetEVRFID 0x0000'0000'ffff'ffff
	670
	671	// Port block allocation
	672	uint16_t block_start; // OffsetPortBlock 0xffff'0000'0000'0000
	673	uint16_t block_end; // OffsetPortBlock 0x0000'ffff'0000'0000
	674	uint16_t block_step; // OffsetPortBlock 0x0000'0000'ffff'0000
	675	uint16_t block_size; // OffsetPortBlock 0x0000'0000'0000'ffff
	676
	677	#ifdef WORDS_BIGENDIAN
	678	# define MaskIVRFID 0xFFFFFFFF00000000LL
	679	# define ShiftIVRFID 32
	680	# define MaskEVRFID 0x00000000FFFFFFFFLL
	681	# define ShiftEVRFID 0
	682	# define MaskPortBlockStart 0xFFFF000000000000LL
	683	# define ShiftPortBlockStart 48
	684	# define MaskPortBlockEnd 0x0000FFFF00000000LL
	685	# define ShiftPortBlockEnd 32
	686	# define MaskPortBlockStep 0x00000000FFFF0000LL
	687	# define ShiftPortBlockStep 16
	688	# define MaskPortBlockSize 0x000000000000FFFFLL
	689	# define ShiftPortBlockSize 0
	690	#else
	691	# define MaskIVRFID 0x00000000FFFFFFFFLL
	692	# define ShiftIVRFID 0
	693	# define MaskEVRFID 0xFFFFFFFF00000000LL
	694	# define ShiftEVRFID 32
	695	# define MaskPortBlockStart 0x000000000000FFFFLL
	696	# define ShiftPortBlockStart 0
	697	# define MaskPortBlockEnd 0x00000000FFFF0000LL
	698	# define ShiftPortBlockEnd 16
	699	# define MaskPortBlockStep 0x0000FFFF00000000LL
	700	# define ShiftPortBlockStep 32
	701	# define MaskPortBlockSize 0xFFFF000000000000LL
	702	# define ShiftPortBlockSize 48
	703	#endif
	704
	705	#endif
	706
	707	// latency extension
	708	uint64_t client_nw_delay_usec; // index LATENCY_BASE_OFFSET 0xffff'ffff'ffff'ffff
	709	uint64_t server_nw_delay_usec; // index LATENCY_BASE_OFFSET + 1 0xffff'ffff'ffff'ffff
	710	uint64_t appl_latency_usec; // index LATENCY_BASE_OFFSET + 2 0xffff'ffff'ffff'ffff
	711
	712	#define LATENCY_BASE_OFFSET (offsetof(master_record_t, client_nw_delay_usec) >> 3)
	713	# define OffsetClientLatency LATENCY_BASE_OFFSET
	714	# define OffsetServerLatency LATENCY_BASE_OFFSET + 1
	715	# define OffsetAppLatency LATENCY_BASE_OFFSET + 2
	716	# define MaskLatency 0xFFFFFFFFFFFFFFFFLL
	717	# define ShiftLatency 0
	718
	719	// flow received time in ms
	720	uint64_t received;
	721
	722	/* possible user extensions may fit here
	723	* - Put each extension into its own #ifdef
	724	* - Define the base offset for the user extension as reference to the first object
	725	* - Refer to this base offset for each of the values in the master record for the extension
	726	* - make sure the extension is 64bit aligned
	727	* - The user extension must be independant of the number of user extensions already defined
	728	* - the extension map must be updated accordingly
50	729	*/
51	730
52		#include "config.h"
53
54		typedef struct FilterParam {
55		uint16_t comp;
56		uint16_t direction;
57		uint32_t data;
58		uint32_t inout;
59		uint32_t acl;
60		uint32_t self;
61		} FilterParam_t;
62
63		/* IP tree type */
64		typedef RB_HEAD(IPtree, IPListNode) IPlist_t;
65
66		/* Port/AS tree type */
67		typedef RB_HEAD(ULongtree, ULongListNode) ULongtree_t;
68
69		/* parser/scanner prototypes */
70		int yyparse(void);
71
72		int yylex(void);
73
74		void lex_cleanup(void);
75
76		void lex_init(char *buf);
77
78		int ScreenIPString(char *string);
79
80		int ScreenIdentString(char *string);
81
82		// Insert the RB prototypes here
83		RB_PROTOTYPE(IPtree, IPListNode, entry, IPNodeCMP);
84
85		RB_PROTOTYPE(ULongtree, ULongListNode, entry, ULNodeCMP);
	731	#ifdef USER_EXTENSION_1
	732	uint64_t u64_1;
	733	# define Offset_BASE_U1 offsetof(master_record_t, u64_1)
	734	# define OffsetUser1_u64 Offset_BASE_U1
	735
	736	uint32_t u32_1;
	737	uint32_t u32_2;
	738	# define OffsetUser1_u32_1 Offset_BASE_U1 + 8
	739	# define MaskUser1_u32_1 0xffffffff00000000LL
	740	# define MaskUser1_u32_2 0x00000000ffffffffLL
	741
	742	#endif
	743
	744	// reference to exporter
	745	exporter_info_record_t *exp_ref;
	746
	747	// last entry in master record
	748	# define Offset_MR_LAST offsetof(master_record_t, map_ref)
	749	extension_map_t *map_ref;
	750
	751	// optional flowlabel
	752	char *label;
	753	} master_record_t;
	754
	755	// convenience type conversion record
	756	typedef struct type_mask_s {
	757	union {
	758	uint8_t val8[8];
	759	uint16_t val16[4];
	760	uint32_t val32[2];
	761	uint64_t val64;
	762	} val;
	763	} type_mask_t;
	764
86	765
87	766	#endif //_NFDUMP_H
88	767

+44

-44

bin/nfdump.test.out less more

13	13	dst port = 25
14	14	fwd status = 1
15	15	tcp flags = 0x01 .......F
16		proto = 6 TCP
	16	proto = 6 TCP
17	17	(src)tos = 2
18	18	(in)packets = 202
19	19	(in)bytes = 303

68	68	dst port = 25
69	69	fwd status = 2
70	70	tcp flags = 0x01 .......F
71		proto = 6 TCP
	71	proto = 6 TCP
72	72	(src)tos = 2
73	73	(in)packets = 202
74	74	(in)bytes = 303

123	123	dst port = 25
124	124	fwd status = 3
125	125	tcp flags = 0x01 .......F
126		proto = 6 TCP
	126	proto = 6 TCP
127	127	(src)tos = 2
128	128	(in)packets = 101
129	129	(in)bytes = 102

178	178	dst port = 25
179	179	fwd status = 4
180	180	tcp flags = 0x01 .......F
181		proto = 6 TCP
	181	proto = 6 TCP
182	182	(src)tos = 2
183	183	(in)packets = 101
184	184	(in)bytes = 102

233	233	dst port = 25
234	234	fwd status = 5
235	235	tcp flags = 0x01 .......F
236		proto = 17 UDP
	236	proto = 17 UDP
237	237	(src)tos = 1
238	238	(in)packets = 1001
239	239	(in)bytes = 1002

288	288	dst port = 25
289	289	fwd status = 6
290	290	tcp flags = 0x02 ......S.
291		proto = 51 AH
	291	proto = 51 AH
292	292	(src)tos = 2
293	293	(in)packets = 10001
294	294	(in)bytes = 10002

343	343	dst port = 25
344	344	fwd status = 7
345	345	tcp flags = 0x04 .....R..
346		proto = 6 TCP
	346	proto = 6 TCP
347	347	(src)tos = 3
348	348	(in)packets = 100001
349	349	(in)bytes = 100002

398	398	dst port = 25
399	399	fwd status = 8
400	400	tcp flags = 0x08 ....P...
401		proto = 6 TCP
	401	proto = 6 TCP
402	402	(src)tos = 4
403	403	(in)packets = 1000001
404	404	(in)bytes = 1000002

453	453	dst port = 25
454	454	fwd status = 9
455	455	tcp flags = 0x01 .......F
456		proto = 6 TCP
	456	proto = 6 TCP
457	457	(src)tos = 4
458	458	(in)packets = 10000001
459	459	(in)bytes = 1001

508	508	dst port = 25
509	509	fwd status = 10
510	510	tcp flags = 0x10 ...A....
511		proto = 6 TCP
	511	proto = 6 TCP
512	512	(src)tos = 5
513	513	(in)packets = 500
514	514	(in)bytes = 10000001

563	563	dst port = 25
564	564	fwd status = 11
565	565	tcp flags = 0x10 ...A....
566		proto = 6 TCP
	566	proto = 6 TCP
567	567	(src)tos = 5
568	568	(in)packets = 500
569	569	(in)bytes = 10000001

618	618	dst port = 25
619	619	fwd status = 12
620	620	tcp flags = 0x20 ..U.....
621		proto = 6 TCP
	621	proto = 6 TCP
622	622	(src)tos = 255
623	623	(in)packets = 5000
624	624	(in)bytes = 100000001

673	673	dst port = 25
674	674	fwd status = 13
675	675	tcp flags = 0x3f ..UAPRSF
676		proto = 6 TCP
	676	proto = 6 TCP
677	677	(src)tos = 0
678	678	(in)packets = 5000
679	679	(in)bytes = 1000000001

727	727	ICMP = 0.8 type.code
728	728	fwd status = 14
729	729	tcp flags = 0x00 ........
730		proto = 1 ICMP
	730	proto = 1 ICMP
731	731	(src)tos = 0
732	732	(in)packets = 50002
733	733	(in)bytes = 50000

782	782	dst port = 25000
783	783	fwd status = 15
784	784	tcp flags = 0x00 ........
785		proto = 6 TCP
	785	proto = 6 TCP
786	786	(src)tos = 0
787	787	(in)packets = 500001
788	788	(in)bytes = 500000

831	831	last = 1089534910 [2004-07-11 10:35:10]
832	832	msec_first = 510
833	833	msec_last = 520
834		src addr = fe80::2..:1234:0
835		dst addr = fe80::2..35:4321
	834	src addr = fe80::2110:abcd:1234:0
	835	dst addr = fe80::2110:abcd:1235:4321
836	836	src port = 1024
837	837	dst port = 25
838	838	fwd status = 16
839	839	tcp flags = 0x1b ...AP.SF
840		proto = 6 TCP
	840	proto = 6 TCP
841	841	(src)tos = 0
842	842	(in)packets = 10
843	843	(in)bytes = 15100

886	886	last = 1089534930 [2004-07-11 10:35:30]
887	887	msec_first = 610
888	888	msec_last = 620
889		src addr = 2001:23..80:d01e
890		dst addr = 2001:62..52:38e5
	889	src addr = 2001:234:aabb:cc:211:24ff:fe80:d01e
	890	dst addr = 2001:620:1f:8:203:baff:fe52:38e5
891	891	src port = 10240
892	892	dst port = 52345
893	893	fwd status = 17
894	894	tcp flags = 0x1b ...AP.SF
895		proto = 6 TCP
	895	proto = 6 TCP
896	896	(src)tos = 0
897	897	(in)packets = 10100
898	898	(in)bytes = 15000000

900	900	output = 14
901	901	src as = 775
902	902	dst as = 8404
903		src mask = 16 2001:234:aabb::/16
904		dst mask = 24 2001:620:0:8::/24
	903	src mask = 88 2001:234:aabb:cc:211:2400::/88
	904	dst mask = 48 2001:620:1f::/48
905	905	dst tos = 128
906	906	direction = 1
907	907	ip next hop = 172.72.1.2

941	941	last = 1089534950 [2004-07-11 10:35:50]
942	942	msec_first = 710
943	943	msec_last = 720
944		src addr = 2001:23..80:d01e
945		dst addr = 2001:62..52:38e5
	944	src addr = 2001:234:aabb:cc:211:24ff:fe80:d01e
	945	dst addr = 2001:620:1f:8:203:baff:fe52:38e5
946	946	src port = 10240
947	947	dst port = 52345
948	948	fwd status = 18
949	949	tcp flags = 0x1b ...AP.SF
950		proto = 6 TCP
	950	proto = 6 TCP
951	951	(src)tos = 0
952	952	(in)packets = 10100000
953	953	(in)bytes = 4294967296

955	955	output = 14
956	956	src as = 775
957	957	dst as = 8404
958		src mask = 16 2001:234:aabb::/16
959		dst mask = 24 2001:620:0:8::/24
	958	src mask = 88 2001:234:aabb:cc:211:2400::/88
	959	dst mask = 48 2001:620:1f::/48
960	960	dst tos = 128
961	961	direction = 1
962	962	ip next hop = 172.72.1.2

996	996	last = 1089534970 [2004-07-11 10:36:10]
997	997	msec_first = 810
998	998	msec_last = 820
999		src addr = 2001:23..80:d01e
1000		dst addr = 2001:62..52:38e5
	999	src addr = 2001:234:aabb:cc:211:24ff:fe80:d01e
	1000	dst addr = 2001:620:1f:8:203:baff:fe52:38e5
1001	1001	src port = 10240
1002	1002	dst port = 52345
1003	1003	fwd status = 19
1004	1004	tcp flags = 0x1b ...AP.SF
1005		proto = 6 TCP
	1005	proto = 6 TCP
1006	1006	(src)tos = 0
1007	1007	(in)packets = 4294967296
1008	1008	(in)bytes = 15000000

1010	1010	output = 14
1011	1011	src as = 775
1012	1012	dst as = 8404
1013		src mask = 16 2001:234:aabb::/16
1014		dst mask = 24 2001:620:0:8::/24
	1013	src mask = 88 2001:234:aabb:cc:211:2400::/88
	1014	dst mask = 48 2001:620:1f::/48
1015	1015	dst tos = 128
1016	1016	direction = 1
1017	1017	ip next hop = 172.72.1.2

1051	1051	last = 1089534990 [2004-07-11 10:36:30]
1052	1052	msec_first = 910
1053	1053	msec_last = 920
1054		src addr = 2001:23..80:d01e
1055		dst addr = 2001:62..52:38e5
	1054	src addr = 2001:234:aabb:cc:211:24ff:fe80:d01e
	1055	dst addr = 2001:620:1f:8:203:baff:fe52:38e5
1056	1056	src port = 10240
1057	1057	dst port = 52345
1058	1058	fwd status = 20
1059	1059	tcp flags = 0x1b ...AP.SF
1060		proto = 6 TCP
	1060	proto = 6 TCP
1061	1061	(src)tos = 0
1062	1062	(in)packets = 4294967296
1063	1063	(in)bytes = 8589934592

1065	1065	output = 14
1066	1066	src as = 775
1067	1067	dst as = 8404
1068		src mask = 16 2001:234:aabb::/16
1069		dst mask = 24 2001:620:0:8::/24
	1068	src mask = 88 2001:234:aabb:cc:211:2400::/88
	1069	dst mask = 48 2001:620:1f::/48
1070	1070	dst tos = 128
1071	1071	direction = 1
1072	1072	ip next hop = 172.72.1.2

1112	1112	dst port = 52345
1113	1113	fwd status = 21
1114	1114	tcp flags = 0x1b ...AP.SF
1115		proto = 6 TCP
	1115	proto = 6 TCP
1116	1116	(src)tos = 0
1117	1117	(in)packets = 10100000
1118	1118	(in)bytes = 4294967296

1167	1167	dst port = 52345
1168	1168	fwd status = 22
1169	1169	tcp flags = 0x1b ...AP.SF
1170		proto = 6 TCP
	1170	proto = 6 TCP
1171	1171	(src)tos = 0
1172	1172	(in)packets = 4294967296
1173	1173	(in)bytes = 15000000

1222	1222	dst port = 52345
1223	1223	fwd status = 23
1224	1224	tcp flags = 0x1b ...AP.SF
1225		proto = 6 TCP
	1225	proto = 6 TCP
1226	1226	(src)tos = 0
1227	1227	(in)packets = 4294967296
1228	1228	(in)bytes = 8589934592

1277	1277	dst port = 52345
1278	1278	fwd status = 24
1279	1279	tcp flags = 0x1b ...AP.SF
1280		proto = 6 TCP
	1280	proto = 6 TCP
1281	1281	(src)tos = 0
1282	1282	(in)packets = 4294967296
1283	1283	(in)bytes = 8589934592

1332	1332	dst port = 52345
1333	1333	fwd status = 25
1334	1334	tcp flags = 0x1b ...AP.SF
1335		proto = 6 TCP
	1335	proto = 6 TCP
1336	1336	(src)tos = 0
1337	1337	(in)packets = 4294967296
1338	1338	(in)bytes = 8589934592

1387	1387	dst port = 52345
1388	1388	fwd status = 26
1389	1389	tcp flags = 0x1b ...AP.SF
1390		proto = 6 TCP
	1390	proto = 6 TCP
1391	1391	(src)tos = 0
1392	1392	(in)packets = 4294967296
1393	1393	(in)bytes = 8589934592

+104

-0

bin/nfdump.test2.diff less more

	0	6c6
	1	< size = 184
	2	---
	3	> size = 172
	4	61c61
	5	< size = 184
	6	---
	7	> size = 172
	8	116c116
	9	< size = 184
	10	---
	11	> size = 172
	12	171c171
	13	< size = 184
	14	---
	15	> size = 172
	16	226c226
	17	< size = 184
	18	---
	19	> size = 172
	20	281c281
	21	< size = 184
	22	---
	23	> size = 172
	24	336c336
	25	< size = 184
	26	---
	27	> size = 172
	28	391c391
	29	< size = 184
	30	---
	31	> size = 172
	32	446c446
	33	< size = 184
	34	---
	35	> size = 172
	36	501c501
	37	< size = 184
	38	---
	39	> size = 172
	40	556c556
	41	< size = 184
	42	---
	43	> size = 172
	44	611c611
	45	< size = 184
	46	---
	47	> size = 172
	48	666c666
	49	< size = 184
	50	---
	51	> size = 172
	52	721c721
	53	< size = 184
	54	---
	55	> size = 172
	56	775c775
	57	< size = 184
	58	---
	59	> size = 172
	60	830c830
	61	< size = 208
	62	---
	63	> size = 196
	64	885c885
	65	< size = 208
	66	---
	67	> size = 196
	68	940c940
	69	< size = 212
	70	---
	71	> size = 200
	72	995c995
	73	< size = 212
	74	---
	75	> size = 200
	76	1050c1050
	77	< size = 216
	78	---
	79	> size = 204
	80	1105c1105
	81	< size = 188
	82	---
	83	> size = 176
	84	1160c1160
	85	< size = 188
	86	---
	87	> size = 176
	88	1215c1215
	89	< size = 192
	90	---
	91	> size = 180
	92	1270c1270
	93	< size = 196
	94	---
	95	> size = 184
	96	1325c1325
	97	< size = 196
	98	---
	99	> size = 184
	100	1380c1380
	101	< size = 200
	102	---
	103	> size = 188

+15

-40

bin/nfexpire.c less more

0	0	/*
1		* Copyright (c) 2017, Peter Haag
2		* Copyright (c) 2014, Peter Haag
3		* Copyright (c) 2009, Peter Haag
	1	* Copyright (c) 2009-2019, Peter Haag
4	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
5	3	* All rights reserved.
6	4	*

37	35	#include <stdlib.h>
38	36	#include <string.h>
39	37	#include <sys/types.h>
	38	#include <sys/param.h>
40	39	#include <sys/stat.h>
41	40	#include <sys/uio.h>
42	41	#include <unistd.h>
43	42	#include <dirent.h>
44		#include <sys/param.h>
45	43	#include <fcntl.h>
46	44	#include <errno.h>
47	45	#include <ctype.h>

62	60	#include <stdint.h>
63	61	#endif
64	62
	63	#include "util.h"
65	64	#include "bookkeeper.h"
66	65	#include "nfstatfile.h"
67	66	#include "expire.h"
68		#include "util.h"
69	67
70	68	static void usage(char *name);
71	69

89	87
90	88	void CheckDataDir( char *datadir) {
91	89	if ( datadir ) {
92		fprintf(stderr, "Specify only one option out of -l -e -r -u or -p \n");
	90	LogError("Only one option allowed out of -l -e -r -u or -p");
93	91	exit(250);
94	92	}
95	93	} // End of CheckDataDir
96
97		static char AbsolutePath(char dirname) {
98		char *path;
99
100		if ( !dirname )
101		return NULL;
102
103		if ( dirname[0] == '/' ) // it's already absolute path
104		return dirname;
105
106		path = (char *)malloc(MAXPATHLEN);
107		if ( !path ) {
108		LogError("malloc() error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
109		return NULL;
110		}
111		memset((void *)path, 0, MAXPATHLEN);
112		getcwd(path, MAXPATHLEN-strlen(dirname)-2); // -2: one for '/' and one for '\0'
113		strncat(path, "/", 1);
114		strncat(path, dirname, strlen(dirname));
115		path[MAXPATHLEN-1] = '\0';
116
117		return path;
118
119		} // End of AbsolutePath
120	94
121	95	channel_t GetChannelList(char datadir, int is_profile, int do_rescan) {
122	96	channel_t *c, channel;

130	104	DIR *PDIR = opendir(datadir);
131	105	struct dirent *entry;
132	106	if ( !PDIR ) {
133		fprintf(stderr, "Can't read profiledir '%s': %s\n",datadir, strerror(errno) );
	107	LogError("Can't read profiledir '%s': %s", datadir, strerror(errno));
134	108	return NULL;
135	109	}
136	110	while ( ( entry = readdir(PDIR)) != NULL ) {

139	113	stringbuf[MAXPATHLEN-1] = '\0';
140	114
141	115	if ( stat(stringbuf, &stat_buf) ) {
142		fprintf(stderr, "Can't stat '%s': %s\n",stringbuf, strerror(errno) );
	116	LogError("Can't stat '%s': %s", stringbuf, strerror(errno));
143	117	continue;
144	118	}
145	119	if ( !S_ISDIR(stat_buf.st_mode) )

163	137
164	138	c = (channel_t )malloc(sizeof(channel_t));
165	139	if ( !*c ) {
166		LogError("malloc() error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
	140	LogError("malloc() error in %s line %d: %s", __FILE__, __LINE__, strerror(errno));
167	141	return NULL;
168	142	}
169	143	memset((void )c, 0, sizeof(channel_t));

191	165
192	166	(c)->books_stat = AccessBookkeeper(&((c)->books), (*c)->datadir);
193	167	if ( (*c)->books_stat == ERR_FAILED ) {
194		fprintf(stderr, "Failed to access bookkeeping record.\n");
	168	LogError("Failed to access bookkeeping record");
195	169	exit(250);
196	170	}
197	171

277	251	case 'w':
278	252	low_water = strtoll(optarg, NULL, 10);
279	253	if ( low_water > 100 ) {
280		fprintf(stderr, "Water mark > 100%%\n");
	254	LogError("Water mark > 100%%");
281	255	exit(250);
282	256	}
283	257	if ( low_water == 0 )

286	260	case 'T':
287	261	runtime = strtoll(optarg, NULL, 10);
288	262	if ( runtime > 3600 ) {
289		fprintf(stderr, "Runtime > 3600 (1h)\n");
	263	LogError("Runtime > 3600 (1h)");
290	264	exit(250);
291	265	}
292	266	break;

300	274
301	275	}
302	276
303		datadir = AbsolutePath(datadir);
	277	datadir = realpath(datadir, NULL);
304	278
305	279	if ( !datadir ) {
306		fprintf(stderr, "Missing data directory\n");
	280	LogError("Data directory %s", datadir);
	281	LogError("realpath() in %s:%d: %s", __FILE__, __LINE__, strerror(errno) );
307	282	usage(argv[0]);
308	283	exit(250);
309	284	}
310	285
311	286	stat(datadir, &fstat);
312	287	if ( !(fstat.st_mode & S_IFDIR) ) {
313		fprintf(stderr, "No such directory: %s\n", datadir);
	288	LogError("No such directory: %s", datadir);
314	289	exit(250);
315	290	}
316	291

369	344	printf("Rescan again, due to file changes in directory!\n");
370	345	}
371	346	if ( BookSequence(current_channel->books) != last_sequence ) {
372		fprintf(stderr, "Could not safely rescan the directory. Data is not consistent.\n");
	347	LogError("Could not safely rescan the directory. Data is not consistent");
373	348	ReleaseBookkeeper(current_channel->books, DETACH_ONLY);
374	349	if ( current_channel->status == OK )
375	350	WriteStatInfo(current_channel->dirstat);

+62

-25

bin/nfexport.c less more

0	0	/*
1		* Copyright (c) 2017, Peter Haag
2		* Copyright (c) 2014, Peter Haag
3		* Copyright (c) 2009, Peter Haag
	1	* Copyright (c) 2009-2020, Peter Haag
4	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
5	3	* All rights reserved.
6	4	*

50	48	#endif
51	49
52	50	#include "util.h"
53		#include "nf_common.h"
	51	#include "nfdump.h"
54	52	#include "nffile.h"
55	53	#include "nfx.h"
56	54	#include "nfstat.h"
57	55	#include "nflowcache.h"
58	56	#include "exporter.h"
59
	57	#include "output_util.h"
60	58	#include "nfexport.h"
61	59
62	60	#include "nfdump_inline.c"

112	110	// parse map for older NEL nat extension
113	111	has_nat = 0;
114	112
	113	int needConvert = 0;
115	114	num_extensions = 0;
116	115	i = 0;
117	116	while ( SourceMap->ex_id[i] ) {
118	117	switch (SourceMap->ex_id[i]) {
119	118	case EX_AGGR_FLOWS_4:
	119	needConvert = 1;
120	120	case EX_AGGR_FLOWS_8:
121	121	has_aggr_flows = 1;
122	122	break;
123	123	case EX_OUT_BYTES_4:
	124	needConvert = 1;
124	125	case EX_OUT_BYTES_8:
125	126	has_out_bytes = 1;
126	127	break;
127	128	case EX_OUT_PKG_4:
	129	needConvert = 1;
128	130	case EX_OUT_PKG_8:
129	131	has_out_packets = 1;
130	132	break;

138	140	num_extensions++;
139	141	}
140	142	#ifdef DEVEL
141		printf("map: num_extensions: %i, has_aggr_flows: %i, has_out_bytes: %i, has_out_packets: %i, has_nat: %i\n",
142		num_extensions, has_aggr_flows, has_out_bytes, has_out_packets, has_nat);
	143	printf("map: num_extensions: %i, has_aggr_flows: %i, has_out_bytes: %i, has_out_packets: %i, has_nat: %i, needConvert: %i\n",
	144	num_extensions, has_aggr_flows, has_out_bytes, has_out_packets, has_nat, needConvert);
143	145	#endif
144	146
145	147	// count missing extensions

161	163	printf("opt_extensions: %i, new_map_size: %i\n", opt_extensions,new_map_size );
162	164	PrintExtensionMap(SourceMap);
163	165	#endif
164		if ( opt_extensions ) {
	166	if ( opt_extensions \|\| needConvert ) {
165	167	// align 32bits
166	168	if (( new_map_size & 0x3 ) != 0 ) {
167	169	new_map_size += 4 - ( new_map_size & 0x3 );

203	205
204	206	new_map->size = new_map_size;
205	207
	208	i = 0;
	209	// convert 4 to 8 byte counters
	210	while ( new_map->ex_id[i] ) {
	211	switch (new_map->ex_id[i]) {
	212	case EX_AGGR_FLOWS_4:
	213	new_map->extension_size -= extension_descriptor[EX_AGGR_FLOWS_4].size;
	214	new_map->extension_size += extension_descriptor[EX_AGGR_FLOWS_8].size;
	215	new_map->ex_id[i] = EX_AGGR_FLOWS_8;
	216	break;
	217	case EX_OUT_BYTES_4:
	218	new_map->extension_size -= extension_descriptor[EX_OUT_BYTES_4].size;
	219	new_map->extension_size += extension_descriptor[EX_OUT_BYTES_8].size;
	220	new_map->ex_id[i] = EX_OUT_BYTES_8;
	221	break;
	222	case EX_OUT_PKG_4:
	223	new_map->extension_size -= extension_descriptor[EX_OUT_PKG_4].size;
	224	new_map->extension_size += extension_descriptor[EX_OUT_PKG_8].size;
	225	new_map->ex_id[i] = EX_OUT_PKG_8;
	226	break;
	227	// default: nothing to do
	228	}
	229	i++;
	230	}
	231
206	232	// add the missing extensions to the output map
207		// skip to end of current map
208		while ( new_map->ex_id[i] )
209		i++;
210
211	233	if ( has_nat ) {
212	234	new_map->ex_id[i++] = EX_NSEL_XLATE_PORTS;
213	235	new_map->extension_size += extension_descriptor[EX_NSEL_XLATE_PORTS].size;
214	236	}
215	237	// add missing map elements
216	238	if ( aggregate && !has_aggr_flows ) {
217		new_map->ex_id[i++] = EX_AGGR_FLOWS_4;
218		new_map->extension_size += extension_descriptor[EX_AGGR_FLOWS_4].size;
	239	new_map->ex_id[i++] = EX_AGGR_FLOWS_8;
	240	new_map->extension_size += extension_descriptor[EX_AGGR_FLOWS_8].size;
219	241	}
220	242	if ( bidir && !has_out_bytes ) {
221	243	new_map->ex_id[i++] = EX_OUT_BYTES_8;

231	253	new_map->ex_id[i] = 0;
232	254
233	255	#ifdef DEVEL
	256	printf("New/converted extension map:\n");
234	257	PrintExtensionMap(new_map);
235	258	#endif
236	259
237		free(extension_map_list->slot[map_id]->map);
238		extension_map_list->slot[map_id]->map = new_map;
	260	// set new export map
	261	extension_map_list->slot[map_id]->exportMap = new_map;
239	262
240	263	// Flush the map to disk
241	264	AppendToBuffer(nffile, (void *)new_map, new_map->size);

244	267
245	268	} // End of ExportExtensionMaps
246	269
247		int ExportFlowTable(nffile_t nffile, int aggregate, int bidir, int date_sorted, extension_map_list_t extension_map_list) {
	270	int ExportFlowTable(nffile_t nffile, int aggregate, int bidir, int GuessDir, int date_sorted, extension_map_list_t extension_map_list) {
248	271	hash_FlowTable *FlowTable;
249	272	FlowTableRecord_t *r;
250	273	SortElement_t *SortList;

310	333	flow_record->dOctets = r->counter[INBYTES];
311	334	flow_record->out_pkts = r->counter[OUTPACKETS];
312	335	flow_record->out_bytes = r->counter[OUTBYTES];
313		flow_record->aggr_flows = r->counter[FLOWS];
	336	flow_record->aggr_flows = r->counter[FLOWS];
314	337
315	338	// apply IP mask from aggregation, to provide a pretty output
316	339	if ( FlowTable->has_masks ) {

327	350	ApplyAggrMask(flow_record, aggr_record_mask);
328	351	}
329	352
	353	if ( GuessDir &&
	354	( flow_record->prot == IPPROTO_TCP \|\| flow_record->prot == IPPROTO_UDP) &&
	355	( flow_record->srcport < 1024 ) && ( flow_record->dstport > 1024 ) &&
	356	( flow_record->srcport < flow_record->dstport ) ) {
	357	SwapFlow(flow_record);
	358	}
	359
330	360	// switch to output extension map
331		flow_record->map_ref = extension_info->map;
332		flow_record->ext_map = extension_info->map->map_id;
	361	flow_record->map_ref = extension_info->exportMap ? extension_info->exportMap : extension_info->map;
	362	flow_record->ext_map = flow_record->map_ref->map_id;
333	363	PackRecord(flow_record, nffile);
334	364	#ifdef DEVEL
335		format_file_block_record((void *)flow_record, &string, 0);
	365	flow_record_to_raw((void *)flow_record, &string, 0);
336	366	printf("%s\n", string);
337	367	#endif
338	368	// Update statistics

352	382	extension_info = r->map_info_ref;
353	383
354	384	flow_record = &(extension_info->master_record);
355		ExpandRecord_v2( raw_record, extension_info, r->exp_ref, flow_record);
	385	ExpandRecord_v2(raw_record, extension_info, r->exp_ref, flow_record);
356	386	flow_record->dPkts = r->counter[INPACKETS];
357	387	flow_record->dOctets = r->counter[INBYTES];
358	388	flow_record->out_pkts = r->counter[OUTPACKETS];

374	404	ApplyAggrMask(flow_record, aggr_record_mask);
375	405	}
376	406
	407	if ( GuessDir &&
	408	( flow_record->prot == IPPROTO_TCP \|\| flow_record->prot == IPPROTO_UDP) &&
	409	( flow_record->srcport < 1024 ) && ( flow_record->dstport > 1024 ) &&
	410	( flow_record->srcport < flow_record->dstport ) ) {
	411	SwapFlow(flow_record);
	412	}
	413
377	414	// switch to output extension map
378		flow_record->map_ref = extension_info->map;
379		flow_record->ext_map = extension_info->map->map_id;
	415	flow_record->map_ref = extension_info->exportMap ? extension_info->exportMap : extension_info->map;
	416	flow_record->ext_map = flow_record->map_ref->map_id;
380	417	PackRecord(flow_record, nffile);
381	418	#ifdef DEVEL
382		format_file_block_record((void *)flow_record, &string, 0);
	419	flow_record_to_raw((void *)flow_record, &string, 0);
383	420	printf("%s\n", string);
384	421	#endif
385	422	// Update statistics

-1

bin/nfexport.h less more

36	36	#include "nffile.h"
37	37	#include "nfx.h"
38	38
39		int ExportFlowTable(nffile_t nffile, int aggregate, int bidir, int date_sorted, extension_map_list_t extension_map_list);
	39	int ExportFlowTable(nffile_t nffile, int aggregate, int bidir, int GuessDir, int date_sorted, extension_map_list_t extension_map_list);
40	40
41	41	#endif //_NFEXPORT_H
42	42

+11

-106

bin/nffile.c less more

0	0	/*
1		* Copyright (c) 2017, Peter Haag
2		* Copyright (c) 2014, Peter Haag
3		* Copyright (c) 2011, Peter Haag
	1	* Copyright (c) 2009-2020, Peter Haag
4	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
5	3	* All rights reserved.
6	4	*

53	51	#include <stdint.h>
54	52	#endif
55	53
	54	#include "util.h"
	55	#include "nfdump.h"
56	56	#include "minilzo.h"
57	57	#include "lz4.h"
58		#include "nf_common.h"
	58	#include "flist.h"
59	59	#include "nffile.h"
60		#include "flist.h"
61		#include "util.h"
	60	#include "nffileV2.h"
62	61
63	62	/* global vars */
64	63
65	64	// required for idet filter in nftree.c
66	65	char *CurrentIdent;
67	66
68
69		#define READ_FILE 1
70		#define WRITE_FILE 1
71	67
72	68	// LZO params
73	69	#define HEAP_ALLOC(var,size) \

629	625	return NULL;
630	626	}
631	627
632		/*
633		XXX catalogs not yet implemented
634		nffile->catalog = calloc(1, sizeof(catalog_t));
635		if ( !nffile->catalog ) {
636		LogError("malloc() error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
637		return NULL;
638		}
639		nffile->catalog->NumRecords = 0;
640		nffile->catalog->size = sizeof(catalog_t) - sizeof(data_block_header_t);
641		nffile->catalog->id = CATALOG_BLOCK;
642		nffile->catalog->pad = 0;
643		nffile->catalog->reserved = 0;
644		*/
645	628	// init data buffer
646	629	nffile->buff_size = 2 * BUFFSIZE;
647	630	for (i=0; i<NUM_BUFFS; i++ ) {

738	721
739	722	nffile->file_header->flags = flags;
740	723
741		/*
742		XXX catalogs not yet implemented
743		if ( nffile->catalog && nffile->catalog->NumRecords ) {
744		memset((void )nffile->catalog->entries, 0, nffile->catalog->NumRecords sizeof(struct catalog_entry_s));
745		nffile->catalog->NumRecords = 0;
746		nffile->catalog->size = 0;
747		}
748		*/
749	724	if ( nffile->stat_record ) {
750	725	memset((void *)nffile->stat_record, 0, sizeof(stat_record_t));
751	726	nffile->stat_record->first_seen = 0x7fffffff;

774	749	nffile->fd = 0;
775	750	return NULL;
776	751	}
777
778		/* skip writing catalog in this test version
779		XXX catalogs not yet implemented
780		if ( WriteExtraBlock(nffile, (data_block_header_t *)nffile->catalog) < 0 ) {
781		LogError("write() error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
782		close(nffile->fd);
783		return NULL;
784		}
785		*/
786	752
787	753	return nffile;
788	754

1076	1042	read_bytes = ret;
1077	1043
1078	1044	// Check for sane buffer size
1079		if ( nffile->block_header->size > BUFFSIZE ) {
	1045	if ( nffile->block_header->size > BUFFSIZE \|\|
	1046	nffile->block_header->size == 0 \|\| nffile->block_header->NumRecords == 0) {
1080	1047	// this is most likely a corrupt file
1081		LogError("Corrupt data file: Requested buffer size %u exceeds max. buffer size.\n", nffile->block_header->size);
	1048	LogError("Corrupt data file: Requested buffer size %u exceeds max. buffer size", nffile->block_header->size);
1082	1049	return NF_CORRUPT;
1083	1050	}
	1051
	1052	// check block compression - defaults to file compression setting
1084	1053
1085	1054	compression = FILE_COMPRESSION(nffile);
1086	1055	ret = read(nffile->fd, nffile->buff_ptr, nffile->block_header->size);

1204	1173
1205	1174	} // End of WriteBlock
1206	1175
1207		inline void ExpandRecord_v1(common_record_t input_record, master_record_t output_record ) {
1208		uint32_t *u;
1209		size_t size;
1210		void p = (void )input_record;
1211
1212		// Copy common data block
1213		size = sizeof(common_record_t) - sizeof(uint8_t[4]);
1214		memcpy((void *)output_record, p, size);
1215		p = (void *)input_record->data;
1216
1217		if ( (input_record->flags & FLAG_IPV6_ADDR) != 0 ) { // IPv6
1218		// IPv6
1219		// keep compiler happy
1220		// memcpy((void )output_record->V6.srcaddr, p, 4 sizeof(uint64_t));
1221		memcpy((void )output_record->ip_union._ip_64.addr, p, 4 sizeof(uint64_t));
1222		p = (void )((pointer_addr_t)p + 4 sizeof(uint64_t));
1223		} else {
1224		// IPv4
1225		u = (uint32_t *)p;
1226		output_record->V6.srcaddr[0] = 0;
1227		output_record->V6.srcaddr[1] = 0;
1228		output_record->V4.srcaddr = u[0];
1229
1230		output_record->V6.dstaddr[0] = 0;
1231		output_record->V6.dstaddr[1] = 0;
1232		output_record->V4.dstaddr = u[1];
1233		p = (void )((pointer_addr_t)p + 2 sizeof(uint32_t));
1234		}
1235
1236		// packet counter
1237		if ( (input_record->flags & FLAG_PKG_64 ) != 0 ) {
1238		// 64bit packet counter
1239		value64_t l, v = (value64_t )p;
1240		l.val.val32[0] = v->val.val32[0];
1241		l.val.val32[1] = v->val.val32[1];
1242		output_record->dPkts = l.val.val64;
1243		p = (void *)((pointer_addr_t)p + sizeof(uint64_t));
1244		} else {
1245		// 32bit packet counter
1246		output_record->dPkts = ((uint32_t )p);
1247		p = (void *)((pointer_addr_t)p + sizeof(uint32_t));
1248		}
1249
1250		// byte counter
1251		if ( (input_record->flags & FLAG_BYTES_64 ) != 0 ) {
1252		// 64bit byte counter
1253		value64_t l, v = (value64_t )p;
1254		l.val.val32[0] = v->val.val32[0];
1255		l.val.val32[1] = v->val.val32[1];
1256		output_record->dOctets = l.val.val64;
1257		p = (void *)((pointer_addr_t)p + sizeof(uint64_t));
1258		} else {
1259		// 32bit bytes counter
1260		output_record->dOctets = ((uint32_t )p);
1261		p = (void *)((pointer_addr_t)p + sizeof(uint32_t));
1262		}
1263
1264		} // End of ExpandRecord_v1
1265
1266	1176	void ModifyCompressFile(char * rfile, char *Rfile, int compress) {
1267	1177	int i, anonymized, compression;
1268	1178	ssize_t ret;

1359	1269	void QueryFile(char *filename) {
1360	1270	int i;
1361	1271	nffile_t *nffile;
1362		uint32_t num_records, type1, type2, type3;
	1272	uint32_t num_records, type1, type2;
1363	1273	struct stat stat_buf;
1364	1274	ssize_t ret;
1365	1275	off_t fsize;

1379	1289	fsize = lseek(nffile->fd, 0, SEEK_CUR);
1380	1290	type1 = 0;
1381	1291	type2 = 0;
1382		type3 = 0;
1383	1292	printf("File : %s\n", filename);
1384	1293	printf ("Version : %u - %s\n", nffile->file_header->version,
1385	1294	FILE_IS_LZO_COMPRESSED (nffile) ? "lzo compressed" :

1419	1328	case DATA_BLOCK_TYPE_2:
1420	1329	type2++;
1421	1330	break;
1422		case Large_BLOCK_Type:
1423		type3++;
1424		break;
1425	1331	default:
1426	1332	printf("block %i has unknown type %u\n", i, nffile->block_header->id);
1427	1333	}

1449	1355
1450	1356	printf(" Type 1 : %u\n", type1);
1451	1357	printf(" Type 2 : %u\n", type2);
1452		printf(" Type 3 : %u\n", type3);
1453	1358	printf("Records : %u\n", num_records);
1454	1359
1455	1360	CloseFile(nffile);

+24

-1860

bin/nffile.h less more

0	0	/*
1		* Copyright (c) 2009-2019, Peter Haag
	1	* Copyright (c) 2009-2020, Peter Haag
2	2	* Copyright (c) 2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
3	3	* All rights reserved.
4	4	*

48	48
49	49	#define NF_DUMPFILE "nfcapd.current"
50	50
	51	#define NOT_COMPRESSED 0
	52	#define LZO_COMPRESSED 1
	53	#define BZ2_COMPRESSED 2
	54	#define LZ4_COMPRESSED 3
	55
51	56	/*
52	57	* output buffer max size, before writing data to the file
53	58	* used to cache flows before writing to disk. size: tradeoff between

69	74	*/
70	75
71	76	/*
72		* nfdump binary file layout
73		* =========================
	77	* nfdump binary file layout 1
	78	* ===========================
74	79	* Each data file starts with a file header, which identifies the file as an nfdump data file.
75	80	* The magic 16bit integer at the beginning of each file must read 0xA50C. This also guarantees
76	81	* that endian dependant files are read correct.

82	87	* +-----------+-------------+-------------+-------------+-----+-------------+
83	88	*/
84	89
85		#define NOT_COMPRESSED 0
86		#define LZO_COMPRESSED 1
87		#define BZ2_COMPRESSED 2
88		#define LZ4_COMPRESSED 3
89	90
90	91	typedef struct file_header_s {
91	92	uint16_t magic; // magic to recognize nfdump file type and endian type

120	121	} file_header_t;
121	122
122	123	/*
123		* Compatible with nfdump x.x.x file format: After the file header an
	124	* In file layout format 1: After the file header an
124	125	* inplicit stat record follows, which contains the statistics
125	126	* information about all netflow records in this file.
126	127	*/

154	155	uint32_t sequence_failure;
155	156	} stat_record_t;
156	157
157		typedef struct stat_header_s {
158		uint16_t type; // stat record type
159		// compatible stat type nfdump 1.5.x in new extended stat record type
160		#define STD_STAT_TYPE 0
161		uint16_t size; // size of the stat record in bytes without this header
162		} stat_header_t;
163
	158
	159	// legacy nfdump 1.5.x data block type
	160	#define DATA_BLOCK_TYPE_1 1
	161
	162	// nfdump 1.6.x data block type
	163	#define DATA_BLOCK_TYPE_2 2
164	164
165	165	/*
166	166	*

178	178	// 1 - block uncompressed
179	179	// 2 - block compressed
180	180	} data_block_header_t;
181
182		// compat nfdump 1.5.x data block type
183		#define DATA_BLOCK_TYPE_1 1
184
185		// nfdump 1.6.x data block type
186		#define DATA_BLOCK_TYPE_2 2
187
188		/*
189		*
190		* Block type 3
191		* ============
192		* same block header as type 2. Used for data other than flow data - e.g. histograms. Important difference:
193		* included data records have type L_record_header_t headers in order to allow larger data records.
194		*
195		*/
196
197		#define Large_BLOCK_Type 3
198
199		typedef struct L_record_header_s {
200		// record header
201		uint32_t type;
202		uint32_t size;
203		} L_record_header_t;
204	181
205	182	/*
206	183	* Generic file handle for reading/writing files

218	195	} nffile_t;
219	196
220	197	/*
221		* The new block type 2 introduces a changed common record and multiple extension records. This allows a more flexible data
	198	* The block type 2 contains a common record and multiple extension records. This allows a more flexible data
222	199	* storage of netflow v9 records and 3rd party extension to nfdump.
223	200	*
224	201	* A block type 2 may contain different record types, as described below.

241	218	#define PortHistogramType 3
242	219	#define BppHistogramType 4
243	220
244		// TC code - phased out
245		#define ExporterRecordType 5
246		#define SamplerRecordype 6
247
248		// replaces TC Types
	221	// Legacy records
	222	#define LegacyRecordType1 5
	223	#define LegacyRecordType2 6
	224
	225	// exporter/sampler types
249	226	#define ExporterInfoRecordType 7
250	227	#define ExporterStatRecordType 8
251	228	#define SamplerInfoRecordype 9
252
253		// new extended Common Record as intermediate solution to overcome 255 exporters
254		// requires moderate changes till 1.7
255		#define CommonRecordType 10
256
257		/*
258		* All records are 32bit aligned and layouted in a 64bit array. The numbers placed in () refer to the netflow v9 type id.
259		*
260		* Record type 1
261		* =============
262		* The record type 1 describes a netflow data record incl. all optional extensions for this record.
263		* A netflow data record requires at least the first 3 extensions 1..3. All other extensions are optional
264		* and described in the extensiion map. The common record contains a reference to the extension map which
265		* applies for this record.
266		*
267		* flags:
268		* bit 0: 0: IPv4 1: IPv6
269		* bit 1: 0: 32bit dPkts 1: 64bit dPkts
270		* bit 2: 0: 32bit dOctets 1: 64bit dOctets
271		* bit 3: 0: IPv4 next hop 1: IPv6 next hop
272		* bit 4: 0: IPv4 BGP next hop 1: BGP IPv6 next hop
273		* bit 5: 0: IPv4 exporter IP 1: IPv6 exporter IP
274		* bit 6: 0: flow 1: event
275		* bit 7: 0: unsampled 1: sampled flow - sampling applied
276		*
277		* Required extensions: 1,2,3
278		* ------------------------------
279		* A netflow record consists at least of a common record ( extension 0 ) and 3 required extension:
280		*
281		* Extension 1: IPv4 or IPv4 src and dst addresses Flags bit 0: 0: IPv4, 1: IPv6
282		* Extension 2: 32 or 64 bit packet counter Flags bit 1: 0: 32bit, 1: 64bit
283		* Extension 3: 32 or 64 bit byte counter Flags bit 2: 0: 32bit, 1: 64bit
284		*
285		* Commmon record - extension 0
286		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
287		* \| - \| 0 \| 1 \| 2 \| 3 \| 4 \| 5 \| 6 \| 7 \|
288		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
289		* \| 0 \| record type == 1 \| size \| flags \| tag \| ext. map \|
290		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
291		* \| 1 \| msec_first \| msec_last \| first (22) \|
292		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
293		* \| 2 \| last (21) \|fwd_status(89)\| tcpflags (6) \| proto (4) \| src tos (5) \|
294		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
295		* \| 3 \| srcport (7) \| dstport(11)/ICMP (32) \|
296		* +----+--------------+--------------+--------------+--------------+
297		*
298		* Commmon record - extension 0 - Type 10
299		* required for larger exporter ID reference
300		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
301		* \| - \| 0 \| 1 \| 2 \| 3 \| 4 \| 5 \| 6 \| 7 \|
302		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
303		* \| 0 \| record type == 10 \| size \| flags \| ext. map \|
304		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
305		* \| 1 \| msec_first \| msec_last \| first (22) \|
306		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
307		* \| 2 \| last (21) \|fwd_status(89)\| tcpflags (6) \| proto (4) \| src tos (5) \|
308		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
309		* \| 3 \| srcport (7) \| dstport(11)/ICMP (32) \| exporter ID \| reserved icmp type/code \|
310		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
311
312		*
313		*/
314
315		#define COMMON_BLOCK_ID 0
316	229
317	230	typedef struct record_header_s {
318	231	// record header

320	233	uint16_t size;
321	234	} record_header_t;
322	235
323		typedef struct common_record_s {
324		// record head
325		uint16_t type;
326		uint16_t size;
327
328		// record meta data
329		uint16_t flags;
330		#define FLAG_IPV6_ADDR 1
331		#define FLAG_PKG_64 2
332		#define FLAG_BYTES_64 4
333		#define FLAG_IPV6_NH 8
334		#define FLAG_IPV6_NHB 16
335		#define FLAG_IPV6_EXP 32
336		#define FLAG_EVENT 64
337		#define FLAG_SAMPLED 128
338
339		#define SetFlag(var, flag) (var \|= flag)
340		#define ClearFlag(var, flag) (var &= ~flag)
341		#define TestFlag(var, flag) (var & flag)
342
343		uint16_t ext_map;
344
345		// netflow common record
346		uint16_t msec_first;
347		uint16_t msec_last;
348		uint32_t first;
349		uint32_t last;
350
351		uint8_t fwd_status;
352		uint8_t tcp_flags;
353		uint8_t prot;
354		uint8_t tos;
355		uint16_t srcport;
356		uint16_t dstport;
357
358		uint16_t exporter_sysid;
359		uint16_t reserved;
360
361		// link to extensions
362		uint32_t data[1];
363		} common_record_t;
364
365		#define COMMON_RECORD_DATA_SIZE (sizeof(common_record_t) - sizeof(uint32_t) )
366
367		#define COMMON_BLOCK 0
368
369		/*
370		* Required extensions:
371		* --------------------
372		* Extension 1:
373		* IPv4/v6 address type
374		* IP version: IPv4
375		* \|
376		* Flags: xxxx xxx0
377		* IPv4:
378		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
379		* \| 0 \| srcip (8) \| dstip (12) \|
380		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
381		*
382		* IPv6:
383		* IP version: IPv6
384		* \|
385		* Flags: xxxx xxx1
386		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
387		* \| 0 \| srcip (27) \|
388		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
389		* \| 1 \| srcip (27) \|
390		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
391		* \| 2 \| dstip (28) \|
392		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
393		* \| 3 \| dstip (28) \|
394		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
395		*
396		*/
397
398		#define EX_IPv4v6 1
399
400		typedef struct ipv4_block_s {
401		uint32_t srcaddr;
402		uint32_t dstaddr;
403		uint8_t data[4]; // .. more data below
404		} ipv4_block_t;
405
406		typedef struct ipv6_block_s {
407		uint64_t srcaddr[2];
408		uint64_t dstaddr[2];
409		uint8_t data[4]; // .. more data below
410		} ipv6_block_t;
411
412		// single IP addr for next hop and bgp next hop
413		typedef struct ip_addr_s {
414		union {
415		struct {
416		#ifdef WORDS_BIGENDIAN
417		uint32_t fill[3];
418		uint32_t _v4;
419		#else
420		uint32_t fill1[2];
421		uint32_t _v4;
422		uint32_t fill2;
423		#endif
424		};
425		uint64_t _v6[2];
426		} ip_union;
427		#define IP_ADDR_T
428		} ip_addr_t;
429
430		#define V4 ip_union._v4
431		#define V6 ip_union._v6
432
433		/*
434		* Extension 2:
435		* In packet counter size
436		*
437		* In packet counter size 4byte
438		* \|
439		* Flags: xxxx xx0x
440		* +---++--------------+--------------+--------------+--------------+
441		* \| 0 \| in pkts (2) \|
442		* +---++--------------+--------------+--------------+--------------+
443		*
444		* In packet counter size 8byte
445		* \|
446		* Flags: xxxx xx1x
447		* +---++--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
448		* \| 0 \| in pkts (2) \|
449		* +---++--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
450		*
451		*/
452
453		#define EX_PACKET_4_8 2
454
455		typedef struct value32_s {
456		uint32_t val;
457		uint8_t data[4]; // .. more data below
458		} value32_t;
459
460		typedef struct value64_s {
461		union val_s {
462		uint64_t val64;
463		uint32_t val32[2];
464		} val;
465		uint8_t data[4]; // .. more data below
466		} value64_t;
467
468
469		/* Extension 3:
470		* in byte counter size
471		* In byte counter size 4byte
472		* \|
473		* Flags: xxxx x0xx
474		*
475		* +---++--------------+--------------+--------------+--------------+
476		* \| 0 \| in bytes (1) \|
477		* +---++--------------+--------------+--------------+--------------+
478		*
479		* In byte counter size 8byte
480		* \|
481		* Flags: xxxx x1xx
482		* +---++--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
483		* \| 0 \| in bytes (1) \|
484		* +---++--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
485		*/
486
487		#define EX_BYTE_4_8 3
488
489		/*
490		*
491		* Optional extension:
492		* ===================
493		*
494		* Interface record
495		* ----------------
496		* Interface records are optional and accepted as either 2 or 4 bytes numbers
497		* Extension 4:
498		* +---++--------------+--------------+--------------+--------------+
499		* \| 0 \| input (10) \| output (14) \|
500		* +---++--------------+--------------+--------------+--------------+
501		*/
502		#define EX_IO_SNMP_2 4
503		typedef struct tpl_ext_4_s {
504		uint16_t input;
505		uint16_t output;
506		uint8_t data[4]; // points to further data
507		} tpl_ext_4_t;
508
509		/*
510		* Extension 5:
511		* +---++--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
512		* \| 0 \| input (10) \| output (14) \|
513		* +---++--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
514		* Extension 4 and 5 are mutually exclusive in the extension map
515		*/
516		#define EX_IO_SNMP_4 5
517		typedef struct tpl_ext_5_s {
518		uint32_t input;
519		uint32_t output;
520		uint8_t data[4]; // points to further data
521		} tpl_ext_5_t;
522
523
524		/*
525		* AS record
526		* ---------
527		* AS records are optional and accepted as either 2 or 4 bytes numbers
528		* Extension 6:
529		* +---++--------------+--------------+--------------+--------------+
530		* \| 0 \| src as (16) \| dst as (17) \|
531		* +---++--------------+--------------+--------------+--------------+
532		*/
533		#define EX_AS_2 6
534		typedef struct tpl_ext_6_s {
535		uint16_t src_as;
536		uint16_t dst_as;
537		uint8_t data[4]; // points to further data
538		} tpl_ext_6_t;
539
540		/*
541		* Extension 7:
542		* +---++--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
543		* \| 0 \| src as (16) \| dst as (17) \|
544		* +---++--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
545		* Extension 6 and 7 are mutually exclusive in the extension map
546		*/
547		#define EX_AS_4 7
548		typedef struct tpl_ext_7_s {
549		uint32_t src_as;
550		uint32_t dst_as;
551		uint8_t data[4]; // points to further data
552		} tpl_ext_7_t;
553
554
555		/*
556		* Multiple fields record
557		* ----------------------
558		* These 4 different fields are grouped together in a 32bit value.
559		* Extension 8:
560		* +---++--------------+--------------+--------------+--------------+
561		* \| 3 \| dst tos(55) \| dir(61) \| srcmask(9,29)\|dstmask(13,30)\|
562		* +---++--------------+--------------+--------------+--------------+
563		*/
564		#define EX_MULIPLE 8
565		typedef struct tpl_ext_8_s {
566		union {
567		struct {
568		uint8_t dst_tos;
569		uint8_t dir;
570		uint8_t src_mask;
571		uint8_t dst_mask;
572		};
573		uint32_t any;
574		};
575		uint8_t data[4]; // points to further data
576		} tpl_ext_8_t;
577
578		/*
579		* IP next hop
580		* -------------
581		* IPv4:
582		* Extension 9:
583		* IP version: IPv6
584		* \|
585		* Flags: xxxx 0xxx
586		* +----+--------------+--------------+--------------+--------------+
587		* \| 0 \| next hop ip (15) \|
588		* +----+--------------+--------------+--------------+--------------+
589		*/
590		#define EX_NEXT_HOP_v4 9
591		typedef struct tpl_ext_9_s {
592		uint32_t nexthop;
593		uint8_t data[4]; // points to further data
594		} tpl_ext_9_t;
595
596		/*
597		* IPv6:
598		* Extension 10:
599		* IP version: IPv6
600		* \|
601		* Flags: xxxx 1xxx
602		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
603		* \| 0 \| next hop ip (62) \|
604		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
605		* \| 1 \| next hop ip (62) \|
606		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
607		* Extension 9 and 10 are mutually exclusive in the extension map
608		*/
609		#define EX_NEXT_HOP_v6 10
610		typedef struct tpl_ext_10_s {
611		uint64_t nexthop[2];
612		uint8_t data[4]; // points to further data
613		} tpl_ext_10_t;
614
615
616		/*
617		* BGP next hop IP
618		* ------------------
619		* IPv4:
620		* Extension 11:
621		* IP version: IPv6
622		* \|
623		* Flags: xxx0 xxxx
624		* +----+--------------+--------------+--------------+--------------+
625		* \| 0 \| bgp next ip (18) \|
626		* +----+--------------+--------------+--------------+--------------+
627		*/
628		#define EX_NEXT_HOP_BGP_v4 11
629		typedef struct tpl_ext_11_s {
630		uint32_t bgp_nexthop;
631		uint8_t data[4]; // points to further data
632		} tpl_ext_11_t;
633
634		/*
635		* IPv6:
636		* Extension 12:
637		* IP version: IPv6
638		* \|
639		* Flags: xxx1 xxxx
640		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
641		* \| 0 \| bgp next ip (63) \|
642		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
643		* \| 1 \| bgp next ip (63) \|
644		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
645		*/
646		#define EX_NEXT_HOP_BGP_v6 12
647		typedef struct tpl_ext_12_s {
648		uint64_t bgp_nexthop[2];
649		uint8_t data[4]; // points to further data
650		} tpl_ext_12_t;
651
652
653		/*
654		* VLAN record
655		* -----------
656		* Extension 13:
657		* +----+--------------+--------------+--------------+--------------+
658		* \| 0 \| src vlan(58) \| dst vlan (59) \|
659		* +----+--------------+--------------+--------------+--------------+
660		*/
661		#define EX_VLAN 13
662		typedef struct tpl_ext_13_s {
663		uint16_t src_vlan;
664		uint16_t dst_vlan;
665		uint8_t data[4]; // points to further data
666		} tpl_ext_13_t;
667
668
669		/*
670		* Out packet counter size
671		* ------------------------
672		* 4 byte
673		* Extension 14:
674		* +----+--------------+--------------+--------------+--------------+
675		* \| 0 \| out pkts (24) \|
676		* +----+--------------+--------------+--------------+--------------+
677		*/
678		#define EX_OUT_PKG_4 14
679		typedef struct tpl_ext_14_s {
680		uint32_t out_pkts;
681		uint8_t data[4]; // points to further data
682		} tpl_ext_14_t;
683
684		/*
685		* 4 byte
686		* Extension 15:
687		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
688		* \| 0 \| out pkts (24) \|
689		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
690		* Extension 14 and 15 are mutually exclusive in the extension map
691		*/
692		#define EX_OUT_PKG_8 15
693		typedef struct tpl_ext_15_s {
694		union {
695		uint64_t out_pkts;
696		uint32_t v[2]; // for strict alignment use 2x32bits
697		};
698		uint8_t data[4]; // points to further data
699		} tpl_ext_15_t;
700
701
702		/*
703		* Out byte counter size
704		* ---------------------
705		* 4 byte
706		* Extension 16:
707		* +----+--------------+--------------+--------------+--------------+
708		* \| 0 \| out bytes (23) \|
709		* +----+--------------+--------------+--------------+--------------+
710		*/
711		#define EX_OUT_BYTES_4 16
712		typedef struct tpl_ext_16_s {
713		uint32_t out_bytes;
714		uint8_t data[4]; // points to further data
715		} tpl_ext_16_t;
716
717
718		/* 8 byte
719		* Extension 17:
720		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
721		* \| 0 \| out bytes (23) \|
722		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
723		* Extension 16 and 17 are mutually exclusive in the extension map
724		*/
725		#define EX_OUT_BYTES_8 17
726		typedef struct tpl_ext_17_s {
727		union {
728		uint64_t out_bytes;
729		uint32_t v[2]; // potential 32bit alignment
730		};
731		uint8_t data[4]; // points to further data
732		} tpl_ext_17_t;
733
734		/*
735		* Aggr flows
736		* ----------
737		* 4 byte
738		* Extension 18:
739		* +----+--------------+--------------+--------------+--------------+
740		* \| 0 \| aggr flows (3) \|
741		* +----+--------------+--------------+--------------+--------------+
742		*/
743		#define EX_AGGR_FLOWS_4 18
744		typedef struct tpl_ext_18_s {
745		uint32_t aggr_flows;
746		uint8_t data[4]; // points to further data
747		} tpl_ext_18_t;
748
749
750		/* 8 byte
751		* Extension 19:
752		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
753		* \| 0 \| aggr flows (3) \|
754		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
755		* Extension 18 and 19 are mutually exclusive in the extension map
756		*/
757		#define EX_AGGR_FLOWS_8 19
758		typedef struct tpl_ext_19_s {
759		union {
760		uint64_t aggr_flows;
761		uint32_t v[2]; // 32bit alignment
762		};
763		uint8_t data[4]; // points to further data
764		} tpl_ext_19_t;
765
766		/* 16 byte
767		* Extension 20:
768		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
769		* \| 0 \| 0 \| in src mac (56) \|
770		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
771		* \| 1 \| 0 \| out dst mac (57) \|
772		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
773		*/
774		#define EX_MAC_1 20
775		typedef struct tpl_ext_20_s {
776		union {
777		uint64_t in_src_mac;
778		uint32_t v1[2];
779		};
780		union {
781		uint64_t out_dst_mac;
782		uint32_t v2[2];
783		};
784		uint8_t data[4]; // points to further data
785		} tpl_ext_20_t;
786
787		/* 16 byte
788		* Extension 21:
789		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
790		* \| 0 \| 0 \| in dst mac (80) \|
791		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
792		* \| 1 \| 0 \| out src mac (81) \|
793		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
794		*/
795		#define EX_MAC_2 21
796		typedef struct tpl_ext_21_s {
797		union {
798		uint64_t in_dst_mac;
799		uint32_t v1[2];
800		};
801		union {
802		uint64_t out_src_mac;
803		uint32_t v2[2];
804		};
805		uint8_t data[4]; // points to further data
806		} tpl_ext_21_t;
807
808		/* 40 byte
809		* Extension 22:
810		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
811		* \| 0 \| 0 \| MPLS_LABEL_2 (71) \| 0 \| MPLS_LABEL_1 (70) \|
812		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
813		* \| 1 \| 0 \| MPLS_LABEL_4 (73) \| 0 \| MPLS_LABEL_3 (72) \|
814		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
815		* \| 2 \| 0 \| MPLS_LABEL_6 (75) \| 0 \| MPLS_LABEL_5 (74) \|
816		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
817		* \| 3 \| 0 \| MPLS_LABEL_8 (77) \| 0 \| MPLS_LABEL_7 (76) \|
818		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
819		* \| 4 \| 0 \| MPLS_LABEL_10 (79) \| 0 \| MPLS_LABEL_9 (78) \|
820		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
821		*/
822		#define EX_MPLS 22
823		typedef struct tpl_ext_22_s {
824		uint32_t mpls_label[10];
825		uint8_t data[4]; // points to further data
826		} tpl_ext_22_t;
827
828		/*
829		* Sending router IP
830		* -----------------
831		* IPv4:
832		* Extension 23:
833		* IP version: IPv6
834		* \|
835		* Flags: xx0x xxxx
836		* +----+--------------+--------------+--------------+--------------+
837		* \| 0 \| router ipv4 () \|
838		* +----+--------------+--------------+--------------+--------------+
839		*/
840		#define EX_ROUTER_IP_v4 23
841		typedef struct tpl_ext_23_s {
842		uint32_t router_ip;
843		uint8_t data[4]; // points to further data
844		} tpl_ext_23_t;
845
846		/*
847		* IPv6:
848		* Extension 24:
849		* IP version: IPv6
850		* \|
851		* Flags: xx1x xxxx
852		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
853		* \| 0 \| router ip v6 () \|
854		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
855		* \| 1 \| router ip v6 () \|
856		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
857		* Extension 23 and 24 are mutually exclusive in the extension map
858		*/
859		#define EX_ROUTER_IP_v6 24
860		typedef struct tpl_ext_24_s {
861		uint64_t router_ip[2];
862		uint8_t data[4]; // points to further data
863		} tpl_ext_24_t;
864
865		/*
866		* router source ID
867		* ----------------
868		* For v5 netflow, it's engine type/engine ID
869		* for v9 it's the source_id
870		* Extension 25:
871		* +----+--------------+--------------+--------------+--------------+
872		* \| 0 \| fill \|engine tpe(38)\|engine ID(39) \|
873		* +----+--------------+--------------+--------------+--------------+
874		*/
875		#define EX_ROUTER_ID 25
876		typedef struct tpl_ext_25_s {
877		uint16_t fill;
878		uint8_t engine_type;
879		uint8_t engine_id;
880		uint8_t data[4]; // points to further data
881		} tpl_ext_25_t;
882
883		/*
884		* BGP prev/next adjacent AS
885		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
886		* \| 0 \| bgpNextAdjacentAsNumber(128) \| bgpPrevAdjacentAsNumber(129) \|
887		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
888		*/
889		#define EX_BGPADJ 26
890		typedef struct tpl_ext_26_s {
891		uint32_t bgpNextAdjacentAS;
892		uint32_t bgpPrevAdjacentAS;
893		uint8_t data[4]; // points to further data
894		} tpl_ext_26_t;
895
896		/*
897		* time flow received in ms
898		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
899		* \| 0 \| T received() \|
900		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
901		*/
902		#define EX_RECEIVED 27
903		typedef struct tpl_ext_27_s {
904		union {
905		uint64_t received;
906		uint32_t v[2];
907		};
908		uint8_t data[4]; // points to further data
909		} tpl_ext_27_t;
910
911
912
913		#define EX_RESERVED_1 28
914		#define EX_RESERVED_2 29
915		#define EX_RESERVED_3 30
916		#define EX_RESERVED_4 31
917		#define EX_RESERVED_5 32
918		#define EX_RESERVED_6 33
919		#define EX_RESERVED_7 34
920		#define EX_RESERVED_8 35
921		#define EX_RESERVED_9 36
922
923		/*
924		* NSEL Common block
925		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
926		* \| 0 \| NF_F_FLOW_CREATE_TIME_MSEC(152) \|
927		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
928		* \| 1 \| NF_F_CONN_ID(148) \|i type(176/8) \|i code(177/9) \|EVT(40005/233)\| fill \|
929		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
930		* \| 2 \| NF_F_FW_EXT_EVENT(33002) \| fill2 \|
931		* +----+--------------+--------------+--------------+--------------+
932		* * EVT: NF_F_FW_EVENT
933		* * XEVT: NF_F_FW_EXT_EVENT
934		*/
935		#define EX_NSEL_COMMON 37
936		typedef struct tpl_ext_37_s {
937		union {
938		uint64_t event_time;
939		uint32_t v[2];
940		};
941		uint32_t conn_id;
942		union {
943		struct {
944		#ifdef WORDS_BIGENDIAN
945		uint8_t icmp_type;
946		uint8_t icmp_code;
947		#else
948		uint8_t icmp_code;
949		uint8_t icmp_type;
950		#endif
951		};
952		uint16_t nsel_icmp;
953		};
954		uint8_t fw_event;
955		uint8_t fill;
956		uint16_t fw_xevent;
957		uint16_t fill2;
958		uint8_t data[4]; // points to further data
959		} tpl_ext_37_t;
960
961		/*
962		* NSEL/NEL xlate ports
963		* +----+--------------+--------------+--------------+--------------+
964		* \| 0 \| NF_F_XLATE_SRC_PORT(227) \| NF_F_XLATE_DST_PORT(228) \|
965		* +----+--------------+--------------+--------------+--------------+
966		* ASA 8.4 compatibility mapping 40003 -> 227
967		* ASA 8.4 compatibility mapping 40004 -> 228
968		*/
969		#define EX_NSEL_XLATE_PORTS 38
970		typedef struct tpl_ext_38_s {
971		uint16_t xlate_src_port;
972		uint16_t xlate_dst_port;
973		uint8_t data[4]; // points to further data
974		} tpl_ext_38_t;
975
976		/*
977		* NSEL xlate v4 IP address
978		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
979		* \| 0 \| NF_F_XLATE_SRC_ADDR_IPV4(225) \| NF_F_XLATE_DST_ADDR_IPV4(226) \|
980		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
981		* ASA 8.4 compatibility mapping 40001 -> 225
982		* ASA 8.4 compatibility mapping 40002 -> 226
983		*/
984		#define EX_NSEL_XLATE_IP_v4 39
985		typedef struct tpl_ext_39_s {
986		uint32_t xlate_src_ip;
987		uint32_t xlate_dst_ip;
988		uint8_t data[4]; // points to further data
989		} tpl_ext_39_t;
990
991		/*
992		* NSEL xlate v6 IP address - not yet implemented by CISCO
993		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
994		* \| 0 \| xlate src ip (281) \|
995		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
996		* \| 1 \| xlate src ip (281) \|
997		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
998		* \| 2 \| xlate dst ip (282) \|
999		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1000		* \| 3 \| xlate dst ip (282) \|
1001		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1002		*/
1003		#define EX_NSEL_XLATE_IP_v6 40
1004		typedef struct tpl_ext_40_s {
1005		uint64_t xlate_src_ip[2];
1006		uint64_t xlate_dst_ip[2];
1007		uint8_t data[4]; // points to further data
1008		} tpl_ext_40_t;
1009
1010
1011		/*
1012		* NSEL ACL ingress/egress acl ID
1013		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1014		* \| 0 \| NF_F_INGRESS_ACL_ID(33000) \|
1015		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1016		* \| 1 \| NF_F_INGRESS_ACL_ID(33000) \| NF_F_EGRESS_ACL_ID(33001) \|
1017		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1018		* \| 2 \| NF_F_EGRESS_ACL_ID(33001) \|
1019		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1020		*/
1021		#define EX_NSEL_ACL 41
1022		typedef struct tpl_ext_41_s {
1023		uint32_t ingress_acl_id[3];
1024		uint32_t egress_acl_id[3];
1025		uint8_t data[4]; // points to further data
1026		} tpl_ext_41_t;
1027
1028		/*
1029		* NSEL ACL username
1030		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1031		* \| 0 \| NF_F_USERNAME(40000) \|
1032		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1033		* \| 1 \| \|
1034		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1035		* \| 2 \| \|
1036		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1037		*/
1038		#define EX_NSEL_USER 42
1039		typedef struct tpl_ext_42_s {
1040		char username[24];
1041		uint8_t data[4]; // points to further data
1042		} tpl_ext_42_t;
1043
1044		/*
1045		* NSEL ACL username max
1046		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1047		* \| 0 \| NF_F_USERNAME(40000) \|
1048		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1049		* \| .. \| \|
1050		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1051		* \| 8 \| \|
1052		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1053		*/
1054		#define EX_NSEL_USER_MAX 43
1055		typedef struct tpl_ext_43_s {
1056		char username[72];
1057		uint8_t data[4]; // points to further data
1058		} tpl_ext_43_t;
1059
1060
1061		#define EX_NSEL_RESERVED 44
1062
1063		/*
1064		* latency extensions, used by nprobe and nfpcapd
1065		*/
1066
1067		/*
1068		* latency extension
1069		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1070		* \| 0 \| client_nw_delay_usec (57554/57554) \|
1071		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1072		* \| 1 \| server_nw_delay_usec (57556/57557) \|
1073		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1074		* \| 2 \| appl_latency_usec (57558/57559) \|
1075		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1076		*/
1077		#define EX_LATENCY 45
1078		typedef struct tpl_ext_latency_s {
1079		uint64_t client_nw_delay_usec;
1080		uint64_t server_nw_delay_usec;
1081		uint64_t appl_latency_usec;
1082		uint8_t data[4]; // points to further data
1083		} tpl_ext_latency_t;
1084
1085		/*
1086		* NEL xlate ports
1087		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1088		* \| 0 \|NAT_EVENT(230)\| flags \| fill \| NF_N_EGRESS_VRFID(235) \|
1089		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1090		* \| 1 \| NF_N_INGRESS_VRFID(234) \|
1091		* +----+--------------+--------------+--------------+--------------+
1092		*/
1093		#define EX_NEL_COMMON 46
1094		typedef struct tpl_ext_46_s {
1095		uint8_t nat_event;
1096		uint8_t flags;
1097		uint16_t fill;
1098		uint32_t egress_vrfid;
1099		uint32_t ingress_vrfid;
1100		uint8_t data[4]; // points to further data
1101		} tpl_ext_46_t;
1102
1103		#define EX_NEL_GLOBAL_IP_v4 47
1104		/*
1105		* no longer used. Mapped to NSEL extension EX_NSEL_XLATE_IP_v4
1106		*/
1107		typedef struct tpl_ext_47_s {
1108		uint32_t nat_inside;
1109		uint32_t nat_outside;
1110		uint8_t data[4]; // points to further data
1111		} tpl_ext_47_t;
1112
1113		/*
1114		* NEL Port Block Allocation
1115		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1116		* \| 0 \| NF_F_XLATE_PORT_BLOCK_START \| NF_F_XLATE_PORT_BLOCK_END \| NF_F_XLATE_PORT_BLOCK_STEP \| NF_F_XLATE_PORT_BLOCK_SIZE \|
1117		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1118		*/
1119		#define EX_PORT_BLOCK_ALLOC 48
1120		typedef struct tpl_ext_48_s {
1121		uint16_t block_start;
1122		uint16_t block_end;
1123		uint16_t block_step;
1124		uint16_t block_size;
1125		uint8_t data[4]; // points to further data
1126		} tpl_ext_48_t;
1127
1128		#define EX_NEL_RESERVED_1 49
1129
1130
1131		/*
1132		*
1133		*
1134		* V1 Extension map:
1135		* =================
1136		* The extension map replaces the individual flags in v1 layout. With many possible extensions and combination of extensions
1137		* an extension map is more efficient and flexible while reading and decoding the record.
1138		* In current version of nfdump, up to 65535 individual extension maps are supported, which is considered to be enough.
1139		*
1140		* For each available extension record, the ids are recorded in the extension map in the order they appear.
1141		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1142		* \| - \| 0 \| 1 \| 2 \| 3 \| 4 \| 5 \| 6 \| 7 \|
1143		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1144		* \| 0 \| record type == 2 \| size \| map id \| extension size \|
1145		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1146		* \| 0 \| extension id 1 \| extension id 2 \| extension id 3 \| extension id 4 \|
1147		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1148		* ...
1149		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1150		* \| 0 \| extension id n \| extension id n+1 \| extension id n+2 \| extension id n+3 \|
1151		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1152		* ...
1153		* +----+--------------+--------------+--------------+--------------+
1154		* \| 0 \| 0 \| opt. 32bit alignment: 0 \|
1155		* +----+--------------+--------------+--------------+--------------+
1156		*/
1157
1158		/* extension IDs above are 16 bit integers. So themax number of available extensions is */
1159		#define MAX_EXTENSIONS 65536
1160
1161		typedef struct extension_map_s {
1162		// record head
1163		uint16_t type; // is ExtensionMapType
1164		uint16_t size; // size of full map incl. header
1165
1166		// map data
1167		#define INIT_ID 0xFFFF
1168		uint16_t map_id; // identifies this map
1169		uint16_t extension_size; // size of all extensions
1170		uint16_t ex_id[1]; // extension id array
1171		} extension_map_t;
1172
1173
1174		/*
1175		*
1176		*
1177		* V2 Extension map:
1178		* =================
1179		* The V2 extension map replaces the V1 extension map. The basic extension architecture remains the same. V2 extensions
1180		* adds extra information about the extension such as size and offset within the packed record. This allows more
1181		* flexibility by using flexible length extensions up to the extension defined maximum. With the introduction of V2
1182		* extension maps, the old master_record will become obsolete in near future.
1183		* Implementation wise: if extension size is set to 0 - the first 2 bytes of the extension data contains the size of
1184		* the extension. Extensions with flexible size must only appended at the end of the record block.
1185		*
1186		* For each available extension record, the ids are recorded in the extension map in the order they appear.
1187		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1188		* \| - \| 0 \| 1 \| 2 \| 3 \| 4 \| 5 \| 6 \| 7 \|
1189		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1190		* \| 0 \| record type == 10 \| size \| map id \| max extension size \|
1191		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1192		* \| 1 \| extension id 1 \| offset extension id 1 \| extension id 2 \| offset extension id 2 \|
1193		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1194		* \| 1 \| extension id 3 \| offset extension id 3 \| extension id 4 \| offset extension id 4 \|
1195		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1196		* ...
1197		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1198		* \| n \| extension id n \| size extension id n \| 0 \| 0 \|
1199		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1200		*/
1201
1202		/* extension IDs above are 16 bit integers. So themax number of available extensions is */
1203		#define MAX_EXTENSIONS 65536
1204		/* XXX extension 1.7
1205		typedef struct extension_map17_s {
1206		// record head
1207		uint16_t type; // is ExtensionMapType2
1208		uint16_t size; // size of full map incl. header
1209
1210		// map data
1211		uint16_t map_id; // identifies this map
1212		uint16_t extension_size; // max size of all extensions
1213		struct ex_def_s {
1214		uint16_t id; // extension id
1215		uint16_t offset; // max extension size
1216		} ex[1]; // array of all extensions
1217		} extension_map17_t;
	236
	237	/*
	238	* for the detailed description of the record definition see nfx.h
1218	239	*/
1219	240
1220		// see nfx.c - extension_descriptor
1221		#ifdef NSEL
1222		// Defaults for NSEL
1223		#define DefaultExtensions "1,8,26,27,28,29,30,31"
1224		#else
1225		// Collector netflow defaults
1226		#define DefaultExtensions "1,2"
1227		#endif
1228
1229		/*
1230		* nfcapd writes an info stat record for each new exporter
1231		*
1232		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1233		* \| - \| 0 \| 1 \| 2 \| 3 \| 4 \| 5 \| 6 \| 7 \|
1234		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1235		* \| 0 \| record type == 7 \| size \| version \|
1236		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1237		* \| 1 \| \|
1238		* +----+--------------+--------------+--------------+---------- ip ------------+--------------+--------------+--------------+
1239		* \| 2 \| \|
1240		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1241		* \| 3 \| sa_family \| sysid \| id \|
1242		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1243		*/
1244		typedef struct exporter_info_record_s {
1245		record_header_t header;
1246
1247		// exporter version
1248		uint32_t version;
1249		#define SFLOW_VERSION 9999
1250
1251		// IP address
1252		ip_addr_t ip;
1253		uint16_t sa_family;
1254
1255		// internal assigned ID
1256		uint16_t sysid;
1257
1258		// exporter ID/Domain ID/Observation Domain ID assigned by the device
1259		uint32_t id;
1260
1261		} exporter_info_record_t;
1262
1263		/*
1264		* nfcapd writes a stat record at the end of the file which contains the exporter statistics.
1265		*
1266		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1267		* \| - \| 0 \| 1 \| 2 \| 3 \| 4 \| 5 \| 6 \| 7 \|
1268		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1269		* \| 0 \| record type == 8 \| size \| stat_count \|
1270		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1271		* \| 1 \| sysid[0] \| sequence_failure[0] \|
1272		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1273		* \| 2 \| packets[0] \|
1274		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1275		* \| 3 \| flows[0] \|
1276		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1277		* ... more stat records [x], one for each exporter
1278		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1279		*/
1280		typedef struct exporter_stats_record_s {
1281		record_header_t header;
1282
1283		uint32_t stat_count; // number of stat records
1284
1285		struct exporter_stat_s {
1286		uint32_t sysid; // identifies the exporter
1287		uint32_t sequence_failure; // number of sequence failues
1288		uint64_t packets; // number of packets sent by this exporter
1289		uint64_t flows; // number of flow records sent by this exporter
1290		} stat[1];
1291
1292		} exporter_stats_record_t;
1293
1294
1295		/*
1296		* nfcapd writes a sampler record for each new sampler announced
1297		*
1298		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1299		* \| - \| 0 \| 1 \| 2 \| 3 \| 4 \| 5 \| 6 \| 7 \|
1300		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1301		* \| 0 \| record type == 9 \| size \| id \|
1302		* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
1303		* \| 1 \| interval \| mode \| exporter_sysid \|
1304		* +----+--------------+--------------+--------------+-----------------------------+--------------+--------------+--------------+
1305		*/
1306		typedef struct sampler_info_record_s {
1307		record_header_t header;
1308
1309		// sampler data
1310		int32_t id; // id assigned by the exporting device
1311		uint32_t interval; // sampling interval
1312		uint16_t mode; // sampling mode
1313		uint16_t exporter_sysid; // internal reference to exporter
1314
1315		} sampler_info_record_t;
1316
1317
1318
1319		/*
1320		* TC test code - records will disappear ..
1321		*/
1322		typedef struct exporter_record_s {
1323		record_header_t header;
1324
1325		// exporter data
1326		uint32_t version; // make sure it's a version 9 exporter
1327
1328		// IP address
1329		uint32_t sa_family;
1330		ip_addr_t ip;
1331
1332		// internal assigned ID
1333		uint32_t sysid;
1334
1335		// exporter info
1336		uint32_t exporter_id;
1337		uint32_t sequence_failure;
1338
1339		} exporter_record_t;
1340
1341		typedef struct sampler_record_s {
1342		record_header_t header;
1343
1344		// reference to exporter
1345		uint32_t exporter_sysid;
1346
1347		// sampler data
1348		int32_t id;
1349		uint32_t interval;
1350		uint8_t mode;
1351		} sampler_record_t;
1352
1353
1354		/* the master record contains all possible records unpacked */
1355		typedef struct master_record_s {
1356		// common information from all netflow versions
1357		// // interpreted as uint64_t[]
1358		// #ifdef WORDS_BIGENDIAN
1359
1360		uint16_t type; // index 0 0xffff 0000 0000 0000
1361		uint16_t size; // index 0 0x0000'ffff'0000 0000
1362		uint16_t flags; // index 0 0x0000'0000'ffff'0000
1363		uint16_t ext_map; // index 0 0x0000'0000'0000'ffff
1364		# define OffsetRecordFlags 0
1365		#ifdef WORDS_BIGENDIAN
1366		# define MaskRecordFlags 0x00000000ff000000LL
1367		# define ShiftRecordFlags 24
1368		#else
1369		# define MaskRecordFlags 0x000000ff00000000LL
1370		# define ShiftRecordFlags 32
1371		#endif
1372
1373		//
1374		uint16_t msec_first; // index 1 0xffff'0000'0000'0000
1375		uint16_t msec_last; // index 1 0x0000'ffff'0000'0000
1376
1377		// 12 bytes offset in master record to first
1378		#define BYTE_OFFSET_first 12
1379
1380		uint32_t first; // index 1 0x0000'0000'ffff'ffff
1381
1382		//
1383		uint32_t last; // index 2 0xffff'ffff'0000'0000
1384		uint8_t fwd_status; // index 2 0x0000'0000'ff00'0000
1385		uint8_t tcp_flags; // index 2 0x0000'0000'00ff'0000
1386		uint8_t prot; // index 2 0x0000'0000'0000'ff00
1387		uint8_t tos; // index 2 0x0000'0000'0000'00ff
1388		#ifdef WORDS_BIGENDIAN
1389		# define OffsetStatus 2
1390		# define MaskStatus 0x00000000ff000000LL
1391		# define ShiftStatus 24
1392
1393		# define OffsetFlags 2
1394		# define MaskFlags 0x0000000000ff0000LL
1395		#define ShiftFlags 16
1396
1397		# define OffsetProto 2
1398		# define MaskProto 0x000000000000ff00LL
1399		# define ShiftProto 8
1400
1401		# define OffsetTos 2
1402		# define MaskTos 0x00000000000000ffLL
1403		# define ShiftTos 0
1404
1405		#else
1406		# define OffsetStatus 2
1407		# define MaskStatus 0x000000ff00000000LL
1408		# define ShiftStatus 32
1409
1410		# define OffsetFlags 2
1411		# define MaskFlags 0x0000ff0000000000LL
1412		# define ShiftFlags 40
1413
1414		# define OffsetProto 2
1415		# define MaskProto 0x00ff000000000000LL
1416		# define ShiftProto 48
1417
1418		# define OffsetTos 2
1419		# define MaskTos 0xff00000000000000LL
1420		# define ShiftTos 56
1421		#endif
1422
1423		uint16_t srcport; // index 3 0xffff'0000'0000'0000
1424		uint16_t dstport; // index 3 0x0000'ffff'0000'0000
1425		uint16_t exporter_sysid; // index 3 0x0000'0000'ffff'0000
1426
1427		union {
1428		struct {
1429		#ifdef WORDS_BIGENDIAN
1430		uint8_t icmp_type; // index 3 0x0000'0000'0000'ff00
1431		uint8_t icmp_code; // index 3 0x0000'0000'0000'00ff
1432		#else
1433		// little endian confusion ...
1434		uint8_t icmp_code;
1435		uint8_t icmp_type;
1436		#endif
1437		};
1438		uint16_t icmp;
1439		};
1440
1441		#ifdef WORDS_BIGENDIAN
1442		# define OffsetPort 3
1443		# define OffsetExporterSysID 3
1444		# define MaskSrcPort 0xffff000000000000LL
1445		# define ShiftSrcPort 48
1446
1447		# define MaskDstPort 0x0000ffff00000000LL
1448		# define ShiftDstPort 32
1449
1450		# define MaskExporterSysID 0x00000000ffff0000LL
1451		# define ShiftExporterSysID 16
1452
1453		# define MaskICMPtype 0x000000000000ff00LL
1454		# define ShiftICMPtype 8
1455		# define MaskICMPcode 0x00000000000000ffLL
1456		# define ShiftICMPcode 0
1457
1458		#else
1459		# define OffsetPort 3
1460		# define OffsetExporterSysID 3
1461		# define MaskSrcPort 0x000000000000ffffLL
1462		# define ShiftSrcPort 0
1463
1464		# define MaskDstPort 0x00000000ffff0000LL
1465		# define ShiftDstPort 16
1466
1467		# define MaskExporterSysID 0x0000ffff00000000LL
1468		# define ShiftExporterSysID 32
1469
1470		# define MaskICMPtype 0xff00000000000000LL
1471		# define ShiftICMPtype 56
1472		# define MaskICMPcode 0x00ff000000000000LL
1473		# define ShiftICMPcode 48
1474		#endif
1475
1476		// extension 4 / 5
1477		uint32_t input; // index 4 0xffff'ffff'0000'0000
1478		uint32_t output; // index 4 0x0000'0000'ffff'ffff
1479		#ifdef WORDS_BIGENDIAN
1480		# define OffsetInOut 4
1481		# define MaskInput 0xffffffff00000000LL
1482		# define ShiftInput 32
1483		# define MaskOutput 0x00000000ffffffffLL
1484		# define ShiftOutput 0
1485
1486		#else
1487		# define OffsetInOut 4
1488		# define MaskInput 0x00000000ffffffffLL
1489		# define ShiftInput 0
1490		# define MaskOutput 0xffffffff00000000LL
1491		# define ShiftOutput 32
1492		#endif
1493
1494		// extension 6 / 7
1495		uint32_t srcas; // index 5 0xffff'ffff'0000'0000
1496		uint32_t dstas; // index 5 0x0000'0000'ffff'ffff
1497		#ifdef WORDS_BIGENDIAN
1498		# define OffsetAS 5
1499		# define MaskSrcAS 0xffffffff00000000LL
1500		# define ShiftSrcAS 32
1501		# define MaskDstAS 0x00000000ffffffffLL
1502		# define ShiftDstAS 0
1503
1504		#else
1505		# define OffsetAS 5
1506		# define MaskSrcAS 0x00000000ffffffffLL
1507		# define ShiftSrcAS 0
1508		# define MaskDstAS 0xffffffff00000000LL
1509		# define ShiftDstAS 32
1510		#endif
1511
1512
1513		// IP address block
1514		union {
1515		struct _ipv4_s {
1516		#ifdef WORDS_BIGENDIAN
1517		uint32_t fill1[3]; // <empty> index 6 0xffff'ffff'ffff'ffff
1518		// <empty> index 7 0xffff'ffff'0000'0000
1519		uint32_t srcaddr; // srcaddr index 7 0x0000'0000'ffff'ffff
1520		uint32_t fill2[3]; // <empty> index 8 0xffff'ffff'ffff'ffff
1521		// <empty> index 9 0xffff'ffff'0000'0000
1522		uint32_t dstaddr; // dstaddr index 9 0x0000'0000'ffff'ffff
1523		#else
1524		uint32_t fill1[2]; // <empty> index 6 0xffff'ffff'ffff'ffff
1525		uint32_t srcaddr; // srcaddr index 7 0xffff'ffff'0000'0000
1526		uint32_t fill2; // <empty> index 7 0x0000'0000'ffff'ffff
1527		uint32_t fill3[2]; // <empty> index 8 0xffff'ffff'ffff'ffff
1528		uint32_t dstaddr; // dstaddr index 9 0xffff'ffff'0000'0000
1529		uint32_t fill4; // <empty> index 9 0xffff'ffff'0000'0000
1530		#endif
1531		} _v4;
1532		struct _ipv6_s {
1533		uint64_t srcaddr[2]; // srcaddr[0-1] index 6 0xffff'ffff'ffff'ffff
1534		// srcaddr[2-3] index 7 0xffff'ffff'ffff'ffff
1535		uint64_t dstaddr[2]; // dstaddr[0-1] index 8 0xffff'ffff'ffff'ffff
1536		// dstaddr[2-3] index 9 0xffff'ffff'ffff'ffff
1537		} _v6;
1538		struct _ip64_s {
1539		uint64_t addr[4];
1540		} _ip_64;
1541		} ip_union;
1542
1543		#ifdef WORDS_BIGENDIAN
1544		# define OffsetSrcIPv4 7
1545		# define MaskSrcIPv4 0x00000000ffffffffLL
1546		# define ShiftSrcIPv4 0
1547
1548		# define OffsetDstIPv4 9
1549		# define MaskDstIPv4 0x00000000ffffffffLL
1550		# define ShiftDstIPv4 0
1551
1552		# define OffsetSrcIPv6a 6
1553		# define OffsetSrcIPv6b 7
1554		# define OffsetDstIPv6a 8
1555		# define OffsetDstIPv6b 9
1556		# define MaskIPv6 0xffffffffffffffffLL
1557		# define ShiftIPv6 0
1558
1559		#else
1560		# define OffsetSrcIPv4 6
1561		# define MaskSrcIPv4 0xffffffff00000000LL
1562		# define ShiftSrcIPv4 32
1563
1564		# define OffsetDstIPv4 8
1565		# define MaskDstIPv4 0xffffffff00000000LL
1566		# define ShiftDstIPv4 32
1567
1568		# define OffsetSrcIPv6a 6
1569		# define OffsetSrcIPv6b 7
1570		# define OffsetDstIPv6a 8
1571		# define OffsetDstIPv6b 9
1572		# define MaskIPv6 0xffffffffffffffffLL
1573		# define ShiftIPv6 0
1574		#endif
1575
1576
1577		// counter block - expanded to 8 bytes
1578		uint64_t dPkts; // index 10 0xffff'ffff'ffff'ffff
1579		# define OffsetPackets 10
1580		# define MaskPackets 0xffffffffffffffffLL
1581		# define ShiftPackets 0
1582
1583		uint64_t dOctets; // index 11 0xffff'ffff'ffff'ffff
1584		# define OffsetBytes 11
1585		# define MaskBytes 0xffffffffffffffffLL
1586		# define ShiftBytes 0
1587
1588		// extension 9 / 10
1589		ip_addr_t ip_nexthop; // ipv4 index 13 0x0000'0000'ffff'ffff
1590		// ipv6 index 12 0xffff'ffff'ffff'ffff
1591		// ipv6 index 13 0xffff'ffff'ffff'ffff
1592
1593		#ifdef WORDS_BIGENDIAN
1594		# define OffsetNexthopv4 13
1595		# define MaskNexthopv4 0x00000000ffffffffLL
1596		# define ShiftNexthopv4 0
1597
1598		# define OffsetNexthopv6a 12
1599		# define OffsetNexthopv6b 13
1600		// MaskIPv6 and ShiftIPv6 already defined
1601
1602		#else
1603		# define OffsetNexthopv4 13
1604		# define MaskNexthopv4 0xffffffff00000000LL
1605		# define ShiftNexthopv4 0
1606
1607		# define OffsetNexthopv6a 12
1608		# define OffsetNexthopv6b 13
1609		#endif
1610
1611		// extension 11 / 12
1612		ip_addr_t bgp_nexthop; // ipv4 index 15 0x0000'0000'ffff'ffff
1613		// ipv6 index 14 0xffff'ffff'ffff'ffff
1614		// ipv6 index 15 0xffff'ffff'ffff'ffff
1615
1616		#ifdef WORDS_BIGENDIAN
1617		# define OffsetBGPNexthopv4 15
1618		# define MaskBGPNexthopv4 0x00000000ffffffffLL
1619		# define ShiftBGPNexthopv4 0
1620
1621		# define OffsetBGPNexthopv6a 14
1622		# define OffsetBGPNexthopv6b 15
1623		// MaskIPv6 and ShiftIPv6 already defined
1624
1625		#else
1626		# define OffsetBGPNexthopv4 15
1627		# define MaskBGPNexthopv4 0xffffffff00000000LL
1628		# define ShiftBGPNexthopv4 0
1629
1630		# define OffsetBGPNexthopv6a 14
1631		# define OffsetBGPNexthopv6b 15
1632		#endif
1633
1634		// extension 8
1635		union {
1636		struct {
1637		uint8_t dst_tos; // index 16 0xff00'0000'0000'0000
1638		uint8_t dir; // index 16 0x00ff'0000'0000'0000
1639		uint8_t src_mask; // index 16 0x0000'ff00'0000'0000
1640		uint8_t dst_mask; // index 16 0x0000'00ff'0000'0000
1641		};
1642		uint32_t any;
1643		};
1644
1645		// extension 13
1646		uint16_t src_vlan; // index 16 0x0000'0000'ffff'0000
1647		uint16_t dst_vlan; // index 16 0x0000'0000'0000'ffff
1648
1649		#ifdef WORDS_BIGENDIAN
1650		# define OffsetDstTos 16
1651		# define MaskDstTos 0xff00000000000000LL
1652		# define ShiftDstTos 56
1653
1654		# define OffsetDir 16
1655		# define MaskDir 0x00ff000000000000LL
1656		# define ShiftDir 48
1657
1658		# define OffsetMask 16
1659		# define MaskSrcMask 0x0000ff0000000000LL
1660		# define ShiftSrcMask 40
1661
1662		# define MaskDstMask 0x000000ff00000000LL
1663		# define ShiftDstMask 32
1664
1665		# define OffsetVlan 16
1666		# define MaskSrcVlan 0x00000000ffff0000LL
1667		# define ShiftSrcVlan 16
1668
1669		# define MaskDstVlan 0x000000000000ffffLL
1670		# define ShiftDstVlan 0
1671
1672		#else
1673		# define OffsetDstTos 16
1674		# define MaskDstTos 0x00000000000000ffLL
1675		# define ShiftDstTos 0
1676
1677		# define OffsetDir 16
1678		# define MaskDir 0x000000000000ff00LL
1679		# define ShiftDir 8
1680
1681		# define OffsetMask 16
1682		# define MaskSrcMask 0x0000000000ff0000LL
1683		# define ShiftSrcMask 16
1684
1685		# define MaskDstMask 0x00000000ff000000LL
1686		# define ShiftDstMask 24
1687
1688		# define OffsetVlan 16
1689		# define MaskSrcVlan 0x0000ffff00000000LL
1690		# define ShiftSrcVlan 32
1691
1692		# define MaskDstVlan 0xffff000000000000LL
1693		# define ShiftDstVlan 48
1694
1695		#endif
1696
1697		// extension 14 / 15
1698		uint64_t out_pkts; // index 17 0xffff'ffff'ffff'ffff
1699		# define OffsetOutPackets 17
1700		// MaskPackets and ShiftPackets already defined
1701
1702		// extension 16 / 17
1703		uint64_t out_bytes; // index 18 0xffff'ffff'ffff'ffff
1704		# define OffsetOutBytes 18
1705
1706		// extension 18 / 19
1707		uint64_t aggr_flows; // index 19 0xffff'ffff'ffff'ffff
1708		# define OffsetAggrFlows 19
1709		# define MaskFlows 0xffffffffffffffffLL
1710
1711		// extension 20
1712		uint64_t in_src_mac; // index 20 0xffff'ffff'ffff'ffff
1713		# define OffsetInSrcMAC 20
1714		# define MaskMac 0xffffffffffffffffLL
1715
1716		// extension 20
1717		uint64_t out_dst_mac; // index 21 0xffff'ffff'ffff'ffff
1718		# define OffsetOutDstMAC 21
1719
1720		// extension 21
1721		uint64_t in_dst_mac; // index 22 0xffff'ffff'ffff'ffff
1722		# define OffsetInDstMAC 22
1723
1724		// extension 21
1725		uint64_t out_src_mac; // index 23 0xffff'ffff'ffff'ffff
1726		# define OffsetOutSrcMAC 23
1727
1728		// extension 22
1729		uint32_t mpls_label[10];
1730		# define OffsetMPLS12 24
1731		# define OffsetMPLS34 25
1732		# define OffsetMPLS56 26
1733		# define OffsetMPLS78 27
1734		# define OffsetMPLS910 28
1735
1736		#ifdef WORDS_BIGENDIAN
1737		# define MaskMPLSlabelOdd 0x00fffff000000000LL
1738		# define ShiftMPLSlabelOdd 36
1739		# define MaskMPLSexpOdd 0x0000000e00000000LL
1740		# define ShiftMPLSexpOdd 33
1741
1742		# define MaskMPLSlabelEven 0x0000000000fffff0LL
1743		# define ShiftMPLSlabelEven 4
1744		# define MaskMPLSexpEven 0x000000000000000eLL
1745		# define ShiftMPLSexpEven 1
1746		#else
1747		# define MaskMPLSlabelOdd 0x000000000000fff0LL
1748		# define ShiftMPLSlabelOdd 4
1749		# define MaskMPLSexpOdd 0x000000000000000eLL
1750		# define ShiftMPLSexpOdd 1
1751
1752		# define MaskMPLSlabelEven 0x00fffff000000000LL
1753		# define ShiftMPLSlabelEven 36
1754		# define MaskMPLSexpEven 0x0000000e00000000LL
1755		# define ShiftMPLSexpEven 33
1756
1757		#endif
1758
1759		// extension 23 / 24
1760		ip_addr_t ip_router; // ipv4 index 30 0x0000'0000'ffff'ffff
1761		// ipv6 index 29 0xffff'ffff'ffff'ffff
1762		// ipv6 index 30 0xffff'ffff'ffff'ffff
1763
1764		#ifdef WORDS_BIGENDIAN
1765		# define OffsetRouterv4 30
1766		# define MaskRouterv4 0x00000000ffffffffLL
1767		# define ShiftRouterv4 0
1768
1769		# define OffsetRouterv6a 29
1770		# define OffsetRouterv6b 30
1771		// MaskIPv6 and ShiftIPv6 already defined
1772
1773		#else
1774		# define OffsetRouterv4 30
1775		# define MaskRouterv4 0xffffffff00000000LL
1776		# define ShiftRouterv4 0
1777
1778		# define OffsetRouterv6a 29
1779		# define OffsetRouterv6b 30
1780		#endif
1781
1782		// extension 25
1783		uint16_t fill; // fill index 31 0xffff'0000'0000'0000
1784		uint8_t engine_type; // type index 31 0x0000'ff00'0000'0000
1785		uint8_t engine_id; // ID index 31 0x0000'00ff'0000'0000
1786		uint32_t fill2;
1787
1788		# define OffsetRouterID 31
1789		#ifdef WORDS_BIGENDIAN
1790		# define MaskEngineType 0x0000FF0000000000LL
1791		# define ShiftEngineType 40
1792		# define MaskEngineID 0x000000FF00000000LL
1793		# define ShiftEngineID 32
1794
1795		#else
1796		# define MaskEngineType 0x0000000000FF0000LL
1797		# define ShiftEngineType 16
1798		# define MaskEngineID 0x00000000FF000000LL
1799		# define ShiftEngineID 24
1800		#endif
1801
1802		// IPFIX extensions in v9
1803		// BGP next/prev AS
1804		uint32_t bgpNextAdjacentAS; // index 32 0xffff'ffff'0000'0000
1805		uint32_t bgpPrevAdjacentAS; // index 32 0x0000'0000'ffff'ffff
1806
1807		// extension 18
1808		# define OffsetBGPadj 32
1809		#ifdef WORDS_BIGENDIAN
1810		# define MaskBGPadjNext 0xFFFFFFFF00000000LL
1811		# define ShiftBGPadjNext 32
1812		# define MaskBGPadjPrev 0x00000000FFFFFFFFLL
1813		# define ShiftBGPadjPrev 0
1814
1815		#else
1816		# define MaskBGPadjNext 0x00000000FFFFFFFFLL
1817		# define ShiftBGPadjNext 0
1818		# define MaskBGPadjPrev 0xFFFFFFFF00000000LL
1819		# define ShiftBGPadjPrev 32
1820		#endif
1821
1822		// NSEL extensions
1823		#ifdef NSEL
1824		#define NSEL_BASE_OFFSET (offsetof(master_record_t, conn_id) >> 3)
1825
1826		// common block
1827		# define OffsetConnID NSEL_BASE_OFFSET
1828		# define OffsetNATevent NSEL_BASE_OFFSET
1829		uint32_t conn_id; // index OffsetConnID 0xffff'ffff'0000'0000
1830		uint8_t event; // index OffsetConnID 0x0000'0000'ff00'0000
1831		#define FW_EVENT 1
1832		#define NAT_EVENT 2
1833		uint8_t event_flag; // index OffsetConnID 0x0000'0000'00ff'0000
1834		uint16_t fw_xevent; // index OffsetConnID 0x0000'0000'0000'ffff
1835		uint64_t event_time; // index OffsetConnID +1 0x1111'1111'1111'1111
1836		#ifdef WORDS_BIGENDIAN
1837		# define MaskConnID 0xFFFFFFFF00000000LL
1838		# define ShiftConnID 32
1839		# define MaskFWevent 0x00000000FF000000LL
1840		# define ShiftFWevent 24
1841		# define MasNATevent 0x00000000FF000000LL
1842		# define ShiftNATevent 24
1843		# define MaskFWXevent 0x000000000000FFFFLL
1844		# define ShiftFWXevent 0
1845		#else
1846		# define MaskConnID 0x00000000FFFFFFFFLL
1847		# define ShiftConnID 0
1848		# define MaskFWevent 0x000000FF00000000LL
1849		# define ShiftFWevent 32
1850		# define MasNATevent 0x000000FF00000000LL
1851		# define ShiftNATevent 32
1852		# define MaskFWXevent 0xFFFF000000000000LL
1853		# define ShiftFWXevent 48
1854
1855		#endif
1856
1857		// xlate ip/port
1858		# define OffsetXLATEPort NSEL_BASE_OFFSET+2
1859		uint16_t xlate_src_port; // index OffsetXLATEPort 0xffff'0000'0000'0000
1860		uint16_t xlate_dst_port; // index OffsetXLATEPort 0x0000'ffff'0000'0000
1861		uint32_t xlate_flags;
1862		# define OffsetXLATESRCIP NSEL_BASE_OFFSET+3
1863		ip_addr_t xlate_src_ip; // ipv4 OffsetXLATESRCIP +1 0x0000'0000'ffff'ffff
1864		// ipv6 OffsetXLATESRCIP 0xffff'ffff'ffff'ffff
1865		// ipv6 OffsetXLATESRCIP 0xffff'ffff'ffff'ffff
1866
1867		ip_addr_t xlate_dst_ip; // ipv4 OffsetXLATEDSTIP +1 0x0000'0000'ffff'ffff
1868		// ipv6 OffsetXLATEDSTIP 0xffff'ffff'ffff'ffff
1869		// ipv6 OffsetXLATEDSTIP 0xffff'ffff'ffff'ffff
1870		#ifdef WORDS_BIGENDIAN
1871		# define MaskXLATESRCPORT 0xFFFF000000000000LL
1872		# define ShiftXLATESRCPORT 48
1873		# define MaskXLATEDSTPORT 0x0000FFFF00000000LL
1874		# define ShiftXLATEDSTPORT 32
1875
1876		# define OffsetXLATESRCv4 OffsetXLATESRCIP+1
1877		# define MaskXLATEIPv4 0x00000000fFFFFFFFLL
1878		# define ShiftXLATEIPv4 0
1879
1880		# define OffsetXLATESRCv6a OffsetXLATESRCIP
1881		# define OffsetXLATESRCv6b OffsetXLATESRCIP+1
1882
1883		# define OffsetXLATEDSTv6a OffsetXLATESRCIP+2
1884		# define OffsetXLATEDSTv6b OffsetXLATESRCIP+3
1885
1886		#else
1887		# define MaskXLATESRCPORT 0x000000000000FFFFLL
1888		# define ShiftXLATESRCPORT 0
1889		# define MaskXLATEDSTPORT 0x00000000FFFF0000LL
1890		# define ShiftXLATEDSTPORT 16
1891
1892		# define OffsetXLATESRCv4 OffsetXLATESRCIP+1
1893		# define MaskXLATEIPv4 0xFFFFFFFF00000000LL
1894		# define ShiftXLATEIPv4 32
1895
1896		# define OffsetXLATESRCv6a OffsetXLATESRCIP
1897		# define OffsetXLATESRCv6b OffsetXLATESRCIP+1
1898
1899		# define OffsetXLATEDSTv6a OffsetXLATESRCIP+2
1900		# define OffsetXLATEDSTv6b OffsetXLATESRCIP+3
1901
1902		#endif
1903
1904
1905		// ingress/egress ACL id
1906		# define OffsetIngressAclId NSEL_BASE_OFFSET+7
1907		# define OffsetIngressAceId NSEL_BASE_OFFSET+7
1908		# define OffsetIngressGrpId NSEL_BASE_OFFSET+8
1909		# define OffsetEgressAclId NSEL_BASE_OFFSET+8
1910		# define OffsetEgressAceId NSEL_BASE_OFFSET+9
1911		# define OffsetEgressGrpId NSEL_BASE_OFFSET+9
1912		uint32_t ingress_acl_id[3]; // index OffsetIngressAclId 0xffff'ffff'0000'0000
1913		// index OffsetIngressAceId 0x0000'0000'ffff'ffff
1914		// index OffsetIngressGrpId 0xffff'ffff'0000'0000
1915		uint32_t egress_acl_id[3]; // index OffsetEgressAclId 0x0000'0000'ffff'ffff
1916		// index OffsetEgressAceId 0xffff'ffff'0000'0000
1917		// index OffsetEgressGrpId 0x0000'0000'ffff'ffff
1918		#ifdef WORDS_BIGENDIAN
1919		#define MaskIngressAclId 0xffffffff00000000LL
1920		#define ShiftIngressAclId 32
1921		#define MaskIngressAceId 0x00000000ffffffffLL
1922		#define ShiftIngressAceId 0
1923		#define MaskIngressGrpId 0xffffffff00000000LL
1924		#define ShiftIngressGrpId 32
1925		#define MaskEgressAclId 0x00000000ffffffffLL
1926		#define ShiftEgressAclId 0
1927		#define MaskEgressAceId 0xffffffff00000000LL
1928		#define ShiftEgressAceId 32
1929		#define MaskEgressGrpId 0x00000000ffffffffLL
1930		#define ShiftEgressGrpId 0
1931		#else
1932		#define MaskIngressAclId 0x00000000ffffffffLL
1933		#define ShiftIngressAclId 0
1934		#define MaskIngressAceId 0xffffffff00000000LL
1935		#define ShiftIngressAceId 32
1936		#define MaskIngressGrpId 0x00000000ffffffffLL
1937		#define ShiftIngressGrpId 0
1938		#define MaskEgressAclId 0xffffffff00000000LL
1939		#define ShiftEgressAclId 32
1940		#define MaskEgressAceId 0x00000000ffffffffLL
1941		#define ShiftEgressAceId 0
1942		#define MaskEgressGrpId 0xffffffff00000000LL
1943		#define ShiftEgressGrpId 32
1944		#endif
1945
1946		// username
1947		# define OffsetUsername NSEL_BASE_OFFSET+10
1948		char username[72];
1949
1950		// NAT extensions
1951		// NAT event is mapped into ASA event
1952		#define NAT_BASE_OFFSET (offsetof(master_record_t, ingress_vrfid) >> 3)
1953		// common block
1954		# define OffsetNELcommon NEL_BASE_OFFSET
1955		# define OffsetIVRFID NAT_BASE_OFFSET
1956		# define OffsetEVRFID NAT_BASE_OFFSET
1957		# define OffsetPortBlock NAT_BASE_OFFSET+1
1958		uint32_t ingress_vrfid; // OffsetIVRFID 0xffff'ffff'0000'0000
1959		uint32_t egress_vrfid; // OffsetEVRFID 0x0000'0000'ffff'ffff
1960
1961		// Port block allocation
1962		uint16_t block_start; // OffsetPortBlock 0xffff'0000'0000'0000
1963		uint16_t block_end; // OffsetPortBlock 0x0000'ffff'0000'0000
1964		uint16_t block_step; // OffsetPortBlock 0x0000'0000'ffff'0000
1965		uint16_t block_size; // OffsetPortBlock 0x0000'0000'0000'ffff
1966
1967		#ifdef WORDS_BIGENDIAN
1968		# define MaskIVRFID 0xFFFFFFFF00000000LL
1969		# define ShiftIVRFID 32
1970		# define MaskEVRFID 0x00000000FFFFFFFFLL
1971		# define ShiftEVRFID 0
1972		# define MaskPortBlockStart 0xFFFF000000000000LL
1973		# define ShiftPortBlockStart 48
1974		# define MaskPortBlockEnd 0x0000FFFF00000000LL
1975		# define ShiftPortBlockEnd 32
1976		# define MaskPortBlockStep 0x00000000FFFF0000LL
1977		# define ShiftPortBlockStep 16
1978		# define MaskPortBlockSize 0x000000000000FFFFLL
1979		# define ShiftPortBlockSize 0
1980		#else
1981		# define MaskIVRFID 0x00000000FFFFFFFFLL
1982		# define ShiftIVRFID 0
1983		# define MaskEVRFID 0xFFFFFFFF00000000LL
1984		# define ShiftEVRFID 32
1985		# define MaskPortBlockStart 0x000000000000FFFFLL
1986		# define ShiftPortBlockStart 0
1987		# define MaskPortBlockEnd 0x00000000FFFF0000LL
1988		# define ShiftPortBlockEnd 16
1989		# define MaskPortBlockStep 0x0000FFFF00000000LL
1990		# define ShiftPortBlockStep 32
1991		# define MaskPortBlockSize 0xFFFF000000000000LL
1992		# define ShiftPortBlockSize 48
1993		#endif
1994
1995		#endif
1996
1997		// latency extension
1998		uint64_t client_nw_delay_usec; // index LATENCY_BASE_OFFSET 0xffff'ffff'ffff'ffff
1999		uint64_t server_nw_delay_usec; // index LATENCY_BASE_OFFSET + 1 0xffff'ffff'ffff'ffff
2000		uint64_t appl_latency_usec; // index LATENCY_BASE_OFFSET + 2 0xffff'ffff'ffff'ffff
2001
2002		#define LATENCY_BASE_OFFSET (offsetof(master_record_t, client_nw_delay_usec) >> 3)
2003		# define OffsetClientLatency LATENCY_BASE_OFFSET
2004		# define OffsetServerLatency LATENCY_BASE_OFFSET + 1
2005		# define OffsetAppLatency LATENCY_BASE_OFFSET + 2
2006		# define MaskLatency 0xFFFFFFFFFFFFFFFFLL
2007		# define ShiftLatency 0
2008
2009		// flow received time in ms
2010		uint64_t received;
2011
2012		/* possible user extensions may fit here
2013		* - Put each extension into its own #ifdef
2014		* - Define the base offset for the user extension as reference to the first object
2015		* - Refer to this base offset for each of the values in the master record for the extension
2016		* - make sure the extension is 64bit aligned
2017		* - The user extension must be independant of the number of user extensions already defined
2018		* - the extension map must be updated accordingly
2019		*/
2020
2021		#ifdef USER_EXTENSION_1
2022		uint64_t u64_1;
2023		# define Offset_BASE_U1 offsetof(master_record_t, u64_1)
2024		# define OffsetUser1_u64 Offset_BASE_U1
2025
2026		uint32_t u32_1;
2027		uint32_t u32_2;
2028		# define OffsetUser1_u32_1 Offset_BASE_U1 + 8
2029		# define MaskUser1_u32_1 0xffffffff00000000LL
2030		# define MaskUser1_u32_2 0x00000000ffffffffLL
2031
2032		#endif
2033
2034		// reference to exporter
2035		exporter_info_record_t *exp_ref;
2036
2037		// last entry in master record
2038		# define Offset_MR_LAST offsetof(master_record_t, map_ref)
2039		extension_map_t *map_ref;
2040
2041		// optional flowlabel
2042		char *label;
2043		} master_record_t;
2044
2045		#define AnyMask 0xffffffffffffffffLL
2046
2047
2048		// convenience type conversion record
2049		typedef struct type_mask_s {
2050		union {
2051		uint8_t val8[8];
2052		uint16_t val16[4];
2053		uint32_t val32[2];
2054		uint64_t val64;
2055		} val;
2056		} type_mask_t;
2057
2058		/*
2059		* offset translation table
2060		* In netflow v9 values may have a different length, and may or may not be present.
2061		* The commmon information ( see data_block_record_t ) is expected to be present
2062		* unconditionally, and has a fixed size. IP addrs as well as counters for packets and
2063		* bytes are expexted to exist as well, but may be variable in size. Further information
2064		* may or may not be present, according the flags. See flags
2065		* To cope with this situation, the offset translation table gives the offset into an
2066		* uint32_t array at which offset the requested value start.
2067		*
2068		* index:
2069		* 0: dstip
2070		* for IPv4 netflow v5/v7 10
2071		* 1: dPkts
2072		* for IPv4 netflow v5/v7 11
2073		* 2: dOctets
2074		* for IPv4 netflow v5/v7 12
2075		*/
2076	241
2077	242	void SumStatRecords(stat_record_t s1, stat_record_t s2);
2078	243

2104	269
2105	270	void ModifyCompressFile(char * rfile, char *Rfile, int compress);
2106	271
2107		void ExpandRecord_v1(common_record_t input_record,master_record_t output_record );
2108	272
2109	273	#endif //_NFFILE_H
2110	274

+111

-0

bin/nffileV2.h less more

	0	/*
	1	* Copyright (c) 2020, Peter Haag
	2	* All rights reserved.
	3	*
	4	* Redistribution and use in source and binary forms, with or without
	5	* modification, are permitted provided that the following conditions are met:
	6	*
	7	* * Redistributions of source code must retain the above copyright notice,
	8	* this list of conditions and the following disclaimer.
	9	* * Redistributions in binary form must reproduce the above copyright notice,
	10	* this list of conditions and the following disclaimer in the documentation
	11	* and/or other materials provided with the distribution.
	12	* * Neither the name of the author nor the names of its contributors may be
	13	* used to endorse or promote products derived from this software without
	14	* specific prior written permission.
	15	*
	16	* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
	17	* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	18	* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	19	* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
	20	* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
	21	* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
	22	* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
	23	* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
	24	* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
	25	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
	26	* POSSIBILITY OF SUCH DAMAGE.
	27	*
	28	*/
	29
	30	#ifndef _NFFILEV2_H
	31	#define _NFFILEV2_H 1
	32
	33	#include "config.h"
	34
	35	#include <stddef.h>
	36	#include <sys/types.h>
	37	#ifdef HAVE_STDINT_H
	38	#include <stdint.h>
	39	#endif
	40
	41	/*
	42	* nfdump binary file layout 2
	43	* ===========================
	44	* Each data file starts with a file header, which identifies the file as an nfdump data file.
	45	* The magic 16bit integer at the beginning of each file must read 0xA50C. This also guarantees
	46	* that endian dependant files are read correct.
	47	*
	48	* Principal layout, recognized as LAYOUT_VERSION_2:
	49	*
	50	* +-----------+-------------+-------------+-------------+-----+-------------+
	51	* \|Fileheader \| datablock 0 \| datablock 1 \| datablock 2 \| ... \| datablock n \|
	52	* +-----------+-------------+-------------+-------------+-----+-------------+
	53	*/
	54
	55
	56	typedef struct fileHeaderV2_s {
	57	uint16_t magic; // magic to recognize nfdump file type and endian type
	58	#define MAGIC 0xA50C
	59
	60	uint16_t version; // version of binary file layout
	61	#define LAYOUT_VERSION_2 2
	62
	63	uint32_t nfversion; // version of nfdump created this file
	64	// 4bytes 1.6.19-1 0x01061301
	65	time_t created; // file create time
	66
	67	uint8_t compression;
	68	#define NOT_COMPRESSED 0
	69	#define LZO_COMPRESSED 1
	70	#define BZ2_COMPRESSED 2
	71	#define LZ4_COMPRESSED 3
	72	uint8_t encryption;
	73	uint16_t flags;
	74	uint32_t unused; // unused 0 - reserved for futur use
	75	uint64_t unused2; // unused 0 - reserved for futur use
	76
	77	uint32_t BlockSize; // max block size of data blocks
	78	uint32_t NumBlocks; // number of data blocks in file
	79	} fileHeaderV2_t;
	80
	81
	82	/*
	83	*
	84	* Generic data block
	85	* ==================
	86	* Data blocks are generic containers for the any type of data records.
	87	* Each data block starts with a block header, which specifies the size, the number of records
	88	* and data block properties. The struct is compatible with type 2 data records
	89	*/
	90
	91	typedef struct dataBlock_s {
	92	uint32_t type; // Block type
	93	#define TYPE_STAT_BLOCK 1
	94	#define TYPE_RECORD_BLOCK 2
	95	uint32_t size; // size of this block in bytes without this header
	96	} dataBlock_t;
	97
	98	/*
	99	* Generic data record
	100	* Contains any type of data, specified by type
	101	*/
	102	typedef struct recordHeader_s {
	103	// record header
	104	uint16_t type;
	105	uint16_t size;
	106	} recordHeader_t;
	107
	108
	109	#endif //_NFFILEV2_H
	110

-2

bin/nffile_inline.c less more

356	356	output_record->event_flag = FW_EVENT;
357	357	output_record->fw_xevent = tpl->fw_xevent;
358	358	output_record->icmp = tpl->nsel_icmp;
	359	output_record->sec_group_tag = tpl->sec_group_tag;
359	360	p = (void *)tpl->data;
360	361	} break;
361	362	case EX_NSEL_XLATE_PORTS: {

719	720	tpl->fw_event = master_record->event;
720	721	tpl->nsel_icmp = master_record->icmp;
721	722	tpl->fill = 0;
722		tpl->fill2 = 0;
	723	tpl->sec_group_tag = master_record->sec_group_tag;
723	724	tpl->fw_xevent = master_record->fw_xevent;
724	725	p = (void *)tpl->data;
725	726	} break;

786	787	}
787	788	}
788	789
789		nffile->block_header->size += required;
	790	nffile->block_header->size += required;
790	791	nffile->block_header->NumRecords++;
791	792	#ifdef DEVEL
792	793	if ( ((pointer_addr_t)p - (pointer_addr_t)nffile->buff_ptr) != required ) {

+10

-18

bin/nfgen.c less more

0	0	/*
1		* Copyright (c) 2017, Peter Haag
2		* Copyright (c) 2014, Peter Haag
3		* Copyright (c) 2009, Peter Haag
	1	* Copyright (c) 2009-2020 Peter Haag
4	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
5	3	* All rights reserved.
6	4	*

28	26	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
29	27	* POSSIBILITY OF SUCH DAMAGE.
30	28	*
31		* $Author: haag $
32		*
33		* $Id: nfgen.c 39 2009-11-25 08:11:15Z haag $
34		*
35		* $LastChangedRevision: 39 $
36		*
37	29	*/
38	30
39	31	#include "config.h"

56	48	#include <stdint.h>
57	49	#endif
58	50
	51	#include "util.h"
	52	#include "nfdump.h"
59	53	#include "nffile.h"
	54	#include "collector.h"
60	55	#include "nfx.h"
61		#include "nfnet.h"
62		#include "nf_common.h"
63		#include "util.h"
64		#include "bookkeeper.h"
65		#include "collector.h"
66		#include "exporter.h"
67		#include "netflow_v5_v7.h"
68	56
69	57	extern extension_descriptor_t extension_descriptor[];
70	58

399	387	SetIPaddress(&record, PF_INET, "172.16.13.66", "192.168.170.112");
400	388	record.srcport = 0;
401	389	record.dstport = 8;
402		record.prot = 1;
	390	record.prot = IPPROTO_ICMP;
403	391	record.tcp_flags = 0;
404	392	record.tos = 0;
405	393	record.dPkts = 50002;

430	418	UpdateRecord(&record);
431	419	PackRecord(&record, nffile);
432	420
433		SetIPaddress(&record, PF_INET6, "2001:234:aabb::211:24ff:fe80:d01e", "2001:620::8:203:baff:fe52:38e5");
	421	SetIPaddress(&record, PF_INET6, "2001:234:aabb:cc:211:24ff:fe80:d01e", "2001:620:1f:8:203:baff:fe52:38e5");
	422	record.src_mask = 88;
	423	record.dst_mask = 48;
434	424	record.srcport = 10240;
435	425	record.dstport = 52345;
436	426	record.dPkts = 10100;

459	449	SetIPaddress(&record, PF_INET, "172.16.14.18", "192.168.170.113");
460	450	// SetNextIPaddress(&record, PF_INET, "172.72.1.2");
461	451	// SetBGPNextIPaddress(&record, PF_INET, "172.73.2.3");
	452	record.src_mask = 16;
	453	record.dst_mask = 24;
462	454	record.srcport = 10240;
463	455	record.dstport = 52345;
464	456	record.dPkts = 10100000;

-9

bin/nflowcache.c less more

0	0	/*
1		* Copyright (c) 2017, Peter Haag
2		* Copyright (c) 2014, Peter Haag
3		* Copyright (c) 2009, Peter Haag
	1	* Copyright (c) 2009-2020, Peter Haag
4	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
5	3	* All rights reserved.
6	4	*

48	46	#include <stdint.h>
49	47	#endif
50	48
	49	#include "util.h"
	50	#include "nfdump.h"
51	51	#include "nffile.h"
52	52	#include "nfx.h"
53	53	#include "nflowcache.h"
54
55		#ifndef DEVEL
56		# define dbg_printf(...) /* printf(__VA_ARGS__) */
57		#else
58		# define dbg_printf(...) printf(__VA_ARGS__)
59		#endif
60	54
61	55	#define ALIGN_BYTES (offsetof (struct { char x; uint64_t y; }, y) - 1)
62	56

-58

bin/nfnet.c less more

0	0	/*
1		* Copyright (c) 2014, Peter Haag
2		* Copyright (c) 2009, Peter Haag
	1	* Copyright (c) 2009-2020, Peter Haag
3	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
4	3	* All rights reserved.
5	4	*

27	26	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
28	27	* POSSIBILITY OF SUCH DAMAGE.
29	28	*
30		* $Author: haag $
31		*
32		* $Id: nfnet.c 39 2009-11-25 08:11:15Z haag $
33		*
34		* $LastChangedRevision: 39 $
35		*
36		*
37	29	*/
38	30
39	31	#include "config.h"

143	135	if (sockfd < 0) {
144	136	freeaddrinfo(ressave);
145	137	fprintf(stderr, "Could not open the requested socket: %s\n", strerror (errno));
146		LogError("Receive socket error: could not open the requested socket", strerror (errno));
	138	LogError("Receive socket error: could not open the requested socket: %s", strerror (errno));
147	139	return -1;
148	140	}
149	141

423	415	*addrlen = res->ai_addrlen;
424	416	memcpy(addr, res->ai_addr, res->ai_addrlen);
425	417
426		/*
427		((struct sockaddr_in *)addr)->sin_family=AF_INET;
428		addr.sin_addr.s_addr=inet_addr(HELLO_GROUP);
429		addr.sin_port=htons(HELLO_PORT);
430		*/
431
432	418	if (joinGroup(sockfd, 1 , 1, (struct sockaddr_storage *)res->ai_addr) <0) {
433	419	close(sockfd);
434	420	freeaddrinfo(ressave);

437	423
438	424	freeaddrinfo(ressave);
439	425
440		/*
441		message = "hello world\n";
442		printf("Send %lu bytes, Message: %s\n", strlen(message)+1, message);
443		while (1) {
444		ret = sendto(sockfd,message,strlen(message)+1,0,(struct sockaddr ) addr, addrlen);
445		if ( ret != strlen(message)+1 ) {
446		perror("sendto");
447		exit(1);
448		}
449		printf("sleep\n");
450		sleep(1);
451		}
452		exit(255);
453		*/
454	426	return sockfd;
455	427
456	428	} /* End of Multicast_send_socket */

474	446	break;
475	447	}
476	448	ret = 0;
477		/*
478		// These two setsockopt may fail because not implemented on every OS, but harmless
479		err = setsockopt(sockfd, IPPROTO_IP, IP_MULTICAST_LOOP, &loopBack, sizeof(loopBack));
480		if ( err ) {
481		fprintf(stderr, "setsockopt IP_MULTICAST_LOOP: %s\n", strerror (errno));
482		LogError("setsockopt IP_MULTICAST_LOOP: %s", strerror (errno));
483		}
484
485		err = setsockopt(sockfd, IPPROTO_IP, IP_MULTICAST_TTL, &mcastTTL, sizeof(mcastTTL));
486		if ( err ) {
487		fprintf(stderr, "setsockopt IP_MULTICAST_LOOP: %s\n", strerror (errno));
488		LogError("setsockopt IP_MULTICAST_LOOP: %s", strerror (errno));
489		}
490		*/
491	449	} break;
492	450
493	451	case AF_INET6: {

503	461	break;
504	462	}
505	463	ret = 0;
506		/*
507		// These two setsockopt may fail because not implemented on every OS, but harmless
508		err = setsockopt(sockfd, IPPROTO_IPV6, IPV6_MULTICAST_LOOP, &loopBack, sizeof(loopBack));
509		if ( err ) {
510		fprintf(stderr, "setsockopt IPV6_MULTICAST_LOOP: %s\n", strerror (errno));
511		LogError("setsockopt IPV6_MULTICAST_LOOP: %s", strerror (errno));
512		}
513
514		err = setsockopt(sockfd, IPPROTO_IPV6, IPV6_MULTICAST_HOPS, &mcastTTL, sizeof(mcastTTL));
515		if ( err ) {
516		fprintf(stderr, "setsockopt IPV6_MULTICAST_HOPS: %s\n", strerror (errno));
517		LogError("setsockopt IPV6_MULTICAST_HOPS: %s", strerror (errno));
518		}
519		*/
520	464	} break;
521	465
522	466	default:

+84

-63

bin/nfpcapd.c less more

0	0	/*
1		* Copyright (c) 2013-2019, Peter Haag
	1	* Copyright (c) 2013-2020, Peter Haag
2	2	* All rights reserved.
3	3	*
4	4	* Redistribution and use in source and binary forms, with or without

82	82	#include <pcap.h>
83	83
84	84	#include "util.h"
	85	#include "nfdump.h"
85	86	#include "nffile.h"
86	87	#include "nfx.h"
87		#include "nf_common.h"
	88	#include "expire.h"
88	89	#include "nfnet.h"
89	90	#include "flist.h"
90	91	#include "nfstatfile.h"
91	92	#include "bookkeeper.h"
92	93	#include "collector.h"
93	94	#include "exporter.h"
94		#include "rbtree.h"
95	95	#include "ipfrag.h"
96
97		#include "expire.h"
98
99	96	#include "flowtree.h"
100	97	#include "netflow_pcap.h"
101	98	#include "pcaproc.h"

148	145	// arguments
149	146	int subdir_index;
150	147	char *pcap_datadir;
	148	char *time_extension;
151	149	pcap_dev_t *pcap_dev;
152	150	pcapfile_t *pcapfile;
153	151	} p_pcap_flush_thread_args_t;

167	165	time_t t_win;
168	166	int subdir_index;
169	167	char *pcap_datadir;
	168	char *time_extension;
170	169	int live;
171	170	} p_packet_thread_args_t;
172	171

182	181	NodeList_t *NodeList; // pop new nodes from this list
183	182	FlowSource_t *fs;
184	183	time_t t_win;
	184	char *time_extension;
185	185	int subdir_index;
186	186	int compress;
187	187	int live;

595	595	__attribute__((noreturn)) static void p_flow_thread(void thread_data) {
596	596	// argument dispatching
597	597	p_flow_thread_args_t args = (p_flow_thread_args_t )thread_data;
598		time_t t_win = args->t_win;
599		int subdir_index = args->subdir_index;
600		int compress = args->compress;
601		int live = args->live;
602		FlowSource_t *fs = args->fs;
	598	time_t t_win = args->t_win;
	599	int subdir_index = args->subdir_index;
	600	int compress = args->compress;
	601	int live = args->live;
	602	FlowSource_t *fs = args->fs;
	603	char *time_extension = args->time_extension;
603	604	static time_t lastExpire = 0;
604	605	time_t t_start, t_clock;
605	606	int err, done;

663	664	char FullName[MAXPATHLEN];
664	665	char netflowFname[128];
665	666	char error[256];
666		char *subdir;
	667	char *subdir, fmt[24];
667	668	uint32_t NumFlows;
668	669
669	670	// flush all flows to disk

674	675	NumFlows = Expire_FlowTree(fs, t_clock);
675	676
676	677	when = localtime(&t_start);
	678	strftime(fmt, sizeof(fmt), time_extension, when);
	679
677	680	nffile = fs->nffile;
678	681
679	682	// prepare sub dir hierarchy

685	688
686	689	// failed to generate subdir path - put flows into base directory
687	690	subdir = NULL;
688		snprintf(netflowFname, 127, "nfcapd.%i%02i%02i%02i%02i",
689		when->tm_year + 1900, when->tm_mon + 1, when->tm_mday, when->tm_hour, when->tm_min);
	691	snprintf(netflowFname, 127, "nfcapd.%s", fmt);
690	692	} else {
691		snprintf(netflowFname, 127, "%s/nfcapd.%i%02i%02i%02i%02i", subdir,
692		when->tm_year + 1900, when->tm_mon + 1, when->tm_mday, when->tm_hour, when->tm_min);
	693	snprintf(netflowFname, 127, "%s/nfcapd.%s", subdir, fmt);
693	694	}
694	695
695	696	} else {
696	697	subdir = NULL;
697		snprintf(netflowFname, 127, "nfcapd.%i%02i%02i%02i%02i",
698		when->tm_year + 1900, when->tm_mon + 1, when->tm_mday, when->tm_hour, when->tm_min);
	698	snprintf(netflowFname, 127, "nfcapd.%s", fmt);
699	699	}
700	700	netflowFname[127] = '\0';
701	701

781	781	}
782	782	}
783	783
784		if ( Node->fin != SIGNAL_NODE ) {
785		// Process the Node
786		ProcessFlowNode(fs, Node);
787		time_t when = Node->t_last.tv_sec;
788		if ( (when - lastExpire) > EXPIREINTERVALL ) {
789		Expire_FlowTree(fs, when);
790		lastExpire = when;
791		}
792		CacheCheck(fs, when, live);
793		}
	784	time_t when;
	785	if ( Node ) {
	786	if ( Node->fin != SIGNAL_NODE ) {
	787	// Process the Node
	788	ProcessFlowNode(fs, Node);
	789	}
	790	when = Node->t_last.tv_sec;
	791	} else {
	792	when = time(NULL);
	793	}
	794	if ( (when - lastExpire) > EXPIREINTERVALL ) {
	795	Expire_FlowTree(fs, when);
	796	lastExpire = when;
	797	}
	798	CacheCheck(fs, when, live);
794	799
795	800	}
796	801

812	817	char *pcap_datadir = args->pcap_datadir;
813	818	pcap_dev_t *pcap_dev = args->pcap_dev;
814	819	pcapfile_t *pcapfile = args->pcapfile;
	820	char *time_extension = args->time_extension;
815	821	char pcap_dumpfile[MAXPATHLEN];
816	822	int err;
817	823	int runs = 0;

876	882	char FullName[MAXPATHLEN];
877	883	char pcapFname[128];
878	884	char error[256];
879		char *subdir;
	885	char *subdir, fmt[24];
880	886	int err;
881	887
882	888	dbg_printf("Flush rotate file\n");
883	889	when = localtime(&pcapfile->t_CloseRename);
	890	strftime(fmt, sizeof(fmt), time_extension, when);
	891
884	892	pcapfile->t_CloseRename = 0;
885	893
886	894	// prepare sub dir hierarchy

892	900
893	901	// failed to generate subdir path - put flows into base directory
894	902	subdir = NULL;
895		snprintf(pcapFname, 127, "pcapd.%i%02i%02i%02i%02i",
896		when->tm_year + 1900, when->tm_mon + 1, when->tm_mday, when->tm_hour, when->tm_min);
	903	snprintf(pcapFname, 127, "pcapd.%s", fmt);
897	904	} else {
898		snprintf(pcapFname, 127, "%s/pcapd.%i%02i%02i%02i%02i", subdir,
899		when->tm_year + 1900, when->tm_mon + 1, when->tm_mday, when->tm_hour, when->tm_min);
	905	snprintf(pcapFname, 127, "%s/pcapd.%s", subdir, fmt);
900	906	}
901	907
902	908	} else {
903	909	subdir = NULL;
904		snprintf(pcapFname, 127, "pcapd.%i%02i%02i%02i%02i",
905		when->tm_year + 1900, when->tm_mon + 1, when->tm_mday, when->tm_hour, when->tm_min);
	910	snprintf(pcapFname, 127, "pcapd.%s", fmt);
906	911	}
907	912	pcapFname[127] = '\0';
908	913

958	963	pcap_dev_t *pcap_dev = args->pcap_dev;
959	964	time_t t_win = args->t_win;
960	965	char *pcap_datadir = args->pcap_datadir;
	966	char *time_extension = args->time_extension;
961	967	int subdir_index = args->subdir_index;
962	968	int live = args->live;
963	969	// locals

993	999	/* NOTREACHED */
994	1000	}
995	1001
996		p_flush_thread_args.done = 0;
997		p_flush_thread_args.exit = 0;
998		p_flush_thread_args.parent = args->tid;
999		p_flush_thread_args.pcap_dev = args->pcap_dev;
1000		p_flush_thread_args.subdir_index = subdir_index;
1001		p_flush_thread_args.pcap_datadir = pcap_datadir;
1002		p_flush_thread_args.pcapfile = pcapfile;
	1002	p_flush_thread_args.done = 0;
	1003	p_flush_thread_args.exit = 0;
	1004	p_flush_thread_args.parent = args->tid;
	1005	p_flush_thread_args.pcap_dev = args->pcap_dev;
	1006	p_flush_thread_args.subdir_index = subdir_index;
	1007	p_flush_thread_args.pcap_datadir = pcap_datadir;
	1008	p_flush_thread_args.time_extension = time_extension;
	1009	p_flush_thread_args.pcapfile = pcapfile;
1003	1010
1004	1011	err = pthread_create(&p_flush_thread_args.tid, NULL, p_pcap_flush_thread, (void *)&p_flush_thread_args);
1005	1012	if ( err ) {

1190	1197	time_t t_win;
1191	1198	char device, pcapfile, filter, datadir, pcap_datadir, extension_tags, pidfile[MAXPATHLEN], pidstr[32];
1192	1199	char Ident, userid, *groupid;
	1200	char *time_extension;
1193	1201	pcap_dev_t *pcap_dev;
1194	1202	p_packet_thread_args_t *p_packet_thread_args;
1195	1203	p_flow_thread_args_t *p_flow_thread_args;

1208	1216	Ident = "none";
1209	1217	fs = NULL;
1210	1218	extension_tags = DefaultExtensions;
	1219	time_extension = "%Y%m%d%H%M";
1211	1220	subdir_index = 0;
1212	1221	compress = NOT_COMPRESSED;
1213	1222	verbose = 0;

1301	1310	} break;
1302	1311	case 't':
1303	1312	t_win = atoi(optarg);
	1313	if (t_win < 2) {
	1314	LogError("time interval <= 2s not allowed");
	1315	exit(EXIT_FAILURE);
	1316	}
1304	1317	if (t_win < 60) {
1305		LogError("WARNING, very small time frame (< 60)!");
1306		}
1307		if (t_win <= 0) {
1308		LogError("ERROR: time frame too small (<= 0)");
1309		exit(EXIT_FAILURE);
	1318	time_extension = "%Y%m%d%H%M%S";
1310	1319	}
1311	1320	break;
1312	1321	case 'j':

1340	1349	exit(255);
1341	1350	}
1342	1351	tmp[MAXPATHLEN-1] = 0;
1343		snprintf(pidfile, MAXPATHLEN - 1 - strlen(tmp), "%s/%s", tmp, optarg);
	1352	if ( (strlen(tmp) + strlen(optarg) + 3) < MAXPATHLEN ) {
	1353	snprintf(pidfile, MAXPATHLEN - 3 - strlen(tmp), "%s/%s", tmp, optarg);
	1354	} else {
	1355	fprintf(stderr, "pidfile MAXPATHLEN error:\n");
	1356	exit(255);
	1357	}
1344	1358	}
1345	1359	// pidfile now absolute path
1346	1360	pidfile[MAXPATHLEN-1] = 0;

1388	1402	exit(EXIT_FAILURE);
1389	1403	}
1390	1404
	1405	if ( !device && !pcapfile ) {
	1406	LogError("Specify either a device or a pcapfile to read packets from");
	1407	exit(EXIT_FAILURE);
	1408	}
	1409
1391	1410	if ( !Init_FlowTree(cache_size, active, inactive)) {
1392	1411	LogError("Init_FlowTree() failed.");
1393	1412	exit(EXIT_FAILURE);

1412	1431	exit(255);
1413	1432	}
1414	1433
1415		if ( do_daemonize && !InitLog(argv[0], SYSLOG_FACILITY)) {
	1434	if ( !InitLog(do_daemonize, argv[0], SYSLOG_FACILITY, verbose) ) {
1416	1435	pcap_close(pcap_dev->handle);
1417	1436	exit(255);
1418	1437	}

1528	1547	__FILE__, __LINE__, strerror(errno) );
1529	1548	exit(255);
1530	1549	}
1531		p_flow_thread_args->fs = fs;
1532		p_flow_thread_args->t_win = t_win;
1533		p_flow_thread_args->compress = compress;
1534		p_flow_thread_args->live = device != NULL;
1535		p_flow_thread_args->subdir_index = subdir_index;
1536		p_flow_thread_args->parent = pthread_self();
1537		p_flow_thread_args->NodeList = NewNodeList();
	1550	p_flow_thread_args->fs = fs;
	1551	p_flow_thread_args->t_win = t_win;
	1552	p_flow_thread_args->compress = compress;
	1553	p_flow_thread_args->live = device != NULL;
	1554	p_flow_thread_args->subdir_index = subdir_index;
	1555	p_flow_thread_args->parent = pthread_self();
	1556	p_flow_thread_args->NodeList = NewNodeList();
	1557	p_flow_thread_args->time_extension = time_extension;
1538	1558
1539	1559	err = 0;
1540	1560

1552	1572	__FILE__, __LINE__, strerror(errno) );
1553	1573	exit(255);
1554	1574	}
1555		p_packet_thread_args->pcap_dev = pcap_dev;
1556		p_packet_thread_args->t_win = t_win;
1557		p_packet_thread_args->subdir_index = subdir_index;
1558		p_packet_thread_args->pcap_datadir = pcap_datadir;
1559		p_packet_thread_args->live = device != NULL;
1560		p_packet_thread_args->parent = pthread_self();
1561		p_packet_thread_args->NodeList = p_flow_thread_args->NodeList;
	1575	p_packet_thread_args->pcap_dev = pcap_dev;
	1576	p_packet_thread_args->t_win = t_win;
	1577	p_packet_thread_args->subdir_index = subdir_index;
	1578	p_packet_thread_args->pcap_datadir = pcap_datadir;
	1579	p_packet_thread_args->live = device != NULL;
	1580	p_packet_thread_args->parent = pthread_self();
	1581	p_packet_thread_args->NodeList = p_flow_thread_args->NodeList;
	1582	p_packet_thread_args->time_extension = p_flow_thread_args->time_extension;
1562	1583
1563	1584	err = pthread_create(&p_packet_thread_args->tid, NULL, p_packet_thread, (void *)p_packet_thread_args);
1564	1585	if ( err ) {

+10

-17

bin/nfprofile.c less more

0	0	/*
1		* Copyright (c) 2009 - 2019, Peter Haag
2		* Copyright (c) 2004 - 2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
	1	* Copyright (c) 2009-2020, Peter Haag
	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
3	3	* All rights reserved.
4	4	*
5	5	* Redistribution and use in source and binary forms, with or without

47	47	#include <stdint.h>
48	48	#endif
49	49
50		#include "nf_common.h"
51		#include "rbtree.h"
	50	#include "util.h"
	51	#include "nfdump.h"
52	52	#include "nftree.h"
53		#include "nfdump.h"
54	53	#include "nffile.h"
55	54	#include "nfx.h"
56	55	#include "nfstat.h"

60	59	#include "exporter.h"
61	60	#include "ipconv.h"
62	61	#include "flist.h"
63		#include "util.h"
64	62	#include "profile.h"
65	63
66	64	/* externals */
67		extern generic_exporter_t **exporter_list;
	65	extern exporter_t **exporter_list;
68	66
69	67	/* Local Variables */
70	68	static const char *nfdump_version = VERSION;

114	112	static void process_data(profile_channel_info_t *channels, unsigned int num_channels, time_t tslot) {
115	113	common_record_t *flow_record;
116	114	nffile_t *nffile;
117		FilterEngine_data_t *engine;
	115	FilterEngine_t *engine;
118	116	int i, j, done, ret ;
119	117
120	118	nffile = GetNextFile(NULL, 0, 0);

162	160	} break; // not really needed
163	161	}
164	162
165		if ( nffile->block_header->id == Large_BLOCK_Type ) {
166		// skip
167		continue;
168		}
169
170	163	if ( nffile->block_header->id != DATA_BLOCK_TYPE_2 ) {
171	164	LogError("Can't process block type %u. Skip block.\n", nffile->block_header->id);
172	165	continue;

183	176
184	177	switch ( flow_record->type ) {
185	178	case CommonRecordType: {
186		generic_exporter_t *exp_info = exporter_list[flow_record->exporter_sysid];
	179	exporter_t *exp_info = exporter_list[flow_record->exporter_sysid];
187	180	uint32_t map_id = flow_record->ext_map;
188	181	master_record_t *master_record;
189	182

278	271	LogError("Failed to add Sampler Record\n");
279	272	}
280	273	} break;
281		case ExporterRecordType:
282		case SamplerRecordype:
	274	case LegacyRecordType1:
	275	case LegacyRecordType2:
283	276	case ExporterStatRecordType:
284	277	// Silently skip exporter records
285	278	break;

520	513	stdin_profile_params = 1;
521	514	break;
522	515	case 'L':
523		if ( !InitLog("nfprofile", optarg) )
	516	if ( !InitLog(0, "nfprofile", optarg, 0) )
524	517	exit(255);
525	518	break;
526	519	case 'Z':

-18

bin/nfreader.c less more

0	0	/*
1		* Copyright (c) 2009-2019, Peter Haag
	1	* Copyright (c) 2009-2020, Peter Haag
2	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
3	3	* All rights reserved.
4	4	*

42	42	* This compiles this code and links the required nfdump files
43	43	* If you do it by hand:
44	44	*
45		* gcc -c nfreader.c
46		* gcc -o nfreader nfreader.o nffile.o flist.o util.o
	45	* gcc -I.. -o nfreader nfreader.c -lnfdump
47	46	*
48	47	*/
49	48

68	67	#include <stdint.h>
69	68	#endif
70	69
	70	#include "util.h"
	71	#include "nfdump.h"
71	72	#include "nffile.h"
72	73	#include "nfx.h"
73	74	#include "bookkeeper.h"
74	75	#include "collector.h"
75	76	#include "exporter.h"
76		#include "util.h"
77	77	#include "flist.h"
78
79		#if ( SIZEOF_VOID_P == 8 )
80		typedef uint64_t pointer_addr_t;
81		#else
82		typedef uint32_t pointer_addr_t;
83		#endif
84	78
85	79	// module limited globals
86	80	extension_map_list_t *extension_map_list;
87	81
88		extern generic_exporter_t **exporter_list;
	82	extern exporter_t **exporter_list;
89	83
90	84	/* Function Prototypes */
91	85	static void usage(char *name);

205	199	} break; // not really needed
206	200	}
207	201
208		if ( nffile->block_header->id == Large_BLOCK_Type ) {
209		// skip
210		continue;
211		}
212
213	202	if ( nffile->block_header->id != DATA_BLOCK_TYPE_2 ) {
214	203	fprintf(stderr, "Can't process block type %u. Skip block.\n", nffile->block_header->id);
215	204	continue;

228	217	switch ( flow_record->type ) {
229	218	case CommonRecordType: {
230	219	uint32_t map_id = flow_record->ext_map;
231		generic_exporter_t *exp_info = exporter_list[flow_record->exporter_sysid];
	220	exporter_t *exp_info = exporter_list[flow_record->exporter_sysid];
232	221	if ( extension_map_list->slot[map_id] == NULL ) {
233	222	snprintf(string, 1024, "Corrupt data file! No such extension map id: %u. Skip record", flow_record->ext_map );
234	223	string[1023] = '\0';

305	294	break;
306	295	break;
307	296	case 'L':
308		if ( !InitLog("argv[0]", optarg) )
	297	if ( !InitLog(0, "argv[0]", optarg, 0) )
309	298	exit(255);
310	299	break;
311	300	case 'r':

+22

-24

bin/nfreplay.c less more

0	0	/*
1		* Copyright (c) 2018, 2017, 2016 Peter Haag
2		* Copyright (c) 2014, Peter Haag
3		* Copyright (c) 2009, Peter Haag
	1	* Copyright (c) 2009-2020, Peter Haag
4	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
5	3	* All rights reserved.
6	4	*

53	51	#include <stdint.h>
54	52	#endif
55	53
	54	#ifdef HAVE_STDIO_EXT_H
	55	#include <stdio_ext.h>
	56	#endif
	57
	58	#include "util.h"
	59	#include "nfdump.h"
56	60	#include "nffile.h"
57	61	#include "nfx.h"
58		#include "nf_common.h"
59		#include "rbtree.h"
	62	#include "flist.h"
60	63	#include "nftree.h"
61		#include "nfdump.h"
62	64	#include "nfnet.h"
63	65	#include "bookkeeper.h"
64	66	#include "collector.h"

66	68	#include "netflow_v5_v7.h"
67	69	#include "netflow_v9.h"
68	70	#include "nfprof.h"
69		#include "flist.h"
70		#include "util.h"
71		#include "grammar.h"
72	71
73	72	#define DEFAULTCISCOPORT "9995"
74	73	#define DEFAULTHOSTNAME "127.0.0.1"

81	80	#define FPURGE fpurge
82	81	#endif
83	82
84		/* Externals */
85		extern int yydebug;
86
87	83	/* Global Variables */
88		FilterEngine_data_t *Engine;
89		int verbose;
	84	FilterEngine_t *Engine;
	85	static int verbose;
90	86
91	87	/* Local Variables */
92	88	static const char *nfdump_version = VERSION;
93	89
94		send_peer_t peer;
	90	static send_peer_t peer;
95	91
96	92	extension_map_list_t *extension_map_list;
97
98		generic_exporter_t **exporter_list;
99	93
100	94	/* Function Prototypes */
101	95	static void usage(char *name);

116	110	printf("usage %s [options] [\"filter\"]\n"
117	111	"-h\t\tthis text you see right here\n"
118	112	"-V\t\tPrint version and exit.\n"
	113	"-E\t\tPrint verbose messages. For debugging purpose only.\n"
119	114	"-H <Host/ip>\tTarget IP address default: 127.0.0.1\n"
120	115	"-j <mcast>\tSend packets to multicast group\n"
121	116	"-4\t\tForce IPv4 protocol.\n"

209	204
210	205	// z-parameter variables
211	206	struct timeval todayTime, currentTime;
212		double first, last, now, today = 0, reftime = 0;
	207	double today = 0, reftime = 0;
213	208	int reducer = 0;
214	209
215	210	// Get the first file handle

365	360	}
366	361
367	362	} break;
368		case ExporterRecordType:
369		case SamplerRecordype:
	363	case LegacyRecordType1:
	364	case LegacyRecordType2:
370	365	case ExporterInfoRecordType:
371	366	case ExporterStatRecordType:
372	367	case SamplerInfoRecordype:

379	374
380	375	// z-parameter
381	376	//first and last are line (tstart and tend) timestamp with milliseconds
382		first = (double) flow_record->first + ((double)flow_record->msec_first / 1000);
383		last = (double) flow_record->last + ((double)flow_record->msec_last / 1000);
	377	// first = (double) flow_record->first + ((double)flow_record->msec_first / 1000);
	378	double last = (double) flow_record->last + ((double)flow_record->msec_last / 1000);
384	379
385	380	gettimeofday(&currentTime, NULL);
386		now = (double)currentTime.tv_sec + (double)currentTime.tv_usec / 1000000;
	381	double now = (double)currentTime.tv_sec + (double)currentTime.tv_usec / 1000000;
387	382
388	383	// remove incoherent values
389	384	if (reftime == 0 && last > 1000000000 && last < 2000000000){

462	457	case 'B':
463	458	blast = 1;
464	459	break;
	460	case 'E':
	461	verbose = 1;
	462	break;
465	463	case 'V':
466	464	printf("%s: Version: %s\n",argv[0], nfdump_version);
467	465	exit(0);

488	486	exit(255);
489	487	break;
490	488	case 'L':
491		if ( !InitLog(argv[0], optarg) )
	489	if ( !InitLog(0, argv[0], optarg, verbose) )
492	490	exit(255);
493	491	break;
494	492	case 'p':

+112

-123

bin/nfstat.c less more

0	0	/*
1		* Copyright (c) 2017, Peter Haag
2		* Copyright (c) 2014, Peter Haag
3		* Copyright (c) 2009, Peter Haag
	1	* Copyright (c) 2009-2020, Peter Haag
4	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
5	3	* All rights reserved.
6	4	*

47	45	#include <stdint.h>
48	46	#endif
49	47
50		#include "rbtree.h"
	48	#include "util.h"
51	49	#include "nfdump.h"
52	50	#include "nffile.h"
53	51	#include "nfx.h"

55	53	#include "collector.h"
56	54	#include "exporter.h"
57	55	#include "nfnet.h"
58		#include "netflow_v5_v7.h"
59		#include "nf_common.h"
60		#include "util.h"
	56	#include "output_util.h"
61	57	#include "nflowcache.h"
62	58	#include "nfstat.h"
63	59

460	456	static inline void PrintSortedFlowcache(SortElement_t *SortList, uint32_t maxindex, int limit_count, int GuessFlowDirection,
461	457	printer_t print_record, int tag, int ascending, extension_map_list_t *extension_map_list );
462	458
463		static void PrintStatLine(stat_record_t stat, uint32_t plain_numbers, StatRecord_t StatData, int type, int order_proto, int tag, int inout);
	459	static void PrintStatLine(stat_record_t stat, uint32_t printPlain, StatRecord_t StatData, int type, int order_proto, int tag, int inout);
464	460
465	461	static void PrintPipeStatLine(StatRecord_t *StatData, int type, int order_proto, int tag, int inout);
466	462
467		static void PrintCvsStatLine(stat_record_t stat, StatRecord_t StatData, int type, int order_proto, int tag, int inout);
	463	static void PrintCvsStatLine(stat_record_t stat, int printPlain, StatRecord_t StatData, int type, int order_proto, int tag, int inout);
468	464
469	465	static inline int TimeMsec_CMP(time_t t1, uint16_t offset1, time_t t2, uint16_t offset2 );
470	466
471	467	static SortElement_t StatTopN(int topN, uint32_t count, int hash_num, int order );
472
473		static void SwapFlow(master_record_t *flow_record);
474	468
475	469	/* locals */
476	470	static hash_StatTable *StatTable;

485	479	#include "applybits_inline.c"
486	480
487	481	static uint64_t null_record(FlowTableRecord_t *record, int inout) {
488		return 0;
	482	return 0;
489	483	}
490	484
491	485	static uint64_t flows_record(FlowTableRecord_t *record, int inout) {
492		return record->counter[FLOWS];
	486	return record->counter[FLOWS];
493	487	}
494	488
495	489	static uint64_t packets_record(FlowTableRecord_t *record, int inout) {
496		if ( GuessDirection && (record->flowrecord.srcport < record->flowrecord.dstport) ) {
497		if (inout == IN)
498		inout = OUT;
499		else if (inout == OUT)
500		inout = IN;
501		}
502		if (inout == IN)
503		return record->counter[INPACKETS];
504		else if (inout == OUT)
505		return record->counter[OUTPACKETS];
506		else
507		return record->counter[INPACKETS] + record->counter[OUTPACKETS];
	490	if ( GuessDirection &&
	491	( record->flowrecord.prot == IPPROTO_TCP \|\| record->flowrecord.prot == IPPROTO_UDP) &&
	492	( record->flowrecord.srcport < 1024 ) && ( record->flowrecord.dstport > 1024 ) &&
	493	( record->flowrecord.srcport < record->flowrecord.dstport) ) {
	494	if (inout == IN)
	495	inout = OUT;
	496	else if (inout == OUT)
	497	inout = IN;
	498	}
	499	if (inout == IN)
	500	return record->counter[INPACKETS];
	501	else if (inout == OUT)
	502	return record->counter[OUTPACKETS];
	503	else
	504	return record->counter[INPACKETS] + record->counter[OUTPACKETS];
508	505	}
509	506
510	507	static uint64_t bytes_record(FlowTableRecord_t *record, int inout) {
511		if ( GuessDirection && (record->flowrecord.srcport < record->flowrecord.dstport) ) {
512		if (inout == IN)
513		inout = OUT;
514		else if (inout == OUT)
515		inout = IN;
516		}
517		if (inout == IN)
518		return record->counter[INBYTES];
519		else if (inout == OUT)
520		return record->counter[OUTBYTES];
521		else
522		return record->counter[INBYTES] + record->counter[OUTBYTES];
	508	if ( GuessDirection &&
	509	( record->flowrecord.prot == IPPROTO_TCP \|\| record->flowrecord.prot == IPPROTO_UDP) &&
	510	( record->flowrecord.srcport < 1024 ) && ( record->flowrecord.dstport > 1024 ) &&
	511	( record->flowrecord.srcport < record->flowrecord.dstport) ) {
	512	if (inout == IN)
	513	inout = OUT;
	514	else if (inout == OUT)
	515	inout = IN;
	516	}
	517	if (inout == IN)
	518	return record->counter[INBYTES];
	519	else if (inout == OUT)
	520	return record->counter[OUTBYTES];
	521	else
	522	return record->counter[INBYTES] + record->counter[OUTBYTES];
523	523	}
524	524
525	525	static uint64_t pps_record(FlowTableRecord_t *record, int inout) {

545	545	if ( duration == 0 )
546	546	return 0;
547	547	else {
548		bytes = bytes_record(record, inout);
	548	bytes = bytes_record(record, inout);
549	549	return ( 8000LL * bytes ) / duration; /* 8 bits per Octet - x 1000 for msec */
550	550	}
551	551

572	572	} // End of tend_record
573	573
574	574	static uint64_t null_element(StatRecord_t *record, int inout) {
575		return 0;
	575	return 0;
576	576	}
577	577
578	578	static uint64_t flows_element(StatRecord_t *record, int inout) {
579		return record->counter[FLOWS];
	579	return record->counter[FLOWS];
580	580	}
581	581
582	582	static uint64_t packets_element(StatRecord_t *record, int inout) {
583		if (inout == IN)
584		return record->counter[INPACKETS];
585		else if (inout == OUT)
586		return record->counter[OUTPACKETS];
587		else
588		return record->counter[INPACKETS] + record->counter[OUTPACKETS];
	583	if (inout == IN)
	584	return record->counter[INPACKETS];
	585	else if (inout == OUT)
	586	return record->counter[OUTPACKETS];
	587	else
	588	return record->counter[INPACKETS] + record->counter[OUTPACKETS];
589	589	}
590	590
591	591	static uint64_t bytes_element(StatRecord_t *record, int inout) {
592		if (inout == IN)
593		return record->counter[INBYTES];
594		else if (inout == OUT)
595		return record->counter[OUTBYTES];
596		else
597		return record->counter[INBYTES] + record->counter[OUTBYTES];
	592	if (inout == IN)
	593	return record->counter[INBYTES];
	594	else if (inout == OUT)
	595	return record->counter[OUTBYTES];
	596	else
	597	return record->counter[INBYTES] + record->counter[OUTBYTES];
598	598	}
599	599
600	600	static uint64_t pps_element(StatRecord_t *record, int inout) {

606	606	if ( duration == 0 )
607	607	return 0;
608	608	else {
609		packets = packets_element(record, inout);
	609	packets = packets_element(record, inout);
610	610	return ( 1000LL * packets ) / duration;
611	611	}
612	612

620	620	if ( duration == 0 )
621	621	return 0;
622	622	else {
623		bytes = bytes_element(record, inout);
	623	bytes = bytes_element(record, inout);
624	624	return ( 8000LL * bytes ) / duration; /* 8 bits per Octet - x 1000 for msec */
625	625	}
626	626

912	912
913	913	// no order is given - default order applies;
914	914	if ( !q ) {
915		free(s);
916		return 1;
	915	q = "/flows"; // default to flows
917	916	}
918	917
919	918	// check if one or more orders are given

1011	1010	if ( StatTable[hash_num].NumBlocks >= StatTable[hash_num].MaxBlocks ) {
1012	1011	StatTable[hash_num].MaxBlocks += MaxMemBlocks;
1013	1012	StatTable[hash_num].memblock = (StatRecord_t **)realloc(StatTable[hash_num].memblock,
1014		StatTable[hash_num].MaxBlocks * sizeof(StatRecord_t *));
	1013	StatTable[hash_num].MaxBlocks * sizeof(StatRecord_t *));
1015	1014	if ( !StatTable[hash_num].memblock ) {
1016	1015	fprintf(stderr, "realloc() error in %s line %d: %s\n", __FILE__, __LINE__, strerror (errno));
1017	1016	exit(250);
1018	1017	}
1019	1018	}
1020	1019	StatTable[hash_num].memblock[StatTable[hash_num].NumBlocks] =
1021		(StatRecord_t *)calloc(StatTable[hash_num].Prealloc, sizeof(StatRecord_t));
	1020	(StatRecord_t *)calloc(StatTable[hash_num].Prealloc, sizeof(StatRecord_t));
1022	1021
1023	1022	if ( !StatTable[hash_num].memblock[StatTable[hash_num].NumBlocks] ) {
1024	1023	fprintf(stderr, "calloc() error in %s line %d: %s\n", __FILE__, __LINE__, strerror (errno));

1121	1120
1122	1121	} // End of AddStat
1123	1122
1124		static void PrintStatLine(stat_record_t stat, uint32_t plain_numbers, StatRecord_t StatData, int type, int order_proto, int tag, int inout) {
1125		char proto[16], valstr[40], datestr[64];
	1123	static void PrintStatLine(stat_record_t stat, uint32_t printPlain, StatRecord_t StatData, int type, int order_proto, int tag, int inout) {
	1124	char valstr[40], datestr[64];
1126	1125	char flows_str[NUMBER_STRING_SIZE], byte_str[NUMBER_STRING_SIZE], packets_str[NUMBER_STRING_SIZE];
1127	1126	char pps_str[NUMBER_STRING_SIZE], bps_str[NUMBER_STRING_SIZE];
1128	1127	char tag_string[2];

1130	1129	double duration, flows_percent, packets_percent, bytes_percent;
1131	1130	uint32_t bpp;
1132	1131	uint64_t pps, bps;
1133		int scale;
1134	1132	time_t first;
1135	1133	struct tm *tbuff;
1136	1134

1150	1148	_key[1] = htonll(StatData->stat_key[1]);
1151	1149	inet_ntop(AF_INET6, _key, valstr, sizeof(valstr));
1152	1150	if ( ! Getv6Mode() )
1153		condense_v6(valstr);
	1151	CondenseV6(valstr);
1154	1152
1155	1153	} else { // IPv4
1156	1154	uint32_t ipv4;

1205	1203	}
1206	1204
1207	1205	valstr[39] = 0;
1208		scale = plain_numbers == 0;
1209	1206	count_flows = StatData->counter[FLOWS];
1210	1207	count_packets = packets_element(StatData, inout);
1211	1208	count_bytes = bytes_element(StatData, inout);
1212		format_number(count_flows, flows_str, scale, FIXED_WIDTH);
1213		format_number(count_packets, packets_str, scale, FIXED_WIDTH);
1214		format_number(count_bytes, byte_str, scale, FIXED_WIDTH);
	1209	format_number(count_flows, flows_str, printPlain, FIXED_WIDTH);
	1210	format_number(count_packets, packets_str, printPlain, FIXED_WIDTH);
	1211	format_number(count_bytes, byte_str, printPlain, FIXED_WIDTH);
1215	1212
1216	1213	flows_percent = stat->numflows ? (double)(count_flows * 100 ) / (double)stat->numflows : 0;
1217	1214	if ( stat->numpackets ) {

1242	1239	bpp = 0;
1243	1240	}
1244	1241
1245		format_number(pps, pps_str, scale, FIXED_WIDTH);
1246		format_number(bps, bps_str, scale, FIXED_WIDTH);
	1242	format_number(pps, pps_str, printPlain, FIXED_WIDTH);
	1243	format_number(bps, bps_str, printPlain, FIXED_WIDTH);
1247	1244
1248	1245	first = StatData->first;
1249	1246	tbuff = localtime(&first);

1253	1250	}
1254	1251	strftime(datestr, 63, "%Y-%m-%d %H:%M:%S", tbuff);
1255	1252
1256		if ( order_proto ) {
1257		Proto_string(StatData->prot, proto);
1258		} else {
1259		snprintf(proto, 15, "any ");
1260		proto[15] = 0;
1261		}
1262
1263	1253	if ( Getv6Mode() && ( type == IS_IPADDR ) )
1264		printf("%s.%03u %9.3f %s %s%39s %8s(%4.1f) %8s(%4.1f) %8s(%4.1f) %8s %8s %5u\n",
1265		datestr, StatData->msec_first, duration, proto, tag_string, valstr,
1266		flows_str, flows_percent, packets_str, packets_percent, byte_str,
1267		bytes_percent, pps_str, bps_str, bpp );
	1254	printf("%s.%03u %9.3f %-5s %s%39s %8s(%4.1f) %8s(%4.1f) %8s(%4.1f) %8s %8s %5u\n",
	1255	datestr, StatData->msec_first, duration,
	1256	order_proto ? ProtoString(StatData->prot, printPlain) : "any", tag_string, valstr,
	1257	flows_str, flows_percent, packets_str, packets_percent, byte_str,
	1258	bytes_percent, pps_str, bps_str, bpp );
1268	1259	else {
1269		printf("%s.%03u %9.3f %s %s%17s %8s(%4.1f) %8s(%4.1f) %8s(%4.1f) %8s %8s %5u\n",
1270		datestr, StatData->msec_first, duration, proto, tag_string, valstr,
1271		flows_str, flows_percent, packets_str, packets_percent, byte_str,
1272		bytes_percent, pps_str, bps_str, bpp );
	1260	printf("%s.%03u %9.3f %-5s %s%17s %8s(%4.1f) %8s(%4.1f) %8s(%4.1f) %8s %8s %5u\n",
	1261	datestr, StatData->msec_first, duration,
	1262	order_proto ? ProtoString(StatData->prot, printPlain) : "any", tag_string, valstr,
	1263	flows_str, flows_percent, packets_str, packets_percent, byte_str,
	1264	bytes_percent, pps_str, bps_str, bpp );
1273	1265	}
1274	1266
1275	1267	} // End of PrintStatLine

1324	1316
1325	1317	if ( type == IS_IPADDR )
1326	1318	printf("%i\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%llu\|%llu\|%llu\|%u\|%u\|%u\n",
1327		af, StatData->first, StatData->msec_first ,StatData->last, StatData->msec_last, StatData->prot,
1328		sa[0], sa[1], sa[2], sa[3], (long long unsigned)count_flows,
1329		(long long unsigned)count_packets, (long long unsigned)count_bytes,
1330		pps, bps, bpp);
	1319	af, StatData->first, StatData->msec_first ,StatData->last, StatData->msec_last, StatData->prot,
	1320	sa[0], sa[1], sa[2], sa[3], (long long unsigned)count_flows,
	1321	(long long unsigned)count_packets, (long long unsigned)count_bytes,
	1322	pps, bps, bpp);
1331	1323	else
1332	1324	printf("%i\|%u\|%u\|%u\|%u\|%u\|%llu\|%llu\|%llu\|%llu\|%u\|%u\|%u\n",
1333		af, StatData->first, StatData->msec_first ,StatData->last, StatData->msec_last, StatData->prot,
1334		(long long unsigned)_key[1], (long long unsigned)count_flows,
1335		(long long unsigned)count_packets, (long long unsigned)count_bytes,
1336		pps, bps, bpp);
	1325	af, StatData->first, StatData->msec_first ,StatData->last, StatData->msec_last, StatData->prot,
	1326	(long long unsigned)_key[1], (long long unsigned)count_flows,
	1327	(long long unsigned)count_packets, (long long unsigned)count_bytes,
	1328	pps, bps, bpp);
1337	1329
1338	1330	} // End of PrintPipeStatLine
1339	1331
1340		static void PrintCvsStatLine(stat_record_t stat, StatRecord_t StatData, int type, int order_proto, int tag, int inout) {
1341		char proto[16], valstr[40], datestr1[64], datestr2[64];
	1332	static void PrintCvsStatLine(stat_record_t stat, int printPlain, StatRecord_t StatData, int type, int order_proto, int tag, int inout) {
	1333	char valstr[40], datestr1[64], datestr2[64];
1342	1334	uint64_t count_flows, count_packets, count_bytes;
1343	1335	double duration, flows_percent, packets_percent, bytes_percent;
1344		uint32_t i, bpp;
	1336	uint32_t bpp;
1345	1337	uint64_t pps, bps;
1346	1338	time_t when;
1347	1339	struct tm *tbuff;

1424	1416	}
1425	1417	strftime(datestr2, 63, "%Y-%m-%d %H:%M:%S", tbuff);
1426	1418
1427		if ( order_proto ) {
1428		Proto_string(StatData->prot, proto);
1429		} else {
1430		snprintf(proto, 15, "any ");
1431		proto[15] = 0;
1432		}
1433
1434		i=0;
1435		while ( proto[i] ) {
1436		if ( proto[i] == ' ' )
1437		proto[i] = '\0';
1438		i++;
1439		}
1440
1441		printf("%s,%s,%.3f,%s,%s,%llu,%.1f,%llu,%.1f,%llu,%.1f,%llu,%llu,%u\n",
1442		datestr1, datestr2, duration, proto, valstr,
1443		(long long unsigned)count_flows, flows_percent,
1444		(long long unsigned)count_packets, packets_percent,
1445		(long long unsigned)count_bytes, bytes_percent,
1446		(long long unsigned)pps,(long long unsigned)bps,bpp);
	1419	printf("%s,%s,%.3f,%s,%s,%llu,%.1f,%llu,%.1f,%llu,%.1f,%llu,%llu,%u\n",
	1420	datestr1, datestr2, duration,
	1421	order_proto ? ProtoString(StatData->prot, printPlain) : "any", valstr,
	1422	(long long unsigned)count_flows, flows_percent,
	1423	(long long unsigned)count_packets, packets_percent,
	1424	(long long unsigned)count_bytes, bytes_percent,
	1425	(long long unsigned)pps,(long long unsigned)bps,bpp);
1447	1426
1448	1427	} // End of PrintCvsStatLine
1449	1428

1566	1545	if ( aggr_record_mask ) {
1567	1546	ApplyAggrMask(flow_record, aggr_record_mask);
1568	1547	}
1569		if ( GuessDir && ( flow_record->srcport < flow_record->dstport ) )
	1548
	1549	if ( GuessDir &&
	1550	( flow_record->prot == IPPROTO_TCP \|\| flow_record->prot == IPPROTO_UDP) &&
	1551	( flow_record->srcport < 1024 ) && ( flow_record->dstport >= 1024 ) &&
	1552	( flow_record->srcport < 32768 ) && ( flow_record->dstport >= 32768 ) &&
	1553	( flow_record->srcport < 49152 ) && ( flow_record->dstport >= 49152 ))
1570	1554	SwapFlow(flow_record);
	1555
1571	1556	print_record((void *)flow_record, &string, tag);
1572	1557	printf("%s\n", string);
1573	1558

1578	1563	}
1579	1564	} // End of PrintFlowTable
1580	1565
1581		void PrintFlowStat(char record_header, printer_t print_record, int topN, int tag, int quiet, int cvs_output, extension_map_list_t extension_map_list) {
	1566	void PrintFlowStat(func_prolog_t record_header, printer_t print_record, int topN, int tag, int quiet, int cvs_output, extension_map_list_t *extension_map_list) {
1582	1567	hash_FlowTable *FlowTable;
1583	1568	FlowTableRecord_t *r;
1584	1569	SortElement_t *SortList;

1655	1640	printf("Top flows ordered by %s:\n", order_mode[order_index].string);
1656	1641	}
1657	1642	if ( record_header )
1658		printf("%s\n", record_header);
	1643	record_header();
1659	1644	}
1660	1645
1661	1646	PrintSortedFlowcache(SortList, maxindex, topN, 0, print_record, tag, DESCENDING, extension_map_list);

1683	1668	printf("Top flows ordered by %s:\n", order_mode[order_index].string);
1684	1669	}
1685	1670	if ( record_header )
1686		printf("%s\n", record_header);
	1671	record_header();
1687	1672	}
1688	1673	PrintSortedFlowcache(SortList, maxindex, topN, 0, print_record, tag, DESCENDING, extension_map_list);
1689	1674

1748	1733	} else if ( aggr_record_mask )
1749	1734	ApplyAggrMask(flow_record, aggr_record_mask);
1750	1735
1751		if ( GuessFlowDirection && ( flow_record->srcport < flow_record->dstport ) )
	1736	if ( GuessFlowDirection &&
	1737	( flow_record->prot == IPPROTO_TCP \|\| flow_record->prot == IPPROTO_UDP) &&
	1738	( flow_record->srcport < 1024 ) && ( flow_record->dstport >= 1024 ) &&
	1739	( flow_record->srcport < 32768 ) && ( flow_record->dstport >= 32768 ) &&
	1740	( flow_record->srcport < 49152 ) && ( flow_record->dstport >= 49152 ))
1752	1741	SwapFlow(flow_record);
1753	1742
1754	1743	print_record((void *)flow_record, &string, tag);

1757	1746
1758	1747	} // End of PrintSortedFlowcache
1759	1748
1760		void PrintElementStat(stat_record_t sum_stat, uint32_t limitflows, char record_header, printer_t print_record, int topN, int tag, int quiet, int pipe_output, int cvs_output) {
	1749	void PrintElementStat(stat_record_t *sum_stat, uint32_t printPlain, printer_t print_record, int topN, int tag, int quiet, int pipe_output, int cvs_output) {
1761	1750	SortElement_t *topN_element_list;
1762	1751	uint32_t numflows;
1763	1752	int32_t i, j, hash_num, order_index;

1812	1801	PrintPipeStatLine((StatRecord_t *)topN_element_list[i].record, type,
1813	1802	StatRequest[hash_num].order_proto, tag, order_mode[order_index].inout);
1814	1803	else if ( cvs_output )
1815		PrintCvsStatLine(sum_stat, (StatRecord_t *)topN_element_list[i].record, type,
	1804	PrintCvsStatLine(sum_stat, printPlain, (StatRecord_t *)topN_element_list[i].record, type,
1816	1805	StatRequest[hash_num].order_proto, tag, order_mode[order_index].inout);
1817	1806	else
1818		PrintStatLine(sum_stat, limitflows, (StatRecord_t *)topN_element_list[i].record,
	1807	PrintStatLine(sum_stat, printPlain, (StatRecord_t *)topN_element_list[i].record,
1819	1808	type, StatRequest[hash_num].order_proto, tag, order_mode[order_index].inout);
1820	1809	}
1821	1810	free((void *)topN_element_list);

1895	1884	} // End of StatTopN
1896	1885
1897	1886
1898		static void SwapFlow(master_record_t *flow_record) {
	1887	void SwapFlow(master_record_t *flow_record) {
1899	1888	uint64_t _tmp_ip[2];
1900	1889	uint64_t _tmp_l;
1901	1890	uint32_t _tmp;

-6

bin/nfstat.h less more

0	0	/*
1		* Copyright (c) 2017, Peter Haag
2		* Copyright (c) 2014, Peter Haag
3		* Copyright (c) 2009, Peter Haag
	1	* Copyright (c) 2009-2020, Peter Haag
4	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
5	3	* All rights reserved.
6	4	*

40	38	#include <stdint.h>
41	39	#endif
42	40
43		#include "nf_common.h"
	41	#include "output_util.h"
	42	#include "output_fmt.h"
44	43	#include "nfx.h"
45	44	#include "nffile.h"
46	45

119	118
120	119	void PrintFlowTable(printer_t print_record, uint32_t limitflows, int tag, int GuessDir, extension_map_list_t *extension_map_list);
121	120
122		void PrintFlowStat(char record_header, printer_t print_record, int topN, int tag, int quiet, int cvs_output, extension_map_list_t extension_map_list);
	121	void PrintFlowStat(func_prolog_t record_header, printer_t print_record, int topN, int tag, int quiet, int cvs_output, extension_map_list_t *extension_map_list);
123	122
124		void PrintElementStat(stat_record_t sum_stat, uint32_t limitflows, char record_header, printer_t print_record, int topN, int tag, int quiet, int pipe_output, int cvs_output);
	123	void PrintElementStat(stat_record_t *sum_stat, uint32_t printPlain, printer_t print_record, int topN, int tag, int quiet, int pipe_output, int cvs_output);
125	124
126	125	int ParseListOrder(char *s, int multiple_orders );
127	126
128	127	void PrintSortedFlows(printer_t print_record, uint32_t limitflows, int tag);
129	128
	129	void SwapFlow(master_record_t *flow_record);
130	130	#endif //_NFSTAT_H

-3

bin/nfstatfile.c less more

0	0	/*
1		* Copyright (c) 2016, Peter Haag
2		* Copyright (c) 2014, Peter Haag
3		* Copyright (c) 2009, Peter Haag
	1	* Copyright (c) 2009-2019, Peter Haag
4	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
5	3	* All rights reserved.
6	4	*

-11

bin/nftest.c less more

0	0	/*
1		* Copyright (c) 2017, Peter Haag
2		* Copyright (c) 2014, Peter Haag
3		* Copyright (c) 2009, Peter Haag
	1	* Copyright (c) 2009-2020, Peter Haag
4	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
5	3	* All rights reserved.
6	4	*

64	62
65	63	#define ALIGN_BYTES (offsetof (struct { char x; uint64_t y; }, y) - 1)
66	64
67		#include "rbtree.h"
	65	#include "util.h"
68	66	#include "nfdump.h"
	67	#include "nffile.h"
69	68	#include "nftree.h"
70		#include "nffile.h"
71		#include "nf_common.h"
	69	#include "filter.h"
72	70	#include "nfx.h"
73		#include "util.h"
74	71
75	72	/* Global Variables */
76	73	extern char *CurrentIdent;
77	74	extern extension_descriptor_t extension_descriptor[];
78	75
79		FilterEngine_data_t *Engine;
	76	FilterEngine_t *Engine;
80	77
81	78	/* exported fuctions */
82	79	int check_filter_block(char filter, master_record_t flow_record, int expect);

100	97	printf("Success: Startnode: %i Numblocks: %i Extended: %i Filter: '%s'\n", Engine->StartNode, nblocks(), Engine->Extended, filter);
101	98	} else {
102	99	printf("** FAILED ** Startnode: %i Numblocks: %i Extended: %i Filter: '%s'\n", Engine->StartNode, nblocks(), Engine->Extended, filter);
103		DumpList(Engine);
	100	DumpEngine(Engine);
104	101	printf("Expected: %i, Found: %i\n", expect, ret);
105	102	printf("Record:\n");
106	103	for(i=0; i <= (Offset_MR_LAST >> 3); i++) {

210	207	master_record_t flow_record;
211	208	common_record_t c_record;
212	209	uint64_t *blocks, l;
213		uint32_t size, in[2];
	210	uint32_t in[2];
214	211	time_t now;
215	212	int ret, i;
216	213	value64_t v;

261	258	}
262	259
263	260
264		size = COMMON_RECORD_DATA_SIZE;
265	261	memset((void *)&flow_record, 0, sizeof(master_record_t));
266	262	blocks = (uint64_t *)&flow_record;
267	263

+61

-69

bin/nftree.c less more

0	0	/*
1		* Copyright (c) 2014, Peter Haag
2		* Copyright (c) 2009, Peter Haag
	1	* Copyright (c) 2009-2020, Peter Haag
3	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
4	3	* All rights reserved.
5	4	*

27	26	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
28	27	* POSSIBILITY OF SUCH DAMAGE.
29	28	*
30		* $Author: haag $
31		*
32		* $Id: nftree.c 39 2009-11-25 08:11:15Z haag $
33		*
34		* $LastChangedRevision: 39 $
35		*
36	29	*/
37	30
38	31	#include "config.h"

48	41	#endif
49	42
50	43	#include "rbtree.h"
	44	#include "filter.h"
51	45	#include "nfdump.h"
52	46	#include "nffile.h"
53		#include "nf_common.h"
54	47	#include "ipconv.h"
55	48	#include "nftree.h"
56	49
57		#include "grammar.h"
	50	// #include "grammar.h"
58	51
59	52	/*
60	53	* netflow filter engine

155	148	ClearFilter();
156	149	} // End of InitTree
157	150
	151
158	152	/*
159	153	* Clear Filter
160	154	*/

169	163
170	164	} /* End of ClearFilter */
171	165
172		FilterEngine_data_t CompileFilter(char FilterSyntax) {
173		FilterEngine_data_t *engine;
	166	FilterEngine_t CompileFilter(char FilterSyntax) {
	167	FilterEngine_t *engine;
174	168	int ret;
175	169
176	170	if ( !FilterSyntax )

182	176	exit(255);
183	177	}
184	178
185		if ( !InitSymbols() )
186		exit(255);
187	179	InitTree();
188	180	lex_init(FilterSyntax);
189	181	ret = yyparse();

193	185	lex_cleanup();
194	186	free(IPstack);
195	187
196		engine = malloc(sizeof(FilterEngine_data_t));
	188	engine = malloc(sizeof(FilterEngine_t));
197	189	if ( !engine ) {
198	190	fprintf(stderr, "Memory allocation error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
199	191	exit(255);

208	200	else
209	201	engine->FilterEngine = RunFilter;
210	202
211		return engine;
	203	return (FilterEngine_t *)engine;
212	204
213	205	} // End of GetTree
214	206

378	370	/*
379	371	* Dump Filterlist
380	372	*/
381		void DumpList(FilterEngine_data_t *args) {
382		uint32_t i, j;
	373	void DumpEngine(FilterEngine_t *engine) {
	374	uint32_t i, j;
383	375
384	376	for (i=1; i<NumBlocks; i++ ) {
385		if ( args->filter[i].invert )
	377	if ( engine->filter[i].invert )
386	378	printf("Index: %u, Offset: %u, Mask: %.16llx, Value: %.16llx, Superblock: %u, Numblocks: %u, !OnTrue: %u, !OnFalse: %u Comp: %u Function: %s, Label: %s\n",
387		i, args->filter[i].offset, (unsigned long long)args->filter[i].mask,
388		(unsigned long long)args->filter[i].value, args->filter[i].superblock,
389		args->filter[i].numblocks, args->filter[i].OnTrue, args->filter[i].OnFalse,
390		args->filter[i].comp, args->filter[i].fname, args->filter[i].label ? args->filter[i].label : "<none>");
	379	i, engine->filter[i].offset, (unsigned long long)engine->filter[i].mask,
	380	(unsigned long long)engine->filter[i].value, engine->filter[i].superblock,
	381	engine->filter[i].numblocks, engine->filter[i].OnTrue, engine->filter[i].OnFalse,
	382	engine->filter[i].comp, engine->filter[i].fname, engine->filter[i].label ? engine->filter[i].label : "<none>");
391	383	else
392	384	printf("Index: %u, Offset: %u, Mask: %.16llx, Value: %.16llx, Superblock: %u, Numblocks: %u, OnTrue: %u, OnFalse: %u Comp: %u Function: %s, Label: %s\n",
393		i, args->filter[i].offset, (unsigned long long)args->filter[i].mask,
394		(unsigned long long)args->filter[i].value, args->filter[i].superblock,
395		args->filter[i].numblocks, args->filter[i].OnTrue, args->filter[i].OnFalse,
396		args->filter[i].comp, args->filter[i].fname, args->filter[i].label ? args->filter[i].label : "<none>");
397		if ( args->filter[i].OnTrue > (memblocks * MAXBLOCKS) \|\| args->filter[i].OnFalse > (memblocks * MAXBLOCKS) ) {
	385	i, engine->filter[i].offset, (unsigned long long)engine->filter[i].mask,
	386	(unsigned long long)engine->filter[i].value, engine->filter[i].superblock,
	387	engine->filter[i].numblocks, engine->filter[i].OnTrue, engine->filter[i].OnFalse,
	388	engine->filter[i].comp, engine->filter[i].fname, engine->filter[i].label ? engine->filter[i].label : "<none>");
	389	if ( engine->filter[i].OnTrue > (memblocks * MAXBLOCKS) \|\| engine->filter[i].OnFalse > (memblocks * MAXBLOCKS) ) {
398	390	fprintf(stderr, "Tree pointer out of range for index %u. * ABORT *\n", i);
399	391	exit(255);
400	392	}
401		if ( args->filter[i].data ) {
402		if ( args->filter[i].comp == CMP_IPLIST ) {
	393	if ( engine->filter[i].data ) {
	394	if ( engine->filter[i].comp == CMP_IPLIST ) {
403	395	struct IPListNode *node;
404		RB_FOREACH(node, IPtree, args->filter[i].data) {
	396	RB_FOREACH(node, IPtree, engine->filter[i].data) {
405	397	printf("value: %.16llx %.16llx mask: %.16llx %.16llx\n",
406	398	(unsigned long long)node->ip[0], (unsigned long long)node->ip[1],
407	399	(unsigned long long)node->mask[0], (unsigned long long)node->mask[1]);
408	400	}
409		} else if ( args->filter[i].comp == CMP_ULLIST ) {
	401	} else if ( engine->filter[i].comp == CMP_ULLIST ) {
410	402	struct ULongListNode *node;
411		RB_FOREACH(node, ULongtree, args->filter[i].data) {
	403	RB_FOREACH(node, ULongtree, engine->filter[i].data) {
412	404	printf("%.16llx \n", (unsigned long long)node->value);
413	405	}
414	406	} else
415		printf("Error comp: %i\n", args->filter[i].comp);
	407	printf("Error comp: %i\n", engine->filter[i].comp);
416	408	}
417	409	printf("\tBlocks: ");
418		for ( j=0; j<args->filter[i].numblocks; j++ )
419		printf("%i ", args->filter[i].blocklist[j]);
	410	for ( j=0; j<engine->filter[i].numblocks; j++ )
	411	printf("%i ", engine->filter[i].blocklist[j]);
420	412	printf("\n");
421	413	}
422	414	printf("NumBlocks: %i\n", NumBlocks - 1);

426	418	} /* End of DumpList */
427	419
428	420	/* fast filter engine */
429		int RunFilter(FilterEngine_data_t *args) {
	421	int RunFilter(FilterEngine_t *engine) {
430	422	uint32_t index, offset;
431	423	int evaluate, invert;
432	424
433		args->label = NULL;
434		index = args->StartNode;
	425	engine->label = NULL;
	426	index = engine->StartNode;
435	427	evaluate = 0;
436	428	invert = 0;
437	429	while ( index ) {
438		offset = args->filter[index].offset;
439		invert = args->filter[index].invert;
440		evaluate = ( args->nfrecord[offset] & args->filter[index].mask ) == args->filter[index].value;
441		index = evaluate ? args->filter[index].OnTrue : args->filter[index].OnFalse;
	430	offset = engine->filter[index].offset;
	431	invert = engine->filter[index].invert;
	432	evaluate = ( engine->nfrecord[offset] & engine->filter[index].mask ) == engine->filter[index].value;
	433	index = evaluate ? engine->filter[index].OnTrue : engine->filter[index].OnFalse;
442	434	}
443	435	return invert ? !evaluate : evaluate;
444	436
445	437	} /* End of RunFilter */
446	438
447	439	/* extended filter engine */
448		int RunExtendedFilter(FilterEngine_data_t *args) {
	440	int RunExtendedFilter(FilterEngine_t *engine) {
449	441	uint32_t index, offset;
450	442	uint64_t comp_value[2];
451	443	int evaluate, invert;
452	444
453		args->label = NULL;
454		index = args->StartNode;
	445	engine->label = NULL;
	446	index = engine->StartNode;
455	447	evaluate = 0;
456	448	invert = 0;
457	449	while ( index ) {
458		offset = args->filter[index].offset;
459		invert = args->filter[index].invert;
460
461		comp_value[0] = args->nfrecord[offset] & args->filter[index].mask;
462		comp_value[1] = args->filter[index].value;
463
464		if (args->filter[index].function != NULL)
465		args->filter[index].function(args->nfrecord, comp_value);
466
467		switch (args->filter[index].comp) {
	450	offset = engine->filter[index].offset;
	451	invert = engine->filter[index].invert;
	452
	453	comp_value[0] = engine->nfrecord[offset] & engine->filter[index].mask;
	454	comp_value[1] = engine->filter[index].value;
	455
	456	if (engine->filter[index].function != NULL)
	457	engine->filter[index].function(engine->nfrecord, comp_value);
	458
	459	switch (engine->filter[index].comp) {
468	460	case CMP_EQ:
469	461	evaluate = comp_value[0] == comp_value[1];
470	462	break;

475	467	evaluate = comp_value[0] < comp_value[1];
476	468	break;
477	469	case CMP_IDENT:
478		evaluate = strncmp(CurrentIdent, args->IdentList[comp_value[1]], IDENTLEN) == 0 ;
	470	evaluate = strncmp(CurrentIdent, engine->IdentList[comp_value[1]], IDENTLEN) == 0 ;
479	471	break;
480	472	case CMP_FLAGS:
481	473	if ( invert )

485	477	break;
486	478	case CMP_IPLIST: {
487	479	struct IPListNode find;
488		find.ip[0] = args->nfrecord[offset];
489		find.ip[1] = args->nfrecord[offset+1];
	480	find.ip[0] = engine->nfrecord[offset];
	481	find.ip[1] = engine->nfrecord[offset+1];
490	482	find.mask[0] = 0xffffffffffffffffLL;
491	483	find.mask[1] = 0xffffffffffffffffLL;
492		evaluate = RB_FIND(IPtree, args->filter[index].data, &find) != NULL; }
	484	evaluate = RB_FIND(IPtree, engine->filter[index].data, &find) != NULL; }
493	485	break;
494	486	case CMP_ULLIST: {
495	487	struct ULongListNode find;
496	488	find.value = comp_value[0];
497		evaluate = RB_FIND(ULongtree, args->filter[index].data, &find ) != NULL; }
	489	evaluate = RB_FIND(ULongtree, engine->filter[index].data, &find ) != NULL; }
498	490	break;
499	491	}
500	492

508	500	*/
509	501	if ( evaluate ) {
510	502	// if filter expression has a label assigned, copy that
511		if ( args->filter[index].label ) {
512		args->label = args->filter[index].label;
	503	if ( engine->filter[index].label ) {
	504	engine->label = engine->filter[index].label;
513	505	}
514		index = args->filter[index].OnTrue;
	506	index = engine->filter[index].OnTrue;
515	507	} else {
516	508	// filter expression does not match - clear previous label if abailable
517		if ( args->label )
518		args->label = NULL;
519		index = args->filter[index].OnFalse;
520		}
521		// index = evaluate ? args->filter[index].OnTrue : args->filter[index].OnFalse;
	509	if ( engine->label )
	510	engine->label = NULL;
	511	index = engine->filter[index].OnFalse;
	512	}
	513	// index = evaluate ? engine->filter[index].OnTrue : engine->filter[index].OnFalse;
522	514	}
523	515	return invert ? !evaluate : evaluate;
524	516

+14

-106

bin/nftree.h less more

0	0	/*
1		* Copyright (c) 2017, Peter Haag
2		* Copyright (c) 2014, Peter Haag
3		* Copyright (c) 2009, Peter Haag
	1	* Copyright (c) 2009-2019, Peter Haag
4	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
5	3	* All rights reserved.
6	4	*

40	38	#include <stdint.h>
41	39	#endif
42	40
43		#include "rbtree.h"
44
45	41	/*
46	42	* type definitions for nf tree
47	43	*/

76	72	uint64_t *nfrecord;
77	73	char *label;
78	74	int (FilterEngine)(struct FilterEngine_data_s );
79		} FilterEngine_data_t;
80
81
82		/*
83		* Definitions
84		*/
85		enum { CMP_EQ = 0, CMP_GT, CMP_LT, CMP_IDENT, CMP_FLAGS, CMP_IPLIST, CMP_ULLIST };
86
87		/*
88		* filter functions:
89		* For some filter functions, netflow records need to be processed first in order to filter them
90		* This involves all data not directly available in the netflow record, such as packets per second etc.
91		* Filter speed is a bit slower due to extra netflow processsing
92		* The sequence of the enum values must correspond with the entries in the flow_procs array
93		*/
94
95		enum { FUNC_NONE = 0, /* no function - just plain filtering - just to be complete here */
96		FUNC_PPS, /* function code for pps ( packet per second ) filter function */
97		FUNC_BPS, /* function code for bps ( bits per second ) filter function */
98		FUNC_BPP, /* function code for bpp ( bytes per packet ) filter function */
99		FUNC_DURATION, /* function code for duration ( in miliseconds ) filter function */
100		FUNC_MPLS_EOS, /* function code for matching End of MPLS Stack label */
101		FUNC_MPLS_ANY, /* function code for matching any MPLS label */
102		FUNC_PBLOCK /* function code for matching ports against pblock start */
103		};
104
105		/*
106		* Tree type defs
107		*/
108
109		/* Definition of the IP list node */
110		struct IPListNode {
111		RB_ENTRY(IPListNode) entry;
112		uint64_t ip[2];
113		uint64_t mask[2];
114		};
115
116		/* Definition of the port/AS list node */
117		struct ULongListNode {
118		RB_ENTRY(ULongListNode) entry;
119		uint64_t value;
120		};
121
	75	} FilterEngine_t;
122	76
123	77	/*
124	78	* Filter Engine Functions
125	79	*/
126		int RunFilter(FilterEngine_data_t *args);
127		int RunExtendedFilter(FilterEngine_data_t *args);
128		/*
129		* For testing purpose only
130		*/
	80	void InitTree(void);
	81
	82	FilterEngine_t CompileFilter(char FilterSyntax);
	83
	84	int RunFilter(FilterEngine_t *engine);
	85
	86	int RunExtendedFilter(FilterEngine_t *engine);
	87
	88	void ClearFilter(void);
	89
	90	void DumpEngine(FilterEngine_t *engine);
	91
131	92	int nblocks(void);
132	93
133		/*
134		* Initialize globals
135		*/
136		void InitTree(void);
137
138		/*
139		* Returns the current Filter Tree
140		*/
141		FilterEngine_data_t CompileFilter(char FilterSyntax);
142
143		/*
144		* Clear Filter
145		*/
146		void ClearFilter(void);
147
148		/*
149		* Returns next free slot in blocklist
150		*/
151		uint32_t NewBlock(uint32_t offset, uint64_t mask, uint64_t value, uint16_t comp, uint32_t function, void *data);
152
153		/*
154		* Connects the to blocks b1 and b2 ( AND ) and returns index of superblock
155		*/
156		uint32_t Connect_AND(uint32_t b1, uint32_t b2);
157
158		/*
159		* Connects the to blocks b1 and b2 ( OR ) and returns index of superblock
160		*/
161		uint32_t Connect_OR(uint32_t b1, uint32_t b2);
162
163		/*
164		* Inverts OnTrue and OnFalse
165		*/
166		uint32_t Invert(uint32_t a );
167
168		/*
169		* Add label to filter index
170		*/
171		void AddLabel(uint32_t index, char *label);
172
173		/*
174		* Add Ident to Identlist
175		*/
176		uint32_t AddIdent(char *Ident);
177
178		/*
179		* Dump Filterlist
180		*/
181		void DumpList(FilterEngine_data_t *args);
182
183		/*
184		* Prints info while filer is running
185		*/
186	94	int RunDebugFilter(uint32_t *block);
187	95
188	96	#endif //_NFTREE_H

-9

bin/nfx.c less more

0	0	/*
1		* Copyright (c) 2009-2019, Peter Haag
	1	* Copyright (c) 2009-2020, Peter Haag
2	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
3	3	* All rights reserved.
4	4	*

44	44	#include <stdint.h>
45	45	#endif
46	46
47		#ifndef DEVEL
48		# define dbg_printf(...) /* printf(__VA_ARGS__) */
49		#else
50		# define dbg_printf(...) printf(__VA_ARGS__)
51		#endif
52
53		#include "nf_common.h"
	47	#include "util.h"
	48	#include "nfdump.h"
54	49	#include "nffile.h"
55		#include "util.h"
56	50	#include "nfx.h"
57	51
58	52	/* global vars */

260	254	}
261	255	l->ref_count = 0;
262	256	l->next = NULL;
	257	l->exportMap = NULL;
263	258	memset((void *)&l->master_record, 0, sizeof(master_record_t));
264	259
265	260	l->map = (extension_map_t *)malloc((ssize_t)map->size);

+899

-2

bin/nfx.h less more

46	46	#define MAX_EXTENSION_MAPS 65536
47	47	#define EXTENSION_MAP_MASK (MAX_EXTENSION_MAPS-1)
48	48
	49	#ifdef NSEL
	50	// Defaults for NSEL
	51	#define DefaultExtensions "1,8,26,27,28,29,30,31"
	52	#else
	53	// Collector netflow defaults
	54	#define DefaultExtensions "1,2"
	55	#endif
	56
	57
	58	#define NEEDS_EXTENSION_LIST 1
	59	#define NO_EXTENSION_LIST 0
	60
	61	// new extended Common Record as intermediate solution to overcome 255 exporters
	62	// requires moderate changes till 1.7
	63	#define CommonRecordType 10
	64
	65	/*
	66	* All records are 32bit aligned and layouted in a 64bit array. The numbers placed in () refer to the netflow v9 type id.
	67	*
	68	* Record type 1
	69	* =============
	70	* The record type 1 describes a netflow data record incl. all optional extensions for this record.
	71	* A netflow data record requires at least the first 3 extensions 1..3. All other extensions are optional
	72	* and described in the extensiion map. The common record contains a reference to the extension map which
	73	* applies for this record.
	74	*
	75	* flags:
	76	* bit 0: 0: IPv4 1: IPv6
	77	* bit 1: 0: 32bit dPkts 1: 64bit dPkts
	78	* bit 2: 0: 32bit dOctets 1: 64bit dOctets
	79	* bit 3: 0: IPv4 next hop 1: IPv6 next hop
	80	* bit 4: 0: IPv4 BGP next hop 1: BGP IPv6 next hop
	81	* bit 5: 0: IPv4 exporter IP 1: IPv6 exporter IP
	82	* bit 6: 0: flow 1: event
	83	* bit 7: 0: unsampled 1: sampled flow - sampling applied
	84	*
	85	* Required extensions: 1,2,3
	86	* ------------------------------
	87	* A netflow record consists at least of a common record ( extension 0 ) and 3 required extension:
	88	*
	89	* Extension 1: IPv4 or IPv4 src and dst addresses Flags bit 0: 0: IPv4, 1: IPv6
	90	* Extension 2: 32 or 64 bit packet counter Flags bit 1: 0: 32bit, 1: 64bit
	91	* Extension 3: 32 or 64 bit byte counter Flags bit 2: 0: 32bit, 1: 64bit
	92	*
	93	* Commmon record - extension 0
	94	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	95	* \| - \| 0 \| 1 \| 2 \| 3 \| 4 \| 5 \| 6 \| 7 \|
	96	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	97	* \| 0 \| record type == 1 \| size \| flags \| tag \| ext. map \|
	98	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	99	* \| 1 \| msec_first \| msec_last \| first (22) \|
	100	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	101	* \| 2 \| last (21) \|fwd_status(89)\| tcpflags (6) \| proto (4) \| src tos (5) \|
	102	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	103	* \| 3 \| srcport (7) \| dstport(11)/ICMP (32) \|
	104	* +----+--------------+--------------+--------------+--------------+
	105	*
	106	* Commmon record - extension 0 - Type 10
	107	* required for larger exporter ID reference
	108	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	109	* \| - \| 0 \| 1 \| 2 \| 3 \| 4 \| 5 \| 6 \| 7 \|
	110	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	111	* \| 0 \| record type == 10 \| size \| flags \| ext. map \|
	112	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	113	* \| 1 \| msec_first \| msec_last \| first (22) \|
	114	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	115	* \| 2 \| last (21) \|fwd_status(89)\| tcpflags (6) \| proto (4) \| src tos (5) \|
	116	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	117	* \| 3 \| srcport (7) \| dstport(11)/ICMP (32) \| exporter ID \| reserved icmp type/code \|
	118	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	119
	120	*
	121	*/
	122
	123	#define COMMON_BLOCK_ID 0
	124
	125
	126	typedef struct common_record_s {
	127	// record head
	128	uint16_t type;
	129	uint16_t size;
	130
	131	// record meta data
	132	uint16_t flags;
	133	#define FLAG_IPV6_ADDR 1
	134	#define FLAG_PKG_64 2
	135	#define FLAG_BYTES_64 4
	136	#define FLAG_IPV6_NH 8
	137	#define FLAG_IPV6_NHB 16
	138	#define FLAG_IPV6_EXP 32
	139	#define FLAG_EVENT 64
	140	#define FLAG_SAMPLED 128
	141
	142	uint16_t ext_map;
	143
	144	// netflow common record
	145	uint16_t msec_first;
	146	uint16_t msec_last;
	147	uint32_t first;
	148	uint32_t last;
	149
	150	uint8_t fwd_status;
	151	uint8_t tcp_flags;
	152	uint8_t prot;
	153	uint8_t tos;
	154	uint16_t srcport;
	155	uint16_t dstport;
	156
	157	uint16_t exporter_sysid;
	158	uint16_t reserved;
	159
	160	// link to extensions
	161	uint32_t data[1];
	162	} common_record_t;
	163
	164	#define COMMON_RECORD_DATA_SIZE (sizeof(common_record_t) - sizeof(uint32_t) )
	165
	166	#define COMMON_BLOCK 0
	167
	168	/*
	169	* Required extensions:
	170	* --------------------
	171	* Extension 1:
	172	* IPv4/v6 address type
	173	* IP version: IPv4
	174	* \|
	175	* Flags: xxxx xxx0
	176	* IPv4:
	177	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	178	* \| 0 \| srcip (8) \| dstip (12) \|
	179	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	180	*
	181	* IPv6:
	182	* IP version: IPv6
	183	* \|
	184	* Flags: xxxx xxx1
	185	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	186	* \| 0 \| srcip (27) \|
	187	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	188	* \| 1 \| srcip (27) \|
	189	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	190	* \| 2 \| dstip (28) \|
	191	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	192	* \| 3 \| dstip (28) \|
	193	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	194	*
	195	*/
	196
	197	#define EX_IPv4v6 1
	198
	199	typedef struct ipv4_block_s {
	200	uint32_t srcaddr;
	201	uint32_t dstaddr;
	202	uint8_t data[4]; // .. more data below
	203	} ipv4_block_t;
	204
	205	typedef struct ipv6_block_s {
	206	uint64_t srcaddr[2];
	207	uint64_t dstaddr[2];
	208	uint8_t data[4]; // .. more data below
	209	} ipv6_block_t;
	210
	211
	212
	213	/*
	214	* Extension 2:
	215	* In packet counter size
	216	*
	217	* In packet counter size 4byte
	218	* \|
	219	* Flags: xxxx xx0x
	220	* +---++--------------+--------------+--------------+--------------+
	221	* \| 0 \| in pkts (2) \|
	222	* +---++--------------+--------------+--------------+--------------+
	223	*
	224	* In packet counter size 8byte
	225	* \|
	226	* Flags: xxxx xx1x
	227	* +---++--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	228	* \| 0 \| in pkts (2) \|
	229	* +---++--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	230	*
	231	*/
	232
	233	#define EX_PACKET_4_8 2
	234
	235	typedef struct value32_s {
	236	uint32_t val;
	237	uint8_t data[4]; // .. more data below
	238	} value32_t;
	239
	240	typedef struct value64_s {
	241	union val_s {
	242	uint64_t val64;
	243	uint32_t val32[2];
	244	} val;
	245	uint8_t data[4]; // .. more data below
	246	} value64_t;
	247
	248
	249	/* Extension 3:
	250	* in byte counter size
	251	* In byte counter size 4byte
	252	* \|
	253	* Flags: xxxx x0xx
	254	*
	255	* +---++--------------+--------------+--------------+--------------+
	256	* \| 0 \| in bytes (1) \|
	257	* +---++--------------+--------------+--------------+--------------+
	258	*
	259	* In byte counter size 8byte
	260	* \|
	261	* Flags: xxxx x1xx
	262	* +---++--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	263	* \| 0 \| in bytes (1) \|
	264	* +---++--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	265	*/
	266
	267	#define EX_BYTE_4_8 3
	268
	269	/*
	270	*
	271	* Optional extension:
	272	* ===================
	273	*
	274	* Interface record
	275	* ----------------
	276	* Interface records are optional and accepted as either 2 or 4 bytes numbers
	277	* Extension 4:
	278	* +---++--------------+--------------+--------------+--------------+
	279	* \| 0 \| input (10) \| output (14) \|
	280	* +---++--------------+--------------+--------------+--------------+
	281	*/
	282	#define EX_IO_SNMP_2 4
	283	typedef struct tpl_ext_4_s {
	284	uint16_t input;
	285	uint16_t output;
	286	uint8_t data[4]; // points to further data
	287	} tpl_ext_4_t;
	288
	289	/*
	290	* Extension 5:
	291	* +---++--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	292	* \| 0 \| input (10) \| output (14) \|
	293	* +---++--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	294	* Extension 4 and 5 are mutually exclusive in the extension map
	295	*/
	296	#define EX_IO_SNMP_4 5
	297	typedef struct tpl_ext_5_s {
	298	uint32_t input;
	299	uint32_t output;
	300	uint8_t data[4]; // points to further data
	301	} tpl_ext_5_t;
	302
	303
	304	/*
	305	* AS record
	306	* ---------
	307	* AS records are optional and accepted as either 2 or 4 bytes numbers
	308	* Extension 6:
	309	* +---++--------------+--------------+--------------+--------------+
	310	* \| 0 \| src as (16) \| dst as (17) \|
	311	* +---++--------------+--------------+--------------+--------------+
	312	*/
	313	#define EX_AS_2 6
	314	typedef struct tpl_ext_6_s {
	315	uint16_t src_as;
	316	uint16_t dst_as;
	317	uint8_t data[4]; // points to further data
	318	} tpl_ext_6_t;
	319
	320	/*
	321	* Extension 7:
	322	* +---++--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	323	* \| 0 \| src as (16) \| dst as (17) \|
	324	* +---++--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	325	* Extension 6 and 7 are mutually exclusive in the extension map
	326	*/
	327	#define EX_AS_4 7
	328	typedef struct tpl_ext_7_s {
	329	uint32_t src_as;
	330	uint32_t dst_as;
	331	uint8_t data[4]; // points to further data
	332	} tpl_ext_7_t;
	333
	334
	335	/*
	336	* Multiple fields record
	337	* ----------------------
	338	* These 4 different fields are grouped together in a 32bit value.
	339	* Extension 8:
	340	* +---++--------------+--------------+--------------+--------------+
	341	* \| 3 \| dst tos(55) \| dir(61) \| srcmask(9,29)\|dstmask(13,30)\|
	342	* +---++--------------+--------------+--------------+--------------+
	343	*/
	344	#define EX_MULIPLE 8
	345	typedef struct tpl_ext_8_s {
	346	union {
	347	struct {
	348	uint8_t dst_tos;
	349	uint8_t dir;
	350	uint8_t src_mask;
	351	uint8_t dst_mask;
	352	};
	353	uint32_t any;
	354	};
	355	uint8_t data[4]; // points to further data
	356	} tpl_ext_8_t;
	357
	358	/*
	359	* IP next hop
	360	* -------------
	361	* IPv4:
	362	* Extension 9:
	363	* IP version: IPv6
	364	* \|
	365	* Flags: xxxx 0xxx
	366	* +----+--------------+--------------+--------------+--------------+
	367	* \| 0 \| next hop ip (15) \|
	368	* +----+--------------+--------------+--------------+--------------+
	369	*/
	370	#define EX_NEXT_HOP_v4 9
	371	typedef struct tpl_ext_9_s {
	372	uint32_t nexthop;
	373	uint8_t data[4]; // points to further data
	374	} tpl_ext_9_t;
	375
	376	/*
	377	* IPv6:
	378	* Extension 10:
	379	* IP version: IPv6
	380	* \|
	381	* Flags: xxxx 1xxx
	382	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	383	* \| 0 \| next hop ip (62) \|
	384	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	385	* \| 1 \| next hop ip (62) \|
	386	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	387	* Extension 9 and 10 are mutually exclusive in the extension map
	388	*/
	389	#define EX_NEXT_HOP_v6 10
	390	typedef struct tpl_ext_10_s {
	391	uint64_t nexthop[2];
	392	uint8_t data[4]; // points to further data
	393	} tpl_ext_10_t;
	394
	395
	396	/*
	397	* BGP next hop IP
	398	* ------------------
	399	* IPv4:
	400	* Extension 11:
	401	* IP version: IPv6
	402	* \|
	403	* Flags: xxx0 xxxx
	404	* +----+--------------+--------------+--------------+--------------+
	405	* \| 0 \| bgp next ip (18) \|
	406	* +----+--------------+--------------+--------------+--------------+
	407	*/
	408	#define EX_NEXT_HOP_BGP_v4 11
	409	typedef struct tpl_ext_11_s {
	410	uint32_t bgp_nexthop;
	411	uint8_t data[4]; // points to further data
	412	} tpl_ext_11_t;
	413
	414	/*
	415	* IPv6:
	416	* Extension 12:
	417	* IP version: IPv6
	418	* \|
	419	* Flags: xxx1 xxxx
	420	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	421	* \| 0 \| bgp next ip (63) \|
	422	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	423	* \| 1 \| bgp next ip (63) \|
	424	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	425	*/
	426	#define EX_NEXT_HOP_BGP_v6 12
	427	typedef struct tpl_ext_12_s {
	428	uint64_t bgp_nexthop[2];
	429	uint8_t data[4]; // points to further data
	430	} tpl_ext_12_t;
	431
	432
	433	/*
	434	* VLAN record
	435	* -----------
	436	* Extension 13:
	437	* +----+--------------+--------------+--------------+--------------+
	438	* \| 0 \| src vlan(58) \| dst vlan (59) \|
	439	* +----+--------------+--------------+--------------+--------------+
	440	*/
	441	#define EX_VLAN 13
	442	typedef struct tpl_ext_13_s {
	443	uint16_t src_vlan;
	444	uint16_t dst_vlan;
	445	uint8_t data[4]; // points to further data
	446	} tpl_ext_13_t;
	447
	448
	449	/*
	450	* Out packet counter size
	451	* ------------------------
	452	* 4 byte
	453	* Extension 14:
	454	* +----+--------------+--------------+--------------+--------------+
	455	* \| 0 \| out pkts (24) \|
	456	* +----+--------------+--------------+--------------+--------------+
	457	*/
	458	#define EX_OUT_PKG_4 14
	459	typedef struct tpl_ext_14_s {
	460	uint32_t out_pkts;
	461	uint8_t data[4]; // points to further data
	462	} tpl_ext_14_t;
	463
	464	/*
	465	* 4 byte
	466	* Extension 15:
	467	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	468	* \| 0 \| out pkts (24) \|
	469	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	470	* Extension 14 and 15 are mutually exclusive in the extension map
	471	*/
	472	#define EX_OUT_PKG_8 15
	473	typedef struct tpl_ext_15_s {
	474	union {
	475	uint64_t out_pkts;
	476	uint32_t v[2]; // for strict alignment use 2x32bits
	477	};
	478	uint8_t data[4]; // points to further data
	479	} tpl_ext_15_t;
	480
	481
	482	/*
	483	* Out byte counter size
	484	* ---------------------
	485	* 4 byte
	486	* Extension 16:
	487	* +----+--------------+--------------+--------------+--------------+
	488	* \| 0 \| out bytes (23) \|
	489	* +----+--------------+--------------+--------------+--------------+
	490	*/
	491	#define EX_OUT_BYTES_4 16
	492	typedef struct tpl_ext_16_s {
	493	uint32_t out_bytes;
	494	uint8_t data[4]; // points to further data
	495	} tpl_ext_16_t;
	496
	497
	498	/* 8 byte
	499	* Extension 17:
	500	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	501	* \| 0 \| out bytes (23) \|
	502	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	503	* Extension 16 and 17 are mutually exclusive in the extension map
	504	*/
	505	#define EX_OUT_BYTES_8 17
	506	typedef struct tpl_ext_17_s {
	507	union {
	508	uint64_t out_bytes;
	509	uint32_t v[2]; // potential 32bit alignment
	510	};
	511	uint8_t data[4]; // points to further data
	512	} tpl_ext_17_t;
	513
	514	/*
	515	* Aggr flows
	516	* ----------
	517	* 4 byte
	518	* Extension 18:
	519	* +----+--------------+--------------+--------------+--------------+
	520	* \| 0 \| aggr flows (3) \|
	521	* +----+--------------+--------------+--------------+--------------+
	522	*/
	523	#define EX_AGGR_FLOWS_4 18
	524	typedef struct tpl_ext_18_s {
	525	uint32_t aggr_flows;
	526	uint8_t data[4]; // points to further data
	527	} tpl_ext_18_t;
	528
	529
	530	/* 8 byte
	531	* Extension 19:
	532	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	533	* \| 0 \| aggr flows (3) \|
	534	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	535	* Extension 18 and 19 are mutually exclusive in the extension map
	536	*/
	537	#define EX_AGGR_FLOWS_8 19
	538	typedef struct tpl_ext_19_s {
	539	union {
	540	uint64_t aggr_flows;
	541	uint32_t v[2]; // 32bit alignment
	542	};
	543	uint8_t data[4]; // points to further data
	544	} tpl_ext_19_t;
	545
	546	/* 16 byte
	547	* Extension 20:
	548	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	549	* \| 0 \| 0 \| in src mac (56) \|
	550	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	551	* \| 1 \| 0 \| out dst mac (57) \|
	552	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	553	*/
	554	#define EX_MAC_1 20
	555	typedef struct tpl_ext_20_s {
	556	union {
	557	uint64_t in_src_mac;
	558	uint32_t v1[2];
	559	};
	560	union {
	561	uint64_t out_dst_mac;
	562	uint32_t v2[2];
	563	};
	564	uint8_t data[4]; // points to further data
	565	} tpl_ext_20_t;
	566
	567	/* 16 byte
	568	* Extension 21:
	569	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	570	* \| 0 \| 0 \| in dst mac (80) \|
	571	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	572	* \| 1 \| 0 \| out src mac (81) \|
	573	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	574	*/
	575	#define EX_MAC_2 21
	576	typedef struct tpl_ext_21_s {
	577	union {
	578	uint64_t in_dst_mac;
	579	uint32_t v1[2];
	580	};
	581	union {
	582	uint64_t out_src_mac;
	583	uint32_t v2[2];
	584	};
	585	uint8_t data[4]; // points to further data
	586	} tpl_ext_21_t;
	587
	588	/* 40 byte
	589	* Extension 22:
	590	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	591	* \| 0 \| 0 \| MPLS_LABEL_2 (71) \| 0 \| MPLS_LABEL_1 (70) \|
	592	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	593	* \| 1 \| 0 \| MPLS_LABEL_4 (73) \| 0 \| MPLS_LABEL_3 (72) \|
	594	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	595	* \| 2 \| 0 \| MPLS_LABEL_6 (75) \| 0 \| MPLS_LABEL_5 (74) \|
	596	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	597	* \| 3 \| 0 \| MPLS_LABEL_8 (77) \| 0 \| MPLS_LABEL_7 (76) \|
	598	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	599	* \| 4 \| 0 \| MPLS_LABEL_10 (79) \| 0 \| MPLS_LABEL_9 (78) \|
	600	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	601	*/
	602	#define EX_MPLS 22
	603	typedef struct tpl_ext_22_s {
	604	uint32_t mpls_label[10];
	605	uint8_t data[4]; // points to further data
	606	} tpl_ext_22_t;
	607
	608	/*
	609	* Sending router IP
	610	* -----------------
	611	* IPv4:
	612	* Extension 23:
	613	* IP version: IPv6
	614	* \|
	615	* Flags: xx0x xxxx
	616	* +----+--------------+--------------+--------------+--------------+
	617	* \| 0 \| router ipv4 () \|
	618	* +----+--------------+--------------+--------------+--------------+
	619	*/
	620	#define EX_ROUTER_IP_v4 23
	621	typedef struct tpl_ext_23_s {
	622	uint32_t router_ip;
	623	uint8_t data[4]; // points to further data
	624	} tpl_ext_23_t;
	625
	626	/*
	627	* IPv6:
	628	* Extension 24:
	629	* IP version: IPv6
	630	* \|
	631	* Flags: xx1x xxxx
	632	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	633	* \| 0 \| router ip v6 () \|
	634	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	635	* \| 1 \| router ip v6 () \|
	636	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	637	* Extension 23 and 24 are mutually exclusive in the extension map
	638	*/
	639	#define EX_ROUTER_IP_v6 24
	640	typedef struct tpl_ext_24_s {
	641	uint64_t router_ip[2];
	642	uint8_t data[4]; // points to further data
	643	} tpl_ext_24_t;
	644
	645	/*
	646	* router source ID
	647	* ----------------
	648	* For v5 netflow, it's engine type/engine ID
	649	* for v9 it's the source_id
	650	* Extension 25:
	651	* +----+--------------+--------------+--------------+--------------+
	652	* \| 0 \| fill \|engine tpe(38)\|engine ID(39) \|
	653	* +----+--------------+--------------+--------------+--------------+
	654	*/
	655	#define EX_ROUTER_ID 25
	656	typedef struct tpl_ext_25_s {
	657	uint16_t fill;
	658	uint8_t engine_type;
	659	uint8_t engine_id;
	660	uint8_t data[4]; // points to further data
	661	} tpl_ext_25_t;
	662
	663	/*
	664	* BGP prev/next adjacent AS
	665	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	666	* \| 0 \| bgpNextAdjacentAsNumber(128) \| bgpPrevAdjacentAsNumber(129) \|
	667	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	668	*/
	669	#define EX_BGPADJ 26
	670	typedef struct tpl_ext_26_s {
	671	uint32_t bgpNextAdjacentAS;
	672	uint32_t bgpPrevAdjacentAS;
	673	uint8_t data[4]; // points to further data
	674	} tpl_ext_26_t;
	675
	676	/*
	677	* time flow received in ms
	678	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	679	* \| 0 \| T received() \|
	680	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	681	*/
	682	#define EX_RECEIVED 27
	683	typedef struct tpl_ext_27_s {
	684	union {
	685	uint64_t received;
	686	uint32_t v[2];
	687	};
	688	uint8_t data[4]; // points to further data
	689	} tpl_ext_27_t;
	690
	691
	692
	693	#define EX_RESERVED_1 28
	694	#define EX_RESERVED_2 29
	695	#define EX_RESERVED_3 30
	696	#define EX_RESERVED_4 31
	697	#define EX_RESERVED_5 32
	698	#define EX_RESERVED_6 33
	699	#define EX_RESERVED_7 34
	700	#define EX_RESERVED_8 35
	701	#define EX_RESERVED_9 36
	702
	703	/*
	704	* NSEL Common block
	705	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	706	* \| 0 \| NF_F_EVENT_TIME_MSEC(323) \|
	707	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	708	* \| 1 \| NF_F_CONN_ID(148) \|i type(176/8) \|i code(177/9) \|EVT(40005/233)\| fill \|
	709	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	710	* \| 2 \| NF_F_FW_EXT_EVENT(33002) \| FW_CTS_SRC_SGT(34000) \|
	711	* +----+--------------+--------------+--------------+--------------+
	712	* * EVT: NF_F_FW_EVENT
	713	* * XEVT: NF_F_FW_EXT_EVENT
	714	*/
	715	#define EX_NSEL_COMMON 37
	716	typedef struct tpl_ext_37_s {
	717	union {
	718	uint64_t event_time;
	719	uint32_t v[2];
	720	};
	721	uint32_t conn_id;
	722	union {
	723	struct {
	724	#ifdef WORDS_BIGENDIAN
	725	uint8_t icmp_type;
	726	uint8_t icmp_code;
	727	#else
	728	uint8_t icmp_code;
	729	uint8_t icmp_type;
	730	#endif
	731	};
	732	uint16_t nsel_icmp;
	733	};
	734	uint8_t fw_event;
	735	uint8_t fill;
	736	uint16_t fw_xevent;
	737	uint16_t sec_group_tag;
	738	uint8_t data[4]; // points to further data
	739	} tpl_ext_37_t;
	740
	741	/*
	742	* NSEL/NEL xlate ports
	743	* +----+--------------+--------------+--------------+--------------+
	744	* \| 0 \| NF_F_XLATE_SRC_PORT(227) \| NF_F_XLATE_DST_PORT(228) \|
	745	* +----+--------------+--------------+--------------+--------------+
	746	* ASA 8.4 compatibility mapping 40003 -> 227
	747	* ASA 8.4 compatibility mapping 40004 -> 228
	748	*/
	749	#define EX_NSEL_XLATE_PORTS 38
	750	typedef struct tpl_ext_38_s {
	751	uint16_t xlate_src_port;
	752	uint16_t xlate_dst_port;
	753	uint8_t data[4]; // points to further data
	754	} tpl_ext_38_t;
	755
	756	/*
	757	* NSEL xlate v4 IP address
	758	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	759	* \| 0 \| NF_F_XLATE_SRC_ADDR_IPV4(225) \| NF_F_XLATE_DST_ADDR_IPV4(226) \|
	760	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	761	* ASA 8.4 compatibility mapping 40001 -> 225
	762	* ASA 8.4 compatibility mapping 40002 -> 226
	763	*/
	764	#define EX_NSEL_XLATE_IP_v4 39
	765	typedef struct tpl_ext_39_s {
	766	uint32_t xlate_src_ip;
	767	uint32_t xlate_dst_ip;
	768	uint8_t data[4]; // points to further data
	769	} tpl_ext_39_t;
	770
	771	/*
	772	* NSEL xlate v6 IP address - not yet implemented by CISCO
	773	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	774	* \| 0 \| xlate src ip (281) \|
	775	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	776	* \| 1 \| xlate src ip (281) \|
	777	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	778	* \| 2 \| xlate dst ip (282) \|
	779	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	780	* \| 3 \| xlate dst ip (282) \|
	781	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	782	*/
	783	#define EX_NSEL_XLATE_IP_v6 40
	784	typedef struct tpl_ext_40_s {
	785	uint64_t xlate_src_ip[2];
	786	uint64_t xlate_dst_ip[2];
	787	uint8_t data[4]; // points to further data
	788	} tpl_ext_40_t;
	789
	790
	791	/*
	792	* NSEL ACL ingress/egress acl ID
	793	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	794	* \| 0 \| NF_F_INGRESS_ACL_ID(33000) \|
	795	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	796	* \| 1 \| NF_F_INGRESS_ACL_ID(33000) \| NF_F_EGRESS_ACL_ID(33001) \|
	797	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	798	* \| 2 \| NF_F_EGRESS_ACL_ID(33001) \|
	799	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	800	*/
	801	#define EX_NSEL_ACL 41
	802	typedef struct tpl_ext_41_s {
	803	uint32_t ingress_acl_id[3];
	804	uint32_t egress_acl_id[3];
	805	uint8_t data[4]; // points to further data
	806	} tpl_ext_41_t;
	807
	808	/*
	809	* NSEL ACL username
	810	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	811	* \| 0 \| NF_F_USERNAME(40000) \|
	812	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	813	* \| 1 \| \|
	814	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	815	* \| 2 \| \|
	816	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	817	*/
	818	#define EX_NSEL_USER 42
	819	typedef struct tpl_ext_42_s {
	820	char username[24];
	821	uint8_t data[4]; // points to further data
	822	} tpl_ext_42_t;
	823
	824	/*
	825	* NSEL ACL username max
	826	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	827	* \| 0 \| NF_F_USERNAME(40000) \|
	828	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	829	* \| .. \| \|
	830	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	831	* \| 8 \| \|
	832	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	833	*/
	834	#define EX_NSEL_USER_MAX 43
	835	typedef struct tpl_ext_43_s {
	836	char username[72];
	837	uint8_t data[4]; // points to further data
	838	} tpl_ext_43_t;
	839
	840
	841	#define EX_NSEL_RESERVED 44
	842
	843	/*
	844	* latency extensions, used by nprobe and nfpcapd
	845	*/
	846
	847	/*
	848	* latency extension
	849	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	850	* \| 0 \| client_nw_delay_usec (57554/57554) \|
	851	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	852	* \| 1 \| server_nw_delay_usec (57556/57557) \|
	853	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	854	* \| 2 \| appl_latency_usec (57558/57559) \|
	855	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	856	*/
	857	#define EX_LATENCY 45
	858	typedef struct tpl_ext_latency_s {
	859	uint64_t client_nw_delay_usec;
	860	uint64_t server_nw_delay_usec;
	861	uint64_t appl_latency_usec;
	862	uint8_t data[4]; // points to further data
	863	} tpl_ext_latency_t;
	864
	865	/*
	866	* NEL xlate ports
	867	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	868	* \| 0 \|NAT_EVENT(230)\| flags \| fill \| NF_N_EGRESS_VRFID(235) \|
	869	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	870	* \| 1 \| NF_N_INGRESS_VRFID(234) \|
	871	* +----+--------------+--------------+--------------+--------------+
	872	*/
	873	#define EX_NEL_COMMON 46
	874	typedef struct tpl_ext_46_s {
	875	uint8_t nat_event;
	876	uint8_t flags;
	877	uint16_t fill;
	878	uint32_t egress_vrfid;
	879	uint32_t ingress_vrfid;
	880	uint8_t data[4]; // points to further data
	881	} tpl_ext_46_t;
	882
	883	#define EX_NEL_GLOBAL_IP_v4 47
	884	/*
	885	* no longer used. Mapped to NSEL extension EX_NSEL_XLATE_IP_v4
	886	*/
	887	typedef struct tpl_ext_47_s {
	888	uint32_t nat_inside;
	889	uint32_t nat_outside;
	890	uint8_t data[4]; // points to further data
	891	} tpl_ext_47_t;
	892
	893	/*
	894	* NEL Port Block Allocation
	895	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	896	* \| 0 \| NF_F_XLATE_PORT_BLOCK_START \| NF_F_XLATE_PORT_BLOCK_END \| NF_F_XLATE_PORT_BLOCK_STEP \| NF_F_XLATE_PORT_BLOCK_SIZE \|
	897	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	898	*/
	899	#define EX_PORT_BLOCK_ALLOC 48
	900	typedef struct tpl_ext_48_s {
	901	uint16_t block_start;
	902	uint16_t block_end;
	903	uint16_t block_step;
	904	uint16_t block_size;
	905	uint8_t data[4]; // points to further data
	906	} tpl_ext_48_t;
	907
	908	#define EX_NEL_RESERVED_1 49
	909
	910	/*
	911	* V1 Extension map:
	912	* =================
	913	* The extension map replaces the individual flags in v1 layout. With many possible extensions and combination of extensions
	914	* an extension map is more efficient and flexible while reading and decoding the record.
	915	* In current version of nfdump, up to 65535 individual extension maps are supported, which is considered to be enough.
	916	*
	917	* For each available extension record, the ids are recorded in the extension map in the order they appear.
	918	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	919	* \| - \| 0 \| 1 \| 2 \| 3 \| 4 \| 5 \| 6 \| 7 \|
	920	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	921	* \| 0 \| record type == 2 \| size \| map id \| extension size \|
	922	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	923	* \| 0 \| extension id 1 \| extension id 2 \| extension id 3 \| extension id 4 \|
	924	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	925	* ...
	926	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	927	* \| 0 \| extension id n \| extension id n+1 \| extension id n+2 \| extension id n+3 \|
	928	* +----+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
	929	* ...
	930	* +----+--------------+--------------+--------------+--------------+
	931	* \| 0 \| 0 \| opt. 32bit alignment: 0 \|
	932	* +----+--------------+--------------+--------------+--------------+
	933	*/
	934
	935	typedef struct extension_map_s {
	936	// record head
	937	uint16_t type; // is ExtensionMapType
	938	uint16_t size; // size of full map incl. header
	939
	940	// map data
	941	#define INIT_ID 0xFFFF
	942	uint16_t map_id; // identifies this map
	943	uint16_t extension_size; // size of all extensions
	944	uint16_t ex_id[1]; // extension id array
	945	} extension_map_t;
	946
49	947	typedef struct extension_descriptor_s {
50	948	uint16_t id; // id number
51	949	uint16_t size; // number of bytes

57	955	typedef struct extension_info_s {
58	956	struct extension_info_s *next;
59	957	extension_map_t *map;
	958	extension_map_t *exportMap;
60	959	uint32_t ref_count;
61	960	uint32_t *offset_cache;
62	961	master_record_t master_record;

69	968	uint32_t max_used;
70	969	} extension_map_list_t;
71	970
72		#define NEEDS_EXTENSION_LIST 1
73		#define NO_EXTENSION_LIST 0
74	971	extension_map_list_t *InitExtensionMaps(int AllocateList);
75	972
76	973	void FreeExtensionMaps(extension_map_list_t *extension_map_list);

+341

-0

bin/output_csv.c less more

	0	/*
	1	* Copyright (c) 2019-2020, Peter Haag
	2	* All rights reserved.
	3	*
	4	* Redistribution and use in source and binary forms, with or without
	5	* modification, are permitted provided that the following conditions are met:
	6	*
	7	* * Redistributions of source code must retain the above copyright notice,
	8	* this list of conditions and the following disclaimer.
	9	* * Redistributions in binary form must reproduce the above copyright notice,
	10	* this list of conditions and the following disclaimer in the documentation
	11	* and/or other materials provided with the distribution.
	12	* * Neither the name of the author nor the names of its contributors may be
	13	* used to endorse or promote products derived from this software without
	14	* specific prior written permission.
	15	*
	16	* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
	17	* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	18	* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	19	* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
	20	* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
	21	* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
	22	* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
	23	* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
	24	* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
	25	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
	26	* POSSIBILITY OF SUCH DAMAGE.
	27	*
	28	*/
	29
	30	#include "config.h"
	31
	32	#include <stdio.h>
	33	#include <stddef.h>
	34	#include <sys/types.h>
	35	#include <sys/socket.h>
	36	#include <netinet/in.h>
	37	#include <arpa/inet.h>
	38	#include <time.h>
	39	#include <string.h>
	40
	41	#ifdef HAVE_STDINT_H
	42	#include <stdint.h>
	43	#endif
	44
	45	#include "util.h"
	46	#include "nfdump.h"
	47	#include "nffile.h"
	48	#include "nfx.h"
	49	#include "output_util.h"
	50	#include "output_csv.h"
	51
	52	#define STRINGSIZE 10240
	53	#define IP_STRING_LEN (INET6_ADDRSTRLEN)
	54
	55	static char data_string[STRINGSIZE];
	56
	57	// record counter
	58	static uint32_t recordCount;
	59
	60	void csv_prolog(void) {
	61	recordCount = 0;
	62	memset(data_string, 0, STRINGSIZE);
	63
	64	printf("ts,te,td,sa,da,sp,dp,pr,flg,fwd,stos,ipkt,ibyt,opkt,obyt,in,out,sas,das,smk,dmk,dtos,dir,nh,nhb,svln,dvln,ismc,odmc,idmc,osmc,mpls1,mpls2,mpls3,mpls4,mpls5,mpls6,mpls7,mpls8,mpls9,mpls10,cl,sl,al,ra,eng,exid,tr");
	65
	66	} // End of csv_prolog
	67
	68	void csv_epilog(void) {
	69
	70	} // End of csv_epilog
	71
	72	void flow_record_to_csv(void record, char * s, int tag) {
	73	char *_s, as[IP_STRING_LEN], ds[IP_STRING_LEN];
	74	char datestr1[64], datestr2[64], datestr3[64];
	75	char s_snet[IP_STRING_LEN], s_dnet[IP_STRING_LEN];
	76	ssize_t slen, _slen;
	77	time_t when;
	78	struct tm *ts;
	79	master_record_t r = (master_record_t )record;
	80
	81	as[0] = 0;
	82	ds[0] = 0;
	83	if ( (r->flags & FLAG_IPV6_ADDR ) != 0 ) { // IPv6
	84	uint64_t snet[2];
	85	uint64_t dnet[2];
	86
	87	snet[0] = htonll(r->V6.srcaddr[0]);
	88	snet[1] = htonll(r->V6.srcaddr[1]);
	89	dnet[0] = htonll(r->V6.dstaddr[0]);
	90	dnet[1] = htonll(r->V6.dstaddr[1]);
	91	inet_ntop(AF_INET6, snet, as, sizeof(as));
	92	inet_ntop(AF_INET6, dnet, ds, sizeof(ds));
	93
	94	inet6_ntop_mask(r->V6.srcaddr, r->src_mask, s_snet, sizeof(s_snet));
	95	inet6_ntop_mask(r->V6.dstaddr, r->dst_mask, s_dnet, sizeof(s_dnet));
	96
	97	} else { // IPv4
	98	uint32_t snet, dnet;
	99	snet = htonl(r->V4.srcaddr);
	100	dnet = htonl(r->V4.dstaddr);
	101	inet_ntop(AF_INET, &snet, as, sizeof(as));
	102	inet_ntop(AF_INET, &dnet, ds, sizeof(ds));
	103
	104	inet_ntop_mask(r->V4.srcaddr, r->src_mask, s_snet, sizeof(s_snet));
	105	inet_ntop_mask(r->V4.dstaddr, r->dst_mask, s_dnet, sizeof(s_dnet));
	106
	107	}
	108	as[IP_STRING_LEN-1] = 0;
	109	ds[IP_STRING_LEN-1] = 0;
	110
	111	when = r->first;
	112	ts = localtime(&when);
	113	strftime(datestr1, 63, "%Y-%m-%d %H:%M:%S", ts);
	114
	115	when = r->last;
	116	ts = localtime(&when);
	117	strftime(datestr2, 63, "%Y-%m-%d %H:%M:%S", ts);
	118
	119	double duration = r->last - r->first;
	120	duration += ((double)r->msec_last - (double)r->msec_first) / 1000.0;
	121
	122	_s = data_string;
	123	slen = STRINGSIZE;
	124	snprintf(_s, slen-1, "%s,%s,%.3f,%s,%s,%u,%u,%s,%s,%u,%u,%llu,%llu,%llu,%llu",
	125	datestr1, datestr2, duration, as,ds,r->srcport, r->dstport, ProtoString(r->prot, 0),
	126	FlagsString(r->tcp_flags), r->fwd_status, r->tos, (unsigned long long)r->dPkts,
	127	(unsigned long long)r->dOctets, (long long unsigned)r->out_pkts, (long long unsigned)r->out_bytes);
	128
	129	_slen = strlen(data_string);
	130	_s += _slen;
	131	slen -= _slen;
	132
	133	// EX_IO_SNMP_2:
	134	// EX_IO_SNMP_4:
	135	snprintf(_s, slen-1, ",%u,%u" , r->input, r->output);
	136	_slen = strlen(data_string);
	137	_s = data_string + _slen;
	138	slen = STRINGSIZE - _slen;
	139
	140	// EX_AS_2:
	141	// EX_AS_4:
	142	snprintf(_s, slen-1, ",%u,%u", r->srcas, r->dstas);
	143	_slen = strlen(data_string);
	144	_s = data_string + _slen;
	145	slen = STRINGSIZE - _slen;
	146
	147	// EX_MULIPLE:
	148	snprintf(_s, slen-1, ",%u,%u,%u,%u" , r->src_mask, r->dst_mask, r->dst_tos, r->dir );
	149	_slen = strlen(data_string);
	150	_s = data_string + _slen;
	151	slen = STRINGSIZE - _slen;
	152
	153	if ( (r->flags & FLAG_IPV6_NH ) != 0 ) { // IPv6
	154	// EX_NEXT_HOP_v6:
	155	as[0] = 0;
	156	r->ip_nexthop.V6[0] = htonll(r->ip_nexthop.V6[0]);
	157	r->ip_nexthop.V6[1] = htonll(r->ip_nexthop.V6[1]);
	158	inet_ntop(AF_INET6, r->ip_nexthop.V6, as, sizeof(as));
	159	as[IP_STRING_LEN-1] = 0;
	160
	161	snprintf(_s, slen-1, ",%s", as);
	162	_slen = strlen(data_string);
	163	_s = data_string + _slen;
	164	slen = STRINGSIZE - _slen;
	165
	166	} else {
	167	// EX_NEXT_HOP_v4:
	168	as[0] = 0;
	169	r->ip_nexthop.V4 = htonl(r->ip_nexthop.V4);
	170	inet_ntop(AF_INET, &r->ip_nexthop.V4, as, sizeof(as));
	171	as[IP_STRING_LEN-1] = 0;
	172
	173	snprintf(_s, slen-1, ",%s", as);
	174	_slen = strlen(data_string);
	175	_s = data_string + _slen;
	176	slen = STRINGSIZE - _slen;
	177	}
	178
	179	if ( (r->flags & FLAG_IPV6_NHB ) != 0 ) { // IPv6
	180	// EX_NEXT_HOP_BGP_v6:
	181	as[0] = 0;
	182	r->bgp_nexthop.V6[0] = htonll(r->bgp_nexthop.V6[0]);
	183	r->bgp_nexthop.V6[1] = htonll(r->bgp_nexthop.V6[1]);
	184	inet_ntop(AF_INET6, r->ip_nexthop.V6, as, sizeof(as));
	185	as[IP_STRING_LEN-1] = 0;
	186
	187	snprintf(_s, slen-1, ",%s", as);
	188	_slen = strlen(data_string);
	189	_s = data_string + _slen;
	190	slen = STRINGSIZE - _slen;
	191
	192	} else {
	193	// EX_NEXT_HOP_BGP_v4:
	194	as[0] = 0;
	195	r->bgp_nexthop.V4 = htonl(r->bgp_nexthop.V4);
	196	inet_ntop(AF_INET, &r->bgp_nexthop.V4, as, sizeof(as));
	197	as[IP_STRING_LEN-1] = 0;
	198
	199	snprintf(_s, slen-1, ",%s", as);
	200	_slen = strlen(data_string);
	201	_s = data_string + _slen;
	202	slen = STRINGSIZE - _slen;
	203
	204	}
	205
	206	// EX_VLAN:
	207	snprintf(_s, slen-1, ",%u,%u", r->src_vlan, r->dst_vlan);
	208	_slen = strlen(data_string);
	209	_s = data_string + _slen;
	210	slen = STRINGSIZE - _slen;
	211
	212
	213	/* already in default output:
	214	EX_OUT_PKG_4:
	215	EX_OUT_PKG_8:
	216	EX_OUT_BYTES_4:
	217	EX_OUT_BYTES_8:
	218	*/
	219
	220	// case EX_MAC_1:
	221	{
	222	int i;
	223	uint8_t mac1[6], mac2[6];
	224
	225	for ( i=0; i<6; i++ ) {
	226	mac1[i] = (r->in_src_mac >> ( i*8 )) & 0xFF;
	227	}
	228	for ( i=0; i<6; i++ ) {
	229	mac2[i] = (r->out_dst_mac >> ( i*8 )) & 0xFF;
	230	}
	231
	232	snprintf(_s, slen-1, ",%.2x:%.2x:%.2x:%.2x:%.2x:%.2x,%.2x:%.2x:%.2x:%.2x:%.2x:%.2x",
	233	mac1[5], mac1[4], mac1[3], mac1[2], mac1[1], mac1[0],
	234	mac2[5], mac2[4], mac2[3], mac2[2], mac2[1], mac2[0] );
	235	_slen = strlen(data_string);
	236	_s = data_string + _slen;
	237	slen = STRINGSIZE - _slen;
	238	}
	239
	240	// EX_MAC_2:
	241	{
	242	int i;
	243	uint8_t mac1[6], mac2[6];
	244
	245	for ( i=0; i<6; i++ ) {
	246	mac1[i] = (r->in_dst_mac >> ( i*8 )) & 0xFF;
	247	}
	248	for ( i=0; i<6; i++ ) {
	249	mac2[i] = (r->out_src_mac >> ( i*8 )) & 0xFF;
	250	}
	251
	252	snprintf(_s, slen-1, ",%.2x:%.2x:%.2x:%.2x:%.2x:%.2x,%.2x:%.2x:%.2x:%.2x:%.2x:%.2x",
	253	mac1[5], mac1[4], mac1[3], mac1[2], mac1[1], mac1[0],
	254	mac2[5], mac2[4], mac2[3], mac2[2], mac2[1], mac2[0] );
	255	_slen = strlen(data_string);
	256	_s = data_string + _slen;
	257	slen = STRINGSIZE - _slen;
	258	}
	259
	260	// EX_MPLS:
	261	{
	262	unsigned int i;
	263	for ( i=0; i<10; i++ ) {
	264	snprintf(_s, slen-1, ",%u-%1u-%1u",
	265	r->mpls_label[i] >> 4 , (r->mpls_label[i] & 0xF ) >> 1, r->mpls_label[i] & 1 );
	266	_slen = strlen(data_string);
	267	_s = data_string + _slen;
	268	slen = STRINGSIZE - _slen;
	269	}
	270	}
	271
	272	{
	273	double f1, f2, f3;
	274	f1 = (double)r->client_nw_delay_usec / 1000.0;
	275	f2 = (double)r->server_nw_delay_usec / 1000.0;
	276	f3 = (double)r->appl_latency_usec / 1000.0;
	277
	278	snprintf(_s, slen-1,
	279	",%9.3f,%9.3f,%9.3f", f1, f2, f3);
	280
	281	_slen = strlen(data_string);
	282	_s = data_string + _slen;
	283	slen = STRINGSIZE - _slen;
	284	}
	285
	286
	287	// EX_ROUTER_IP_v4:
	288	if ( (r->flags & FLAG_IPV6_EXP ) != 0 ) { // IPv6
	289	// EX_NEXT_HOP_v6:
	290	as[0] = 0;
	291	r->ip_router.V6[0] = htonll(r->ip_router.V6[0]);
	292	r->ip_router.V6[1] = htonll(r->ip_router.V6[1]);
	293	inet_ntop(AF_INET6, r->ip_router.V6, as, sizeof(as));
	294	as[IP_STRING_LEN-1] = 0;
	295
	296	snprintf(_s, slen-1, ",%s", as);
	297	_slen = strlen(data_string);
	298	_s = data_string + _slen;
	299	slen = STRINGSIZE - _slen;
	300
	301	} else {
	302	// EX_NEXT_HOP_v4:
	303	as[0] = 0;
	304	r->ip_router.V4 = htonl(r->ip_router.V4);
	305	inet_ntop(AF_INET, &r->ip_router.V4, as, sizeof(as));
	306	as[IP_STRING_LEN-1] = 0;
	307
	308	snprintf(_s, slen-1, ",%s", as);
	309	_slen = strlen(data_string);
	310	_s = data_string + _slen;
	311	slen = STRINGSIZE - _slen;
	312	}
	313
	314	// EX_ROUTER_ID
	315	snprintf(_s, slen-1, ",%u/%u", r->engine_type, r->engine_id);
	316	_slen = strlen(data_string);
	317	_s = data_string + _slen;
	318	slen = STRINGSIZE - _slen;
	319
	320	// Exporter SysID
	321	snprintf(_s, slen-1, ",%u", r->exporter_sysid);
	322	_slen = strlen(data_string);
	323	_s = data_string + _slen;
	324	slen = STRINGSIZE - _slen;
	325
	326	// Date flow received
	327	when = r->received / 1000LL;
	328	ts = localtime(&when);
	329	strftime(datestr3, 63, ",%Y-%m-%d %H:%M:%S", ts);
	330
	331	snprintf(_s, slen-1, "%s.%03llu", datestr3, (long long unsigned)r->received % 1000LL);
	332	_slen = strlen(data_string);
	333	_s = data_string + _slen;
	334	slen = STRINGSIZE - _slen;
	335
	336	// snprintf(_s, slen-1, "\n");
	337	data_string[STRINGSIZE-1] = 0;
	338	*s = data_string;
	339
	340	} // End of flow_record_to_csv

+40

-0

bin/output_csv.h less more

	0	/*
	1	* Copyright (c) 2019, Peter Haag
	2	* All rights reserved.
	3	*
	4	* Redistribution and use in source and binary forms, with or without
	5	* modification, are permitted provided that the following conditions are met:
	6	*
	7	* * Redistributions of source code must retain the above copyright notice,
	8	* this list of conditions and the following disclaimer.
	9	* * Redistributions in binary form must reproduce the above copyright notice,
	10	* this list of conditions and the following disclaimer in the documentation
	11	* and/or other materials provided with the distribution.
	12	* * Neither the name of the author nor the names of its contributors may be
	13	* used to endorse or promote products derived from this software without
	14	* specific prior written permission.
	15	*
	16	* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
	17	* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	18	* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	19	* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
	20	* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
	21	* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
	22	* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
	23	* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
	24	* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
	25	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
	26	* POSSIBILITY OF SUCH DAMAGE.
	27	*
	28	*/
	29
	30	#ifndef _OUTPUT_CSV_H
	31	#define _OUTPUT_CSV_H 1
	32
	33	void csv_prolog(void);
	34
	35	void csv_epilog(void);
	36
	37	void flow_record_to_csv(void record, char *s, int tag);
	38
	39	#endif // _OUTPUT_CSV_H

+1686

-0

bin/output_fmt.c less more

	0	/*
	1	* Copyright (c) 2009-2020, Peter Haag
	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
	3	* All rights reserved.
	4	*
	5	* Redistribution and use in source and binary forms, with or without
	6	* modification, are permitted provided that the following conditions are met:
	7	*
	8	* * Redistributions of source code must retain the above copyright notice,
	9	* this list of conditions and the following disclaimer.
	10	* * Redistributions in binary form must reproduce the above copyright notice,
	11	* this list of conditions and the following disclaimer in the documentation
	12	* and/or other materials provided with the distribution.
	13	* * Neither the name of the author nor the names of its contributors may be
	14	* used to endorse or promote products derived from this software without
	15	* specific prior written permission.
	16	*
	17	* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
	18	* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	19	* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	20	* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
	21	* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
	22	* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
	23	* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
	24	* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
	25	* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
	26	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
	27	* POSSIBILITY OF SUCH DAMAGE.
	28	*
	29	*/
	30
	31	#include "config.h"
	32
	33	#include <stdio.h>
	34	#include <stddef.h>
	35	#include <sys/types.h>
	36	#include <sys/socket.h>
	37	#include <netinet/in.h>
	38	#include <arpa/inet.h>
	39	#include <time.h>
	40	#include <string.h>
	41	#include <ctype.h>
	42	#include <stdlib.h>
	43	#include <errno.h>
	44
	45	#ifdef HAVE_STDINT_H
	46	#include <stdint.h>
	47	#endif
	48
	49	#include "util.h"
	50	#include "nfdump.h"
	51	#include "nffile.h"
	52	#include "nfx.h"
	53	#include "output_util.h"
	54	#include "output_fmt.h"
	55
	56	typedef void (string_function_t)(master_record_t , char *);
	57
	58	static struct token_list_s {
	59	string_function_t string_function; // function generation output string
	60	char *string_buffer; // buffer for output string
	61	} *token_list;
	62
	63	static int max_token_index = 0;
	64	static int token_index = 0;
	65
	66	#define BLOCK_SIZE 32
	67
	68	static char **format_list; // ordered list of all individual strings formating the output line
	69	static int max_format_index = 0;
	70	static int format_index = 0;
	71
	72	static int do_tag = 0;
	73	static int long_v6 = 0;
	74	static int printPlain = 0;
	75	static double duration;
	76
	77	#define STRINGSIZE 10240
	78	#define IP_STRING_LEN (INET6_ADDRSTRLEN)
	79
	80	static char header_string[STRINGSIZE];
	81	static char data_string[STRINGSIZE];
	82
	83	// tag
	84	static char tag_string[2];
	85
	86	/* prototypes */
	87	static char ICMP_Port_decode(master_record_t r);
	88
	89	static void InitFormatParser(void);
	90
	91	static void AddToken(int index);
	92
	93	static void AddString(char *string);
	94
	95	static void String_FlowFlags(master_record_t r, char string);
	96
	97	static void String_FirstSeen(master_record_t r, char string);
	98
	99	static void String_LastSeen(master_record_t r, char string);
	100
	101	static void String_Received(master_record_t r, char string);
	102
	103	static void String_FirstSeenRaw(master_record_t r, char string);
	104
	105	static void String_LastSeenRaw(master_record_t r, char string);
	106
	107	static void String_ReceivedRaw(master_record_t r, char string);
	108
	109	static void String_Duration(master_record_t r, char string);
	110
	111	static void String_Protocol(master_record_t r, char string);
	112
	113	static void String_SrcAddr(master_record_t r, char string);
	114
	115	static void String_DstAddr(master_record_t r, char string);
	116
	117	static void String_SrcAddrPort(master_record_t r, char string);
	118
	119	static void String_DstAddrPort(master_record_t r, char string);
	120
	121	static void String_SrcNet(master_record_t r, char string);
	122
	123	static void String_DstNet(master_record_t r, char string);
	124
	125	static void String_NextHop(master_record_t r, char string);
	126
	127	static void String_BGPNextHop(master_record_t r, char string);
	128
	129	static void String_RouterIP(master_record_t r, char string);
	130
	131	static void String_SrcPort(master_record_t r, char string);
	132
	133	static void String_DstPort(master_record_t r, char string);
	134
	135	static void String_ICMP_code(master_record_t r, char string);
	136
	137	static void String_ICMP_type(master_record_t r, char string);
	138
	139	static void String_SrcAS(master_record_t r, char string);
	140
	141	static void String_DstAS(master_record_t r, char string);
	142
	143	static void String_NextAS(master_record_t r, char string);
	144
	145	static void String_PrevAS(master_record_t r, char string);
	146
	147	static void String_Input(master_record_t r, char string);
	148
	149	static void String_Output(master_record_t r, char string);
	150
	151	static void String_InPackets(master_record_t r, char string);
	152
	153	static void String_OutPackets(master_record_t r, char string);
	154
	155	static void String_InBytes(master_record_t r, char string);
	156
	157	static void String_OutBytes(master_record_t r, char string);
	158
	159	static void String_Flows(master_record_t r, char string);
	160
	161	static void String_Tos(master_record_t r, char string);
	162
	163	static void String_Dir(master_record_t r, char string);
	164
	165	static void String_SrcTos(master_record_t r, char string);
	166
	167	static void String_DstTos(master_record_t r, char string);
	168
	169	static void String_SrcMask(master_record_t r, char string);
	170
	171	static void String_DstMask(master_record_t r, char string);
	172
	173	static void String_SrcVlan(master_record_t r, char string);
	174
	175	static void String_DstVlan(master_record_t r, char string);
	176
	177	static void String_FwdStatus(master_record_t r, char string);
	178
	179	static void String_Flags(master_record_t r, char string);
	180
	181	static void String_InSrcMac(master_record_t r, char string);
	182
	183	static void String_OutDstMac(master_record_t r, char string);
	184
	185	static void String_InDstMac(master_record_t r, char string);
	186
	187	static void String_OutSrcMac(master_record_t r, char string);
	188
	189	static void String_MPLS_1(master_record_t r, char string);
	190
	191	static void String_MPLS_2(master_record_t r, char string);
	192
	193	static void String_MPLS_3(master_record_t r, char string);
	194
	195	static void String_MPLS_4(master_record_t r, char string);
	196
	197	static void String_MPLS_5(master_record_t r, char string);
	198
	199	static void String_MPLS_6(master_record_t r, char string);
	200
	201	static void String_MPLS_7(master_record_t r, char string);
	202
	203	static void String_MPLS_8(master_record_t r, char string);
	204
	205	static void String_MPLS_9(master_record_t r, char string);
	206
	207	static void String_MPLS_10(master_record_t r, char string);
	208
	209	static void String_MPLSs(master_record_t r, char string);
	210
	211	static void String_Engine(master_record_t r, char string);
	212
	213	static void String_Label(master_record_t r, char string);
	214
	215	static void String_ClientLatency(master_record_t r, char string);
	216
	217	static void String_ServerLatency(master_record_t r, char string);
	218
	219	static void String_AppLatency(master_record_t r, char string);
	220
	221	static void String_bps(master_record_t r, char string);
	222
	223	static void String_pps(master_record_t r, char string);
	224
	225	static void String_bpp(master_record_t r, char string);
	226
	227	static void String_ExpSysID(master_record_t r, char string);
	228
	229	#ifdef NSEL
	230	static void String_EventTime(master_record_t r, char string);
	231
	232	static void String_nfc(master_record_t r, char string);
	233
	234	static void String_evt(master_record_t r, char string);
	235
	236	static void String_xevt(master_record_t r, char string);
	237
	238	static void String_sgt(master_record_t r, char string);
	239
	240	static void String_msec(master_record_t r, char string);
	241
	242	static void String_iacl(master_record_t r, char string);
	243
	244	static void String_eacl(master_record_t r, char string);
	245
	246	static void String_xlateSrcAddr(master_record_t r, char string);
	247
	248	static void String_xlateDstAddr(master_record_t r, char string);
	249
	250	static void String_xlateSrcPort(master_record_t r, char string);
	251
	252	static void String_xlateDstPort(master_record_t r, char string);
	253
	254	static void String_xlateSrcAddrPort(master_record_t r, char string);
	255
	256	static void String_xlateDstAddrPort(master_record_t r, char string);
	257
	258	static void String_userName(master_record_t r, char string);
	259
	260	static void String_ivrf(master_record_t r, char string);
	261
	262	static void String_evrf(master_record_t r, char string);
	263
	264	static void String_PortBlockStart(master_record_t r, char string);
	265
	266	static void String_PortBlockEnd(master_record_t r, char string);
	267
	268	static void String_PortBlockStep(master_record_t r, char string);
	269
	270	static void String_PortBlockSize(master_record_t r, char string);
	271
	272	#endif
	273
	274	static struct format_token_list_s {
	275	char *token; // token
	276	int is_address; // is an IP address
	277	char *header; // header line description
	278	string_function_t string_function; // function generation output string
	279	} format_token_list[] = {
	280	{ "%ff", 0, "Flow Flags", String_FlowFlags }, // flow flags in hex
	281	{ "%tfs", 0, "Date first seen ", String_FirstSeen }, // Start Time - first seen
	282	{ "%ts", 0, "Date first seen ", String_FirstSeen }, // Start Time - first seen
	283	{ "%tsr", 0, "Date first seen (raw) ", String_FirstSeenRaw }, // Start Time - first seen, seconds
	284	{ "%te", 0, "Date last seen ", String_LastSeen }, // End Time - last seen
	285	{ "%ter", 0, "Date last seen (raw) ", String_LastSeenRaw }, // End Time - first seen, seconds
	286	{ "%tr", 0, "Date flow received ", String_Received }, // Received Time
	287	{ "%trr", 0, "Date flow received (raw) ", String_ReceivedRaw }, // Received Time, seconds
	288	{ "%td", 0, " Duration", String_Duration }, // Duration
	289	{ "%exp", 0, "Exp ID", String_ExpSysID }, // Exporter SysID
	290	{ "%pr", 0, "Proto", String_Protocol }, // Protocol
	291	{ "%sa", 1, " Src IP Addr", String_SrcAddr }, // Source Address
	292	{ "%da", 1, " Dst IP Addr", String_DstAddr }, // Destination Address
	293	{ "%sn", 1, " Src Network", String_SrcNet }, // Source Address applied source netmask
	294	{ "%dn", 1, " Dst Network", String_DstNet }, // Destination Address applied source netmask
	295	{ "%nh", 1, " Next-hop IP", String_NextHop }, // Next-hop IP Address
	296	{ "%nhb", 1, " BGP next-hop IP", String_BGPNextHop }, // BGP Next-hop IP Address
	297	{ "%ra", 1, " Router IP", String_RouterIP }, // Router IP Address
	298	{ "%sap", 1, " Src IP Addr:Port ", String_SrcAddrPort }, // Source Address:Port
	299	{ "%dap", 1, " Dst IP Addr:Port ", String_DstAddrPort }, // Destination Address:Port
	300	{ "%sp", 0, "Src Pt", String_SrcPort }, // Source Port
	301	{ "%dp", 0, "Dst Pt", String_DstPort }, // Destination Port
	302	{ "%it", 0, "ICMP-T", String_ICMP_type }, // ICMP type
	303	{ "%ic", 0, "ICMP-C", String_ICMP_code }, // ICMP code
	304	{ "%sas", 0, "Src AS", String_SrcAS }, // Source AS
	305	{ "%das", 0, "Dst AS", String_DstAS }, // Destination AS
	306	{ "%nas", 0, "Next AS", String_NextAS }, // Next AS
	307	{ "%pas", 0, "Prev AS", String_PrevAS }, // Previous AS
	308	{ "%in", 0, " Input", String_Input }, // Input Interface num
	309	{ "%out", 0, "Output", String_Output }, // Output Interface num
	310	{ "%pkt", 0, " Packets", String_InPackets }, // Packets - default input - compat
	311	{ "%ipkt", 0, " In Pkt", String_InPackets }, // In Packets
	312	{ "%opkt", 0, " Out Pkt", String_OutPackets }, // Out Packets
	313	{ "%byt", 0, " Bytes", String_InBytes }, // Bytes - default input - compat
	314	{ "%ibyt", 0, " In Byte", String_InBytes }, // In Bytes
	315	{ "%obyt", 0, "Out Byte", String_OutBytes }, // In Bytes
	316	{ "%fl", 0, "Flows", String_Flows }, // Flows
	317	{ "%flg", 0, " Flags", String_Flags }, // TCP Flags
	318	{ "%tos", 0, "Tos", String_Tos }, // Tos - compat
	319	{ "%stos", 0, "STos", String_SrcTos }, // Tos - Src tos
	320	{ "%dtos", 0, "DTos", String_DstTos }, // Tos - Dst tos
	321	{ "%dir", 0, "Dir", String_Dir }, // Direction: ingress, egress
	322	{ "%smk", 0, "SMask", String_SrcMask }, // Src mask
	323	{ "%dmk", 0, "DMask", String_DstMask }, // Dst mask
	324	{ "%fwd", 0, "Fwd", String_FwdStatus }, // Forwarding Status
	325	{ "%svln", 0, "SVlan", String_SrcVlan }, // Src Vlan
	326	{ "%dvln", 0, "DVlan", String_DstVlan }, // Dst Vlan
	327	{ "%ismc", 0, " In src MAC Addr", String_InSrcMac }, // Input Src Mac Addr
	328	{ "%odmc", 0, " Out dst MAC Addr", String_OutDstMac }, // Output Dst Mac Addr
	329	{ "%idmc", 0, " In dst MAC Addr", String_InDstMac }, // Input Dst Mac Addr
	330	{ "%osmc", 0, " Out src MAC Addr", String_OutSrcMac }, // Output Src Mac Addr
	331	{ "%mpls1", 0, " MPLS lbl 1 ", String_MPLS_1 }, // MPLS Label 1
	332	{ "%mpls2", 0, " MPLS lbl 2 ", String_MPLS_2 }, // MPLS Label 2
	333	{ "%mpls3", 0, " MPLS lbl 3 ", String_MPLS_3 }, // MPLS Label 3
	334	{ "%mpls4", 0, " MPLS lbl 4 ", String_MPLS_4 }, // MPLS Label 4
	335	{ "%mpls5", 0, " MPLS lbl 5 ", String_MPLS_5 }, // MPLS Label 5
	336	{ "%mpls6", 0, " MPLS lbl 6 ", String_MPLS_6 }, // MPLS Label 6
	337	{ "%mpls7", 0, " MPLS lbl 7 ", String_MPLS_7 }, // MPLS Label 7
	338	{ "%mpls8", 0, " MPLS lbl 8 ", String_MPLS_8 }, // MPLS Label 8
	339	{ "%mpls9", 0, " MPLS lbl 9 ", String_MPLS_9 }, // MPLS Label 9
	340	{ "%mpls10", 0, " MPLS lbl 10", String_MPLS_10 }, // MPLS Label 10
	341	{ "%mpls", 0, " MPLS labels 1-10 ", String_MPLSs }, // All MPLS labels
	342	//
	343	{ "%bps", 0, " bps", String_bps }, // bps - bits per second
	344	{ "%pps", 0, " pps", String_pps }, // pps - packets per second
	345	{ "%bpp", 0, " Bpp", String_bpp }, // bpp - Bytes per package
	346	{ "%eng", 0, " engine", String_Engine }, // Engine Type/ID
	347	{ "%lbl", 0, " label", String_Label }, // Flow Label
	348
	349	#ifdef NSEL
	350	// NSEL specifics
	351	{ "%nfc", 0, " Conn-ID", String_nfc }, // NSEL connection ID
	352	{ "%tevt", 0, "Event time ",String_EventTime }, // NSEL Flow start time
	353	{ "%evt", 0, " Event", String_evt }, // NSEL event
	354	{ "%xevt", 0, " XEvent", String_xevt }, // NSEL xevent
	355	{ "%sgt", 0, " SGT ", String_sgt }, // NSEL xevent
	356	{ "%msec", 0, " Event Time", String_msec}, // NSEL event time in msec
	357	{ "%iacl", 0, "Ingress ACL ", String_iacl}, // NSEL ingress ACL
	358	{ "%eacl", 0, "Egress ACL ", String_eacl}, // NSEL egress ACL
	359	{ "%xsa", 0, " X-late Src IP", String_xlateSrcAddr}, // NSEL XLATE src IP
	360	{ "%xda", 0, " X-late Dst IP", String_xlateDstAddr}, // NSEL XLATE dst IP
	361	{ "%xsp", 0, "XsPort", String_xlateSrcPort}, // NSEL XLATE src port
	362	{ "%xdp", 0, "XdPort", String_xlateDstPort}, // NSEL SLATE dst port
	363	{ "%xsap", 1, " X-Src IP Addr:Port ", String_xlateSrcAddrPort }, // Xlate Source Address:Port
	364	{ "%xdap", 1, " X-Dst IP Addr:Port ", String_xlateDstAddrPort }, // Xlate Destination Address:Port
	365	{ "%uname", 0, "UserName", String_userName}, // NSEL user name
	366
	367	// NEL
	368	// for v.1.6.10 compatibility, keep NEL specific addr/port format tokens
	369	{ "%nevt", 0, " Event", String_evt }, // NAT event
	370	{ "%vrf", 0, " I-VRF-ID", String_ivrf }, // NAT ivrf ID - compatible
	371	{ "%ivrf", 0, " I-VRF-ID", String_ivrf }, // NAT ivrf ID
	372	{ "%evrf", 0, " E-VRF-ID", String_evrf }, // NAT ivrf ID
	373	{ "%nsa", 0, " X-late Src IP", String_xlateSrcAddr}, // NAT XLATE src IP
	374	{ "%nda", 0, " X-late Dst IP", String_xlateDstAddr}, // NAT XLATE dst IP
	375	{ "%nsp", 0, "XsPort", String_xlateSrcPort}, // NAT XLATE src port
	376	{ "%ndp", 0, "XdPort", String_xlateDstPort}, // NAT SLATE dst port
	377	{ "%nsap", 1, " X-Src IP Addr:Port ", String_xlateSrcAddrPort },// NAT Xlate Source Address:Port
	378	{ "%ndap", 1, " X-Dst IP Addr:Port ", String_xlateDstAddrPort },// NAT Xlate Destination Address:Port
	379
	380	// Port block allocation
	381	{ "%pbstart", 0, "Pb-Start", String_PortBlockStart}, // Port block start
	382	{ "%pbend", 0, "Pb-End", String_PortBlockEnd}, // Port block end
	383	{ "%pbstep", 0, "Pb-Step", String_PortBlockStep}, // Port block step
	384	{ "%pbsize", 0, "Pb-Size", String_PortBlockSize}, // Port block size
	385	#endif
	386
	387	// latency extension for nfpcapd and nprobe
	388	{ "%cl", 0, "C Latency", String_ClientLatency }, // client latency
	389	{ "%sl", 0, "S latency", String_ServerLatency }, // server latency
	390	{ "%al", 0, "A latency", String_AppLatency }, // app latency
	391
	392	{ NULL, 0, NULL, NULL }
	393	};
	394
	395	/* each of the tokens above must not generate output strings larger than this */
	396	#define MAX_STRING_LENGTH 256
	397
	398
	399	#include "applybits_inline.c"
	400
	401	/* functions */
	402
	403
	404	void Setv6Mode(int mode) {
	405	long_v6 += mode;
	406	}
	407
	408	int Getv6Mode(void) {
	409	return long_v6;
	410	}
	411
	412	void format_special(void record, char * s, int tag) {
	413	master_record_t r = (master_record_t )record;
	414	int i, index;
	415
	416	do_tag = tag;
	417	tag_string[0] = do_tag ? TAG_CHAR : '\0';
	418	tag_string[1] = '\0';
	419
	420	duration = r->last - r->first;
	421	duration += ((double)r->msec_last - (double)r->msec_first) / 1000.0;
	422	for ( i=0; i<token_index; i++ ) {
	423	token_list[i].string_function(r, token_list[i].string_buffer);
	424	}
	425
	426	// concat all strings together for the output line
	427	i = 0;
	428	for ( index=0; index<format_index; index++ ) {
	429	int j = 0;
	430	while ( format_list[index][j] && i < STRINGSIZE )
	431	data_string[i++] = format_list[index][j++];
	432	}
	433	if ( i < STRINGSIZE )
	434	data_string[i] = '\0';
	435
	436	data_string[STRINGSIZE-1] = '\0';
	437	*s = data_string;
	438
	439	} // End of format_special
	440
	441	void text_prolog(void) {
	442	printf("%s\n", header_string);
	443	} // End of text_prolog
	444
	445	void text_epilog(void) {
	446	// empty
	447	} // End of text_epilog
	448
	449	static void InitFormatParser(void) {
	450
	451	max_format_index = max_token_index = BLOCK_SIZE;
	452	format_list = (char *)malloc(max_format_index sizeof(char *));
	453	token_list = (struct token_list_s )malloc(max_token_index sizeof(struct token_list_s));
	454	if ( !format_list \|\| !token_list ) {
	455	fprintf(stderr, "Memory allocation error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
	456	exit(255);
	457	}
	458
	459	} // End of InitFormatParser
	460
	461	static void AddToken(int index) {
	462
	463	if ( token_index >= max_token_index ) { // no slot available - expand table
	464	max_token_index += BLOCK_SIZE;
	465	token_list = (struct token_list_s )realloc(token_list, max_token_index sizeof(struct token_list_s));
	466	if ( !token_list ) {
	467	fprintf(stderr, "Memory allocation error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
	468	exit(255);
	469	}
	470	}
	471	token_list[token_index].string_function = format_token_list[index].string_function;
	472	token_list[token_index].string_buffer = malloc(MAX_STRING_LENGTH);
	473	if ( !token_list[token_index].string_buffer ) {
	474	fprintf(stderr, "Memory allocation error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
	475	exit(255);
	476	}
	477
	478	AddString(token_list[token_index].string_buffer);
	479	token_index++;
	480
	481	} // End of AddToken
	482
	483	/* Add either a static string or the memory for a variable string from a token to the list */
	484	static void AddString(char *string) {
	485
	486	if ( !string ) {
	487	fprintf(stderr, "Panic! NULL string in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
	488	exit(255);
	489	}
	490
	491	if ( format_index >= max_format_index ) { // no slot available - expand table
	492	max_format_index += BLOCK_SIZE;
	493	format_list = (char *)realloc(format_list, max_format_index sizeof(char *));
	494	if ( !format_list ) {
	495	fprintf(stderr, "Memory allocation error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
	496	exit(255);
	497	}
	498	}
	499	format_list[format_index++] = string;
	500
	501	} // End of AddString
	502
	503	static char* RecursiveReplace(char format, printmap_t printmap) {
	504	int i = 0;
	505
	506	while ( printmap[i].printmode ) {
	507	char s, r;
	508	// check for printmode string
	509	s = strstr(format, printmap[i].printmode);
	510	if ( s && printmap[i].Format && s != format ) {
	511	int len = strlen(printmap[i].printmode);
	512	if ( !isalpha((int)s[len]) ) {
	513	s--;
	514	if ( s[0] == '%' ) {
	515	int newlen = strlen(format) + strlen(printmap[i].Format);
	516	r = malloc(newlen);
	517	if ( !r ) {
	518	LogError("malloc() allocation error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
	519	exit(255);
	520	}
	521	s[0] = '\0';
	522	snprintf(r, newlen, "%s%s%s", format, printmap[i].Format, &(s[len+1]) );
	523	r[newlen-1] = '\0';
	524	free(format);
	525	format = r;
	526	}
	527	}
	528	}
	529	i++;
	530	}
	531
	532	return format;
	533
	534	} // End of RecursiveReplace
	535
	536	int ParseOutputFormat(char format, int plain_numbers, printmap_t printmap) {
	537	char c, s, *h;
	538	int i, remaining;
	539
	540	printPlain = plain_numbers;
	541
	542	InitFormatParser();
	543
	544	s = strdup(format);
	545	if ( !s ) {
	546	fprintf(stderr, "Memory allocation error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
	547	exit(255);
	548	}
	549	s = RecursiveReplace(s, printmap);
	550	c = s;
	551
	552	h = header_string;
	553	*h = '\0';
	554	while ( *c ) {
	555	if ( *c == '%' ) { // it's a token from format_token_list
	556	i = 0;
	557	remaining = strlen(c);
	558	while ( format_token_list[i].token ) { // sweep through the list
	559	int len = strlen(format_token_list[i].token);
	560
	561	// a token is separated by either a space, another token, or end of string
	562	if ( remaining >= len && !isalpha((int)c[len]) ) {
	563	// separator found a expected position
	564	char p = c[len]; // save separator;
	565	c[len] = '\0';
	566	if ( strncmp(format_token_list[i].token, c, len) == 0 ) { // token found
	567	AddToken(i);
	568	if ( long_v6 && format_token_list[i].is_address )
	569	snprintf(h, STRINGSIZE-1-strlen(h), "%23s%s", "", format_token_list[i].header);
	570	else
	571	snprintf(h, STRINGSIZE-1-strlen(h), "%s", format_token_list[i].header);
	572	h += strlen(h);
	573	c[len] = p;
	574	c += len;
	575	break;
	576	} else {
	577	c[len] = p;
	578	}
	579	}
	580	i++;
	581	}
	582	if ( format_token_list[i].token == NULL ) {
	583	fprintf(stderr, "Output format parse error at: %s\n", c);
	584	free(s);
	585	return 0;
	586	}
	587	} else { // it's a static string
	588	/* a static string goes up to next '%' or end of string */
	589	char *p = strchr(c, '%');
	590	char format[32];
	591	if ( p ) {
	592	// p points to next '%' token
	593	*p = '\0';
	594	AddString(strdup(c));
	595	snprintf(format, 31, "%%%zus", strlen(c));
	596	format[31] = '\0';
	597	snprintf(h, STRINGSIZE-1-strlen(h), format, "");
	598	h += strlen(h);
	599	*p = '%';
	600	c = p;
	601	} else {
	602	// static string up to end of format string
	603	AddString(strdup(c));
	604	snprintf(format, 31, "%%%zus", strlen(c));
	605	format[31] = '\0';
	606	snprintf(h, STRINGSIZE-1-strlen(h), format, "");
	607	h += strlen(h);
	608	*c = '\0';
	609	}
	610	}
	611	}
	612
	613	free(s);
	614	return 1;
	615
	616	} // End of ParseOutputFormat
	617
	618	static char ICMP_Port_decode(master_record_t r) {
	619	#define ICMPSTRLEN 16
	620	static char icmp_string[ICMPSTRLEN];
	621
	622	if ( r->prot == IPPROTO_ICMP \|\| r->prot == IPPROTO_ICMPV6 ) { // ICMP
	623	snprintf(icmp_string, ICMPSTRLEN-1, "%u.%u", r->icmp_type, r->icmp_code);
	624	} else { // dst port
	625	snprintf(icmp_string, ICMPSTRLEN-1, "%u", r->dstport);
	626	}
	627	icmp_string[ICMPSTRLEN-1] = '\0';
	628
	629	return icmp_string;
	630
	631	} // End of ICMP_Port_decode
	632
	633	/* functions, which create the individual strings for the output line */
	634	static void String_FlowFlags(master_record_t r, char string) {
	635
	636	snprintf(string, MAX_STRING_LENGTH-1, "0x%.2x", r->flags);
	637	string[MAX_STRING_LENGTH-1] = '\0';
	638
	639	} // End of String_FlowFlags
	640
	641	static void String_FirstSeen(master_record_t r, char string) {
	642	time_t tt;
	643	struct tm * ts;
	644	char *s;
	645
	646	tt = r->first;
	647	ts = localtime(&tt);
	648	strftime(string, MAX_STRING_LENGTH-1, "%Y-%m-%d %H:%M:%S", ts);
	649	s = string + strlen(string);
	650	snprintf(s, MAX_STRING_LENGTH-strlen(string)-1,".%03u", r->msec_first);
	651	string[MAX_STRING_LENGTH-1] = '\0';
	652
	653	} // End of String_FirstSeen
	654
	655	static void String_LastSeen(master_record_t r, char string) {
	656	time_t tt;
	657	struct tm * ts;
	658	char *s;
	659
	660	tt = r->last;
	661	ts = localtime(&tt);
	662	strftime(string, MAX_STRING_LENGTH-1, "%Y-%m-%d %H:%M:%S", ts);
	663	s = string + strlen(string);
	664	snprintf(s, MAX_STRING_LENGTH-strlen(string)-1,".%03u", r->msec_last);
	665	string[MAX_STRING_LENGTH-1] = '\0';
	666
	667	} // End of String_LastSeen
	668
	669	static void String_Received(master_record_t r, char string) {
	670	time_t tt;
	671	struct tm * ts;
	672	char *s;
	673
	674	tt = r->received / 1000LL;
	675	ts = localtime(&tt);
	676	strftime(string, MAX_STRING_LENGTH-1, "%Y-%m-%d %H:%M:%S", ts);
	677	s = string + strlen(string);
	678	snprintf(s, MAX_STRING_LENGTH-strlen(string)-1,".%03llu", r->received % 1000LL);
	679	string[MAX_STRING_LENGTH-1] = '\0';
	680
	681	} // End of String_Received
	682
	683	static void String_ReceivedRaw(master_record_t r, char string) {
	684
	685	/* snprintf does write \0, and the max is INCL the terminating \0 */
	686	snprintf(string, MAX_STRING_LENGTH, "%.3f", r->received/1000.0);
	687
	688	} // End of String_ReceivedRaw
	689
	690	static void String_FirstSeenRaw(master_record_t r, char string) {
	691
	692	/* snprintf does write \0, and the max is INCL the terminating \0 */
	693	snprintf(string, MAX_STRING_LENGTH, "%u.%03u", r->first, r->msec_first);
	694
	695	} // End of String_FirstSeenRaw
	696
	697	static void String_LastSeenRaw(master_record_t r, char string) {
	698
	699	/* snprintf does write \0, and the max is INCL the terminating \0 */
	700	snprintf(string, MAX_STRING_LENGTH, "%u.%03u", r->last, r->msec_last);
	701
	702	} // End of String_LastSeenRaw
	703
	704
	705	#ifdef NSEL
	706	static void String_EventTime(master_record_t r, char string) {
	707	time_t tt;
	708	struct tm * ts;
	709	char *s;
	710
	711	tt = r->event_time / 1000LL;
	712	ts = localtime(&tt);
	713	strftime(string, MAX_STRING_LENGTH-1, "%Y-%m-%d %H:%M:%S", ts);
	714	s = string + strlen(string);
	715	snprintf(s, MAX_STRING_LENGTH-strlen(string)-1,".%03llu", r->event_time % 1000LL);
	716	string[MAX_STRING_LENGTH-1] = '\0';
	717
	718	} // End of String_EventTime
	719	#endif
	720
	721	static void String_Duration(master_record_t r, char string) {
	722
	723	snprintf(string, MAX_STRING_LENGTH-1 ,"%9.3f", duration);
	724	string[MAX_STRING_LENGTH-1] = '\0';
	725
	726	} // End of String_Duration
	727
	728	static void String_Protocol(master_record_t r, char string) {
	729
	730	snprintf(string, MAX_STRING_LENGTH-1 ,"%-5s", ProtoString(r->prot, printPlain));
	731	string[MAX_STRING_LENGTH-1] = '\0';
	732
	733	} // End of String_Protocol
	734
	735	static void String_SrcAddr(master_record_t r, char string) {
	736	char tmp_str[IP_STRING_LEN];
	737
	738	tmp_str[0] = 0;
	739	if ( (r->flags & FLAG_IPV6_ADDR ) != 0 ) { // IPv6
	740	uint64_t ip[2];
	741
	742	ip[0] = htonll(r->V6.srcaddr[0]);
	743	ip[1] = htonll(r->V6.srcaddr[1]);
	744	inet_ntop(AF_INET6, ip, tmp_str, sizeof(tmp_str));
	745	if ( ! long_v6 ) {
	746	CondenseV6(tmp_str);
	747	}
	748	} else { // IPv4
	749	uint32_t ip;
	750	ip = htonl(r->V4.srcaddr);
	751	inet_ntop(AF_INET, &ip, tmp_str, sizeof(tmp_str));
	752	}
	753	tmp_str[IP_STRING_LEN-1] = 0;
	754	if ( long_v6 )
	755	snprintf(string, MAX_STRING_LENGTH-1, "%s%39s", tag_string, tmp_str);
	756	else
	757	snprintf(string, MAX_STRING_LENGTH-1, "%s%16s", tag_string, tmp_str);
	758
	759	string[MAX_STRING_LENGTH-1] = 0;
	760
	761
	762	} // End of String_SrcAddr
	763
	764	static void String_SrcAddrPort(master_record_t r, char string) {
	765	char tmp_str[IP_STRING_LEN], portchar;
	766
	767	tmp_str[0] = 0;
	768	if ( (r->flags & FLAG_IPV6_ADDR ) != 0 ) { // IPv6
	769	uint64_t ip[2];
	770
	771	ip[0] = htonll(r->V6.srcaddr[0]);
	772	ip[1] = htonll(r->V6.srcaddr[1]);
	773	inet_ntop(AF_INET6, ip, tmp_str, sizeof(tmp_str));
	774	if ( ! long_v6 ) {
	775	CondenseV6(tmp_str);
	776	}
	777	portchar = '.';
	778	} else { // IPv4
	779	uint32_t ip;
	780	ip = htonl(r->V4.srcaddr);
	781	inet_ntop(AF_INET, &ip, tmp_str, sizeof(tmp_str));
	782	portchar = ':';
	783	}
	784	tmp_str[IP_STRING_LEN-1] = 0;
	785
	786	if ( long_v6 )
	787	snprintf(string, MAX_STRING_LENGTH-1, "%s%39s%c%-5i", tag_string, tmp_str, portchar, r->srcport);
	788	else
	789	snprintf(string, MAX_STRING_LENGTH-1, "%s%16s%c%-5i", tag_string, tmp_str, portchar, r->srcport);
	790
	791	string[MAX_STRING_LENGTH-1] = 0;
	792
	793	} // End of String_SrcAddrPort
	794
	795	static void String_DstAddr(master_record_t r, char string) {
	796	char tmp_str[IP_STRING_LEN];
	797
	798	tmp_str[0] = 0;
	799	if ( (r->flags & FLAG_IPV6_ADDR ) != 0 ) { // IPv6
	800	uint64_t ip[2];
	801
	802	ip[0] = htonll(r->V6.dstaddr[0]);
	803	ip[1] = htonll(r->V6.dstaddr[1]);
	804	inet_ntop(AF_INET6, ip, tmp_str, sizeof(tmp_str));
	805	if ( ! long_v6 ) {
	806	CondenseV6(tmp_str);
	807	}
	808	} else { // IPv4
	809	uint32_t ip;
	810	ip = htonl(r->V4.dstaddr);
	811	inet_ntop(AF_INET, &ip, tmp_str, sizeof(tmp_str));
	812	}
	813	tmp_str[IP_STRING_LEN-1] = 0;
	814	if ( long_v6 )
	815	snprintf(string, MAX_STRING_LENGTH-1, "%s%39s", tag_string, tmp_str);
	816	else
	817	snprintf(string, MAX_STRING_LENGTH-1, "%s%16s", tag_string, tmp_str);
	818
	819	string[MAX_STRING_LENGTH-1] = 0;
	820
	821
	822	} // End of String_DstAddr
	823
	824
	825	static void String_NextHop(master_record_t r, char string) {
	826	char tmp_str[IP_STRING_LEN];
	827
	828	tmp_str[0] = 0;
	829	if ( (r->flags & FLAG_IPV6_NH ) != 0 ) { // IPv6
	830	uint64_t ip[2];
	831
	832	ip[0] = htonll(r->ip_nexthop.V6[0]);
	833	ip[1] = htonll(r->ip_nexthop.V6[1]);
	834	inet_ntop(AF_INET6, ip, tmp_str, sizeof(tmp_str));
	835	if ( ! long_v6 ) {
	836	CondenseV6(tmp_str);
	837	}
	838	} else { // IPv4
	839	uint32_t ip;
	840	ip = htonl(r->ip_nexthop.V4);
	841	inet_ntop(AF_INET, &ip, tmp_str, sizeof(tmp_str));
	842	}
	843	tmp_str[IP_STRING_LEN-1] = 0;
	844	if ( long_v6 )
	845	snprintf(string, MAX_STRING_LENGTH-1, "%s%39s", tag_string, tmp_str);
	846	else
	847	snprintf(string, MAX_STRING_LENGTH-1, "%s%16s", tag_string, tmp_str);
	848
	849	string[MAX_STRING_LENGTH-1] = 0;
	850
	851
	852	} // End of String_NextHop
	853
	854	static void String_BGPNextHop(master_record_t r, char string) {
	855	char tmp_str[IP_STRING_LEN];
	856
	857	tmp_str[0] = 0;
	858	if ( (r->flags & FLAG_IPV6_NHB ) != 0 ) { // IPv6
	859	uint64_t ip[2];
	860
	861	ip[0] = htonll(r->bgp_nexthop.V6[0]);
	862	ip[1] = htonll(r->bgp_nexthop.V6[1]);
	863	inet_ntop(AF_INET6, ip, tmp_str, sizeof(tmp_str));
	864	if ( ! long_v6 ) {
	865	CondenseV6(tmp_str);
	866	}
	867	} else { // IPv4
	868	uint32_t ip;
	869	ip = htonl(r->bgp_nexthop.V4);
	870	inet_ntop(AF_INET, &ip, tmp_str, sizeof(tmp_str));
	871	}
	872	tmp_str[IP_STRING_LEN-1] = 0;
	873	if ( long_v6 )
	874	snprintf(string, MAX_STRING_LENGTH-1, "%s%39s", tag_string, tmp_str);
	875	else
	876	snprintf(string, MAX_STRING_LENGTH-1, "%s%16s", tag_string, tmp_str);
	877
	878	string[MAX_STRING_LENGTH-1] = 0;
	879
	880
	881	} // End of String_NextHop
	882
	883	static void String_RouterIP(master_record_t r, char string) {
	884	char tmp_str[IP_STRING_LEN];
	885
	886	tmp_str[0] = 0;
	887	if ( (r->flags & FLAG_IPV6_EXP ) != 0 ) { // IPv6
	888	uint64_t ip[2];
	889
	890	ip[0] = htonll(r->ip_router.V6[0]);
	891	ip[1] = htonll(r->ip_router.V6[1]);
	892	inet_ntop(AF_INET6, ip, tmp_str, sizeof(tmp_str));
	893	if ( ! long_v6 ) {
	894	CondenseV6(tmp_str);
	895	}
	896	} else { // IPv4
	897	uint32_t ip;
	898	ip = htonl(r->ip_router.V4);
	899	inet_ntop(AF_INET, &ip, tmp_str, sizeof(tmp_str));
	900	}
	901	tmp_str[IP_STRING_LEN-1] = 0;
	902	if ( long_v6 )
	903	snprintf(string, MAX_STRING_LENGTH-1, "%s%39s", tag_string, tmp_str);
	904	else
	905	snprintf(string, MAX_STRING_LENGTH-1, "%s%16s", tag_string, tmp_str);
	906
	907	string[MAX_STRING_LENGTH-1] = 0;
	908
	909
	910	} // End of String_RouterIP
	911
	912
	913	static void String_DstAddrPort(master_record_t r, char string) {
	914	char tmp_str[IP_STRING_LEN], portchar;
	915
	916	tmp_str[0] = 0;
	917	if ( (r->flags & FLAG_IPV6_ADDR ) != 0 ) { // IPv6
	918	uint64_t ip[2];
	919
	920	ip[0] = htonll(r->V6.dstaddr[0]);
	921	ip[1] = htonll(r->V6.dstaddr[1]);
	922	inet_ntop(AF_INET6, ip, tmp_str, sizeof(tmp_str));
	923	if ( ! long_v6 ) {
	924	CondenseV6(tmp_str);
	925	}
	926	portchar = '.';
	927	} else { // IPv4
	928	uint32_t ip;
	929	ip = htonl(r->V4.dstaddr);
	930	inet_ntop(AF_INET, &ip, tmp_str, sizeof(tmp_str));
	931	portchar = ':';
	932	}
	933	tmp_str[IP_STRING_LEN-1] = 0;
	934
	935	if ( long_v6 )
	936	snprintf(string, MAX_STRING_LENGTH-1, "%s%39s%c%-5s", tag_string, tmp_str, portchar, ICMP_Port_decode(r));
	937	else
	938	snprintf(string, MAX_STRING_LENGTH-1, "%s%16s%c%-5s", tag_string, tmp_str, portchar, ICMP_Port_decode(r));
	939
	940	string[MAX_STRING_LENGTH-1] = 0;
	941
	942	} // End of String_DstAddrPort
	943
	944	static void String_SrcNet(master_record_t r, char string) {
	945	char tmp_str[IP_STRING_LEN];
	946
	947	ApplyNetMaskBits(r, 1);
	948
	949	tmp_str[0] = 0;
	950	if ( (r->flags & FLAG_IPV6_ADDR ) != 0 ) { // IPv6
	951	uint64_t ip[2];
	952
	953	ip[0] = htonll(r->V6.srcaddr[0]);
	954	ip[1] = htonll(r->V6.srcaddr[1]);
	955	inet_ntop(AF_INET6, ip, tmp_str, sizeof(tmp_str));
	956	if ( ! long_v6 ) {
	957	CondenseV6(tmp_str);
	958	}
	959	} else { // IPv4
	960	uint32_t ip;
	961	ip = htonl(r->V4.srcaddr);
	962	inet_ntop(AF_INET, &ip, tmp_str, sizeof(tmp_str));
	963	}
	964	tmp_str[IP_STRING_LEN-1] = 0;
	965	if ( long_v6 )
	966	snprintf(string, MAX_STRING_LENGTH-1, "%s%39s/%-2u", tag_string, tmp_str, r->src_mask );
	967	else
	968	snprintf(string, MAX_STRING_LENGTH-1, "%s%16s/%-2u", tag_string, tmp_str, r->src_mask );
	969
	970	string[MAX_STRING_LENGTH-1] = 0;
	971
	972
	973	} // End of String_SrcNet
	974
	975	static void String_DstNet(master_record_t r, char string) {
	976	char tmp_str[IP_STRING_LEN];
	977
	978	ApplyNetMaskBits(r, 2);
	979
	980	tmp_str[0] = 0;
	981	if ( (r->flags & FLAG_IPV6_ADDR ) != 0 ) { // IPv6
	982	uint64_t ip[2];
	983
	984	ip[0] = htonll(r->V6.dstaddr[0]);
	985	ip[1] = htonll(r->V6.dstaddr[1]);
	986	inet_ntop(AF_INET6, ip, tmp_str, sizeof(tmp_str));
	987	if ( ! long_v6 ) {
	988	CondenseV6(tmp_str);
	989	}
	990	} else { // IPv4
	991	uint32_t ip;
	992	ip = htonl(r->V4.dstaddr);
	993	inet_ntop(AF_INET, &ip, tmp_str, sizeof(tmp_str));
	994	}
	995	tmp_str[IP_STRING_LEN-1] = 0;
	996	if ( long_v6 )
	997	snprintf(string, MAX_STRING_LENGTH-1, "%s%39s/%-2u", tag_string, tmp_str, r->dst_mask );
	998	else
	999	snprintf(string, MAX_STRING_LENGTH-1, "%s%16s/%-2u", tag_string, tmp_str, r->dst_mask );
	1000
	1001	string[MAX_STRING_LENGTH-1] = 0;
	1002
	1003
	1004	} // End of String_DstNet
	1005
	1006	static void String_SrcPort(master_record_t r, char string) {
	1007
	1008	snprintf(string, MAX_STRING_LENGTH-1 ,"%6u", r->srcport);
	1009	string[MAX_STRING_LENGTH-1] = '\0';
	1010
	1011	} // End of String_SrcPort
	1012
	1013	static void String_DstPort(master_record_t r, char string) {
	1014
	1015	snprintf(string, MAX_STRING_LENGTH-1 ,"%6s", ICMP_Port_decode(r));
	1016	string[MAX_STRING_LENGTH-1] = '\0';
	1017
	1018	} // End of String_DstPort
	1019
	1020	static void String_ICMP_type(master_record_t r, char string) {
	1021	uint8_t type;
	1022
	1023	type = ( r->prot == IPPROTO_ICMP \|\| r->prot == IPPROTO_ICMPV6 ) ? r->icmp_type : 0;
	1024	snprintf(string, MAX_STRING_LENGTH-1, "%6u", type);
	1025	string[MAX_STRING_LENGTH-1] = 0;
	1026
	1027	} // End of String_ICMP_type
	1028
	1029	static void String_ICMP_code(master_record_t r, char string) {
	1030	uint8_t code;
	1031
	1032	code = ( r->prot == IPPROTO_ICMP \|\| r->prot == IPPROTO_ICMPV6 ) ? r->icmp_code : 0;
	1033	snprintf(string, MAX_STRING_LENGTH-1, "%6u", code);
	1034	string[MAX_STRING_LENGTH-1] = 0;
	1035
	1036	} // End of String_ICMP_code
	1037
	1038	static void String_SrcAS(master_record_t r, char string) {
	1039
	1040	snprintf(string, MAX_STRING_LENGTH-1 ,"%6u", r->srcas);
	1041	string[MAX_STRING_LENGTH-1] = '\0';
	1042
	1043	} // End of String_SrcAS
	1044
	1045	static void String_DstAS(master_record_t r, char string) {
	1046
	1047	snprintf(string, MAX_STRING_LENGTH-1 ,"%6u", r->dstas);
	1048	string[MAX_STRING_LENGTH-1] = '\0';
	1049
	1050	} // End of String_DstAS
	1051
	1052	static void String_NextAS(master_record_t r, char string) {
	1053
	1054	snprintf(string, MAX_STRING_LENGTH-1 ," %6u", r->bgpNextAdjacentAS);
	1055	string[MAX_STRING_LENGTH-1] = '\0';
	1056
	1057	} // End of String_NextAS
	1058
	1059	static void String_PrevAS(master_record_t r, char string) {
	1060
	1061	snprintf(string, MAX_STRING_LENGTH-1 ," %6u", r->bgpPrevAdjacentAS);
	1062	string[MAX_STRING_LENGTH-1] = '\0';
	1063
	1064	} // End of String_PrevAS
	1065
	1066	static void String_Input(master_record_t r, char string) {
	1067
	1068	snprintf(string, MAX_STRING_LENGTH-1 ,"%6u", r->input);
	1069	string[MAX_STRING_LENGTH-1] = '\0';
	1070
	1071	} // End of String_Input
	1072
	1073	static void String_Output(master_record_t r, char string) {
	1074
	1075	snprintf(string, MAX_STRING_LENGTH-1 ,"%6u", r->output);
	1076	string[MAX_STRING_LENGTH-1] = '\0';
	1077
	1078	} // End of String_Output
	1079
	1080	static void String_InPackets(master_record_t r, char string) {
	1081	char s[NUMBER_STRING_SIZE];
	1082
	1083	format_number(r->dPkts, s, printPlain, FIXED_WIDTH);
	1084	snprintf(string, MAX_STRING_LENGTH-1 ,"%8s", s);
	1085	string[MAX_STRING_LENGTH-1] = '\0';
	1086
	1087	} // End of String_InPackets
	1088
	1089	static void String_OutPackets(master_record_t r, char string) {
	1090	char s[NUMBER_STRING_SIZE];
	1091
	1092	format_number(r->out_pkts, s, printPlain, FIXED_WIDTH);
	1093	snprintf(string, MAX_STRING_LENGTH-1 ,"%8s", s);
	1094	string[MAX_STRING_LENGTH-1] = '\0';
	1095
	1096	} // End of String_OutPackets
	1097
	1098	static void String_InBytes(master_record_t r, char string) {
	1099	char s[NUMBER_STRING_SIZE];
	1100
	1101	format_number(r->dOctets, s, printPlain, FIXED_WIDTH);
	1102	snprintf(string, MAX_STRING_LENGTH-1 ,"%8s", s);
	1103	string[MAX_STRING_LENGTH-1] = '\0';
	1104
	1105	} // End of String_InBytes
	1106
	1107	static void String_OutBytes(master_record_t r, char string) {
	1108	char s[NUMBER_STRING_SIZE];
	1109
	1110	format_number(r->out_bytes, s, printPlain, FIXED_WIDTH);
	1111	snprintf(string, MAX_STRING_LENGTH-1 ,"%8s", s);
	1112	string[MAX_STRING_LENGTH-1] = '\0';
	1113
	1114	} // End of String_OutBytes
	1115
	1116	static void String_Flows(master_record_t r, char string) {
	1117
	1118	// snprintf(string, MAX_STRING_LENGTH-1 ,"%5llu", r->aggr_flows ? (unsigned long long)r->aggr_flows : 1 );
	1119	snprintf(string, MAX_STRING_LENGTH-1 ,"%5llu", (unsigned long long)r->aggr_flows );
	1120	string[MAX_STRING_LENGTH-1] = '\0';
	1121
	1122	} // End of String_Flows
	1123
	1124	static void String_Tos(master_record_t r, char string) {
	1125
	1126	snprintf(string, MAX_STRING_LENGTH-1 ,"%3u", r->tos);
	1127	string[MAX_STRING_LENGTH-1] = '\0';
	1128
	1129	} // End of String_Tos
	1130
	1131	static void String_SrcTos(master_record_t r, char string) {
	1132
	1133	snprintf(string, MAX_STRING_LENGTH-1 ,"%4u", r->tos);
	1134	string[MAX_STRING_LENGTH-1] = '\0';
	1135
	1136	} // End of String_SrcTos
	1137
	1138	static void String_DstTos(master_record_t r, char string) {
	1139
	1140	snprintf(string, MAX_STRING_LENGTH-1 ,"%4u", r->dst_tos);
	1141	string[MAX_STRING_LENGTH-1] = '\0';
	1142
	1143	} // End of String_DstTos
	1144
	1145	static void String_SrcMask(master_record_t r, char string) {
	1146
	1147	snprintf(string, MAX_STRING_LENGTH-1 ,"%5u", r->src_mask);
	1148	string[MAX_STRING_LENGTH-1] = '\0';
	1149
	1150	} // End of String_SrcMask
	1151
	1152	static void String_DstMask(master_record_t r, char string) {
	1153
	1154	snprintf(string, MAX_STRING_LENGTH-1 ,"%5u", r->dst_mask);
	1155	string[MAX_STRING_LENGTH-1] = '\0';
	1156
	1157	} // End of String_DstMask
	1158
	1159	static void String_SrcVlan(master_record_t r, char string) {
	1160
	1161	snprintf(string, MAX_STRING_LENGTH-1 ,"%5u", r->src_vlan);
	1162	string[MAX_STRING_LENGTH-1] = '\0';
	1163
	1164	} // End of String_SrcVlan
	1165
	1166	static void String_DstVlan(master_record_t r, char string) {
	1167
	1168	snprintf(string, MAX_STRING_LENGTH-1 ,"%5u", r->dst_vlan);
	1169	string[MAX_STRING_LENGTH-1] = '\0';
	1170
	1171	} // End of String_DstVlan
	1172
	1173	static void String_Dir(master_record_t r, char string) {
	1174
	1175	snprintf(string, MAX_STRING_LENGTH-1 ,"%3c", r->dir ? 'E' : 'I' );
	1176	string[MAX_STRING_LENGTH-1] = '\0';
	1177
	1178	} // End of String_Dir
	1179
	1180	static void String_FwdStatus(master_record_t r, char string) {
	1181
	1182	snprintf(string, MAX_STRING_LENGTH-1 ,"%3u", r->fwd_status);
	1183	string[MAX_STRING_LENGTH-1] = '\0';
	1184
	1185	} // End of String_FwdStatus
	1186
	1187	static void String_Flags(master_record_t r, char string) {
	1188
	1189	snprintf(string, MAX_STRING_LENGTH-1 ,"%8s", FlagsString(r->tcp_flags));
	1190	string[MAX_STRING_LENGTH-1] = '\0';
	1191
	1192	} // End of String_Flags
	1193
	1194	static void String_InSrcMac(master_record_t r, char string) {
	1195	int i;
	1196	uint8_t mac[6];
	1197
	1198	for ( i=0; i<6; i++ ) {
	1199	mac[i] = (r->in_src_mac >> ( i*8 )) & 0xFF;
	1200	}
	1201	snprintf(string, MAX_STRING_LENGTH-1 ,"%.2x:%.2x:%.2x:%.2x:%.2x:%.2x", mac[5], mac[4], mac[3], mac[2], mac[1], mac[0]);
	1202	string[MAX_STRING_LENGTH-1] = '\0';
	1203
	1204	} // End of String_InSrcMac
	1205
	1206	static void String_OutDstMac(master_record_t r, char string) {
	1207	int i;
	1208	uint8_t mac[6];
	1209
	1210	for ( i=0; i<6; i++ ) {
	1211	mac[i] = (r->out_dst_mac >> ( i*8 )) & 0xFF;
	1212	}
	1213	snprintf(string, MAX_STRING_LENGTH-1 ,"%.2x:%.2x:%.2x:%.2x:%.2x:%.2x", mac[5], mac[4], mac[3], mac[2], mac[1], mac[0]);
	1214	string[MAX_STRING_LENGTH-1] = '\0';
	1215
	1216	} // End of String_OutDstMac
	1217
	1218	static void String_InDstMac(master_record_t r, char string) {
	1219	int i;
	1220	uint8_t mac[6];
	1221
	1222	for ( i=0; i<6; i++ ) {
	1223	mac[i] = (r->in_dst_mac >> ( i*8 )) & 0xFF;
	1224	}
	1225	snprintf(string, MAX_STRING_LENGTH-1 ,"%.2x:%.2x:%.2x:%.2x:%.2x:%.2x", mac[5], mac[4], mac[3], mac[2], mac[1], mac[0]);
	1226	string[MAX_STRING_LENGTH-1] = '\0';
	1227
	1228	} // End of String_InDstMac
	1229
	1230	static void String_OutSrcMac(master_record_t r, char string) {
	1231	int i;
	1232	uint8_t mac[6];
	1233
	1234	for ( i=0; i<6; i++ ) {
	1235	mac[i] = (r->out_src_mac >> ( i*8 )) & 0xFF;
	1236	}
	1237	snprintf(string, MAX_STRING_LENGTH-1 ,"%.2x:%.2x:%.2x:%.2x:%.2x:%.2x", mac[5], mac[4], mac[3], mac[2], mac[1], mac[0]);
	1238	string[MAX_STRING_LENGTH-1] = '\0';
	1239
	1240	} // End of String_OutSrcMac
	1241
	1242	static void String_MPLS_1(master_record_t r, char string) {
	1243
	1244	snprintf(string, MAX_STRING_LENGTH-1 ,"%8u-%1u-%1u",
	1245	r->mpls_label[0] >> 4 , (r->mpls_label[0] & 0xF ) >> 1, r->mpls_label[0] & 1);
	1246	string[MAX_STRING_LENGTH-1] = '\0';
	1247
	1248	} // End of String_MPLS
	1249
	1250	static void String_MPLS_2(master_record_t r, char string) {
	1251
	1252	snprintf(string, MAX_STRING_LENGTH-1 ,"%8u-%1u-%1u",
	1253	r->mpls_label[1] >> 4 , (r->mpls_label[1] & 0xF ) >> 1, r->mpls_label[1] & 1);
	1254	string[MAX_STRING_LENGTH-1] = '\0';
	1255
	1256	} // End of String_MPLS
	1257
	1258	static void String_MPLS_3(master_record_t r, char string) {
	1259
	1260	snprintf(string, MAX_STRING_LENGTH-1 ,"%8u-%1u-%1u",
	1261	r->mpls_label[2] >> 4 , (r->mpls_label[2] & 0xF ) >> 1, r->mpls_label[2] & 1);
	1262	string[MAX_STRING_LENGTH-1] = '\0';
	1263
	1264	} // End of String_MPLS
	1265
	1266	static void String_MPLS_4(master_record_t r, char string) {
	1267
	1268	snprintf(string, MAX_STRING_LENGTH-1 ,"%8u-%1u-%1u",
	1269	r->mpls_label[3] >> 4 , (r->mpls_label[3] & 0xF ) >> 1, r->mpls_label[3] & 1);
	1270	string[MAX_STRING_LENGTH-1] = '\0';
	1271
	1272	} // End of String_MPLS
	1273
	1274	static void String_MPLS_5(master_record_t r, char string) {
	1275
	1276	snprintf(string, MAX_STRING_LENGTH-1 ,"%8u-%1u-%1u",
	1277	r->mpls_label[4] >> 4 , (r->mpls_label[4] & 0xF ) >> 1, r->mpls_label[4] & 1);
	1278	string[MAX_STRING_LENGTH-1] = '\0';
	1279
	1280	} // End of String_MPLS
	1281
	1282	static void String_MPLS_6(master_record_t r, char string) {
	1283
	1284	snprintf(string, MAX_STRING_LENGTH-1 ,"%8u-%1u-%1u",
	1285	r->mpls_label[5] >> 4 , (r->mpls_label[5] & 0xF ) >> 1, r->mpls_label[5] & 1);
	1286	string[MAX_STRING_LENGTH-1] = '\0';
	1287
	1288	} // End of String_MPLS
	1289
	1290	static void String_MPLS_7(master_record_t r, char string) {
	1291
	1292	snprintf(string, MAX_STRING_LENGTH-1 ,"%8u-%1u-%1u",
	1293	r->mpls_label[6] >> 4 , (r->mpls_label[6] & 0xF ) >> 1, r->mpls_label[6] & 1);
	1294	string[MAX_STRING_LENGTH-1] = '\0';
	1295
	1296	} // End of String_MPLS
	1297
	1298	static void String_MPLS_8(master_record_t r, char string) {
	1299
	1300	snprintf(string, MAX_STRING_LENGTH-1 ,"%8u-%1u-%1u",
	1301	r->mpls_label[7] >> 4 , (r->mpls_label[7] & 0xF ) >> 1, r->mpls_label[7] & 1);
	1302	string[MAX_STRING_LENGTH-1] = '\0';
	1303
	1304	} // End of String_MPLS
	1305
	1306	static void String_MPLS_9(master_record_t r, char string) {
	1307
	1308	snprintf(string, MAX_STRING_LENGTH-1 ,"%8u-%1u-%1u",
	1309	r->mpls_label[8] >> 4 , (r->mpls_label[8] & 0xF ) >> 1, r->mpls_label[8] & 1);
	1310	string[MAX_STRING_LENGTH-1] = '\0';
	1311
	1312	} // End of String_MPLS
	1313
	1314	static void String_MPLS_10(master_record_t r, char string) {
	1315
	1316	snprintf(string, MAX_STRING_LENGTH-1 ,"%8u-%1u-%1u",
	1317	r->mpls_label[9] >> 4 , (r->mpls_label[9] & 0xF ) >> 1, r->mpls_label[9] & 1);
	1318	string[MAX_STRING_LENGTH-1] = '\0';
	1319
	1320	} // End of String_MPLS
	1321
	1322	static void String_MPLSs(master_record_t r, char string) {
	1323
	1324	snprintf(string, MAX_STRING_LENGTH-1 ,"%8u-%1u-%1u %8u-%1u-%1u %8u-%1u-%1u %8u-%1u-%1u %8u-%1u-%1u %8u-%1u-%1u %8u-%1u-%1u %8u-%1u-%1u %8u-%1u-%1u %8u-%1u-%1u ",
	1325	r->mpls_label[0] >> 4 , (r->mpls_label[0] & 0xF ) >> 1, r->mpls_label[0] & 1,
	1326	r->mpls_label[1] >> 4 , (r->mpls_label[1] & 0xF ) >> 1, r->mpls_label[1] & 1,
	1327	r->mpls_label[2] >> 4 , (r->mpls_label[2] & 0xF ) >> 1, r->mpls_label[2] & 1,
	1328	r->mpls_label[3] >> 4 , (r->mpls_label[3] & 0xF ) >> 1, r->mpls_label[3] & 1,
	1329	r->mpls_label[4] >> 4 , (r->mpls_label[4] & 0xF ) >> 1, r->mpls_label[4] & 1,
	1330	r->mpls_label[5] >> 4 , (r->mpls_label[5] & 0xF ) >> 1, r->mpls_label[5] & 1,
	1331	r->mpls_label[6] >> 4 , (r->mpls_label[6] & 0xF ) >> 1, r->mpls_label[6] & 1,
	1332	r->mpls_label[7] >> 4 , (r->mpls_label[7] & 0xF ) >> 1, r->mpls_label[7] & 1,
	1333	r->mpls_label[8] >> 4 , (r->mpls_label[8] & 0xF ) >> 1, r->mpls_label[8] & 1,
	1334	r->mpls_label[9] >> 4 , (r->mpls_label[9] & 0xF ) >> 1, r->mpls_label[9] & 1
	1335	);
	1336	string[MAX_STRING_LENGTH-1] = '\0';
	1337
	1338	} // End of String_MPLSs
	1339
	1340	static void String_Engine(master_record_t r, char string) {
	1341
	1342	snprintf(string, MAX_STRING_LENGTH-1 ,"%3u/%-3u", r->engine_type, r->engine_id);
	1343	string[MAX_STRING_LENGTH-1] = '\0';
	1344
	1345	} // End of String_Engine
	1346
	1347	static void String_Label(master_record_t r, char string) {
	1348
	1349	if ( r->label )
	1350	snprintf(string, MAX_STRING_LENGTH-1 ,"%16s", r->label);
	1351	else
	1352	snprintf(string, MAX_STRING_LENGTH-1 ,"<none>");
	1353
	1354	string[MAX_STRING_LENGTH-1] = '\0';
	1355
	1356	} // End of String_Label
	1357
	1358	static void String_ClientLatency(master_record_t r, char string) {
	1359	double latency;
	1360
	1361	latency = (double)r->client_nw_delay_usec / 1000.0;
	1362	snprintf(string, MAX_STRING_LENGTH-1 ,"%9.3f", latency);
	1363	string[MAX_STRING_LENGTH-1] = '\0';
	1364
	1365	} // End of String_ClientLatency
	1366
	1367	static void String_ServerLatency(master_record_t r, char string) {
	1368	double latency;
	1369
	1370	latency = (double)r->server_nw_delay_usec / 1000.0;
	1371	snprintf(string, MAX_STRING_LENGTH-1 ,"%9.3f", latency);
	1372	string[MAX_STRING_LENGTH-1] = '\0';
	1373
	1374	} // End of String_ServerLatency
	1375
	1376	static void String_AppLatency(master_record_t r, char string) {
	1377	double latency;
	1378
	1379	latency = (double)r->appl_latency_usec / 1000.0;
	1380	snprintf(string, MAX_STRING_LENGTH-1 ,"%9.3f", latency);
	1381	string[MAX_STRING_LENGTH-1] = '\0';
	1382
	1383	} // End of String_AppLatency
	1384
	1385	static void String_bps(master_record_t r, char string) {
	1386	uint64_t bps;
	1387	char s[NUMBER_STRING_SIZE];
	1388
	1389	if ( duration ) {
	1390	bps = (( r->dOctets << 3 ) / duration); // bits per second. ( >> 3 ) -> * 8 to convert octets into bits
	1391	} else {
	1392	bps = 0;
	1393	}
	1394	format_number(bps, s, printPlain, FIXED_WIDTH);
	1395	snprintf(string, MAX_STRING_LENGTH-1 ,"%8s", s);
	1396	string[MAX_STRING_LENGTH-1] = '\0';
	1397
	1398	} // End of String_bps
	1399
	1400	static void String_pps(master_record_t r, char string) {
	1401	uint64_t pps;
	1402	char s[NUMBER_STRING_SIZE];
	1403
	1404	if ( duration ) {
	1405	pps = r->dPkts / duration; // packets per second
	1406	} else {
	1407	pps = 0;
	1408	}
	1409	format_number(pps, s, printPlain, FIXED_WIDTH);
	1410	snprintf(string, MAX_STRING_LENGTH-1 ,"%8s", s);
	1411	string[MAX_STRING_LENGTH-1] = '\0';
	1412
	1413	} // End of String_Duration
	1414
	1415	static void String_bpp(master_record_t r, char string) {
	1416	uint32_t Bpp;
	1417
	1418	string[MAX_STRING_LENGTH-1] = '\0';
	1419
	1420	if ( r->dPkts )
	1421	Bpp = r->dOctets / r->dPkts; // Bytes per Packet
	1422	else
	1423	Bpp = 0;
	1424	snprintf(string, MAX_STRING_LENGTH-1 ,"%6u", Bpp);
	1425	string[MAX_STRING_LENGTH-1] = '\0';
	1426
	1427	} // End of String_bpp
	1428
	1429	static void String_ExpSysID(master_record_t r, char string) {
	1430
	1431	string[MAX_STRING_LENGTH-1] = '\0';
	1432
	1433	snprintf(string, MAX_STRING_LENGTH-1 ,"%6u", r->exporter_sysid);
	1434	string[MAX_STRING_LENGTH-1] = '\0';
	1435
	1436	} // End of String_ExpSysID
	1437
	1438	#ifdef NSEL
	1439	static void String_nfc(master_record_t r, char string) {
	1440
	1441	snprintf(string, MAX_STRING_LENGTH-1, "%10u", r->conn_id);
	1442	string[MAX_STRING_LENGTH-1] = '\0';
	1443
	1444	} // End of String_nfc
	1445
	1446	static void String_evt(master_record_t r, char string) {
	1447
	1448	if (r->fw_xevent) {
	1449	snprintf(string, MAX_STRING_LENGTH-1 ,"%7s", FwEventString(r->event));
	1450	} else {
	1451	snprintf(string, MAX_STRING_LENGTH-1 ,"%7s", EventString(r->event));
	1452	}
	1453
	1454	} // End of String_evt
	1455
	1456	static void String_xevt(master_record_t r, char string) {
	1457
	1458	snprintf(string, MAX_STRING_LENGTH-1 ,"%7s", EventXString(r->fw_xevent));
	1459
	1460	} // End of String_xevt
	1461
	1462	static void String_sgt(master_record_t r, char string) {
	1463
	1464	snprintf(string, MAX_STRING_LENGTH-1 ,"%5u", r->sec_group_tag);
	1465
	1466	} // End of String_sgt
	1467
	1468	static void String_msec(master_record_t r, char string) {
	1469	unsigned long long etime;
	1470
	1471	etime = 1000LL * (unsigned long long)r->first + (unsigned long long)r->msec_first;
	1472	snprintf(string, MAX_STRING_LENGTH-1,"%13llu", etime);
	1473	string[MAX_STRING_LENGTH-1] = '\0';
	1474
	1475	} // End of String_msec
	1476
	1477	static void String_iacl(master_record_t r, char string) {
	1478
	1479	snprintf(string, MAX_STRING_LENGTH-1, "0x%-8x 0x%-8x 0x%-8x",
	1480	r->ingress_acl_id[0], r->ingress_acl_id[1], r->ingress_acl_id[2]);
	1481	string[MAX_STRING_LENGTH-1] = 0;
	1482
	1483	} // End of String_iacl
	1484
	1485	static void String_eacl(master_record_t r, char string) {
	1486
	1487	snprintf(string, MAX_STRING_LENGTH-1, "%10u %10u %10u",
	1488	r->egress_acl_id[0], r->egress_acl_id[1], r->egress_acl_id[2]);
	1489
	1490	string[MAX_STRING_LENGTH-1] = 0;
	1491
	1492	} // End of String_eacl
	1493
	1494	static void String_xlateSrcAddr(master_record_t r, char string) {
	1495	char tmp_str[IP_STRING_LEN];
	1496
	1497	tmp_str[0] = 0;
	1498	if ( (r->xlate_flags & 1 ) != 0 ) { // IPv6
	1499	uint64_t ip[2];
	1500
	1501	ip[0] = htonll(r->xlate_src_ip.V6[0]);
	1502	ip[1] = htonll(r->xlate_src_ip.V6[1]);
	1503	inet_ntop(AF_INET6, ip, tmp_str, sizeof(tmp_str));
	1504	if ( ! long_v6 ) {
	1505	CondenseV6(tmp_str);
	1506	}
	1507	} else { // IPv4
	1508	uint32_t ip;
	1509	ip = htonl(r->xlate_src_ip.V4);
	1510	inet_ntop(AF_INET, &ip, tmp_str, sizeof(tmp_str));
	1511	}
	1512	tmp_str[IP_STRING_LEN-1] = 0;
	1513	if ( long_v6 )
	1514	snprintf(string, MAX_STRING_LENGTH-1, "%s%39s", tag_string, tmp_str);
	1515	else
	1516	snprintf(string, MAX_STRING_LENGTH-1, "%s%16s", tag_string, tmp_str);
	1517
	1518	string[MAX_STRING_LENGTH-1] = 0;
	1519
	1520	} // End of String_xlateSrcAddr
	1521
	1522	static void String_xlateDstAddr(master_record_t r, char string) {
	1523	char tmp_str[IP_STRING_LEN];
	1524
	1525	tmp_str[0] = 0;
	1526	if ( (r->xlate_flags & 1 ) != 0 ) { // IPv6
	1527	uint64_t ip[2];
	1528
	1529	ip[0] = htonll(r->xlate_dst_ip.V6[0]);
	1530	ip[1] = htonll(r->xlate_dst_ip.V6[1]);
	1531	inet_ntop(AF_INET6, ip, tmp_str, sizeof(tmp_str));
	1532	if ( ! long_v6 ) {
	1533	CondenseV6(tmp_str);
	1534	}
	1535	} else { // IPv4
	1536	uint32_t ip;
	1537	ip = htonl(r->xlate_dst_ip.V4);
	1538	inet_ntop(AF_INET, &ip, tmp_str, sizeof(tmp_str));
	1539	}
	1540	tmp_str[IP_STRING_LEN-1] = 0;
	1541	if ( long_v6 )
	1542	snprintf(string, MAX_STRING_LENGTH-1, "%s%39s", tag_string, tmp_str);
	1543	else
	1544	snprintf(string, MAX_STRING_LENGTH-1, "%s%16s", tag_string, tmp_str);
	1545
	1546	string[MAX_STRING_LENGTH-1] = 0;
	1547
	1548	} // End of String_xlateDstAddr
	1549
	1550	static void String_xlateSrcPort(master_record_t r, char string) {
	1551
	1552	snprintf(string, MAX_STRING_LENGTH-1 ,"%6u", r->xlate_src_port);
	1553	string[MAX_STRING_LENGTH-1] = '\0';
	1554
	1555	} // End of String_xlateSrcPort
	1556
	1557	static void String_xlateDstPort(master_record_t r, char string) {
	1558
	1559	snprintf(string, MAX_STRING_LENGTH-1 ,"%6u", r->xlate_dst_port);
	1560	string[MAX_STRING_LENGTH-1] = '\0';
	1561
	1562	} // End of String_xlateDstPort
	1563
	1564	static void String_xlateSrcAddrPort(master_record_t r, char string) {
	1565	char tmp_str[IP_STRING_LEN], portchar;
	1566
	1567	tmp_str[0] = 0;
	1568	if ( (r->xlate_flags & 1 ) != 0 ) { // IPv6
	1569	uint64_t ip[2];
	1570
	1571	ip[0] = htonll(r->xlate_src_ip.V6[0]);
	1572	ip[1] = htonll(r->xlate_src_ip.V6[1]);
	1573	inet_ntop(AF_INET6, ip, tmp_str, sizeof(tmp_str));
	1574	if ( ! long_v6 ) {
	1575	CondenseV6(tmp_str);
	1576	}
	1577
	1578	portchar = '.';
	1579	} else { // IPv4
	1580	uint32_t ip;
	1581	ip = htonl(r->xlate_src_ip.V4);
	1582	inet_ntop(AF_INET, &ip, tmp_str, sizeof(tmp_str));
	1583
	1584	portchar = ':';
	1585	}
	1586	tmp_str[IP_STRING_LEN-1] = 0;
	1587
	1588	if ( long_v6 )
	1589	snprintf(string, MAX_STRING_LENGTH-1, "%s%39s%c%-5i", tag_string, tmp_str, portchar, r->xlate_src_port);
	1590	else
	1591	snprintf(string, MAX_STRING_LENGTH-1, "%s%16s%c%-5i", tag_string, tmp_str, portchar, r->xlate_src_port);
	1592
	1593	string[MAX_STRING_LENGTH-1] = 0;
	1594
	1595	} // End of String_xlateSrcAddrPort
	1596
	1597	static void String_xlateDstAddrPort(master_record_t r, char string) {
	1598	char tmp_str[IP_STRING_LEN], portchar;
	1599
	1600	tmp_str[0] = 0;
	1601	if ( (r->xlate_flags & 1 ) != 0 ) { // IPv6
	1602	uint64_t ip[2];
	1603
	1604	ip[0] = htonll(r->xlate_dst_ip.V6[0]);
	1605	ip[1] = htonll(r->xlate_dst_ip.V6[1]);
	1606	inet_ntop(AF_INET6, ip, tmp_str, sizeof(tmp_str));
	1607	if ( ! long_v6 ) {
	1608	CondenseV6(tmp_str);
	1609	}
	1610
	1611	portchar = '.';
	1612	} else { // IPv4
	1613	uint32_t ip;
	1614	ip = htonl(r->xlate_dst_ip.V4);
	1615	inet_ntop(AF_INET, &ip, tmp_str, sizeof(tmp_str));
	1616
	1617	portchar = ':';
	1618	}
	1619	tmp_str[IP_STRING_LEN-1] = 0;
	1620
	1621	if ( long_v6 )
	1622	snprintf(string, MAX_STRING_LENGTH-1, "%s%39s%c%-5i", tag_string, tmp_str, portchar, r->xlate_dst_port);
	1623	else
	1624	snprintf(string, MAX_STRING_LENGTH-1, "%s%16s%c%-5i", tag_string, tmp_str, portchar, r->xlate_dst_port);
	1625
	1626	string[MAX_STRING_LENGTH-1] = 0;
	1627
	1628
	1629	} // End of String_xlateDstAddrPort
	1630
	1631	static void String_userName(master_record_t r, char string) {
	1632
	1633	if ( r->username[0] == '\0' )
	1634	snprintf(string, MAX_STRING_LENGTH-1 ,"%s", "<empty>");
	1635	else
	1636	snprintf(string, MAX_STRING_LENGTH-1 ,"%s", r->username);
	1637
	1638	string[MAX_STRING_LENGTH-1] = '\0';
	1639
	1640	} // End of String_userName
	1641
	1642	static void String_ivrf(master_record_t r, char string) {
	1643
	1644	snprintf(string, MAX_STRING_LENGTH-1, "%10u", r->ingress_vrfid);
	1645	string[MAX_STRING_LENGTH-1] = '\0';
	1646
	1647	} // End of String_ivrf
	1648
	1649	static void String_evrf(master_record_t r, char string) {
	1650
	1651	snprintf(string, MAX_STRING_LENGTH-1, "%10u", r->egress_vrfid);
	1652	string[MAX_STRING_LENGTH-1] = '\0';
	1653
	1654	} // End of String_evrf
	1655
	1656	static void String_PortBlockStart(master_record_t r, char string) {
	1657
	1658	snprintf(string, MAX_STRING_LENGTH-1 ,"%7u", r->block_start);
	1659	string[MAX_STRING_LENGTH-1] = '\0';
	1660
	1661	} // End of String_PortBlockStart
	1662
	1663	static void String_PortBlockEnd(master_record_t r, char string) {
	1664
	1665	snprintf(string, MAX_STRING_LENGTH-1 ,"%7u", r->block_end);
	1666	string[MAX_STRING_LENGTH-1] = '\0';
	1667
	1668	} // End of String_PortBlockEnd
	1669
	1670	static void String_PortBlockStep(master_record_t r, char string) {
	1671
	1672	snprintf(string, MAX_STRING_LENGTH-1 ,"%7u", r->block_step);
	1673	string[MAX_STRING_LENGTH-1] = '\0';
	1674
	1675	} // End of String_PortBlockStep
	1676
	1677	static void String_PortBlockSize(master_record_t r, char string) {
	1678
	1679	snprintf(string, MAX_STRING_LENGTH-1 ,"%7u", r->block_size);
	1680	string[MAX_STRING_LENGTH-1] = '\0';
	1681
	1682	} // End of String_PortBlockSize
	1683
	1684
	1685	#endif

+62

-0

bin/output_fmt.h less more

	0	/*
	1	* Copyright (c) 2009-2020, Peter Haag
	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
	3	* All rights reserved.
	4	*
	5	* Redistribution and use in source and binary forms, with or without
	6	* modification, are permitted provided that the following conditions are met:
	7	*
	8	* * Redistributions of source code must retain the above copyright notice,
	9	* this list of conditions and the following disclaimer.
	10	* * Redistributions in binary form must reproduce the above copyright notice,
	11	* this list of conditions and the following disclaimer in the documentation
	12	* and/or other materials provided with the distribution.
	13	* * Neither the name of the author nor the names of its contributors may be
	14	* used to endorse or promote products derived from this software without
	15	* specific prior written permission.
	16	*
	17	* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
	18	* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	19	* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	20	* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
	21	* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
	22	* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
	23	* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
	24	* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
	25	* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
	26	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
	27	* POSSIBILITY OF SUCH DAMAGE.
	28	*
	29	*/
	30
	31	#ifndef _OUTPUT_FMT_H
	32	#define _OUTPUT_FMT_H 1
	33
	34	#include "config.h"
	35
	36	#include <sys/types.h>
	37	#include <time.h>
	38
	39	#ifdef HAVE_STDINT_H
	40	#include <stdint.h>
	41	#endif
	42
	43
	44	/* prototypes */
	45
	46	void Setv6Mode(int mode);
	47
	48	int Getv6Mode(void);
	49
	50	void text_prolog(void);
	51
	52	void text_epilog(void);
	53
	54	int ParseOutputFormat(char format, int printPlain, printmap_t printmap);
	55
	56	void format_special(void record, char * s, int tag);
	57
	58	#define TAG_CHAR ''
	59
	60	#endif //_OUTPUT_FMT_H
	61

+35

-30

bin/output_json.c less more

0	0	/*
1		* Copyright (c) 2017, Peter Haag
	1	* Copyright (c) 2019-2020, Peter Haag
2	2	* All rights reserved.
3	3	*
4	4	* Redistribution and use in source and binary forms, with or without

34	34
35	35	#include <stdio.h>
36	36	#include <stddef.h>
	37	#include <stdlib.h>
37	38	#include <sys/types.h>
38	39	#include <sys/socket.h>
39	40	#include <netinet/in.h>
40	41	#include <arpa/inet.h>
41	42	#include <time.h>
42	43	#include <string.h>
43		#include <ctype.h>
44		#include <stdlib.h>
45		#include <errno.h>
46	44
47	45	#ifdef HAVE_STDINT_H
48	46	#include <stdint.h>
49	47	#endif
50	48
	49	#include "util.h"
	50	#include "nfdump.h"
51	51	#include "nffile.h"
52		#include "util.h"
53		#include "nf_common.h"
	52	#include "nfx.h"
	53	#include "output_util.h"
54	54	#include "output_json.h"
55	55
56	56	#define STRINGSIZE 10240
57	57	#define IP_STRING_LEN (INET6_ADDRSTRLEN)
58	58
59		#ifdef NSEL
60		static char *NSEL_event_string[6] = {
61		"IGNORE", "CREATE", "DELETE", "DENIED", "ALERT", "UPDATE"
62		};
63
64		static char *NEL_event_string[3] = {
65		"INVALID", "ADD", "DELETE"
66		};
67		#endif
68
69	59	static char data_string[STRINGSIZE];
	60
	61	// record counter
	62	static uint32_t recordCount;
70	63
71	64	static void String_Flags(master_record_t r, char string) {
72	65

82	75
83	76	} // End of String_Flags
84	77
	78	void json_prolog(void) {
	79	recordCount = 0;
	80	memset(data_string, 0, STRINGSIZE);
	81	printf("[\n");
	82	} // End of json_prolog
	83
	84	void json_epilog(void) {
	85	printf("]\n");
	86	} // End of json_epilog
	87
	88
85	89	void flow_record_to_json(void record, char * s, int tag) {
86	90	char _s, as[IP_STRING_LEN], ds[IP_STRING_LEN], datestr1, *datestr2, datebuff[64], flags_str[16];
87	91	int i, id;

103	107
104	108	String_Flags(record, flags_str);
105	109
106		_s = data_string;
107		slen = STRINGSIZE;
	110	if ( recordCount ) {
	111	strncpy(data_string, ",\n", STRINGSIZE-1);
	112	}
	113	recordCount++;
	114
	115	_slen = strlen(data_string);
	116	_s = data_string + _slen;
	117	slen = STRINGSIZE - _slen;
108	118	snprintf(_s, slen-1, "{\n"
109	119	" \"type\" : \"%s\",\n"
110	120	" \"sampled\" : %u,\n"

391	401	} break;
392	402	#ifdef NSEL
393	403	case EX_NSEL_COMMON: {
394		char *event = "UNKNOWN";
395	404	char *datestr, datebuff[64];
396		if ( r->event <= 5 ) {
397		event = NSEL_event_string[r->event];
398		}
399	405	when = r->event_time / 1000LL;
400	406	ts = localtime(&when);
401	407	strftime(datebuff, 63, "%Y-%m-%dT%H:%M:%S", ts);

405	411	" \"event_id\" : \"%u\",\n"
406	412	" \"event\" : \"%s\",\n"
407	413	" \"xevent_id\" : \"%u\",\n"
	414	" \"sgt_id\" : \"%u\",\n"
408	415	" \"t_event\" : \"%s\",\n"
409		, r->conn_id, r->event, event, r->fw_xevent, datestr);
	416	, r->conn_id, r->event, r->event_flag == FW_EVENT ? FwEventString(r->event) : EventString(r->event)
	417	, r->fw_xevent, r->sec_group_tag, datestr);
410	418	free(datestr);
411	419	} break;
412	420	case EX_NEL_COMMON: {
413		char *event = "UNKNOWN";
414		if ( r->event <= 2 ) {
415		event = NEL_event_string[r->event];
416		}
417	421	snprintf(_s, slen-1,
418	422	" \"nat_event_id\" : \"%u\",\n"
419	423	" \"nat_event\" : \"%s\",\n"
420	424	" \"ingress_vrf\" : \"%u\",\n"
421	425	" \"egress_vrf\" : \"%u\",\n"
422		, r->event, event, r->ingress_vrfid, r->egress_vrfid);
	426	, r->event, r->event_flag == FW_EVENT ? FwEventString(r->event) : EventString(r->event)
	427	, r->ingress_vrfid, r->egress_vrfid);
423	428	} break;
424	429	case EX_NSEL_XLATE_PORTS: {
425	430	snprintf(_s, slen-1,

494	499	// add label and close json object
495	500	snprintf(_s, slen-1,
496	501	" \"label\" : \"%s\"\n"
497		"}\n", r->label ? r->label : "<none>");
	502	"}", r->label ? r->label : "<none>");
498	503
499	504	data_string[STRINGSIZE-1] = 0;
500	505	*s = data_string;
501	506
502	507
503		} // End of format_file_block_record
	508	} // End of flow_record_to_json

-1

bin/output_json.h less more

0	0	/*
1		* Copyright (c) 2017, Peter Haag
	1	* Copyright (c) 2019, Peter Haag
2	2	* All rights reserved.
3	3	*
4	4	* Redistribution and use in source and binary forms, with or without

27	27	*
28	28	*/
29	29
	30	#ifndef _OUTPUT_JSON_H
	31	#define _OUTPUT_JSON_H 1
	32
	33	void json_prolog(void);
	34
	35	void json_epilog(void);
30	36
31	37	void flow_record_to_json(void record, char *s, int tag);
32	38
	39	#endif // _OUTPUT_JSON_H

+100

-0

bin/output_pipe.c less more

	0	/*
	1	* Copyright (c) 2019-2020, Peter Haag
	2	* All rights reserved.
	3	*
	4	* Redistribution and use in source and binary forms, with or without
	5	* modification, are permitted provided that the following conditions are met:
	6	*
	7	* * Redistributions of source code must retain the above copyright notice,
	8	* this list of conditions and the following disclaimer.
	9	* * Redistributions in binary form must reproduce the above copyright notice,
	10	* this list of conditions and the following disclaimer in the documentation
	11	* and/or other materials provided with the distribution.
	12	* * Neither the name of the author nor the names of its contributors may be
	13	* used to endorse or promote products derived from this software without
	14	* specific prior written permission.
	15	*
	16	* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
	17	* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	18	* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	19	* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
	20	* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
	21	* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
	22	* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
	23	* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
	24	* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
	25	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
	26	* POSSIBILITY OF SUCH DAMAGE.
	27	*
	28	*/
	29
	30	#include "config.h"
	31
	32	#include <stdio.h>
	33	#include <stddef.h>
	34	#include <sys/types.h>
	35	#include <sys/socket.h>
	36	#include <netinet/in.h>
	37	#include <arpa/inet.h>
	38	#include <time.h>
	39	#include <string.h>
	40
	41	#ifdef HAVE_STDINT_H
	42	#include <stdint.h>
	43	#endif
	44
	45	#include "nfdump.h"
	46	#include "nffile.h"
	47	#include "nfx.h"
	48	#include "output_pipe.h"
	49
	50	#define STRINGSIZE 10240
	51	#define IP_STRING_LEN (INET6_ADDRSTRLEN)
	52
	53	static char data_string[STRINGSIZE];
	54
	55	// record counter
	56	static uint32_t recordCount;
	57
	58	void pipe_prolog(void) {
	59	recordCount = 0;
	60	memset(data_string, 0, STRINGSIZE);
	61	} // End of pipe_prolog
	62
	63	void pipe_epilog(void) {
	64	// empty
	65	} // End of pipe_epilog
	66
	67	void flow_record_to_pipe(void record, char * s, int tag) {
	68	uint32_t sa[4], da[4];
	69	int af;
	70	master_record_t r = (master_record_t )record;
	71
	72	if ( (r->flags & FLAG_IPV6_ADDR ) != 0 ) { // IPv6
	73	af = PF_INET6;
	74	} else { // IPv4
	75	af = PF_INET;
	76	}
	77
	78	// Make sure Endian does not screw us up
	79	sa[0] = ( r->V6.srcaddr[0] >> 32 ) & 0xffffffffLL;
	80	sa[1] = r->V6.srcaddr[0] & 0xffffffffLL;
	81	sa[2] = ( r->V6.srcaddr[1] >> 32 ) & 0xffffffffLL;
	82	sa[3] = r->V6.srcaddr[1] & 0xffffffffLL;
	83
	84	da[0] = ( r->V6.dstaddr[0] >> 32 ) & 0xffffffffLL;
	85	da[1] = r->V6.dstaddr[0] & 0xffffffffLL;
	86	da[2] = ( r->V6.dstaddr[1] >> 32 ) & 0xffffffffLL;
	87	da[3] = r->V6.dstaddr[1] & 0xffffffffLL;
	88
	89	snprintf(data_string, STRINGSIZE-1 ,"%i\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%llu\|%llu",
	90	af, r->first, r->msec_first ,r->last, r->msec_last, r->prot,
	91	sa[0], sa[1], sa[2], sa[3], r->srcport, da[0], da[1], da[2], da[3], r->dstport,
	92	r->srcas, r->dstas, r->input, r->output,
	93	r->tcp_flags, r->tos, (unsigned long long)r->dPkts, (unsigned long long)r->dOctets);
	94
	95	data_string[STRINGSIZE-1] = 0;
	96
	97	*s = data_string;
	98
	99	} // End of flow_record_to_pipe

+40

-0

bin/output_pipe.h less more

	0	/*
	1	* Copyright (c) 2019, Peter Haag
	2	* All rights reserved.
	3	*
	4	* Redistribution and use in source and binary forms, with or without
	5	* modification, are permitted provided that the following conditions are met:
	6	*
	7	* * Redistributions of source code must retain the above copyright notice,
	8	* this list of conditions and the following disclaimer.
	9	* * Redistributions in binary form must reproduce the above copyright notice,
	10	* this list of conditions and the following disclaimer in the documentation
	11	* and/or other materials provided with the distribution.
	12	* * Neither the name of the author nor the names of its contributors may be
	13	* used to endorse or promote products derived from this software without
	14	* specific prior written permission.
	15	*
	16	* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
	17	* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	18	* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	19	* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
	20	* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
	21	* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
	22	* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
	23	* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
	24	* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
	25	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
	26	* POSSIBILITY OF SUCH DAMAGE.
	27	*
	28	*/
	29
	30	#ifndef _OUTPUT_PIPE_H
	31	#define _OUTPUT_PIPE_H 1
	32
	33	void pipe_prolog(void);
	34
	35	void pipe_epilog(void);
	36
	37	void flow_record_to_pipe(void record, char *s, int tag);
	38
	39	#endif // _OUTPUT_PIPE_H

+532

-0

bin/output_raw.c less more

	0	/*
	1	* Copyright (c) 2019-2020, Peter Haag
	2	* All rights reserved.
	3	*
	4	* Redistribution and use in source and binary forms, with or without
	5	* modification, are permitted provided that the following conditions are met:
	6	*
	7	* * Redistributions of source code must retain the above copyright notice,
	8	* this list of conditions and the following disclaimer.
	9	* * Redistributions in binary form must reproduce the above copyright notice,
	10	* this list of conditions and the following disclaimer in the documentation
	11	* and/or other materials provided with the distribution.
	12	* * Neither the name of the author nor the names of its contributors may be
	13	* used to endorse or promote products derived from this software without
	14	* specific prior written permission.
	15	*
	16	* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
	17	* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	18	* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	19	* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
	20	* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
	21	* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
	22	* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
	23	* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
	24	* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
	25	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
	26	* POSSIBILITY OF SUCH DAMAGE.
	27	*
	28	*/
	29
	30	#include "config.h"
	31
	32	#include <stdio.h>
	33	#include <stddef.h>
	34	#include <sys/types.h>
	35	#include <sys/socket.h>
	36	#include <netinet/in.h>
	37	#include <arpa/inet.h>
	38	#include <time.h>
	39	#include <string.h>
	40
	41	#ifdef HAVE_STDINT_H
	42	#include <stdint.h>
	43	#endif
	44
	45	#include "util.h"
	46	#include "nfdump.h"
	47	#include "nffile.h"
	48	#include "nfx.h"
	49	#include "output_util.h"
	50	#include "output_raw.h"
	51
	52	#define STRINGSIZE 10240
	53	#define IP_STRING_LEN (INET6_ADDRSTRLEN)
	54
	55	static char data_string[STRINGSIZE];
	56
	57	// record counter
	58	static uint32_t recordCount;
	59
	60	void raw_prolog(void) {
	61	recordCount = 0;
	62	memset(data_string, 0, STRINGSIZE);
	63	} // End of raw_prolog
	64
	65	void raw_epilog(void) {
	66	// empty
	67	} // End of raw_epilog
	68
	69	void flow_record_to_raw(void record, char * s, int tag) {
	70	char *_s, as[IP_STRING_LEN], ds[IP_STRING_LEN], datestr1[64], datestr2[64], datestr3[64];
	71	char s_snet[IP_STRING_LEN], s_dnet[IP_STRING_LEN];
	72	int i, id;
	73	ssize_t slen, _slen;
	74	time_t when;
	75	struct tm *ts;
	76	master_record_t r = (master_record_t )record;
	77	extension_map_t *extension_map = r->map_ref;
	78
	79	as[0] = 0;
	80	ds[0] = 0;
	81	if ( TestFlag(r->flags,FLAG_IPV6_ADDR ) != 0 ) { // IPv6
	82	uint64_t snet[2];
	83	uint64_t dnet[2];
	84
	85	snet[0] = htonll(r->V6.srcaddr[0]);
	86	snet[1] = htonll(r->V6.srcaddr[1]);
	87	dnet[0] = htonll(r->V6.dstaddr[0]);
	88	dnet[1] = htonll(r->V6.dstaddr[1]);
	89	inet_ntop(AF_INET6, snet, as, sizeof(as));
	90	inet_ntop(AF_INET6, dnet, ds, sizeof(ds));
	91
	92	inet6_ntop_mask(r->V6.srcaddr, r->src_mask, s_snet, sizeof(s_snet));
	93	inet6_ntop_mask(r->V6.dstaddr, r->dst_mask, s_dnet, sizeof(s_dnet));
	94
	95	} else { // IPv4
	96	uint32_t snet, dnet;
	97	snet = htonl(r->V4.srcaddr);
	98	dnet = htonl(r->V4.dstaddr);
	99	inet_ntop(AF_INET, &snet, as, sizeof(as));
	100	inet_ntop(AF_INET, &dnet, ds, sizeof(ds));
	101
	102	inet_ntop_mask(r->V4.srcaddr, r->src_mask, s_snet, sizeof(s_snet));
	103	inet_ntop_mask(r->V4.dstaddr, r->dst_mask, s_dnet, sizeof(s_dnet));
	104
	105	}
	106	as[IP_STRING_LEN-1] = 0;
	107	ds[IP_STRING_LEN-1] = 0;
	108
	109	when = r->first;
	110	ts = localtime(&when);
	111	strftime(datestr1, 63, "%Y-%m-%d %H:%M:%S", ts);
	112
	113	when = r->last;
	114	ts = localtime(&when);
	115	strftime(datestr2, 63, "%Y-%m-%d %H:%M:%S", ts);
	116
	117	_s = data_string;
	118	slen = STRINGSIZE;
	119	snprintf(_s, slen-1, "\n"
	120	"Flow Record: \n"
	121	" Flags = 0x%.2x %s, %s\n"
	122	" label = %16s\n"
	123	" export sysid = %5u\n"
	124	" size = %5u\n"
	125	" first = %10u [%s]\n"
	126	" last = %10u [%s]\n"
	127	" msec_first = %5u\n"
	128	" msec_last = %5u\n"
	129	" src addr = %16s\n"
	130	" dst addr = %16s\n"
	131	,
	132	r->flags, TestFlag(r->flags, FLAG_EVENT) ? "EVENT" : "FLOW",
	133	TestFlag(r->flags, FLAG_SAMPLED) ? "Sampled" : "Unsampled",
	134	r->label ? r->label : "<none>",
	135	r->exporter_sysid, r->size, r->first,
	136	datestr1, r->last, datestr2, r->msec_first, r->msec_last,
	137	as, ds );
	138
	139	_slen = strlen(data_string);
	140	_s = data_string + _slen;
	141	slen = STRINGSIZE - _slen;
	142
	143	if ( r->prot == IPPROTO_ICMP \|\| r->prot == IPPROTO_ICMPV6 ) { // ICMP
	144	snprintf(_s, slen-1,
	145	" ICMP = %2u.%-2u type.code\n",
	146	r->icmp_type, r->icmp_code);
	147	} else {
	148	snprintf(_s, slen-1,
	149	" src port = %5u\n"
	150	" dst port = %5u\n",
	151	r->srcport, r->dstport);
	152	}
	153
	154	_slen = strlen(data_string);
	155	_s = data_string + _slen;
	156	slen = STRINGSIZE - _slen;
	157
	158	snprintf(_s, slen-1,
	159	" fwd status = %3u\n"
	160	" tcp flags = 0x%.2x %s\n"
	161	" proto = %3u %s\n"
	162	" (src)tos = %3u\n"
	163	" (in)packets = %10llu\n"
	164	" (in)bytes = %10llu\n",
	165	r->fwd_status, r->tcp_flags, FlagsString(r->tcp_flags), r->prot, ProtoString(r->prot, 0), r->tos,
	166	(unsigned long long)r->dPkts, (unsigned long long)r->dOctets);
	167
	168	_slen = strlen(data_string);
	169	_s = data_string + _slen;
	170	slen = STRINGSIZE - _slen;
	171
	172	i = 0;
	173	while ( (id = extension_map->ex_id[i]) != 0 ) {
	174	if ( slen <= 20 ) {
	175	fprintf(stderr, "String too short! Missing record data!\n");
	176	data_string[STRINGSIZE-1] = 0;
	177	*s = data_string;
	178	}
	179	switch(id) {
	180	case EX_IO_SNMP_2:
	181	case EX_IO_SNMP_4:
	182	snprintf(_s, slen-1,
	183	" input = %5u\n"
	184	" output = %5u\n"
	185	, r->input, r->output);
	186	_slen = strlen(data_string);
	187	_s = data_string + _slen;
	188	slen = STRINGSIZE - _slen;
	189	break;
	190	case EX_AS_2:
	191	case EX_AS_4:
	192	snprintf(_s, slen-1,
	193	" src as = %5u\n"
	194	" dst as = %5u\n"
	195	, r->srcas, r->dstas);
	196	_slen = strlen(data_string);
	197	_s = data_string + _slen;
	198	slen = STRINGSIZE - _slen;
	199	break;
	200	case EX_BGPADJ:
	201	snprintf(_s, slen-1,
	202	" next as = %5u\n"
	203	" prev as = %5u\n"
	204	, r->bgpNextAdjacentAS, r->bgpPrevAdjacentAS);
	205	_slen = strlen(data_string);
	206	_s = data_string + _slen;
	207	slen = STRINGSIZE - _slen;
	208	break;
	209	case EX_MULIPLE:
	210	snprintf(_s, slen-1,
	211	" src mask = %5u %s/%u\n"
	212	" dst mask = %5u %s/%u\n"
	213	" dst tos = %3u\n"
	214	" direction = %3u\n"
	215	, r->src_mask, s_snet, r->src_mask, r->dst_mask, s_dnet, r->dst_mask, r->dst_tos, r->dir );
	216	_slen = strlen(data_string);
	217	_s = data_string + _slen;
	218	slen = STRINGSIZE - _slen;
	219	break;
	220	case EX_NEXT_HOP_v4:
	221	case EX_NEXT_HOP_v6:
	222	if ( (r->flags & FLAG_IPV6_NH ) != 0 ) { // IPv6
	223	as[0] = 0;
	224	r->ip_nexthop.V6[0] = htonll(r->ip_nexthop.V6[0]);
	225	r->ip_nexthop.V6[1] = htonll(r->ip_nexthop.V6[1]);
	226	inet_ntop(AF_INET6, r->ip_nexthop.V6, as, sizeof(as));
	227	as[IP_STRING_LEN-1] = 0;
	228	} else {
	229	as[0] = 0;
	230	r->ip_nexthop.V4 = htonl(r->ip_nexthop.V4);
	231	inet_ntop(AF_INET, &r->ip_nexthop.V4, as, sizeof(as));
	232	as[IP_STRING_LEN-1] = 0;
	233	}
	234
	235	snprintf(_s, slen-1,
	236	" ip next hop = %16s\n"
	237	, as);
	238	_slen = strlen(data_string);
	239	_s = data_string + _slen;
	240	slen = STRINGSIZE - _slen;
	241	break;
	242	case EX_NEXT_HOP_BGP_v4:
	243	case EX_NEXT_HOP_BGP_v6:
	244	if ( (r->flags & FLAG_IPV6_NHB ) != 0 ) { // IPv6
	245	as[0] = 0;
	246	r->bgp_nexthop.V6[0] = htonll(r->bgp_nexthop.V6[0]);
	247	r->bgp_nexthop.V6[1] = htonll(r->bgp_nexthop.V6[1]);
	248	inet_ntop(AF_INET6, r->ip_nexthop.V6, as, sizeof(as));
	249	as[IP_STRING_LEN-1] = 0;
	250	} else {
	251	as[0] = 0;
	252	r->bgp_nexthop.V4 = htonl(r->bgp_nexthop.V4);
	253	inet_ntop(AF_INET, &r->bgp_nexthop.V4, as, sizeof(as));
	254	as[IP_STRING_LEN-1] = 0;
	255	}
	256	snprintf(_s, slen-1,
	257	" bgp next hop = %16s\n"
	258	, as);
	259	_slen = strlen(data_string);
	260	_s = data_string + _slen;
	261	slen = STRINGSIZE - _slen;
	262	break;
	263	case EX_VLAN:
	264	snprintf(_s, slen-1,
	265	" src vlan = %5u\n"
	266	" dst vlan = %5u\n"
	267	, r->src_vlan, r->dst_vlan);
	268	_slen = strlen(data_string);
	269	_s = data_string + _slen;
	270	slen = STRINGSIZE - _slen;
	271	break;
	272	case EX_OUT_PKG_4:
	273	case EX_OUT_PKG_8:
	274	snprintf(_s, slen-1,
	275	" out packets = %10llu\n"
	276	, (long long unsigned)r->out_pkts);
	277	_slen = strlen(data_string);
	278	_s = data_string + _slen;
	279	slen = STRINGSIZE - _slen;
	280	break;
	281	case EX_OUT_BYTES_4:
	282	case EX_OUT_BYTES_8:
	283	snprintf(_s, slen-1,
	284	" out bytes = %10llu\n"
	285	, (long long unsigned)r->out_bytes);
	286	_slen = strlen(data_string);
	287	_s = data_string + _slen;
	288	slen = STRINGSIZE - _slen;
	289	break;
	290	case EX_AGGR_FLOWS_4:
	291	case EX_AGGR_FLOWS_8:
	292	snprintf(_s, slen-1,
	293	" aggr flows = %10llu\n"
	294	, (long long unsigned)r->aggr_flows);
	295	_slen = strlen(data_string);
	296	_s = data_string + _slen;
	297	slen = STRINGSIZE - _slen;
	298	break;
	299	case EX_MAC_1: {
	300	int i;
	301	uint8_t mac1[6], mac2[6];
	302
	303	for ( i=0; i<6; i++ ) {
	304	mac1[i] = (r->in_src_mac >> ( i*8 )) & 0xFF;
	305	}
	306	for ( i=0; i<6; i++ ) {
	307	mac2[i] = (r->out_dst_mac >> ( i*8 )) & 0xFF;
	308	}
	309
	310	snprintf(_s, slen-1,
	311	" in src mac = %.2x:%.2x:%.2x:%.2x:%.2x:%.2x\n"
	312	" out dst mac = %.2x:%.2x:%.2x:%.2x:%.2x:%.2x\n"
	313	, mac1[5], mac1[4], mac1[3], mac1[2], mac1[1], mac1[0], mac2[5], mac2[4], mac2[3], mac2[2], mac2[1], mac2[0] );
	314	_slen = strlen(data_string);
	315	_s = data_string + _slen;
	316	slen = STRINGSIZE - _slen;
	317	} break;
	318	case EX_MAC_2: {
	319	int i;
	320	uint8_t mac1[6], mac2[6];
	321
	322	for ( i=0; i<6; i++ ) {
	323	mac1[i] = (r->in_dst_mac >> ( i*8 )) & 0xFF;
	324	}
	325	for ( i=0; i<6; i++ ) {
	326	mac2[i] = (r->out_src_mac >> ( i*8 )) & 0xFF;
	327	}
	328
	329	snprintf(_s, slen-1,
	330	" in dst mac = %.2x:%.2x:%.2x:%.2x:%.2x:%.2x\n"
	331	" out src mac = %.2x:%.2x:%.2x:%.2x:%.2x:%.2x\n"
	332	, mac1[5], mac1[4], mac1[3], mac1[2], mac1[1], mac1[0], mac2[5], mac2[4], mac2[3], mac2[2], mac2[1], mac2[0] );
	333	_slen = strlen(data_string);
	334	_s = data_string + _slen;
	335	slen = STRINGSIZE - _slen;
	336	} break;
	337	case EX_MPLS: {
	338	unsigned int i;
	339	for ( i=0; i<10; i++ ) {
	340	snprintf(_s, slen-1,
	341	" MPLS Lbl %2u = %8u-%1u-%1u\n", i+1
	342	, r->mpls_label[i] >> 4 , (r->mpls_label[i] & 0xF ) >> 1, r->mpls_label[i] & 1 );
	343	_slen = strlen(data_string);
	344	_s = data_string + _slen;
	345	slen = STRINGSIZE - _slen;
	346	}
	347	} break;
	348	case EX_ROUTER_IP_v4:
	349	as[0] = 0;
	350	r->ip_router.V4 = htonl(r->ip_router.V4);
	351	inet_ntop(AF_INET, &r->ip_router.V4, as, sizeof(as));
	352	as[IP_STRING_LEN-1] = 0;
	353
	354	snprintf(_s, slen-1,
	355	" ip router = %16s\n"
	356	, as);
	357	_slen = strlen(data_string);
	358	_s = data_string + _slen;
	359	slen = STRINGSIZE - _slen;
	360
	361	break;
	362	case EX_ROUTER_IP_v6:
	363	as[0] = 0;
	364	r->ip_router.V6[0] = htonll(r->ip_router.V6[0]);
	365	r->ip_router.V6[1] = htonll(r->ip_router.V6[1]);
	366	inet_ntop(AF_INET6, &r->ip_router.V6, as, sizeof(as));
	367	as[IP_STRING_LEN-1] = 0;
	368
	369	snprintf(_s, slen-1,
	370	" ip router = %16s\n"
	371	, as);
	372	_slen = strlen(data_string);
	373	_s = data_string + _slen;
	374	slen = STRINGSIZE - _slen;
	375	break;
	376	case EX_LATENCY: {
	377	double f1, f2, f3;
	378	f1 = (double)r->client_nw_delay_usec / 1000.0;
	379	f2 = (double)r->server_nw_delay_usec / 1000.0;
	380	f3 = (double)r->appl_latency_usec / 1000.0;
	381
	382	snprintf(_s, slen-1,
	383	" cli latency = %9.3f ms\n"
	384	" srv latency = %9.3f ms\n"
	385	" app latency = %9.3f ms\n"
	386	, f1, f2, f3);
	387
	388	_slen = strlen(data_string);
	389	_s = data_string + _slen;
	390	slen = STRINGSIZE - _slen;
	391
	392	} break;
	393	case EX_ROUTER_ID:
	394	snprintf(_s, slen-1,
	395	" engine type = %5u\n"
	396	" engine ID = %5u\n"
	397	, r->engine_type, r->engine_id);
	398	_slen = strlen(data_string);
	399	_s = data_string + _slen;
	400	slen = STRINGSIZE - _slen;
	401	break;
	402	case EX_RECEIVED:
	403	when = r->received / 1000LL;
	404	ts = localtime(&when);
	405	strftime(datestr3, 63, "%Y-%m-%d %H:%M:%S", ts);
	406
	407	snprintf(_s, slen-1,
	408	" received at = %13llu [%s.%03llu]\n"
	409	, (long long unsigned)r->received, datestr3, (long long unsigned)(r->received % 1000L));
	410	_slen = strlen(data_string);
	411	_s = data_string + _slen;
	412	slen = STRINGSIZE - _slen;
	413	break;
	414	#ifdef NSEL
	415	case EX_NSEL_COMMON: {
	416	when = r->event_time / 1000LL;
	417	ts = localtime(&when);
	418	strftime(datestr3, 63, "%Y-%m-%d %H:%M:%S", ts);
	419	snprintf(_s, slen-1,
	420	" connect ID = %10u\n"
	421	" fw event = %5u: %s\n"
	422	" fw ext event = %5u: %s\n"
	423	" secgroup tag = %5u\n"
	424	" Event time = %13llu [%s.%03llu]\n"
	425	, r->conn_id, r->event, r->event_flag == FW_EVENT ? FwEventString(r->event) : EventString(r->event)
	426	, r->fw_xevent, EventXString(r->fw_xevent), r->sec_group_tag
	427	, (long long unsigned)r->event_time, datestr3, (long long unsigned)(r->event_time % 1000L));
	428	_slen = strlen(data_string);
	429	_s = data_string + _slen;
	430	slen = STRINGSIZE - _slen;
	431	} break;
	432	case EX_NEL_COMMON: {
	433	snprintf(_s, slen-1,
	434	" nat event = %5u: %s\n"
	435	" ingress VRF = %10u\n"
	436	" egress VRF = %10u\n"
	437	, r->event, r->event_flag == FW_EVENT ? FwEventString(r->event) : EventString(r->event)
	438	, r->ingress_vrfid, r->egress_vrfid);
	439	_slen = strlen(data_string);
	440	_s = data_string + _slen;
	441	slen = STRINGSIZE - _slen;
	442	} break;
	443	case EX_NSEL_XLATE_PORTS: {
	444	snprintf(_s, slen-1,
	445	" src xlt port = %5u\n"
	446	" dst xlt port = %5u\n"
	447	, r->xlate_src_port, r->xlate_dst_port );
	448	_slen = strlen(data_string);
	449	_s = data_string + _slen;
	450	slen = STRINGSIZE - _slen;
	451	} break;
	452	case EX_PORT_BLOCK_ALLOC: {
	453	snprintf(_s, slen-1,
	454	" pblock start = %5u\n"
	455	" pblock end = %5u\n"
	456	" pblock step = %5u\n"
	457	" pblock size = %5u\n"
	458	, r->block_start, r->block_end, r->block_step, r->block_size );
	459	_slen = strlen(data_string);
	460	_s = data_string + _slen;
	461	slen = STRINGSIZE - _slen;
	462	} break;
	463	case EX_NSEL_XLATE_IP_v4:
	464	as[0] = 0;
	465	ds[0] = 0;
	466	r->xlate_src_ip.V4 = htonl(r->xlate_src_ip.V4);
	467	r->xlate_dst_ip.V4 = htonl(r->xlate_dst_ip.V4);
	468	inet_ntop(AF_INET, &r->xlate_src_ip.V4, as, sizeof(as));
	469	inet_ntop(AF_INET, &r->xlate_dst_ip.V4, ds, sizeof(ds));
	470	as[IP_STRING_LEN-1] = 0;
	471	ds[IP_STRING_LEN-1] = 0;
	472
	473	snprintf(_s, slen-1,
	474	" src xlt ip = %16s\n"
	475	" dst xlt ip = %16s\n"
	476	, as, ds);
	477	_slen = strlen(data_string);
	478	_s = data_string + _slen;
	479	slen = STRINGSIZE - _slen;
	480	break;
	481	case EX_NSEL_XLATE_IP_v6:
	482	as[0] = 0;
	483	ds[0] = 0;
	484	r->xlate_src_ip.V6[0] = htonll(r->xlate_src_ip.V6[0]);
	485	r->xlate_src_ip.V6[1] = htonll(r->xlate_src_ip.V6[1]);
	486	r->xlate_dst_ip.V6[0] = htonll(r->xlate_dst_ip.V6[0]);
	487	r->xlate_dst_ip.V6[1] = htonll(r->xlate_dst_ip.V6[1]);
	488	inet_ntop(AF_INET6, &r->xlate_src_ip.V6, as, sizeof(as));
	489	inet_ntop(AF_INET6, &r->xlate_dst_ip.V6, ds, sizeof(ds));
	490	as[IP_STRING_LEN-1] = 0;
	491	ds[IP_STRING_LEN-1] = 0;
	492
	493	snprintf(_s, slen-1,
	494	" src xlate ip = %16s\n"
	495	" dst xlate ip = %16s\n"
	496	, as, ds);
	497	_slen = strlen(data_string);
	498	_s = data_string + _slen;
	499	slen = STRINGSIZE - _slen;
	500	break;
	501	case EX_NSEL_ACL:
	502	snprintf(_s, slen-1,
	503	" Ingress ACL = 0x%x/0x%x/0x%x\n"
	504	" Egress ACL = 0x%x/0x%x/0x%x\n"
	505	, r->ingress_acl_id[0], r->ingress_acl_id[1], r->ingress_acl_id[2],
	506	r->egress_acl_id[0], r->egress_acl_id[1], r->egress_acl_id[2]);
	507	_slen = strlen(data_string);
	508	_s = data_string + _slen;
	509	slen = STRINGSIZE - _slen;
	510	break;
	511	case EX_NSEL_USER:
	512	case EX_NSEL_USER_MAX:
	513	snprintf(_s, slen-1,
	514	" User name = %s\n"
	515	, r->username[0] ? r->username : " <empty>");
	516	_slen = strlen(data_string);
	517	_s = data_string + _slen;
	518	slen = STRINGSIZE - _slen;
	519	break;
	520	#endif
	521	default:
	522	snprintf(_s, slen-1, "Type %u not implemented\n", id);
	523
	524	}
	525	i++;
	526	}
	527
	528	data_string[STRINGSIZE-1] = 0;
	529	*s = data_string;
	530
	531	} // End of flow_record_to_raw

+40

-0

bin/output_raw.h less more

	0	/*
	1	* Copyright (c) 2019, Peter Haag
	2	* All rights reserved.
	3	*
	4	* Redistribution and use in source and binary forms, with or without
	5	* modification, are permitted provided that the following conditions are met:
	6	*
	7	* * Redistributions of source code must retain the above copyright notice,
	8	* this list of conditions and the following disclaimer.
	9	* * Redistributions in binary form must reproduce the above copyright notice,
	10	* this list of conditions and the following disclaimer in the documentation
	11	* and/or other materials provided with the distribution.
	12	* * Neither the name of the author nor the names of its contributors may be
	13	* used to endorse or promote products derived from this software without
	14	* specific prior written permission.
	15	*
	16	* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
	17	* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	18	* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	19	* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
	20	* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
	21	* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
	22	* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
	23	* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
	24	* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
	25	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
	26	* POSSIBILITY OF SUCH DAMAGE.
	27	*
	28	*/
	29
	30	#ifndef _OUTPUT_RAW_H
	31	#define _OUTPUT_RAW_H 1
	32
	33	void raw_prolog(void);
	34
	35	void raw_epilog(void);
	36
	37	void flow_record_to_raw(void record, char *s, int tag);
	38
	39	#endif // _OUTPUT_RAW_H

+361

-0

bin/output_util.c less more

	0	/*
	1	* Copyright (c) 2020, Peter Haag
	2	* All rights reserved.
	3	*
	4	* Redistribution and use in source and binary forms, with or without
	5	* modification, are permitted provided that the following conditions are met:
	6	*
	7	* * Redistributions of source code must retain the above copyright notice,
	8	* this list of conditions and the following disclaimer.
	9	* * Redistributions in binary form must reproduce the above copyright notice,
	10	* this list of conditions and the following disclaimer in the documentation
	11	* and/or other materials provided with the distribution.
	12	* * Neither the name of the author nor the names of its contributors may be
	13	* used to endorse or promote products derived from this software without
	14	* specific prior written permission.
	15	*
	16	* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
	17	* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	18	* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	19	* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
	20	* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
	21	* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
	22	* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
	23	* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
	24	* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
	25	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
	26	* POSSIBILITY OF SUCH DAMAGE.
	27	*
	28	*/
	29
	30	#include "config.h"
	31
	32	#include <stdio.h>
	33	#include <stddef.h>
	34	#include <sys/types.h>
	35	#include <string.h>
	36
	37	#ifdef HAVE_STDINT_H
	38	#include <stdint.h>
	39	#endif
	40
	41	#include "nfdump.h"
	42	#include "nffile.h"
	43	#include "output_util.h"
	44
	45	#define NumProtos 138
	46	static char *protoList[NumProtos] = {
	47	"0", // 0 masked out - no protocol info - set to '0'
	48	"ICMP", // 1 Internet Control Message
	49	"IGMP", // 2 Internet Group Management
	50	"GGP", // 3 Gateway-to-Gateway
	51	"IPIP", // 4 IP in IP (encapsulation)
	52	"ST", // 5 Stream
	53	"TCP", // 6 Transmission Control
	54	"CBT", // 7 CBT
	55	"EGP", // 8 Exterior Gateway Protocol
	56	"IGP", // 9 any private interior gateway (used by Cisco for their IGRP)
	57	"BBN", // 10 BBN RCC Monitoring
	58	"NVPII", // 11 Network Voice Protocol
	59	"PUP", // 12 PUP
	60	"ARGUS", // 13 ARGUS
	61	"ENCOM", // 14 EMCON
	62	"XNET", // 15 Cross Net Debugger
	63	"CHAOS", // 16 Chaos
	64	"UDP", // 17 User Datagram
	65	"MUX", // 18 Multiplexing
	66	"DCN", // 19 DCN Measurement Subsystems
	67	"HMP", // 20 Host Monitoring
	68	"PRM", // 21 Packet Radio Measurement
	69	"XNS", // 22 XEROX NS IDP
	70	"Trnk1", // 23 Trunk-1
	71	"Trnk2", // 24 Trunk-2
	72	"Leaf1", // 25 Leaf-1
	73	"Leaf2", // 26 Leaf-2
	74	"RDP", // 27 Reliable Data Protocol
	75	"IRTP", // 28 Internet Reliable Transaction
	76	"ISO-4", // 29 ISO Transport Protocol Class 4
	77	"NETBK", // 30 Bulk Data Transfer Protocol
	78	"MFESP", // 31 MFE Network Services Protocol
	79	"MEINP", // 32 MERIT Internodal Protocol
	80	"DCCP", // 33 Datagram Congestion Control Protocol
	81	"3PC", // 34 Third Party Connect Protocol
	82	"IDPR", // 35 Inter-Domain Policy Routing Protocol
	83	"XTP", // 36 XTP
	84	"DDP", // 37 Datagram Delivery Protocol
	85	"IDPR", // 38 IDPR Control Message Transport Proto
	86	"TP++", // 39 TP++ Transport Protocol
	87	"IL", // 40 IL Transport Protocol
	88	"IPv6", // 41 IPv6
	89	"SDRP", // 42 Source Demand Routing Protocol
	90	"Rte6", // 43 Routing Header for IPv6
	91	"Frag6", // 44 Fragment Header for IPv6
	92	"IDRP", // 45 Inter-Domain Routing Protocol
	93	"RSVP", // 46 Reservation Protocol
	94	"GRE", // 47 General Routing Encapsulation
	95	"MHRP", // 48 Mobile Host Routing Protocol
	96	"BNA", // 49 BNA
	97	"ESP", // 50 Encap Security Payload
	98	"AH", // 51 Authentication Header
	99	"INLSP", // 52 Integrated Net Layer Security TUBA
	100	"SWIPE", // 53 IP with Encryption
	101	"NARP", // 54 NBMA Address Resolution Protocol
	102	"MOBIL", // 55 IP Mobility
	103	"TLSP", // 56 Transport Layer Security Protocol
	104	"SKIP", // 57 SKIP
	105	"ICMP6", // 58 ICMP for IPv6
	106	"NOHE6", // 59 No Next Header for IPv6
	107	"OPTS6", // 60 Destination Options for IPv6
	108	"HOST", // 61 any host internal protocol
	109	"CFTP", // 62 CFTP
	110	"NET", // 63 any local network
	111	"SATNT", // 64 SATNET and Backroom EXPAK
	112	"KLAN", // 65 Kryptolan
	113	"RVD", // 66 MIT Remote Virtual Disk Protocol
	114	"IPPC", // 67 Internet Pluribus Packet Core
	115	"FS", // 68 any distributed file system
	116	"SATM", // 69 SATNET Monitoring
	117	"VISA", // 70 VISA Protocol
	118	"IPCV", // 71 Internet Packet Core Utility
	119	"CPNX", // 72 Computer Protocol Network Executive
	120	"CPHB", // 73 Computer Protocol Heart Beat
	121	"WSN", // 74 Wang Span Network
	122	"PVP", // 75 Packet Video Protocol
	123	"BSATM", // 76 Backroom SATNET Monitoring
	124	"SUNND", // 77 SUN ND PROTOCOL-Temporary
	125	"WBMON", // 78 WIDEBAND Monitoring
	126	"WBEXP", // 79 WIDEBAND EXPAK
	127	"ISOIP", // 80 ISO Internet Protocol
	128	"VMTP", // 81 VMTP
	129	"SVMTP", // 82 SECURE-VMTP
	130	"VINES", // 83 VINES
	131	"TTP", // 84 TTP
	132	"NSIGP", // 85 NSFNET-IGP
	133	"DGP", // 86 Dissimilar Gateway Protocol
	134	"TCF", // 87 TCF
	135	"EIGRP", // 88 EIGRP
	136	"OSPF", // 89 OSPFIGP
	137	"S-RPC", // 90 Sprite RPC Protocol
	138	"LARP", // 91 Locus Address Resolution Protocol
	139	"MTP", // 92 Multicast Transport Protocol
	140	"AX.25", // 93 AX.25 Frames
	141	"IPIP", // 94 IP-within-IP Encapsulation Protocol
	142	"MICP", // 95 Mobile Internetworking Control Protocol
	143	"SCCSP", // 96 Semaphore Communications Sec. Protocol
	144	"ETHIP", // 97 Ethernet-within-IP Encapsulation
	145	"ENCAP", // 98 Encapsulation Header
	146	"99", // 99 any private encryption scheme
	147	"GMTP", // 100 GMTP
	148	"IFMP", // 101 Ipsilon Flow Management Protocol
	149	"PNNI", // 102 PNNI over IP
	150	"PIM", // 103 Protocol Independent Multicast
	151	"ARIS", // 104 ARIS
	152	"SCPS", // 105 SCPS
	153	"QNX", // 106 QNX
	154	"A/N", // 107 Active Networks
	155	"IPcmp", // 108 IP Payload Compression Protocol
	156	"SNP", // 109 Sitara Networks Protocol
	157	"CpqPP", // 110 Compaq Peer Protocol
	158	"IPXIP", // 111 IPX in IP
	159	"VRRP", // 112 Virtual Router Redundancy Protocol
	160	"PGM", // 113 PGM Reliable Transport Protocol
	161	"0hop", // 114 any 0-hop protocol
	162	"L2TP", // 115 Layer Two Tunneling Protocol
	163	"DDX", // 116 D-II Data Exchange (DDX)
	164	"IATP", // 117 Interactive Agent Transfer Protocol
	165	"STP", // 118 Schedule Transfer Protocol
	166	"SRP", // 119 SpectraLink Radio Protocol
	167	"UTI", // 120 UTI
	168	"SMP", // 121 Simple Message Protocol
	169	"SM", // 122 SM
	170	"PTP", // 123 Performance Transparency Protocol
	171	"ISIS4", // 124 ISIS over IPv4
	172	"FIRE", // 125 FIRE
	173	"CRTP", // 126 Combat Radio Transport Protocol
	174	"CRUDP", // 127 Combat Radio User Datagram
	175	"128", // 128 SSCOPMCE
	176	"IPLT", // 129 IPLP
	177	"SPS", // 130 Secure Packet Shield
	178	"PIPE", // 131 Private IP Encapsulation within IP
	179	"SCTP", // 132 Stream Control Transmission Protocol
	180	"FC", // 133 Fibre Channel
	181	"134", // 134 RSVP-E2E-IGNORE
	182	"MHEAD", // 135 Mobility Header
	183	"UDP-L", // 136 UDPLite
	184	"MPLS" // 137 MPLS-in-IP
	185	};
	186
	187	char *ProtoString(uint8_t protoNum, uint32_t printPlain) {
	188	static char s[16];
	189
	190	if ( protoNum >= NumProtos \|\| printPlain) {
	191	snprintf(s,15,"%-5i", protoNum );
	192	s[15] = '\0';
	193	return s;
	194	} else {
	195	return protoList[protoNum];
	196	}
	197
	198	} // End of ProtoString
	199
	200	int ProtoNum(char *protoString) {
	201	int len;
	202
	203	if ( (len = strlen(protoString)) >= 6 )
	204	return -1;
	205
	206	for ( int i=0; i<NumProtos; i++ ) {
	207	if ( strncasecmp(protoString,protoList[i], len) == 0 &&
	208	( strlen(protoList[i]) == len) )
	209	return i;
	210	}
	211
	212	return -1;
	213
	214	} // End of ProtoNum
	215
	216	char *FlagsString(uint16_t flags) {
	217	static char string[16];
	218
	219	string[0] = flags & 128 ? 'C' : '.'; // Congestion window reduced - CWR
	220	string[1] = flags & 64 ? 'E' : '.'; // ECN-Echo
	221	string[2] = flags & 32 ? 'U' : '.'; // Urgent
	222	string[3] = flags & 16 ? 'A' : '.'; // Ack
	223	string[4] = flags & 8 ? 'P' : '.'; // Push
	224	string[5] = flags & 4 ? 'R' : '.'; // Reset
	225	string[6] = flags & 2 ? 'S' : '.'; // Syn
	226	string[7] = flags & 1 ? 'F' : '.'; // Fin
	227	string[8] = '\0';
	228
	229	return string;
	230	} // End of FlagsString
	231
	232
	233	void CondenseV6(char *s) {
	234	size_t len = strlen(s);
	235	char p, q;
	236
	237	if ( len <= 16 )
	238	return;
	239
	240	// orig: 2001:620:1000:cafe:20e:35ff:fec0:fed5 len = 37
	241	// condensed: 2001:62..e0:fed5
	242	p = s + 7;
	243	*p++ = '.';
	244	*p++ = '.';
	245	q = s + len - 7;
	246	while ( *q ) {
	247	p++ = q++;
	248	}
	249	*p = 0;
	250
	251	} // End of CondenseV6
	252
	253	char *FwEventString(int event) {
	254
	255	switch(event) {
	256	#ifdef JUNOS
	257	case 0:
	258	return "IGNORE";
	259	break;
	260	case 1:
	261	case 4:
	262	case 6:
	263	case 8:
	264	case 12:
	265	return "CREATE";
	266	break;
	267	case 2:
	268	case 5:
	269	case 7:
	270	case 9:
	271	case 13:
	272	return "DELETE";
	273	break;
	274	case 3:
	275	case 10:
	276	return "EXHAUSTED";
	277	break;
	278	case 11:
	279	return "QUOTA EXCEED";
	280	break;
	281	case 14:
	282	return "NAT PORT ALLOC";
	283	break;
	284	case 15:
	285	return "NAT PORT RELEASE";
	286	break;
	287	case 16:
	288	return "NAT PORT ACTIVE";
	289	break;
	290	#else
	291	case 0:
	292	return "IGNORE";
	293	break;
	294	case 1:
	295	return "CREATE";
	296	break;
	297	case 2:
	298	return "DELETE";
	299	break;
	300	case 3:
	301	return "DENIED";
	302	break;
	303	case 4:
	304	return "ALERT";
	305	break;
	306	case 5:
	307	return "UPDATE";
	308	break;
	309	#endif
	310	default:
	311	return "UNKNOW";
	312	}
	313
	314	} // End of FwEventString
	315
	316	char *EventString(int event) {
	317
	318	switch(event) {
	319	case 0:
	320	return "INVALID";
	321	break;
	322	case 1:
	323	return "ADD";
	324	break;
	325	case 2:
	326	return "DELETE";
	327	break;
	328	default:
	329	return "UNKNOW";
	330	}
	331
	332	} // End of EventString
	333
	334	char *EventXString(int xevent) {
	335	static char s[16];
	336
	337	switch( xevent) {
	338	case 0:
	339	return "Ignore";
	340	break;
	341	case 1001:
	342	return "I-ACL";
	343	break;
	344	case 1002:
	345	return "E-ACL";
	346	break;
	347	case 1003:
	348	return "Adap";
	349	break;
	350	case 1004:
	351	return "No Syn";
	352	break;
	353	default:
	354	snprintf(s,15,"%u",xevent);
	355	s[15] = '\0';
	356	return s;
	357	}
	358
	359	// not reached
	360	} // End of EventXString

+61

-0

bin/output_util.h less more

	0	/*
	1	* Copyright (c) 2019-2020, Peter Haag
	2	* All rights reserved.
	3	*
	4	* Redistribution and use in source and binary forms, with or without
	5	* modification, are permitted provided that the following conditions are met:
	6	*
	7	* * Redistributions of source code must retain the above copyright notice,
	8	* this list of conditions and the following disclaimer.
	9	* * Redistributions in binary form must reproduce the above copyright notice,
	10	* this list of conditions and the following disclaimer in the documentation
	11	* and/or other materials provided with the distribution.
	12	* * Neither the name of the author nor the names of its contributors may be
	13	* used to endorse or promote products derived from this software without
	14	* specific prior written permission.
	15	*
	16	* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
	17	* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	18	* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	19	* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
	20	* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
	21	* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
	22	* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
	23	* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
	24	* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
	25	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
	26	* POSSIBILITY OF SUCH DAMAGE.
	27	*
	28	*/
	29
	30	#ifndef _OUTPUT_UTIL_H
	31	#define _OUTPUT_UTIL_H 1
	32
	33	typedef void (printer_t)(void , char **, int);
	34	typedef void (*func_prolog_t)(void);
	35	typedef void (*func_epilog_t)(void);
	36
	37	typedef struct printmap_s {
	38	char *printmode; // name of the output format
	39	printer_t func_record; // prints the record
	40	func_prolog_t func_prolog; // prints the output prolog
	41	func_epilog_t func_epilog; // prints the output epilog
	42	char *Format; // output format definition
	43	} printmap_t;
	44
	45
	46	char *ProtoString(uint8_t protoNum, uint32_t plainNumbers);
	47
	48	int ProtoNum(char *protoString);
	49
	50	char *FlagsString(uint16_t flags);
	51
	52	void CondenseV6(char *s);
	53
	54	char *FwEventString(int event);
	55
	56	char *EventString(int event);
	57
	58	char *EventXString(int xevent);
	59
	60	#endif // _OUTPUT_UTIL_H

+24

-26

bin/pcaproc.c less more

0	0	/*
1		* Copyright (c) 2014-2019, Peter Haag
	1	* Copyright (c) 2014-2020, Peter Haag
2	2	* All rights reserved.
3	3	*
4	4	* Redistribution and use in source and binary forms, with or without

62	62
63	63	#include <pcap.h>
64	64
	65	#include "util.h"
	66	#include "nfdump.h"
65	67	#include "nffile.h"
66	68	#include "bookkeeper.h"
67	69	#include "collector.h"

69	71	#include "ipfrag.h"
70	72	#include "pcaproc.h"
71	73	#include "content_dns.h"
72		#include "util.h"
73	74	#include "netflow_pcap.h"
74	75
75	76	struct pcap_timeval {

133	134	}
134	135
135	136	if ( filename ) {
136		pcapfile->pd = pcap_dump_open(p, filename);
	137	FILE* pFile = fopen(filename, "wb");
	138	if ( !pFile ) {
	139	LogError("fopen() error in %s line %d: %s", __FILE__, __LINE__, strerror(errno) );
	140	return NULL;
	141	}
	142	pcapfile->pd = pcap_dump_fopen(p, pFile);
137	143	if ( !pcapfile->pd ) {
138	144	LogError("Fatal: pcap_dump_open() failed for file '%s': %s", filename, pcap_geterr(p));
139	145	return NULL;
140	146	} else {
141		fflush((FILE *)pcapfile->pd);
142		pcapfile->pfd = fileno((FILE *)pcapfile->pd);
	147	fflush(pFile);
	148	pcapfile->pfd = fileno((FILE *)pFile);
143	149	return pcapfile;
144	150	}
145	151	} else

150	156	int ClosePcapFile(pcapfile_t *pcapfile) {
151	157	int err = 0;
152	158
153		if ( fclose((FILE *)pcapfile->pd) < 0 ) {
154		LogError("close() error in %s line %d: %s", __FILE__, __LINE__, strerror(errno) );
155		err = errno;
156		}
	159	pcap_dump_close(pcapfile->pd);
157	160	pcapfile->pfd = -1;
158	161
159	162	return err;

424	427	// redo ethertype evaluation
425	428	goto REDO_LINK;
426	429	} break;
	430	case 0x26: // ?? multicast router termination ??
	431	case 0x32:
427	432	case 0x806: // skip ARP
428		// silently skip ARP
429		pcap_dev->proc_stat.skipped++;
430		goto END_FUNC;
431		break;
432		case 0x26: // ?? multicast router termination ??
433	433	case 0x4305: // B.A.T.M.A.N. BATADV
434	434	case 0x886f: // MS NLB heartbeat
435	435	case 0x88a2: // ATA over ethernet
436	436	case 0x88cc: // CISCO LLDP
437	437	case 0x9000: // Loop
	438	case 0x9003:
438	439	case 0x880b: // PPP - rfc 7042
439	440	case 0x6558: // Ethernet Bridge
440	441	pcap_dev->proc_stat.skipped++;
441		if ( Node->proto ) {
442		// if it's an encap which we do not understand yet - push tunnel
443		Push_Node(NodeList, Node);
444		} else {
445		pcap_dev->proc_stat.skipped++;
446		dbg_printf("Skip Ethertype 0x%x", ethertype);
447		Free_Node(Node);
448		}
	442	Free_Node(Node);
449	443	goto END_FUNC;
450	444	break;
451	445	default:
452	446	pcap_dev->proc_stat.unknown++;
453		LogInfo("Unsupported link type: 0x%x, packet: %u", ethertype, pkg_cnt);
	447	LogInfo("Unsupported ether type: 0x%x, packet: %u", ethertype, pkg_cnt);
454	448	Free_Node(Node);
455	449	goto END_FUNC;
456	450	}
	451	} else {
	452	LogInfo("Unsupported link type: 0x%x, packet: %u", pcap_dev->linktype, pkg_cnt);
	453	Free_Node(Node);
	454	return;
457	455	}
458	456
459	457	if (hdr->caplen < offset) {

592	590	Node->dst_addr.v4 = ntohl(ip->ip_dst.s_addr);
593	591	Node->version = AF_INET;
594	592	} else {
595		LogInfo("ProcessPacket() Unsupprted protocol version: %i", version);
	593	LogInfo("ProcessPacket() Unsupported protocol version: %i", version);
596	594	pcap_dev->proc_stat.unknown++;
597	595	Free_Node(Node);
598	596	goto END_FUNC;

777	775
778	776	} break;
779	777	default:
780		// not handled protocol - simply save node
781		Push_Node(NodeList, Node);
	778	// not handled protocol
	779	Free_Node(Node);
782	780	pcap_dev->proc_stat.unknown++;
783	781	break;
784	782	}

-6

bin/profile.c less more

0	0	/*
1		* Copyright (c) 2009 - 2018, Peter Haag
2		* Copyright (c) 2004 - 2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
	1	* Copyright (c) 2009-2020, Peter Haag
	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
3	3	* All rights reserved.
4	4	*
5	5	* Redistribution and use in source and binary forms, with or without

49	49
50	50	#include <rrd.h>
51	51
52		#include "nf_common.h"
53	52	#include "rbtree.h"
54	53	#include "nfdump.h"
55	54	#include "nffile.h"

73	72	static profile_channel_info_t *profile_channels;
74	73	static unsigned int num_channels;
75	74
76		static inline int AppendString(char stack, char string, size_t *buff_size);
	75	static int AppendString(char stack, char string, size_t *buff_size);
77	76
78	77	static void SetupProfileChannels(char profile_datadir, char profile_statdir, profile_param_info_t *profile_param,
79	78	int subdir_index, char filterfile, char filename, int verify_only, int compress);

82	81	return profile_channels;
83	82	} // End of GetProfiles
84	83
85		static inline int AppendString(char stack, char string, size_t *buff_size) {
	84	static int AppendString(char stack, char string, size_t *buff_size) {
86	85	size_t len = strlen(string);
87	86
88	87	if ( *buff_size <= len ) {

118	117
119	118	static void SetupProfileChannels(char profile_datadir, char profile_statdir, profile_param_info_t *profile_param,
120	119	int subdir_index, char filterfile, char filename, int verify_only, int compress ) {
121		FilterEngine_data_t *engine;
	120	FilterEngine_t *engine;
122	121	struct stat stat_buf;
123	122	char p, filter, subdir, wfile, ofile, rrdfile, *source_filter;
124	123	char path[MAXPATHLEN];

+14

-14

bin/profile.h less more

0	0	/*
1		* Copyright (c) 2009 - 2018, Peter Haag
2		* Copyright (c) 2004 - 2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
	1	* Copyright (c) 2009-2020, Peter Haag
	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
3	3	* All rights reserved.
4	4	*
5	5	* Redistribution and use in source and binary forms, with or without

45	45	} profile_param_info_t;
46	46
47	47	typedef struct profile_channel_info_s {
48		FilterEngine_data_t *engine;
49		char *group;
50		char *profile;
51		char *channel;
52		char *ofile; // tmp output file
53		char *wfile; // final filename
54		char *rrdfile; // rrd filename for update
55		char *dirstat_path; // pathname for dirstat file
56		nffile_t *nffile;
57		stat_record_t stat_record;
58		int type;
59		dirstat_t *dirstat;
	48	FilterEngine_t *engine;
	49	char *group;
	50	char *profile;
	51	char *channel;
	52	char *ofile; // tmp output file
	53	char *wfile; // final filename
	54	char *rrdfile; // rrd filename for update
	55	char *dirstat_path; // pathname for dirstat file
	56	nffile_t *nffile;
	57	stat_record_t stat_record;
	58	int type;
	59	dirstat_t *dirstat;
60	60	} profile_channel_info_t;
61	61
62	62	profile_channel_info_t *GetProfiles(void);

-23

bin/scanner.l less more

55	55	#include <stdint.h>
56	56	#endif
57	57
58		#include "rbtree.h"
59		#include "nfdump.h"
	58	#include "filter.h"
	59	#include "nftree.h"
60	60	#include "grammar.h"
61	61
62	62	extern char yyerror_buff[256];

281	281	return 1;
282	282	}
283	283
284		int ScreenIPString(char *string) {
285		char *c;
286
287		// [0-9A-Fa-f:][0-9A-Fa-f\.:]+[0-9A-Fa-f:] {
288		int len = strlen(string);
289		if ( len < 3 \|\| len > 39 )
290		return 0;
291
292		if ( !isxdigit(string[0]) )
293		return 0;
294
295		c = &string[1];
296		while ( *c ) {
297		if ( c != '.' \|\| c != ':' \|\| !isxdigit(*c) )
298		return 0;
299		c++;
300		}
301		return 1;
302
303		} // End of ScreenString
304
305	284	int ScreenIdentString(char *string) {
306	285	char *c;
307	286

+75

-56

bin/sfcapd.c less more

0	0	/*
1		* Copyright (c) 2018, Peter Haag
2		* Copyright (c) 2017, Peter Haag
3		* Copyright (c) 2016, Peter Haag
4		* Copyright (c) 2014, Peter Haag
5		* Copyright (c) 2009, Peter Haag
	1	* Copyright (c) 2009-2020, Peter Haag
6	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
7	3	* All rights reserved.
8	4	*

69	65	#endif
70	66
71	67	#include "util.h"
	68	#include "nfdump.h"
72	69	#include "nffile.h"
73	70	#include "nfx.h"
74		#include "nf_common.h"
75	71	#include "nfnet.h"
76	72	#include "bookkeeper.h"
77	73	#include "collector.h"
	74	#include "launch.h"
78	75	#include "flist.h"
79	76	#include "nfstatfile.h"
80	77

96	93	#define DEFAULTSFLOWPORT "6343"
97	94
98	95	/* Global Variables */
99		caddr_t shmem;
	96	void *shmem;
100	97
101	98	/* globals */
102	99	int verbose = 0;

329	326
330	327	} // End of SetPriv
331	328
	329	static void format_file_block_header(data_block_header_t *header) {
	330
	331	printf("\n"
	332	"File Block Header: \n"
	333	" NumBlocks = %10u\n"
	334	" Size = %10u\n"
	335	" id = %10u\n",
	336	header->NumRecords,
	337	header->size,
	338	header->id);
	339
	340	} // End of format_file_block_header
	341
332	342	#include "nffile_inline.c"
333	343	#include "collector_inline.c"
334	344

343	353	ssize_t cnt;
344	354	void *in_buff;
345	355	int err;
346		char *string;
347	356	srecord_t *commbuff;
348	357
349		Init_sflow();
	358	Init_sflow(verbose);
350	359
351	360	in_buff = malloc(NETWORK_INPUT_BUFF_SIZE);
352	361	if ( !in_buff ) {

356	365
357	366	// init vars
358	367	commbuff = (srecord_t *)shmem;
359		string = NULL;
360	368
361	369	// Init each netflow source output data buffer
362	370	fs = FlowSource;

383	391	cnt = 0;
384	392	ignored_packets = 0;
385	393
386		// wake up at least at next time slot (twin) + some Overdue time
387		alarm(t_start + twin + OVERDUE_TIME - time(NULL));
	394	// wake up at least at next time slot (twin) + 1s
	395	alarm(t_start + twin + 1 - time(NULL));
388	396	/*
389	397	* Main processing loop:
390	398	* this loop, continues until done = 1, set by the signal handler

433	441	t_now = tv.tv_sec;
434	442
435	443	if ( ((t_now - t_start) >= twin) \|\| done ) {
436		char subfilename[64];
437	444	struct tm *now;
438		char *subdir, fmt[64];
	445	char *subdir, fmt[MAXTIMESTRING];
	446
439	447	alarm(0);
440	448	now = localtime(&t_start);
441		strftime(fmt, sizeof fmt, time_extension, now);
	449	strftime(fmt, sizeof(fmt), time_extension, now);
442	450
443	451	// prepare sub dir hierarchy
444	452	if ( use_subdirs ) {

449	457
450	458	// failed to generate subdir path - put flows into base directory
451	459	subdir = NULL;
452		snprintf(subfilename, 63, "nfcapd.%s", fmt);
453		} else {
454		snprintf(subfilename, 63, "%s/nfcapd.%s", subdir, fmt);
455	460	}
456	461	} else {
457	462	subdir = NULL;
458		snprintf(subfilename, 63, "nfcapd.%s", fmt);
459	463	}
460		subfilename[63] = '\0';
461	464
462	465	// for each flow source update the stats, close the file and re-initialize the new file
463	466	fs = FlowSource;

468	471
469	472	if ( verbose ) {
470	473	// Dump to stdout
471		format_file_block_header(nffile->block_header, &string, 0);
472		printf("%s\n", string);
	474	format_file_block_header(nffile->block_header);
473	475	}
474	476
475	477	if ( nffile->block_header->NumRecords ) {
476	478	// flush current buffer to disc
477	479	if ( WriteBlock(nffile) <= 0 )
478		LogError("Ident: %s, failed to write output buffer to disk: '%s'" , fs->Ident, strerror(errno));
	480	LogError("Ident: %s, failed to write output buffer to disk: '%s'" ,
	481	fs->Ident, strerror(errno));
479	482	} // else - no new records in current block
480	483
481	484
482	485	// prepare filename
483		snprintf(nfcapd_filename, MAXPATHLEN-1, "%s/%s", fs->datadir, subfilename);
	486	if ( subdir ) {
	487	if ( SetupSubDir(fs->datadir, subdir, error, 255) ) {
	488	snprintf(nfcapd_filename, MAXPATHLEN-1, "%s/%s/nfcapd.%s", fs->datadir, subdir, fmt);
	489	} else {
	490	LogError("Ident: %s, Failed to create sub hier directories: %s", fs->Ident, error );
	491	// skip subdir - put flows directly into current directory
	492	snprintf(nfcapd_filename, MAXPATHLEN-1, "%s/nfcapd.%s", fs->datadir, fmt);
	493	}
	494	} else {
	495	snprintf(nfcapd_filename, MAXPATHLEN-1, "%s/nfcapd.%s", fs->datadir, fmt);
	496	}
484	497	nfcapd_filename[MAXPATHLEN-1] = '\0';
485
	498
486	499	// update stat record
487		nffile->stat_record->first_seen = fs->first_seen/1000;
	500	// if no flows were collected, fs->last_seen is still 0
	501	// set first_seen to start of this time slot, with twin window size.
	502	if ( fs->last_seen == 0 ) {
	503	fs->first_seen = (uint64_t)1000 * (uint64_t)t_start;
	504	fs->last_seen = (uint64_t)1000 * (uint64_t)(t_start + twin);
	505	}
	506	nffile->stat_record->first_seen = fs->first_seen/1000;
488	507	nffile->stat_record->msec_first = fs->first_seen - nffile->stat_record->first_seen*1000;
489	508	nffile->stat_record->last_seen = fs->last_seen/1000;
490	509	nffile->stat_record->msec_last = fs->last_seen - nffile->stat_record->last_seen*1000;

494	513	// Write Stat record and close file
495	514	CloseUpdateFile(nffile, fs->Ident);
496	515
497		if ( subdir && !SetupSubDir(fs->datadir, subdir, error, 255) ) {
498		// in this case the flows get lost! - the rename will fail
499		// but this should not happen anyway, unless i/o problems, inode problems etc.
500		LogError("Ident: %s, Failed to create sub hier directories: %s", fs->Ident, error );
501		}
502
503	516	// if rename fails, we are in big trouble, as we need to get rid of the old .current file
504	517	// otherwise, we will loose flows and can not continue collecting new flows
505	518	err = rename(fs->current, nfcapd_filename);

523	536
524	537	// log stats
525	538	LogInfo("Ident: '%s' Flows: %llu, Packets: %llu, Bytes: %llu, Sequence Errors: %u, Bad Packets: %u",
526		fs->Ident, (unsigned long long)nffile->stat_record->numflows, (unsigned long long)nffile->stat_record->numpackets,
	539	fs->Ident, (unsigned long long)nffile->stat_record->numflows,
	540	(unsigned long long)nffile->stat_record->numpackets,
527	541	(unsigned long long)nffile->stat_record->numbytes, nffile->stat_record->sequence_failure, fs->bad_packets);
528	542
529	543	// reset stat record

550	564	if ( launcher_pid ) {
551	565	// Signal launcher
552	566
553		// prepare filename for %f expansion
554		strncpy(commbuff->fname, subfilename, FNAME_SIZE-1);
555		commbuff->fname[FNAME_SIZE-1] = 0;
556		snprintf(commbuff->tstring, 16, "%i%02i%02i%02i%02i",
557		now->tm_year + 1900, now->tm_mon + 1, now->tm_mday, now->tm_hour, now->tm_min);
558		commbuff->tstring[15] = 0;
	567	strncpy(commbuff->tstring, fmt, MAXTIMESTRING);
	568	commbuff->tstring[MAXTIMESTRING-1] = '\0';
	569
559	570	commbuff->tstamp = t_start;
560		if ( subdir )
561		strncpy(commbuff->subdir, subdir, FNAME_SIZE);
562		else
563		commbuff->subdir[0] = '\0';
	571	if ( subdir ) {
	572	snprintf(commbuff->fname, MAXPATHLEN-1, "%s/nfcapd.%s", subdir, fmt);
	573	} else {
	574	snprintf(commbuff->fname, MAXPATHLEN-1, "nfcapd.%s", fmt);
	575	}
	576	commbuff->fname[MAXPATHLEN-1] = '\0';
564	577
565	578	if ( launcher_alive ) {
566	579	LogInfo("Signal launcher");

580	593	t_start += twin;
581	594	/* t_start = filename time stamp: begin of slot
582	595	* + twin = end of next time interval
583		* + OVERDUE_TIME = if no data is collected, this is at latest to act
	596	* + 1 = if no data is collected, this is at latest to act
584	597	* - t_now = difference value to now
585	598	*/
586		alarm(t_start + twin + OVERDUE_TIME - t_now);
	599	alarm(t_start + twin + 1 - t_now);
587	600	}
588	601
589	602	/* check for error condition or done . errno may only be EINTR */

659	672
660	673	char bindhost, datadir, pidstr[32], *launch_process;
661	674	char userid, groupid, checkptr, listenport, mcastgroup, extension_tags;
662		char Ident, pcap_file, *time_extension, pidfile[MAXPATHLEN];
	675	char Ident, time_extension, pidfile[MAXPATHLEN];
663	676	struct stat fstat;
664	677	packet_function_t receive_packet;
665	678	repeater_t repeater[MAX_REPEATERS];

670	683	int sock, synctime, do_daemonize, expire, spec_time_extension, report_sequence;
671	684	int subdir_index, compress;
672	685	int c, i;
	686	#ifdef PCAP
	687	char *pcap_file = NULL;
	688	#endif
673	689
674	690	receive_packet = recvfrom;
675	691	verbose = synctime = do_daemonize = 0;

688	704	datadir = NULL;
689	705	subdir_index = 0;
690	706	time_extension = "%Y%m%d%H%M";
	707	spec_time_extension = 0;
691	708	expire = 0;
692		spec_time_extension = 0;
693	709	compress = NOT_COMPRESSED;
694	710	memset((void *)&repeater, 0, sizeof(repeater));
695	711	for ( i = 0; i < MAX_REPEATERS; i++ ) {

698	714	Ident = "none";
699	715	FlowSource = NULL;
700	716	extension_tags = DefaultExtensions;
701		pcap_file = NULL;
702	717
703	718	while ((c = getopt(argc, argv, "46ewhEVI:DB:b:f:jl:n:p:J:P:R:S:T:t:x:ru:g:zZ")) != EOF) {
704	719	switch (c) {

794	809	exit(255);
795	810	}
796	811	tmp[MAXPATHLEN-1] = 0;
797		snprintf(pidfile, MAXPATHLEN - 1 - strlen(tmp), "%s/%s", tmp, optarg);
	812	if ( (strlen(tmp) + strlen(optarg) + 3) < MAXPATHLEN ) {
	813	snprintf(pidfile, MAXPATHLEN - 3 - strlen(tmp), "%s/%s", tmp, optarg);
	814	} else {
	815	fprintf(stderr, "pidfile MAXPATHLEN error:\n");
	816	exit(255);
	817	}
798	818	}
799	819	// pidfile now absolute path
800	820	pidfile[MAXPATHLEN-1] = 0;

847	867	break;
848	868	case 't':
849	869	twin = atoi(optarg);
850		if ( twin <= 0 ) {
851		fprintf(stderr, "ERROR: time frame <= 0\n");
	870	if ( twin < 2 ) {
	871	LogError("time interval <= 2s not allowed");
852	872	exit(255);
853	873	}
854	874	if (twin < 60) {
855		fprintf(stderr, "WARNING, Very small time frame - < 60s!\n");
856		exit(255);
	875	time_extension = "%Y%m%d%H%M%S";
857	876	}
858	877	break;
859	878	case 'x':

906	925	exit(255);
907	926	}
908	927
909		if ( !InitLog(argv[0], SYSLOG_FACILITY)) {
	928	if ( !InitLog(do_daemonize, argv[0], SYSLOG_FACILITY, verbose) ) {
910	929	exit(255);
911	930	}
912	931

1023	1042	// as well as shared memory
1024	1043	// prepare shared memory
1025	1044	shmem = mmap(0, sizeof(srecord_t), PROT_READ \| PROT_WRITE, MAP_ANON \| MAP_SHARED, -1, 0);
1026		if ( shmem == (caddr_t)-1 ) {
	1045	if ( shmem == MAP_FAILED ) {
1027	1046	LogError("mmap() error: %s", strerror(errno));
1028	1047	close(sock);
1029	1048	exit(255);

1034	1053	case 0:
1035	1054	// child
1036	1055	close(sock);
1037		launcher((char *)shmem, FlowSource, launch_process, expire);
	1056	launcher(shmem, FlowSource, launch_process, expire);
1038	1057	exit(0);
1039	1058	break;
1040	1059	case -1:

+15

-24

bin/sflow_nfdump.c less more

0	0	/*
1		* Copyright (c) 2017, Peter Haag
2		* Copyright (c) 2016, Peter Haag
3		* Copyright (c) 2014, Peter Haag
4		* Copyright (c) 2009, Peter Haag
	1	* Copyright (c) 2009-2020, Peter Haag
5	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
6	3	* All rights reserved.
7	4	*

60	57	#include <stdint.h>
61	58	#endif
62	59
	60	#include "util.h"
	61	#include "nfdump.h"
63	62	#include "nffile.h"
64	63	#include "nfx.h"
65		#include "nf_common.h"
66		#include "util.h"
	64	#include "output_raw.h"
67	65	#include "bookkeeper.h"
68	66	#include "collector.h"
69	67

72	70	#include "sflow_process.h"
73	71	#include "sflow_nfdump.h"
74	72
75		#ifndef DEVEL
76		# define dbg_printf(...) /* printf(__VA_ARGS__) */
77		#else
78		# define dbg_printf(...) printf(__VA_ARGS__)
79		#endif
80
81	73	#define MAX_SFLOW_EXTENSIONS 8
82	74
83	75	typedef struct exporter_sflow_s {
84	76	// link chain
85	77	struct exporter_sflow_s *next;
86	78
87		// generic exporter information
	79	// exporter information
88	80	exporter_info_record_t info;
89	81
90	82	uint64_t packets; // number of packets sent by this exporter
91	83	uint64_t flows; // number of flow records sent by this exporter
92	84	uint32_t sequence_failure; // number of sequence failues
93	85
94		generic_sampler_t *sampler;
	86	sampler_t *sampler;
95	87
96	88	// extension map
97	89	// extension maps are common for all exporters

100	92	} exporter_sflow_t;
101	93
102	94	extern extension_descriptor_t extension_descriptor[];
103		extern FlowSource_t *FlowSource;
104	95
105	96	/* module limited globals */
106	97

151	142	#define SFLOW_NEXT_HOP_BGP 2
152	143	#define SFLOW_ROUTER_IP 4
153	144
154		extern int verbose;
155
	145	static int verbose = 0;
156	146	static int IP_extension_mask = 0;
157	147
158	148	static int Setup_Extension_Info(FlowSource_t fs, exporter_sflow_t exporter, int num);

161	151
162	152	#include "inline.c"
163	153	#include "nffile_inline.c"
164		#include "collector_inline.c"
165
166		void Init_sflow(void) {
	154
	155	void Init_sflow(int v) {
167	156	int i, id;
	157
	158	verbose = v;
168	159
169	160	i=0;
170	161	Num_enabled_extensions = 0;

250	241	map_size += 2;
251	242
252	243
253		// Create a generic sflow extension map
	244	// Create a sflow extension map
254	245	exporter->sflow_extension_info[num].map = (extension_map_t *)malloc((size_t)map_size);
255	246	if ( !exporter->sflow_extension_info[num].map ) {
256	247	LogError("SFLOW: malloc() allocation error in %s line %d: %s", __FILE__, __LINE__, strerror(errno) );

321	312
322	313	static exporter_sflow_t GetExporter(FlowSource_t fs, uint32_t agentSubId, uint32_t meanSkipCount) {
323	314	exporter_sflow_t e = (exporter_sflow_t )&(fs->exporter_data);
324		generic_sampler_t *sampler;
	315	sampler_t *sampler;
325	316	#define IP_STRING_LEN 40
326	317	char ipstr[IP_STRING_LEN];
327	318	int i;

369	360	(*e)->sflow_extension_info[i].map = NULL;
370	361	}
371	362
372		sampler = (generic_sampler_t *)malloc(sizeof(generic_sampler_t));
	363	sampler = (sampler_t *)malloc(sizeof(sampler_t));
373	364	if ( !sampler ) {
374	365	LogError("SFLOW: malloc() error in %s line %d: %s", __FILE__, __LINE__, strerror (errno));
375	366	return NULL;

670	661	master_record_t master_record;
671	662	char *string;
672	663	ExpandRecord_v2((common_record_t *)common_record, &exporter->sflow_extension_info[ip_flags], &(exporter->info), &master_record);
673		format_file_block_record(&master_record, &string, 0);
	664	flow_record_to_raw(&master_record, &string, 0);
674	665	printf("%s\n", string);
675	666	}
676	667

-2

bin/sflow_nfdump.h less more

0	0	/*
1		* Copyright (c) 2017, Peter Haag
	1	* Copyright (c) 2017-2020, Peter Haag
2	2	* All rights reserved.
3	3	*
4	4	* Redistribution and use in source and binary forms, with or without

39	39	#include "collector.h"
40	40	#include "sflow_process.h"
41	41
42		void Init_sflow(void);
	42	void Init_sflow(int v);
43	43
44	44	void Process_sflow(void in_buff, ssize_t in_buff_cnt, FlowSource_t fs);
45	45

-5

bin/sflow_process.c less more

0	0	/*
1		* Copyright (c) 2017, Peter Haag
	1	* Copyright (c) 2017-2020, Peter Haag
2	2	* All rights reserved.
3	3	*
4	4	* Redistribution and use in source and binary forms, with or without

63	63	#include <stdint.h>
64	64	#endif
65	65
	66	#include "nfdump.h"
66	67	#include "sflow.h" /* sFlow v5 */
67	68	#include "sflow_v2v4.h" /* sFlow v2/4 */
68	69	#include "util.h"

3373	3374	}
3374	3375	}
3375	3376
3376		if(sample->gotIPV4 \|\| sample->gotIPV6)
3377		StoreSflowRecord(sample, fs);
	3377	StoreSflowRecord(sample, fs);
3378	3378
3379	3379	if ( verbose )
3380	3380	writeFlowLine(sample);

3513	3513	}
3514	3514	lengthCheck(sample, "flow_sample", sampleStart, sampleLength);
3515	3515
3516		if ( sample->gotIPV4 \|\| sample->gotIPV6 )
3517		StoreSflowRecord(sample, fs);
	3516	StoreSflowRecord(sample, fs);
3518	3517
3519	3518	/* or line-by-line output... */
3520	3519	if ( verbose )

-1

bin/test.sh less more

53	53
54	54	./nfdump -r test.flows -O tstart -z -w test2.flows
55	55	./nfdump -q -r test2.flows -o raw > test4.out
56		diff -u test4.out nfdump.test.out
	56	diff test4.out nfdump.test.out > test4.diff \|\| true
	57	diff test4.diff nfdump.test2.diff
	58
57	59
58	60	# uncompressed flow test
59	61	rm -f test.flows test2.out

+60

-18

bin/util.c less more

0	0	/*
1		* Copyright (c) 2016, Peter Haag
2		* Copyright (c) 2014, Peter Haag
3		* Copyright (c) 2009, Peter Haag
	1	* Copyright (c) 2009-2020, Peter Haag
4	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
5	3	* All rights reserved.
6	4	*

42	40	#include <errno.h>
43	41	#include <sys/types.h>
44	42	#include <sys/select.h>
	43	#include <sys/socket.h>
	44	#include <arpa/inet.h>
45	45
46	46	#ifndef SYSLOG_NAMES
47	47	# define SYSLOG_NAMES 1

58	58
59	59	/* Global vars */
60	60
61		extern char *CurrentIdent;
62
63		enum { NONE, LESS, MORE };
	61	static int verbose = 0;
64	62
65	63	/* Function prototypes */
66	64	static int check_number(char *s, int len);
67	65
68	66	static int ParseTime(char s, time_t t_start);
69	67
70		uint32_t twin_first, twin_last;
	68	uint32_t twin_first, twin_last;
71	69
72	70	static int use_syslog = 0;
73	71

119	117	closelog();
120	118	} // End of CloseLog
121	119
122		int InitLog(char name, char facility) {
	120	int InitLog(int use_syslog, char name, char facility, int verbose_log) {
123	121	int i;
124	122	char *logname;
	123
	124	verbose = verbose_log;
	125	if ( !use_syslog )
	126	return 1;
125	127
126	128	if ( !facility \|\| strlen(facility) > 32 ) {
127	129	fprintf(stderr, "Invalid syslog facility name '%s'!\n", facility);

346	348	char *p;
347	349
348	350	if ( !tstring ) {
349		fprintf(stderr,"Time Window format error '%s'\n", tstring);
	351	fprintf(stderr,"Time Window format error\n");
350	352	return 0;
351	353	}
352	354

417	419
418	420	char *UNIX2ISO(time_t t) {
419	421	struct tm *when;
420		static char timestring[16];
	422	static char timestring[32];
421	423
422	424	when = localtime(&t);
423	425	when->tm_isdst = -1;
424		snprintf(timestring, 15, "%i%02i%02i%02i%02i",
425		when->tm_year + 1900, when->tm_mon + 1, when->tm_mday, when->tm_hour, when->tm_min);
426		timestring[15] = '\0';
	426	snprintf(timestring, 31, "%4i%02i%02i%02i%02i%02i",
	427	when->tm_year + 1900, when->tm_mon + 1, when->tm_mday, when->tm_hour, when->tm_min, when->tm_sec);
	428	timestring[31] = '\0';
427	429
428	430	return timestring;
429	431

442	444	when.tm_yday = 0;
443	445	when.tm_isdst = -1;
444	446
445		if ( strlen(timestring) != 12 ) {
	447	size_t len = strlen(timestring);
	448	if ( len != 12 && len != 14) {
446	449	LogError( "Wrong time format '%s'\n", timestring);
447	450	return 0;
448	451	}
449		// 2006 05 05 12 00
	452	// 2019 05 05 12 00 (10)
450	453	// year
451	454	p = timestring;
452	455	c = p[4];

477	480
478	481	// minute
479	482	p += 2;
	483	c = p[2];
	484	p[2] = '\0';
480	485	when.tm_min = atoi(p);
	486	p[2] = c;
481	487
	488	if ( len == 14 ) {
	489	p += 2;
	490	when.tm_sec = atoi(p);
	491	}
	492
482	493	t = mktime(&when);
483	494	if ( t == -1 ) {
484	495	LogError( "Failed to convert string '%s'\n", timestring);

524	535
525	536	} // End of InsertString
526	537
527		void format_number(uint64_t num, char *s, int scale, int fixed_width) {
	538	void format_number(uint64_t num, char *s, int printPlain, int fixed_width) {
528	539	double f = num;
529	540
530		if ( !scale ) {
	541	if ( printPlain ) {
531	542	snprintf(s, 31, "%llu", (long long unsigned)num);
532	543	} else {
533	544

557	568
558	569	} // End of format_number
559	570
560
	571	void inet_ntop_mask(uint32_t ipv4, int mask, char *s, size_t sSize) {
	572
	573	if ( mask ) {
	574	ipv4 &= 0xffffffffL << ( 32 - mask );
	575	ipv4 = htonl(ipv4);
	576	inet_ntop(AF_INET, &ipv4, s, sSize);
	577	} else {
	578	s[0] = '\0';
	579	}
	580
	581	} // End of inet_ntop_mask
	582
	583	void inet6_ntop_mask(uint64_t ipv6[2], int mask, char *s, size_t sSize) {
	584	uint64_t ip[2];
	585
	586	ip[0] = ipv6[0];
	587	ip[1] = ipv6[1];
	588	if ( mask ) {
	589	if ( mask <= 64 ) {
	590	ip[0] = ip[0] & (0xffffffffffffffffLL << (64 - mask));
	591	ip[1] = 0;
	592	} else {
	593	ip[1] = ip[1] & (0xffffffffffffffffLL << (128 - mask));
	594	}
	595	ip[0] = htonll(ip[0]);
	596	ip[1] = htonll(ip[1]);
	597	inet_ntop(AF_INET6, ip, s, sSize);
	598
	599	} else {
	600	s[0] = '\0';
	601	}
	602	} // End of inet_ntop_mask

+21

-8

bin/util.h less more

0	0	/*
1		* Copyright (c) 2017, Peter Haag
2		* Copyright (c) 2016, Peter Haag
3		* Copyright (c) 2014, Peter Haag
4		* Copyright (c) 2009, Peter Haag
	1	* Copyright (c) 2009-2020, Peter Haag
5	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
6	3	* All rights reserved.
7	4	*

49	46	# include <assert.h>
50	47	# define dbg_printf(...) printf(__VA_ARGS__)
51	48	# define dbg_assert(a) assert(a)
	49	# define dbg(a) a
52	50	#else
53	51	# define dbg_printf(...) /* printf(__VA_ARGS__) */
54	52	# define dbg_assert(a) /* assert(a) */
	53	# define dbg(a) /* a */
55	54	#endif
56	55
57	56	#define UNUSED(expr) do { (void)(expr); } while (0)

68	67	#endif
69	68	#endif
70	69
	70	#if ( SIZEOF_VOID_P == 8 )
	71	typedef uint64_t pointer_addr_t;
	72	#else
	73	typedef uint32_t pointer_addr_t;
	74	#endif
	75
71	76	#define _1KB (double)(1000.0)
72	77	#define _1MB (double)(1000.0 * 1000.0)
73	78	#define _1GB (double)(1000.0 * 1000.0 * 1000.0)
74	79	#define _1TB (double)(1000.0 * 1000.0 * 1000.0 * 1000.0)
	80
	81	#define SetFlag(var, flag) (var \|= flag)
	82	#define ClearFlag(var, flag) (var &= ~flag)
	83	#define TestFlag(var, flag) (var & flag)
	84
75	85
76	86	typedef struct stringlist_s {
77	87	uint32_t block_size;

80	90	char **list;
81	91	} stringlist_t;
82	92
	93
83	94	void xsleep(long sec);
84	95
85	96	void EndLog(void);
86	97
87		int InitLog(char name, char facility);
	98	int InitLog(int use_syslog, char name, char facility, int verbose_log);
88	99
89	100	void LogError(char *format, ...);
90	101

103	114	time_t ISO2UNIX(char *timestring);
104	115
105	116	#define NUMBER_STRING_SIZE 32
106		#define DONT_SCALE_NUMBER 0
107		#define DO_SCALE_NUMBER 1
108	117	#define FIXED_WIDTH 1
109	118	#define VAR_LENGTH 0
110		void format_number(uint64_t num, char *s, int scale, int fixed_width);
	119	void format_number(uint64_t num, char *s, int printPlain, int fixed_width);
111	120
112	121	void SetupInputFileSequence(char multiple_dirs, char single_file, char *multiple_files);
113	122

115	124
116	125	void Setv6Mode(int mode);
117	126
	127	void inet_ntop_mask(uint32_t ipv4, int mask, char *s, size_t sSize);
	128
	129	void inet6_ntop_mask(uint64_t ipv6[2], int mask, char *s, size_t sSize);
	130
118	131	#endif //_UTIL_H

+31

-32

configure.ac less more

1	1	# Process this file with autoconf to produce a configure script.
2	2
3	3	AC_PREREQ(2.59)
4		AC_REVISION($Revision: 243 $)dnl
5		AC_INIT(nfdump, 1.6.18, peter@people.ops-trust.net)
6		# $Date: 2014-11-16 14:10:20 +0100 (Sun, 16 Nov 2014) $
7		#AC_CONFIG_SRCDIR([grammar.y])
	4	AC_REVISION($Revision: 244 $)dnl
	5	AC_INIT(nfdump, 1.6.20, peter@people.ops-trust.net)
	6
8	7	AC_CONFIG_HEADER([config.h])
9	8	AM_INIT_AUTOMAKE([subdir-objects])
10	9
	10	# Checks for programs.
	11	CFLAGS="-g -O3"
	12	AC_PROG_CC([clang gcc])
	13	AX_CHECK_C11
	14	CFLAGS="$CFLAGS -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wmissing-noreturn -fno-strict-aliasing"
	15
11	16	LT_INIT
12		# AC_ENABLE_SHARED
13		# AC_ENABLE_STATIC
14
15		# Checks for programs.
16		AC_PROG_CC
17		AM_PROG_CC_C_O
18
19		dnl get the flags
20		CFLAGS="${CFLAGS=}"
21		if test $ac_cv_prog_gcc = yes -a "x$CFLAGS" = "x-g -O2"; then
22		CFLAGS="-g -O2 -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wmissing-noreturn -fno-strict-aliasing"
23		fi
24	17
25	18	AC_ARG_ENABLE(devel,
26	19	[ --enable-devel compile debug and development code into nfdump; default is NO])

28	21	# Which way is better?
29	22	if test "${enable_devel}" = "yes" ; then
30	23	CFLAGS="$CFLAGS -DDEVEL"
31		cat >>config.h <<_ACEOF
32		#define dbg_printf(...) printf(__VA_ARGS__)
33		_ACEOF
34		else
35		cat >>config.h <<_ACEOF
36		#define dbg_printf(...) /* printf(__VA_ARGS__) */
37		_ACEOF
38	24	fi
39	25
40	26	AC_ARG_ENABLE(nsel,

42	28
43	29	if test "${enable_nsel}" = "yes" ; then
44	30	CFLAGS="$CFLAGS -DNSEL"
	31	fi
	32
	33	AC_ARG_ENABLE(jnat,
	34	[ --enable-jnat compile nfdump, to read and process Junos NAT event logging; default is NO])
	35
	36	if test "${enable_jnat}" = "yes" ; then
	37	CFLAGS="$CFLAGS -DNSEL -DJUNOS"
	38	fi
	39
	40	if test "${enable_nsel}" = "yes" -a "${enable_jnat}" = "yes"; then
	41	AC_MSG_ERROR(You can use only one of --enable-nsel or --enable-jnat. CISCO and Juniper are not compatible.)
45	42	fi
46	43
47	44	AC_ARG_ENABLE(nel,

117	114	if test ! -f "$WHERE_FTPATH/include/ftlib.h"; then
118	115	AC_MSG_ERROR(ftlib.h file not found in flow-tools directory '$WHERE_FTPATH'. Use --with-ftpath=PATH)
119	116	fi
120		if test ! -f "$WHERE_FTPATH/lib/libft.a" -a -f "$WHERE_FTPATH/lib64/libft.a" ! -f "$WHERE_FTPATH/lib/libft.so" -a -f "$WHERE_FTPATH/lib64/libft.so"; then
	117	if test -f "$WHERE_FTPATH/lib/libft.a" -o -f "$WHERE_FTPATH/lib64/libft.a" -o -f "$WHERE_FTPATH/lib/libft.so" -o -f "$WHERE_FTPATH/lib64/libft.so"; then
	118	FT_INCLUDES="-I$WHERE_FTPATH/include -I$WHERE_FTPATH/lib"
	119	FT_LDFLAGS="-L$WHERE_FTPATH/lib"
	120	else
121	121	AC_MSG_ERROR(libft.a not found in flow-tools directory '$WHERE_FTPATH'. Build flow tools first)
122	122	fi
123		FT_INCLUDES="-I$WHERE_FTPATH/include -I$WHERE_FTPATH/lib"
124		FT_LDFLAGS="-L$WHERE_FTPATH/lib"
125	123	else
126	124	AC_MSG_ERROR(flow-tools directory '$WHERE_FTPATH' does not exists. Use --with-ftpath=PATH)
127	125	fi

148	146	else
149	147	AC_MSG_ERROR(Required rrd.h header file not found!)
150	148	fi
151		AC_RUN_IFELSE(
	149	AC_COMPILE_IFELSE(
152	150	[ AC_LANG_PROGRAM(
153	151	[[
154	152	#include <stdio.h>

203	201	else
204	202	AC_MSG_ERROR(Required rrd.h header file not found!)
205	203	fi
206		AC_RUN_IFELSE(
	204	AC_COMPILE_IFELSE(
207	205	[ AC_LANG_PROGRAM(
208	206	[[
209	207	#include <stdio.h>

244	242	else
245	243	AC_MSG_ERROR(Required pcap.h header file not found!)
246	244	fi
247		AC_RUN_IFELSE(
	245	AC_COMPILE_IFELSE(
248	246	[ AC_LANG_PROGRAM(
249	247	[[
250	248	#include <stdio.h>

312	310	# Checks for header files.
313	311	AC_HEADER_DIRENT
314	312	AC_HEADER_STDC
	313	AC_CHECK_HEADERS(stdio_ext.h)
315	314	AC_CHECK_HEADERS([nameser8_compat.h])
316	315	AC_CHECK_HEADERS([features.h arpa/inet.h fcntl.h netinet/in.h fts.h stdint.h stdlib.h stddef.h string.h sys/socket.h syslog.h unistd.h iso/limits_iso.h])
317	316	AC_CHECK_HEADERS(pcap-bpf.h net/bpf.h)

394	393	AC_CHECK_SIZEOF(__int64)
395	394	AC_CHECK_SIZEOF(void *)
396	395	AC_CHECK_SIZEOF(size_t)
	396	AC_CHECK_SIZEOF(time_t)
397	397	AC_CHECK_SIZEOF(ptrdiff_t)
398	398	AC_C_CONST
399	399	AC_CHECK_FUNCS(memcmp memcpy memmove memset)

459	459
460	460	echo ""
461	461	echo "* Many thanks for using nfdump tools"
	462	echo "* Please send bug reports back to me: peter@people.ops-trust.net"
	463	echo "* or open a ticket at https://github.com/phaag/nfdump/issues"
462	464	echo "* You may want to subscribe to the nfdump-discuss and/or"
463	465	echo "* nfsen-discuss mailing list:"
464	466	echo "* http://lists.sourceforge.net/lists/listinfo/nfdump-discuss"
465	467	echo "* http://lists.sourceforge.net/lists/listinfo/nfsen-discuss"
466		echo "* Please send bug reports back to me: phaag@users.sourceforge.net"
467		echo "* or to one of the lists."
468

+59

-0

extra/docker/readme less more

	0	Dockerfile
	1	#
	2	# Ubuntu Dockerfile
	3	#
	4	# https://github.com/dockerfile/ubuntu
	5	#
	6
	7	# Pull base image.
	8	FROM ubuntu:latest
	9
	10	#Expose sflow port
	11	EXPOSE 6343/udp
	12
	13
	14	# Install.
	15	RUN apt-get update && apt-get install -y \
	16	wget \
	17	unzip \
	18	man \
	19	apt-utils \
	20	dialog \
	21	pkg-config \
	22	rrdtool \
	23	librrd-dev \
	24	libtool \
	25	autoconf \
	26	autogen \
	27	bison \
	28	byacc \
	29	flex \
	30	libbz2-dev \
	31	&& rm -rf /var/lib/apt/lists/*
	32
	33	RUN cd /usr/src \
	34	&& wget https://github.com/phaag/nfdump/archive/v1.6.20.zip \
	35	&& unzip v1.6.20.zip \
	36	&& cd nfdump-1.6.20\
	37	&& mkdir m4 \
	38	&& ./autogen.sh \
	39	&& ./configure --enable-nfprofile --enable-sflow \
	40	&& make \
	41	&& make install
	42
	43	RUN ldconfig
	44
	45	# Add files.
	46	#ADD root/.bashrc /root/.bashrc
	47	#ADD root/.gitconfig /root/.gitconfig
	48	#ADD root/.scripts /root/.scripts
	49
	50	# Set environment variables.
	51	#ENV HOME /root
	52
	53	# Define working directory.
	54	WORKDIR /usr/src
	55
	56	# Define default command.
	57	CMD ["bash"]
	58

-2

extra/nfdump.spec less more

0	0	Name: nfdump
1	1	Summary: A set of command-line tools to collect and process netflow data
2		Version: 1.6.18
	2	Version: 1.6.20
3	3	Release: 1
4	4	License: BSD
5	5	Group: Applications/System

40	40
41	41	%changelog
42	42	* Thu Jan 03 2019 Richard REY <Rexy>
43		- Version 1.6.18 (used in ALCASAR 3.3.3)
	43	- Version 1.6.20 (used in ALCASAR 3.3.3)

-25

extra/nftrack/nftrack.c less more

0	0	/*
1		* Copyright (c) 2014, Peter Haag
2		* Copyright (c) 2009, Peter Haag
3		* Copyright (c) 2004, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
	1	* Copyright (c) 2009-2020, Peter Haag
	2	* Copyright (c) 2004-2008, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
4	3	* All rights reserved.
5	4	*
6	5	* Redistribution and use in source and binary forms, with or without

27	26	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
28	27	* POSSIBILITY OF SUCH DAMAGE.
29	28	*
30		* $Author: peter $
31		*
32		* $Id: nftrack.c 224 2014-02-16 12:59:29Z peter $
33		*
34		* $LastChangedRevision: 224 $
35		*
36	29	*/
37	30
38	31	#include <stdio.h>

57	50	#include <stdint.h>
58	51	#endif
59	52
60		#include "nf_common.h"
	53	#include "util.h"
	54	#include "nfdump.h"
61	55	#include "nffile.h"
	56	#include "nfx.h"
	57	#include "exporter.h"
62	58	#include "flist.h"
63		#include "rbtree.h"
64	59	#include "nftree.h"
65		#include "nfdump.h"
66		#include "nfx.h"
67		#include "util.h"
68		#include "grammar.h"
69	60
70	61	#include "nftrack_stat.h"
71	62	#include "nftrack_rrd.h"

73	64	// We have 288 slot ( 1 day ) for stat record
74	65	#define AVG_STAT 1
75	66
76		/* Externals */
77		extern int yydebug;
78
79	67	/* Global Variables */
80		FilterEngine_data_t *Engine;
	68	FilterEngine_t *Engine;
81	69	int byte_mode, packet_mode;
82	70	uint32_t byte_limit, packet_limit; // needed for linking purpose only
83	71

228	216	total_bytes += ret;
229	217	}
230	218
231		if ( nffile->block_header->id == Large_BLOCK_Type ) {
232		// skip
233		continue;
234		}
235
236	219	if ( nffile->block_header->id != DATA_BLOCK_TYPE_2 ) {
237	220	LogError("Can't process block type %u\n", nffile->block_header->id);
238	221	continue;

375	358	AddDB = 1;
376	359	break;
377	360	case 'L':
378		if ( !InitLog("nftrack", optarg) )
	361	if ( !InitLog(0, "nftrack", optarg, 0) )
379	362	exit(255);
380	363	break;
381	364	case 'S':

+50

-0

m4/ax_append_flag.m4 less more

	0	# ===========================================================================
	1	# https://www.gnu.org/software/autoconf-archive/ax_append_flag.html
	2	# ===========================================================================
	3	#
	4	# SYNOPSIS
	5	#
	6	# AX_APPEND_FLAG(FLAG, [FLAGS-VARIABLE])
	7	#
	8	# DESCRIPTION
	9	#
	10	# FLAG is appended to the FLAGS-VARIABLE shell variable, with a space
	11	# added in between.
	12	#
	13	# If FLAGS-VARIABLE is not specified, the current language's flags (e.g.
	14	# CFLAGS) is used. FLAGS-VARIABLE is not changed if it already contains
	15	# FLAG. If FLAGS-VARIABLE is unset in the shell, it is set to exactly
	16	# FLAG.
	17	#
	18	# NOTE: Implementation based on AX_CFLAGS_GCC_OPTION.
	19	#
	20	# LICENSE
	21	#
	22	# Copyright (c) 2008 Guido U. Draheim <guidod@gmx.de>
	23	# Copyright (c) 2011 Maarten Bosmans <mkbosmans@gmail.com>
	24	#
	25	# Copying and distribution of this file, with or without modification, are
	26	# permitted in any medium without royalty provided the copyright notice
	27	# and this notice are preserved. This file is offered as-is, without any
	28	# warranty.
	29
	30	#serial 8
	31
	32	AC_DEFUN([AX_APPEND_FLAG],
	33	[dnl
	34	AC_PREREQ(2.64)dnl for _AC_LANG_PREFIX and AS_VAR_SET_IF
	35	AS_VAR_PUSHDEF([FLAGS], [m4_default($2,_AC_LANG_PREFIX[FLAGS])])
	36	AS_VAR_SET_IF(FLAGS,[
	37	AS_CASE([" AS_VAR_GET(FLAGS) "],
	38	[" $1 "], [AC_RUN_LOG([: FLAGS already contains $1])],
	39	[
	40	AS_VAR_APPEND(FLAGS,[" $1"])
	41	AC_RUN_LOG([: FLAGS="$FLAGS"])
	42	])
	43	],
	44	[
	45	AS_VAR_SET(FLAGS,[$1])
	46	AC_RUN_LOG([: FLAGS="$FLAGS"])
	47	])
	48	AS_VAR_POPDEF([FLAGS])dnl
	49	])dnl AX_APPEND_FLAG

+53

-0

m4/ax_check_compile_flag.m4 less more

	0	# ===========================================================================
	1	# https://www.gnu.org/software/autoconf-archive/ax_check_compile_flag.html
	2	# ===========================================================================
	3	#
	4	# SYNOPSIS
	5	#
	6	# AX_CHECK_COMPILE_FLAG(FLAG, [ACTION-SUCCESS], [ACTION-FAILURE], [EXTRA-FLAGS], [INPUT])
	7	#
	8	# DESCRIPTION
	9	#
	10	# Check whether the given FLAG works with the current language's compiler
	11	# or gives an error. (Warnings, however, are ignored)
	12	#
	13	# ACTION-SUCCESS/ACTION-FAILURE are shell commands to execute on
	14	# success/failure.
	15	#
	16	# If EXTRA-FLAGS is defined, it is added to the current language's default
	17	# flags (e.g. CFLAGS) when the check is done. The check is thus made with
	18	# the flags: "CFLAGS EXTRA-FLAGS FLAG". This can for example be used to
	19	# force the compiler to issue an error when a bad flag is given.
	20	#
	21	# INPUT gives an alternative input source to AC_COMPILE_IFELSE.
	22	#
	23	# NOTE: Implementation based on AX_CFLAGS_GCC_OPTION. Please keep this
	24	# macro in sync with AX_CHECK_{PREPROC,LINK}_FLAG.
	25	#
	26	# LICENSE
	27	#
	28	# Copyright (c) 2008 Guido U. Draheim <guidod@gmx.de>
	29	# Copyright (c) 2011 Maarten Bosmans <mkbosmans@gmail.com>
	30	#
	31	# Copying and distribution of this file, with or without modification, are
	32	# permitted in any medium without royalty provided the copyright notice
	33	# and this notice are preserved. This file is offered as-is, without any
	34	# warranty.
	35
	36	#serial 6
	37
	38	AC_DEFUN([AX_CHECK_COMPILE_FLAG],
	39	[AC_PREREQ(2.64)dnl for _AC_LANG_PREFIX and AS_VAR_IF
	40	AS_VAR_PUSHDEF([CACHEVAR],[ax_cv_check_[]_AC_LANG_ABBREV[]flags_$4_$1])dnl
	41	AC_CACHE_CHECK([whether _AC_LANG compiler accepts $1], CACHEVAR, [
	42	ax_check_save_flags=$[]_AC_LANG_PREFIX[]FLAGS
	43	_AC_LANG_PREFIX[]FLAGS="$[]_AC_LANG_PREFIX[]FLAGS $4 $1"
	44	AC_COMPILE_IFELSE([m4_default([$5],[AC_LANG_PROGRAM()])],
	45	[AS_VAR_SET(CACHEVAR,[yes])],
	46	[AS_VAR_SET(CACHEVAR,[no])])
	47	_AC_LANG_PREFIX[]FLAGS=$ax_check_save_flags])
	48	AS_VAR_IF(CACHEVAR,yes,
	49	[m4_default([$2], :)],
	50	[m4_default([$3], :)])
	51	AS_VAR_POPDEF([CACHEVAR])dnl
	52	])dnl AX_CHECK_COMPILE_FLAGS

+11

-0

m4/c11.m4 less more

	0	AC_DEFUN([AX_CHECK_C11],
	1	[AX_CHECK_COMPILE_FLAG([-std=gnu11],
	2	[AX_APPEND_FLAG([-std=gnu11])],
	3	[AX_CHECK_COMPILE_FLAG([-std=c11],
	4	[AX_APPEND_FLAG([-std=c11])],
	5	[AX_CHECK_COMPILE_FLAG([-std=c99],
	6	[AX_APPEND_FLAG([-std=c99])],
	7	[AC_MSG_ERROR([C compiled does not support at least C99!])])
	8	])
	9	])
	10	])

-2

man/nfcapd.1 less more

9	9	data from the network and stores it into files. The output file
10	10	is automatically rotated and renamed every n minutes \- typically
11	11	5 min \- according the timestamp YYYYMMddhhmm of the interval e.g.
12		nfcapd.201107110845 contains the data from July 11th 2011 08:45 onward.
	12	nfcapd.201907110845 contains the data from July 11th 2019 08:45 onward.
	13	If the time interval is smaller then 60s, the naming extends to seconds
	14	e.g. nfcapd.20190711084510.
	15
13	16	.P
14	17	Netflow version v1, v5, v7 and v9 and IPFIX are transparently supported.
15	18	.P

240	243	.TP 3
241	244	.B -t \fIinterval
242	245	Specifies the time interval in seconds to rotate files. The default value
243		is 300s ( 5min ).
	246	is 300s ( 5min ). The smallest interval is 2s.
244	247	.TP 3
245	248	.B -w
246	249	Align file rotation with next n minute ( specified by \-t ) interval.

-4

man/nfdump.1 less more

239	239	.TP 3
240	240	.B -B
241	241	Like \-b but automagically swaps flows if src port is < dst port
242		as some exporters do not care sending the flows in proper order. It's
243		considered to be a convenient option. Please note - for some peer-to-peer flows
244		this my lead to errornous swapping.
	242	for TCP and UDP flows and src port < 1024 and dst port > 1024.
	243	as some exporters do not care sending the flows in proper order. Other
	244	flows are not affected. It's considered to be a convenient option.
245	245	.TP 3
246	246	.B -I
247	247	Print flow statistics from file specified by \-r, or timeslot specified by \-R/\-M.

740	740	\fB%evt\fR NSEL event
741	741	.br
742	742	\fB%xevt\fR NSEL extended event
	743	.br
	744	\fB%sgt\fR NSEL Source security group tag
743	745	.br
744	746	\fB%msec\fR NSEL event time in msec
745	747	.br

1149	1151	.I Type of Service (TOS)
1150	1152	\fI[SourceDestination]\fR \fBtos <num>\fR
1151	1153	.br
1152		With \fI<num>\fR 0..255. For compatibility with nfump 1.5.x:
	1154	With \fI<num>\fR 0..255. For compatibility with nfdump 1.5.x:
1153	1155	\fBtos <num>\fR is equivalent with \fBsrc tos <num>\fR
1154	1156	.TP 4
1155	1157	.I Packets per second: Calculated value.

-1

man/nfpcapd.1 less more

80	80	.TP 3
81	81	.B -t \fIinterval
82	82	Specifies the time interval in seconds to rotate files. The default value
83		is 300s ( 5min ). The intervalls are in sync with wall clock 5min intervals 5,10,15, ..
	83	is 300s ( 5min ). The smallest interval can be set to 2s. The intervalls are in sync
	84	with wall clock.
84	85	.TP 3
85	86	.B -P \fIpidfile
86	87	Specify name of pidfile. Default is no pidfile.

-4

man/sfcapd.1 less more

9	9	data from the network and stores it into nfcapd compatible files.
10	10	The output file is automatically rotated and renamed every n
11	11	minutes - typically 5 min - according the timestamp YYYYMMddhhmm
12		of the interval e.g. nfcapd.200407110845 contains the data from
13		July 11th 2004 08:45 onward. sfcapd supports sFlow version 4 and
14		5 datagrams.
	12	of the interval e.g. nfcapd.201907110845 contains the data from
	13	July 11th 2019 08:45 onward. sfcapd supports sFlow version 4 and
	14	5 datagrams. If the time interval is smaller then 60s, the naming
	15	extends to seconds e.g. nfcapd.20190711084510.
15	16	.P
16	17	Sflow is an industry standard developed by InMon Corporation.
17	18	For more information see http://sflow.org.

152	153	.TP 3
153	154	.B -t \fIinterval
154	155	Specifies the time interval in seconds to rotate files. The default value
155		is 300s ( 5min ).
	156	is 300s ( 5min ). The smallest interval can be set to 2s.
156	157	.TP 3
157	158	.B -w
158	159	Align file rotation with next n minute ( specified by -t ) interval.