Commit 28da9129908af1eaef195727461ee2541c514739 - nfdump

Imported Upstream version 1.5.2 Erik Wenzel 15 years ago

61 changed file(s) with 15078 addition(s) and 3711 deletion(s). Raw diff Collapse all Expand all

+99

-0

ChangeLog less more

	0	2006-05-17 v.1.5.1
	1	Add summary line at the end of output
	2	Improve configure process to automatically detect large file
	3	support as well as printf %z format
	4	make v9 module to accept any byte size (1-8) integer.
	5	2006-05-10 Fix a minor bug in v9 processing module.
	6	Fix bug in nfdump.c: Writing anonymized flows to file did not work corretly
	7	stdin input format now compatible with file format, therefore
	8	'nfdump < file' works again as it did in nfdump 1.4.
	9	Fix bug in nfcapd.c: Error handling not correct when receiving a non
	10	recognized netflow packet. Resulted in an endless loop
	11	2006-03-27 snapshot 1.5-20060327
	12	Make all element statistics -s transport layer protocol
	13	independant by default. Add :p to stat name ( e.g. srcip:p ) to
	14	enable transport layer dependant statistics on request.
	15	2006-03-20 snapshot 1.5-20060320
	16	Fix bug in filter engine: 'not flags xyz' produces wrong results
	17	when more than a single flag is specified.
	18	Minor man page fixes.
	19
	20	2006-03-06 v1.5
	21	Fix bug nfcapd. Laucher signaled too early. File not yet properly
	22	closed.
	23	2006-02-14 v1.5-beta-5
	24	Add srcas, dstas, input and output interfaces in aggregated
	25	output.
	26	Fix IPv6 bug in filter: accept 1234:: address.
	27	rename nfcapd.curent tmp file to nfcapd.curren.<pid>. Poorly
	28	configured nfcapd processes may mess up themselves otherwise.
	29	2006-02-02 v1.5-beta-4
	30	Fix netflow v5 dPkts <-> dOctets collector bug.
	31	Update pipe format to include more information
	32	Allow AS number 0 in filter syntax.
	33	Add some more boundary checking - netflow exporters aren't bug free either - sigh ..
	34	2006-01-11 v1.5-beta-3
	35	Fix isnumber incompatibility in grammer.y
	36	Add 'if' statistics
	37	2006-01-10 v1.5-beta-2
	38	nf_common.c Fix bug in format parser.
	39	Extended 'proto <protocol>' syntax to support all protocols
	40	Change time format in summary line to ISO format
	41	2005-12-20 v1.5-beta-1
	42	. A lot of internal changes, not mentioned here. :(
	43
	44	nfdump Add subnet aggregation for option -A
	45	A new syntax e.g. srcip4/24, dstip6/64 is supported for subnet wise aggregation.
	46	example: traffic of a whole subnet -A srcip4/24 -s srcip/bytes
	47
	48	nfdump Add more stat element option. -s <stat> now supports:
	49	srcip, dstip, ip, srcport, dstport, port, srcas, dstas, as, inif, outif, proto
	50
	51	nfdump Add -z. Suppress writing flows to data files. Only stat information is written.
	52	nfprofile Used only be nfsen for upcoming shadow profiles. If you don't understand this
	53	simply ignore it.
	54
	55	nfdump Add -q option to suppress header as well as stat information at the bottom
	56	nfprofile for easier post processing with external programms.
	57
	58	nf_common.c Output format processsing rewritting for more flexibility. Besides standard
	59	nfdump.c output formats line, long extended etc., user defined output formats are now
	60	possible and can even be compiled into nfdump for easy access. See -o fmt:<format>
	61	and nfdump.c around line 100.
	62
	63	. Integrate netflow v9 into nfdump. Only a subset of v9 is stored into
	64	the data files, basically everything needed for nfdump to work as it did before.
	65	This also includes IPv6 support for any nfdump options. CryptoPAN extended
	66	to work with IPv6. IPv6 condensed output format for better readability.
	67	Output formats available in long and condensed mode: e.g. line -> line6
	68	extended -> extended6
	69
	70	. Replace binary data file format. Old format not flexible enough for
	71	upcoming netflow v9/sflow data. *.stat files are gone. The same
	72	information is now available under nfdump -I
	73	New format about 5% larger in size, but faster for reading and writing.
	74	speed gain eaten up by more complex processing - sigh ..
	75	compat14 mode enables transparent reading of old style format.
	76	nffile.[ch] now handles all data file stuff.
	77
	78	nfreplay Multicast enabled:
	79	Add -j <join group>. Joins the specified multicast group ( v4 or v6 )
	80	sending flows to this group.
	81
	82	nfreplay IPv6 enabled:
	83	Add option -4 and -6 to force a specific protocol, otherwise
	84	protocol is automatically selected according the hostname to send flows to.
	85	Add -K key, to send data anonymized, using CryptoPAn
	86
	87	nfcapd Multicast enabled:
	88	Add -j <join group>. Joins the specified multicast group ( v4 or v6 )
	89	for listening.
	90
	91	nfcapd IPv6 enabled:
	92	Add option -4 and -6 for IPv4 and IPv6. By default, listen on IPv4.
	93	Option -b <host/IP> to bind for a specific host/IP address automatically
	94	selects appropriate protocol.
	95
	96	nfnet.c All functions to setup network sockets for listening/sending are
	97	put into this file.
	98
0	99	2005-08-22 v1.4
1	100	- nfreplay: Bug fix sending flows.
2	101	- nfdump: Add CryptoPAn code to anonymize IP addresses. New option -K

+33

-13

INSTALL less more

0	0	Installation:
1	1	-------------
2	2	The tools should work on any *nix flavor. They have been compiled
3		and tested on Linux, Solaris (32/64bit), OpenBSD (32/64bit) as well
4		as MacOSX.
	3	and tested on Linux(32/64bit) AMD64, Solaris (32/64bit),
	4	OpenBSD (32/64bit) as well as MacOSX.
5	5
6	6	./configure
7	7	make

10	10	should do the job.
11	11
12	12	configure options:
13		--enable-64
14		When using Snore CC on Solaris, you may compile 64 bit code.
15		default is NO
	13	--enable-compat14
	14	Build nfdump to read pre 1.5 data files. Use this option
	15	if you have data files nfdump <= 1.4.1
16	16
17	17	--enable-ftconv
18	18	Build ft2nfdump converter. The flow-tools sources are required

23	23	flow-tools sources in ../flow-tools-0.67. In case of another
24	24	path, the flow-tools directory is specified.
25	25
	26	--enable-64
	27	When using SUNWspro CC on Solaris, you may compile 64 bit code.
	28	default is NO
	29
	30	--enable-sflow
	31	Build sflow collector daemon.
	32
26	33	The parser code produced by yacc or bison can generate some
27	34	warnings, when being compiled. These can safely be ignored.
28	35
	36	Note on Solaris:
29	37	On any Solaris, make sure you have flex installed, as lex
30	38	on Solaris does not compile the scanner properly.
	39	When using SUNWspro set CC environment variable and run
	40	configure:
	41	setenv CC /opt/SUNWspro/bin/cc
	42	./configure ...
	43
	44	Notes on compatibility:
	45	As of nfdump 1.5, a new binary data format was introduced, to cope
	46	with any kind of input data such as netflow v9 and sflow data. This
	47	new format is not readable by nfdump <= 1.4.1. nfdump 1.5 can read
	48	old style data files when compat14 mode is enabled. ( see configure
	49	options ), but will always create new style data files. The new
	50	binary format also integrates old style .stat files, which no
	51	longer exist.
	52
	53	Although, nfdump 1.5 should smoothly replace nfdump <= 1.4.1
	54
	55	nfsen <= 1.2.1 does not work together with nfdump 1.5,
	56	use at least nfsen > v1.2.3 instead.
	57
31	58
32	59	Uninstalling:
33
	60	-------------
34	61	make uninstall
35	62
36		Notes on compatibility:
37		nfdump 1.4 introduces milisecond resolution. This resulted in a small
38		change of the binary format of nfdump files. However, this change is
39		fully transparent to the user, which means, nfdump 1.4 is able to read
40		and process older files, and nfdump < 1.4 is able to read and process
41		nfdump 1,4 files. But note: The milisecond resolution is only available
42		in data files captured with nfcapd v1.4 and processed with nfdump 1.4.

+35

-46

Makefile.in less more

31	31	#
32	32	# $Author: peter $
33	33	#
34		# $Id: Makefile.in 40 2005-08-24 08:07:54Z peter $
	34	# $Id: Makefile.in 70 2006-05-17 08:38:01Z peter $
35	35	#
36	36
37	37	srcdir = .

40	40	YACC = @YACC@
41	41	LEX = @LEX@
42	42	INSTALL = @INSTALL@
43		CFLAGS = @CFLAGS@
44		CPPFLAGS = -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64
	43	CFLAGS = @CFLAGS@ $(WARNINGS)
	44	CPPFLAGS =
45	45	YFLAGS = -d -v
46	46	LFLAGS = @LFLAGS@
47	47	LDFLAGS = @LIBS@
48	48	prefix = @prefix@
49	49	PREFIX = $(DESTDIR)/$(prefix)
50	50	MANDIR = $(DESTDIR)/@mandir@
51		CRYPTO_PAN_OBJ = panonymizer.o rijndael.o
52		FILTER_OBJ = grammar.o scanner.o nftree.o
53		NFDUMP_OBJ = nfdump.o $(FILTER_OBJ) nf_common.o nfstat.o nfprof.o util.o $(CRYPTO_PAN_OBJ)
54		NFPROF_OBJ = nfprofile.o profile.o $(FILTER_OBJ) nf_common.o util.o $(CRYPTO_PAN_OBJ)
55		NFRPLY_OBJ = nfreplay.o $(FILTER_OBJ) nf_common.o util.o $(CRYPTO_PAN_OBJ)
56		NFCAPD_OBJ = nfcapd.o nf_common.o launch.o $(CRYPTO_PAN_OBJ)
57		FT2NF_OBJ = ft2nfdump.o nf_common.o $(CRYPTO_PAN_OBJ)
58		OBJECTS = nfcapd nfdump nfprofile nfreplay nftree_check nfgen @opt_objects@
	51	COMMON_OBJ = nffile.o nf_common.o util.o panonymizer.o rijndael.o
	52	FILTER_OBJ = grammar.o scanner.o nftree.o ipconv.o
	53	NFDUMP_OBJ = nfdump.o $(COMMON_OBJ) $(FILTER_OBJ) nfstat.o nfprof.o
	54	NFRPLY_OBJ = nfreplay.o $(COMMON_OBJ) $(FILTER_OBJ) nfprof.o nfnet.o netflow_v9.o netflow_v5_v7.o
	55	NFPROF_OBJ = nfprofile.o profile.o $(COMMON_OBJ) $(FILTER_OBJ)
	56	NFCAPD_OBJ = nfcapd.o $(COMMON_OBJ) nfnet.o netflow_v9.o netflow_v5_v7.o launch.o
	57	SFCAPD_OBJ = sfcapd.o $(COMMON_OBJ) nfnet.o sflow.o launch.o
	58	FT2NF_OBJ = ft2nfdump.o $(COMMON_OBJ)
	59	OBJECTS = nfcapd nfdump nfprofile nfreplay nftest nfgen @opt_objects@
59	60	INCS = @FT_INCLUDES@
60	61	LIBS =
61	62	FT_LIBS = -lft -lz
62	63	FT_LDFLAGS = @FT_LDFLAGS@
	64	WARNINGS = -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wmissing-noreturn
63	65
64	66	.c.o:
65	67	$(CC) -c $(CPPFLAGS) $(CFLAGS) $(INCS) $<

69	71	nfcapd: $(NFCAPD_OBJ)
70	72	$(CC) $(CFLAGS) -o $@ $(NFCAPD_OBJ) $(LDFLAGS)
71	73
72		nftree_check: nftree_check.o $(FILTER_OBJ)
73		$(CC) $(CFLAGS) $(INCS) -o $@ nftree_check.o $(FILTER_OBJ) $(LDFLAGS)
	74	sfcapd: $(SFCAPD_OBJ)
	75	$(CC) $(CFLAGS) -o $@ $(SFCAPD_OBJ) $(LDFLAGS)
74	76
75		nfgen: nfgen.o
	77	nftest: nftest.o $(COMMON_OBJ) $(FILTER_OBJ)
	78	$(CC) $(CFLAGS) $(INCS) -o $@ nftest.o $(COMMON_OBJ) $(FILTER_OBJ) $(LDFLAGS)
	79
	80	nfgen: nfgen.o
76	81	$(CC) $(CFLAGS) $(INCS) -o $@ $? $(LDFLAGS)
77	82
78	83	nfdump: $(NFDUMP_OBJ)

92	97	test -d $(PREFIX)/bin \|\| install -d -o root -g bin -m 755 $(PREFIX)/bin
93	98	test -d $(MANDIR) \|\| install -d -o root -g bin -m 755 $(MANDIR)
94	99	test -d $(MANDIR)/man1 \|\| install -d -o root -g bin -m 755 $(MANDIR)/man1
95		$(INSTALL) -o root -g bin -m 755 nfcapd $(PREFIX)/bin
96		$(INSTALL) -o root -g bin -m 755 nfdump $(PREFIX)/bin
97		$(INSTALL) -o root -g bin -m 755 nfprofile $(PREFIX)/bin
98		$(INSTALL) -o root -g bin -m 755 nfreplay $(PREFIX)/bin
	100	for file in $(OBJECTS); do \
	101	if [ -f $$file.1 ]; then \
	102	$(INSTALL) -o root -g bin -m 755 $$file $(PREFIX)/bin ; \
	103	$(INSTALL) -o root -g bin -m 644 $$file.1 $(MANDIR)/man1 ; \
	104	fi \
	105	done;
99	106	test -f $(PREFIX)/bin/nfclean.pl \|\| $(INSTALL) -o root -g bin -m 755 nfclean.pl $(PREFIX)/bin
100	107
101		$(INSTALL) -o root -g bin -m 644 nfcapd.1 $(MANDIR)/man1
102		$(INSTALL) -o root -g bin -m 644 nfdump.1 $(MANDIR)/man1
103		$(INSTALL) -o root -g bin -m 644 nfprofile.1 $(MANDIR)/man1
104		$(INSTALL) -o root -g bin -m 644 nfreplay.1 $(MANDIR)/man1
105		if test -f ft2nfdump; then \
106		$(INSTALL) -o root -g bin -m 755 ft2nfdump $(PREFIX)/bin; \
107		$(INSTALL) -o root -g bin -m 644 ft2nfdump.1 $(MANDIR)/man1; \
108		fi;
109
110	108	uninstall:
111		/bin/rm -f $(PREFIX)/bin/nfcapd
112		/bin/rm -f $(PREFIX)/bin/nfdump
113		/bin/rm -f $(PREFIX)/bin/nfprofile
114		/bin/rm -f $(PREFIX)/bin/nfreplay
	109	for file in $(OBJECTS); do \
	110	if [ -f $$file.1 ]; then \
	111	/bin/rm -f $(PREFIX)/bin/$$file ; \
	112	/bin/rm -f $(MANDIR)/man1/$$file.1 ; \
	113	fi \
	114	done;
115	115	/bin/rm -f $(PREFIX)/bin/nfclean.pl
116	116
117		/bin/rm -f $(MANDIR)/man1/nfcapd.1
118		/bin/rm -f $(MANDIR)/man1/nfdump.1
119		/bin/rm -f $(MANDIR)/man1/nfprofile.1
120		/bin/rm -f $(MANDIR)/man1/nfreplay.1
121
122		test: nftree_check
123		./nftree_check
124		./nfgen \| ./nfdump -o extended \| head -17 > test1.out
125		diff test1.out nfdump.test.out
126		./nfgen > test.flows
127		./nfdump -r test.flows -o extended \| head -17 > test2.out
128		diff test2.out nfdump.test.out
129		rm -r test1.out test2.out
	117	test: nftest
	118	./test.sh
130	119
131	120	clean:
132	121	rm -f .o ~ y.* .tab. core $(OBJECTS) grammar.h grammar.c scanner.c

136	125	rm -f config.log config.h config.status Makefile
137	126
138	127	lint:
139		splint -warnposix $(FSRC)
	128	splint -warnposix *.c
140	129
141	130	grammar.h: grammar.c
142	131	grammar.c: grammar.y

151	140	mv lex.yy.c $@
152	141
153	142	depend: grammar.c scanner.c
154		makedepend $(CPPFLAGS) -- $(ASRC)
	143	makedepend $(CPPFLAGS) -- .c .h
155	144
156	145	# EOF
157	146	# DO NOT DELETE

+53

-9

README less more

4	4	They are part of the NFSEN project which is explained more detailed at
5	5	http://www.terena.nl/tech/task-forces/tf-csirt/meeting12/nfsen-Haag.pdf
6	6
7		The Web interface mentioned is not part of nfdump and will be released
8		separately.
	7	The Web interface mentioned is not part of nfdump and is available at
	8	http://nfsen.sourceforge.net
9	9
10	10	nfdump tools overview:
11	11	----------------------
12	12
13		nfcapd - netflow capture daemon.
	13	nfcapd - netflow collector daemon.
14	14	Reads the netflow data from the network and stores the data into files.
15	15	Automatically rotate files every n minutes. ( typically ever 5 min )
16		nfcapd reads netflow v5 and v7 flows transparently. v7 flows are
17		converted into v5 flows. You need one nfcapd process for each netflow
18		stream.
	16	nfcapd reads netflow v5, v7 and v9 flows transparently.
	17	You need one nfcapd process for each netflow stream.
19	18
20	19	nfdump - netflow dump.
21	20	Reads the netflow data from the files stored by nfcapd. It's syntax is

36	35	Sample script to cleanup old data. You may run this script every
37	36	hour or so.
38	37
	38	Optional binaries:
	39
39	40	ft2nfdump - read flow-tools format - Optional tool
40	41	ft2nfdump acts as a pipe converter for flow-tools data. It allows
41	42	to read any flow-tools data and process and save it in nfdump format.
	43
	44	sfcapd - sflow collector daemon
	45	scfapd collects sflow data and stores it into nfcapd comaptible files.
	46	"sfcapd includes sFlow(TM), freely available from http://www.inmon.com/".
	47
	48	Note for sflow users:
	49	sfcapd and nfcapd can be used concurrently to collect netflow and sflow
	50	data at the same time. Generic command line options apply to both
	51	collectors likewise. Due to lack of availability of sflow devices,
	52	I could not test the correct output of IPv6 records. Users are requested
	53	to send feedback to the list or directly to me. As of this first version,
	54	sfcapd supports the same fields as nfcapd does for netflow v9, which is a
	55	subset of all available sflow fields in an sflow record. More fields will
	56	be integrated in future versions of sfcapd.
42	57
43	58
44	59	Principle of Operation:

111	126	flow-cache timeout to lower values. All nfdump tools use 64 bit counters
112	127	internally, which means, all aggregated values are correctly reported.
113	128
114		The binary format of the data files is similar to the v5 netflow format
115		except the date fields, which are converted into UNIX time format. For
116		speed reasons the binary format is machine architecture dependent, and
	129	The binary format of the data files is netflow version independant.
	130	For speed reasons the binary format is machine architecture dependent, and
117	131	as such can not be exchanged between little and big endian systems.
	132	Internally nfdump does all processing IP protocol independant, which means
	133	everything works for IPv4 as well as IPv6 addresses.
	134	See the nfdump(1) man page for details.
	135
	136	netflow version 9:
	137	Even if netflow v9 is support, not all in netflow v9 defined elements
	138	are store in the data files. As of version 1.5 nfdump supports the fol-
	139	lowing fields:
	140	NF9_LAST_SWITCHED
	141	NF9_FIRST_SWITCHED
	142	NF9_IN_BYTES
	143	NF9_IN_PACKETS
	144	NF9_FLOWS
	145	NF9_IN_PROTOCOL
	146	NF9_SRC_TOS
	147	NF9_TCP_FLAGS
	148	NF9_IPV4_SRC_ADDR
	149	NF9_IPV6_SRC_ADDR
	150	NF9_IPV4_DST_ADDR
	151	NF9_IPV6_DST_ADDR
	152	NF9_L4_SRC_PORT
	153	NF9_L4_DST_PORT
	154	NF9_INPUT_SNMP
	155	NF9_OUTPUT_SNMP
	156	NF9_SRC_AS
	157	NF9_DST_AS
	158	32 and 64 bit counters are supported for Bytes and Packets. More
	159	fields may be supported in future.
	160
	161	nfcapd can listen on IPv6 or IPv4. Furthermore multicast is supported.
118	162
119	163	Flow-tools compatibility
120	164	------------------------

-0

ToDo less more

	0	- I'm urged to support sflow - so I put it on the todo list.
	1	- a better nfclean.pl script. It's already too old
	2	- Add table support in filter syntax to optimize expressions like
	3	.. or host .. or host .. or host ...
	4	- related filtering - a dream I have already or a long time .. sigh

+12

-0

config.h.in less more

18	18	/* Define to 1 if you have the `fork' function. */
19	19	#undef HAVE_FORK
20	20
	21	/* Define to 1 if you have the <fts.h> header file. */
	22	#undef HAVE_FTS_H
	23
21	24	/* Define to 1 if you have the `gethostbyname' function. */
22	25	#undef HAVE_GETHOSTBYNAME
23	26

62	65	/* Define to 1 if you have the `setsockopt' function. */
63	66	#undef HAVE_SETSOCKOPT
64	67
	68	/* Define to 1 if you have a printf() that supports the %z format string. */
	69	#undef HAVE_SIZE_T_Z_FORMAT
	70
65	71	/* Define to 1 if you have the `socket' function. */
66	72	#undef HAVE_SOCKET
67	73

181	187	/* Define to 1 if `lex' declares `yytext' as a `char *' by default, not a
182	188	`char[]'. */
183	189	#undef YYTEXT_POINTER
	190
	191	/* Number of bits in a file offset, on hosts where this is settable. */
	192	#undef _FILE_OFFSET_BITS
	193
	194	/* Define for large files, on AIX-style hosts. */
	195	#undef _LARGE_FILES
184	196
185	197	/* Define to empty if `const' does not conform to ANSI C. */
186	198	#undef const

+1188

-14

configure less more

0	0	#! /bin/sh
1		# From configure.in Revision: 40 .
	1	# From configure.in Revision: 55 .
2	2	# Guess values for system-dependent variables and create Makefiles.
3		# Generated by GNU Autoconf 2.59 for nfdump 1.4.
	3	# Generated by GNU Autoconf 2.59 for nfdump 1.5.
4	4	#
5	5	# Report bugs to <haag@switch.ch>.
6	6	#

269	269	# Identity of this package.
270	270	PACKAGE_NAME='nfdump'
271	271	PACKAGE_TARNAME='nfdump'
272		PACKAGE_VERSION='1.4'
273		PACKAGE_STRING='nfdump 1.4'
	272	PACKAGE_VERSION='1.5'
	273	PACKAGE_STRING='nfdump 1.5'
274	274	PACKAGE_BUGREPORT='haag@switch.ch'
275	275
276	276	ac_unique_file="grammar.y"

311	311	# include <unistd.h>
312	312	#endif"
313	313
314		ac_subst_vars='SHELL PATH_SEPARATOR PACKAGE_NAME PACKAGE_TARNAME PACKAGE_VERSION PACKAGE_STRING PACKAGE_BUGREPORT exec_prefix prefix program_transform_name bindir sbindir libexecdir datadir sysconfdir sharedstatedir localstatedir libdir includedir oldincludedir infodir mandir build_alias host_alias target_alias DEFS ECHO_C ECHO_N ECHO_T LIBS CC CFLAGS LDFLAGS CPPFLAGS ac_ct_CC EXEEXT OBJEXT YACC LEX LEXLIB LEX_OUTPUT_ROOT INSTALL_PROGRAM INSTALL_SCRIPT INSTALL_DATA opt_objects FT_INCLUDES FT_LDFLAGS LFLAGS CPP EGREP LIBOBJS HA_HAVE_SCANDIR LTLIBOBJS'
	314	ac_subst_vars='SHELL PATH_SEPARATOR PACKAGE_NAME PACKAGE_TARNAME PACKAGE_VERSION PACKAGE_STRING PACKAGE_BUGREPORT exec_prefix prefix program_transform_name bindir sbindir libexecdir datadir sysconfdir sharedstatedir localstatedir libdir includedir oldincludedir infodir mandir build_alias host_alias target_alias DEFS ECHO_C ECHO_N ECHO_T LIBS CC CFLAGS LDFLAGS CPPFLAGS ac_ct_CC EXEEXT OBJEXT YACC LEX LEXLIB LEX_OUTPUT_ROOT INSTALL_PROGRAM INSTALL_SCRIPT INSTALL_DATA opt_objects FT_INCLUDES FT_LDFLAGS LFLAGS CPP EGREP FTS_OBJ LIBOBJS HA_HAVE_SCANDIR LTLIBOBJS'
315	315	ac_subst_files=''
316	316
317	317	# Initialize some variables set by options.

780	780	# Omit some internal or obsolete options to make the list less imposing.
781	781	# This message is too long to be a string in the A/UX 3.1 sh.
782	782	cat <<_ACEOF
783		\`configure' configures nfdump 1.4 to adapt to many kinds of systems.
	783	\`configure' configures nfdump 1.5 to adapt to many kinds of systems.
784	784
785	785	Usage: $0 [OPTION]... [VAR=VALUE]...
786	786

837	837
838	838	if test -n "$ac_init_help"; then
839	839	case $ac_init_help in
840		short \| recursive ) echo "Configuration of nfdump 1.4:";;
	840	short \| recursive ) echo "Configuration of nfdump 1.5:";;
841	841	esac
842	842	cat <<\_ACEOF
843	843

845	845	--disable-FEATURE do not include FEATURE (same as --enable-FEATURE=no)
846	846	--enable-FEATURE[=ARG] include FEATURE [ARG=yes]
847	847	--enable-64 compile 64 bit on solaris with SUNPRO cc; default is NO
	848	--disable-largefile omit support for large files
	849	--enable-compat14 compile nfdump, to read nfdump data files <= v1.4; default is NO
848	850	--enable-ftconv Build the flow-tools to nfdump converter; default is NO
	851	--enable-sflow Build sflow collector sfcpad; default is NO
849	852
850	853	Optional Packages:
851	854	--with-PACKAGE[=ARG] use PACKAGE [ARG=yes]

960	963	test -n "$ac_init_help" && exit 0
961	964	if $ac_init_version; then
962	965	cat <<\_ACEOF
963		nfdump configure 1.4
	966	nfdump configure 1.5
964	967	generated by GNU Autoconf 2.59
965	968
966	969	Copyright (C) 2003 Free Software Foundation, Inc.

974	977	This file contains any messages produced by compilers while
975	978	running configure, to aid debugging if configure makes a mistake.
976	979
977		It was created by nfdump $as_me 1.4, which was
	980	It was created by nfdump $as_me 1.5, which was
978	981	generated by GNU Autoconf 2.59. Invocation command line was
979	982
980	983	$ $0 $@

1320	1323
1321	1324	fi;
1322	1325
1323		# Checks for programs.
1324	1326	ac_ext=c
1325	1327	ac_cpp='$CPP $CPPFLAGS'
1326	1328	ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5'

2246	2248	ac_compiler_gnu=$ac_cv_c_compiler_gnu
2247	2249
2248	2250
	2251	# Check whether --enable-largefile or --disable-largefile was given.
	2252	if test "${enable_largefile+set}" = set; then
	2253	enableval="$enable_largefile"
	2254
	2255	fi;
	2256	if test "$enable_largefile" != no; then
	2257
	2258	echo "$as_me:$LINENO: checking for special C compiler options needed for large files" >&5
	2259	echo $ECHO_N "checking for special C compiler options needed for large files... $ECHO_C" >&6
	2260	if test "${ac_cv_sys_largefile_CC+set}" = set; then
	2261	echo $ECHO_N "(cached) $ECHO_C" >&6
	2262	else
	2263	ac_cv_sys_largefile_CC=no
	2264	if test "$GCC" != yes; then
	2265	ac_save_CC=$CC
	2266	while :; do
	2267	# IRIX 6.2 and later do not support large files by default,
	2268	# so use the C compiler's -n32 option if that helps.
	2269	cat >conftest.$ac_ext <<_ACEOF
	2270	/* confdefs.h. */
	2271	_ACEOF
	2272	cat confdefs.h >>conftest.$ac_ext
	2273	cat >>conftest.$ac_ext <<_ACEOF
	2274	/* end confdefs.h. */
	2275	#include <sys/types.h>
	2276	/* Check that off_t can represent 2**63 - 1 correctly.
	2277	We can't simply define LARGE_OFF_T to be 9223372036854775807,
	2278	since some C++ compilers masquerading as C compilers
	2279	incorrectly reject 9223372036854775807. */
	2280	#define LARGE_OFF_T (((off_t) 1 << 62) - 1 + ((off_t) 1 << 62))
	2281	int off_t_is_large[(LARGE_OFF_T % 2147483629 == 721
	2282	&& LARGE_OFF_T % 2147483647 == 1)
	2283	? 1 : -1];
	2284	int
	2285	main ()
	2286	{
	2287
	2288	;
	2289	return 0;
	2290	}
	2291	_ACEOF
	2292	rm -f conftest.$ac_objext
	2293	if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5
	2294	(eval $ac_compile) 2>conftest.er1
	2295	ac_status=$?
	2296	grep -v '^ *+' conftest.er1 >conftest.err
	2297	rm -f conftest.er1
	2298	cat conftest.err >&5
	2299	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	2300	(exit $ac_status); } &&
	2301	{ ac_try='test -z "$ac_c_werror_flag" \|\| test ! -s conftest.err'
	2302	{ (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5
	2303	(eval $ac_try) 2>&5
	2304	ac_status=$?
	2305	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	2306	(exit $ac_status); }; } &&
	2307	{ ac_try='test -s conftest.$ac_objext'
	2308	{ (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5
	2309	(eval $ac_try) 2>&5
	2310	ac_status=$?
	2311	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	2312	(exit $ac_status); }; }; then
	2313	break
	2314	else
	2315	echo "$as_me: failed program was:" >&5
	2316	sed 's/^/\| /' conftest.$ac_ext >&5
	2317
	2318	fi
	2319	rm -f conftest.err conftest.$ac_objext
	2320	CC="$CC -n32"
	2321	rm -f conftest.$ac_objext
	2322	if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5
	2323	(eval $ac_compile) 2>conftest.er1
	2324	ac_status=$?
	2325	grep -v '^ *+' conftest.er1 >conftest.err
	2326	rm -f conftest.er1
	2327	cat conftest.err >&5
	2328	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	2329	(exit $ac_status); } &&
	2330	{ ac_try='test -z "$ac_c_werror_flag" \|\| test ! -s conftest.err'
	2331	{ (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5
	2332	(eval $ac_try) 2>&5
	2333	ac_status=$?
	2334	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	2335	(exit $ac_status); }; } &&
	2336	{ ac_try='test -s conftest.$ac_objext'
	2337	{ (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5
	2338	(eval $ac_try) 2>&5
	2339	ac_status=$?
	2340	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	2341	(exit $ac_status); }; }; then
	2342	ac_cv_sys_largefile_CC=' -n32'; break
	2343	else
	2344	echo "$as_me: failed program was:" >&5
	2345	sed 's/^/\| /' conftest.$ac_ext >&5
	2346
	2347	fi
	2348	rm -f conftest.err conftest.$ac_objext
	2349	break
	2350	done
	2351	CC=$ac_save_CC
	2352	rm -f conftest.$ac_ext
	2353	fi
	2354	fi
	2355	echo "$as_me:$LINENO: result: $ac_cv_sys_largefile_CC" >&5
	2356	echo "${ECHO_T}$ac_cv_sys_largefile_CC" >&6
	2357	if test "$ac_cv_sys_largefile_CC" != no; then
	2358	CC=$CC$ac_cv_sys_largefile_CC
	2359	fi
	2360
	2361	echo "$as_me:$LINENO: checking for _FILE_OFFSET_BITS value needed for large files" >&5
	2362	echo $ECHO_N "checking for _FILE_OFFSET_BITS value needed for large files... $ECHO_C" >&6
	2363	if test "${ac_cv_sys_file_offset_bits+set}" = set; then
	2364	echo $ECHO_N "(cached) $ECHO_C" >&6
	2365	else
	2366	while :; do
	2367	ac_cv_sys_file_offset_bits=no
	2368	cat >conftest.$ac_ext <<_ACEOF
	2369	/* confdefs.h. */
	2370	_ACEOF
	2371	cat confdefs.h >>conftest.$ac_ext
	2372	cat >>conftest.$ac_ext <<_ACEOF
	2373	/* end confdefs.h. */
	2374	#include <sys/types.h>
	2375	/* Check that off_t can represent 2**63 - 1 correctly.
	2376	We can't simply define LARGE_OFF_T to be 9223372036854775807,
	2377	since some C++ compilers masquerading as C compilers
	2378	incorrectly reject 9223372036854775807. */
	2379	#define LARGE_OFF_T (((off_t) 1 << 62) - 1 + ((off_t) 1 << 62))
	2380	int off_t_is_large[(LARGE_OFF_T % 2147483629 == 721
	2381	&& LARGE_OFF_T % 2147483647 == 1)
	2382	? 1 : -1];
	2383	int
	2384	main ()
	2385	{
	2386
	2387	;
	2388	return 0;
	2389	}
	2390	_ACEOF
	2391	rm -f conftest.$ac_objext
	2392	if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5
	2393	(eval $ac_compile) 2>conftest.er1
	2394	ac_status=$?
	2395	grep -v '^ *+' conftest.er1 >conftest.err
	2396	rm -f conftest.er1
	2397	cat conftest.err >&5
	2398	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	2399	(exit $ac_status); } &&
	2400	{ ac_try='test -z "$ac_c_werror_flag" \|\| test ! -s conftest.err'
	2401	{ (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5
	2402	(eval $ac_try) 2>&5
	2403	ac_status=$?
	2404	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	2405	(exit $ac_status); }; } &&
	2406	{ ac_try='test -s conftest.$ac_objext'
	2407	{ (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5
	2408	(eval $ac_try) 2>&5
	2409	ac_status=$?
	2410	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	2411	(exit $ac_status); }; }; then
	2412	break
	2413	else
	2414	echo "$as_me: failed program was:" >&5
	2415	sed 's/^/\| /' conftest.$ac_ext >&5
	2416
	2417	fi
	2418	rm -f conftest.err conftest.$ac_objext conftest.$ac_ext
	2419	cat >conftest.$ac_ext <<_ACEOF
	2420	/* confdefs.h. */
	2421	_ACEOF
	2422	cat confdefs.h >>conftest.$ac_ext
	2423	cat >>conftest.$ac_ext <<_ACEOF
	2424	/* end confdefs.h. */
	2425	#define _FILE_OFFSET_BITS 64
	2426	#include <sys/types.h>
	2427	/* Check that off_t can represent 2**63 - 1 correctly.
	2428	We can't simply define LARGE_OFF_T to be 9223372036854775807,
	2429	since some C++ compilers masquerading as C compilers
	2430	incorrectly reject 9223372036854775807. */
	2431	#define LARGE_OFF_T (((off_t) 1 << 62) - 1 + ((off_t) 1 << 62))
	2432	int off_t_is_large[(LARGE_OFF_T % 2147483629 == 721
	2433	&& LARGE_OFF_T % 2147483647 == 1)
	2434	? 1 : -1];
	2435	int
	2436	main ()
	2437	{
	2438
	2439	;
	2440	return 0;
	2441	}
	2442	_ACEOF
	2443	rm -f conftest.$ac_objext
	2444	if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5
	2445	(eval $ac_compile) 2>conftest.er1
	2446	ac_status=$?
	2447	grep -v '^ *+' conftest.er1 >conftest.err
	2448	rm -f conftest.er1
	2449	cat conftest.err >&5
	2450	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	2451	(exit $ac_status); } &&
	2452	{ ac_try='test -z "$ac_c_werror_flag" \|\| test ! -s conftest.err'
	2453	{ (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5
	2454	(eval $ac_try) 2>&5
	2455	ac_status=$?
	2456	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	2457	(exit $ac_status); }; } &&
	2458	{ ac_try='test -s conftest.$ac_objext'
	2459	{ (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5
	2460	(eval $ac_try) 2>&5
	2461	ac_status=$?
	2462	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	2463	(exit $ac_status); }; }; then
	2464	ac_cv_sys_file_offset_bits=64; break
	2465	else
	2466	echo "$as_me: failed program was:" >&5
	2467	sed 's/^/\| /' conftest.$ac_ext >&5
	2468
	2469	fi
	2470	rm -f conftest.err conftest.$ac_objext conftest.$ac_ext
	2471	break
	2472	done
	2473	fi
	2474	echo "$as_me:$LINENO: result: $ac_cv_sys_file_offset_bits" >&5
	2475	echo "${ECHO_T}$ac_cv_sys_file_offset_bits" >&6
	2476	if test "$ac_cv_sys_file_offset_bits" != no; then
	2477
	2478	cat >>confdefs.h <<_ACEOF
	2479	#define _FILE_OFFSET_BITS $ac_cv_sys_file_offset_bits
	2480	_ACEOF
	2481
	2482	fi
	2483	rm -f conftest*
	2484	echo "$as_me:$LINENO: checking for _LARGE_FILES value needed for large files" >&5
	2485	echo $ECHO_N "checking for _LARGE_FILES value needed for large files... $ECHO_C" >&6
	2486	if test "${ac_cv_sys_large_files+set}" = set; then
	2487	echo $ECHO_N "(cached) $ECHO_C" >&6
	2488	else
	2489	while :; do
	2490	ac_cv_sys_large_files=no
	2491	cat >conftest.$ac_ext <<_ACEOF
	2492	/* confdefs.h. */
	2493	_ACEOF
	2494	cat confdefs.h >>conftest.$ac_ext
	2495	cat >>conftest.$ac_ext <<_ACEOF
	2496	/* end confdefs.h. */
	2497	#include <sys/types.h>
	2498	/* Check that off_t can represent 2**63 - 1 correctly.
	2499	We can't simply define LARGE_OFF_T to be 9223372036854775807,
	2500	since some C++ compilers masquerading as C compilers
	2501	incorrectly reject 9223372036854775807. */
	2502	#define LARGE_OFF_T (((off_t) 1 << 62) - 1 + ((off_t) 1 << 62))
	2503	int off_t_is_large[(LARGE_OFF_T % 2147483629 == 721
	2504	&& LARGE_OFF_T % 2147483647 == 1)
	2505	? 1 : -1];
	2506	int
	2507	main ()
	2508	{
	2509
	2510	;
	2511	return 0;
	2512	}
	2513	_ACEOF
	2514	rm -f conftest.$ac_objext
	2515	if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5
	2516	(eval $ac_compile) 2>conftest.er1
	2517	ac_status=$?
	2518	grep -v '^ *+' conftest.er1 >conftest.err
	2519	rm -f conftest.er1
	2520	cat conftest.err >&5
	2521	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	2522	(exit $ac_status); } &&
	2523	{ ac_try='test -z "$ac_c_werror_flag" \|\| test ! -s conftest.err'
	2524	{ (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5
	2525	(eval $ac_try) 2>&5
	2526	ac_status=$?
	2527	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	2528	(exit $ac_status); }; } &&
	2529	{ ac_try='test -s conftest.$ac_objext'
	2530	{ (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5
	2531	(eval $ac_try) 2>&5
	2532	ac_status=$?
	2533	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	2534	(exit $ac_status); }; }; then
	2535	break
	2536	else
	2537	echo "$as_me: failed program was:" >&5
	2538	sed 's/^/\| /' conftest.$ac_ext >&5
	2539
	2540	fi
	2541	rm -f conftest.err conftest.$ac_objext conftest.$ac_ext
	2542	cat >conftest.$ac_ext <<_ACEOF
	2543	/* confdefs.h. */
	2544	_ACEOF
	2545	cat confdefs.h >>conftest.$ac_ext
	2546	cat >>conftest.$ac_ext <<_ACEOF
	2547	/* end confdefs.h. */
	2548	#define _LARGE_FILES 1
	2549	#include <sys/types.h>
	2550	/* Check that off_t can represent 2**63 - 1 correctly.
	2551	We can't simply define LARGE_OFF_T to be 9223372036854775807,
	2552	since some C++ compilers masquerading as C compilers
	2553	incorrectly reject 9223372036854775807. */
	2554	#define LARGE_OFF_T (((off_t) 1 << 62) - 1 + ((off_t) 1 << 62))
	2555	int off_t_is_large[(LARGE_OFF_T % 2147483629 == 721
	2556	&& LARGE_OFF_T % 2147483647 == 1)
	2557	? 1 : -1];
	2558	int
	2559	main ()
	2560	{
	2561
	2562	;
	2563	return 0;
	2564	}
	2565	_ACEOF
	2566	rm -f conftest.$ac_objext
	2567	if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5
	2568	(eval $ac_compile) 2>conftest.er1
	2569	ac_status=$?
	2570	grep -v '^ *+' conftest.er1 >conftest.err
	2571	rm -f conftest.er1
	2572	cat conftest.err >&5
	2573	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	2574	(exit $ac_status); } &&
	2575	{ ac_try='test -z "$ac_c_werror_flag" \|\| test ! -s conftest.err'
	2576	{ (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5
	2577	(eval $ac_try) 2>&5
	2578	ac_status=$?
	2579	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	2580	(exit $ac_status); }; } &&
	2581	{ ac_try='test -s conftest.$ac_objext'
	2582	{ (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5
	2583	(eval $ac_try) 2>&5
	2584	ac_status=$?
	2585	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	2586	(exit $ac_status); }; }; then
	2587	ac_cv_sys_large_files=1; break
	2588	else
	2589	echo "$as_me: failed program was:" >&5
	2590	sed 's/^/\| /' conftest.$ac_ext >&5
	2591
	2592	fi
	2593	rm -f conftest.err conftest.$ac_objext conftest.$ac_ext
	2594	break
	2595	done
	2596	fi
	2597	echo "$as_me:$LINENO: result: $ac_cv_sys_large_files" >&5
	2598	echo "${ECHO_T}$ac_cv_sys_large_files" >&6
	2599	if test "$ac_cv_sys_large_files" != no; then
	2600
	2601	cat >>confdefs.h <<_ACEOF
	2602	#define _LARGE_FILES $ac_cv_sys_large_files
	2603	_ACEOF
	2604
	2605	fi
	2606	rm -f conftest*
	2607	fi
	2608
	2609
	2610	# Checks for programs.
	2611	ac_ext=c
	2612	ac_cpp='$CPP $CPPFLAGS'
	2613	ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5'
	2614	ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'
	2615	ac_compiler_gnu=$ac_cv_c_compiler_gnu
	2616	if test -n "$ac_tool_prefix"; then
	2617	# Extract the first word of "${ac_tool_prefix}gcc", so it can be a program name with args.
	2618	set dummy ${ac_tool_prefix}gcc; ac_word=$2
	2619	echo "$as_me:$LINENO: checking for $ac_word" >&5
	2620	echo $ECHO_N "checking for $ac_word... $ECHO_C" >&6
	2621	if test "${ac_cv_prog_CC+set}" = set; then
	2622	echo $ECHO_N "(cached) $ECHO_C" >&6
	2623	else
	2624	if test -n "$CC"; then
	2625	ac_cv_prog_CC="$CC" # Let the user override the test.
	2626	else
	2627	as_save_IFS=$IFS; IFS=$PATH_SEPARATOR
	2628	for as_dir in $PATH
	2629	do
	2630	IFS=$as_save_IFS
	2631	test -z "$as_dir" && as_dir=.
	2632	for ac_exec_ext in '' $ac_executable_extensions; do
	2633	if $as_executable_p "$as_dir/$ac_word$ac_exec_ext"; then
	2634	ac_cv_prog_CC="${ac_tool_prefix}gcc"
	2635	echo "$as_me:$LINENO: found $as_dir/$ac_word$ac_exec_ext" >&5
	2636	break 2
	2637	fi
	2638	done
	2639	done
	2640
	2641	fi
	2642	fi
	2643	CC=$ac_cv_prog_CC
	2644	if test -n "$CC"; then
	2645	echo "$as_me:$LINENO: result: $CC" >&5
	2646	echo "${ECHO_T}$CC" >&6
	2647	else
	2648	echo "$as_me:$LINENO: result: no" >&5
	2649	echo "${ECHO_T}no" >&6
	2650	fi
	2651
	2652	fi
	2653	if test -z "$ac_cv_prog_CC"; then
	2654	ac_ct_CC=$CC
	2655	# Extract the first word of "gcc", so it can be a program name with args.
	2656	set dummy gcc; ac_word=$2
	2657	echo "$as_me:$LINENO: checking for $ac_word" >&5
	2658	echo $ECHO_N "checking for $ac_word... $ECHO_C" >&6
	2659	if test "${ac_cv_prog_ac_ct_CC+set}" = set; then
	2660	echo $ECHO_N "(cached) $ECHO_C" >&6
	2661	else
	2662	if test -n "$ac_ct_CC"; then
	2663	ac_cv_prog_ac_ct_CC="$ac_ct_CC" # Let the user override the test.
	2664	else
	2665	as_save_IFS=$IFS; IFS=$PATH_SEPARATOR
	2666	for as_dir in $PATH
	2667	do
	2668	IFS=$as_save_IFS
	2669	test -z "$as_dir" && as_dir=.
	2670	for ac_exec_ext in '' $ac_executable_extensions; do
	2671	if $as_executable_p "$as_dir/$ac_word$ac_exec_ext"; then
	2672	ac_cv_prog_ac_ct_CC="gcc"
	2673	echo "$as_me:$LINENO: found $as_dir/$ac_word$ac_exec_ext" >&5
	2674	break 2
	2675	fi
	2676	done
	2677	done
	2678
	2679	fi
	2680	fi
	2681	ac_ct_CC=$ac_cv_prog_ac_ct_CC
	2682	if test -n "$ac_ct_CC"; then
	2683	echo "$as_me:$LINENO: result: $ac_ct_CC" >&5
	2684	echo "${ECHO_T}$ac_ct_CC" >&6
	2685	else
	2686	echo "$as_me:$LINENO: result: no" >&5
	2687	echo "${ECHO_T}no" >&6
	2688	fi
	2689
	2690	CC=$ac_ct_CC
	2691	else
	2692	CC="$ac_cv_prog_CC"
	2693	fi
	2694
	2695	if test -z "$CC"; then
	2696	if test -n "$ac_tool_prefix"; then
	2697	# Extract the first word of "${ac_tool_prefix}cc", so it can be a program name with args.
	2698	set dummy ${ac_tool_prefix}cc; ac_word=$2
	2699	echo "$as_me:$LINENO: checking for $ac_word" >&5
	2700	echo $ECHO_N "checking for $ac_word... $ECHO_C" >&6
	2701	if test "${ac_cv_prog_CC+set}" = set; then
	2702	echo $ECHO_N "(cached) $ECHO_C" >&6
	2703	else
	2704	if test -n "$CC"; then
	2705	ac_cv_prog_CC="$CC" # Let the user override the test.
	2706	else
	2707	as_save_IFS=$IFS; IFS=$PATH_SEPARATOR
	2708	for as_dir in $PATH
	2709	do
	2710	IFS=$as_save_IFS
	2711	test -z "$as_dir" && as_dir=.
	2712	for ac_exec_ext in '' $ac_executable_extensions; do
	2713	if $as_executable_p "$as_dir/$ac_word$ac_exec_ext"; then
	2714	ac_cv_prog_CC="${ac_tool_prefix}cc"
	2715	echo "$as_me:$LINENO: found $as_dir/$ac_word$ac_exec_ext" >&5
	2716	break 2
	2717	fi
	2718	done
	2719	done
	2720
	2721	fi
	2722	fi
	2723	CC=$ac_cv_prog_CC
	2724	if test -n "$CC"; then
	2725	echo "$as_me:$LINENO: result: $CC" >&5
	2726	echo "${ECHO_T}$CC" >&6
	2727	else
	2728	echo "$as_me:$LINENO: result: no" >&5
	2729	echo "${ECHO_T}no" >&6
	2730	fi
	2731
	2732	fi
	2733	if test -z "$ac_cv_prog_CC"; then
	2734	ac_ct_CC=$CC
	2735	# Extract the first word of "cc", so it can be a program name with args.
	2736	set dummy cc; ac_word=$2
	2737	echo "$as_me:$LINENO: checking for $ac_word" >&5
	2738	echo $ECHO_N "checking for $ac_word... $ECHO_C" >&6
	2739	if test "${ac_cv_prog_ac_ct_CC+set}" = set; then
	2740	echo $ECHO_N "(cached) $ECHO_C" >&6
	2741	else
	2742	if test -n "$ac_ct_CC"; then
	2743	ac_cv_prog_ac_ct_CC="$ac_ct_CC" # Let the user override the test.
	2744	else
	2745	as_save_IFS=$IFS; IFS=$PATH_SEPARATOR
	2746	for as_dir in $PATH
	2747	do
	2748	IFS=$as_save_IFS
	2749	test -z "$as_dir" && as_dir=.
	2750	for ac_exec_ext in '' $ac_executable_extensions; do
	2751	if $as_executable_p "$as_dir/$ac_word$ac_exec_ext"; then
	2752	ac_cv_prog_ac_ct_CC="cc"
	2753	echo "$as_me:$LINENO: found $as_dir/$ac_word$ac_exec_ext" >&5
	2754	break 2
	2755	fi
	2756	done
	2757	done
	2758
	2759	fi
	2760	fi
	2761	ac_ct_CC=$ac_cv_prog_ac_ct_CC
	2762	if test -n "$ac_ct_CC"; then
	2763	echo "$as_me:$LINENO: result: $ac_ct_CC" >&5
	2764	echo "${ECHO_T}$ac_ct_CC" >&6
	2765	else
	2766	echo "$as_me:$LINENO: result: no" >&5
	2767	echo "${ECHO_T}no" >&6
	2768	fi
	2769
	2770	CC=$ac_ct_CC
	2771	else
	2772	CC="$ac_cv_prog_CC"
	2773	fi
	2774
	2775	fi
	2776	if test -z "$CC"; then
	2777	# Extract the first word of "cc", so it can be a program name with args.
	2778	set dummy cc; ac_word=$2
	2779	echo "$as_me:$LINENO: checking for $ac_word" >&5
	2780	echo $ECHO_N "checking for $ac_word... $ECHO_C" >&6
	2781	if test "${ac_cv_prog_CC+set}" = set; then
	2782	echo $ECHO_N "(cached) $ECHO_C" >&6
	2783	else
	2784	if test -n "$CC"; then
	2785	ac_cv_prog_CC="$CC" # Let the user override the test.
	2786	else
	2787	ac_prog_rejected=no
	2788	as_save_IFS=$IFS; IFS=$PATH_SEPARATOR
	2789	for as_dir in $PATH
	2790	do
	2791	IFS=$as_save_IFS
	2792	test -z "$as_dir" && as_dir=.
	2793	for ac_exec_ext in '' $ac_executable_extensions; do
	2794	if $as_executable_p "$as_dir/$ac_word$ac_exec_ext"; then
	2795	if test "$as_dir/$ac_word$ac_exec_ext" = "/usr/ucb/cc"; then
	2796	ac_prog_rejected=yes
	2797	continue
	2798	fi
	2799	ac_cv_prog_CC="cc"
	2800	echo "$as_me:$LINENO: found $as_dir/$ac_word$ac_exec_ext" >&5
	2801	break 2
	2802	fi
	2803	done
	2804	done
	2805
	2806	if test $ac_prog_rejected = yes; then
	2807	# We found a bogon in the path, so make sure we never use it.
	2808	set dummy $ac_cv_prog_CC
	2809	shift
	2810	if test $# != 0; then
	2811	# We chose a different compiler from the bogus one.
	2812	# However, it has the same basename, so the bogon will be chosen
	2813	# first if we set CC to just the basename; use the full file name.
	2814	shift
	2815	ac_cv_prog_CC="$as_dir/$ac_word${1+' '}$@"
	2816	fi
	2817	fi
	2818	fi
	2819	fi
	2820	CC=$ac_cv_prog_CC
	2821	if test -n "$CC"; then
	2822	echo "$as_me:$LINENO: result: $CC" >&5
	2823	echo "${ECHO_T}$CC" >&6
	2824	else
	2825	echo "$as_me:$LINENO: result: no" >&5
	2826	echo "${ECHO_T}no" >&6
	2827	fi
	2828
	2829	fi
	2830	if test -z "$CC"; then
	2831	if test -n "$ac_tool_prefix"; then
	2832	for ac_prog in cl
	2833	do
	2834	# Extract the first word of "$ac_tool_prefix$ac_prog", so it can be a program name with args.
	2835	set dummy $ac_tool_prefix$ac_prog; ac_word=$2
	2836	echo "$as_me:$LINENO: checking for $ac_word" >&5
	2837	echo $ECHO_N "checking for $ac_word... $ECHO_C" >&6
	2838	if test "${ac_cv_prog_CC+set}" = set; then
	2839	echo $ECHO_N "(cached) $ECHO_C" >&6
	2840	else
	2841	if test -n "$CC"; then
	2842	ac_cv_prog_CC="$CC" # Let the user override the test.
	2843	else
	2844	as_save_IFS=$IFS; IFS=$PATH_SEPARATOR
	2845	for as_dir in $PATH
	2846	do
	2847	IFS=$as_save_IFS
	2848	test -z "$as_dir" && as_dir=.
	2849	for ac_exec_ext in '' $ac_executable_extensions; do
	2850	if $as_executable_p "$as_dir/$ac_word$ac_exec_ext"; then
	2851	ac_cv_prog_CC="$ac_tool_prefix$ac_prog"
	2852	echo "$as_me:$LINENO: found $as_dir/$ac_word$ac_exec_ext" >&5
	2853	break 2
	2854	fi
	2855	done
	2856	done
	2857
	2858	fi
	2859	fi
	2860	CC=$ac_cv_prog_CC
	2861	if test -n "$CC"; then
	2862	echo "$as_me:$LINENO: result: $CC" >&5
	2863	echo "${ECHO_T}$CC" >&6
	2864	else
	2865	echo "$as_me:$LINENO: result: no" >&5
	2866	echo "${ECHO_T}no" >&6
	2867	fi
	2868
	2869	test -n "$CC" && break
	2870	done
	2871	fi
	2872	if test -z "$CC"; then
	2873	ac_ct_CC=$CC
	2874	for ac_prog in cl
	2875	do
	2876	# Extract the first word of "$ac_prog", so it can be a program name with args.
	2877	set dummy $ac_prog; ac_word=$2
	2878	echo "$as_me:$LINENO: checking for $ac_word" >&5
	2879	echo $ECHO_N "checking for $ac_word... $ECHO_C" >&6
	2880	if test "${ac_cv_prog_ac_ct_CC+set}" = set; then
	2881	echo $ECHO_N "(cached) $ECHO_C" >&6
	2882	else
	2883	if test -n "$ac_ct_CC"; then
	2884	ac_cv_prog_ac_ct_CC="$ac_ct_CC" # Let the user override the test.
	2885	else
	2886	as_save_IFS=$IFS; IFS=$PATH_SEPARATOR
	2887	for as_dir in $PATH
	2888	do
	2889	IFS=$as_save_IFS
	2890	test -z "$as_dir" && as_dir=.
	2891	for ac_exec_ext in '' $ac_executable_extensions; do
	2892	if $as_executable_p "$as_dir/$ac_word$ac_exec_ext"; then
	2893	ac_cv_prog_ac_ct_CC="$ac_prog"
	2894	echo "$as_me:$LINENO: found $as_dir/$ac_word$ac_exec_ext" >&5
	2895	break 2
	2896	fi
	2897	done
	2898	done
	2899
	2900	fi
	2901	fi
	2902	ac_ct_CC=$ac_cv_prog_ac_ct_CC
	2903	if test -n "$ac_ct_CC"; then
	2904	echo "$as_me:$LINENO: result: $ac_ct_CC" >&5
	2905	echo "${ECHO_T}$ac_ct_CC" >&6
	2906	else
	2907	echo "$as_me:$LINENO: result: no" >&5
	2908	echo "${ECHO_T}no" >&6
	2909	fi
	2910
	2911	test -n "$ac_ct_CC" && break
	2912	done
	2913
	2914	CC=$ac_ct_CC
	2915	fi
	2916
	2917	fi
	2918
	2919
	2920	test -z "$CC" && { { echo "$as_me:$LINENO: error: no acceptable C compiler found in \$PATH
	2921	See \`config.log' for more details." >&5
	2922	echo "$as_me: error: no acceptable C compiler found in \$PATH
	2923	See \`config.log' for more details." >&2;}
	2924	{ (exit 1); exit 1; }; }
	2925
	2926	# Provide some information about the compiler.
	2927	echo "$as_me:$LINENO:" \
	2928	"checking for C compiler version" >&5
	2929	ac_compiler=`set X $ac_compile; echo $2`
	2930	{ (eval echo "$as_me:$LINENO: \"$ac_compiler --version </dev/null >&5\"") >&5
	2931	(eval $ac_compiler --version </dev/null >&5) 2>&5
	2932	ac_status=$?
	2933	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	2934	(exit $ac_status); }
	2935	{ (eval echo "$as_me:$LINENO: \"$ac_compiler -v </dev/null >&5\"") >&5
	2936	(eval $ac_compiler -v </dev/null >&5) 2>&5
	2937	ac_status=$?
	2938	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	2939	(exit $ac_status); }
	2940	{ (eval echo "$as_me:$LINENO: \"$ac_compiler -V </dev/null >&5\"") >&5
	2941	(eval $ac_compiler -V </dev/null >&5) 2>&5
	2942	ac_status=$?
	2943	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	2944	(exit $ac_status); }
	2945
	2946	echo "$as_me:$LINENO: checking whether we are using the GNU C compiler" >&5
	2947	echo $ECHO_N "checking whether we are using the GNU C compiler... $ECHO_C" >&6
	2948	if test "${ac_cv_c_compiler_gnu+set}" = set; then
	2949	echo $ECHO_N "(cached) $ECHO_C" >&6
	2950	else
	2951	cat >conftest.$ac_ext <<_ACEOF
	2952	/* confdefs.h. */
	2953	_ACEOF
	2954	cat confdefs.h >>conftest.$ac_ext
	2955	cat >>conftest.$ac_ext <<_ACEOF
	2956	/* end confdefs.h. */
	2957
	2958	int
	2959	main ()
	2960	{
	2961	#ifndef __GNUC__
	2962	choke me
	2963	#endif
	2964
	2965	;
	2966	return 0;
	2967	}
	2968	_ACEOF
	2969	rm -f conftest.$ac_objext
	2970	if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5
	2971	(eval $ac_compile) 2>conftest.er1
	2972	ac_status=$?
	2973	grep -v '^ *+' conftest.er1 >conftest.err
	2974	rm -f conftest.er1
	2975	cat conftest.err >&5
	2976	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	2977	(exit $ac_status); } &&
	2978	{ ac_try='test -z "$ac_c_werror_flag" \|\| test ! -s conftest.err'
	2979	{ (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5
	2980	(eval $ac_try) 2>&5
	2981	ac_status=$?
	2982	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	2983	(exit $ac_status); }; } &&
	2984	{ ac_try='test -s conftest.$ac_objext'
	2985	{ (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5
	2986	(eval $ac_try) 2>&5
	2987	ac_status=$?
	2988	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	2989	(exit $ac_status); }; }; then
	2990	ac_compiler_gnu=yes
	2991	else
	2992	echo "$as_me: failed program was:" >&5
	2993	sed 's/^/\| /' conftest.$ac_ext >&5
	2994
	2995	ac_compiler_gnu=no
	2996	fi
	2997	rm -f conftest.err conftest.$ac_objext conftest.$ac_ext
	2998	ac_cv_c_compiler_gnu=$ac_compiler_gnu
	2999
	3000	fi
	3001	echo "$as_me:$LINENO: result: $ac_cv_c_compiler_gnu" >&5
	3002	echo "${ECHO_T}$ac_cv_c_compiler_gnu" >&6
	3003	GCC=`test $ac_compiler_gnu = yes && echo yes`
	3004	ac_test_CFLAGS=${CFLAGS+set}
	3005	ac_save_CFLAGS=$CFLAGS
	3006	CFLAGS="-g"
	3007	echo "$as_me:$LINENO: checking whether $CC accepts -g" >&5
	3008	echo $ECHO_N "checking whether $CC accepts -g... $ECHO_C" >&6
	3009	if test "${ac_cv_prog_cc_g+set}" = set; then
	3010	echo $ECHO_N "(cached) $ECHO_C" >&6
	3011	else
	3012	cat >conftest.$ac_ext <<_ACEOF
	3013	/* confdefs.h. */
	3014	_ACEOF
	3015	cat confdefs.h >>conftest.$ac_ext
	3016	cat >>conftest.$ac_ext <<_ACEOF
	3017	/* end confdefs.h. */
	3018
	3019	int
	3020	main ()
	3021	{
	3022
	3023	;
	3024	return 0;
	3025	}
	3026	_ACEOF
	3027	rm -f conftest.$ac_objext
	3028	if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5
	3029	(eval $ac_compile) 2>conftest.er1
	3030	ac_status=$?
	3031	grep -v '^ *+' conftest.er1 >conftest.err
	3032	rm -f conftest.er1
	3033	cat conftest.err >&5
	3034	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	3035	(exit $ac_status); } &&
	3036	{ ac_try='test -z "$ac_c_werror_flag" \|\| test ! -s conftest.err'
	3037	{ (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5
	3038	(eval $ac_try) 2>&5
	3039	ac_status=$?
	3040	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	3041	(exit $ac_status); }; } &&
	3042	{ ac_try='test -s conftest.$ac_objext'
	3043	{ (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5
	3044	(eval $ac_try) 2>&5
	3045	ac_status=$?
	3046	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	3047	(exit $ac_status); }; }; then
	3048	ac_cv_prog_cc_g=yes
	3049	else
	3050	echo "$as_me: failed program was:" >&5
	3051	sed 's/^/\| /' conftest.$ac_ext >&5
	3052
	3053	ac_cv_prog_cc_g=no
	3054	fi
	3055	rm -f conftest.err conftest.$ac_objext conftest.$ac_ext
	3056	fi
	3057	echo "$as_me:$LINENO: result: $ac_cv_prog_cc_g" >&5
	3058	echo "${ECHO_T}$ac_cv_prog_cc_g" >&6
	3059	if test "$ac_test_CFLAGS" = set; then
	3060	CFLAGS=$ac_save_CFLAGS
	3061	elif test $ac_cv_prog_cc_g = yes; then
	3062	if test "$GCC" = yes; then
	3063	CFLAGS="-g -O2"
	3064	else
	3065	CFLAGS="-g"
	3066	fi
	3067	else
	3068	if test "$GCC" = yes; then
	3069	CFLAGS="-O2"
	3070	else
	3071	CFLAGS=
	3072	fi
	3073	fi
	3074	echo "$as_me:$LINENO: checking for $CC option to accept ANSI C" >&5
	3075	echo $ECHO_N "checking for $CC option to accept ANSI C... $ECHO_C" >&6
	3076	if test "${ac_cv_prog_cc_stdc+set}" = set; then
	3077	echo $ECHO_N "(cached) $ECHO_C" >&6
	3078	else
	3079	ac_cv_prog_cc_stdc=no
	3080	ac_save_CC=$CC
	3081	cat >conftest.$ac_ext <<_ACEOF
	3082	/* confdefs.h. */
	3083	_ACEOF
	3084	cat confdefs.h >>conftest.$ac_ext
	3085	cat >>conftest.$ac_ext <<_ACEOF
	3086	/* end confdefs.h. */
	3087	#include <stdarg.h>
	3088	#include <stdio.h>
	3089	#include <sys/types.h>
	3090	#include <sys/stat.h>
	3091	/* Most of the following tests are stolen from RCS 5.7's src/conf.sh. */
	3092	struct buf { int x; };
	3093	FILE * (rcsopen) (struct buf , struct stat *, int);
	3094	static char *e (p, i)
	3095	char **p;
	3096	int i;
	3097	{
	3098	return p[i];
	3099	}
	3100	static char f (char (g) (char , int), char *p, ...)
	3101	{
	3102	char *s;
	3103	va_list v;
	3104	va_start (v,p);
	3105	s = g (p, va_arg (v,int));
	3106	va_end (v);
	3107	return s;
	3108	}
	3109
	3110	/* OSF 4.0 Compaq cc is some sort of almost-ANSI by default. It has
	3111	function prototypes and stuff, but not '\xHH' hex character constants.
	3112	These don't provoke an error unfortunately, instead are silently treated
	3113	as 'x'. The following induces an error, until -std1 is added to get
	3114	proper ANSI mode. Curiously '\x00'!='x' always comes out true, for an
	3115	array size at least. It's necessary to write '\x00'==0 to get something
	3116	that's true only with -std1. */
	3117	int osf4_cc_array ['\x00' == 0 ? 1 : -1];
	3118
	3119	int test (int i, double x);
	3120	struct s1 {int (*f) (int a);};
	3121	struct s2 {int (*f) (double a);};
	3122	int pairnames (int, char *, FILE ()(struct buf , struct stat *, int), int, int);
	3123	int argc;
	3124	char **argv;
	3125	int
	3126	main ()
	3127	{
	3128	return f (e, argv, 0) != argv[0] \|\| f (e, argv, 1) != argv[1];
	3129	;
	3130	return 0;
	3131	}
	3132	_ACEOF
	3133	# Don't try gcc -ansi; that turns off useful extensions and
	3134	# breaks some systems' header files.
	3135	# AIX -qlanglvl=ansi
	3136	# Ultrix and OSF/1 -std1
	3137	# HP-UX 10.20 and later -Ae
	3138	# HP-UX older versions -Aa -D_HPUX_SOURCE
	3139	# SVR4 -Xc -D__EXTENSIONS__
	3140	for ac_arg in "" -qlanglvl=ansi -std1 -Ae "-Aa -D_HPUX_SOURCE" "-Xc -D__EXTENSIONS__"
	3141	do
	3142	CC="$ac_save_CC $ac_arg"
	3143	rm -f conftest.$ac_objext
	3144	if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5
	3145	(eval $ac_compile) 2>conftest.er1
	3146	ac_status=$?
	3147	grep -v '^ *+' conftest.er1 >conftest.err
	3148	rm -f conftest.er1
	3149	cat conftest.err >&5
	3150	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	3151	(exit $ac_status); } &&
	3152	{ ac_try='test -z "$ac_c_werror_flag" \|\| test ! -s conftest.err'
	3153	{ (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5
	3154	(eval $ac_try) 2>&5
	3155	ac_status=$?
	3156	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	3157	(exit $ac_status); }; } &&
	3158	{ ac_try='test -s conftest.$ac_objext'
	3159	{ (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5
	3160	(eval $ac_try) 2>&5
	3161	ac_status=$?
	3162	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	3163	(exit $ac_status); }; }; then
	3164	ac_cv_prog_cc_stdc=$ac_arg
	3165	break
	3166	else
	3167	echo "$as_me: failed program was:" >&5
	3168	sed 's/^/\| /' conftest.$ac_ext >&5
	3169
	3170	fi
	3171	rm -f conftest.err conftest.$ac_objext
	3172	done
	3173	rm -f conftest.$ac_ext conftest.$ac_objext
	3174	CC=$ac_save_CC
	3175
	3176	fi
	3177
	3178	case "x$ac_cv_prog_cc_stdc" in
	3179	x\|xno)
	3180	echo "$as_me:$LINENO: result: none needed" >&5
	3181	echo "${ECHO_T}none needed" >&6 ;;
	3182	*)
	3183	echo "$as_me:$LINENO: result: $ac_cv_prog_cc_stdc" >&5
	3184	echo "${ECHO_T}$ac_cv_prog_cc_stdc" >&6
	3185	CC="$CC $ac_cv_prog_cc_stdc" ;;
	3186	esac
	3187
	3188	# Some people use a C++ compiler to compile C. Since we use `exit',
	3189	# in C++ we need to declare it. In case someone uses the same compiler
	3190	# for both compiling C and C++ we need to have the C++ compiler decide
	3191	# the declaration of exit, since it's the most demanding environment.
	3192	cat >conftest.$ac_ext <<_ACEOF
	3193	#ifndef __cplusplus
	3194	choke me
	3195	#endif
	3196	_ACEOF
	3197	rm -f conftest.$ac_objext
	3198	if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5
	3199	(eval $ac_compile) 2>conftest.er1
	3200	ac_status=$?
	3201	grep -v '^ *+' conftest.er1 >conftest.err
	3202	rm -f conftest.er1
	3203	cat conftest.err >&5
	3204	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	3205	(exit $ac_status); } &&
	3206	{ ac_try='test -z "$ac_c_werror_flag" \|\| test ! -s conftest.err'
	3207	{ (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5
	3208	(eval $ac_try) 2>&5
	3209	ac_status=$?
	3210	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	3211	(exit $ac_status); }; } &&
	3212	{ ac_try='test -s conftest.$ac_objext'
	3213	{ (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5
	3214	(eval $ac_try) 2>&5
	3215	ac_status=$?
	3216	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	3217	(exit $ac_status); }; }; then
	3218	for ac_declaration in \
	3219	'' \
	3220	'extern "C" void std::exit (int) throw (); using std::exit;' \
	3221	'extern "C" void std::exit (int); using std::exit;' \
	3222	'extern "C" void exit (int) throw ();' \
	3223	'extern "C" void exit (int);' \
	3224	'void exit (int);'
	3225	do
	3226	cat >conftest.$ac_ext <<_ACEOF
	3227	/* confdefs.h. */
	3228	_ACEOF
	3229	cat confdefs.h >>conftest.$ac_ext
	3230	cat >>conftest.$ac_ext <<_ACEOF
	3231	/* end confdefs.h. */
	3232	$ac_declaration
	3233	#include <stdlib.h>
	3234	int
	3235	main ()
	3236	{
	3237	exit (42);
	3238	;
	3239	return 0;
	3240	}
	3241	_ACEOF
	3242	rm -f conftest.$ac_objext
	3243	if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5
	3244	(eval $ac_compile) 2>conftest.er1
	3245	ac_status=$?
	3246	grep -v '^ *+' conftest.er1 >conftest.err
	3247	rm -f conftest.er1
	3248	cat conftest.err >&5
	3249	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	3250	(exit $ac_status); } &&
	3251	{ ac_try='test -z "$ac_c_werror_flag" \|\| test ! -s conftest.err'
	3252	{ (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5
	3253	(eval $ac_try) 2>&5
	3254	ac_status=$?
	3255	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	3256	(exit $ac_status); }; } &&
	3257	{ ac_try='test -s conftest.$ac_objext'
	3258	{ (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5
	3259	(eval $ac_try) 2>&5
	3260	ac_status=$?
	3261	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	3262	(exit $ac_status); }; }; then
	3263	:
	3264	else
	3265	echo "$as_me: failed program was:" >&5
	3266	sed 's/^/\| /' conftest.$ac_ext >&5
	3267
	3268	continue
	3269	fi
	3270	rm -f conftest.err conftest.$ac_objext conftest.$ac_ext
	3271	cat >conftest.$ac_ext <<_ACEOF
	3272	/* confdefs.h. */
	3273	_ACEOF
	3274	cat confdefs.h >>conftest.$ac_ext
	3275	cat >>conftest.$ac_ext <<_ACEOF
	3276	/* end confdefs.h. */
	3277	$ac_declaration
	3278	int
	3279	main ()
	3280	{
	3281	exit (42);
	3282	;
	3283	return 0;
	3284	}
	3285	_ACEOF
	3286	rm -f conftest.$ac_objext
	3287	if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5
	3288	(eval $ac_compile) 2>conftest.er1
	3289	ac_status=$?
	3290	grep -v '^ *+' conftest.er1 >conftest.err
	3291	rm -f conftest.er1
	3292	cat conftest.err >&5
	3293	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	3294	(exit $ac_status); } &&
	3295	{ ac_try='test -z "$ac_c_werror_flag" \|\| test ! -s conftest.err'
	3296	{ (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5
	3297	(eval $ac_try) 2>&5
	3298	ac_status=$?
	3299	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	3300	(exit $ac_status); }; } &&
	3301	{ ac_try='test -s conftest.$ac_objext'
	3302	{ (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5
	3303	(eval $ac_try) 2>&5
	3304	ac_status=$?
	3305	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	3306	(exit $ac_status); }; }; then
	3307	break
	3308	else
	3309	echo "$as_me: failed program was:" >&5
	3310	sed 's/^/\| /' conftest.$ac_ext >&5
	3311
	3312	fi
	3313	rm -f conftest.err conftest.$ac_objext conftest.$ac_ext
	3314	done
	3315	rm -f conftest*
	3316	if test -n "$ac_declaration"; then
	3317	echo '#ifdef __cplusplus' >>confdefs.h
	3318	echo $ac_declaration >>confdefs.h
	3319	echo '#endif' >>confdefs.h
	3320	fi
	3321
	3322	else
	3323	echo "$as_me: failed program was:" >&5
	3324	sed 's/^/\| /' conftest.$ac_ext >&5
	3325
	3326	fi
	3327	rm -f conftest.err conftest.$ac_objext conftest.$ac_ext
	3328	ac_ext=c
	3329	ac_cpp='$CPP $CPPFLAGS'
	3330	ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5'
	3331	ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'
	3332	ac_compiler_gnu=$ac_cv_c_compiler_gnu
	3333
	3334
2249	3335	echo "$as_me:$LINENO: checking whether we are using SunPro C" >&5
2250	3336	echo $ECHO_N "checking whether we are using SunPro C... $ECHO_C" >&6
2251
2252	3337	cat >conftest.$ac_ext <<_ACEOF
2253	3338	/* confdefs.h. */
2254	3339	_ACEOF

2309	3394	if test "${enable_64}" = "yes" ; then
2310	3395	CFLAGS="-xarch=v9 $CFLAGS"
2311	3396	fi
	3397	fi
	3398
	3399	# Check whether --enable-compat14 or --disable-compat14 was given.
	3400	if test "${enable_compat14+set}" = set; then
	3401	enableval="$enable_compat14"
	3402
	3403	fi;
	3404	if test "${enable_compat14}" = "yes" ; then
	3405	CFLAGS="$CFLAGS -DCOMPAT14"
2312	3406	fi
2313	3407
2314	3408	for ac_prog in 'bison -y' byacc

2856	3950
2857	3951	fi;
2858	3952
	3953	# Check whether --enable-sflow or --disable-sflow was given.
	3954	if test "${enable_sflow+set}" = set; then
	3955	enableval="$enable_sflow"
	3956	opt_objects="$opt_objects sfcapd"
	3957
	3958	fi;
	3959
2859	3960
2860	3961
2861	3962

4110	5211
4111	5212
4112	5213
4113		for ac_header in arpa/inet.h fcntl.h netinet/in.h stdint.h stdlib.h string.h sys/socket.h syslog.h unistd.h iso/limits_iso.h
	5214
	5215	for ac_header in arpa/inet.h fcntl.h netinet/in.h fts.h stdint.h stdlib.h string.h sys/socket.h syslog.h unistd.h iso/limits_iso.h
4114	5216	do
4115	5217	as_ac_Header=`echo "ac_cv_header_$ac_header" \| $as_tr_sh`
4116	5218	if eval "test \"\${$as_ac_Header+set}\" = set"; then

4257	5359	fi
4258	5360
4259	5361	done
	5362
	5363
	5364
	5365	if test "$ac_cv_header_fts_h" != yes; then
	5366	FTS_OBJ=fts_compat.o
	5367	fi
4260	5368
4261	5369
4262	5370	# Checks for typedefs, structures, and compiler characteristics.

7380	8488	done
7381	8489
7382	8490
	8491	echo "$as_me:$LINENO: checking for the %z format string in printf()" >&5
	8492	echo $ECHO_N "checking for the %z format string in printf()... $ECHO_C" >&6
	8493	if test "$cross_compiling" = yes; then
	8494
	8495	echo "$as_me:$LINENO: result: no" >&5
	8496	echo "${ECHO_T}no" >&6
	8497
	8498
	8499	else
	8500	cat >conftest.$ac_ext <<_ACEOF
	8501	/* confdefs.h. */
	8502	_ACEOF
	8503	cat confdefs.h >>conftest.$ac_ext
	8504	cat >>conftest.$ac_ext <<_ACEOF
	8505	/* end confdefs.h. */
	8506
	8507	#include <stdio.h>
	8508	#include <sys/types.h>
	8509
	8510	int main() {
	8511	int i;
	8512	size_t s;
	8513	char string[16];
	8514
	8515	s = 12345;
	8516	i = snprintf(string,16,"%zu", s);
	8517
	8518	return i == 5 ? 0 : 1;
	8519	}
	8520
	8521	_ACEOF
	8522	rm -f conftest$ac_exeext
	8523	if { (eval echo "$as_me:$LINENO: \"$ac_link\"") >&5
	8524	(eval $ac_link) 2>&5
	8525	ac_status=$?
	8526	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	8527	(exit $ac_status); } && { ac_try='./conftest$ac_exeext'
	8528	{ (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5
	8529	(eval $ac_try) 2>&5
	8530	ac_status=$?
	8531	echo "$as_me:$LINENO: \$? = $ac_status" >&5
	8532	(exit $ac_status); }; }; then
	8533
	8534	echo "$as_me:$LINENO: result: yes" >&5
	8535	echo "${ECHO_T}yes" >&6
	8536
	8537	cat >>confdefs.h <<\_ACEOF
	8538	#define HAVE_SIZE_T_Z_FORMAT 1
	8539	_ACEOF
	8540
	8541
	8542	else
	8543	echo "$as_me: program exited with status $ac_status" >&5
	8544	echo "$as_me: failed program was:" >&5
	8545	sed 's/^/\| /' conftest.$ac_ext >&5
	8546
	8547	( exit $ac_status )
	8548
	8549	echo "$as_me:$LINENO: result: no" >&5
	8550	echo "${ECHO_T}no" >&6
	8551
	8552	fi
	8553	rm -f core *.core gmon.out bb.out conftest$ac_exeext conftest.$ac_objext conftest.$ac_ext
	8554	fi
	8555
7383	8556
7384	8557
7385	8558	ac_config_files="$ac_config_files Makefile"

7746	8919	} >&5
7747	8920	cat >&5 <<_CSEOF
7748	8921
7749		This file was extended by nfdump $as_me 1.4, which was
	8922	This file was extended by nfdump $as_me 1.5, which was
7750	8923	generated by GNU Autoconf 2.59. Invocation command line was
7751	8924
7752	8925	CONFIG_FILES = $CONFIG_FILES

7806	8979
7807	8980	cat >>$CONFIG_STATUS <<_ACEOF
7808	8981	ac_cs_version="\\
7809		nfdump config.status 1.4
	8982	nfdump config.status 1.5
7810	8983	configured by $0, generated by GNU Autoconf 2.59,
7811	8984	with options \\"`echo "$ac_configure_args" \| sed 's/[\\""\`\$]/\\\\&/g'`\\"
7812	8985

8015	9188	s,@LFLAGS@,$LFLAGS,;t t
8016	9189	s,@CPP@,$CPP,;t t
8017	9190	s,@EGREP@,$EGREP,;t t
	9191	s,@FTS_OBJ@,$FTS_OBJ,;t t
8018	9192	s,@LIBOBJS@,$LIBOBJS,;t t
8019	9193	s,@HA_HAVE_SCANDIR@,$HA_HAVE_SCANDIR,;t t
8020	9194	s,@LTLIBOBJS@,$LTLIBOBJS,;t t

-0

ft2nfdump.1 less more

15	15	.B -r <flow-tools-file>
16	16	Read flow-tools formated netflow data from file \fIflow-tools-file.
17	17	Default: Read from stdin.
	18	.TP 3
	19	.B -c \fInum
	20	Read only \fBnum\fR flows from input file.
18	21	.TP 3
19	22	.B -E
20	23	Print raw netflow records to stdout. This option is for debugging purpose

+118

-58

ft2nfdump.c less more

32	32	*
33	33	* $Author: peter $
34	34	*
35		* $Id: ft2nfdump.c 53 2005-11-17 07:45:34Z peter $
	35	* $Id: ft2nfdump.c 70 2006-05-17 08:38:01Z peter $
36	36	*
37		* $LastChangedRevision: 53 $
	37	* $LastChangedRevision: 70 $
38	38	*
39	39	*
40	40	*/

53	53
54	54	#include <string.h>
55	55	#include <errno.h>
56		#include "ftbuild.h"
57	56
58	57	#include <sys/stat.h>
59	58

64	63	#endif
65	64
66	65	#include "version.h"
	66	#include "ftbuild.h"
67	67	#include "nf_common.h"
	68	#include "nffile.h"
	69	#include "launch.h"
68	70
69	71	/* Global defines */
70	72	#define MAXRECORDS 30
71	73
72	74	/* Global consts */
73		static int const BUFFSIZE = sizeof(flow_header_t) + MAXRECORDS * sizeof(flow_record_t);
74		static char const *vers_id = "$Id: ft2nfdump.c 53 2005-11-17 07:45:34Z peter $";
	75
	76	#define HIGHWATER BUFFSIZE * 0.9
	77
	78	// Keep linker happy
	79	uint32_t byte_limit, packet_limit;
	80	int byte_mode, packet_mode;
	81
	82	extern uint16_t MAGIC;
	83	extern uint16_t VERSION;
	84
	85	static char const *vers_id = "$Id: ft2nfdump.c 70 2006-05-17 08:38:01Z peter $";
	86
	87	typedef struct v5_block_s {
	88	uint32_t srcaddr;
	89	uint32_t dstaddr;
	90	uint32_t dPkts;
	91	uint32_t dOctets;
	92	uint8_t data[4]; // link to next record
	93	} v5_block_t;
75	94
76	95	/* prototypes */
77	96	void usage(char *name);
78	97
79		int flows2nfdump(struct ftio *ftio, int extended);
	98	int flows2nfdump(struct ftio *ftio, int extended, uint32_t limitflows);
80	99
81	100	void usage(char *name) {
82	101	printf("usage %s [options] \n"
83	102	"-h\t\tthis text you see right here.\n"
84	103	"-E\t\tDump records in ASCII extended format to stdout.\n"
	104	"-c\t\tLimit number of records to convert.\n"
85	105	"-V\t\tPrint version and exit.\n"
86	106	"-r\t\tread input from file\n"
87	107	"Convert flow-tools format to nfdump format:\n"

90	110
91	111	} // End of usage
92	112
93		int flows2nfdump(struct ftio *ftio, int extended) {
	113	int flows2nfdump(struct ftio *ftio, int extended, uint32_t limitflows) {
94	114	struct fttime ftt;
95	115	struct fts3rec_offsets fo;
96	116	struct ftver ftv;
97		flow_header_t *nf_header;
98		flow_record_t record_buff, nf_record;
99		char rec, string;
100		uint32_t when, unix_secs, unix_nsecs, sysUpTime;
101		int rec_count;
102		void *flow_buff;
	117	data_block_header_t *nf_header;
	118	file_header_t *file_header;
	119	common_record_t record_buff, nf_record;
	120	v5_block_t *v5_block;
	121	char rec, string;
	122	uint32_t when, unix_secs, unix_nsecs, sysUpTime, cnt, output_record_length;
	123	void *flow_buff;
	124	size_t len;
103	125
104	126	/* setup memory buffer */
105	127	flow_buff = malloc(BUFFSIZE);
106	128	if ( !flow_buff ) {
107	129	fterr_errx(1, "Buffer allocation error: %s.", strerror(errno));
108	130	}
109		nf_header = (flow_header_t *)flow_buff;
110		record_buff = (flow_record_t *)(flow_buff + sizeof(flow_header_t));
111		rec_count = 0;
112
	131	nf_header = (data_block_header_t *)flow_buff;
	132	record_buff = (common_record_t *)((pointer_addr_t)flow_buff + sizeof(data_block_header_t));
	133
	134	output_record_length = sizeof(common_record_t) + sizeof(v5_block_t) - 2 * sizeof(uint8_t[4]);
	135
113	136	/* Init defaults in header */
114		nf_header->version = 5;
115		nf_header->count = MAXRECORDS;
116		nf_header->flow_sequence = 1;
117		nf_header->engine_type = 0;
118		nf_header->engine_id = 0;
119		nf_header->layout_version = 1;
	137	nf_header->NumBlocks = 0;
	138	nf_header->size = 0;
	139	nf_header->id = DATA_BLOCK_TYPE_1;
	140	nf_header->pad = 0;
120	141
121	142	if (ftio_check_xfield(ftio, FT_XFIELD_DPKTS \|
122	143	FT_XFIELD_DOCTETS \| FT_XFIELD_FIRST \| FT_XFIELD_LAST \| FT_XFIELD_INPUT \|

128	149	return -1;
129	150	}
130	151
	152	// initialize file header and dummy stat record
	153	len = sizeof(file_header_t) + sizeof(stat_record_t);
	154	file_header = (file_header_t *)malloc(len);
	155	memset((void *)file_header, 0, len);
	156	file_header->magic = MAGIC;
	157	file_header->version = VERSION;
	158	strncpy(file_header->ident, "none", IDENT_SIZE);
	159	write(STDOUT_FILENO, (void *)file_header, len) ;
	160
	161	cnt = 0;
131	162	ftio_get_ver(ftio, &ftv);
132	163	fts3rec_compute_offsets(&fo, &ftv);
133	164
	165	nf_record = record_buff;
	166	v5_block = (v5_block_t *)nf_record->data;
134	167	while ((rec = ftio_read(ftio))) {
135		nf_record = &record_buff[rec_count];
136
137		nf_record->pad = 0;
138		nf_record->nexthop = 0;
139
140		nf_record->dOctets = ((u_int32)(rec+fo.dOctets));
141		nf_record->dPkts = ((u_int32)(rec+fo.dPkts));
	168
	169	nf_record->flags = 0;
	170	nf_record->mark = 0;
	171	nf_record->size = output_record_length;
142	172
143	173	unix_secs = ((u_int32)(rec+fo.unix_secs));
144	174	unix_nsecs = ((u_int32)(rec+fo.unix_nsecs));

146	176
147	177	when = ((u_int32)(rec+fo.First));
148	178	ftt = ftltime(sysUpTime, unix_secs, unix_nsecs, when);
149		nf_record->First = ftt.secs;
	179	nf_record->first = ftt.secs;
150	180	nf_record->msec_first = ftt.msecs;
151	181
152	182	when = ((u_int32)(rec+fo.Last));
153	183	ftt = ftltime(sysUpTime, unix_secs, unix_nsecs, when);
154		nf_record->Last = ftt.secs;
	184	nf_record->last = ftt.secs;
155	185	nf_record->msec_last = ftt.msecs;
156	186
157		nf_record->srcaddr = ((u_int32)(rec+fo.srcaddr));
158		nf_record->dstaddr = ((u_int32)(rec+fo.dstaddr));
159	187	nf_record->input = ((u_int16)(rec+fo.input));
160	188	nf_record->output = ((u_int16)(rec+fo.output));
161	189	nf_record->srcport = ((u_int16)(rec+fo.srcport));

163	191	nf_record->prot = ((u_int8)(rec+fo.prot));
164	192	nf_record->tcp_flags = ((u_int8)(rec+fo.tcp_flags));
165	193	nf_record->tos = ((u_int8)(rec+fo.tos));
166		nf_record->src_as = ((u_int16)(rec+fo.src_as));
167		nf_record->dst_as = ((u_int16)(rec+fo.dst_as));
168
169		rec_count++;
170		if ( rec_count == MAXRECORDS ) {
171		rec_count = 0;
172		if ( !extended ) {
173		nf_header->flow_sequence += MAXRECORDS;
174		nf_header->SysUptime = ((u_int32)(rec+fo.sysUpTime));
175		nf_header->unix_secs = ((u_int32)(rec+fo.unix_secs));
176		nf_header->unix_nsecs = ((u_int32)(rec+fo.unix_nsecs));
177		write(STDOUT_FILENO, flow_buff, BUFFSIZE);
178		}
179		}
	194	nf_record->srcas = ((u_int16)(rec+fo.src_as));
	195	nf_record->dstas = ((u_int16)(rec+fo.dst_as));
	196
	197	v5_block->srcaddr = ((u_int32)(rec+fo.srcaddr));
	198	v5_block->dstaddr = ((u_int32)(rec+fo.dstaddr));
	199	v5_block->dOctets = ((u_int32)(rec+fo.dOctets));
	200	v5_block->dPkts = ((u_int32)(rec+fo.dPkts));
	201
	202	nf_header->NumBlocks++;
	203	nf_header->size += output_record_length;
180	204
181	205	if ( extended ) {
182		flow_record_raw(nf_record, 0, 0, 0, &string, 0);
	206	master_record_t print_record;
	207	size_t size = sizeof(common_record_t) - sizeof(uint8_t[4]);
	208	memcpy((void )&print_record, (void )nf_record, size);
	209	print_record.v6.srcaddr[0] = 0;
	210	print_record.v6.srcaddr[1] = 0;
	211	print_record.v6.dstaddr[0] = 0;
	212	print_record.v6.dstaddr[1] = 0;
	213	print_record.v4.srcaddr = v5_block->srcaddr;
	214	print_record.v4.dstaddr = v5_block->dstaddr;
	215	print_record.dPkts = v5_block->dPkts;
	216	print_record.dOctets = v5_block->dOctets;
	217
	218	format_file_block_record(&print_record, 1, &string, 0);
183	219	printf("%s\n", string);
184	220	}
185	221
	222	if ( nf_header->size >= HIGHWATER ) {
	223	if ( !extended ) {
	224	write(STDOUT_FILENO, flow_buff, sizeof(data_block_header_t) + nf_header->size);
	225	}
	226	nf_header->NumBlocks = 0;
	227	nf_header->size = 0;
	228	nf_record = record_buff;
	229	v5_block = (v5_block_t *)nf_record->data;
	230	} else {
	231	nf_record = (common_record_t *)((pointer_addr_t)nf_record + output_record_length);
	232	v5_block = (v5_block_t *)nf_record->data;
	233	}
	234	cnt++;
	235	if ( cnt == limitflows )
	236	break;
186	237
187	238	} /* while */
188	239
189	240	// write the last records in buffer
190		if ( !extended && rec_count ) {
191		nf_header->count = rec_count;
192		nf_header->flow_sequence += rec_count;
193		write(STDOUT_FILENO, flow_buff, sizeof(flow_header_t) + rec_count * sizeof(flow_record_t));
	241	if ( !extended && nf_header->size ) {
	242	write(STDOUT_FILENO, flow_buff, sizeof(data_block_header_t) + nf_header->size);
194	243	}
	244
195	245	free(flow_buff);
196	246
197	247	return 0;

201	251	int main(int argc, char **argv) {
202	252	struct ftio ftio;
203	253	struct stat statbuf;
	254	uint32_t limitflows;
204	255	int i, extended, ret, fd;
205	256	char *ftfile;
206	257
207	258	/* init fterr */
208	259	fterr_setid(argv[0]);
209	260
210		extended = 0;
211		ftfile = NULL;
212
213		while ((i = getopt(argc, argv, "EVhr:?")) != -1)
	261	extended = 0;
	262	limitflows = 0;
	263	ftfile = NULL;
	264
	265	while ((i = getopt(argc, argv, "EVc:hr:?")) != -1)
214	266	switch (i) {
215	267	case 'h': /* help */
216	268	case '?':

227	279	extended = 1;
228	280	break;
229	281
	282	case 'c':
	283	limitflows = atoi(optarg);
	284	if ( !limitflows ) {
	285	fprintf(stderr, "Option -c needs a number > 0\n");
	286	exit(255);
	287	}
	288	break;
	289
230	290	case 'r':
231	291	ftfile = optarg;
232	292	if ( (stat(ftfile, &statbuf) < 0 ) \|\| !(statbuf.st_mode & S_IFREG) ) {

259	319	if (ftio_init(&ftio, fd, FT_IO_FLAG_READ) < 0)
260	320	fterr_errx(1, "ftio_init(): failed");
261	321
262		ret = flows2nfdump(&ftio, extended);
	322	ret = flows2nfdump(&ftio, extended, limitflows);
263	323
264	324	return ret;
265	325

+384

-291

grammar.y less more

29	29	*
30	30	* $Author: peter $
31	31	*
32		* $Id: grammar.y 34 2005-08-22 12:01:31Z peter $
	32	* $Id: grammar.y 70 2006-05-17 08:38:01Z peter $
33	33	*
34		* $LastChangedRevision: 34 $
	34	* $LastChangedRevision: 70 $
35	35	*
36	36	*
37	37	*

41	41
42	42	#include <stdio.h>
43	43	#include <sys/types.h>
	44	#include <sys/socket.h>
44	45	#include <string.h>
45	46	#include <stdlib.h>
	47	#include <ctype.h>
46	48
47	49	#include "config.h"
48	50

50	52	#include <stdint.h>
51	53	#endif
52	54
	55	#include "nf_common.h"
53	56	#include "nfdump.h"
	57	#include "nffile.h"
54	58	#include "nftree.h"
	59	#include "ipconv.h"
	60	#include "util.h"
55	61
56	62	/*
57	63	* function prototypes
58	64	*/
59	65	static void yyerror(char *msg);
60
61		static uint32_t stoipaddr(char s, uint32_t ipaddr);
62	66
63	67	enum { SOURCE = 1, DESTINATION, SOURCE_AND_DESTINATION, SOURCE_OR_DESTINATION };
64	68

72	76	%}
73	77
74	78	%union {
75		uint32_t value;
	79	uint64_t value;
76	80	char *s;
77	81	FilterParam_t param;
78	82	}
79	83
80		%token ANY IP IF NEXT TCP UDP ICMP GRE ESP AH RSVP PROTO TOS FLAGS HOST NET PORT IN OUT SRC DST EQ LT GT
81		%token NUMBER QUADDOT ALPHA_FLAGS PORTNUM ICMPTYPE AS PACKETS BYTES PPS BPS BPP DURATION
	84	%token ANY IP IF IDENT TOS FLAGS HOST NET PORT IN OUT SRC DST EQ LT GT
	85	%token NUMBER IPSTRING ALPHA_FLAGS PROTOSTR PORTNUM ICMPTYPE AS PACKETS BYTES PPS BPS BPP DURATION
	86	%token IPV4 IPV6
82	87	%token NOT END
83		%type <value> expr NUMBER PORTNUM NETNUM ICMPTYPE
84		%type <s> QUADDOT ALPHA_FLAGS
	88	%type <value> expr NUMBER PORTNUM ICMPTYPE
	89	%type <s> IPSTRING IDENT ALPHA_FLAGS PROTOSTR
85	90	%type <param> dqual inout term comp scale
86	91
87	92	%left '+' OR

91	96	%%
92	97	prog: /* empty */
93	98	\| expr {
94		StartNode = $1;
	99	StartNode = $1;
	100	}
	101	;
	102
	103	term: ANY { /* this is an unconditionally true expression, as a filter applies in any case */
	104	$$.self = NewBlock(OffsetProto, 0, 0, CMP_EQ, FUNC_NONE );
	105	}
	106
	107	\| IDENT {
	108	uint32_t index = AddIdent($1);
	109	$$.self = NewBlock(0, 0, index, CMP_IDENT, FUNC_NONE );
	110	}
	111
	112	\| IPV4 {
	113	$$.self = NewBlock(OffsetRecordFlags, (1LL << ShiftRecordFlags) & MaskRecordFlags,
	114	(0LL << ShiftRecordFlags) & MaskRecordFlags, CMP_EQ, FUNC_NONE);
	115	}
	116
	117	\| IPV6 {
	118	$$.self = NewBlock(OffsetRecordFlags, (1LL << ShiftRecordFlags) & MaskRecordFlags,
	119	(1LL << ShiftRecordFlags) & MaskRecordFlags, CMP_EQ, FUNC_NONE);
	120	}
	121
	122	\| PROTOSTR {
	123	int64_t proto;
	124	char *s = $1;
	125	while ( *s && isdigit((int)s[0]) ) s++;
	126	if ( *s ) // alpha string for protocol
	127	proto = Proto_num($1);
	128	else
	129	proto = atoi($1);
	130
	131	if ( proto > 255 ) {
	132	yyerror("Protocol number > 255");
	133	YYABORT;
	134	}
	135	if ( proto < 0 ) {
	136	yyerror("Unknown protocol");
	137	YYABORT;
	138	}
	139	$$.self = NewBlock(OffsetProto, MaskProto, (proto << ShiftProto) & MaskProto, CMP_EQ, FUNC_NONE);
	140	}
	141
	142	\| PACKETS comp NUMBER scale {
	143	$$.self = NewBlock(OffsetPackets, MaskPackets, $3 * $4.scale, $2.comp, FUNC_NONE);
	144	}
	145
	146	\| BYTES comp NUMBER scale {
	147	$$.self = NewBlock(OffsetBytes, MaskBytes, $3 * $4.scale , $2.comp, FUNC_NONE);
	148	}
	149
	150	\| PPS comp NUMBER scale {
	151	$$.self = NewBlock(0, AnyMask, $3 * $4.scale , $2.comp, FUNC_PPS);
	152	}
	153
	154	\| BPS comp NUMBER scale {
	155	$$.self = NewBlock(0, AnyMask, $3 * $4.scale , $2.comp, FUNC_BPS);
	156	}
	157
	158	\| BPP comp NUMBER scale {
	159	$$.self = NewBlock(0, AnyMask, $3 * $4.scale , $2.comp, FUNC_BPP);
	160	}
	161
	162	\| DURATION comp NUMBER {
	163	$$.self = NewBlock(0, AnyMask, $3, $2.comp, FUNC_DURATION);
	164	}
	165
	166	\| TOS comp NUMBER {
	167	if ( $3 > 255 ) {
	168	yyerror("TOS must be 0..255");
	169	YYABORT;
	170	}
	171	$$.self = NewBlock(OffsetTos, MaskTos, ($3 << ShiftTos) & MaskTos, $2.comp, FUNC_NONE);
	172	}
	173
	174	\| FLAGS comp NUMBER {
	175	if ( $3 > 63 ) {
	176	yyerror("Flags must be 0..63");
	177	YYABORT;
	178	}
	179	$$.self = NewBlock(OffsetFlags, MaskFlags, ($3 << ShiftFlags) & MaskFlags, $2.comp, FUNC_NONE);
	180	}
	181
	182	\| FLAGS ALPHA_FLAGS {
	183	uint64_t fl = 0;
	184	if ( strlen($2) > 7 ) {
	185	yyerror("Too many flags");
	186	YYABORT;
	187	}
	188
	189	if ( strchr($2, 'F') ) fl \|= 1;
	190	if ( strchr($2, 'S') ) fl \|= 2;
	191	if ( strchr($2, 'R') ) fl \|= 4;
	192	if ( strchr($2, 'P') ) fl \|= 8;
	193	if ( strchr($2, 'A') ) fl \|= 16;
	194	if ( strchr($2, 'U') ) fl \|= 32;
	195	if ( strchr($2, 'X') ) fl = 63;
	196
	197	$$.self = NewBlock(OffsetFlags, (fl << ShiftFlags) & MaskFlags,
	198	(fl << ShiftFlags) & MaskFlags, CMP_FLAGS, FUNC_NONE);
	199	}
	200
	201	\| dqual IP IPSTRING {
	202	int af, bytes;
	203	if ( parse_ip(&af, $3, $$.ip, &bytes) == 0 ) {
	204	yyerror("Invalid IP address");
	205	YYABORT;
	206	}
	207	if ( ( af == PF_INET && bytes != 4 ) \|\| ( af == PF_INET6 && bytes != 16 )) {
	208	yyerror("incomplete IP address");
	209	YYABORT;
	210	}
	211
	212	$$.direction = $1.direction;
	213	if ( $$.direction == SOURCE ) {
	214	$$.self = Connect_AND(
	215	NewBlock(OffsetSrcIPv6b, MaskIPv6, $$.ip[1] , CMP_EQ, FUNC_NONE ),
	216	NewBlock(OffsetSrcIPv6a, MaskIPv6, $$.ip[0] , CMP_EQ, FUNC_NONE )
	217	);
	218	} else if ( $$.direction == DESTINATION) {
	219	$$.self = Connect_AND(
	220	NewBlock(OffsetDstIPv6b, MaskIPv6, $$.ip[1] , CMP_EQ, FUNC_NONE ),
	221	NewBlock(OffsetDstIPv6a, MaskIPv6, $$.ip[0] , CMP_EQ, FUNC_NONE )
	222	);
	223	} else if ( $$.direction == SOURCE_OR_DESTINATION ) {
	224	$$.self = Connect_OR(
	225	Connect_AND(
	226	NewBlock(OffsetSrcIPv6b, MaskIPv6, $$.ip[1] , CMP_EQ, FUNC_NONE ),
	227	NewBlock(OffsetSrcIPv6a, MaskIPv6, $$.ip[0] , CMP_EQ, FUNC_NONE )
	228	),
	229	Connect_AND(
	230	NewBlock(OffsetDstIPv6b, MaskIPv6, $$.ip[1] , CMP_EQ, FUNC_NONE ),
	231	NewBlock(OffsetDstIPv6a, MaskIPv6, $$.ip[0] , CMP_EQ, FUNC_NONE )
	232	)
	233	);
	234	} else if ( $$.direction == SOURCE_AND_DESTINATION ) {
	235	$$.self = Connect_AND(
	236	Connect_AND(
	237	NewBlock(OffsetSrcIPv6b, MaskIPv6, $$.ip[1] , CMP_EQ, FUNC_NONE ),
	238	NewBlock(OffsetSrcIPv6a, MaskIPv6, $$.ip[0] , CMP_EQ, FUNC_NONE )
	239	),
	240	Connect_AND(
	241	NewBlock(OffsetDstIPv6b, MaskIPv6, $$.ip[1] , CMP_EQ, FUNC_NONE ),
	242	NewBlock(OffsetDstIPv6a, MaskIPv6, $$.ip[0] , CMP_EQ, FUNC_NONE )
	243	)
	244	);
	245	} else {
	246	/* should never happen */
	247	yyerror("Internal parser error");
	248	YYABORT;
	249	}
	250	}
	251
	252	\| dqual PORT comp NUMBER {
	253	$$.direction = $1.direction;
	254	if ( $4 > 65535 ) {
	255	yyerror("Port outside of range 0..65535");
	256	YYABORT;
	257	}
	258
	259	if ( $$.direction == SOURCE ) {
	260	$$.self = NewBlock(OffsetPort, MaskSrcPort, ($4 << ShiftSrcPort) & MaskSrcPort, $3.comp, FUNC_NONE );
	261	} else if ( $$.direction == DESTINATION) {
	262	$$.self = NewBlock(OffsetPort, MaskDstPort, ($4 << ShiftDstPort) & MaskDstPort, $3.comp, FUNC_NONE );
	263	} else if ( $$.direction == SOURCE_OR_DESTINATION ) {
	264	$$.self = Connect_OR(
	265	NewBlock(OffsetPort, MaskSrcPort, ($4 << ShiftSrcPort) & MaskSrcPort, $3.comp, FUNC_NONE ),
	266	NewBlock(OffsetPort, MaskDstPort, ($4 << ShiftDstPort) & MaskDstPort, $3.comp, FUNC_NONE )
	267	);
	268	} else if ( $$.direction == SOURCE_AND_DESTINATION ) {
	269	$$.self = Connect_AND(
	270	NewBlock(OffsetPort, MaskSrcPort, ($4 << ShiftSrcPort) & MaskSrcPort, $3.comp, FUNC_NONE ),
	271	NewBlock(OffsetPort, MaskDstPort, ($4 << ShiftDstPort) & MaskDstPort, $3.comp, FUNC_NONE )
	272	);
	273	} else {
	274	/* should never happen */
	275	yyerror("Internal parser error");
	276	YYABORT;
	277	}
	278	}
	279
	280	\| dqual AS NUMBER {
	281	$$.direction = $1.direction;
	282	if ( $3 > 65535 \|\| $3 < 0 ) {
	283	yyerror("AS number outside of range 0..65535");
	284	YYABORT;
	285	}
	286
	287	if ( $$.direction == SOURCE ) {
	288	$$.self = NewBlock(OffsetAS, MaskSrcAS, ($3 << ShiftSrcAS) & MaskSrcAS, CMP_EQ, FUNC_NONE );
	289	} else if ( $$.direction == DESTINATION) {
	290	$$.self = NewBlock(OffsetAS, MaskDstAS, ($3 << ShiftDstAS) & MaskDstAS, CMP_EQ, FUNC_NONE);
	291	} else if ( $$.direction == SOURCE_OR_DESTINATION ) {
	292	$$.self = Connect_OR(
	293	NewBlock(OffsetAS, MaskSrcAS, ($3 << ShiftSrcAS) & MaskSrcAS, CMP_EQ, FUNC_NONE ),
	294	NewBlock(OffsetAS, MaskDstAS, ($3 << ShiftDstAS) & MaskDstAS, CMP_EQ, FUNC_NONE)
	295	);
	296	} else if ( $$.direction == SOURCE_AND_DESTINATION ) {
	297	$$.self = Connect_AND(
	298	NewBlock(OffsetPort, MaskSrcAS, ($3 << ShiftSrcAS) & MaskSrcAS, CMP_EQ, FUNC_NONE ),
	299	NewBlock(OffsetPort, MaskDstAS, ($3 << ShiftDstAS) & MaskDstAS, CMP_EQ, FUNC_NONE)
	300	);
	301	} else {
	302	/* should never happen */
	303	yyerror("Internal parser error");
	304	YYABORT;
	305	}
	306	}
	307
	308	\| dqual NET IPSTRING IPSTRING {
	309	int af, bytes;
	310	uint64_t mask[2];
	311	if ( parse_ip(&af, $3, $$.ip, &bytes) == 0 ) {
	312	yyerror("Invalid IP address");
	313	YYABORT;
	314	}
	315	if ( af != PF_INET ) {
	316	yyerror("IP netmask syntax valid only for IPv4");
	317	YYABORT;
	318	}
	319	if ( bytes != 4 ) {
	320	yyerror("Need complete IP address");
	321	YYABORT;
	322	}
	323	if ( parse_ip(&af, $4, mask, &bytes) == 0 ) {
	324	yyerror("Invalid IP address");
	325	YYABORT;
	326	}
	327	if ( af != PF_INET \|\| bytes != 4 ) {
	328	yyerror("Invalid netmask for IPv4 address");
	329	YYABORT;
	330	}
	331
	332	$$.ip[0] &= mask[0];
	333	$$.ip[1] &= mask[1];
	334
	335	$$.direction = $1.direction;
	336
	337	if ( $$.direction == SOURCE ) {
	338	$$.self = Connect_AND(
	339	NewBlock(OffsetSrcIPv6b, mask[1], $$.ip[1] , CMP_EQ, FUNC_NONE ),
	340	NewBlock(OffsetSrcIPv6a, mask[0], $$.ip[0] , CMP_EQ, FUNC_NONE )
	341	);
	342	} else if ( $$.direction == DESTINATION) {
	343	$$.self = Connect_AND(
	344	NewBlock(OffsetDstIPv6b, mask[1], $$.ip[1] , CMP_EQ, FUNC_NONE ),
	345	NewBlock(OffsetDstIPv6a, mask[0], $$.ip[0] , CMP_EQ, FUNC_NONE )
	346	);
	347	} else if ( $$.direction == SOURCE_OR_DESTINATION ) {
	348	$$.self = Connect_OR(
	349	Connect_AND(
	350	NewBlock(OffsetSrcIPv6b, mask[1], $$.ip[1] , CMP_EQ, FUNC_NONE ),
	351	NewBlock(OffsetSrcIPv6a, mask[0], $$.ip[0] , CMP_EQ, FUNC_NONE )
	352	),
	353	Connect_AND(
	354	NewBlock(OffsetDstIPv6b, mask[1], $$.ip[1] , CMP_EQ, FUNC_NONE ),
	355	NewBlock(OffsetDstIPv6a, mask[0], $$.ip[0] , CMP_EQ, FUNC_NONE )
	356	)
	357	);
	358	} else if ( $$.direction == SOURCE_AND_DESTINATION ) {
	359	$$.self = Connect_AND(
	360	Connect_AND(
	361	NewBlock(OffsetSrcIPv6b, mask[1], $$.ip[1] , CMP_EQ, FUNC_NONE ),
	362	NewBlock(OffsetSrcIPv6a, mask[0], $$.ip[0] , CMP_EQ, FUNC_NONE )
	363	),
	364	Connect_AND(
	365	NewBlock(OffsetDstIPv6b, mask[1], $$.ip[1] , CMP_EQ, FUNC_NONE ),
	366	NewBlock(OffsetDstIPv6a, mask[0], $$.ip[0] , CMP_EQ, FUNC_NONE )
	367	)
	368	);
	369	} else {
	370	/* should never happen */
	371	yyerror("Internal parser error");
	372	YYABORT;
	373	}
	374	}
	375
	376	\| dqual NET IPSTRING '/' NUMBER {
	377	int af, bytes;
	378	uint64_t mask[2];
	379	if ( parse_ip(&af, $3, $$.ip, &bytes) == 0 ) {
	380	yyerror("Invalid IP address");
	381	YYABORT;
	382	}
	383
	384	if ( $5 > (bytes*8) ) {
	385	yyerror("Too many netbits for this IP addresss");
	386	YYABORT;
	387	}
	388
	389	if ( af == PF_INET ) {
	390	mask[0] = 0xffffffffffffffffLL;
	391	mask[1] = 0xffffffffffffffffLL << ( 32 - $5 );
	392	} else { // PF_INET6
	393	if ( $5 > 64 ) {
	394	mask[0] = 0xffffffffffffffffLL;
	395	mask[1] = 0xffffffffffffffffLL << ( 128 - $5 );
	396	} else {
	397	mask[0] = 0xffffffffffffffffLL << ( 64 - $5 );
	398	mask[1] = 0;
95	399	}
96		;
97
98		term: ANY { /* this is an unconditionally true expression, as a filter applies in any case */
99		$$.self = NewBlock(OffsetProto, 0, 0, CMP_EQ, FUNC_NONE ); }
100
101		\| ICMP { $$.self = NewBlock(OffsetProto, MaskProto, (uint32_t)(1 << ShiftProto) & MaskProto, CMP_EQ, FUNC_NONE); }
102
103		\| TCP { $$.self = NewBlock(OffsetProto, MaskProto, (uint32_t)(6 << ShiftProto) & MaskProto, CMP_EQ, FUNC_NONE); }
104
105		\| UDP { $$.self = NewBlock(OffsetProto, MaskProto, (uint32_t)(17 << ShiftProto) & MaskProto, CMP_EQ, FUNC_NONE); }
106
107		\| RSVP { $$.self = NewBlock(OffsetProto, MaskProto, (uint32_t)(46 << ShiftProto) & MaskProto, CMP_EQ, FUNC_NONE); }
108
109		\| GRE { $$.self = NewBlock(OffsetProto, MaskProto, (uint32_t)(47 << ShiftProto) & MaskProto, CMP_EQ, FUNC_NONE); }
110
111		\| ESP { $$.self = NewBlock(OffsetProto, MaskProto, (uint32_t)(50 << ShiftProto) & MaskProto, CMP_EQ, FUNC_NONE); }
112
113		\| AH { $$.self = NewBlock(OffsetProto, MaskProto, (uint32_t)(51 << ShiftProto) & MaskProto, CMP_EQ, FUNC_NONE); }
114
115		\| PROTO NUMBER { $$.self = NewBlock(OffsetProto, MaskProto, (uint32_t)($2 << ShiftProto) & MaskProto, CMP_EQ, FUNC_NONE); }
116		\| PACKETS comp NUMBER scale {
117		$$.self = NewBlock(OffsetPackets, MaskSize, (uint32_t)$3 * $4.scale, $2.comp, FUNC_NONE);
118		}
119
120		\| BYTES comp NUMBER scale {
121		$$.self = NewBlock(OffsetBytes, MaskSize, (uint32_t)$3 * $4.scale , $2.comp, FUNC_NONE);
122		}
123
124		\| PPS comp NUMBER scale {
125		$$.self = NewBlock(0, AnyMask, (uint32_t)$3 * $4.scale , $2.comp, FUNC_PPS);
126		}
127
128		\| BPS comp NUMBER scale {
129		$$.self = NewBlock(0, AnyMask, (uint32_t)$3 * $4.scale , $2.comp, FUNC_BPS);
130		}
131
132		\| BPP comp NUMBER scale {
133		$$.self = NewBlock(0, AnyMask, (uint32_t)$3 * $4.scale , $2.comp, FUNC_BPP);
134		}
135
136		\| DURATION comp NUMBER {
137		$$.self = NewBlock(0, AnyMask, (uint32_t)$3, $2.comp, FUNC_DURATION);
138		}
139
140		\| TOS comp NUMBER {
141		if ( $3 > 255 ) {
142		yyerror("TOS must be 0..255");
143		YYABORT;
144		}
145		$$.self = NewBlock(OffsetTos, MaskTos, (uint32_t)($3 << ShiftTos) & MaskTos, $2.comp, FUNC_NONE);
146
147		}
148
149		\| FLAGS comp NUMBER { if ( $3 > 63 ) {
150		yyerror("Flags must be 0..63");
151		YYABORT;
152		}
153		$$.self = NewBlock(OffsetFlags, MaskFlags, (uint32_t)($3 << ShiftFlags) & MaskFlags, $2.comp, FUNC_NONE);
154		}
155
156		\| FLAGS ALPHA_FLAGS {
157		int fl = 0;
158		if ( strlen($2) > 7 ) {
159		yyerror("Too many flags");
160		YYABORT;
161		}
162
163		if ( strchr($2, 'F') ) fl \|= 1;
164		if ( strchr($2, 'S') ) fl \|= 2;
165		if ( strchr($2, 'R') ) fl \|= 4;
166		if ( strchr($2, 'P') ) fl \|= 8;
167		if ( strchr($2, 'A') ) fl \|= 16;
168		if ( strchr($2, 'U') ) fl \|= 32;
169		if ( strchr($2, 'X') ) fl = 63;
170
171		$$.self = NewBlock(OffsetFlags, (uint32_t)(fl << ShiftFlags) & MaskFlags, (uint32_t)(fl << ShiftFlags) & MaskFlags, CMP_EQ, FUNC_NONE);
172		}
173
174		\| dqual HOST QUADDOT { if ( !stoipaddr($3, &$$.ip) )
175		YYABORT;
176		$$.direction = $1.direction;
177		if ( $$.direction == SOURCE ) {
178		$$.self = NewBlock(OffsetSrcIP, MaskIP, $$.ip, CMP_EQ, FUNC_NONE );
179		} else if ( $$.direction == DESTINATION) {
180		$$.self = NewBlock(OffsetDstIP, MaskIP, $$.ip, CMP_EQ, FUNC_NONE );
181		} else if ( $$.direction == SOURCE_OR_DESTINATION ) {
182		$$.self = Connect_OR(
183		NewBlock(OffsetSrcIP, MaskIP, $$.ip, CMP_EQ, FUNC_NONE ),
184		NewBlock(OffsetDstIP, MaskIP, $$.ip, CMP_EQ, FUNC_NONE )
185		);
186		} else if ( $$.direction == SOURCE_AND_DESTINATION ) {
187		$$.self = Connect_AND(
188		NewBlock(OffsetSrcIP, MaskIP, $$.ip, CMP_EQ, FUNC_NONE ),
189		NewBlock(OffsetDstIP, MaskIP, $$.ip, CMP_EQ, FUNC_NONE )
190		);
191		} else {
192		/* should never happen */
193		yyerror("Internal parser error");
194		YYABORT;
195		}
196		}
197
198		\| dqual IP QUADDOT { if ( !stoipaddr($3, &$$.ip) )
199		YYABORT;
200
201		$$.direction = $1.direction;
202		if ( $$.direction == SOURCE ) {
203		$$.self = NewBlock(OffsetSrcIP, MaskIP, $$.ip, CMP_EQ, FUNC_NONE );
204		} else if ( $$.direction == DESTINATION) {
205		$$.self = NewBlock(OffsetDstIP, MaskIP, $$.ip, CMP_EQ, FUNC_NONE );
206		} else if ( $$.direction == SOURCE_OR_DESTINATION ) {
207		$$.self = Connect_OR(
208		NewBlock(OffsetSrcIP, MaskIP, $$.ip, CMP_EQ, FUNC_NONE ),
209		NewBlock(OffsetDstIP, MaskIP, $$.ip, CMP_EQ, FUNC_NONE )
210		);
211		} else if ( $$.direction == SOURCE_AND_DESTINATION ) {
212		$$.self = Connect_AND(
213		NewBlock(OffsetSrcIP, MaskIP, $$.ip, CMP_EQ, FUNC_NONE ),
214		NewBlock(OffsetDstIP, MaskIP, $$.ip, CMP_EQ, FUNC_NONE )
215		);
216		} else {
217		/* should never happen */
218		yyerror("Internal parser error");
219		YYABORT;
220		}
221
222		}
223
224		\| NEXT QUADDOT { if ( !stoipaddr($2, &$$.ip) )
225		YYABORT;
226
227		$$.self = NewBlock(OffsetNext, MaskIP, $$.ip, CMP_EQ, FUNC_NONE );
228
229		}
230
231		\| dqual PORT comp NUMBER { $$.direction = $1.direction;
232		if ( $4 > 65535 ) {
233		yyerror("Port outside of range 0..65535");
234		YYABORT;
235		}
236
237		if ( $$.direction == SOURCE ) {
238		$$.self = NewBlock(OffsetPort, MaskSrcPort, ($4 << ShiftSrcPort) & MaskSrcPort, $3.comp, FUNC_NONE );
239		} else if ( $$.direction == DESTINATION) {
240		$$.self = NewBlock(OffsetPort, MaskDstPort, ($4 << ShiftDstPort) & MaskDstPort, $3.comp, FUNC_NONE );
241		} else if ( $$.direction == SOURCE_OR_DESTINATION ) {
242		$$.self = Connect_OR(
243		NewBlock(OffsetPort, MaskSrcPort, ($4 << ShiftSrcPort) & MaskSrcPort, $3.comp, FUNC_NONE ),
244		NewBlock(OffsetPort, MaskDstPort, ($4 << ShiftDstPort) & MaskDstPort, $3.comp, FUNC_NONE )
245		);
246		} else if ( $$.direction == SOURCE_AND_DESTINATION ) {
247		$$.self = Connect_AND(
248		NewBlock(OffsetPort, MaskSrcPort, ($4 << ShiftSrcPort) & MaskSrcPort, $3.comp, FUNC_NONE ),
249		NewBlock(OffsetPort, MaskDstPort, ($4 << ShiftDstPort) & MaskDstPort, $3.comp, FUNC_NONE )
250		);
251		} else {
252		/* should never happen */
253		yyerror("Internal parser error");
254		YYABORT;
255		}
256		}
257
258		\| dqual AS NUMBER { $$.direction = $1.direction;
259		if ( $3 > 65535 \|\| $3 < 1 ) {
260		yyerror("AS number outside of range 1..65535");
261		YYABORT;
262		}
263
264		if ( $$.direction == SOURCE ) {
265		$$.self = NewBlock(OffsetAS, MaskSrcAS, ($3 << ShiftSrcAS) & MaskSrcAS, CMP_EQ, FUNC_NONE );
266		} else if ( $$.direction == DESTINATION) {
267		$$.self = NewBlock(OffsetAS, MaskDstAS, ($3 << ShiftDstAS) & MaskDstAS, CMP_EQ, FUNC_NONE);
268		} else if ( $$.direction == SOURCE_OR_DESTINATION ) {
269		$$.self = Connect_OR(
270		NewBlock(OffsetAS, MaskSrcAS, ($3 << ShiftSrcAS) & MaskSrcAS, CMP_EQ, FUNC_NONE ),
271		NewBlock(OffsetAS, MaskDstAS, ($3 << ShiftDstAS) & MaskDstAS, CMP_EQ, FUNC_NONE)
272		);
273		} else if ( $$.direction == SOURCE_AND_DESTINATION ) {
274		$$.self = Connect_AND(
275		NewBlock(OffsetPort, MaskSrcAS, ($3 << ShiftSrcAS) & MaskSrcAS, CMP_EQ, FUNC_NONE ),
276		NewBlock(OffsetPort, MaskDstAS, ($3 << ShiftDstAS) & MaskDstAS, CMP_EQ, FUNC_NONE)
277		);
278		} else {
279		/* should never happen */
280		yyerror("Internal parser error");
281		YYABORT;
282		}
283		}
284
285		\| dqual NET QUADDOT NETNUM { if ( !stoipaddr($3, &$$.ip) )
286		YYABORT;
287		$$.direction = $1.direction;
288		if ( $$.direction == SOURCE ) {
289		$$.self = NewBlock(OffsetSrcIP, $4, $$.ip & $4, CMP_EQ, FUNC_NONE);
290		} else if ( $$.direction == DESTINATION) {
291		$$.self = NewBlock(OffsetDstIP, $4, $$.ip & $4, CMP_EQ, FUNC_NONE);
292		} else if ( $$.direction == SOURCE_OR_DESTINATION ) {
293		$$.self = Connect_OR(
294		NewBlock(OffsetSrcIP, $4, $$.ip & $4, CMP_EQ, FUNC_NONE),
295		NewBlock(OffsetDstIP, $4, $$.ip & $4, CMP_EQ, FUNC_NONE)
296		);
297		} else if ( $$.direction == SOURCE_AND_DESTINATION ) {
298		$$.self = Connect_AND(
299		NewBlock(OffsetSrcIP, $4, $$.ip & $4, CMP_EQ, FUNC_NONE),
300		NewBlock(OffsetDstIP, $4, $$.ip & $4, CMP_EQ, FUNC_NONE)
301		);
302		} else {
303		/* should never happen */
304		yyerror("Internal parser error");
305		YYABORT;
306		}
307
308		}
309
310		\| inout IF NUMBER {
311		if ( $3 > 65535 ) {
312		yyerror("Input interface number must be 0..65535");
313		YYABORT;
314		}
315		if ( $$.direction == SOURCE ) {
316		$$.self = NewBlock(OffsetInOut, MaskInput, (uint32_t)($3 << ShiftInput) & MaskInput, CMP_EQ, FUNC_NONE);
317		} else if ( $$.direction == DESTINATION) {
318		$$.self = NewBlock(OffsetInOut, MaskOutput, (uint32_t)($3 << ShiftOutput) & MaskOutput, CMP_EQ, FUNC_NONE);
319		} else if ( $$.direction == SOURCE_OR_DESTINATION ) {
320		$$.self = Connect_OR(
321		NewBlock(OffsetInOut, MaskInput, (uint32_t)($3 << ShiftInput) & MaskInput, CMP_EQ, FUNC_NONE),
322		NewBlock(OffsetInOut, MaskOutput, (uint32_t)($3 << ShiftOutput) & MaskOutput, CMP_EQ, FUNC_NONE)
323		);
324		} else {
325		/* should never happen */
326		yyerror("Internal parser error");
327		YYABORT;
328		}
329		}
330
331		\| ICMP ICMPTYPE { $$.proto = 1;
332
333		}
334		;
335
336		NETNUM: QUADDOT { if ( !stoipaddr($1, &$$) )
337		YYABORT;
338		}
339		\| '/' NUMBER { if ( $2 > 32 \|\| $2 < 1 ) {
340		yyerror("Mask bits outside of range 1..32");
341		YYABORT;
342		}
343		$$ = 0xffffffff << ( 32 - $2 );
344		/* $$ = !( 1 << ( $2 -1 ) ); */
345
346		}
	400	}
	401	// IP aadresses are stored in network representation
	402	mask[0] = mask[0];
	403	mask[1] = mask[1];
	404
	405	$$.ip[0] &= mask[0];
	406	$$.ip[1] &= mask[1];
	407
	408	$$.direction = $1.direction;
	409	if ( $$.direction == SOURCE ) {
	410	$$.self = Connect_AND(
	411	NewBlock(OffsetSrcIPv6b, mask[1], $$.ip[1] , CMP_EQ, FUNC_NONE ),
	412	NewBlock(OffsetSrcIPv6a, mask[0], $$.ip[0] , CMP_EQ, FUNC_NONE )
	413	);
	414	} else if ( $$.direction == DESTINATION) {
	415	$$.self = Connect_AND(
	416	NewBlock(OffsetDstIPv6b, mask[1], $$.ip[1] , CMP_EQ, FUNC_NONE ),
	417	NewBlock(OffsetDstIPv6a, mask[0], $$.ip[0] , CMP_EQ, FUNC_NONE )
	418	);
	419	} else if ( $$.direction == SOURCE_OR_DESTINATION ) {
	420	$$.self = Connect_OR(
	421	Connect_AND(
	422	NewBlock(OffsetSrcIPv6b, mask[1], $$.ip[1] , CMP_EQ, FUNC_NONE ),
	423	NewBlock(OffsetSrcIPv6a, mask[0], $$.ip[0] , CMP_EQ, FUNC_NONE )
	424	),
	425	Connect_AND(
	426	NewBlock(OffsetDstIPv6b, mask[1], $$.ip[1] , CMP_EQ, FUNC_NONE ),
	427	NewBlock(OffsetDstIPv6a, mask[0], $$.ip[0] , CMP_EQ, FUNC_NONE )
	428	)
	429	);
	430	} else if ( $$.direction == SOURCE_AND_DESTINATION ) {
	431	$$.self = Connect_AND(
	432	Connect_AND(
	433	NewBlock(OffsetSrcIPv6b, mask[1], $$.ip[1] , CMP_EQ, FUNC_NONE ),
	434	NewBlock(OffsetSrcIPv6a, mask[0], $$.ip[0] , CMP_EQ, FUNC_NONE )
	435	),
	436	Connect_AND(
	437	NewBlock(OffsetDstIPv6b, mask[1], $$.ip[1] , CMP_EQ, FUNC_NONE ),
	438	NewBlock(OffsetDstIPv6a, mask[0], $$.ip[0] , CMP_EQ, FUNC_NONE )
	439	)
	440	);
	441	} else {
	442	/* should never happen */
	443	yyerror("Internal parser error");
	444	YYABORT;
	445	}
	446	}
	447
	448	\| inout IF NUMBER {
	449	if ( $3 > 65535 ) {
	450	yyerror("Input interface number must be 0..65535");
	451	YYABORT;
	452	}
	453	if ( $$.direction == SOURCE ) {
	454	$$.self = NewBlock(OffsetInOut, MaskInput, ($3 << ShiftInput) & MaskInput, CMP_EQ, FUNC_NONE);
	455	} else if ( $$.direction == DESTINATION) {
	456	$$.self = NewBlock(OffsetInOut, MaskOutput, ($3 << ShiftOutput) & MaskOutput, CMP_EQ, FUNC_NONE);
	457	} else if ( $$.direction == SOURCE_OR_DESTINATION ) {
	458	$$.self = Connect_OR(
	459	NewBlock(OffsetInOut, MaskInput, ($3 << ShiftInput) & MaskInput, CMP_EQ, FUNC_NONE),
	460	NewBlock(OffsetInOut, MaskOutput, ($3 << ShiftOutput) & MaskOutput, CMP_EQ, FUNC_NONE)
	461	);
	462	} else {
	463	/* should never happen */
	464	yyerror("Internal parser error");
	465	YYABORT;
	466	}
	467	}
	468
347	469	;
348	470
349	471	/* scaling qualifiers */

388	510	fprintf(stderr,"line %d: %s at '%s'\n", lineno, msg, yytext);
389	511	} /* End of yyerror */
390	512
391		uint32_t stoipaddr(char s, uint32_t ipaddr) {
392		uint n, i;
393		char p, q;
394
395		*ipaddr = 0;
396		p = s;
397		for ( i=0; i < 4; i++ ) {
398		if ( p ) {
399		if ((q = strchr(p,'.')) != NULL ) {
400		*q = 0;
401		}
402		n = atoi(p);
403		if ( n < 0 \|\| n > 255 ) {
404		yyerror("Bad IP address");
405		return 0;
406		}
407		if ( q )
408		p = q + 1;
409		else
410		p = NULL;
411		} else {
412		n = 0;
413		}
414		ipaddr = ( ipaddr << 8 ) \| n;
415		}
416		return 1;
417
418		} /* End of stoipaddr */
419
420
	513

+150

-0

inline.c less more

	0	/*
	1	* This file is part of the nfdump project.
	2	*
	3	* Copyright (c) 2004, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
	4	* All rights reserved.
	5	*
	6	* Redistribution and use in source and binary forms, with or without
	7	* modification, are permitted provided that the following conditions are met:
	8	*
	9	* * Redistributions of source code must retain the above copyright notice,
	10	* this list of conditions and the following disclaimer.
	11	* * Redistributions in binary form must reproduce the above copyright notice,
	12	* this list of conditions and the following disclaimer in the documentation
	13	* and/or other materials provided with the distribution.
	14	* * Neither the name of SWITCH nor the names of its contributors may be
	15	* used to endorse or promote products derived from this software without
	16	* specific prior written permission.
	17	*
	18	* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
	19	* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	20	* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	21	* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
	22	* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
	23	* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
	24	* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
	25	* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
	26	* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
	27	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
	28	* POSSIBILITY OF SUCH DAMAGE.
	29	*
	30	* $Author: peter $
	31	*
	32	* $Id: inline.c 70 2006-05-17 08:38:01Z peter $
	33	*
	34	* $LastChangedRevision: 70 $
	35	*
	36	*/
	37
	38	static uint16_t Get_val16(void *p);
	39
	40	static uint32_t Get_val24(void *p);
	41
	42	static uint32_t Get_val32(void *p);
	43
	44	static uint64_t Get_val40(void *p);
	45
	46	static uint64_t Get_val48(void *p);
	47
	48	static uint64_t Get_val56(void *p);
	49
	50	static uint64_t Get_val64(void *p);
	51
	52	static uint16_t Get_val16(void *p) {
	53	uint8_t in = (uint8_t )p;
	54	type_mask_t mask;
	55
	56	mask.val.val8[0] = in[0];
	57	mask.val.val8[1] = in[1];
	58	return mask.val.val16[0];
	59
	60	} // End of Get_val16
	61
	62	static uint32_t Get_val24(void *p) {
	63	uint8_t in = (uint8_t )p;
	64	type_mask_t mask;
	65
	66	mask.val.val8[0] = 0;
	67	mask.val.val8[1] = in[0];
	68	mask.val.val8[2] = in[1];
	69	mask.val.val8[3] = in[2];
	70	return mask.val.val32[0];
	71
	72	} // End of Get_val24
	73
	74	static uint32_t Get_val32(void *p) {
	75	uint8_t in = (uint8_t )p;
	76	type_mask_t mask;
	77
	78	mask.val.val8[0] = in[0];
	79	mask.val.val8[1] = in[1];
	80	mask.val.val8[2] = in[2];
	81	mask.val.val8[3] = in[3];
	82	return mask.val.val32[0];
	83
	84	} // End of Get_val32
	85
	86	static uint64_t Get_val64(void *p) {
	87	uint8_t in = (uint8_t )p;
	88	type_mask_t mask;
	89
	90	mask.val.val8[0] = in[0];
	91	mask.val.val8[1] = in[1];
	92	mask.val.val8[2] = in[2];
	93	mask.val.val8[3] = in[3];
	94	mask.val.val8[4] = in[4];
	95	mask.val.val8[5] = in[5];
	96	mask.val.val8[6] = in[6];
	97	mask.val.val8[7] = in[7];
	98	return mask.val.val64;
	99
	100	} // End of Get_val64
	101
	102	static uint64_t Get_val40(void *p) {
	103	uint8_t in = (uint8_t )p;
	104	type_mask_t mask;
	105
	106	mask.val.val8[0] = 0;
	107	mask.val.val8[1] = 0;
	108	mask.val.val8[2] = 0;
	109	mask.val.val8[3] = in[0];
	110	mask.val.val8[4] = in[1];
	111	mask.val.val8[5] = in[2];
	112	mask.val.val8[6] = in[3];
	113	mask.val.val8[7] = in[4];
	114	return mask.val.val64;
	115
	116	} // End of Get_val40
	117
	118	static uint64_t Get_val48(void *p) {
	119	uint8_t in = (uint8_t )p;
	120	type_mask_t mask;
	121
	122	mask.val.val8[0] = 0;
	123	mask.val.val8[1] = 0;
	124	mask.val.val8[2] = in[0];
	125	mask.val.val8[3] = in[1];
	126	mask.val.val8[4] = in[2];
	127	mask.val.val8[5] = in[3];
	128	mask.val.val8[6] = in[4];
	129	mask.val.val8[7] = in[5];
	130	return mask.val.val64;
	131
	132	} // End of Get_val48
	133
	134	static uint64_t Get_val56(void *p) {
	135	uint8_t in = (uint8_t )p;
	136	type_mask_t mask;
	137
	138	mask.val.val8[0] = 0;
	139	mask.val.val8[1] = in[0];
	140	mask.val.val8[2] = in[1];
	141	mask.val.val8[3] = in[2];
	142	mask.val.val8[4] = in[3];
	143	mask.val.val8[5] = in[4];
	144	mask.val.val8[6] = in[5];
	145	mask.val.val8[7] = in[6];
	146	return mask.val.val64;
	147
	148	} // End of Get_val56
	149

+207

-0

ipconv.c less more

	0	/* Copyright (c) 1996 by Internet Software Consortium.
	1	*
	2	* Permission to use, copy, modify, and distribute this software for any
	3	* purpose with or without fee is hereby granted, provided that the above
	4	* copyright notice and this permission notice appear in all copies.
	5	*
	6	* THE SOFTWARE IS PROVIDED "AS IS" AND INTERNET SOFTWARE CONSORTIUM DISCLAIMS
	7	* ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES
	8	* OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL INTERNET SOFTWARE
	9	* CONSORTIUM BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL
	10	* DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR
	11	* PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS
	12	* ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS
	13	* SOFTWARE.
	14	*/
	15
	16	#include <sys/types.h>
	17	#include <sys/socket.h>
	18	#include <arpa/inet.h>
	19	#include <string.h>
	20
	21	#include "config.h"
	22
	23	#ifdef HAVE_STDINT_H
	24	#include <stdint.h>
	25	#endif
	26
	27	#include "nffile.h"
	28	#include "util.h"
	29	#include "ipconv.h"
	30
	31	static int parse_ipv4(const char src, uint32_t dst, int *bytes);
	32	static int parse_ipv6(const char src, uint64_t dst, int *bytes);
	33
	34	int parse_ip(int af, const char src, uint64_t dst, int bytes) {
	35	uint32_t v4addr;
	36	int ret;
	37
	38	if ( strchr(src, ':') != NULL )
	39	*af = PF_INET6;
	40	else
	41	*af = PF_INET;
	42
	43	switch (*af) {
	44	case AF_INET:
	45	ret = (parse_ipv4(src, &v4addr, bytes));
	46	dst[0] = 0;
	47	dst[1] = ntohl(v4addr) & 0xffffffffLL ;
	48	return ret;
	49	break;
	50	case AF_INET6:
	51	ret = (parse_ipv6(src, dst, bytes));
	52	dst[0] = ntohll(dst[0]);
	53	dst[1] = ntohll(dst[1]);
	54	return ret;
	55	break;
	56	}
	57	/* NOTREACHED */
	58
	59	return 0;
	60	}
	61
	62	static int parse_ipv4(const char src, uint32_t dst, int *bytes) {
	63	static const char digits[] = "0123456789";
	64	int saw_digit, ch;
	65	uint8_t tmp[4], *tp;
	66
	67	saw_digit = 0;
	68	*bytes = 0;
	69	*(tp = tmp) = 0;
	70	memset(tmp, 0, sizeof(tmp));
	71	while ((ch = *src++) != '\0') {
	72	const char *pch;
	73
	74	if ((pch = strchr(digits, ch)) != NULL) {
	75	unsigned int new = tp 10 + (pch - digits);
	76
	77	if (new > 255)
	78	return (0);
	79	if (! saw_digit) {
	80	if (++(*bytes) > 4)
	81	return (0);
	82	saw_digit = 1;
	83	}
	84	*tp = new;
	85	} else if (ch == '.' && saw_digit) {
	86	if (*bytes == 4)
	87	return (0);
	88	*++tp = 0;
	89	saw_digit = 0;
	90	if ( !(*src) )
	91	return 0;
	92	} else
	93	return (0);
	94	}
	95
	96	memcpy(dst, tmp, sizeof(tmp));
	97	return (1);
	98	}
	99
	100	static int parse_ipv6(const char src, uint64_t dst, int *bytes) {
	101	static const char xdigits_l[] = "0123456789abcdef",
	102	xdigits_u[] = "0123456789ABCDEF";
	103	uint8_t tmp[16], tp, endp, *colonp;
	104	const char xdigits, curtok;
	105	int ch, saw_xdigit;
	106	u_int val;
	107
	108	memset((tp = tmp), '\0', sizeof(tmp));
	109	endp = tp + sizeof(tmp);
	110	colonp = NULL;
	111	/* Leading :: requires some special handling. */
	112	if (*src == ':')
	113	if (*++src != ':')
	114	return (0);
	115	curtok = src;
	116	saw_xdigit = 0;
	117	val = 0;
	118	while ((ch = *src++) != '\0') {
	119	const char *pch;
	120
	121	if ((pch = strchr((xdigits = xdigits_l), ch)) == NULL)
	122	pch = strchr((xdigits = xdigits_u), ch);
	123	if (pch != NULL) {
	124	val <<= 4;
	125	val \|= (pch - xdigits);
	126	if (val > 0xffff)
	127	return (0);
	128	saw_xdigit = 1;
	129	continue;
	130	}
	131	if (ch == ':') {
	132	curtok = src;
	133	if (!saw_xdigit) {
	134	if (colonp)
	135	return (0);
	136	colonp = tp;
	137	continue;
	138	} else if (*src == '\0') {
	139	return (0);
	140	}
	141	if (tp + sizeof(uint16_t) > endp)
	142	return (0);
	143	*tp++ = (u_char) (val >> 8) & 0xff;
	144	*tp++ = (u_char) val & 0xff;
	145	saw_xdigit = 0;
	146	val = 0;
	147	continue;
	148	}
	149	if (ch == '.' && ((tp + 4) <= endp) &&
	150	parse_ipv4(curtok, (uint32_t *)tp, bytes) > 0) {
	151	tp += 4;
	152	saw_xdigit = 0;
	153	break; /* '\0' was seen by parse_ipv4(). */
	154	}
	155	return (0);
	156	}
	157	if (saw_xdigit) {
	158	if (tp + sizeof(uint16_t) > endp)
	159	return (0);
	160	*tp++ = (u_char) (val >> 8) & 0xff;
	161	*tp++ = (u_char) val & 0xff;
	162	}
	163	if (colonp != NULL) {
	164	/*
	165	* Since some memmove()'s erroneously fail to handle
	166	* overlapping regions, we'll do the shift by hand.
	167	*/
	168	const int n = tp - colonp;
	169	int i;
	170
	171	for (i = 1; i <= n; i++) {
	172	endp[- i] = colonp[n - i];
	173	colonp[n - i] = 0;
	174	}
	175	tp = endp;
	176	}
	177	*bytes = 16 - ( endp - tp );
	178
	179	memcpy(dst, tmp, sizeof(tmp));
	180	return (1);
	181	}
	182
	183	/*
	184	int main( int argc, char **argv ) {
	185
	186	char *s, t[64];
	187	uint64_t anyaddr[2];
	188	int af, ret, bytes;
	189
	190	s = argv[1];
	191	ret = parse_ip(&af, s, anyaddr, &bytes);
	192	if ( ret != 1 ) {
	193	printf("Parse failed!\n");
	194	exit(0);
	195	}
	196
	197	if ( af == PF_INET )
	198	inet_ntop(af, &(((uint32_t *)anyaddr)[3]), t, 64);
	199	else
	200	inet_ntop(af, anyaddr, t, 64);
	201
	202	printf("Convert back: %s => %s %i bytes\n", s, t, bytes);
	203
	204	}
	205
	206	*/

-0

ipconv.h less more

	0
	1	int parse_ip(int af, const char src, uint64_t dst, int bytes);
	2

-2

launch.c less more

27	27	*
28	28	* $Author: peter $
29	29	*
30		* $Id: launch.c 34 2005-08-22 12:01:31Z peter $
	30	* $Id: launch.c 70 2006-05-17 08:38:01Z peter $
31	31	*
32		* $LastChangedRevision: 34 $
	32	* $LastChangedRevision: 70 $
33	33	*
34	34	*
35	35	*/

46	46	#include <stdarg.h>
47	47	#include <errno.h>
48	48	#include <unistd.h>
	49
	50	#include "config.h"
	51
	52	#ifdef HAVE_STDINT_H
	53	#include <stdint.h>
	54	#endif
49	55
50	56	#include "launch.h"
51	57

-88

~~netflow_v5.h~~ less more

0		/*
1		* This file is part of the nfdump project.
2		*
3		* Copyright (c) 2004, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
4		* All rights reserved.
5		*
6		* Redistribution and use in source and binary forms, with or without
7		* modification, are permitted provided that the following conditions are met:
8		*
9		* * Redistributions of source code must retain the above copyright notice,
10		* this list of conditions and the following disclaimer.
11		* * Redistributions in binary form must reproduce the above copyright notice,
12		* this list of conditions and the following disclaimer in the documentation
13		* and/or other materials provided with the distribution.
14		* * Neither the name of SWITCH nor the names of its contributors may be
15		* used to endorse or promote products derived from this software without
16		* specific prior written permission.
17		*
18		* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
19		* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
20		* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
21		* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
22		* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
23		* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
24		* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
25		* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
26		* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
27		* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
28		* POSSIBILITY OF SUCH DAMAGE.
29		*
30		* $Author: peter $
31		*
32		* $Id: netflow_v5.h 34 2005-08-22 12:01:31Z peter $
33		*
34		* $LastChangedRevision: 34 $
35		*
36		*/
37
38		#define NETFLOW_V5_HEADER_LENGTH 24
39		#define NETFLOW_V5_RECORD_LENGTH 48
40		#define NETFLOW_V5_MAX_RECORDS 30
41
42		typedef struct netflow_v5_header {
43		uint16_t version;
44		uint16_t count;
45		uint32_t SysUptime;
46		uint32_t unix_secs;
47		uint32_t unix_nsecs;
48		uint32_t flow_sequence;
49		uint8_t engine_type;
50		uint8_t engine_id;
51		uint16_t reserved;
52		} netflow_v5_header_t;
53
54		typedef struct netflow_v5_record {
55		uint32_t srcaddr;
56		uint32_t dstaddr;
57		uint32_t nexthop;
58		uint16_t input;
59		uint16_t output;
60		uint32_t dPkts;
61		uint32_t dOctets;
62		uint32_t First;
63		uint32_t Last;
64		uint16_t srcport;
65		uint16_t dstport;
66		uint8_t pad1;
67		uint8_t tcp_flags;
68		uint8_t prot;
69		uint8_t tos;
70		uint16_t src_as;
71		uint16_t dst_as;
72		uint8_t src_mask;
73		uint8_t dst_mask;
74		uint16_t pad2;
75		} netflow_v5_record_t;
76
77		/* prototypes */
78
79		void netflow_v5_header_to_string(void header, char *s);
80
81		void netflow_v5_record_to_block(void record, char *s);
82
83		void netflow_v5_record_to_line(void record, char *s);
84
85		void netflow_v5_record_to_line_long(void record, char *s);
86
87		void netflow_v5_record_to_pipe(void record, char * s);

+386

-0

netflow_v5_v7.c less more

	0	/*
	1	* This file is part of the nfdump project.
	2	*
	3	* Copyright (c) 2004, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
	4	* All rights reserved.
	5	*
	6	* Redistribution and use in source and binary forms, with or without
	7	* modification, are permitted provided that the following conditions are met:
	8	*
	9	* * Redistributions of source code must retain the above copyright notice,
	10	* this list of conditions and the following disclaimer.
	11	* * Redistributions in binary form must reproduce the above copyright notice,
	12	* this list of conditions and the following disclaimer in the documentation
	13	* and/or other materials provided with the distribution.
	14	* * Neither the name of SWITCH nor the names of its contributors may be
	15	* used to endorse or promote products derived from this software without
	16	* specific prior written permission.
	17	*
	18	* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
	19	* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	20	* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	21	* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
	22	* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
	23	* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
	24	* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
	25	* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
	26	* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
	27	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
	28	* POSSIBILITY OF SUCH DAMAGE.
	29	*
	30	* $Author: peter $
	31	*
	32	* $Id: netflow_v5_v7.c 70 2006-05-17 08:38:01Z peter $
	33	*
	34	* $LastChangedRevision: 70 $
	35	*
	36	*/
	37
	38	#include <stdio.h>
	39	#include <sys/types.h>
	40	#include <sys/socket.h>
	41	#include <unistd.h>
	42	#include <stdlib.h>
	43	#include <syslog.h>
	44	#include <netinet/in.h>
	45
	46	#include "config.h"
	47
	48	#ifdef HAVE_STDINT_H
	49	#include <stdint.h>
	50	#endif
	51
	52	#include "nffile.h"
	53	#include "nfnet.h"
	54	#include "nf_common.h"
	55	#include "netflow_v5_v7.h"
	56
	57	extern int verbose;
	58
	59	/* module limited globals */
	60	static int64_t last_sequence, sequence, distance, last_count;
	61
	62	static int first;
	63
	64	// for sending netflow v5
	65	static netflow_v5_header_t *v5_output_header;
	66	static netflow_v5_record_t *v5_output_record;
	67
	68	typedef struct v5_block_s {
	69	uint32_t srcaddr;
	70	uint32_t dstaddr;
	71	uint32_t dPkts;
	72	uint32_t dOctets;
	73	uint8_t data[4]; // link to next record
	74	} v5_block_t;
	75
	76	/* functions */
	77	void Init_v5_v7_input(void) {
	78	first = 1;
	79	} // End of Init_v5_input
	80
	81	/*
	82	* functions used for receiving netflow v5 records
	83	*/
	84
	85	void Process_v5_v7(void in_buff, ssize_t in_buff_cnt, data_block_header_t data_header, void writeto,
	86	stat_record_t stat_record, uint64_t first_seen, uint64_t *last_seen) {
	87
	88	netflow_v5_header_t *v5_header;
	89	netflow_v5_record_t *v5_record;
	90	common_record_t *common_record;
	91	v5_block_t *v5_block;
	92	uint64_t start_time, end_time, boot_time;
	93	uint32_t First, Last;
	94	uint16_t count;
	95	int i, done, version, record_length, output_record_length;
	96	ssize_t size_left;
	97	pointer_addr_t bsize;
	98	char *string;
	99
	100	/* Init
	101	* v7 is treated as v5. It differes only in the record length, for what we process.
	102	*/
	103
	104	// map v5 data structure to input buffer
	105	v5_header = (netflow_v5_header_t *)in_buff;
	106
	107	// map file record to output buffer
	108	common_record = (common_record_t *)writeto;
	109	v5_block = (v5_block_t *)common_record->data;
	110
	111	// sanity check for buffer size
	112	bsize = (pointer_addr_t)writeto - (pointer_addr_t)data_header;
	113	// The save margin is a full data record. The master record is a bit more
	114	// as no record will use more space than this master_record
	115	if ( bsize > (BUFFSIZE-sizeof(master_record_t)) ) {
	116	syslog(LOG_WARNING,"Process_v5: Outputbuffer full. Flush buffer but have to skip records.");
	117	return writeto;
	118	}
	119
	120	// common size + 2 * 32bit ip addresses + 2 * 32bit counters
	121	output_record_length = sizeof(common_record_t) + sizeof(v5_block_t) - 2 * sizeof(uint8_t[4]);
	122
	123
	124	version = ntohs(v5_header->version);
	125	record_length = version == 5 ? NETFLOW_V5_RECORD_LENGTH : NETFLOW_V7_RECORD_LENGTH;
	126
	127	// this many data to process
	128	size_left = in_buff_cnt;
	129
	130	done = 0;
	131	while ( !done ) {
	132	/* Process header */
	133
	134	// count and buffer size check
	135	count = ntohs(v5_header->count);
	136	if ( count > NETFLOW_V5_MAX_RECORDS ) {
	137	syslog(LOG_ERR,"Process_v5: Unexpected record count in header: %i. Abort v5/v7 record processing", count);
	138	return writeto;
	139	}
	140	if ( size_left < ( NETFLOW_V5_HEADER_LENGTH + count * record_length) ) {
	141	syslog(LOG_ERR,"Process_v5: Not enough data to process v5 record. Abort v5/v7 record processing");
	142	return writeto;
	143	}
	144
	145	// output buffer size check
	146	if ( (data_header->size + count * output_record_length) > OUTPUT_BUFF_SIZE ) {
	147	// this should really never occur, because the buffer gets flushed ealier
	148	syslog(LOG_ERR,"Process_v5: output buffer size error. Abort v5/v7 record processing");
	149	return writeto;
	150	}
	151
	152	// sequence check
	153	if ( first ) {
	154	last_sequence = ntohl(v5_header->flow_sequence);
	155	sequence = last_sequence;
	156	first = 0;
	157	} else {
	158	last_sequence = sequence;
	159	sequence = ntohl(v5_header->flow_sequence);
	160	distance = sequence - last_sequence;
	161	// handle overflow
	162	if (distance < 0) {
	163	distance = 0xffffffff + distance +1;
	164	}
	165	if (distance != last_count) {
	166	#define delta(a,b) ( (a)>(b) ? (a)-(b) : (b)-(a) )
	167	stat_record->sequence_failure++;
	168	/*
	169	syslog(LOG_ERR,"Flow sequence mismatch. Missing: %lli flows", delta(last_count,distance));
	170	syslog(LOG_ERR,"sequence %llu. last sequence: %lli", sequence, last_sequence);
	171	if ( report_seq )
	172	syslog(LOG_ERR,"Flow sequence mismatch. Missing: %lli flows", delta(last_count,distance));
	173	*/
	174	}
	175	}
	176	last_count = count;
	177
	178	v5_header->SysUptime = ntohl(v5_header->SysUptime);
	179	v5_header->unix_secs = ntohl(v5_header->unix_secs);
	180	v5_header->unix_nsecs = ntohl(v5_header->unix_nsecs);
	181
	182	/* calculate boot time in msec */
	183	boot_time = ((uint64_t)(v5_header->unix_secs)*1000 +
	184	((uint64_t)(v5_header->unix_nsecs) / 1000000) ) - (uint64_t)(v5_header->SysUptime);
	185
	186	// process all records
	187	v5_record = (netflow_v5_record_t *)((pointer_addr_t)v5_header + NETFLOW_V5_HEADER_LENGTH);
	188
	189	/* loop over each records associated with this header */
	190	for (i = 0; i < count; i++) {
	191	common_record->srcport = ntohs(v5_record->srcport);
	192	common_record->dstport = ntohs(v5_record->dstport);
	193	common_record->input = ntohs(v5_record->input);
	194	common_record->output = ntohs(v5_record->output);
	195	common_record->srcas = ntohs(v5_record->src_as);
	196	common_record->dstas = ntohs(v5_record->dst_as);
	197	common_record->tcp_flags= v5_record->tcp_flags;
	198	common_record->prot = v5_record->prot;
	199	common_record->tos = v5_record->tos;
	200	common_record->dir = 0;
	201	common_record->mark = 0;
	202	common_record->size = output_record_length;
	203
	204	v5_block->srcaddr = ntohl(v5_record->srcaddr);
	205	v5_block->dstaddr = ntohl(v5_record->dstaddr);
	206	v5_block->dPkts = ntohl(v5_record->dPkts);
	207	v5_block->dOctets = ntohl(v5_record->dOctets);
	208
	209	// Time issues
	210	First = ntohl(v5_record->First);
	211	Last = ntohl(v5_record->Last);
	212	if ( First > Last )
	213	/* Last in msec, in case of msec overflow, between start and end */
	214	end_time = 0x100000000LL + Last + boot_time;
	215	else
	216	end_time = (uint64_t)Last + boot_time;
	217
	218	/* start time in msecs */
	219	start_time = (uint64_t)First + boot_time;
	220
	221	common_record->first = start_time/1000;
	222	common_record->msec_first = start_time - common_record->first*1000;
	223
	224	common_record->last = end_time/1000;
	225	common_record->msec_last = end_time - common_record->last*1000;
	226
	227	// update first_seen, last_seen
	228	if ( start_time < *first_seen )
	229	*first_seen = start_time;
	230	if ( end_time > *last_seen )
	231	*last_seen = end_time;
	232
	233
	234	// Update stats
	235	switch (common_record->prot) {
	236	case 1:
	237	stat_record->numflows_icmp++;
	238	stat_record->numpackets_icmp += v5_block->dPkts;
	239	stat_record->numbytes_icmp += v5_block->dOctets;
	240	break;
	241	case 6:
	242	stat_record->numflows_tcp++;
	243	stat_record->numpackets_tcp += v5_block->dPkts;
	244	stat_record->numbytes_tcp += v5_block->dOctets;
	245	break;
	246	case 17:
	247	stat_record->numflows_udp++;
	248	stat_record->numpackets_udp += v5_block->dPkts;
	249	stat_record->numbytes_udp += v5_block->dOctets;
	250	break;
	251	default:
	252	stat_record->numflows_other++;
	253	stat_record->numpackets_other += v5_block->dPkts;
	254	stat_record->numbytes_other += v5_block->dOctets;
	255	}
	256	stat_record->numflows++;
	257	stat_record->numpackets += v5_block->dPkts;
	258	stat_record->numbytes += v5_block->dOctets;
	259
	260	if ( verbose ) {
	261	master_record_t master_record;
	262	ExpandRecord((common_record_t *)common_record, &master_record);
	263	format_file_block_record(&master_record, 1, &string, 0);
	264	printf("%s\n", string);
	265	}
	266
	267	v5_record = (netflow_v5_record_t *)((pointer_addr_t)v5_record + record_length);
	268	common_record = (common_record_t *)v5_block->data;
	269	v5_block = (v5_block_t *)common_record->data;
	270
	271	// buffer size sanity check
	272	bsize = (pointer_addr_t)common_record - (pointer_addr_t)data_header;
	273	if ( bsize >= OUTPUT_BUFF_SIZE ) {
	274	syslog(LOG_ERR,"Process_v5: Output buffer overflow! Flush buffer and skip records.");
	275	return (void *)common_record;
	276	}
	277
	278	} // End of foreach v5 record
	279
	280	// update file record size ( -> output buffer size )
	281	data_header->NumBlocks += count;
	282	data_header->size += count * output_record_length;
	283
	284	// still to go for
	285	size_left -= NETFLOW_V5_HEADER_LENGTH + count * record_length;
	286
	287	// next header
	288	v5_header = (netflow_v5_header_t *)v5_record;
	289
	290	done = size_left <= 0;
	291
	292	} // End of while !done
	293
	294	return (void *)common_record;
	295
	296	} /* End of Process_v5 */
	297
	298	/*
	299	* functions used for sending netflow v5 records
	300	*/
	301	void Init_v5_v7_output(send_peer_t *peer) {
	302
	303	v5_output_header = (netflow_v5_header_t *)peer->send_buffer;
	304	v5_output_header->version = htons(5);
	305	v5_output_header->SysUptime = 0;
	306	v5_output_header->unix_secs = 0;
	307	v5_output_header->unix_nsecs = 0;
	308	v5_output_header->count = 0;
	309	first = 1;
	310
	311	sequence = 0;
	312	last_sequence = 0;
	313	last_count = 0;
	314	v5_output_record = (netflow_v5_record_t *)((pointer_addr_t)v5_output_header + (pointer_addr_t)sizeof(netflow_v5_header_t));
	315
	316	} // End of Init_v5_v7_output
	317
	318	int Add_v5_output_record(master_record_t master_record, send_peer_t peer) {
	319	static uint64_t boot_time; // in msec
	320	static int cnt;
	321	uint32_t t1, t2;
	322	// char *s;
	323
	324	// Skip IPv6 records
	325	if ( (master_record->flags & FLAG_IPV6_ADDR ) != 0 )
	326	return 0;
	327
	328	//format_file_block_record(master_record, 1, &s, 0);
	329	//printf("%s\n", s);
	330
	331	if ( first ) { // first time a record is added
	332	// boot time is set one day back - assuming that the start time of every flow does not start ealier
	333	boot_time = (uint64_t)(master_record->first - 86400)*1000;
	334	v5_output_header->unix_secs = htonl(master_record->first - 86400);
	335	cnt = 0;
	336	first = 0;
	337	}
	338	if ( cnt == 0 ) {
	339	peer->writeto = (void *)((pointer_addr_t)peer->send_buffer + NETFLOW_V5_HEADER_LENGTH);
	340	v5_output_record = (netflow_v5_record_t *)((pointer_addr_t)v5_output_header + (pointer_addr_t)sizeof(netflow_v5_header_t));
	341	sequence = last_sequence + last_count;
	342	v5_output_header->flow_sequence = htonl(sequence);
	343	last_sequence = sequence;
	344	}
	345	v5_output_record->srcaddr = htonl(master_record->v4.srcaddr);
	346	v5_output_record->dstaddr = htonl(master_record->v4.dstaddr);
	347	v5_output_record->input = htons(master_record->input);
	348	v5_output_record->output = htons(master_record->output);
	349
	350	// the 64bit counters are cut down to 32 bits for v5
	351	v5_output_record->dPkts = htonl((uint32_t)master_record->dPkts);
	352	v5_output_record->dOctets = htonl((uint32_t)master_record->dOctets);
	353
	354	t1 = (uint32_t)(1000LL * (uint64_t)master_record->first + (uint64_t)master_record->msec_first - boot_time);
	355	t2 = (uint32_t)(1000LL * (uint64_t)master_record->last + (uint64_t)master_record->msec_last - boot_time);
	356	v5_output_record->First = htonl(t1);
	357	v5_output_record->Last = htonl(t2);
	358
	359	v5_output_record->srcport = htons(master_record->srcport);
	360	v5_output_record->dstport = htons(master_record->dstport);
	361	v5_output_record->src_as = htons(master_record->srcas);
	362	v5_output_record->dst_as = htons(master_record->dstas);
	363	v5_output_record->tcp_flags = master_record->tcp_flags;
	364	v5_output_record->prot = master_record->prot;
	365	v5_output_record->tos = master_record->tos;
	366	v5_output_record->src_mask = 0;
	367	v5_output_record->dst_mask = 0;
	368	v5_output_record->pad1 = 0;
	369	v5_output_record->pad2 = 0;
	370	v5_output_record->nexthop = 0;
	371
	372	cnt++;
	373
	374	v5_output_header->count = htons(cnt);
	375	peer->writeto = (void *)((pointer_addr_t)peer->writeto + NETFLOW_V5_RECORD_LENGTH);
	376	v5_output_record++;
	377	if ( cnt == NETFLOW_V5_MAX_RECORDS ) {
	378	peer->flush = 1;
	379	last_count = cnt;
	380	cnt = 0;
	381	}
	382
	383	return 0;
	384
	385	} // End of Add_v5_output_record

+127

-0

netflow_v5_v7.h less more

	0	/*
	1	* This file is part of the nfdump project.
	2	*
	3	* Copyright (c) 2004, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
	4	* All rights reserved.
	5	*
	6	* Redistribution and use in source and binary forms, with or without
	7	* modification, are permitted provided that the following conditions are met:
	8	*
	9	* * Redistributions of source code must retain the above copyright notice,
	10	* this list of conditions and the following disclaimer.
	11	* * Redistributions in binary form must reproduce the above copyright notice,
	12	* this list of conditions and the following disclaimer in the documentation
	13	* and/or other materials provided with the distribution.
	14	* * Neither the name of SWITCH nor the names of its contributors may be
	15	* used to endorse or promote products derived from this software without
	16	* specific prior written permission.
	17	*
	18	* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
	19	* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	20	* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	21	* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
	22	* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
	23	* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
	24	* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
	25	* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
	26	* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
	27	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
	28	* POSSIBILITY OF SUCH DAMAGE.
	29	*
	30	* $Author: peter $
	31	*
	32	* $Id: netflow_v5_v7.h 55 2006-01-13 10:04:34Z peter $
	33	*
	34	* $LastChangedRevision: 55 $
	35	*
	36	*/
	37
	38	#define NETFLOW_V5_HEADER_LENGTH 24
	39	#define NETFLOW_V5_RECORD_LENGTH 48
	40	#define NETFLOW_V5_MAX_RECORDS 30
	41
	42	#define NETFLOW_V7_HEADER_LENGTH 24
	43	#define NETFLOW_V7_RECORD_LENGTH 52
	44	#define NETFLOW_V7_MAX_RECORDS 28
	45
	46	/* v5 structures */
	47	typedef struct netflow_v5_header {
	48	uint16_t version;
	49	uint16_t count;
	50	uint32_t SysUptime;
	51	uint32_t unix_secs;
	52	uint32_t unix_nsecs;
	53	uint32_t flow_sequence;
	54	uint8_t engine_type;
	55	uint8_t engine_id;
	56	uint16_t reserved;
	57	} netflow_v5_header_t;
	58
	59	typedef struct netflow_v5_record {
	60	uint32_t srcaddr;
	61	uint32_t dstaddr;
	62	uint32_t nexthop;
	63	uint16_t input;
	64	uint16_t output;
	65	uint32_t dPkts;
	66	uint32_t dOctets;
	67	uint32_t First;
	68	uint32_t Last;
	69	uint16_t srcport;
	70	uint16_t dstport;
	71	uint8_t pad1;
	72	uint8_t tcp_flags;
	73	uint8_t prot;
	74	uint8_t tos;
	75	uint16_t src_as;
	76	uint16_t dst_as;
	77	uint8_t src_mask;
	78	uint8_t dst_mask;
	79	uint16_t pad2;
	80	} netflow_v5_record_t;
	81
	82
	83	/* v7 structures */
	84	typedef struct netflow_v7_header {
	85	uint16_t version;
	86	uint16_t count;
	87	uint32_t SysUptime;
	88	uint32_t unix_secs;
	89	uint32_t unix_nsecs;
	90	uint32_t flow_sequence;
	91	uint32_t reserved;
	92	} netflow_v7_header_t;
	93
	94	typedef struct netflow_v7_record {
	95	uint32_t srcaddr;
	96	uint32_t dstaddr;
	97	uint32_t nexthop;
	98	uint16_t input;
	99	uint16_t output;
	100	uint32_t dPkts;
	101	uint32_t dOctets;
	102	uint32_t First;
	103	uint32_t Last;
	104	uint16_t srcport;
	105	uint16_t dstport;
	106	uint8_t flags;
	107	uint8_t tcp_flags;
	108	uint8_t prot;
	109	uint8_t tos;
	110	uint16_t src_as;
	111	uint16_t dst_as;
	112	uint8_t src_mask;
	113	uint8_t dst_mask;
	114	uint16_t pad;
	115	uint32_t router_sc;
	116	} netflow_v7_record_t;
	117
	118	/* prototypes */
	119	void Init_v5_v7_input(void);
	120
	121	void Process_v5_v7(void in_buff, ssize_t in_buff_cnt, data_block_header_t data_header, void writeto,
	122	stat_record_t stat_record, uint64_t first_seen, uint64_t *last_seen);
	123
	124	void Init_v5_v7_output(send_peer_t *peer);
	125
	126	int Add_v5_output_record(master_record_t master_record, send_peer_t peer);

-75

~~netflow_v7.h~~ less more

0		/*
1		* This file is part of the nfdump project.
2		*
3		* Copyright (c) 2004, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
4		* All rights reserved.
5		*
6		* Redistribution and use in source and binary forms, with or without
7		* modification, are permitted provided that the following conditions are met:
8		*
9		* * Redistributions of source code must retain the above copyright notice,
10		* this list of conditions and the following disclaimer.
11		* * Redistributions in binary form must reproduce the above copyright notice,
12		* this list of conditions and the following disclaimer in the documentation
13		* and/or other materials provided with the distribution.
14		* * Neither the name of SWITCH nor the names of its contributors may be
15		* used to endorse or promote products derived from this software without
16		* specific prior written permission.
17		*
18		* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
19		* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
20		* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
21		* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
22		* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
23		* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
24		* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
25		* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
26		* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
27		* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
28		* POSSIBILITY OF SUCH DAMAGE.
29		*
30		* $Author: peter $
31		*
32		* $Id: netflow_v7.h 34 2005-08-22 12:01:31Z peter $
33		*
34		* $LastChangedRevision: 34 $
35		*
36		*/
37
38		#define NETFLOW_V7_HEADER_LENGTH 24
39		#define NETFLOW_V7_RECORD_LENGTH 52
40		#define NETFLOW_V7_MAX_RECORDS 28
41
42		typedef struct netflow_v7_header {
43		uint16_t version;
44		uint16_t count;
45		uint32_t SysUptime;
46		uint32_t unix_secs;
47		uint32_t unix_nsecs;
48		uint32_t flow_sequence;
49		uint32_t reserved;
50		} netflow_v7_header_t;
51
52		typedef struct netflow_v7_record {
53		uint32_t srcaddr;
54		uint32_t dstaddr;
55		uint32_t nexthop;
56		uint16_t input;
57		uint16_t output;
58		uint32_t dPkts;
59		uint32_t dOctets;
60		uint32_t First;
61		uint32_t Last;
62		uint16_t srcport;
63		uint16_t dstport;
64		uint8_t flags;
65		uint8_t tcp_flags;
66		uint8_t prot;
67		uint8_t tos;
68		uint16_t src_as;
69		uint16_t dst_as;
70		uint8_t src_mask;
71		uint8_t dst_mask;
72		uint16_t pad;
73		uint32_t router_sc;
74		} netflow_v7_record_t;

+1267

-0

netflow_v9.c less more

	0	/*
	1	* This file is part of the nfdump project.
	2	*
	3	* Copyright (c) 2004, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
	4	* All rights reserved.
	5	*
	6	* Redistribution and use in source and binary forms, with or without
	7	* modification, are permitted provided that the following conditions are met:
	8	*
	9	* * Redistributions of source code must retain the above copyright notice,
	10	* this list of conditions and the following disclaimer.
	11	* * Redistributions in binary form must reproduce the above copyright notice,
	12	* this list of conditions and the following disclaimer in the documentation
	13	* and/or other materials provided with the distribution.
	14	* * Neither the name of SWITCH nor the names of its contributors may be
	15	* used to endorse or promote products derived from this software without
	16	* specific prior written permission.
	17	*
	18	* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
	19	* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	20	* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	21	* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
	22	* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
	23	* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
	24	* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
	25	* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
	26	* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
	27	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
	28	* POSSIBILITY OF SUCH DAMAGE.
	29	*
	30	* $Author: peter $
	31	*
	32	* $Id: netflow_v9.c 70 2006-05-17 08:38:01Z peter $
	33	*
	34	* $LastChangedRevision: 70 $
	35	*
	36	*/
	37
	38	#include <stdio.h>
	39	#include <sys/types.h>
	40	#include <sys/socket.h>
	41	#include <unistd.h>
	42	#include <stdlib.h>
	43	#include <syslog.h>
	44	#include <string.h>
	45	#include <errno.h>
	46	#include <time.h>
	47	#include <netinet/in.h>
	48
	49	#include "config.h"
	50
	51	#ifdef HAVE_STDINT_H
	52	#include <stdint.h>
	53	#endif
	54
	55	#include "nffile.h"
	56	#include "nfnet.h"
	57	#include "nf_common.h"
	58	#include "util.h"
	59	#include "netflow_v5_v7.h"
	60	#include "netflow_v9.h"
	61
	62
	63	#include "inline.c"
	64
	65	extern int verbose;
	66
	67	typedef struct translation_element_s {
	68	uint16_t input_offset;
	69	uint16_t output_offset;
	70	uint16_t length;
	71	} translation_element_t;
	72
	73	typedef struct input_translation_s {
	74	struct input_translation_s *next;
	75	uint32_t flags;
	76	time_t updated;
	77	uint32_t id;
	78	uint32_t input_record_size;
	79	uint32_t output_record_size;
	80	uint32_t input_index;
	81	uint32_t zero_index;
	82	uint32_t packet_offset;
	83	uint32_t byte_offset;
	84	translation_element_t element[NumElements];
	85	} input_translation_t;
	86
	87	typedef struct exporter_domain_s {
	88	struct exporter_domain_s *next;
	89	// identifier
	90	uint32_t exporter_id;
	91	// exporter parameters
	92	uint64_t boot_time;
	93	// sequence
	94	int64_t last_sequence;
	95	int64_t sequence;
	96	int first;
	97	input_translation_t *input_translation_table;
	98	input_translation_t *current_table;
	99	} exporter_domain_t;
	100
	101	/* module limited globals */
	102	static struct element_info_s {
	103	uint16_t min;
	104	uint16_t max;
	105	} element_info[128] = {
	106	{ 0, 0 }, // 0 - empty
	107	{ 4, 8 }, // 1 - NF9_IN_BYTES
	108	{ 4, 8 }, // 2 - NF9_IN_PACKETS
	109	{ 4, 8 }, // 3 - NF9_FLOWS
	110	{ 1, 1 }, // 4 - NF9_IN_PROTOCOL
	111	{ 1, 1 }, // 5 - NF9_SRC_TOS
	112	{ 1, 1 }, // 6 - NF9_TCP_FLAGS
	113	{ 2, 2 }, // 7 - NF9_L4_SRC_PORT
	114	{ 4, 4 }, // 8 - NF9_IPV4_SRC_ADDR
	115	{ 2, 2 }, // 9 - NF9_SRC_MASK
	116	{ 2, 2 }, // 10 - NF9_INPUT_SNMP
	117	{ 2, 2 }, // 11 - NF9_L4_DST_PORT
	118	{ 4, 4 }, // 12 - NF9_IPV4_DST_ADDR
	119	{ 2, 2 }, // 13 - NF9_DST_MASK
	120	{ 2, 2 }, // 14 - NF9_OUTPUT_SNMP
	121	{ 4, 4 }, // 15 - NF9_IPV4_NEXT_HOP
	122	{ 2, 2 }, // 16 - NF9_SRC_AS
	123	{ 2, 2 }, // 17 - NF9_DST_AS
	124
	125	{ 0, 0 }, { 0, 0 }, { 0, 0 }, // 18 - 20 not implemented
	126
	127	{ 4, 4 }, // 21 - NF9_LAST_SWITCHED
	128	{ 4, 4 }, // 22 - NF9_FIRST_SWITCHED
	129	{ 4, 8 }, // 23 - NF9_OUT_BYTES
	130	{ 4, 8 }, // 24 - NF9_OUT_PKTS
	131
	132	{ 0, 0 }, { 0, 0 }, // 25 - 26 not implemented
	133
	134	{ 16, 16 }, // 27 - NF9_IPV6_SRC_ADDR
	135	{ 16, 16 }, // 28 - NF9_IPV6_DST_ADDR
	136	{ 4, 4 }, // 29 - NF9_IPV6_SRC_MASK
	137	{ 4, 4 }, // 30 - NF9_IPV6_DST_MASK
	138	{ 4, 4 }, // 31 - NF9_IPV6_FLOW_LABEL
	139	{ 4, 4 }, // 32 - NF9_ICMP_TYPE
	140
	141	{ 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, // 33 - 37 not implemented
	142
	143	{ 4, 4 }, // 38 - NF9_ENGINE_TYPE
	144	{ 4, 4 }, // 39 - NF9_ENGINE_ID
	145
	146	// 40 - 47 not implemented
	147	{ 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 },
	148	// 48 - 55 not implemented
	149	{ 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 },
	150	// 56 - 60 not implemented
	151	{ 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 },
	152
	153	{ 1, 1 }, // 61 - NF9_DIRECTION
	154
	155	// 62 - 63 not implemented
	156	{ 0, 0 }, { 0, 0 },
	157	// 64 - 71 not implemented
	158	{ 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 },
	159	// 72 - 79 not implemented
	160	{ 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 },
	161	// 80 - 87 not implemented
	162	{ 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 },
	163	// 88 - 95 not implemented
	164	{ 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 },
	165	// 96 - 103 not implemented
	166	{ 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 },
	167	// 104 - 111 not implemented
	168	{ 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 },
	169	// 112 - 119 not implemented
	170	{ 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 },
	171	// 120 - 127 not implemented
	172	{ 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }, { 0, 0 }
	173
	174	};
	175
	176
	177	typedef struct output_templates_s {
	178	struct output_templates_s *next;
	179	uint32_t flags;
	180	time_t time_sent;
	181	uint32_t record_length; // length of the data record resulting from this template
	182	uint32_t flowset_length; // length of the flowset record
	183	template_flowset_t *template_flowset;
	184	} output_template_t;
	185
	186	#define MAX_LIFETIME 60
	187
	188	static output_template_t *output_templates;
	189	static uint16_t template_id;
	190
	191	static uint32_t processed_records;
	192	static exporter_domain_t *exporter;
	193
	194	/* local function prototypes */
	195	static inline uint16_t CheckElementLength(int element, uint16_t in_length);
	196
	197	static inline void FillElement(input_translation_t table, int element, uint32_t offset);
	198
	199	static inline void Process_v9_templates(exporter_domain_t exporter, template_flowset_t template_flowset);
	200
	201	static inline void Process_v9_data(exporter_domain_t exporter, data_flowset_t data_flowset, data_block_header_t data_header,
	202	void writeto, stat_record_t stat_record, uint64_t first_seen, uint64_t last_seen);
	203
	204	static inline exporter_domain_t *GetExporter(uint32_t exporter_id);
	205
	206	static inline input_translation_t GetTranslationTable(exporter_domain_t exporter, uint16_t id);
	207
	208	static void setup_translation_table (exporter_domain_t *exporter, uint16_t id, uint16_t input_record_size);
	209
	210	static input_translation_t add_translation_table(exporter_domain_t exporter, uint16_t id);
	211
	212	static output_template_t *GetOutputTemplate(uint32_t flags);
	213
	214	static uint16_t Get_val16(void *p);
	215
	216	static uint32_t Get_val32(void *p);
	217
	218	static uint64_t Get_val64(void *p);
	219
	220	/* local variables */
	221
	222	static struct input_table_s {
	223	uint16_t offset;
	224	uint16_t length;
	225	} input_template[128];
	226
	227	// for sending netflow v9
	228	static netflow_v9_header_t *v9_output_header;
	229
	230	/* functions */
	231
	232
	233	void Init_v9(void) {
	234	exporter = NULL;
	235	output_templates = NULL;
	236	template_id = NF9_MIN_RECORD_FLOWSET_ID;
	237	} // End of Init_v9
	238
	239	static inline exporter_domain_t *GetExporter(uint32_t exporter_id) {
	240	exporter_domain_t **e;
	241
	242	e = &exporter;
	243	while ( *e ) {
	244	if ( (*e)->exporter_id == exporter_id )
	245	return *e;
	246	e = &((*e)->next);
	247	}
	248
	249	syslog(LOG_INFO, "Process_v9: New exporter domain %u\n", exporter_id);
	250
	251	// nothing found
	252	e = (exporter_domain_t )malloc(sizeof(exporter_domain_t));
	253	if ( !(*e)) {
	254	syslog(LOG_ERR, "Process_v9: Panic! %s line %d: %s\n", __FILE__, __LINE__, strerror (errno));
	255	return NULL;
	256	}
	257	memset((void )(e), 0, sizeof(exporter_domain_t));
	258	(*e)->exporter_id = exporter_id;
	259	(*e)->first = 1;
	260	(*e)->next = NULL;
	261	return (*e);
	262
	263	} // End of GetExporter
	264
	265	static inline input_translation_t GetTranslationTable(exporter_domain_t exporter, uint16_t id) {
	266	input_translation_t *table;
	267
	268	if ( exporter->current_table && ( exporter->current_table->id == id ) )
	269	return exporter->current_table;
	270
	271	table = exporter->input_translation_table;
	272	while ( table ) {
	273	if ( table->id == id ) {
	274	exporter->current_table = table;
	275	return table;
	276	}
	277
	278	table = table->next;
	279	}
	280
	281	// printf("[%u] Get translation table %u: %s\n", exporter->exporter_id, id, table == NULL ? "not found" : "found");
	282
	283	exporter->current_table = table;
	284	return table;
	285
	286	} // End of GetTranslationTable
	287
	288	static input_translation_t add_translation_table(exporter_domain_t exporter, uint16_t id) {
	289	input_translation_t **table;
	290
	291	table = &(exporter->input_translation_table);
	292	while ( *table ) {
	293	table = &((*table)->next);
	294	}
	295	*table = malloc(sizeof(input_translation_t));
	296	if ( !(*table) ) {
	297	syslog(LOG_ERR, "Process_v9: Panic! %s line %d: %s\n", __FILE__, __LINE__, strerror (errno));
	298	return NULL;
	299	}
	300	(*table)->id = id;
	301	(*table)->next = NULL;
	302
	303	// printf("[%u] Get new translation table %u\n", exporter->exporter_id, id);
	304	return *table;
	305
	306	} // End of add_translation_table
	307
	308	static inline uint16_t CheckElementLength(int element, uint16_t in_length) {
	309
	310	if ( in_length == element_info[element].min )
	311	return element_info[element].min;
	312	if ( in_length == element_info[element].max )
	313	return element_info[element].max;
	314	if ( (in_length > element_info[element].min) && (in_length < element_info[element].max) )
	315	return element_info[element].max;
	316
	317	return 0;
	318
	319	} // End of CheckElementLength
	320
	321
	322	inline void FillElement(input_translation_t table, int element, uint32_t offset) {
	323	uint16_t output_length;
	324	uint32_t input_index = table->input_index;
	325	uint32_t zero_index = table->zero_index;
	326
	327	output_length = CheckElementLength(element, input_template[element].length);
	328	if ( output_length ) {
	329	/*
	330	printf("Index: %u Elem %i, IO %u, OO %u, len: %u\n",
	331	input_index, element, input_template[element].offset, *offset, input_template[element].length);
	332	*/
	333	table->element[input_index].output_offset = *offset;
	334	table->element[input_index].input_offset = input_template[element].offset;
	335	table->element[input_index].length = input_template[element].length;
	336	table->input_index++;
	337	(*offset) += output_length;
	338	} else {
	339	/*
	340	printf("Zero: %u, Elem: %i, OO %u, len: %u\n",
	341	zero_index, element, *offset, element_info[element].min);
	342	*/
	343	table->element[zero_index].output_offset = *offset;
	344	table->element[zero_index].length = element_info[element].min;
	345	table->zero_index--;
	346	(*offset) += element_info[element].min;
	347	}
	348
	349	} // End of FillElement
	350
	351	static void setup_translation_table (exporter_domain_t *exporter, uint16_t id, uint16_t input_record_size) {
	352	input_translation_t *table;
	353	uint32_t offset;
	354
	355	table = GetTranslationTable(exporter, id);
	356	if ( !table ) {
	357	syslog(LOG_INFO, "Process_v9: [%u] Add template %u\n", exporter->exporter_id, id);
	358	table = add_translation_table(exporter, id);
	359	} else
	360	// very noisy with somee exporters
	361	// syslog(LOG_DEBUG, "Process_v9: [%u] Refresh template %u\n", exporter->exporter_id, id);
	362
	363	if ( !table ) {
	364	return;
	365	}
	366
	367	// clear current table
	368	memset((void )table->element, 0, NumElements sizeof(translation_element_t));
	369	table->updated = time(NULL);
	370	table->flags = 0;
	371
	372	// printf("[%u] Fill translation table %u\n", exporter->exporter_id, id);
	373
	374	// fill table
	375
	376	table->id = id;
	377	table->input_index = 0;
	378	table->zero_index = NumElements - 1;
	379
	380	/*
	381	* common data block: The common record is expected in the output stream. If not available
	382	* in the template, fill values with 0
	383	*/
	384
	385	offset = BYTE_OFFSET_first;
	386	FillElement( table, NF9_FIRST_SWITCHED, &offset);
	387	FillElement( table, NF9_LAST_SWITCHED, &offset);
	388	FillElement( table, NF9_DIRECTION, &offset);
	389	FillElement( table, NF9_TCP_FLAGS, &offset);
	390	FillElement( table, NF9_IN_PROTOCOL, &offset);
	391	FillElement( table, NF9_SRC_TOS, &offset);
	392	FillElement( table, NF9_INPUT_SNMP, &offset);
	393	FillElement( table, NF9_OUTPUT_SNMP, &offset);
	394	FillElement( table, NF9_L4_SRC_PORT, &offset);
	395	FillElement( table, NF9_L4_DST_PORT, &offset);
	396	FillElement( table, NF9_SRC_AS, &offset);
	397	FillElement( table, NF9_DST_AS, &offset);
	398
	399	/* IP addresss record
	400	* This record is expected in the output stream. If not available
	401	* in the template, assume empty v4 address.
	402	*/
	403	if ( input_template[NF9_IPV4_SRC_ADDR].length ) {
	404	// IPv4 addresses
	405	FillElement( table, NF9_IPV4_SRC_ADDR, &offset);
	406	FillElement( table, NF9_IPV4_DST_ADDR, &offset);
	407	} else if ( input_template[NF9_IPV6_SRC_ADDR].length == 16 ) {
	408	// IPv6 addresses
	409	FillElement( table, NF9_IPV6_SRC_ADDR, &offset);
	410	FillElement( table, NF9_IPV6_DST_ADDR, &offset);
	411	// mark IPv6
	412	table->flags \|= FLAG_IPV6_ADDR;
	413	} else {
	414	// should not happen, assume empty IPv4 addresses
	415	FillElement( table, NF9_IPV4_SRC_ADDR, &offset);
	416	FillElement( table, NF9_IPV4_DST_ADDR, &offset);
	417	}
	418
	419
	420	/* packet record
	421	* This record is expected in the output stream. If not available
	422	* in the template, assume empty 4 bytes value
	423	*/
	424	if ( input_template[NF9_IN_PACKETS].length ) {
	425	table->packet_offset = offset;
	426	FillElement( table, NF9_IN_PACKETS, &offset);
	427	if ( input_template[NF9_IN_PACKETS].length == 8 )
	428	table->flags \|= FLAG_PKG_64;
	429	} else
	430	table->packet_offset = 0;
	431
	432	/* byte record
	433	* This record is expected in the output stream. If not available
	434	* in the template, assume empty 4 bytes value
	435	*/
	436	if ( input_template[NF9_IN_BYTES].length ) {
	437	table->byte_offset = offset;
	438	FillElement( table, NF9_IN_BYTES, &offset);
	439	if ( input_template[NF9_IN_BYTES].length == 8 )
	440	table->flags \|= FLAG_BYTES_64;
	441	} else
	442	table->byte_offset = 0;
	443
	444	table->input_record_size = input_record_size;
	445	table->output_record_size = offset;
	446
	447	/*
	448	printf("Table %u Flags: %u, index: %u, Zero: %u input_size: %u, output_size: %u\n",
	449	table->id, table->flags, table->input_index, table->zero_index, table->input_record_size, table->output_record_size);
	450	*/
	451
	452	} // End of setup_translation_table
	453
	454	static inline void Process_v9_templates(exporter_domain_t exporter, template_flowset_t template_flowset) {
	455	template_record_t *template;
	456	uint16_t id, count, field_type, field_length, offset;
	457	uint32_t size_left, template_size;
	458	int i;
	459
	460	size_left = ntohs(template_flowset->length) - 4; // -4 for flowset header -> id and length
	461	template = template_flowset->fields;
	462
	463	// process all templates in flowset, as long as any bytes are left
	464	template_size = 0;
	465	while (size_left) {
	466	template = (template_record_t *)((pointer_addr_t)template + template_size);
	467
	468	id = ntohs(template->template_id);
	469	count = ntohs(template->count);
	470	// printf("\n[%u] Template ID: %u\n", exporter->exporter_id, id);
	471
	472	template_size = 4 + 4 * count; // id + count = 4 bytes, and 2 x 2 bytes for each entry
	473	// printf("template size: %u buffersize: %u\n", template_size, size_left);
	474
	475	if ( size_left < template_size ) {
	476	syslog(LOG_ERR, "Process_v9: [%u] buffer size error: expected %u available %u\n",
	477	exporter->exporter_id, template_size, size_left);
	478	size_left = 0;
	479	continue;
	480	}
	481
	482	offset = 0;
	483	memset((void *)&input_template, 0, sizeof(input_template));
	484	for(i=0; i<count; i++ ) {
	485	field_type = ntohs(template->record[i].type) & 0x007f; // make sure field < 128
	486	field_length = ntohs(template->record[i].length);
	487	input_template[field_type].offset = offset;
	488	input_template[field_type].length = field_length;
	489	offset += field_length;
	490
	491	// printf("Type: %u, Length %u\n", field_type, field_length);
	492	}
	493	setup_translation_table(exporter, id, offset);
	494	size_left -= template_size;
	495	processed_records++;
	496	// printf("\n");
	497
	498	} // End of while size_left
	499
	500	} // End of Process_v9_templates
	501
	502	inline void Process_v9_data(exporter_domain_t exporter, data_flowset_t data_flowset, data_block_header_t data_header, void *writeto,
	503	stat_record_t stat_record, uint64_t first_seen, uint64_t *last_seen) {
	504
	505	input_translation_t *table;
	506	common_record_t *data_record;
	507	uint64_t start_time, end_time, packets, bytes;
	508	uint32_t size_left, First, Last;
	509	uint8_t in, out;
	510	pointer_addr_t bsize;
	511	int i;
	512	char *string;
	513
	514	data_flowset->flowset_id = ntohs(data_flowset->flowset_id);
	515	table = GetTranslationTable(exporter, data_flowset->flowset_id);
	516	if ( !table ) {
	517	// syslog(LOG_WARNING,"Process v9: [%u] No table for id %u -> Skip record\n",
	518	// exporter->exporter_id, data_flowset->flowset_id);
	519	return writeto;
	520	}
	521
	522	// map file record to output buffer
	523	data_record = (common_record_t *)writeto;
	524
	525	// sanity check for buffer size
	526	bsize = (pointer_addr_t)writeto - (pointer_addr_t)data_header;
	527	// The save margin is a full data record. The master record is a bit more
	528	// as no record will use more space than this master_record
	529	if ( bsize > (BUFFSIZE-sizeof(master_record_t)) ) {
	530	syslog(LOG_WARNING,"Process v9: Outputbuffer full. Flush buffer but have to skip records.");
	531	return writeto;
	532	}
	533
	534
	535	size_left = ntohs(data_flowset->length) - 4; // -4 for data flowset header -> id and length
	536
	537	// map byte arrays
	538	in = (uint8_t *)data_flowset->data;
	539	out = (uint8_t *)data_record;
	540
	541	// printf("[%u] Process data flowset size: %u\n", exporter->exporter_id, size_left);
	542
	543	while (size_left) {
	544
	545	if ( (size_left < table->input_record_size) ) {
	546	if ( size_left > 3 )
	547	syslog(LOG_WARNING,"Process_v9: Corrupt data flowset? Pad bytes: %u\n", size_left);
	548	size_left = 0;
	549	continue;
	550	}
	551
	552	// check for enough space in output buffer
	553	if ( (data_header->size + table->output_record_size) > OUTPUT_BUFF_SIZE ) {
	554	// this should really never occur, because the buffer gets flushed ealier
	555	syslog(LOG_ERR,"Process_v9: output buffer size error. Abort v9 record processing");
	556	return writeto;
	557	}
	558	processed_records++;
	559
	560	/*
	561	printf("[%u] Process data record: %u addr: %u %u buffersize: %u\n",
	562	exporter->exporter_id, processed_records, (uint32_t)in - (uint32_t)data_flowset, table->input_record_size, size_left);
	563	*/
	564
	565	// fill the data record
	566	data_record->flags = table->flags;
	567	data_record->size = table->output_record_size;
	568	data_record->mark = 0;
	569
	570	// pop up the table to fill the data record
	571	for ( i=0; i<table->input_index; i++ ) {
	572	int input_offset = table->element[i].input_offset;
	573	int output_offset = table->element[i].output_offset;
	574	switch ( table->element[i].length ) {
	575	case 1:
	576	out[output_offset] = in[input_offset];
	577	break;
	578	case 2:
	579	((uint16_t )&out[output_offset]) = ntohs(Get_val16((void *)&in[input_offset]));
	580	break;
	581	case 3:
	582	((uint32_t )&out[output_offset]) = ntohl(Get_val24((void *)&in[input_offset]));
	583	break;
	584	case 4:
	585	((uint32_t )&out[output_offset]) = ntohl(Get_val32((void *)&in[input_offset]));
	586	break;
	587	case 5:
	588	/* 64bit access to potentially unaligned output buffer. use 2 x 32bit for _LP64 CPUs */
	589	{ type_mask_t t;
	590
	591	t.val.val64 = ntohll(Get_val40((void *)&in[input_offset]));
	592	((uint32_t )&out[output_offset]) = t.val.val32[0];
	593	((uint32_t )&out[output_offset+4]) = t.val.val32[1];
	594	}
	595	break;
	596	case 6:
	597	/* 64bit access to potentially unaligned output buffer. use 2 x 32bit for _LP64 CPUs */
	598	{ type_mask_t t;
	599
	600	t.val.val64 = ntohll(Get_val48((void *)&in[input_offset]));
	601	((uint32_t )&out[output_offset]) = t.val.val32[0];
	602	((uint32_t )&out[output_offset+4]) = t.val.val32[1];
	603	}
	604	break;
	605	case 7:
	606	/* 64bit access to potentially unaligned output buffer. use 2 x 32bit for _LP64 CPUs */
	607	{ type_mask_t t;
	608
	609	t.val.val64 = ntohll(Get_val56((void *)&in[input_offset]));
	610	((uint32_t )&out[output_offset]) = t.val.val32[0];
	611	((uint32_t )&out[output_offset+4]) = t.val.val32[1];
	612	}
	613	break;
	614	case 8:
	615	/* 64bit access to potentially unaligned output buffer. use 2 x 32bit for _LP64 CPUs */
	616	{ type_mask_t t;
	617
	618	t.val.val64 = ntohll(Get_val64((void *)&in[input_offset]));
	619	((uint32_t )&out[output_offset]) = t.val.val32[0];
	620	((uint32_t )&out[output_offset+4]) = t.val.val32[1];
	621	}
	622	break;
	623	case 16:
	624	/* 64bit access to potentially unaligned output buffer. use 2 x 32bit for _LP64 CPUs */
	625	{ type_mask_t t;
	626
	627	t.val.val64 = ntohll(Get_val64((void *)&in[input_offset]));
	628	((uint32_t )&out[output_offset]) = t.val.val32[0];
	629	((uint32_t )&out[output_offset+4]) = t.val.val32[1];
	630
	631	t.val.val64 = ntohll(Get_val64((void *)&in[input_offset+8]));
	632	((uint32_t )&out[output_offset+8]) = t.val.val32[0];
	633	((uint32_t )&out[output_offset+12]) = t.val.val32[1];
	634	}
	635	break;
	636	default:
	637	memcpy((void )&out[output_offset], (void )&in[input_offset], table->element[i].length);
	638	}
	639	} // End for
	640
	641	// pop down the table to zero unavailable elements
	642	for ( i=NumElements - 1; i>table->zero_index; i-- ) {
	643	int output_offset = table->element[i].output_offset;
	644	switch ( table->element[i].length ) {
	645	case 1:
	646	out[output_offset] = 0;
	647	break;
	648	case 2:
	649	((uint16_t )&out[output_offset]) = 0;
	650	break;
	651	case 4:
	652	((uint32_t )&out[output_offset]) = 0;
	653	break;
	654	case 8:
	655	((uint64_t )&out[output_offset]) = 0;
	656	break;
	657	case 16:
	658	memset((void *)&out[output_offset], 0, 16);
	659	break;
	660	default:
	661	memset((void *)&out[output_offset], 0, table->element[i].length);
	662	}
	663	} // End for
	664
	665	First = data_record->first;
	666	Last = data_record->last;
	667
	668	if ( First > Last )
	669	/* Last in msec, in case of msec overflow, between start and end */
	670	end_time = 0x100000000LL + Last + exporter->boot_time;
	671	else
	672	end_time = (uint64_t)Last + exporter->boot_time;
	673
	674	/* start time in msecs */
	675	start_time = (uint64_t)First + exporter->boot_time;
	676
	677	data_record->first = start_time/1000;
	678	data_record->msec_first = start_time - data_record->first*1000;
	679
	680	data_record->last = end_time/1000;
	681	data_record->msec_last = end_time - data_record->last*1000;
	682
	683	// update first_seen, last_seen
	684	if ( start_time < *first_seen )
	685	*first_seen = start_time;
	686	if ( end_time > *last_seen )
	687	*last_seen = end_time;
	688
	689	// Update stats
	690	if ( table->packet_offset ) {
	691	if ( (data_record->flags & FLAG_PKG_64 ) == 0 ) // 32bit packet counter
	692	packets = ((uint32_t )&(out[table->packet_offset]));
	693	else {
	694	/* 64bit access to potentially unaligned output buffer. use 2 x 32bit for _LP64 CPUs */
	695	value64_t v;
	696	uint32_t ptr = (uint32_t )&(out[table->packet_offset]);
	697
	698	v.val.val32[0] = ptr[0];
	699	v.val.val32[1] = ptr[1];
	700	packets = v.val.val64;
	701	}
	702	} else
	703	packets = 0;
	704
	705	if ( table->byte_offset ) {
	706	if ( (data_record->flags & FLAG_BYTES_64 ) == 0 ) // 32bit byte counter
	707	bytes = ((uint32_t )&(out[table->byte_offset]));
	708	else {
	709	/* 64bit access to potentially unaligned output buffer. use 2 x 32bit for _LP64 CPUs */
	710	value64_t v;
	711	uint32_t ptr = (uint32_t )&(out[table->byte_offset]);
	712
	713	v.val.val32[0] = ptr[0];
	714	v.val.val32[1] = ptr[1];
	715	bytes = v.val.val64;
	716	}
	717	} else
	718	bytes = 0;
	719
	720	switch (data_record->prot ) { // switch protocol of
	721	case 1:
	722	stat_record->numflows_icmp++;
	723	stat_record->numpackets_icmp += packets;
	724	stat_record->numbytes_icmp += bytes;
	725	break;
	726	case 6:
	727	stat_record->numflows_tcp++;
	728	stat_record->numpackets_tcp += packets;
	729	stat_record->numbytes_tcp += bytes;
	730	break;
	731	case 17:
	732	stat_record->numflows_udp++;
	733	stat_record->numpackets_udp += packets;
	734	stat_record->numbytes_udp += bytes;
	735	break;
	736	default:
	737	stat_record->numflows_other++;
	738	stat_record->numpackets_other += packets;
	739	stat_record->numbytes_other += bytes;
	740	}
	741	stat_record->numflows++;
	742	stat_record->numpackets += packets;
	743	stat_record->numbytes += bytes;
	744
	745	if ( verbose ) {
	746	master_record_t master_record;
	747	ExpandRecord((common_record_t *)data_record, &master_record);
	748	format_file_block_record(&master_record, 1, &string, 0);
	749	printf("%s\n", string);
	750	}
	751
	752	data_header->size += data_record->size;
	753	data_header->NumBlocks++;
	754	size_left -= table->input_record_size;
	755	in += table->input_record_size;
	756	data_record = (common_record_t *)((pointer_addr_t)data_record + data_record->size);
	757	out = (uint8_t *)data_record;
	758
	759	// buffer size sanity check
	760	bsize = (pointer_addr_t)data_record - (pointer_addr_t)data_header;
	761	if ( bsize >= OUTPUT_BUFF_SIZE ) {
	762	syslog(LOG_ERR,"Process v9: Output buffer overflow! Flush buffer and skip records.");
	763	return (void *)data_record;
	764	}
	765
	766	}
	767	return (void *)data_record;
	768
	769	} // End of Process_v9_data
	770
	771	void Process_v9(void in_buff, ssize_t in_buff_cnt, data_block_header_t data_header, void writeto,
	772	stat_record_t stat_record, uint64_t first_seen, uint64_t *last_seen) {
	773
	774	exporter_domain_t *exporter;
	775	common_header_t *common_header;
	776	option_template_flowset_t *option_flowset;
	777	netflow_v9_header_t *v9_header;
	778	int64_t distance;
	779	uint32_t expected_records, flowset_id, flowset_length, exporter_id;
	780	ssize_t size_left;
	781
	782	size_left = in_buff_cnt;
	783	if ( size_left < NETFLOW_V9_HEADER_LENGTH ) {
	784	syslog(LOG_ERR, "Process_v9: Too little data for v9 packets: '%u'\n", size_left);
	785	return writeto;
	786	}
	787
	788	// map v9 data structure to input buffer
	789	v9_header = (netflow_v9_header_t *)in_buff;
	790	exporter_id = ntohl(v9_header->source_id);
	791
	792	exporter = GetExporter(exporter_id);
	793	if ( !exporter )
	794	return writeto;
	795
	796	/* calculate boot time in msec */
	797	v9_header->SysUptime = ntohl(v9_header->SysUptime);
	798	v9_header->unix_secs = ntohl(v9_header->unix_secs);
	799	exporter->boot_time = (uint64_t)1000 * (uint64_t)(v9_header->unix_secs) - (uint64_t)v9_header->SysUptime;
	800
	801	expected_records = ntohs(v9_header->count);
	802	common_header = (common_header_t *)((pointer_addr_t)v9_header + NETFLOW_V9_HEADER_LENGTH);
	803
	804	size_left -= NETFLOW_V9_HEADER_LENGTH;
	805
	806	// printf("\n[%u] Next packet: %u records, buffer: %u \n", exporter_id, expected_records, size_left);
	807
	808	// sequence check
	809	if ( exporter->first ) {
	810	exporter->last_sequence = ntohl(v9_header->sequence);
	811	exporter->sequence = exporter->last_sequence;
	812	exporter->first = 0;
	813	} else {
	814	exporter->last_sequence = exporter->sequence;
	815	exporter->sequence = ntohl(v9_header->sequence);
	816	distance = exporter->sequence - exporter->last_sequence;
	817	// handle overflow
	818	if (distance < 0) {
	819	distance = 0xffffffff + distance +1;
	820	}
	821	if (distance != 1) {
	822	stat_record->sequence_failure++;
	823	/*
	824	printf("[%u] Sequence error: last seq: %lli, seq %lli dist %lli\n",
	825	exporter->exporter_id, exporter->last_sequence, exporter->sequence, distance);
	826	*/
	827	/*
	828	if ( report_seq )
	829	syslog(LOG_ERR,"Flow sequence mismatch. Missing: %lli packets", delta(last_count,distance));
	830	*/
	831	}
	832	}
	833
	834	processed_records = 0;
	835
	836	// iterate over all flowsets in export packet, while there are bytes left
	837	flowset_length = 0;
	838	while (size_left) {
	839	common_header = (common_header_t *)((pointer_addr_t)common_header + flowset_length);
	840
	841	flowset_id = ntohs(common_header->flowset_id);
	842	flowset_length = ntohs(common_header->length);
	843
	844	/*
	845	printf("[%u] Next flowset: %u, length: %u buffersize: %u addr: %u\n",
	846	exporter->exporter_id, flowset_id, flowset_length, size_left,
	847	(uint32_t)common_header - (uint32_t)in_buff );
	848	*/
	849
	850	if ( flowset_length <= 4 ) {
	851	/* this should never happen, as 4 is an empty flowset
	852	and smaller is an illegal flowset anyway ...
	853	if it happends, we can't determine the next flowset, so skip the entire export packet
	854	*/
	855	syslog(LOG_ERR,"Process_v9: flowset length error. '%u' is too short for a flowset", flowset_length);
	856	// printf("Process_v9: flowset length error. '%u' is too short for a flowset\n", flowset_length);
	857	return writeto;
	858	}
	859
	860	if ( flowset_length > size_left ) {
	861	syslog(LOG_ERR,"Process_v9: flowset length error. Expected bytes: %u but buffersize: %u\n", flowset_length, size_left);
	862	size_left = 0;
	863	continue;
	864	}
	865
	866	switch (flowset_id) {
	867	case NF9_TEMPLATE_FLOWSET_ID:
	868	Process_v9_templates(exporter, (template_flowset_t *)common_header);
	869	break;
	870	case NF9_OPTIONS_FLOWSET_ID:
	871	option_flowset = (option_template_flowset_t *)common_header;
	872	syslog(LOG_DEBUG,"Process_v9: Ignore options flowset: template %u\n", ntohs(option_flowset->template_id));
	873	break;
	874	default:
	875	if ( flowset_id < NF9_MIN_RECORD_FLOWSET_ID ) {
	876	// printf("Invalid flowset id: %u\n", flowset_id);
	877	syslog(LOG_ERR,"Process_v9: Invalid flowset id: %u\n", flowset_id);
	878	}
	879
	880	// printf("[%u] ID %u Data flowset\n", exporter->exporter_id, flowset_id);
	881	writeto = Process_v9_data(exporter, (data_flowset_t *)common_header, data_header, writeto, stat_record, first_seen, last_seen);
	882	}
	883
	884	// next flowset
	885	size_left -= flowset_length;
	886
	887	} // End of while
	888
	889	/*
	890	if ( processed_records != expected_records ) {
	891	syslog(LOG_INFO,"Process_v9: Processed records %u, expected %u\n", processed_records, expected_records);
	892	}
	893	*/
	894	return writeto;
	895
	896	} /* End of Process_v9 */
	897
	898	/*
	899	* functions for sending netflow v9 records
	900	*/
	901
	902	void Init_v9_output(send_peer_t *peer) {
	903
	904	v9_output_header = (netflow_v9_header_t *)peer->send_buffer;
	905	v9_output_header->version = htons(9);
	906	v9_output_header->SysUptime = 0;
	907	v9_output_header->unix_secs = 0;
	908	v9_output_header->count = 0;
	909	v9_output_header->source_id = htonl(1);
	910	template_id = NF9_MIN_RECORD_FLOWSET_ID;
	911	peer->writeto = (void *)((pointer_addr_t)v9_output_header + (pointer_addr_t)sizeof(netflow_v9_header_t));
	912
	913	} // End of Init_v9_output
	914
	915	static output_template_t *GetOutputTemplate(uint32_t flags) {
	916	output_template_t **t;
	917	template_record_t *fields;
	918	uint32_t count, record_length;
	919
	920	t = &output_templates;
	921	// search for the template, which corresponds to our flags
	922	while ( *t ) {
	923	if ( (*t)->flags == flags )
	924	return *t;
	925	t = &((*t)->next);
	926	}
	927
	928	// nothing found, otherwise we would not get here
	929	t = (output_template_t )malloc(sizeof(output_template_t));
	930	if ( !(*t)) {
	931	fprintf(stderr, "Memory error:%s\n", strerror (errno));
	932	return NULL;
	933	}
	934	memset((void )(t), 0, sizeof(output_template_t));
	935	(*t)->next = NULL;
	936	(*t)->flags = flags;
	937	(t)->template_flowset = malloc(sizeof(template_flowset_t) + ((MAX_TEMPLATE_ELEMENTS 4)));
	938
	939	count = 0;
	940	record_length = 0;
	941	fields = (*t)->template_flowset->fields;
	942	// index 0 and 1 are filled in at the end
	943	fields->record[count].type = htons(NF9_FIRST_SWITCHED);
	944	fields->record[count].length = htons(element_info[NF9_FIRST_SWITCHED].min);
	945	record_length += element_info[NF9_FIRST_SWITCHED].min;
	946	count++;
	947
	948	fields->record[count].type = htons(NF9_LAST_SWITCHED);
	949	fields->record[count].length = htons(element_info[NF9_LAST_SWITCHED].min);
	950	record_length += element_info[NF9_LAST_SWITCHED].min;
	951	count++;
	952
	953	fields->record[count].type = htons(NF9_DIRECTION);
	954	fields->record[count].length = htons(element_info[NF9_DIRECTION].min);
	955	record_length += element_info[NF9_DIRECTION].min;
	956	count++;
	957
	958	fields->record[count].type = htons(NF9_TCP_FLAGS);
	959	fields->record[count].length = htons(element_info[NF9_TCP_FLAGS].min);
	960	record_length += element_info[NF9_TCP_FLAGS].min;
	961	count++;
	962
	963	fields->record[count].type = htons(NF9_IN_PROTOCOL);
	964	fields->record[count].length = htons(element_info[NF9_IN_PROTOCOL].min);
	965	record_length += element_info[NF9_IN_PROTOCOL].min;
	966	count++;
	967
	968	fields->record[count].type = htons(NF9_SRC_TOS);
	969	fields->record[count].length = htons(element_info[NF9_SRC_TOS].min);
	970	record_length += element_info[NF9_SRC_TOS].min;
	971	count++;
	972
	973	fields->record[count].type = htons(NF9_INPUT_SNMP);
	974	fields->record[count].length = htons(element_info[NF9_INPUT_SNMP].min);
	975	record_length += element_info[NF9_INPUT_SNMP].min;
	976	count++;
	977
	978	fields->record[count].type = htons(NF9_OUTPUT_SNMP);
	979	fields->record[count].length = htons(element_info[NF9_OUTPUT_SNMP].min);
	980	record_length += element_info[NF9_OUTPUT_SNMP].min;
	981	count++;
	982
	983	fields->record[count].type = htons(NF9_L4_SRC_PORT);
	984	fields->record[count].length = htons(element_info[NF9_L4_SRC_PORT].min);
	985	record_length += element_info[NF9_L4_SRC_PORT].min;
	986	count++;
	987
	988	fields->record[count].type = htons(NF9_L4_DST_PORT);
	989	fields->record[count].length = htons(element_info[NF9_L4_DST_PORT].min);
	990	record_length += element_info[NF9_L4_DST_PORT].min;
	991	count++;
	992
	993	fields->record[count].type = htons(NF9_SRC_AS);
	994	fields->record[count].length = htons(element_info[NF9_SRC_AS].min);
	995	record_length += element_info[NF9_SRC_AS].min;
	996	count++;
	997
	998	fields->record[count].type = htons(NF9_DST_AS);
	999	fields->record[count].length = htons(element_info[NF9_DST_AS].min);
	1000	record_length += element_info[NF9_DST_AS].min;
	1001	count++;
	1002
	1003	if ( (flags & FLAG_IPV6_ADDR) != 0 ) { // IPv6 addresses
	1004	fields->record[count].type = htons(NF9_IPV6_SRC_ADDR);
	1005	fields->record[count].length = htons(element_info[NF9_IPV6_SRC_ADDR].min);
	1006	record_length += element_info[NF9_IPV6_SRC_ADDR].min;
	1007	count++;
	1008	fields->record[count].type = htons(NF9_IPV6_DST_ADDR);
	1009	fields->record[count].length = htons(element_info[NF9_IPV6_DST_ADDR].min);
	1010	record_length += element_info[NF9_IPV6_DST_ADDR].min;
	1011	} else { // IPv4 addresses
	1012	fields->record[count].type = htons(NF9_IPV4_SRC_ADDR);
	1013	fields->record[count].length = htons(element_info[NF9_IPV4_SRC_ADDR].min);
	1014	record_length += element_info[NF9_IPV4_SRC_ADDR].min;
	1015	count++;
	1016	fields->record[count].type = htons(NF9_IPV4_DST_ADDR);
	1017	fields->record[count].length = htons(element_info[NF9_IPV4_DST_ADDR].min);
	1018	record_length += element_info[NF9_IPV4_DST_ADDR].min;
	1019	}
	1020	count++;
	1021
	1022	fields->record[count].type = htons(NF9_IN_PACKETS);
	1023	if ( (flags & FLAG_PKG_64) != 0 ) { // 64bit packet counter
	1024	fields->record[count].length = htons(element_info[NF9_IN_PACKETS].max);
	1025	record_length += element_info[NF9_IN_PACKETS].max;
	1026	} else {
	1027	fields->record[count].length = htons(element_info[NF9_IN_PACKETS].min);
	1028	record_length += element_info[NF9_IN_PACKETS].min;
	1029	}
	1030	count++;
	1031
	1032	fields->record[count].type = htons(NF9_IN_BYTES);
	1033	if ( (flags & FLAG_BYTES_64) != 0 ) { // 64bit byte counter
	1034	fields->record[count].length = htons(element_info[NF9_IN_BYTES].max);
	1035	record_length += element_info[NF9_IN_BYTES].max;
	1036	} else {
	1037	fields->record[count].length = htons(element_info[NF9_IN_BYTES].min);
	1038	record_length += element_info[NF9_IN_BYTES].min;
	1039	}
	1040	count++;
	1041
	1042	(*t)->template_flowset->flowset_id = htons(NF9_TEMPLATE_FLOWSET_ID);
	1043	(t)->flowset_length = 4 (2+count); // + 2 for the header
	1044	(t)->template_flowset->length = htons((t)->flowset_length);
	1045	(*t)->record_length = record_length;
	1046
	1047	fields->template_id = htons(template_id++);
	1048	fields->count = htons(count);
	1049
	1050	return *t;
	1051
	1052	} // End of GetOutputTemplate
	1053
	1054	int Add_v9_output_record(master_record_t master_record, send_peer_t peer) {
	1055	static data_flowset_t *data_flowset;
	1056	static output_template_t *template;
	1057	static uint64_t boot_time; // in msec
	1058	static uint32_t last_flags, common_block_size;
	1059	static int record_count, template_count, flowset_count, packet_count;
	1060	uint32_t required_size, t1, t2;
	1061	void *endwrite;
	1062	time_t now = time(NULL);
	1063
	1064	/*
	1065	char *string;
	1066	format_file_block_record(master_record, 1, &string, 0);
	1067	printf("%s\n", string);
	1068	*/
	1069	if ( !v9_output_header->unix_secs ) { // first time a record is added
	1070	// boot time is set one day back - assuming that the start time of every flow does not start ealier
	1071	boot_time = (uint64_t)(master_record->first - 86400)*1000;
	1072	v9_output_header->unix_secs = htonl(master_record->first - 86400);
	1073	v9_output_header->sequence = 0;
	1074	peer->writeto = (void *)((pointer_addr_t)peer->send_buffer + NETFLOW_V9_HEADER_LENGTH);
	1075	record_count = 0;
	1076	template_count = 0;
	1077	flowset_count = 0;
	1078	packet_count = 0;
	1079	data_flowset = NULL;
	1080
	1081	// write common blocksize from frst up to including dstas for one write (memcpy)
	1082	common_block_size = (pointer_addr_t)&master_record->fill - (pointer_addr_t)&master_record->first;
	1083
	1084	} else if ( flowset_count == 0 ) { // after a buffer flush
	1085	packet_count++;
	1086	v9_output_header->sequence = htonl(packet_count);
	1087	}
	1088
	1089	if ( data_flowset ) {
	1090	// output buffer contains already a data flowset
	1091	if ( last_flags == master_record->flags ) {
	1092	// same id as last record
	1093	// if ( now - template->time_sent > MAX_LIFETIME )
	1094	if ( (record_count & 0xFFF) == 0 ) { // every 4096 flow records
	1095	// template refresh is needed
	1096	// terminate the current data flowset
	1097	data_flowset = NULL;
	1098	if ( (pointer_addr_t)peer->writeto + template->flowset_length > (pointer_addr_t)peer->endp ) {
	1099	// not enough space for template flowset => flush buffer first
	1100	record_count = 0;
	1101	flowset_count = 0;
	1102	template_count = 0;
	1103	peer->flush = 1;
	1104	return 1; // return to flush buffer
	1105	}
	1106	memcpy(peer->writeto, (void *)template->template_flowset, template->flowset_length);
	1107	peer->writeto = (void *)((pointer_addr_t)peer->writeto + template->flowset_length);
	1108	template->time_sent = now;
	1109	flowset_count++;
	1110	template_count++;
	1111
	1112	// open a new data flow set at this point in the output buffer
	1113	data_flowset = (data_flowset_t *)peer->writeto;
	1114	data_flowset->flowset_id = template->template_flowset->fields[0].template_id;
	1115	peer->writeto = (void *)data_flowset->data;
	1116	flowset_count++;
	1117	} // else Add record
	1118
	1119	} else {
	1120	// record with different id
	1121	// terminate the current data flowset
	1122	data_flowset = NULL;
	1123
	1124	last_flags = master_record->flags;
	1125	template = GetOutputTemplate(last_flags);
	1126	if ( now - template->time_sent > MAX_LIFETIME ) {
	1127	// refresh template is needed
	1128	endwrite= (void *)((pointer_addr_t)peer->writeto + template->flowset_length + sizeof(data_flowset_t));
	1129	if ( endwrite > peer->endp ) {
	1130	// not enough space for template flowset => flush buffer first
	1131	record_count = 0;
	1132	flowset_count = 0;
	1133	template_count = 0;
	1134	peer->flush = 1;
	1135	return 1; // return to flush the buffer
	1136	}
	1137	memcpy(peer->writeto, (void *)template->template_flowset, template->flowset_length);
	1138	peer->writeto = (void *)((pointer_addr_t)peer->writeto + template->flowset_length);
	1139	template->time_sent = now;
	1140	flowset_count++;
	1141	template_count++;
	1142	}
	1143	// open a new data flow set at this point in the output buffer
	1144	data_flowset = (data_flowset_t *)peer->writeto;
	1145	data_flowset->flowset_id = template->template_flowset->fields[0].template_id;
	1146	peer->writeto = (void *)data_flowset->data;
	1147	flowset_count++;
	1148	}
	1149	} else {
	1150	// output buffer does not contain a data flowset
	1151	peer->writeto = (void *)((pointer_addr_t)v9_output_header + (pointer_addr_t)sizeof(netflow_v9_header_t));
	1152	last_flags = master_record->flags;
	1153	template = GetOutputTemplate(last_flags);
	1154	if ( now - template->time_sent > MAX_LIFETIME ) {
	1155	// refresh template
	1156	endwrite= (void *)((pointer_addr_t)peer->writeto + template->flowset_length + sizeof(data_flowset_t));
	1157	if ( endwrite > peer->endp ) {
	1158	// this must never happen!
	1159	fprintf(stderr, "Panic: Software error in %s line %d\n", __FILE__, __LINE__);
	1160	fprintf(stderr, "buffer %p, writeto %p template length %x, endbuff %p\n",
	1161	peer->send_buffer, peer->writeto, template->flowset_length + (uint32_t)sizeof(data_flowset_t), peer->endp );
	1162	exit(255);
	1163	}
	1164	memcpy(peer->writeto, (void *)template->template_flowset, template->flowset_length);
	1165	peer->writeto = (void *)((pointer_addr_t)peer->writeto + template->flowset_length);
	1166	template->time_sent = now;
	1167	flowset_count++;
	1168	template_count++;
	1169	}
	1170	// open a new data flow set at this point in the output buffer
	1171	data_flowset = (data_flowset_t *)peer->writeto;
	1172	data_flowset->flowset_id = template->template_flowset->fields[0].template_id;
	1173	peer->writeto = (void *)data_flowset->data;
	1174	flowset_count++;
	1175	}
	1176	// now add the record
	1177
	1178	required_size = template->record_length;
	1179
	1180	endwrite = (void *)((pointer_addr_t)peer->writeto + required_size);
	1181	if ( endwrite > peer->endp ) {
	1182	uint16_t length = (pointer_addr_t)peer->writeto - (pointer_addr_t)data_flowset;
	1183	// flush the buffer
	1184	data_flowset->length = htons(length);
	1185	if ( length == 4 ) { // empty flowset
	1186	peer->writeto = (void *)data_flowset;
	1187	}
	1188	data_flowset = NULL;
	1189	v9_output_header->count = htons(record_count+template_count);
	1190	record_count = 0;
	1191	template_count = 0;
	1192	flowset_count = 0;
	1193	peer->flush = 1;
	1194	return 1; // return to flush buffer
	1195	}
	1196
	1197	// this was a long way up to here, now we can add the data
	1198
	1199	master_record->input = htons(master_record->input);
	1200	master_record->output = htons(master_record->output);
	1201
	1202	t1 = (uint32_t)(1000LL * (uint64_t)master_record->first + master_record->msec_first - boot_time);
	1203	t2 = (uint32_t)(1000LL * (uint64_t)master_record->last + master_record->msec_last - boot_time);
	1204	master_record->first = htonl(t1);
	1205	master_record->last = htonl(t2);
	1206
	1207	master_record->srcport = htons(master_record->srcport);
	1208	master_record->dstport = htons(master_record->dstport);
	1209	master_record->srcas = htons(master_record->srcas);
	1210	master_record->dstas = htons(master_record->dstas);
	1211
	1212	memcpy(peer->writeto, (void *)&master_record->first,common_block_size);
	1213	peer->writeto = (void *)((pointer_addr_t)peer->writeto + common_block_size);
	1214
	1215	if ((master_record->flags & FLAG_IPV6_ADDR) != 0 ) { // IPv6
	1216	master_record->v6.srcaddr[0] = htonll(master_record->v6.srcaddr[0]);
	1217	master_record->v6.srcaddr[1] = htonll(master_record->v6.srcaddr[1]);
	1218	master_record->v6.dstaddr[0] = htonll(master_record->v6.dstaddr[0]);
	1219	master_record->v6.dstaddr[1] = htonll(master_record->v6.dstaddr[1]);
	1220	memcpy(peer->writeto, master_record->v6.srcaddr, sizeof(ipv6_block_t));
	1221	peer->writeto = (void *)((pointer_addr_t)peer->writeto + sizeof(ipv6_block_t));
	1222	} else {
	1223	uint32_t addr = (uint32_t )peer->writeto;
	1224	addr[0] = htonl(master_record->v4.srcaddr);
	1225	addr[1] = htonl(master_record->v4.dstaddr);
	1226	peer->writeto = (void )((pointer_addr_t)peer->writeto + 2sizeof(uint32_t));
	1227	}
	1228
	1229	if ((master_record->flags & FLAG_PKG_64) != 0 ) { // 64bit counters
	1230	/* 64bit access to potentially unaligned output buffer. use 2 x 32bit for _LP64 CPUs */
	1231	uint32_t outbuffer = (uint32_t )peer->writeto;
	1232	value64_t v;
	1233
	1234	v.val.val64 = htonll(master_record->dPkts);
	1235	outbuffer[0] = v.val.val32[0];
	1236	outbuffer[1] = v.val.val32[1];
	1237	peer->writeto = (void *)((pointer_addr_t)peer->writeto + sizeof(uint64_t));
	1238	} else {
	1239	uint32_t v = (uint32_t )peer->writeto;
	1240	*v = htonl(master_record->dPkts);
	1241	peer->writeto = (void *)((pointer_addr_t)peer->writeto + sizeof(uint32_t));
	1242	}
	1243
	1244	if ((master_record->flags & FLAG_BYTES_64) != 0 ) { // 64bit counters
	1245	/* 64bit access to potentially unaligned output buffer. use 2 x 32bit for _LP64 CPUs */
	1246	uint32_t outbuffer = (uint32_t )peer->writeto;
	1247	value64_t v;
	1248
	1249	v.val.val64 = htonll(master_record->dOctets);
	1250	outbuffer[0] = v.val.val32[0];
	1251	outbuffer[1] = v.val.val32[1];
	1252	peer->writeto = (void *)((pointer_addr_t)peer->writeto + sizeof(uint64_t));
	1253	} else {
	1254	uint32_t v = (uint32_t )peer->writeto;
	1255	*v = htonl(master_record->dOctets);
	1256	peer->writeto = (void *)((pointer_addr_t)peer->writeto + sizeof(uint32_t));
	1257	}
	1258
	1259	data_flowset->length = htons((pointer_addr_t)peer->writeto - (pointer_addr_t)data_flowset);
	1260	record_count++;
	1261	v9_output_header->count = htons(record_count+template_count);
	1262
	1263	return 0;
	1264
	1265	} // End of Add_v9_output_record
	1266

+204

-0

netflow_v9.h less more

	0	/*
	1	* This file is part of the nfdump project.
	2	*
	3	* Copyright (c) 2004, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
	4	* All rights reserved.
	5	*
	6	* Redistribution and use in source and binary forms, with or without
	7	* modification, are permitted provided that the following conditions are met:
	8	*
	9	* * Redistributions of source code must retain the above copyright notice,
	10	* this list of conditions and the following disclaimer.
	11	* * Redistributions in binary form must reproduce the above copyright notice,
	12	* this list of conditions and the following disclaimer in the documentation
	13	* and/or other materials provided with the distribution.
	14	* * Neither the name of SWITCH nor the names of its contributors may be
	15	* used to endorse or promote products derived from this software without
	16	* specific prior written permission.
	17	*
	18	* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
	19	* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	20	* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	21	* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
	22	* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
	23	* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
	24	* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
	25	* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
	26	* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
	27	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
	28	* POSSIBILITY OF SUCH DAMAGE.
	29	*
	30	* $Author: peter $
	31	*
	32	* $Id: netflow_v9.h 70 2006-05-17 08:38:01Z peter $
	33	*
	34	* $LastChangedRevision: 70 $
	35	*
	36	*/
	37
	38	/* v9 structures */
	39
	40	/* Packet Header Field Descriptions
	41	*
	42	* Version
	43	* Version of Flow Record format exported in this packet. The
	44	* value of this field is 9 for the current version.
	45	*
	46	* Count
	47	* The total number of records in the Export Packet, which is the
	48	* sum of Options FlowSet records, Template FlowSet records, and
	49	* Data FlowSet records.
	50	*
	51	* sysUpTime
	52	* Time in milliseconds since this device was first booted.
	53	*
	54	* UNIX Secs
	55	* Time in seconds since 0000 UTC 1970, at which the Export Packet
	56	* leaves the Exporter.
	57	*
	58	* Sequence Number
	59	* Incremental sequence counter of all Export Packets sent from
	60	* the current Observation Domain by the Exporter. This value
	61	* MUST be cumulative, and SHOULD be used by the Collector to
	62	* identify whether any Export Packets have been missed.
	63	*
	64	* Source ID
	65	* A 32-bit value that identifies the Exporter Observation Domain.
	66	* NetFlow Collectors SHOULD use the combination of the source IP
	67	* address and the Source ID field to separate different export
	68	* streams originating from the same Exporter.
	69	*/
	70
	71	typedef struct netflow_v9_header {
	72	uint16_t version;
	73	uint16_t count;
	74	uint32_t SysUptime;
	75	uint32_t unix_secs;
	76	uint32_t sequence;
	77	uint32_t source_id;
	78	} netflow_v9_header_t;
	79
	80	#define NETFLOW_V9_HEADER_LENGTH sizeof(netflow_v9_header_t)
	81
	82	/* FlowSet ID
	83	* FlowSet ID value of 0 is reserved for the Template FlowSet.
	84	* Length
	85	* Total length of this FlowSet. Because an individual Template
	86	* FlowSet MAY contain multiple Template Records, the Length value
	87	* MUST be used to determine the position of the next FlowSet
	88	* record, which could be any type of FlowSet. Length is the sum
	89	* of the lengths of the FlowSet ID, the Length itself, and all
	90	* Template Records within this FlowSet.
	91	*
	92	* Template ID
	93	* Each of the newly generated Template Records is given a unique
	94	* Template ID. This uniqueness is local to the Observation
	95	* Domain that generated the Template ID. Template IDs 0-255 are
	96	* reserved for Template FlowSets, Options FlowSets, and other
	97	* reserved FlowSets yet to be created. Template IDs of Data
	98	* FlowSets are numbered from 256 to 65535.
	99	*
	100	* Field Count
	101	* Number of fields in this Template Record. Because a Template
	102	* FlowSet usually contains multiple Template Records, this field
	103	* allows the Collector to determine the end of the current
	104	* Template Record and the start of the next.
	105	*
	106	* Field Type
	107	* A numeric value that represents the type of the field. Refer
	108	* to the "Field Type Definitions" section.
	109	*
	110	* Field Length
	111	* The length of the corresponding Field Type, in bytes. Refer to
	112	* the "Field Type Definitions" section.
	113	*/
	114
	115	typedef struct template_record_s {
	116	uint16_t template_id;
	117	uint16_t count;
	118	struct {
	119	uint16_t type;
	120	uint16_t length;
	121	} record[1];
	122	} template_record_t;
	123
	124	typedef struct template_flowset_s {
	125	uint16_t flowset_id;
	126	uint16_t length;
	127	template_record_t fields[1];
	128	} template_flowset_t;
	129
	130	typedef struct data_flowset_s {
	131	uint16_t flowset_id;
	132	uint16_t length;
	133	uint8_t data[4];
	134	} data_flowset_t;
	135
	136	typedef struct option_template_flowset_s {
	137	uint16_t flowset_id;
	138	uint16_t length;
	139	uint16_t template_id;
	140	uint16_t count;
	141	} option_template_flowset_t;
	142
	143	typedef struct common_header_s {
	144	uint16_t flowset_id;
	145	uint16_t length;
	146	} common_header_t;
	147
	148	#define NF9_TEMPLATE_FLOWSET_ID 0
	149	#define NF9_OPTIONS_FLOWSET_ID 1
	150	#define NF9_MIN_RECORD_FLOWSET_ID 256
	151
	152	/* Flowset record types implemented */
	153	#define NF9_FIRST_SWITCHED 22
	154	#define NF9_LAST_SWITCHED 21
	155	#define NF9_DIRECTION 61
	156	#define NF9_TCP_FLAGS 6
	157	#define NF9_SRC_TOS 5
	158	#define NF9_IN_PROTOCOL 4
	159	#define NF9_INPUT_SNMP 10
	160	#define NF9_OUTPUT_SNMP 14
	161	#define NF9_L4_SRC_PORT 7
	162	#define NF9_L4_DST_PORT 11
	163	#define NF9_SRC_AS 16
	164	#define NF9_DST_AS 17
	165	#define NF9_OUT_BYTES 23
	166	#define NF9_OUT_PKTS 24
	167
	168	// count as two elements only
	169	#define NF9_IPV4_SRC_ADDR 8
	170	#define NF9_IPV4_DST_ADDR 12
	171	#define NF9_IPV6_SRC_ADDR 27
	172	#define NF9_IPV6_DST_ADDR 28
	173
	174	#define NF9_IN_BYTES 1
	175	#define NF9_IN_PACKETS 2
	176
	177	#define MAX_TEMPLATE_ELEMENTS 18
	178
	179	/* Flowset record types not implemented */
	180	#define NF9_FLOWS 3
	181	#define NF9_SRC_MASK 9
	182	#define NF9_DST_MASK 13
	183	#define NF9_IPV4_NEXT_HOP 15
	184	#define NF9_IPV6_SRC_MASK 29
	185	#define NF9_IPV6_DST_MASK 30
	186	#define NF9_IPV6_FLOW_LABEL 31
	187	#define NF9_ICMP_TYPE 32
	188	#define NF9_ENGINE_TYPE 38
	189	#define NF9_ENGINE_ID 39
	190	#define NF9_DST_TOS 55
	191	#define NF9_IPV6_NEXT_HOP 62
	192
	193
	194	/* prototypes */
	195	void Init_v9(void);
	196
	197	void Process_v9(void in_buff, ssize_t in_buff_cnt, data_block_header_t data_header, void writeto,
	198	stat_record_t stat_record, uint64_t first_seen, uint64_t *last_seen);
	199
	200	void Init_v9_output(send_peer_t *peer);
	201
	202	int Add_v9_output_record(master_record_t master_record, send_peer_t peer);
	203

+950

-430

nf_common.c less more

29	29	*
30	30	* $Author: peter $
31	31	*
32		* $Id: nf_common.c 53 2005-11-17 07:45:34Z peter $
	32	* $Id: nf_common.c 75 2006-05-21 15:32:48Z peter $
33	33	*
34		* $LastChangedRevision: 53 $
	34	* $LastChangedRevision: 75 $
35	35	*
36	36	*/
37	37

42	42	#include <arpa/inet.h>
43	43	#include <time.h>
44	44	#include <string.h>
	45	#include <ctype.h>
45	46	#include <stdlib.h>
	47	#include <errno.h>
46	48
47	49	#include "config.h"
48	50

50	52	#include <stdint.h>
51	53	#endif
52	54
	55	#include "nffile.h"
	56	#include "panonymizer.h"
53	57	#include "nf_common.h"
54		#include "panonymizer.h"
55
56		/* locals */
57		#define STRINGSIZE 1024
58
59		static char string[STRINGSIZE];
	58	#include "util.h"
	59
	60	typedef void (string_function_t)(master_record_t , char *);
	61
	62	static struct token_list_s {
	63	string_function_t string_function; // function generation output string
	64	char *string_buffer; // buffer for output string
	65	} *token_list;
	66
	67	static int max_token_index = 0;
	68	static int token_index = 0;
	69
	70	#define BLOCK_SIZE 32
	71
	72	static char **format_list; // ordered list of all individual strings formating the output line
	73	static int max_format_index = 0;
	74	static int format_index = 0;
	75
	76	static int do_anonymize;
	77	static int long_v6 = 0;
	78	static uint64_t numflows;
	79	static double duration;
	80
	81	#define STRINGSIZE 1024
	82	#define IP_STRING_LEN 40
	83
	84	static char header_string[STRINGSIZE];
	85	static char data_string[STRINGSIZE];
60	86
61	87	static const double _1KB = 1024.0;
62	88	static const double _1MB = 1024.0 * 1024.0;
63	89	static const double _1GB = 1024.0 * 1024.0 * 1024.0;
64	90	static const double _1TB = 1024.0 * 1024.0 * 1024.0 * 1024.0;
65	91
66		extern int byte_mode, packet_mode;
67		enum { NONE, LESS, MORE };
	92	/* prototypes */
	93	static void InitFormatParser(void);
	94
	95	static void AddToken(int index);
	96
	97	static void AddString(char *string);
	98
	99	static void String_FirstSeen(master_record_t r, char string);
	100
	101	static void String_LastSeen(master_record_t r, char string);
	102
	103	static void String_Duration(master_record_t r, char string);
	104
	105	static void String_Protocol(master_record_t r, char string);
	106
	107	static void String_SrcAddr(master_record_t r, char string);
	108
	109	static void String_SrcAddrPort(master_record_t r, char string);
	110
	111	static void String_DstAddr(master_record_t r, char string);
	112
	113	static void String_DstAddrPort(master_record_t r, char string);
	114
	115	static void String_SrcPort(master_record_t r, char string);
	116
	117	static void String_DstPort(master_record_t r, char string);
	118
	119	static void String_SrcAS(master_record_t r, char string);
	120
	121	static void String_DstAS(master_record_t r, char string);
	122
	123	static void String_Input(master_record_t r, char string);
	124
	125	static void String_Output(master_record_t r, char string);
	126
	127	static void String_Packets(master_record_t r, char string);
	128
	129	static void String_Bytes(master_record_t r, char string);
	130
	131	static void String_Flows(master_record_t r, char string);
	132
	133	static void String_Tos(master_record_t r, char string);
	134
	135	static void String_Flags(master_record_t r, char string);
	136
	137	static void String_bps(master_record_t r, char string);
	138
	139	static void String_pps(master_record_t r, char string);
	140
	141	static void String_bpp(master_record_t r, char string);
	142
	143	static struct format_token_list_s {
	144	char *token; // token
	145	int is_address; // is an IP address
	146	char *header; // header line description
	147	string_function_t string_function; // function generation output string
	148	} format_token_list[] = {
	149	{ "%ts", 0, "Date flow start ", String_FirstSeen }, // Start Time - first seen
	150	{ "%te", 0, "Date flow end ", String_LastSeen }, // End Time - last seen
	151	{ "%td", 0, " Duration", String_Duration }, // Duration
	152	{ "%pr", 0, "Proto", String_Protocol }, // Protocol
	153	{ "%sa", 1, " Src IP Addr", String_SrcAddr }, // Source Address
	154	{ "%da", 1, " Dst IP Addr", String_DstAddr }, // Destination Address
	155	{ "%sap", 1, " Src IP Addr:Port ", String_SrcAddrPort }, // Source Address:Port
	156	{ "%dap", 1, " Dst IP Addr:Port ", String_DstAddrPort }, // Destination Address:Port
	157	{ "%sp", 0, "Src Pt", String_SrcPort }, // Source Port
	158	{ "%dp", 0, "Dst Pt", String_DstPort }, // Destination Port
	159	{ "%sas", 0, "Src AS", String_SrcAS }, // Source AS
	160	{ "%das", 0, "Dst AS", String_DstAS }, // Destination AS
	161	{ "%in", 0, " Input", String_Input }, // Input Interface num
	162	{ "%out", 0, "Output", String_Output }, // Output Interface num
	163	{ "%pkt", 0, " Packets", String_Packets }, // Packets
	164	{ "%byt", 0, " Bytes", String_Bytes }, // Bytes
	165	{ "%fl", 0, "Flows", String_Flows }, // Flows
	166	{ "%dp", 0, "Src AS", String_DstAS }, // Destination AS
	167	{ "%pkt", 0, "Dst AS", String_Packets }, // Packets
	168	{ "%flg", 0, " Flags", String_Flags }, // TCP Flags
	169	{ "%tos", 0, "Tos", String_Tos }, // Tos
	170	{ "%bps", 0, " bps", String_bps }, // bps - bits per second
	171	{ "%pps", 0, " pps", String_pps }, // pps - packets per second
	172	{ "%bpp", 0, " Bpp", String_bpp }, // bps - Bytes perl package
	173	{ NULL, 0, NULL, NULL }
	174	};
	175
	176	/* each of the tokens above must not generate output strings larger than this */
	177	#define MAX_STRING_LENGTH 64
	178
	179	#define NumProtos 138
	180
	181	char protolist[NumProtos][6] = {
	182	"HOP6 ", // 0 IPv6 Hop-by-Hop Option
	183	"ICMP ", // 1 Internet Control Message
	184	"IGMP ", // 2 Internet Group Management
	185	"GGP ", // 3 Gateway-to-Gateway
	186	"IPIP ", // 4 IP in IP (encapsulation)
	187	"ST ", // 5 Stream
	188	"TCP ", // 6 Transmission Control
	189	"CBT ", // 7 CBT
	190	"EGP ", // 8 Exterior Gateway Protocol
	191	"IGP ", // 9 any private interior gateway (used by Cisco for their IGRP)
	192	"BBN ", // 10 BBN RCC Monitoring
	193	"NVPII", // 11 Network Voice Protocol
	194	"PUP ", // 12 PUP
	195	"ARGUS", // 13 ARGUS
	196	"ENCOM", // 14 EMCON
	197	"XNET ", // 15 Cross Net Debugger
	198	"CHAOS", // 16 Chaos
	199	"UDP ", // 17 User Datagram
	200	"MUX ", // 18 Multiplexing
	201	"DCN ", // 19 DCN Measurement Subsystems
	202	"HMP ", // 20 Host Monitoring
	203	"PRM ", // 21 Packet Radio Measurement
	204	"XNS ", // 22 XEROX NS IDP
	205	"Trnk1", // 23 Trunk-1
	206	"Trnk2", // 24 Trunk-2
	207	"Leaf1", // 25 Leaf-1
	208	"Leaf2", // 26 Leaf-2
	209	"RDP ", // 27 Reliable Data Protocol
	210	"IRTP ", // 28 Internet Reliable Transaction
	211	"ISO-4", // 29 ISO Transport Protocol Class 4
	212	"NETBK", // 30 Bulk Data Transfer Protocol
	213	"MFESP", // 31 MFE Network Services Protocol
	214	"MEINP", // 32 MERIT Internodal Protocol
	215	"DCCP ", // 33 Datagram Congestion Control Protocol
	216	"3PC ", // 34 Third Party Connect Protocol
	217	"IDPR ", // 35 Inter-Domain Policy Routing Protocol
	218	"XTP ", // 36 XTP
	219	"DDP ", // 37 Datagram Delivery Protocol
	220	"IDPR ", // 38 IDPR Control Message Transport Proto
	221	"TP++ ", // 39 TP++ Transport Protocol
	222	"IL ", // 40 IL Transport Protocol
	223	"IPv6 ", // 41 IPv6
	224	"SDRP ", // 42 Source Demand Routing Protocol
	225	"Rte6 ", // 43 Routing Header for IPv6
	226	"Frag6", // 44 Fragment Header for IPv6
	227	"IDRP ", // 45 Inter-Domain Routing Protocol
	228	"RSVP ", // 46 Reservation Protocol
	229	"GRE ", // 47 General Routing Encapsulation
	230	"MHRP ", // 48 Mobile Host Routing Protocol
	231	"BNA ", // 49 BNA
	232	"ESP ", // 50 Encap Security Payload
	233	"AH ", // 51 Authentication Header
	234	"INLSP", // 52 Integrated Net Layer Security TUBA
	235	"SWIPE", // 53 IP with Encryption
	236	"NARP ", // 54 NBMA Address Resolution Protocol
	237	"MOBIL", // 55 IP Mobility
	238	"TLSP ", // 56 Transport Layer Security Protocol
	239	"SKIP ", // 57 SKIP
	240	"ICMP6", // 58 ICMP for IPv6
	241	"NOHE6", // 59 No Next Header for IPv6
	242	"OPTS6", // 60 Destination Options for IPv6
	243	"HOST ", // 61 any host internal protocol
	244	"CFTP ", // 62 CFTP
	245	"NET ", // 63 any local network
	246	"SATNT", // 64 SATNET and Backroom EXPAK
	247	"KLAN ", // 65 Kryptolan
	248	"RVD ", // 66 MIT Remote Virtual Disk Protocol
	249	"IPPC ", // 67 Internet Pluribus Packet Core
	250	"FS ", // 68 any distributed file system
	251	"SATM ", // 69 SATNET Monitoring
	252	"VISA ", // 70 VISA Protocol
	253	"IPCV ", // 71 Internet Packet Core Utility
	254	"CPNX ", // 72 Computer Protocol Network Executive
	255	"CPHB ", // 73 Computer Protocol Heart Beat
	256	"WSN ", // 74 Wang Span Network
	257	"PVP ", // 75 Packet Video Protocol
	258	"BSATM", // 76 Backroom SATNET Monitoring
	259	"SUNND", // 77 SUN ND PROTOCOL-Temporary
	260	"WBMON", // 78 WIDEBAND Monitoring
	261	"WBEXP", // 79 WIDEBAND EXPAK
	262	"ISOIP", // 80 ISO Internet Protocol
	263	"VMTP ", // 81 VMTP
	264	"SVMTP", // 82 SECURE-VMTP
	265	"VINES", // 83 VINES
	266	"TTP ", // 84 TTP
	267	"NSIGP", // 85 NSFNET-IGP
	268	"DGP ", // 86 Dissimilar Gateway Protocol
	269	"TCP ", // 87 TCF
	270	"EIGRP", // 88 EIGRP
	271	"OSPF ", // 89 OSPFIGP
	272	"S-RPC", // 90 Sprite RPC Protocol
	273	"LARP ", // 91 Locus Address Resolution Protocol
	274	"MTP ", // 92 Multicast Transport Protocol
	275	"AX.25", // 93 AX.25 Frames
	276	"IPIP ", // 94 IP-within-IP Encapsulation Protocol
	277	"MICP ", // 95 Mobile Internetworking Control Protocol
	278	"SCCSP", // 96 Semaphore Communications Sec. Protocol
	279	"ETHIP", // 97 Ethernet-within-IP Encapsulation
	280	"ENCAP", // 98 Encapsulation Header
	281	"99 ", // 99 any private encryption scheme
	282	"GMTP ", // 100 GMTP
	283	"IFMP ", // 101 Ipsilon Flow Management Protocol
	284	"PNNI ", // 102 PNNI over IP
	285	"PIM ", // 103 Protocol Independent Multicast
	286	"ARIS ", // 104 ARIS
	287	"SCPS ", // 105 SCPS
	288	"QNX ", // 106 QNX
	289	"A/N ", // 107 Active Networks
	290	"IPcmp", // 108 IP Payload Compression Protocol
	291	"SNP ", // 109 Sitara Networks Protocol
	292	"CpqPP", // 110 Compaq Peer Protocol
	293	"IPXIP", // 111 IPX in IP
	294	"VRRP ", // 112 Virtual Router Redundancy Protocol
	295	"PGM ", // 113 PGM Reliable Transport Protocol
	296	"0hop ", // 114 any 0-hop protocol
	297	"L2TP ", // 115 Layer Two Tunneling Protocol
	298	"DDX ", // 116 D-II Data Exchange (DDX)
	299	"IATP ", // 117 Interactive Agent Transfer Protocol
	300	"STP ", // 118 Schedule Transfer Protocol
	301	"SRP ", // 119 SpectraLink Radio Protocol
	302	"UTI ", // 120 UTI
	303	"SMP ", // 121 Simple Message Protocol
	304	"SM ", // 122 SM
	305	"PTP ", // 123 Performance Transparency Protocol
	306	"ISIS4", // 124 ISIS over IPv4
	307	"FIRE ", // 125 FIRE
	308	"CRTP ", // 126 Combat Radio Transport Protocol
	309	"CRUDP", // 127 Combat Radio User Datagram
	310	"128 ", // 128 SSCOPMCE
	311	"IPLT ", // 129 IPLP
	312	"SPS ", // 130 Secure Packet Shield
	313	"PIPE ", // 131 Private IP Encapsulation within IP
	314	"SCTP ", // 132 Stream Control Transmission Protocol
	315	"FC ", // 133 Fibre Channel
	316	"134 ", // 134 RSVP-E2E-IGNORE
	317	"MHEAD", // 135 Mobility Header
	318	"UDP-L", // 136 UDPLite
	319	"MPLS " // 137 MPLS-in-IP
	320	};
	321
	322
	323	/* functions */
	324
	325	void Setv6Mode(int mode) {
	326	long_v6 += mode;
	327	}
	328
	329	int Getv6Mode(void) {
	330	return long_v6;
	331	}
68	332
69	333	#ifdef __SUNPRO_C
70	334	extern
71	335	#endif
72		inline int TimeMsec_CMP(time_t t1, uint16_t offset1, time_t t2, uint16_t offset2 ) {
73
74		if ( t1 > t2 )
75		return 1;
76		if ( t2 > t1 )
77		return 2;
78		// else t1 == t2 - offset is now relevant
79		if ( offset1 > offset2 )
80		return 1;
81		if ( offset2 > offset1 )
82		return 2;
83		else
84		// both times are the same
85		return 0;
86		} // End of TimeMsec_CMP
87
88		void flow_header_raw(void header, uint64_t numflows, uint64_t pkts, uint64_t bytes, char * s, int anon) {
89		/* Allocates and fills a string with a verbose representation of
90		the header pased as argument.
91		Result: 0 on success, !=0 on error
92		*/
93		char * t;
94		flow_header_t h = (flow_header_t )header;
95		time_t now;
	336	inline void Proto_string(uint8_t protonum, char *protostr) {
	337
	338	if ( protonum >= NumProtos ) {
	339	snprintf(protostr,16,"%-5i", protonum );
	340	} else {
	341	strncpy(protostr, protolist[protonum], 16);
	342	}
	343
	344	} // End of Proto_string
	345
	346	int Proto_num(char *protostr) {
	347	int i, len;
	348
	349	if ( (len = strlen(protostr)) >= 6 )
	350	return -1;
	351
	352	for ( i=0; i<NumProtos; i++ ) {
	353	if ( strncasecmp(protostr,protolist[i], len) == 0 &&
	354	( protolist[i][len] == 0 \|\| protolist[i][len] == ' ') )
	355	return i;
	356	}
	357
	358	return -1;
	359
	360	} // End of Proto_num
	361
	362	void format_file_block_header(void header, uint64_t numflows, char * s, int anon) {
	363	data_block_header_t h = (data_block_header_t )header;
96	364
97		now = h->unix_secs;
98		t = ctime(&now);
99		t[strlen(t)-1] = 0;
100
101		snprintf(string,STRINGSIZE-1 ,""
102		"Flow Header: binary version %2u\n"
103		" count = %10u\n"
104		" SysUptime = %10u\n"
105		" unix_secs = %10d [%s]\n"
106		" unix_nsecs = %10d\n"
107		" flow_sequence = %11u\n"
108		" engine_type = %2d\n"
109		" engine_id = %2d\n",
110		h->layout_version,
111		h->count,
112		h->SysUptime,h->unix_secs,t,h->unix_nsecs,
113		h->flow_sequence,h->engine_type,h->engine_id);
114		*s = string;
115
116		} // End of flow_header_raw
117
118
119		void flow_record_raw(void record, uint64_t numflows, uint64_t pkts, uint64_t bytes, char * s, int anon) {
120		struct in_addr a,d,n;
121		char as[16], ds[16], ns[16], datestr1[64], datestr2[64];
122		char * str;
123		time_t when;
124		struct tm *ts;
125		flow_record_t r = (flow_record_t )record;
126
127		if ( anon ) {
128		if ( r->srcaddr )
129		r->srcaddr = anonymize(r->srcaddr);
130		if ( r->dstaddr )
131		r->dstaddr = anonymize(r->dstaddr);
132
133		r->nexthop = anonymize(r->nexthop);
134		}
135		a.s_addr = htonl(r->srcaddr);
136		d.s_addr = htonl(r->dstaddr);
137		n.s_addr = htonl(r->nexthop);
138		str = inet_ntoa(a);
139		strncpy(as, inet_ntoa(a), 15);
140		str = inet_ntoa(d);
141		strncpy(ds, str, 15);
142		str = inet_ntoa(n);
143		strncpy(ns, str, 15);
144		as[15] = 0;
145		ds[15] = 0;
146		ns[15] = 0;
147
148		when = r->First;
	365	snprintf(data_string,STRINGSIZE-1 ,""
	366	"File Block Header: \n"
	367	" NumBlocks = %10u\n"
	368	" Size = %10u\n"
	369	" id = %10u\n",
	370	h->NumBlocks,
	371	h->size,
	372	h->id);
	373	*s = data_string;
	374
	375	} // End of format_file_block_header
	376
	377	void format_file_block_record(void record, uint64_t numflows, char * s, int anon) {
	378	uint64_t anon_ip[2];
	379	char as[IP_STRING_LEN], ds[IP_STRING_LEN], datestr1[64], datestr2[64], flags_str[16];
	380	time_t when;
	381	struct tm *ts;
	382	master_record_t r = (master_record_t )record;
	383
	384	if ( (r->flags & FLAG_IPV6_ADDR ) != 0 ) { // IPv6
	385	if ( anon ) {
	386	anonymize_v6(r->v6.srcaddr, anon_ip);
	387	r->v6.srcaddr[0] = anon_ip[0];
	388	r->v6.srcaddr[1] = anon_ip[1];
	389
	390	anonymize_v6(r->v6.dstaddr, anon_ip);
	391	r->v6.dstaddr[0] = anon_ip[0];
	392	r->v6.dstaddr[1] = anon_ip[1];
	393	}
	394	r->v6.srcaddr[0] = htonll(r->v6.srcaddr[0]);
	395	r->v6.srcaddr[1] = htonll(r->v6.srcaddr[1]);
	396	r->v6.dstaddr[0] = htonll(r->v6.dstaddr[0]);
	397	r->v6.dstaddr[1] = htonll(r->v6.dstaddr[1]);
	398	inet_ntop(AF_INET6, r->v6.srcaddr, as, sizeof(as));
	399	inet_ntop(AF_INET6, r->v6.dstaddr, ds, sizeof(ds));
	400	if ( ! long_v6 ) {
	401	condense_v6(as);
	402	condense_v6(ds);
	403	}
	404	} else { // IPv4
	405	if ( anon ) {
	406	r->v4.srcaddr = anonymize(r->v4.srcaddr);
	407	r->v4.dstaddr = anonymize(r->v4.dstaddr);
	408	}
	409	r->v4.srcaddr = htonl(r->v4.srcaddr);
	410	r->v4.dstaddr = htonl(r->v4.dstaddr);
	411	inet_ntop(AF_INET, &r->v4.srcaddr, as, sizeof(as));
	412	inet_ntop(AF_INET, &r->v4.dstaddr, ds, sizeof(ds));
	413	}
	414	as[IP_STRING_LEN-1] = 0;
	415	ds[IP_STRING_LEN-1] = 0;
	416
	417	when = r->first;
149	418	ts = localtime(&when);
150	419	strftime(datestr1, 63, "%Y-%m-%d %H:%M:%S", ts);
151	420
152		when = r->Last;
	421	when = r->last;
153	422	ts = localtime(&when);
154	423	strftime(datestr2, 63, "%Y-%m-%d %H:%M:%S", ts);
155	424
156		snprintf(string, STRINGSIZE-1, "\n"
	425	String_Flags(record, flags_str);
	426	snprintf(data_string, STRINGSIZE-1, "\n"
157	427	"Flow Record: \n"
158		" addr = %15s\n"
159		" dstaddr = %15s\n"
160		" nexthop = %15s\n"
161		" input = %5u\n"
162		" output = %5u\n"
163		" dPkts = %10llu\n"
164		" dOctets = %10llu\n"
165		" First = %10u [%s]\n"
166		" Last = %10u [%s]\n"
167		" port = %5u\n"
168		" dstport = %5u\n"
169		" tcp_flags = %3u\n"
170		" prot = %3u\n"
171		" tos = %3u\n"
172		" src_as = %5u\n"
173		" dst_as = %5u\n"
174		" msec_first = %5u\n"
175		" msec_last = %5u"
	428	" Flags = 0x%.8x\n"
	429	" size = %5u\n"
	430	" mark = %5u\n"
	431	" srcaddr = %16s\n"
	432	" dstaddr = %16s\n"
	433	" first = %10u [%s]\n"
	434	" last = %10u [%s]\n"
	435	" msec_first = %5u\n"
	436	" msec_last = %5u\n"
	437	" dir = %3u\n"
	438	" tcp_flags = 0x%2x %s\n"
	439	" prot = %3u\n"
	440	" tos = %3u\n"
	441	" input = %5u\n"
	442	" output = %5u\n"
	443	" srcas = %5u\n"
	444	" dstas = %5u\n"
	445	" srcport = %5u\n"
	446	" dstport = %5u\n"
	447	" dPkts = %10llu\n"
	448	" dOctets = %10llu\n"
176	449	,
177		as, ds, ns,
178		r->input, r->output, pkts, bytes, r->First, datestr1,
179		r->Last, datestr2, r->srcport, r->dstport, r->tcp_flags, r->prot, r->tos,
180		r->src_as, r->dst_as, r->msec_first, r->msec_last );
181
182		string[STRINGSIZE-1] = 0;
183
184		*s = string;
185
186		} // End of flow_record_raw
187
188		void flow_record_to_line(void record, uint64_t numflows, uint64_t pkts, uint64_t bytes, char * s, int anon) {
189		double duration;
190		time_t tt;
191		struct in_addr a,d;
192		char as[16], ds[16], bytes_str[32], packets_str[32];
193		char *str;
194		char *prot;
195		char prot_long[16], datestr[64];
196		struct tm *ts;
197		flow_record_t r = (flow_record_t )record;
198
199		if ( anon ) {
200		if ( r->srcaddr )
201		r->srcaddr = anonymize(r->srcaddr);
202		if ( r->dstaddr )
203		r->dstaddr = anonymize(r->dstaddr);
204		}
205
206		a.s_addr = htonl(r->srcaddr);
207		d.s_addr = htonl(r->dstaddr);
208		str = inet_ntoa(a);
209		strncpy(as, inet_ntoa(a), 15);
210		str = inet_ntoa(d);
211		strncpy(ds, str, 15);
212		as[15] = 0;
213		ds[15] = 0;
214
215		duration = r->Last - r->First;
	450	r->flags, r->size, r->mark, as, ds, r->first, datestr1, r->last, datestr2,
	451	r->msec_first, r->msec_last, r->dir, r->tcp_flags, flags_str, r->prot, r->tos,
	452	r->input, r->output, r->srcas, r->dstas, r->srcport, r->dstport,
	453	r->dPkts, r->dOctets);
	454
	455	data_string[STRINGSIZE-1] = 0;
	456	*s = data_string;
	457
	458
	459	} // End of format_file_block_record
	460
	461	void flow_record_to_pipe(void record, uint64_t numflows, char * s, int anon) {
	462	uint64_t anon_ip[2];
	463	uint32_t sa[4], da[4];
	464	int af;
	465	master_record_t r = (master_record_t )record;
	466
	467	if ( (r->flags & FLAG_IPV6_ADDR ) != 0 ) { // IPv6
	468	if ( anon ) {
	469	anonymize_v6(r->v6.srcaddr, anon_ip);
	470	r->v6.srcaddr[0] = anon_ip[0];
	471	r->v6.srcaddr[1] = anon_ip[1];
	472
	473	anonymize_v6(r->v6.dstaddr, anon_ip);
	474	r->v6.dstaddr[0] = anon_ip[0];
	475	r->v6.dstaddr[1] = anon_ip[1];
	476	}
	477	af = PF_INET6;
	478	} else { // IPv4
	479	if ( anon ) {
	480	r->v4.srcaddr = anonymize(r->v4.srcaddr);
	481	r->v4.dstaddr = anonymize(r->v4.dstaddr);
	482	}
	483	af = PF_INET;
	484	}
	485
	486	// Make sure Endian does not screw us up
	487	sa[0] = ( r->v6.srcaddr[0] >> 32 ) & 0xffffffffLL;
	488	sa[1] = r->v6.srcaddr[0] & 0xffffffffLL;
	489	sa[2] = ( r->v6.srcaddr[1] >> 32 ) & 0xffffffffLL;
	490	sa[3] = r->v6.srcaddr[1] & 0xffffffffLL;
	491
	492	da[0] = ( r->v6.dstaddr[0] >> 32 ) & 0xffffffffLL;
	493	da[1] = r->v6.dstaddr[0] & 0xffffffffLL;
	494	da[2] = ( r->v6.dstaddr[1] >> 32 ) & 0xffffffffLL;
	495	da[3] = r->v6.dstaddr[1] & 0xffffffffLL;
	496
	497	snprintf(data_string, STRINGSIZE-1 ,"%i\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%llu\|%llu",
	498	af, r->first, r->msec_first ,r->last, r->msec_last, r->prot,
	499	sa[0], sa[1], sa[2], sa[3], r->srcport, da[0], da[1], da[2], da[3], r->dstport,
	500	r->srcas, r->dstas, r->input, r->output,
	501	r->tcp_flags, r->tos, r->dPkts, r->dOctets);
	502
	503	data_string[STRINGSIZE-1] = 0;
	504
	505	*s = data_string;
	506
	507	} // End of flow_record_pipe
	508
	509	void format_special(void record, uint64_t flows, char * s, int anon) {
	510	master_record_t r = (master_record_t )record;
	511	int i, index;
	512
	513	do_anonymize = anon;
	514	numflows = flows;
	515
	516	duration = r->last - r->first;
216	517	duration += ((double)r->msec_last - (double)r->msec_first) / 1000.0;
217
218		tt = r->First;
219		ts = localtime(&tt);
220		strftime(datestr, 63, "%Y-%m-%d %H:%M:%S", ts);
221
222		switch (r->prot) {
223		case 1:
224		prot = "ICMP";
225		break;
226		case 6:
227		prot = "TCP ";
228		break;
229		case 17:
230		prot = "UDP ";
231		break;
232		case 41:
233		prot = "IPv6";
234		break;
235		case 46:
236		prot = "RSVP";
237		break;
238		case 47:
239		prot = "GRE ";
240		break;
241		case 50:
242		prot = "ESP "; // Encap Security Payload
243		break;
244		case 51:
245		prot = "AH "; // Authentication Header
246		break;
247		case 58:
248		prot = "ICM6";
249		break;
250		case 89:
251		prot = "OSPF";
252		break;
253		case 94:
254		prot = "IPIP";
255		break;
256		case 103:
257		prot = "PIM ";
258		break;
259		default:
260		snprintf(prot_long,15,"%4d",r->prot);
261		prot = prot_long;
262		}
263
264		format_number(bytes, bytes_str);
265		format_number(pkts, packets_str);
266
267		snprintf(string, STRINGSIZE-1 ,"%s.%03u %8.3f %s %15s:%-5i -> %15s:%-5i %8s %8s %5llu",
268		datestr, r->msec_first, duration, prot, as, r->srcport, ds, r->dstport, packets_str, bytes_str, numflows);
269		string[STRINGSIZE-1] = 0;
270
271		*s = string;
272
273		} // End of flow_record_to_line
274
275
276		void flow_record_to_line_long(void record, uint64_t numflows, uint64_t pkts, uint64_t bytes, char * s, int anon) {
277		double duration;
278		time_t tt;
279		struct in_addr a,d;
280		char as[16], ds[16], TCP_flags[7], bytes_str[32], packets_str[32];
281		char *str;
282		char *prot;
283		char prot_long[16], datestr[64];
284		struct tm * ts;
285		flow_record_t r = (flow_record_t )record;
286
287		if ( anon ) {
288		if ( r->srcaddr )
289		r->srcaddr = anonymize(r->srcaddr);
290		if ( r->dstaddr )
291		r->dstaddr = anonymize(r->dstaddr);
292		}
293		a.s_addr = htonl(r->srcaddr);
294		d.s_addr = htonl(r->dstaddr);
295		str = inet_ntoa(a);
296		strncpy(as, inet_ntoa(a), 15);
297		str = inet_ntoa(d);
298		strncpy(ds, str, 15);
299		as[15] = 0;
300		ds[15] = 0;
301
302		duration = r->Last - r->First;
303		duration += ((double)r->msec_last - (double)r->msec_first) / 1000.0;
304
305		tt = r->First;
306		ts = localtime(&tt);
307		strftime(datestr, 63, "%Y-%m-%d %H:%M:%S", ts);
308
309		switch (r->prot) {
310		case 1:
311		prot = "ICMP";
312		break;
313		case 6:
314		prot = "TCP ";
315		break;
316		case 17:
317		prot = "UDP ";
318		break;
319		case 41:
320		prot = "IPv6";
321		break;
322		case 46:
323		prot = "RSVP";
324		break;
325		case 47:
326		prot = "GRE ";
327		break;
328		case 50:
329		prot = "ESP "; // Encap Security Payload
330		break;
331		case 51:
332		prot = "AH "; // Authentication Header
333		break;
334		case 58:
335		prot = "ICM6";
336		break;
337		case 94:
338		prot = "IPIP";
339		break;
340		case 89:
341		prot = "OSPF";
342		break;
343		case 103:
344		prot = "PIM ";
345		break;
346		default:
347		snprintf(prot_long,15,"%4d",r->prot);
348		prot = prot_long;
349		}
350
351		format_number(bytes, bytes_str);
352		format_number(pkts, packets_str);
353
354		TCP_flags[0] = r->tcp_flags & 32 ? 'U' : '.';
355		TCP_flags[1] = r->tcp_flags & 16 ? 'A' : '.';
356		TCP_flags[2] = r->tcp_flags & 8 ? 'P' : '.';
357		TCP_flags[3] = r->tcp_flags & 4 ? 'R' : '.';
358		TCP_flags[4] = r->tcp_flags & 2 ? 'S' : '.';
359		TCP_flags[5] = r->tcp_flags & 1 ? 'F' : '.';
360		TCP_flags[6] = '\0';
361
362		snprintf(string, STRINGSIZE-1 ,"%s.%03u %8.3f %s %15s:%-5i -> %15s:%-5i %s %3i %8s %8s %5llu",
363		datestr, r->msec_first, duration, prot, as, r->srcport, ds, r->dstport, TCP_flags, r->tos, packets_str,
364		bytes_str, numflows);
365		string[STRINGSIZE-1] = 0;
366
367		*s = string;
368
369		} // End of flow_record_to_line_long
370
371		void flow_record_to_line_extended(void record, uint64_t numflows, uint64_t pkts, uint64_t bytes, char * s, int anon) {
372		uint32_t Bpp;
373		uint64_t pps, bps;
374		double duration;
375		time_t tt;
376		struct in_addr a,d;
377		char as[16], ds[16], TCP_flags[7], bytes_str[32], packets_str[32], pps_str[32], bps_str[32];
378		char *str;
379		char *prot;
380		char prot_long[16], datestr[64];
381		struct tm *ts;
382		flow_record_t r = (flow_record_t )record;
383
384
385		if ( anon ) {
386		if ( r->srcaddr )
387		r->srcaddr = anonymize(r->srcaddr);
388		if ( r->dstaddr )
389		r->dstaddr = anonymize(r->dstaddr);
390		}
391		a.s_addr = htonl(r->srcaddr);
392		d.s_addr = htonl(r->dstaddr);
393		str = inet_ntoa(a);
394		strncpy(as, inet_ntoa(a), 15);
395		str = inet_ntoa(d);
396		strncpy(ds, str, 15);
397		as[15] = 0;
398		ds[15] = 0;
399
400		duration = r->Last - r->First;
401		duration += ((double)r->msec_last - (double)r->msec_first) / 1000.0;
402
403		tt = r->First;
404		ts = localtime(&tt);
405		strftime(datestr, 63, "%Y-%m-%d %H:%M:%S", ts);
406
407		switch (r->prot) {
408		case 1:
409		prot = "ICMP";
410		break;
411		case 6:
412		prot = "TCP ";
413		break;
414		case 17:
415		prot = "UDP ";
416		break;
417		case 41:
418		prot = "IPv6";
419		break;
420		case 46:
421		prot = "RSVP";
422		break;
423		case 47:
424		prot = "GRE ";
425		break;
426		case 50:
427		prot = "ESP "; // Encap Security Payload
428		break;
429		case 51:
430		prot = "AH "; // Authentication Header
431		break;
432		case 58:
433		prot = "ICM6";
434		break;
435		case 94:
436		prot = "IPIP";
437		break;
438		case 89:
439		prot = "OSPF";
440		break;
441		case 103:
442		prot = "PIM ";
443		break;
444		default:
445		snprintf(prot_long,15,"%4d",r->prot);
446		prot = prot_long;
447		}
448
449		TCP_flags[0] = r->tcp_flags & 32 ? 'U' : '.';
450		TCP_flags[1] = r->tcp_flags & 16 ? 'A' : '.';
451		TCP_flags[2] = r->tcp_flags & 8 ? 'P' : '.';
452		TCP_flags[3] = r->tcp_flags & 4 ? 'R' : '.';
453		TCP_flags[4] = r->tcp_flags & 2 ? 'S' : '.';
454		TCP_flags[5] = r->tcp_flags & 1 ? 'F' : '.';
455		TCP_flags[6] = '\0';
456
457		if ( duration ) {
458		pps = pkts / duration; // packets per second
459		bps = ( bytes << 3 ) / duration; // bits per second. ( >> 3 ) -> * 8 to convert octets into bits
460		} else {
461		pps = bps = 0;
462		}
463		Bpp = bytes / pkts; // Bytes per Packet
464		format_number(bytes, bytes_str);
465		format_number(pkts, packets_str);
466		format_number(pps, pps_str);
467		format_number(bps, bps_str);
468		format_number(bps, bps_str);
469
470		snprintf(string, STRINGSIZE-1 ,"%s.%03u %8.3f %s %15s:%-5i -> %15s:%-5i %s %3i %8s %8s %8s %8s %6u %5llu",
471		datestr, r->msec_first, duration, prot, as, r->srcport, ds, r->dstport, TCP_flags, r->tos, packets_str,
472		bytes_str, pps_str, bps_str, Bpp, numflows);
473		string[STRINGSIZE-1] = 0;
474
475		*s = string;
476
477		} // End of flow_record_line_extended
478
479		void flow_record_to_pipe(void record, uint64_t numflows, uint64_t pkts, uint64_t bytes, char * s, int anon) {
480		flow_record_t r = (flow_record_t )record;
481
482		if ( anon ) {
483		if ( r->srcaddr )
484		r->srcaddr = anonymize(r->srcaddr);
485		if ( r->dstaddr )
486		r->dstaddr = anonymize(r->dstaddr);
487		}
488		snprintf(string, STRINGSIZE-1 ,"%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%u\|%llu\|%llu",
489		r->First, r->msec_first ,r->Last, r->msec_last, r->prot, r->srcaddr, r->srcport, r->dstaddr, r->dstport,
490		r->tcp_flags, r->tos, pkts, bytes);
491
492		string[STRINGSIZE-1] = 0;
493
494		*s = string;
495
496		} // End of flow_record_pipe
	518	for ( i=0; i<token_index; i++ ) {
	519	token_list[i].string_function(r, token_list[i].string_buffer);
	520	}
	521
	522	// concat all strings together for the output line
	523	i = 0;
	524	for ( index=0; index<format_index; index++ ) {
	525	int j = 0;
	526	while ( format_list[index][j] && i < STRINGSIZE )
	527	data_string[i++] = format_list[index][j++];
	528	}
	529	if ( i < STRINGSIZE )
	530	data_string[i] = '\0';
	531
	532	data_string[STRINGSIZE-1] = 0;
	533	*s = data_string;
	534
	535	} // End of format_special
	536
	537	char *format_special_header(void) {
	538	return header_string;
	539	} // End of format_special_header
	540
	541	static void InitFormatParser(void) {
	542
	543	max_format_index = max_token_index = BLOCK_SIZE;
	544	format_list = (char *)malloc(max_format_index sizeof(char *));
	545	token_list = (struct token_list_s )malloc(max_token_index sizeof(struct token_list_s));
	546	if ( !format_list \|\| !token_list ) {
	547	fprintf(stderr, "Memory allocation error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
	548	exit(255);
	549	}
	550
	551	} // End of InitFormatParser
	552
	553	static void AddToken(int index) {
	554
	555	if ( token_index >= max_token_index ) { // no slot available - expand table
	556	max_token_index += BLOCK_SIZE;
	557	token_list = (struct token_list_s )realloc(token_list, max_token_index sizeof(struct token_list_s));
	558	if ( !token_list ) {
	559	fprintf(stderr, "Memory allocation error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
	560	exit(255);
	561	}
	562	}
	563	token_list[token_index].string_function = format_token_list[index].string_function;
	564	token_list[token_index].string_buffer = malloc(MAX_STRING_LENGTH);
	565	AddString(token_list[token_index].string_buffer);
	566	token_index++;
	567
	568	} // End of AddToken
	569
	570	/* Add either a static string or the memory for a variable string from a token to the list */
	571	static void AddString(char *string) {
	572
	573	if ( !string ) {
	574	fprintf(stderr, "Panic! NULL string in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
	575	exit(255);
	576	}
	577
	578	if ( format_index >= max_format_index ) { // no slot available - expand table
	579	max_format_index += BLOCK_SIZE;
	580	format_list = (char *)realloc(format_list, max_format_index sizeof(char *));
	581	if ( !format_list ) {
	582	fprintf(stderr, "Memory allocation error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
	583	exit(255);
	584	}
	585	}
	586	format_list[format_index++] = string;
	587
	588	} // End of AddString
	589
	590	int ParseOutputFormat(char *format) {
	591	char c, s, *h;
	592	int i, remaining;
	593
	594	InitFormatParser();
	595
	596	c = s = strdup(format);
	597	if ( !s ) {
	598	fprintf(stderr, "Memory allocation error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
	599	exit(255);
	600	}
	601	h = header_string;
	602	*h = '\0';
	603	while ( *c ) {
	604	if ( *c == '%' ) { // it's a token from format_token_list
	605	i = 0;
	606	remaining = strlen(c);
	607	while ( format_token_list[i].token ) { // sweep through the list
	608	int len = strlen(format_token_list[i].token);
	609
	610	// a token is separated by either a space, another token, or end of string
	611	if ( remaining >= len && !isalpha((int)c[len]) ) {
	612	// separator found a expected position
	613	char p = c[len]; // save separator;
	614	c[len] = '\0';
	615	if ( strncmp(format_token_list[i].token, c, len) == 0 ) { // token found
	616	AddToken(i);
	617	if ( long_v6 && format_token_list[i].is_address )
	618	snprintf(h, STRINGSIZE-1-strlen(h), "%23s%s", "", format_token_list[i].header);
	619	else
	620	snprintf(h, STRINGSIZE-1-strlen(h), "%s", format_token_list[i].header);
	621	h += strlen(h);
	622	c[len] = p;
	623	c += len;
	624	break;
	625	} else {
	626	c[len] = p;
	627	}
	628	}
	629	i++;
	630	}
	631	if ( format_token_list[i].token == NULL ) {
	632	fprintf(stderr, "Output format parse error at: %s\n", c);
	633	free(s);
	634	return 0;
	635	}
	636	} else { // it's a static string
	637	/* a static string goes up to next '%' or end of string */
	638	char *p = strchr(c, '%');
	639	char format[16];
	640	if ( p ) {
	641	// p points to next '%' token
	642	*p = '\0';
	643	AddString(strdup(c));
	644	snprintf(format, 15, "%%%zus", strlen(c));
	645	format[15] = '\0';
	646	snprintf(h, STRINGSIZE-1-strlen(h), format, "");
	647	h += strlen(h);
	648	*p = '%';
	649	c = p;
	650	} else {
	651	// static string up to end of format string
	652	AddString(strdup(c));
	653	snprintf(format, 15, "%%%zus", strlen(c));
	654	format[15] = '\0';
	655	snprintf(h, STRINGSIZE-1-strlen(h), format, "");
	656	h += strlen(h);
	657	*c = '\0';
	658	}
	659	}
	660	}
	661
	662	free(s);
	663	return 1;
	664
	665	} // End of ParseOutputFormat
497	666
498	667	#ifdef __SUNPRO_C
499	668	extern
500	669	#endif
501		inline void format_number(uint64_t num, char *s) {
	670	inline void format_number(uint64_t num, char *s, int fixed_width) {
502	671	double f = num;
503	672
504	673	if ( f >= _1TB ) {
505		snprintf(s, 31, "%5.1f T", f / _1TB );
	674	if ( fixed_width )
	675	snprintf(s, 31, "%5.1f T", f / _1TB );
	676	else
	677	snprintf(s, 31, "%.1f T", f / _1TB );
506	678	} else if ( f >= _1GB ) {
507		snprintf(s, 31, "%5.1f G", f / _1GB );
	679	if ( fixed_width )
	680	snprintf(s, 31, "%5.1f G", f / _1GB );
	681	else
	682	snprintf(s, 31, "%.1f G", f / _1GB );
508	683	} else if ( f >= _1MB ) {
509		snprintf(s, 31, "%5.1f M", f / _1MB );
	684	if ( fixed_width )
	685	snprintf(s, 31, "%5.1f M", f / _1MB );
	686	else
	687	snprintf(s, 31, "%.1f M", f / _1MB );
510	688	/*
511	689	} else if ( f >= _1KB ) {
512	690	snprintf(s, 31, "%5.1f K", f / _1KB );
513	691	*/
514	692	} else {
515		snprintf(s, 31, "%4.0f", f );
	693	if ( fixed_width )
	694	snprintf(s, 31, "%4.0f", f );
	695	else
	696	snprintf(s, 31, "%.0f", f );
516	697	}
517	698
518	699	} // End of format_number
519	700
	701	inline void condense_v6(char *s) {
	702	size_t len = strlen(s);
	703	char p, q;
	704
	705	if ( len <= 16 )
	706	return;
	707
	708	// orig: 2001:620:1000:cafe:20e:35ff:fec0:fed5 len = 37
	709	// condensed: 2001:62..e0:fed5
	710	p = s + 7;
	711	*p++ = '.';
	712	*p++ = '.';
	713	q = s + len - 7;
	714	while ( *q ) {
	715	p++ = q++;
	716	}
	717	*p = 0;
	718
	719	} // End of condense_v6
	720
	721	/* functions, which create the individual strings for the output line */
	722	static void String_FirstSeen(master_record_t r, char string) {
	723	time_t tt;
	724	struct tm * ts;
	725	char *s;
	726
	727	tt = r->first;
	728	ts = localtime(&tt);
	729	strftime(string, MAX_STRING_LENGTH-1, "%Y-%m-%d %H:%M:%S", ts);
	730	s = string + strlen(string);
	731	snprintf(s, MAX_STRING_LENGTH-strlen(string)-1,".%03u", r->msec_first);
	732	string[MAX_STRING_LENGTH-1] = '\0';
	733
	734	} // End of String_FirstSeen
	735
	736	static void String_LastSeen(master_record_t r, char string) {
	737	time_t tt;
	738	struct tm * ts;
	739	char *s;
	740
	741	tt = r->last;
	742	ts = localtime(&tt);
	743	strftime(string, MAX_STRING_LENGTH-1, "%Y-%m-%d %H:%M:%S", ts);
	744	s = string + strlen(string);
	745	snprintf(s, MAX_STRING_LENGTH-strlen(string)-1,".%03u", r->msec_last);
	746	string[MAX_STRING_LENGTH-1] = '\0';
	747
	748	} // End of String_LastSeen
	749
	750	static void String_Duration(master_record_t r, char string) {
	751
	752	snprintf(string, MAX_STRING_LENGTH-1 ,"%9.3f", duration);
	753	string[MAX_STRING_LENGTH-1] = '\0';
	754
	755	} // End of String_Duration
	756
	757	static void String_Protocol(master_record_t r, char string) {
	758	char s[16];
	759
	760	Proto_string(r->prot, s);
	761	snprintf(string, MAX_STRING_LENGTH-1 ,"%s", s);
	762	string[MAX_STRING_LENGTH-1] = '\0';
	763
	764	} // End of String_Protocol
	765
	766	static void String_SrcAddr(master_record_t r, char string) {
	767	char tmp_str[40];
	768
	769	if ( (r->flags & FLAG_IPV6_ADDR ) != 0 ) { // IPv6
	770	uint64_t ip[2];
	771
	772	if ( do_anonymize ) {
	773	anonymize_v6(r->v6.srcaddr, ip);
	774	} else {
	775	ip[0] = r->v6.srcaddr[0];
	776	ip[1] = r->v6.srcaddr[1];
	777	}
	778
	779	ip[0] = htonll(ip[0]);
	780	ip[1] = htonll(ip[1]);
	781	inet_ntop(AF_INET6, ip, tmp_str, 39);
	782	if ( ! long_v6 ) {
	783	condense_v6(tmp_str);
	784	}
	785	} else { // IPv4
	786	uint32_t ip;
	787	ip = do_anonymize ? anonymize(r->v4.srcaddr) : r->v4.srcaddr;
	788	ip = htonl(ip);
	789	inet_ntop(AF_INET, &ip, tmp_str, 39);
	790	}
	791	if ( long_v6 )
	792	snprintf(string, MAX_STRING_LENGTH-1, "%39s", tmp_str);
	793	else
	794	snprintf(string, MAX_STRING_LENGTH-1, "%16s", tmp_str);
	795
	796	string[MAX_STRING_LENGTH-1] = 0;
	797
	798
	799	} // End of String_SrcAddr
	800
	801	static void String_SrcAddrPort(master_record_t r, char string) {
	802	char tmp_str[40], portchar;
	803
	804	if ( (r->flags & FLAG_IPV6_ADDR ) != 0 ) { // IPv6
	805	uint64_t ip[2];
	806
	807	if ( do_anonymize ) {
	808	anonymize_v6(r->v6.srcaddr, ip);
	809	} else {
	810	ip[0] = r->v6.srcaddr[0];
	811	ip[1] = r->v6.srcaddr[1];
	812	}
	813
	814	ip[0] = htonll(ip[0]);
	815	ip[1] = htonll(ip[1]);
	816	inet_ntop(AF_INET6, ip, tmp_str, 39);
	817	if ( ! long_v6 ) {
	818	condense_v6(tmp_str);
	819	}
	820	portchar = '.';
	821	} else { // IPv4
	822	uint32_t ip;
	823	ip = do_anonymize ? anonymize(r->v4.srcaddr) : r->v4.srcaddr;
	824	ip = htonl(ip);
	825	inet_ntop(AF_INET, &ip, tmp_str, 39);
	826	portchar = ':';
	827	}
	828
	829	if ( long_v6 )
	830	snprintf(string, MAX_STRING_LENGTH-1, "%39s%c%-5i", tmp_str, portchar, r->srcport);
	831	else
	832	snprintf(string, MAX_STRING_LENGTH-1, "%16s%c%-5i", tmp_str, portchar, r->srcport);
	833
	834	string[MAX_STRING_LENGTH-1] = 0;
	835
	836	} // End of String_SrcAddrPort
	837
	838	static void String_DstAddr(master_record_t r, char string) {
	839	char tmp_str[40];
	840
	841	if ( (r->flags & FLAG_IPV6_ADDR ) != 0 ) { // IPv6
	842	uint64_t ip[2];
	843
	844	if ( do_anonymize ) {
	845	anonymize_v6(r->v6.dstaddr, ip);
	846	} else {
	847	ip[0] = r->v6.dstaddr[0];
	848	ip[1] = r->v6.dstaddr[1];
	849	}
	850
	851	ip[0] = htonll(ip[0]);
	852	ip[1] = htonll(ip[1]);
	853	inet_ntop(AF_INET6, ip, tmp_str, 39);
	854	if ( ! long_v6 ) {
	855	condense_v6(tmp_str);
	856	}
	857	} else { // IPv4
	858	uint32_t ip;
	859	ip = do_anonymize ? anonymize(r->v4.dstaddr) : r->v4.dstaddr;
	860	ip = htonl(ip);
	861	inet_ntop(AF_INET, &ip, tmp_str, 39);
	862	}
	863	if ( long_v6 )
	864	snprintf(string, MAX_STRING_LENGTH-1, "%39s", tmp_str);
	865	else
	866	snprintf(string, MAX_STRING_LENGTH-1, "%16s", tmp_str);
	867
	868	string[MAX_STRING_LENGTH-1] = 0;
	869
	870
	871	} // End of String_DstAddr
	872
	873	static void String_DstAddrPort(master_record_t r, char string) {
	874	char tmp_str[40], portchar;
	875
	876	if ( (r->flags & FLAG_IPV6_ADDR ) != 0 ) { // IPv6
	877	uint64_t ip[2];
	878
	879	if ( do_anonymize ) {
	880	anonymize_v6(r->v6.dstaddr, ip);
	881	} else {
	882	ip[0] = r->v6.dstaddr[0];
	883	ip[1] = r->v6.dstaddr[1];
	884	}
	885
	886	ip[0] = htonll(ip[0]);
	887	ip[1] = htonll(ip[1]);
	888	inet_ntop(AF_INET6, ip, tmp_str, 39);
	889	if ( ! long_v6 ) {
	890	condense_v6(tmp_str);
	891	}
	892	portchar = '.';
	893	} else { // IPv4
	894	uint32_t ip;
	895	ip = do_anonymize ? anonymize(r->v4.dstaddr) : r->v4.dstaddr;
	896	ip = htonl(ip);
	897	inet_ntop(AF_INET, &ip, tmp_str, 39);
	898	portchar = ':';
	899	}
	900
	901	if ( long_v6 )
	902	snprintf(string, MAX_STRING_LENGTH-1, "%39s%c%-5i", tmp_str, portchar, r->dstport);
	903	else
	904	snprintf(string, MAX_STRING_LENGTH-1, "%16s%c%-5i", tmp_str, portchar, r->dstport);
	905
	906	string[MAX_STRING_LENGTH-1] = 0;
	907
	908	} // End of String_DstAddrPort
	909
	910	static void String_SrcPort(master_record_t r, char string) {
	911
	912	snprintf(string, MAX_STRING_LENGTH-1 ,"%6u", r->srcport);
	913	string[MAX_STRING_LENGTH-1] = '\0';
	914
	915	} // End of String_SrcPort
	916
	917	static void String_DstPort(master_record_t r, char string) {
	918
	919	snprintf(string, MAX_STRING_LENGTH-1 ,"%6u", r->dstport);
	920	string[MAX_STRING_LENGTH-1] = '\0';
	921
	922	} // End of String_DstPort
	923
	924	static void String_SrcAS(master_record_t r, char string) {
	925
	926	snprintf(string, MAX_STRING_LENGTH-1 ,"%6u", r->srcas);
	927	string[MAX_STRING_LENGTH-1] = '\0';
	928
	929	} // End of String_SrcAS
	930
	931	static void String_DstAS(master_record_t r, char string) {
	932
	933	snprintf(string, MAX_STRING_LENGTH-1 ,"%6u", r->dstas);
	934	string[MAX_STRING_LENGTH-1] = '\0';
	935
	936	} // End of String_DstAS
	937
	938	static void String_Input(master_record_t r, char string) {
	939
	940	snprintf(string, MAX_STRING_LENGTH-1 ,"%6u", r->input);
	941	string[MAX_STRING_LENGTH-1] = '\0';
	942
	943	} // End of String_Input
	944
	945	static void String_Output(master_record_t r, char string) {
	946
	947	snprintf(string, MAX_STRING_LENGTH-1 ,"%6u", r->output);
	948	string[MAX_STRING_LENGTH-1] = '\0';
	949
	950	} // End of String_Output
	951
	952	static void String_Packets(master_record_t r, char string) {
	953	char s[32];
	954
	955	format_number(r->dPkts, s, FIXED_WIDTH);
	956	snprintf(string, MAX_STRING_LENGTH-1 ,"%8s", s);
	957	string[MAX_STRING_LENGTH-1] = '\0';
	958
	959	} // End of String_Packets
	960
	961	static void String_Bytes(master_record_t r, char string) {
	962	char s[32];
	963
	964	format_number(r->dOctets, s, FIXED_WIDTH);
	965	snprintf(string, MAX_STRING_LENGTH-1 ,"%8s", s);
	966	string[MAX_STRING_LENGTH-1] = '\0';
	967
	968	} // End of String_Bytes
	969
	970	static void String_Flows(master_record_t r, char string) {
	971
	972	snprintf(string, MAX_STRING_LENGTH-1 ,"%5llu", numflows);
	973	string[MAX_STRING_LENGTH-1] = '\0';
	974
	975	} // End of String_Flows
	976
	977	static void String_Tos(master_record_t r, char string) {
	978
	979	snprintf(string, MAX_STRING_LENGTH-1 ,"%3u", r->tos);
	980	string[MAX_STRING_LENGTH-1] = '\0';
	981
	982	} // End of String_Tos
	983
	984	static void String_Flags(master_record_t r, char string) {
	985
	986	string[0] = r->tcp_flags & 32 ? 'U' : '.';
	987	string[1] = r->tcp_flags & 16 ? 'A' : '.';
	988	string[2] = r->tcp_flags & 8 ? 'P' : '.';
	989	string[3] = r->tcp_flags & 4 ? 'R' : '.';
	990	string[4] = r->tcp_flags & 2 ? 'S' : '.';
	991	string[5] = r->tcp_flags & 1 ? 'F' : '.';
	992	string[6] = '\0';
	993
	994	} // End of String_Flags
	995
	996	static void String_bps(master_record_t r, char string) {
	997	uint64_t bps;
	998	char s[32];
	999
	1000	if ( duration ) {
	1001	bps = ( r->dOctets << 3 ) / duration; // bits per second. ( >> 3 ) -> * 8 to convert octets into bits
	1002	} else {
	1003	bps = 0;
	1004	}
	1005	format_number(bps, s, FIXED_WIDTH);
	1006	snprintf(string, MAX_STRING_LENGTH-1 ,"%8s", s);
	1007	string[MAX_STRING_LENGTH-1] = '\0';
	1008
	1009	} // End of String_bps
	1010
	1011	static void String_pps(master_record_t r, char string) {
	1012	uint64_t pps;
	1013	char s[32];
	1014
	1015	if ( duration ) {
	1016	pps = r->dPkts / duration; // packets per second
	1017	} else {
	1018	pps = 0;
	1019	}
	1020	format_number(pps, s, FIXED_WIDTH);
	1021	snprintf(string, MAX_STRING_LENGTH-1 ,"%8s", s);
	1022	string[MAX_STRING_LENGTH-1] = '\0';
	1023
	1024	} // End of String_Duration
	1025
	1026	static void String_bpp(master_record_t r, char string) {
	1027	uint32_t Bpp;
	1028
	1029	string[MAX_STRING_LENGTH-1] = '\0';
	1030
	1031	if ( r->dPkts )
	1032	Bpp = r->dOctets / r->dPkts; // Bytes per Packet
	1033	else
	1034	Bpp = 0;
	1035	snprintf(string, MAX_STRING_LENGTH-1 ,"%6u", Bpp);
	1036	string[MAX_STRING_LENGTH-1] = '\0';
	1037
	1038	} // End of String_bpp
	1039

+51

-58

nf_common.h less more

30	30	*
31	31	* $Author: peter $
32	32	*
33		* $Id: nf_common.h 53 2005-11-17 07:45:34Z peter $
	33	* $Id: nf_common.h 75 2006-05-21 15:32:48Z peter $
34	34	*
35		* $LastChangedRevision: 53 $
	35	* $LastChangedRevision: 75 $
36	36	*
37	37	*
38	38	*/
39	39
40		#include "config.h"
41
42		typedef void (printer_t)(void , uint64_t, uint64_t, uint64_t, char **, int);
	40	typedef void (printer_t)(void , uint64_t, char **, int);
43	41
44	42	#if ( SIZEOF_VOID_P == 8 )
45	43	typedef uint64_t pointer_addr_t;

52	50	uint16_t msec;
53	51	} msec_time_tt;
54	52
55		/*
56		* binary file layout:
57		* mostly compatible with v5 format
58		* flow_header is identical to the v5 header
59		* flow_record has some changes, to speed up flow processing
60		* and overcome the ugly netflow time format ...
61		*/
62
63		#define FLOW_HEADER_LENGTH sizeof(flow_header_t)
64		#define FLOW_RECORD_LENGTH sizeof(flow_record_t)
65
66
67		/* max records in binary files */
68		#define MAX_RECORDS 30
69
70		typedef struct flow_header {
	53	/* common minimum netflow header for all versions */
	54	typedef struct common_flow_header {
71	55	uint16_t version;
72	56	uint16_t count;
73		uint32_t SysUptime;
74		uint32_t unix_secs;
75		uint32_t unix_nsecs;
76		uint32_t flow_sequence;
77		uint8_t engine_type;
78		uint8_t engine_id;
79		uint16_t layout_version; /* binary layout version */
80		} flow_header_t;
	57	} common_flow_header_t;
81	58
82		typedef struct flow_record {
83		uint32_t srcaddr;
84		uint32_t dstaddr;
85		uint32_t nexthop;
86		uint16_t input;
87		uint16_t output;
88		uint32_t dPkts;
89		uint32_t dOctets;
90		uint32_t First; /* First seen timestamp in UNIX time format. msec offset at end of record */
91		uint32_t Last; /* Last seen timestamp in UNIX time format. msec offset at end of record */
92		uint16_t srcport;
93		uint16_t dstport;
94		uint8_t pad;
95		uint8_t tcp_flags;
96		uint8_t prot;
97		uint8_t tos;
98		uint16_t src_as;
99		uint16_t dst_as;
100		uint16_t msec_first; /* msec offset from First */
101		uint16_t msec_last; /* msec offset from Last */
102		} flow_record_t;
	59	/* buffer size issues */
	60
	61	// 100MB max buffer size when dynamically extending
	62	#define MAX_BUFFER_SIZE 104857600
	63
	64	/* input buffer size, to read data from the network */
	65	#define NETWORK_INPUT_BUFF_SIZE 65535 // Maximum UDP message size
	66
	67	/* output buffer size, tmp buffer, before writing data to the file
	68	* when this buffer is 85% full, it gets written to disk.
	69	* no read cycle must ever produce more output data than it reads from the network
	70	* so 8,5 MB + 1 MB = 9.5MB of 10MB
	71	*/
	72	#define BUFFSIZE 1048576
	73	#define OUTPUT_BUFF_SIZE BUFFSIZE
	74
	75	/* if the output buffer reaches this limit, it gets flushed. This means,
	76	* that 0.5MB input data may produce max 1MB data in output buffer, otherwise
	77	* a buffer overflow may occur, and data does not get processed correctly.
	78	* However, every Process_vx function checks buffer boundaries.
	79	*/
	80	#define OUTPUT_FLUSH_LIMIT BUFFSIZE * 0.8
	81
103	82
104	83	/* prototypes */
105	84
106		void flow_header_raw(void header, uint64_t numflows, uint64_t pkts, uint64_t bytes, char *s, int anon);
	85	void Setv6Mode(int mode);
107	86
108		void flow_record_raw(void record, uint64_t numflows, uint64_t pkts, uint64_t bytes, char *s, int anon);
	87	int Getv6Mode(void);
109	88
110		void flow_record_to_line(void record, uint64_t numflows, uint64_t pkts, uint64_t bytes, char *s, int anon);
	89	int Proto_num(char *protostr);
111	90
112		void flow_record_to_line_long(void record, uint64_t numflows, uint64_t pkts, uint64_t bytes, char *s, int anon);
	91	void format_file_block_header(void header, uint64_t numflows, char *s, int anon);
113	92
114		void flow_record_to_line_extended(void record, uint64_t numflows, uint64_t pkts, uint64_t bytes, char * s, int anon);
	93	void format_file_block_record(void record, uint64_t numflows, char *s, int anon);
115	94
116		void flow_record_to_pipe(void record, uint64_t numflows, uint64_t pkts, uint64_t bytes, char * s, int anon);
	95	void flow_record_to_pipe(void record, uint64_t numflows, char * s, int anon);
	96
	97	int ParseOutputFormat(char *format);
	98
	99	void format_special(void record, uint64_t flows, char * s, int anon);
	100
	101	char *format_special_header(void);
	102
	103	#define FIXED_WIDTH 1
	104	#define VAR_LENGTH 0
117	105
118	106	#ifdef __SUNPRO_C
119	107	extern
120	108	#endif
121		inline int TimeMsec_CMP(time_t t1, uint16_t offset1, time_t t2, uint16_t offset2 );
	109	inline void Proto_string(uint8_t protonum, char *protostr);
122	110
123	111	#ifdef __SUNPRO_C
124	112	extern
125	113	#endif
126		inline void format_number(uint64_t num, char *s);
	114	inline void format_number(uint64_t num, char *s, int fixed_width);
	115
	116	#ifdef __SUNPRO_C
	117	extern
	118	#endif
	119	inline void condense_v6(char *s);

+67

-13

nfcapd.1 less more

10	10	is automatically rotated and renamed every n minutes - typically
11	11	5 min - according the timestamp YYYYMMddhhmm of the interval e.g.
12	12	nfcapd.200407110845 contains the data from July 11th 2004 08:45 onward.
13		Additionally a file named nfcapd.200407110845.stat is created which
14		contains an ASCII readable statistical summary of the data. At the
15		end of every interval you may run any external programm, for example
16		nfprofile, which allows you to process the data.
17	13	.P
18		Netflow version v5 and v7 are transparently supported.
	14	Netflow version v5, v7 and v9 are transparently supported.
19	15
20	16	.SH OPTIONS
21	17	.TP 3
22	18	.B -p \fIportnum
23	19	Specifies the port number to listen. Default port is 9995
24	20	.TP 3
25		.B -b \fIIPaddr
26		Specifies the IP address to bind for listening. Default is any
27		available IPv4 interface.
	21	.B -b \fIbindhost
	22	Specifies the hostname/IPv4/IPv6 address to bind for listening. Can be an IP
	23	address or a hostname, resolving to an IP address attached to an interface.
	24	Defaults to any available IPv4 interface, if not specified.
	25	.TP 3
	26	.B -4
	27	Forces nfcapd to listen on IPv4 addresses only. Can be used together with -b
	28	if a hostname has an IPv4 and IPv6 address record.
	29	.TP 3
	30	.B -6
	31	Forces nfcapd to listen on IPv6 addresses only. Can be used together with -b
	32	if a hostname has an IPv4 and IPv6 address record.
	33	.TP 3
	34	.B -j \fIMulticastGroup
	35	Join the specified IPv4 or IPv6 multicast group for listening.
28	36	.TP 3
29	37	.B -l \fIdirectory
30	38	Specifies the directory to store the output files. Default is /var/tmp

94	102	.SH "EXAMPLES"
95	103	nfcapd -w -D -l /netflow/spool/router1 -p 23456 -B 128000 -I router1 -x '/path/nfprofile -p /to/profile/dir -s router1 -r %d/%f' -P /var/run/nfcapd/nfcapd.router1
96	104	.SH NOTES
97		Setting the socket buffer size is system dependent. When starting up,
98		nfcapd returns the number of bytes the buffer was actually set. This
99		is done by reading back the buffer size and may differ from what you
100		requested.
	105	Even if netflow v9 is support, not all in netflow v9 defined elements
	106	are store in the data files. As of version 1.5 nfdump supports the
	107	following fields:
	108	.PD 0
	109	.RS 4
101	110	.P
102		Nfdump tools version 1.4 support netflow v5 and v7 transparently.
	111	\fBNF9_LAST_SWITCHED\fR
	112	.P
	113	\fBNF9_FIRST_SWITCHED\fR
	114	.P
	115	\fBNF9_IN_BYTES\fR
	116	.P
	117	\fBNF9_IN_PACKETS\fR
	118	.P
	119	\fBNF9_FLOWS\fR
	120	.P
	121	\fBNF9_IN_PROTOCOL\fR
	122	.P
	123	\fBNF9_SRC_TOS\fR
	124	.P
	125	\fBNF9_TCP_FLAGS\fR
	126	.P
	127	\fBNF9_IPV4_SRC_ADDR\fR
	128	.P
	129	\fBNF9_IPV6_SRC_ADDR\fR
	130	.P
	131	\fBNF9_IPV4_DST_ADDR\fR
	132	.P
	133	\fBNF9_IPV6_DST_ADDR\fR
	134	.P
	135	\fBNF9_L4_SRC_PORT\fR
	136	.P
	137	\fBNF9_L4_DST_PORT\fR
	138	.P
	139	\fBNF9_INPUT_SNMP\fR
	140	.P
	141	\fBNF9_OUTPUT_SNMP\fR
	142	.P
	143	\fBNF9_SRC_AS\fR
	144	.P
	145	\fBNF9_DST_AS\fR
	146	.RE
	147	.PD
	148	32 and 64 bit counters are supported for Bytes and Packets.
	149	More fields may be supported in future.
	150	.P
	151	The format of the data files is netflow version independant.
	152	.P
	153	Socket buffer: Setting the socket buffer size is system dependent.
	154	When starting up, nfcapd returns the number of bytes the buffer was
	155	actually set. This is done by reading back the buffer size and may
	156	differ from what you requested.
103	157	.SH "SEE ALSO"
104	158	nfdump(1), nfprofile(1), nfreplay(1)
105	159	.SH BUGS

+301

-491

nfcapd.c less more

30	30	*
31	31	* $Author: peter $
32	32	*
33		* $Id: nfcapd.c 53 2005-11-17 07:45:34Z peter $
	33	* $Id: nfcapd.c 75 2006-05-21 15:32:48Z peter $
34	34	*
35		* $LastChangedRevision: 53 $
	35	* $LastChangedRevision: 75 $
36	36	*
37	37	*
38	38	*/

45	45	* previous datagram plus the number of flows in the previous datagram. After
46	46	* receiving a new datagram, the receiving application can subtract the expected
47	47	* sequence number from the sequence number in the header to derive the number
48		* of missed flows. ⏎
49
	48	* of missed flows.
50	49	*/
51	50
52	51	#include <stdio.h>

55	54	#include <errno.h>
56	55	#include <sys/types.h>
57	56	#include <sys/socket.h>
	57	#include <sys/param.h>
	58	#include <netdb.h>
58	59	#include <pwd.h>
59	60	#include <grp.h>
60	61	#include <unistd.h>

67	68	#include <signal.h>
68	69	#include <syslog.h>
69	70	#include <sys/mman.h>
	71	#include <string.h>
	72	#include <dirent.h>
70	73
71	74	#include "config.h"
72	75

75	78	#endif
76	79
77	80	#include "version.h"
	81	#include "nffile.h"
78	82	#include "nf_common.h"
	83	#include "nfnet.h"
79	84	#include "launch.h"
80		#include "netflow_v5.h"
81		#include "netflow_v7.h"
82
83		#define DEFAULTCISCOPORT 9995
84		#define TIME_WINDOW 300 // The Default Time Window is 5min
85		#define OVERDUE_TIME 20 // Rename File latest, after end of time window
	85	#include "netflow_v5_v7.h"
	86	#include "netflow_v9.h"
	87
	88	/* default path to store data - not really attractive, but anyway ... */
86	89	#define DEFAULT_DIR "/var/tmp"
87	90
88		#define BUFFSIZE 655350
89		#define NF_DUMPFILE "nfcapd.current"
90
91		#define delta(a,b) ( (a)>(b) ? (a)-(b) : (b)-(a) )
	91	#define NF_DUMPFILE "nfcapd.current"
	92
	93	#define DEFAULTCISCOPORT "9995"
	94
	95	/* Default time window in seconds to rotate files */
	96	#define TIME_WINDOW 300
	97
	98	/* overdue time:
	99	* if nfcapd does not get any data, wake up the receive system call
	100	* at least after OVERDUE_TIME seconds after the time window
	101	*/
	102	#define OVERDUE_TIME 20
92	103
93	104	#define SYSLOG_FACILITY LOG_DAEMON
94	105

97	108	int byte_mode, packet_mode;
98	109	caddr_t shmem;
99	110
100		/*
101		* local static vars used by interrupt routine
102		*/
103		static int done, launcher_alive, rename_trigger, launcher_pid, verbose = 0;
104		static char Ident[32];
105
106		static char const *rcsid = "$Id: nfcapd.c 53 2005-11-17 07:45:34Z peter $";
107
108		/* Function Prototypes */
	111	/* globals */
	112	int verbose = 0;
	113
	114
	115	/* module limited globals */
	116	static int done, launcher_alive, rename_trigger, launcher_pid;
	117
	118	static char Ident[IdentLen];
	119
	120	static char const *rcsid = "$Id: nfcapd.c 75 2006-05-21 15:32:48Z peter $";
	121
	122	/* Local function Prototypes */
109	123	static void IntHandler(int signal);
110	124
111	125	static void usage(char *name);
112	126
113	127	static void SetPriv(char userid, char groupid );
114
115		static int Setup_Socket(char *IPAddr, int portnum, int sockbuflen );
116	128
117	129	static void kill_launcher(int pid);
118	130

134	146	} // End of kill_launcher
135	147
136	148
137		static void run(int socket, unsigned int bufflen, time_t twin, time_t t_begin, int report_seq);
	149	static void run(int socket, time_t twin, time_t t_begin, int report_seq);
138	150
139	151	/* Functions */
140	152	static void usage(char *name) {

144	156	"-g groupid\tChange group to groupid\n"
145	157	"-w\t\tSync file rotation with next 5min (default) interval\n"
146	158	"-t interval\tset the interval to rotate nfcapd files\n"
147		"-b ipaddr\tbind socket to IP addr\n"
	159	"-b host\tbind socket to host/IP addr\n"
	160	"-j mcastgroup\tJoin multicast group <mcastgroup>\n"
148	161	"-p portnum\tlisten on port portnum\n"
149	162	"-l logdir \tset the output directory. (default /var/tmp) \n"
150	163	"-I Ident\tset the ident string for stat file. (default 'none')\n"
151	164	"-P pidfile\tset the PID file\n"
152		"-r\t\tReport missing flows to syslog\n"
153	165	"-x process\tlauch process after a new file becomes available\n"
154	166	"-B bufflen\tSet socket buffer to bufflen bytes\n"
155	167	"-D\t\tFork to background\n"
156	168	"-E\t\tPrint extended format of netflow data. for debugging purpose only.\n"
	169	"-4\t\tListen on IPv4 (default).\n"
	170	"-6\t\tListen on IPv6.\n"
157	171	"-V\t\tPrint version and exit.\n"
158	172	, name);
159	173	} /* usage */

217	231
218	232	err = setgid(newgid);
219	233	if ( err ) {
220		syslog(LOG_ERR, "Can't set group id %u for group '%s': %s", newgid, groupid, strerror(errno));
221		fprintf (stderr,"Can't set group id %u for group '%s': %s\n", newgid, groupid, strerror(errno));
	234	syslog(LOG_ERR, "Can't set group id %ld for group '%s': %s", (long)newgid, groupid, strerror(errno));
	235	fprintf (stderr,"Can't set group id %ld for group '%s': %s\n", (long)newgid, groupid, strerror(errno));
222	236	exit(255);
223	237	}
224	238

227	241	if ( newuid ) {
228	242	err = setuid(newuid);
229	243	if ( err ) {
230		syslog(LOG_ERR, "Can't set user id %u for user '%s': %s", newuid, userid, strerror(errno));
231		fprintf (stderr,"Can't set user id %u for user '%s': %s\n", newuid, userid, strerror(errno));
	244	syslog(LOG_ERR, "Can't set user id %ld for user '%s': %s", (long)newuid, userid, strerror(errno));
	245	fprintf (stderr,"Can't set user id %ld for user '%s': %s\n", (long)newuid, userid, strerror(errno));
232	246	exit(255);
233	247	}
234	248	}
235	249
236	250	} // End of SetPriv
237	251
238		static int Setup_Socket(char *IPAddr, int portnum, int sockbuflen ) {
239		struct sockaddr_in server;
240		int s, p;
241		socklen_t optlen;
242
243
244		if ( !portnum )
245		portnum = DEFAULTCISCOPORT;
246
247		s = socket (AF_INET, SOCK_DGRAM, 0);
248		if ( s < 0 ) {
249		fprintf(stderr, "Can't open socket: %s\n", strerror(errno));
250		syslog(LOG_ERR, "Can't open socket: %s", strerror(errno));
251		return -1;
252		}
253
254		memset ((char *) &server, 0, sizeof (server));
255
256		server.sin_addr.s_addr = IPAddr ? inet_addr(IPAddr) : INADDR_ANY;
257		server.sin_family = AF_INET;
258		server.sin_port = htons(portnum);
259
260		if ( (bind (s, (struct sockaddr *)&server, sizeof(server))) < 0 ) {
261		fprintf(stderr, "bind to %s:%i failed: %s\n", inet_ntoa(server.sin_addr), portnum, strerror(errno));
262		syslog(LOG_ERR, "bind to %s:%i failed: %s", inet_ntoa(server.sin_addr), portnum, strerror(errno));
263		close(s);
264		return -1;
265		}
266
267		if ( sockbuflen ) {
268		optlen = sizeof(p);
269		getsockopt(s,SOL_SOCKET,SO_RCVBUF,&p,&optlen);
270		syslog(LOG_INFO,"Standard setsockopt, SO_RCVBUF is %i Requested length is %i bytes",p, sockbuflen);
271		if ((setsockopt(s, SOL_SOCKET, SO_RCVBUF, &sockbuflen, sizeof(sockbuflen)) != 0) ) {
272		fprintf (stderr, "setsockopt(SO_RCVBUF,%d): %s\n", sockbuflen, strerror (errno));
273		syslog (LOG_ERR, "setsockopt(SO_RCVBUF,%d): %s", sockbuflen, strerror (errno));
274		close(s);
275		return -1;
276		} else {
277		getsockopt(s,SOL_SOCKET,SO_RCVBUF,&p,&optlen);
278		syslog(LOG_INFO,"System set setsockopt, SO_RCVBUF to %d bytes", p);
279		}
280		}
281
282		return s;
283
284		} /* End of Setup_Socket */
285
286
287		static void run(int socket, unsigned int bufflen, time_t twin, time_t t_begin, int report_seq) {
288		size_t writesize;
289		ssize_t cnt;
290		netflow_v5_header_t *nf_header_in; // v5/v7 common header
291		netflow_v5_record_t *v5_record;
292		flow_header_t *nf_header_out; // file flow header
293		flow_record_t *nf_record_out; // file flow record
	252	static void run(int socket, time_t twin, time_t t_begin, int report_seq) {
	253	common_flow_header_t *nf_header;
	254	data_block_header_t *data_header;
	255	stat_record_t stat_record;
294	256	time_t t_start, t_now;
295		uint32_t buffsize;
296		uint64_t numflows, numbytes, numpackets;
297		uint64_t numflows_tcp, numflows_udp, numflows_icmp, numflows_other;
298		uint64_t numbytes_tcp, numbytes_udp, numbytes_icmp, numbytes_other;
299		uint64_t numpackets_tcp, numpackets_udp, numpackets_icmp, numpackets_other;
300		uint64_t start_time, end_time, boot_time, first_seen, last_seen;
301		uint32_t First, Last, sequence_failure, bad_packets;
302		int64_t last_sequence, sequence, distance, last_count;
	257	uint64_t first_seen, last_seen, export_packets;
	258	uint32_t bad_packets, file_blocks, blast_cnt, blast_failures;
	259	uint16_t version;
303	260	struct tm *now;
304		void in_buff, out_buff, p, q;
305		int i, err, nffd, header_length, record_length, first;
306		char *string, tmpstring[64];
	261	ssize_t cnt;
	262	void in_buff, out_buff, *writeto;
	263	int err, nffd, first;
	264	char *string, nfcapd_filename[64], dumpfile[64];
307	265	srecord_t *commbuff;
308
309		if ( !bufflen \|\| bufflen < BUFFSIZE )
310		bufflen = BUFFSIZE;
311
312		in_buff = malloc(bufflen);
313		out_buff = malloc(bufflen);
	266
	267	Init_v5_v7_input();
	268	Init_v9();
	269
	270	in_buff = malloc(NETWORK_INPUT_BUFF_SIZE);
	271	out_buff = malloc(OUTPUT_BUFF_SIZE);
314	272	if ( !in_buff \|\| !out_buff ) {
315	273	syslog(LOG_ERR, "Buffer allocation error: %s", strerror(errno));
316	274	return;

318	276
319	277	// init vars
320	278	commbuff = (srecord_t *)shmem;
321
322		p = in_buff;
323		q = out_buff;
324		cnt = 0;
325		nf_header_in = NULL;
326		nf_header_out = NULL;
327		v5_record = NULL;
328		nf_record_out = NULL;
329		header_length = NETFLOW_V5_HEADER_LENGTH; // v5 and v7 have same length
330
331		nffd = open(NF_DUMPFILE, O_CREAT \| O_RDWR, S_IRUSR \| S_IWUSR \| S_IRGRP \| S_IROTH );
332		if ( nffd == -1 ) {
333		syslog(LOG_ERR, "Can't open file: '%s': %s", NF_DUMPFILE, strerror(errno));
	279	nf_header = (common_flow_header_t *)in_buff;
	280
	281	// Init header
	282	data_header = (data_block_header_t *)out_buff;
	283	data_header->NumBlocks = 0;
	284	data_header->size = 0;
	285	data_header->id = DATA_BLOCK_TYPE_1;
	286	data_header->pad = 0;
	287	writeto = (void *)((pointer_addr_t)data_header + sizeof(data_block_header_t) );
	288
	289	cnt = 0;
	290	export_packets = blast_cnt = blast_failures = 0;
	291
	292	snprintf(dumpfile, 63, "%s.%u",NF_DUMPFILE, getpid());
	293	dumpfile[63] = 0;
	294	nffd = OpenNewFile(dumpfile, &string);
	295	if ( string != NULL ) {
	296	syslog(LOG_ERR, "%s", string);
334	297	return;
335	298	}
336	299
337	300	bad_packets = 0;
338	301
339	302	// init sequence check vars
340		last_sequence = 0;
341		sequence = 0;
342		distance = 0;
343		last_count = 0;
344		sequence_failure = 0;
345	303	first = 1;
	304	file_blocks = 0;
346	305
347	306	first_seen = (uint64_t)0xffffffffffffLL;
348	307	last_seen = 0;
349	308	t_start = t_begin;
350		numflows = numbytes = numpackets = buffsize = 0;
351		numflows_tcp = numflows_udp = numflows_icmp = numflows_other = 0;
352		numbytes_tcp = numbytes_udp = numbytes_icmp = numbytes_other = 0;
353		numpackets_tcp = numpackets_udp = numpackets_icmp = numpackets_other = 0;
354
	309	memset((void *)&stat_record, 0, sizeof(stat_record_t));
	310
	311	cnt = 0;
355	312	rename_trigger = 0;
356	313	alarm(t_start + twin + OVERDUE_TIME - time(NULL));
357		// convert all v7 records to v5 records while processing them
358		// this ignores the router_sc field in v7
359		while ( !done ) {
360		/* check for too little data */
361		if ( buffsize > 0 && buffsize < header_length ) {
362		syslog(LOG_WARNING, "Data length error: too little data for netflow v5 header: %i", buffsize);
363		buffsize = 0;
364		p = in_buff;
365		q = out_buff;
366		bad_packets++;
367		continue;
368		}
	314	/*
	315	* Main processing loop:
	316	* this loop, continues until done = 1, set by the signal handler
	317	* The while loop will be breaked by the periodic file renaming code
	318	* for proper cleanup
	319	*/
	320	while ( 1 ) {
	321
369	322	/* read next bunch of data into beginn of input buffer */
370		if ( buffsize == 0 ) {
371		cnt = recv (socket, in_buff, bufflen , 0);
372		buffsize = cnt > 0 ? cnt : 0;
373		p = in_buff;
374		q = out_buff;
375		/* check for too little data */
376		if ( buffsize > 0 && buffsize < header_length ) {
377		syslog(LOG_WARNING, "Data length error: too little data for netflow header: %i", buffsize);
378		buffsize = 0;
379		p = in_buff;
380		q = out_buff;
381		bad_packets++;
	323	if ( !done) {
	324	cnt = recvfrom (socket, in_buff, NETWORK_INPUT_BUFF_SIZE , 0, NULL, 0);
	325	if ( cnt < 0 && errno != EINTR ) {
	326	syslog(LOG_ERR, "ERROR: recvfrom: %s", strerror(errno));
382	327	continue;
383	328	}
384	329	}
385		/* paranoia check */
386		if ( ( (pointer_addr_t)p - (pointer_addr_t)in_buff ) > bufflen ) {
387		/* should never happen, but catch it anyway */
388		syslog(LOG_ERR, "Buffer space error");
389		buffsize = 0;
390		p = in_buff;
391		q = out_buff;
392		continue;
393		}
394
395		/* File renaming */
	330
	331	/* Periodic file renaming, if time limit reached or we are done. */
396	332	t_now = time(NULL);
397	333	if ( ((t_now - t_start) >= twin) \|\| done ) {
398	334	alarm(0);
399	335	now = localtime(&t_start);
400		snprintf(tmpstring, 64, "nfcapd.%i%02i%02i%02i%02i",
	336
	337	if ( verbose ) {
	338	// Dump to stdout
	339	format_file_block_header(out_buff, 0, &string, 0);
	340	}
	341
	342	if ( data_header->NumBlocks ) {
	343	// flush current buffer to disc
	344	if ( write(nffd, out_buff, sizeof(data_block_header_t) + data_header->size) <= 0 )
	345	syslog(LOG_ERR, "Failed to write output buffer to disk: '%s'" , strerror(errno));
	346	else
	347	// update successful written blocks
	348	file_blocks++;
	349	}
	350
	351	// Initialize header and write pointer
	352	data_header->NumBlocks = 0;
	353	data_header->size = 0;
	354	writeto = (void *)((pointer_addr_t)data_header + sizeof(data_block_header_t) );
	355
	356	snprintf(nfcapd_filename, 64, "nfcapd.%i%02i%02i%02i%02i",
401	357	now->tm_year + 1900, now->tm_mon + 1, now->tm_mday, now->tm_hour, now->tm_min);
402		close(nffd);
403		err = rename(NF_DUMPFILE, tmpstring);
	358
	359	t_start += twin;
	360	alarm(t_start + twin + OVERDUE_TIME - t_now);
	361
	362	stat_record.first_seen = first_seen/1000;
	363	stat_record.msec_first = first_seen - stat_record.first_seen*1000;
	364	stat_record.last_seen = last_seen/1000;
	365	stat_record.msec_last = last_seen - stat_record.last_seen*1000;
	366
	367	/* Write Stat Info */
	368	CloseUpdateFile(nffd, &stat_record, file_blocks, Ident, &string );
	369	if ( string != NULL ) {
	370	syslog(LOG_ERR, "%s", string);
	371	}
	372
	373	err = rename(dumpfile, nfcapd_filename);
404	374	if ( err ) {
405	375	syslog(LOG_ERR, "Can't rename dump file: %s", strerror(errno));
406		continue;
407		}
	376	if (done) break; else continue;
	377	}
	378
408	379	if ( launcher_pid ) {
409		strncpy(commbuff->fname, tmpstring, FNAME_SIZE);
	380	// Signal launcher
	381	strncpy(commbuff->fname, nfcapd_filename, FNAME_SIZE);
410	382	commbuff->fname[FNAME_SIZE-1] = 0;
411	383	snprintf(commbuff->tstring, 16, "%i%02i%02i%02i%02i",
412	384	now->tm_year + 1900, now->tm_mon + 1, now->tm_mday, now->tm_hour, now->tm_min);
413	385	commbuff->tstring[15] = 0;
414	386	commbuff->tstamp = t_start;
415		}
416		snprintf(tmpstring, 64, "nfcapd.%i%02i%02i%02i%02i.stat",
417		now->tm_year + 1900, now->tm_mon + 1, now->tm_mday, now->tm_hour, now->tm_min);
418		nffd = open(tmpstring, O_CREAT \| O_RDWR, S_IRUSR \| S_IWUSR \| S_IRGRP \| S_IROTH );
419		if ( nffd == -1 ) {
420		syslog(LOG_ERR, "Can't open stat file: %s", strerror(errno));
421		return;
422		}
423
424		/* Statfile */
425		#if defined __OpenBSD__ \|\| defined __FreeBSD__
426		snprintf(tmpstring, 64, "Time: %i\n", t_start);
427		#else
428		snprintf(tmpstring, 64, "Time: %li\n", t_start);
429		#endif
430		write(nffd, tmpstring, strlen(tmpstring));
431		// t_start = t_now - ( t_now % twin);
432		t_start += twin;
433		alarm(t_start + twin + OVERDUE_TIME - t_now);
434
435		snprintf(tmpstring, 64, "Ident: %s\n", Ident);
436		write(nffd, tmpstring, strlen(tmpstring));
437		snprintf(tmpstring, 64, "Flows: %llu\n", numflows);
438		write(nffd, tmpstring, strlen(tmpstring));
439		snprintf(tmpstring, 64, "Flows_tcp: %llu\n", numflows_tcp);
440		write(nffd, tmpstring, strlen(tmpstring));
441		snprintf(tmpstring, 64, "Flows_udp: %llu\n", numflows_udp);
442		write(nffd, tmpstring, strlen(tmpstring));
443		snprintf(tmpstring, 64, "Flows_icmp: %llu\n", numflows_icmp);
444		write(nffd, tmpstring, strlen(tmpstring));
445		snprintf(tmpstring, 64, "Flows_other: %llu\n", numflows_other);
446		write(nffd, tmpstring, strlen(tmpstring));
447		snprintf(tmpstring, 64, "Packets: %llu\n", numpackets);
448		write(nffd, tmpstring, strlen(tmpstring));
449		snprintf(tmpstring, 64, "Packets_tcp: %llu\n", numpackets_tcp);
450		write(nffd, tmpstring, strlen(tmpstring));
451		snprintf(tmpstring, 64, "Packets_udp: %llu\n", numpackets_udp);
452		write(nffd, tmpstring, strlen(tmpstring));
453		snprintf(tmpstring, 64, "Packets_icmp: %llu\n", numpackets_icmp);
454		write(nffd, tmpstring, strlen(tmpstring));
455		snprintf(tmpstring, 64, "Packets_other: %llu\n", numpackets_other);
456		write(nffd, tmpstring, strlen(tmpstring));
457		snprintf(tmpstring, 64, "Bytes: %llu\n", numbytes);
458		write(nffd, tmpstring, strlen(tmpstring));
459		snprintf(tmpstring, 64, "Bytes_tcp: %llu\n", numbytes_tcp);
460		write(nffd, tmpstring, strlen(tmpstring));
461		snprintf(tmpstring, 64, "Bytes_udp: %llu\n", numbytes_udp);
462		write(nffd, tmpstring, strlen(tmpstring));
463		snprintf(tmpstring, 64, "Bytes_icmp: %llu\n", numbytes_icmp);
464		write(nffd, tmpstring, strlen(tmpstring));
465		snprintf(tmpstring, 64, "Bytes_other: %llu\n", numbytes_other);
466		write(nffd, tmpstring, strlen(tmpstring));
467		snprintf(tmpstring, 64, "First: %llu\n", first_seen/1000LL);
468		write(nffd, tmpstring, strlen(tmpstring));
469		snprintf(tmpstring, 64, "Last: %llu\n", last_seen/1000LL);
470		write(nffd, tmpstring, strlen(tmpstring));
471		close(nffd);
472		syslog(LOG_INFO,"Ident: '%s' Flows: %llu, Packets: %llu, Bytes: %llu, Sequence Errors: %u, Bad Packets: %u",
473		Ident, numflows, numpackets, numbytes, sequence_failure, bad_packets);
474		if ( launcher_pid ) {
	387
475	388	if ( launcher_alive ) {
476	389	syslog(LOG_DEBUG, "Signal launcher");
477	390	kill(launcher_pid, SIGHUP);

479	392	syslog(LOG_ERR, "ERROR: Launcher did unexpectedly!");
480	393	}
481	394	}
482		numflows = numbytes = numpackets = 0;
483		sequence_failure = bad_packets = 0;
484		numflows_tcp = numflows_udp = numflows_icmp = numflows_other = 0;
485		numbytes_tcp = numbytes_udp = numbytes_icmp = numbytes_other = 0;
486		numpackets_tcp = numpackets_udp = numpackets_icmp = numpackets_other = 0;
487		first_seen = 0xffffffffffffLL;
488		last_seen = 0;
489
490		nffd = open(NF_DUMPFILE, O_CREAT \| O_RDWR, S_IRUSR \| S_IWUSR \| S_IRGRP \| S_IROTH );
491		if ( nffd == -1 ) {
492		syslog(LOG_ERR, "Can't open dump file: %s", strerror(errno));
	395
	396	syslog(LOG_INFO,"Ident: '%s' Flows: %llu, Packets: %llu, Bytes: %llu, Sequence Errors: %u, Bad Packets: %u",
	397	Ident, stat_record.numflows, stat_record.numpackets, stat_record.numbytes, stat_record.sequence_failure, bad_packets);
	398
	399	memset((void *)&stat_record, 0, sizeof(stat_record_t));
	400	bad_packets = 0;
	401	first_seen = 0xffffffffffffLL;
	402	last_seen = 0;
	403	file_blocks = 0;
	404
	405	if ( done )
	406	break;
	407
	408	nffd = OpenNewFile(dumpfile, &string);
	409	if ( string != NULL ) {
	410	syslog(LOG_ERR, "%s", string);
493	411	return;
494	412	}
495		}
496
497		/* check for error condition or done */
	413
	414	}
	415
	416	/* check for error condition or done . errno may only be EINTR */
498	417	if ( cnt < 0 ) {
499	418	if ( rename_trigger ) {
500	419	rename_trigger = 0;
501	420	continue;
502	421	}
503	422	if ( done )
504		break;
	423	continue;
505	424	else {
506		syslog(LOG_ERR, "Data receive error while expecting header data: %s", strerror(errno));
507		p = in_buff;
508		q = out_buff;
	425	/* this should never be executed as it should be caught in other places */
	426	syslog(LOG_ERR, "error condition in '%s', line '%d', cnt: %i", __FILE__, __LINE__ ,cnt);
	427	if (done) break; else continue;
	428	}
	429	}
	430
	431	/* enough data? */
	432	if ( cnt == 0 )
	433	continue;
	434
	435	/* check for too little data - cnt must be > 0 at this point */
	436	if ( cnt < sizeof(common_flow_header_t) ) {
	437	syslog(LOG_WARNING, "Data length error: too little data for common netflow header. cnt: %i",
	438	cnt);
	439	bad_packets++;
	440	continue;
	441	}
	442
	443
	444	/* Process data - have a look at the common header */
	445	version = ntohs(nf_header->version);
	446	switch (version) {
	447	case 5: // fall through
	448	case 7:
	449	writeto = Process_v5_v7(in_buff, cnt, data_header, writeto, &stat_record, &first_seen, &last_seen);
	450	break;
	451	case 9:
	452	writeto = Process_v9(in_buff, cnt, data_header, writeto, &stat_record, &first_seen, &last_seen);
	453	break;
	454	case 255:
	455	// blast test header
	456	if ( verbose ) {
	457	uint16_t count = ntohs(nf_header->count);
	458	if ( blast_cnt != count ) {
	459	// fprintf(stderr, "Missmatch blast check: Expected %u got %u\n", blast_cnt, count);
	460	blast_cnt = count;
	461	blast_failures++;
	462	} else {
	463	blast_cnt++;
	464	}
	465	if ( blast_cnt == 65535 ) {
	466	fprintf(stderr, "Total missed packets: %u\n", blast_failures);
	467	done = 1;
	468	}
	469	break;
	470	}
	471	default:
	472	// data error, while reading data from socket
	473	syslog(LOG_ERR,"Error reading netflow header: Unexpected netflow version %i", nf_header->version);
509	474	bad_packets++;
510	475	continue;
511		}
512		}
513
514		/* Process header */
515		nf_header_in = (netflow_v5_header_t *)p;
516		nf_header_out = (flow_header_t *)q;
517
518		nf_header_out->version = ntohs(nf_header_in->version);
519		nf_header_out->count = ntohs(nf_header_in->count);
520		nf_header_out->SysUptime = ntohl(nf_header_in->SysUptime);
521		nf_header_out->unix_secs = ntohl(nf_header_in->unix_secs);
522		nf_header_out->unix_nsecs = ntohl(nf_header_in->unix_nsecs);
523		nf_header_out->flow_sequence = ntohl(nf_header_in->flow_sequence);
524		nf_header_out->layout_version = 1;
525
526		// check version and set appropriate params
527		switch (nf_header_out->version) {
528		case 5:
529		record_length = NETFLOW_V5_RECORD_LENGTH;
530		if ( nf_header_out->count == 0 \|\| nf_header_out->count > NETFLOW_V5_MAX_RECORDS ) {
531		syslog(LOG_ERR,"Error netflow header: Unexpected number of records in v5 header: %i.", nf_header_out->count);
532		buffsize = 0;
533		p = in_buff;
534		q = out_buff;
535		bad_packets++;
536		continue;
537		}
538		break;
539		case 7:
540		record_length = NETFLOW_V7_RECORD_LENGTH;
541		if ( nf_header_out->count == 0 \|\| nf_header_out->count > NETFLOW_V7_MAX_RECORDS ) {
542		syslog(LOG_ERR,"Error netflow header: Unexpected number of records in v7 header: %i.", nf_header_out->count);
543		buffsize = 0;
544		p = in_buff;
545		q = out_buff;
546		bad_packets++;
547		continue;
548		}
549		nf_header_out->version = 5;
550		break;
551		default:
552		// force data error, when reading data from socket
553		record_length = 0;
554		syslog(LOG_ERR,"Error netflow header: Unexpected netflow version %i, found.", nf_header_out->version);
555		buffsize = 0;
556		p = in_buff;
557		q = out_buff;
558		bad_packets++;
559		continue;
560		break;
561		}
562
563		if ( first ) {
564		last_sequence = nf_header_out->flow_sequence;
565		sequence = last_sequence;
566		first = 0;
567		} else {
568		last_sequence = sequence;
569		sequence = nf_header_out->flow_sequence;
570		distance = sequence - last_sequence;
571		// handle overflow
572		if (distance < 0) {
573		distance = 0xffffffff + distance +1;
574		}
575		if (distance != last_count) {
576		sequence_failure++;
577		if ( report_seq )
578		syslog(LOG_ERR,"Flow sequence mismatch. Missing: %lli flows", delta(last_count,distance));
579		}
580		}
581		last_count = nf_header_out->count;
582
583		if ( verbose ) {
584		flow_header_raw(nf_header_out, 0, 0, 0, &string, 0);
585		printf("%s", string);
586		}
587
588		/* advance buffer */
589		p = (void *)((pointer_addr_t)p + header_length);
590		q = (void *)((pointer_addr_t)q + header_length);
591		buffsize -= header_length;
592
593		/* calculate boot time in msec */
594		boot_time = ((uint64_t)(nf_header_out->unix_secs)*1000 +
595		((uint64_t)(nf_header_out->unix_nsecs) / 1000000) ) - (uint64_t)(nf_header_out->SysUptime);
596
597		/* records associated with the header */
598		for (i = 0; i < nf_header_out->count; i++) {
599		/* make sure enough data is available in the buffer */
600		if ( buffsize > 0 && buffsize < record_length ) {
601		syslog(LOG_WARNING, "Data length error: too little data for netflow record!");
602		buffsize = 0;
603		p = in_buff;
604		q = out_buff;
605		break;
606		}
607		if ( buffsize == 0 ) {
608		/* read next bunch of data - append to end of data already read */
609		cnt = recv (socket, p, (pointer_addr_t)bufflen - ((pointer_addr_t)p - (pointer_addr_t)in_buff), 0);
610		buffsize = cnt;
611		if ( cnt < 0 ) {
612		if ( done ) {
613		break;
614		} else {
615		syslog(LOG_ERR, "Data receive error while expecting record data: %s", strerror(errno));
616		break;
617		}
618		}
619		/* make sure enough data is available in the buffer */
620		if ( buffsize > 0 && ((buffsize < record_length) \|\| (record_length == 0)) ) {
621		syslog(LOG_WARNING, "Data length error: too little data for v5 record: %i", buffsize);
622		buffsize = 0;
623		p = in_buff;
624		q = out_buff;
625		break;
626		}
627		}
628		if ( ( (pointer_addr_t)p - (pointer_addr_t)in_buff ) > bufflen ) {
629		/* should never happen, but catch it anyway */
630		syslog(LOG_ERR, "Buffer space error");
631		buffsize = 0;
632		p = in_buff;
633		q = out_buff;
634		break;
635		}
636		if ( buffsize < record_length ) {
637		syslog(LOG_WARNING, "Data read error: Too little data for v5 record");
638		buffsize = 0;
639		p = in_buff;
640		q = out_buff;
641		break;
642		}
643
644		// process record
645		// whatever netflow version it is ( v5 or v7 allowed here ) both
646		// version have the same common fields below
647		// fields other than those assigned below or ignored, and not
648		// not relevant for nfdump
649		v5_record = (netflow_v5_record_t *)p;
650		nf_record_out = (flow_record_t *)q;
651		nf_record_out->srcaddr = ntohl(v5_record->srcaddr);
652		nf_record_out->dstaddr = ntohl(v5_record->dstaddr);
653		nf_record_out->nexthop = ntohl(v5_record->nexthop);
654		nf_record_out->input = ntohs(v5_record->input);
655		nf_record_out->output = ntohs(v5_record->output);
656		nf_record_out->dPkts = ntohl(v5_record->dPkts);
657		nf_record_out->dOctets = ntohl(v5_record->dOctets);
658		First = ntohl(v5_record->First);
659		Last = ntohl(v5_record->Last);
660		nf_record_out->srcport = ntohs(v5_record->srcport);
661		nf_record_out->dstport = ntohs(v5_record->dstport);
662		nf_record_out->pad = 0;
663		nf_record_out->tcp_flags = v5_record->tcp_flags;
664		nf_record_out->prot = v5_record->prot;
665		nf_record_out->tos = v5_record->tos;
666		nf_record_out->src_as = ntohs(v5_record->src_as);
667		nf_record_out->dst_as = ntohs(v5_record->dst_as);
668
669		switch (nf_record_out->prot) {
670		case 1:
671		numflows_icmp++;
672		numpackets_icmp += nf_record_out->dPkts;
673		numbytes_icmp += nf_record_out->dOctets;
674		break;
675		case 6:
676		numflows_tcp++;
677		numpackets_tcp += nf_record_out->dPkts;
678		numbytes_tcp += nf_record_out->dOctets;
679		break;
680		case 17:
681		numflows_udp++;
682		numpackets_udp += nf_record_out->dPkts;
683		numbytes_udp += nf_record_out->dOctets;
684		break;
685		default:
686		numflows_other++;
687		numpackets_other += nf_record_out->dPkts;
688		numbytes_other += nf_record_out->dOctets;
689		}
690		numflows++;
691		numpackets += nf_record_out->dPkts;
692		numbytes += nf_record_out->dOctets;
693
694		p = (void *)((pointer_addr_t)p + record_length);
695		q = (void *)((pointer_addr_t)q + NETFLOW_V5_RECORD_LENGTH);
696		buffsize -= record_length;
697
698		/* start time in msecs */
699		start_time = (uint64_t)First + boot_time;
700
701		if ( First > Last )
702		/* Last in msec, in case of msec overflow, between start and end */
703		end_time = 0x100000000LL + Last + boot_time;
704		else
705		end_time = (uint64_t)Last + boot_time;
706
707		nf_record_out->First = start_time/1000;
708		nf_record_out->msec_first = start_time - nf_record_out->First*1000;
709
710		nf_record_out->Last = end_time/1000;
711		nf_record_out->msec_last = end_time - nf_record_out->Last*1000;
712
713		if ( start_time < first_seen )
714		first_seen = start_time;
715		if ( end_time > last_seen )
716		last_seen = end_time;
717
718		if ( verbose ) {
719		flow_record_raw(nf_record_out, 1, (uint64_t)nf_record_out->dPkts, (uint64_t)nf_record_out->dOctets, &string, 0);
720		printf("%s\n", string);
721		}
722		}
723		if ( i != nf_header_out->count ) {
724		syslog(LOG_WARNING, "Expected %i records, but found only %i", nf_header_out->count, i);
725		nf_header_out->count = i;
726		}
727		writesize = header_length + nf_header_out->count * NETFLOW_V5_RECORD_LENGTH;
728		if ( write(nffd, (void *)nf_header_out, writesize) <= 0 ) {
729		syslog(LOG_ERR, "Failed to write records to disk: '%s'" , strerror(errno));
730		}
	476
	477	// not reached
	478	break;
	479	}
	480	// each Process_xx function has to process the entire input buffer, therefore it's empty now.
	481	export_packets++;
	482
	483	// flush current buffer to disc
	484	if ( data_header->size > BUFFSIZE ) {
	485	syslog(LOG_ERR, "output buffer overflow: expect memory inconsitency");
	486	}
	487	if ( data_header->size > OUTPUT_FLUSH_LIMIT ) {
	488	if ( write(nffd, out_buff, sizeof(data_block_header_t) + data_header->size) <= 0 ) {
	489	syslog(LOG_ERR, "Failed to write output buffer to disk: '%s'" , strerror(errno));
	490	} else {
	491	data_header->size = 0;
	492	data_header->NumBlocks = 0;
	493	writeto = (void *)((pointer_addr_t)data_header + sizeof(data_block_header_t) );
	494	file_blocks++;
	495	}
	496	}
	497	}
	498
	499	if ( verbose && blast_failures ) {
	500	fprintf(stderr, "Total missed packets: %u\n", blast_failures);
731	501	}
732	502	free(in_buff);
733	503	free(out_buff);
734	504	close(nffd);
	505	unlink(dumpfile);
735	506
736	507	} /* End of run */
737	508
738	509	int main(int argc, char **argv) {
739	510
740		char bindaddr, pidfile, filter, datadir, pidstr[32], *lauch_process;
741		char userid, groupid, *checkptr;
	511	char bindhost, filter, datadir, pidstr[32], lauch_process;
	512	char userid, groupid, checkptr, listenport, *mcastgroup;
	513	char pidfile[MAXNAMLEN];
742	514	struct stat fstat;
743	515	srecord_t *commbuff;
744	516	struct sigaction act;
745		int bufflen;
	517	int family, bufflen;
746	518	time_t twin, t_start, t_tmp;
747		int portnum, sock, pidf, fd, err, synctime, daemonize, report_sequence;
	519	int sock, pidf, fd, err, synctime, daemonize, report_sequence;
748	520	char c;
749	521	pid_t pid;
750	522
751		portnum = verbose = synctime = daemonize = 0;
	523	verbose = synctime = daemonize = 0;
752	524	bufflen = 0;
	525	family = AF_UNSPEC;
753	526	launcher_pid = 0;
754	527	launcher_alive = 0;
755	528	report_sequence = 0;
756		bindaddr = NULL;
757		pidfile = NULL;
	529	listenport = DEFAULTCISCOPORT;
	530	bindhost = NULL;
	531	mcastgroup = NULL;
	532	pidfile[0] = 0;
758	533	filter = NULL;
759	534	lauch_process = NULL;
760	535	userid = groupid = NULL;
761	536	twin = TIME_WINDOW;
762	537	datadir = DEFAULT_DIR;
763	538	strncpy(Ident, "none", IDENT_SIZE);
764		while ((c = getopt(argc, argv, "whEVI:DB:b:l:p:P:t:x:ru:g:")) != EOF) {
	539	while ((c = getopt(argc, argv, "46whEVI:DB:b:j:l:p:P:t:x:ru:g:")) != EOF) {
765	540	switch (c) {
766	541	case 'h':
767	542	usage(argv[0]);

801	576	fprintf(stderr,"Argument error for -B\n");
802	577	exit(255);
803	578	case 'b':
804		bindaddr = optarg;
	579	bindhost = optarg;
	580	break;
	581	case 'j':
	582	mcastgroup = optarg;
805	583	break;
806	584	case 'p':
807		portnum = strtol(optarg, &checkptr, 10);
808		if ( (checkptr != NULL && *checkptr == 0) && portnum > 0 )
809		break;
810		fprintf(stderr,"Argument error for -p\n");
811		exit(255);
	585	listenport = optarg;
812	586	break;
813	587	case 'P':
814		pidfile = optarg;
	588	if ( optarg[0] == '/' ) { // absolute path given
	589	strncpy(pidfile, optarg, MAXNAMLEN-1);
	590	} else { // path relative to current working directory
	591	char tmp[MAXPATHLEN];
	592	if ( !getcwd(tmp, MAXPATHLEN-1) ) {
	593	fprintf(stderr, "Failed to get current working directory: %s\n", strerror(errno));
	594	exit(255);
	595	}
	596	tmp[MAXPATHLEN-1] = 0;
	597	snprintf(pidfile, MAXPATHLEN - 1 - strlen(tmp), "%s/%s", tmp, optarg);
	598	}
	599	// pidfile now absolute path
	600	pidfile[MAXNAMLEN-1] = 0;
815	601	break;
816	602	case 'r':
817	603	report_sequence = 1;

837	623	case 'x':
838	624	lauch_process = optarg;
839	625	break;
	626	case '4':
	627	if ( family == AF_UNSPEC )
	628	family = AF_INET;
	629	else {
	630	fprintf(stderr, "ERROR, Accepts only one protocol IPv4 or IPv6!\n");
	631	exit(255);
	632	}
	633	break;
	634	case '6':
	635	if ( family == AF_UNSPEC )
	636	family = AF_INET6;
	637	else {
	638	fprintf(stderr, "ERROR, Accepts only one protocol IPv4 or IPv6!\n");
	639	exit(255);
	640	}
	641	break;
840	642	default:
841	643	usage(argv[0]);
842	644	exit(255);
843	645	}
844	646	}
	647
	648	if ( bindhost && mcastgroup ) {
	649	fprintf(stderr, "ERROR, -b and -j are mutually exclusive!!\n");
	650	exit(255);
	651	}
845	652
846	653	openlog(argv[0] , LOG_CONS\|LOG_PID, SYSLOG_FACILITY);
847	654
848	655	SetPriv(userid, groupid);
849	656
850		if ( pidfile ) {
	657	if ( strlen(pidfile) ) {
851	658	pidf = open(pidfile, O_CREAT\|O_RDWR, 0644);
852	659	if ( pidf == -1 ) {
853		syslog(LOG_ERR, "Error opening pid file '%s': %s", pidfile, strerror(errno));
854		fprintf(stderr,"Terminated due to errors.\n");
	660	fprintf(stderr, "Error opening pid file '%s': %s\n", pidfile, strerror(errno));
855	661	exit(255);
856	662	}
857	663	pid = getpid();

907	713	filter = argv[optind];
908	714	}
909	715
910		sock = Setup_Socket(bindaddr, portnum, bufflen );
	716	if ( mcastgroup )
	717	sock = Multicast_receive_socket (mcastgroup, listenport, family, bufflen);
	718	else
	719	sock = Unicast_receive_socket(bindhost, listenport, family, bufflen );
	720
911	721	if ( sock == -1 ) {
912	722	kill_launcher(launcher_pid);
913	723	fprintf(stderr,"Terminated due to errors.\n");

925	735	if ((pid = fork()) < 0 ) {
926	736	perror("Can't fork()");
927	737	} else if (pid) {
928		if (pidfile) {
	738	if (strlen(pidfile)) {
929	739	pidf = open(pidfile, O_CREAT\|O_RDWR, 0644);
930	740	if ( pidf == -1 ) {
931	741	syslog(LOG_ERR, "Error opening pid file: '%s' %s", pidfile, strerror(errno));

950	760	close(2);
951	761	fd = open("/dev/null",O_RDWR); /* open stdin */
952	762	dup(fd); /* stdout */
953		dup(fd); /* stdout */
	763	dup(fd); /* stderr */
954	764	}
955	765
956	766	if ( chdir(datadir)) {

972	782	sigaction(SIGCHLD, &act, NULL);
973	783
974	784	syslog(LOG_INFO, "Startup.");
975		run(sock, bufflen, twin, t_start, report_sequence);
	785	run(sock, twin, t_start, report_sequence);
976	786	close(sock);
977	787	syslog(LOG_INFO, "Terminating nfcapd.");
978	788
979		if ( pidfile )
	789	if ( strlen(pidfile) )
980	790	unlink(pidfile);
981	791
982	792	closelog();
983	793	kill_launcher(launcher_pid);
984		exit(0);
	794	return 0;
985	795
986	796	} /* End of main */

+213

-49

nfdump.1 less more

17	17	Read input data from \fIinputfile\fR. Default is read from stdin.
18	18	.TP 3
19	19	.B -R \fIexpr
20		Read input from a sequence of concatenated files. The \fIexpr\fR
	20	Read input from a sequence of files in the same directory. \fIexpr\fR
21	21	may be one of:
22	22	.PD 0
23	23	.RS 4

30	30	.PD
31	31	.TP 3
32	32	.B -M \fIexpr
33		Read input from multiple directories. The \fIexpr\fR looks like:
	33	Read input from multiple directories. \fIexpr\fR looks like:
34	34	\fI/any/path/to/dir1:dir2:dir3\fR etc. and will be expanded to the
35	35	directories: \fI/any/path/to/dir1\fR, \fI/any/path/to/dir2\fR and
36	36	\fI/any/path/to/dir3\fR Any number of colon separated directories may

58	58	the time spec may be omitted e.g YYYY/MM/dd expands to
59	59	YYYY/MM/dd.00:00:00-YYYY/MM/dd.23:59:59 and processes all flow from a
60	60	given day. The time window may also be specified as +/- n. In this case
61		it is relative to the beginning or end of all flows. +10 means the first
	61	it is relativ to the beginning or end of all flows. +10 means the first
62	62	10 seconds of all flows, -10 means the last 10 seconds of all flows.
63	63	.TP 3
64	64	.B -c \fInum

67	67	.B -a
68	68	Aggregate netflow data. Aggregation is done at connection level.
69	69	.TP 3
70		.B -A \fIfields
	70	.B -A \fIfields[/netmask]
71	71	Aggregate netflow data using the specified fields, where fields is a ','
72	72	separated list out of \fIsrcip dstip srcport dstport\fR. The default is using
73		all fields: \fIsrcip,dstip,srcport,dstport\fR. Only flows of the same protocol
74		are aggregated.
	73	all fields: \fIsrcip,dstip,srcport,dstport\fR. An additional netmask may be
	74	given. In that case flows from the same subnets are aggregated. In order
	75	to do proper aggregation, the IP version is important, for which the mask
	76	applies. Therefore the IP protocol version must be given in the form of:
	77	srcip4/24 for IPv4 or srcip6/64 for IPv6 address aggregation. Apply the
	78	protocol version for dstip respectively.
	79	Only flows of the same IP protocol tcp, udp, icmp etc. are aggregated.
	80	.TP 3
	81	.B -I
	82	Print flow statistics from file specified by -r, or timeslot specified by -R/-M.
	83	The printed information corresponds to pre nfdump 1.5 nfcapd stat files.
75	84	.TP 3
76	85	.B -S
77	86	Compatibility option with pre 1.4 nfdump. Is equal to \fI-s record/packets/bytes\fR.
78	87	.TP 3
79		.B -s \fIstatistic[/orderby]
	88	.B -s \fIstatistic[:p][/orderby]
80	89	Generate the Top N flow or flow element statistic. \fIstatistic\fR can be:
81	90	.PD 0
82	91	.RS 5
83		record Statistic about aggregated netflow records.
	92	record Statistic about arregated netflow records.
84	93	.P
85	94	srcip Statistic about source IP addresses
86	95	.P

100	109	.P
101	110	as Statistic about any (source or destination) AS numbers
102	111	.P
	112	inif Statistic about input interface
	113	.P
	114	outif Statistic about output interface
	115	.P
	116	proto Statistic about IP protocols
103	117	.RE
104	118	.RS 3
	119	.P
	120	By adding \fI:p\fR to the statistic name, the resulting statistic is splitted up into
	121	transport layer protocols. Default is transport protocol independant statistics.
105	122	.P
106	123	\fIorderby\fR is optional and specifies the order by which the statistics is
107	124	ordered and can be \fIflows\fR, \fIpackets\fR, \fIbytes\fR, \fIpps\fR, \fIbps\fR

138	155	the number is unlimited.
139	156	.TP 3
140	157	.B -o \fIformat
141		Selects the output format to print flows or flow statistics (-S). The following
	158	Selects the output format to print flows or flow record statistics (-s record). The following
142	159	formats are available:
143	160	.PD 0
144	161	.RS 5
145		raw Print each flow on multiple lines.
	162	raw Print each file flow record on multiple lines.
146	163	.P
147	164	line Print each flow on one line. Default format.
148	165	.P

151	168	extended Print each flow on one line with even more details.
152	169	.P
153	170	pipe Machine readable format: Print all fields '\|' separated.
	171	.P
	172	fmt:\fIformat\fR
	173	User defined output format.
154	174	.RE
155	175	.RS 3
156		See \fIoutput formats\fR below for more information
	176	For each defined output format except -o fmt:<format> an IPv6 long output format exists.
	177	\fBline6, long6 and extended6\fR. See \fIoutput formts\fR below for more information.
157	178	.RE
158	179	.PD
159	180	.TP 3

169	190	See http://www.cc.gatech.edu/computing/Telecomm/cryptopan/ for
170	191	more information about CryptoPAn.
171	192	.RE
	193	.PD
	194	.TP 3
	195	.B -q
	196	Suppress the header line and the statistics at the bottom.
	197	.TP 3
	198	.B -z
	199	Zero flows. Do not dump flows into the output file, but only the statistics record.
172	200	.TP 3
173	201	.B -Z
174	202	Check filter syntax and exit. Sets the return value accordingly.

198	226	.SH "OUTPUT FORMATS"
199	227	The output format \fBraw\fR prints each flow record on multiple lines, including
200	228	all information available in the record. This is the most detailed view on a
201		flow. Use this output format, if you are interested in the details of flows, not
202		printed in the other output formats.
203		.P
	229	flow.
	230	.P
	231	Other output formats print each flow on a single line. Predefined output formats are
	232	\fBline\fR, \fBlong\fR and \fBextended\fR
204	233	The output format \fBline\fR is the default output format when no format is specified.
205		It prints the flow record each on a single line and limits the information to the
206		connection details as well as number of packets, bytes and flows.
	234	It limits the imformation to the connection details as well as number of packets,
	235	bytes and flows.
207	236	.P
208	237	The output format \fBlong\fR is identical to the format \fBline\fR, and includes
209	238	additional information such as TCP flags and Type of Service.

215	244	.P
216	245	.RS 3
217	246	\fBDate flow start:\fR Start time flow first seen. ISO 8601 format
218		including milliseconds.
219		.P
220		\fBDuration:\fR Duration of the flow in seconds and milliseconds.
	247	including miliseconds.
	248	.P
	249	\fBDuration:\fR Duration of the flow in seconds and miliseconds.
221	250	If flows are aggregated, \fIduration\fR is the time span over the
222		entire period of time from first seen to last seen.
	251	entire periode of time from first seen to last seen.
223	252	.P
224	253	\fBProto:\fR Protocol used in the connection.
225	254	.P

238	267	the bytes are summed up.
239	268	.P
240	269	\fBpps:\fR The calculated packets per second: number of packets / duration.
241		If flows are aggregated this results in the average pps during this period of time.
	270	If flows are aggregated this results in the average pps during this periode of time.
242	271	.P
243	272	\fBbps:\fR The calculated bits per second: 8 * number of bytes / duration. If flows
244		are aggregated this results in the average bps during this period of time.
	273	are aggregated this results in the average bps during this periode of time.
245	274	.P
246	275	\fBBpp:\fR The calculated bytes per packet: number of bytes / number of packets. If flows
247		are aggregated this results in the average bpp during this period of time.
248		.P
249		\fBFlows:\fR Number of flows. If flows are listed only, this number is always 1. If flows
	276	are aggregated this results in the average bpp during this periode of time.
	277	.P
	278	\fBFlows:\fR Number of flows. If flows are listed only, this number is alwasy 1. If flows
250	279	are aggregated, this shows the number of aggregated flows to one record.
	280	.RE
	281	.PD
251	282	.P
252	283	Numbers larger than 1048576 (1024*1024), are scaled to 4 digits and one decimal digit including the
253	284	scaling factor \fBM\fR, \fBG\fR or \fBT\fR for cleaner output, e.g. \fB923.4 M\fR
254		.PE
255		.SH "FILTER
256		"
	285	.P
	286	To make the output more readable, IPv6 addresses are shrinked down to 16 characters. The seven
	287	most and seven least digits connected with two dots \fB'..'\fR are displayed in any normal output
	288	formats. To display the full IPv6 address, use the appropriate long format, which is the format name
	289	followed by a \fB6\fR.
	290	.P
	291	Example: \fB-o line\fR displays an IPv6 address as \fB2001:23..80:d01e\fR where as the format
	292	\fB-o line6\fR displays the IPv6 address in full length \fB2001:234:aabb::211:24ff:fe80:d01e\fR.
	293	The combination of \fB-o line -6\fR is equivalent to \fB-o line6\fR.
	294	.P
	295	The \fBpipe\fR output format is intended to be read by another programm for further processing.
	296	Values are separated by a '\|'. IP addresses are printed as 4 consecutive 32bit numbers.
	297	Output sequence:
	298	.P
	299	.PD 0
	300	.RS 3
	301	\fBAddress family\fR PF_INET or PF_INET6
	302	.P
	303	\fBTime first seen\fR UNIX time seconds
	304	.P
	305	\fBmsec first seen\fR Mili seconds first seen
	306	.P
	307	\fBTime last seen\fR UNIX time seconds
	308	.P
	309	\fBmsec last seen\fR Mili seconds first seen
	310	.P
	311	\fBProtocol\fR Protocol
	312	.P
	313	\fBSrc address\fR Src address as 4 consecutive 32bit numbers.
	314	.P
	315	\fBSrc port\fR Src port
	316	.P
	317	\fBDst address\fR Dst address as 4 consecutive 32bit numbers.
	318	.P
	319	\fBDst port\fR Dst port
	320	.P
	321	\fBSrc AS\fR Src AS number
	322	.P
	323	\fBDst AS\fR Dst AS number
	324	.P
	325	\fBInput IF\fR Input Interface
	326	.P
	327	\fBOutput IF\fR Output Interface
	328	.P
	329	\fBTCP Flags\fR TCP Flags
	330	.P
	331	000001 FIN.
	332	.P
	333	000010 SYN
	334	.P
	335	000100 RESET
	336	.P
	337	001000 PUSH
	338	.P
	339	010000 ACK
	340	.P
	341	100000 URGENT
	342	.P
	343	e.g. 6 => SYN + RESET
	344	.P
	345	\fBTos\fR Type of Service
	346	.P
	347	\fBPackets\fR Packets
	348	.P
	349	\fBBytes\fR Bytes
	350	.P
	351	.RE
	352	.PD
	353	.P
	354	For IPv4 addresses only the last 32bit integer is used. All others are set to zero.
	355	.P
	356	The output format \fBfmt:<format>\fR allows you to define your own output format.
	357	A format description \fBformat\fR consists of a single line containing arbitrary strings
	358	and format specifier as described below
	359	.P
	360	.PD 0
	361	.RS 3
	362	\fB%ts\fR Start Time - first seen
	363	.P
	364	\fB%te\fR End Time - last seen
	365	.P
	366	\fB%td\fR Duration
	367	.P
	368	\fB%pr\fR Protocol
	369	.P
	370	\fB%sa\fR Source Address
	371	.P
	372	\fB%da\fR Destination Address
	373	.P
	374	\fB%sap\fR Source Address:Port
	375	.P
	376	\fB%dap\fR Destination Address:Port
	377	.P
	378	\fB%sp\fR Source Port
	379	.P
	380	\fB%dp\fR Destination Port
	381	.P
	382	\fB%sas\fR Source AS
	383	.P
	384	\fB%das\fR Destination AS
	385	.P
	386	\fB%in\fR Input Interface num
	387	.P
	388	\fB%out\fR Output Interface num
	389	.P
	390	\fB%pkt\fR Packets
	391	.P
	392	\fB%byt\fR Bytes
	393	.P
	394	\fB%fl\fR Flows
	395	.P
	396	\fB%pkt\fR Packets
	397	.P
	398	\fB%flg\fR TCP Flags
	399	.P
	400	\fB%tos\fR Tos
	401	.P
	402	\fB%bps\fR bps - bits per second
	403	.P
	404	\fB%pps\fR pps - packets per second
	405	.P
	406	\fB%bpp\fR bps - Bytes per package
	407	.RE
	408	.PD
	409	.P
	410	For example the standard output format \fBlong\fR can be created as
	411	.P
	412	\fB-o "fmt:%ts %td %pr %sap -> %dap %flg %tos %pkt %byt %fl"\fR
	413	.P
	414	You may also define your own output format and have it compiled into nfdump.
	415	See nfdump.c around line 100 for more details.
	416	.P
	417	.SH "FILTER"
257	418	The filter syntax is similar to the well known pcap library used by tcpdump.
258	419	The filter can be either specified on the command line after all options or
259	420	in a separate file. It can span several lines. Anything after a '#' is treated as a

267	428	.P
268	429	\fIExpr\fR can be one of the following filter primitives:
269	430	.TP 4
	431	.I protocol version
	432	\fBinet\fR for IPv4 and \fBinet6\fR for IPv6
	433	.TP 4
270	434	.I protocol
271		\fBTCP, UDP, ICMP, GRE, ESP, AH, RSVP\fR or \fBPROTO\fR \fInum\fR where \fInum\fR is the protocol
272		number.
	435	\fBproto <protocol>\fR
	436	where \fBprotocol\fR can be any known protocol such as TCP, UDP, ICMP, ICMP6 GRE, ESP, AH, or a valid protocol number.
273	437	.TP 4
274	438	.I IP address
275	439	.PD 0
276	440	.RS 4
277		\fB[SourceDestination]\fR \fBIP\fR \fIa.b.c.d\fR or
278		.P
279		\fB[SourceDestination]\fR \fBHOST\fR \fIa.b.c.d\fR with \fIa.b.c.d\fR as any valid IP address.
	441	\fB[SourceDestination]\fR \fBIP <ipaddr>\fR or
	442	.P
	443	\fB[SourceDestination]\fR \fBHOST <ipaddr>\fR with \fI<ipaddr>\fR as any valid IPv4 or IPv6 address.
280	444	\fISourceDestination\fR may be omitted.
281	445	.RE
282	446	.PD
283	447	.TP 4
284	448	.I SourceDestination
285	449	defines the IP address to be selected and can be \fBSRC\fR
286		\fBDST\fR or any combination of \fBSRC and\|or DST\fR. Omitting \fISourceDestination\fR is
	450	\fBDST\fR or any combination of \fBSRC and\|or DST\fR. Ommiting \fISourceDestination\fR is
287	451	equivalent to \fBSRC or DST\fR.
288	452	.TP 4
	453	.I inout
	454	defines the interface to be selected and can be \fBIN\fR or
	455	\fBOUT\fR.
	456	.TP 4
289	457	.I network
290		\fI[SourceDestination]\fR \fBNET\fR \fIa.b.c.d\fR \fIm.n.r.s\fR
	458	\fI[SourceDestination]\fR \fBNET\fR \fIa.b.c.d\fR \fIm.n.r.s\fR. for IPv4 with \fIm.n.r.s\fR as netmask.
291	459	.PD 0
292	460	.RS 4
293		\fI[SourceDestination]\fR \fBNET\fR \fIa.b.c.d\fR / \fInum\fR with \fIa.b.c.d\fR
294		as network number, \fIm.n.r.s\fR as netmask or \fInum\fR as maskbits respectively.
295		The network may be given as \fIa.b\fR, \fIa.b.c\fR, where a B or C-class equivalent netmask
296		is assumed.
	461	\fI[SourceDestination]\fR \fBNET\fR \fI<net>\fR / \fInum\fR with \fI<net>\fR
	462	as a valid IPv4 or IPv6 network and \fInum\fR as maskbits. The number of mask bits must match
	463	the appropriate address familiy IPv4 or IPv6. Networks may be abreviated such as 172.16/16
	464	if they are unambiguous.
297	465	.RE
298	466	.PD
299	467	.TP 4

304	472	.I Interface
305	473	\fI[inout]\fR \fBIF\fR \fInum\fR with \fInum\fR as an interface number.
306	474	.TP 4
307		.I inout
308		defines the interface to be selected and can be \fBIN\fR or
309		\fBOUT\fR.
310		.TP 4
311	475	.I Flags
312	476	\fBflags\fR \fItcpflags\fR with \fItcpflags\fR as a combination of:
313	477	.PD 0

329	493	.PD
330	494	The ordering of the flags is not relevant. Flags not mentioned are treated as don't care.
331	495	In order to get those flows with only the SYN flag set, use the syntax '\fBflags S and not
332		flags AFRPU\R '.
	496	flags AFRPU\fR'.
333	497	.TP 4
334	498	.I TOS
335	499	Type of service: \fBtos\fR \fIvalue\fR with \fIvalue\fR 0..255.

344	508	\fBpps\fR \fI[comp]\fR \fInum\fR \fI[scale]\fR to specify the pps of the flow.
345	509	.TP 4
346	510	.I Duration: Calculated value
347		\fBduration\fR \fI[comp]\fR \fInum\fR to specify the duration in milliseconds of the flow.
	511	\fBduration\fR \fI[comp]\fR \fInum\fR to specify the duration in miliseconds of the flow.
348	512	.TP 4
349	513	.I Bits per second: Calculated value.
350	514	\fBbps\fR \fI[comp]\fR \fInum\fR \fI[scale]\fR to specify the bps of the flow.

374	538	Generates the Top 20 statistics, extended output format
375	539	.P
376	540	.B nfdump -r /and/dir/nfcapd.200407110845 -S -n 20 'in if 5 and bps > 10k'
377		Generates the Top 20 statistics from flows coming from interface 5
378		.P
379		.B cat file1 file2 file3 \| nfdump 'tcp and ( src port > 1024 and dst port 80 )
380		nfdump reads from stdin and dumps all port 80 connections to any web server.
	541	Generates the Top 20 statistics from flows comming from interface 5
	542	.P
	543	.B nfdump -r /and/dir/nfcapd.200407110845 'inet6 and tcp and ( src port > 1024 and dst port 80 )
	544	Dumps all port 80 IPv6 connections to any web server.
381	545	.SH NOTES
382	546	Generating the statistics for data files of a few hundred MB is no problem. However
383	547	be careful if you want to create statistics of several GB of data. This may consume a lot

+706

-378

nfdump.c less more

31	31	*
32	32	* $Author: peter $
33	33	*
34		* $Id: nfdump.c 53 2005-11-17 07:45:34Z peter $
	34	* $Id: nfdump.c 76 2006-05-21 16:29:48Z peter $
35	35	*
36		* $LastChangedRevision: 53 $
	36	* $LastChangedRevision: 76 $
37	37	*
38	38	*
39	39	*/

46	46	#include <string.h>
47	47	#include <ctype.h>
48	48	#include <sys/types.h>
	49	#include <sys/socket.h>
49	50	#include <sys/stat.h>
50	51	#include <fcntl.h>
	52	#include <sys/resource.h>
51	53
52	54	#include "config.h"
53	55

55	57	#include <stdint.h>
56	58	#endif
57	59
58
59		#include "netflow_v5.h"
	60	#include "nffile.h"
	61	#include "nfnet.h"
60	62	#include "nf_common.h"
	63	#include "netflow_v5_v7.h"
61	64	#include "nftree.h"
62	65	#include "nfprof.h"
63	66	#include "nfdump.h"
64	67	#include "nfstat.h"
65	68	#include "version.h"
	69	#include "launch.h"
66	70	#include "util.h"
67	71	#include "panonymizer.h"
68	72
69	73	/* hash parameters */
70	74	#define HashBits 20
71	75	#define NumPrealloc 128000
	76
	77	#define AGGR_SIZE 5
72	78
73	79
74	80	/* Global Variables */

77	83	int byte_mode, packet_mode;
78	84
79	85	/* Local Variables */
80		static char const *rcsid = "$Id: nfdump.c 53 2005-11-17 07:45:34Z peter $";
	86	static char const *rcsid = "$Id: nfdump.c 76 2006-05-21 16:29:48Z peter $";
81	87	static uint64_t total_bytes;
82	88	static uint32_t total_flows;
83
84		// Header Legends
85		#define HEADER_LINE "Date flow start Duration Proto Src IP Addr:Port Dst IP Addr:Port Packets Bytes Flows"
86		// 2004-07-11 10:31:50.110 120.010 TCP 172.16.8.66:8024 -> 172.16.12.18:25 5000 953.7 M 1
87
88
89		#define HEADER_LINE_LONG "Date flow start Duration Proto Src IP Addr:Port Dst IP Addr:Port Flags Tos Packets Bytes Flows"
90		// 2004-07-11 10:31:50.110 120.010 TCP 172.16.8.66:8024 -> 172.16.12.18:25 UAPRSF 0 5000 953.7 M 1
91
92
93		#define HEADER_LINE_EXTENDED "Date flow start Duration Proto Src IP Addr:Port Dst IP Addr:Port Flags Tos Packets Bytes pps bps Bpp Flows"
94		// 2004-07-11 10:31:50.110 120.010 TCP 172.16.8.66:8024 -> 172.16.12.18:25 UAPRSF 0 5000 953.7 M 41 1041579 200000 1
95
	89	static uint32_t skipped_flows;
	90	static time_t t_first_flow, t_last_flow;
	91
	92	extern const uint16_t MAGIC;
	93	extern const uint16_t VERSION;
	94
	95	/*
	96	* Output formats:
	97	* User defined output formats can be compiled into nfdump, for easy access
	98	* The format has the same syntax as describe in nfdump(1) -o fmt:<format>
	99	*
	100	* A format description consists of a single line containing arbitrary strings
	101	* and format specifier as described below:
	102	*
	103	* %ts // Start Time - first seen
	104	* %te // End Time - last seen
	105	* %td // Duration
	106	* %pr // Protocol
	107	* %sa // Source Address
	108	* %da // Destination Address
	109	* %sap // Source Address:Port
	110	* %dap // Destination Address:Port
	111	* %sp // Source Port
	112	* %dp // Destination Port
	113	* %sas // Source AS
	114	* %das // Destination AS
	115	* %in // Input Interface num
	116	* %out // Output Interface num
	117	* %pkt // Packets
	118	* %byt // Bytes
	119	* %fl // Flows
	120	* %pkt // Packets
	121	* %flg // TCP Flags
	122	* %tos // Tos
	123	* %bps // bps - bits per second
	124	* %pps // pps - packets per second
	125	* %bpp // bps - Bytes perl package
	126	*
	127	* The nfdump standard output formats line, long and extended are defined as follows:
	128	*/
	129
	130	#define FORMAT_line "%ts %td %pr %sap -> %dap %pkt %byt %fl"
	131
	132	#define FORMAT_long "%ts %td %pr %sap -> %dap %flg %tos %pkt %byt %fl"
	133
	134	#define FORMAT_extended "%ts %td %pr %sap -> %dap %flg %tos %pkt %byt %pps %bps %bpp %fl"
	135
	136	/* The appropriate header line is compiled automatically.
	137	*
	138	* For each defined output format a v6 long format automatically exists as well e.g.
	139	* line -> line6, long -> long6, extended -> extended6
	140	* v6 long formats need more space to print IP addresses, as IPv6 addresses are printed in full length,
	141	* where as in standard output format IPv6 addresses are condensed for better readability.
	142	*
	143	* Define your own output format and compile it into nfdumnp:
	144	* 1. Define your output format string.
	145	* 2. Test the format using standard syntax -o "fmt:<your format>"
	146	* 3. Create a #define statement for your output format, similar than the standard output formats above.
	147	* 4. Add another line into the printmap[] struct below BEFORE the last NULL line for you format:
	148	* { "formatname", format_special, FORMAT_definition, NULL },
	149	* The first parameter is the name of your format as recognized on the command line as -o <formatname>
	150	* The second parameter is always 'format_special' - the printing function.
	151	* The third parameter is you format definition as defined in #define.
	152	* The forth parameter is always NULL for user defined formats.
	153	* 5. Recompile nfdump
	154	*/
96	155
97	156	// Assign print functions for all output options -o
98	157	// Teminated with a NULL record
99	158	struct printmap_s {
100		char *printmode; // name of mode
101		int sorted; // does it make sense to sort the output in this mode?
	159	char *printmode; // name of the output format
102	160	printer_t func; // name of the function, which prints the record
103		char *HeaderLine; // Header line for each output format, if needed. NULL otherwise
	161	char *Format; // output format definition
	162	char *HeaderLine; // Header line for static output formats if needed. NULL otherwise
104	163	} printmap[] = {
105		{ "raw", 0, flow_record_raw, NULL },
106		{ "line", 1, flow_record_to_line, HEADER_LINE },
107		{ "long", 1, flow_record_to_line_long, HEADER_LINE_LONG },
108		{ "extended", 1, flow_record_to_line_extended, HEADER_LINE_EXTENDED },
109		{ "pipe", 0, flow_record_to_pipe, NULL },
110		{ NULL, 0, NULL, NULL }
	164	{ "raw", format_file_block_record, NULL, NULL },
	165	{ "line", format_special, FORMAT_line, NULL },
	166	{ "long", format_special, FORMAT_long, NULL },
	167	{ "extended", format_special, FORMAT_extended, NULL },
	168	{ "pipe", flow_record_to_pipe, NULL, NULL },
	169	// add your formats here
	170
	171	// This is always the last line
	172	{ NULL, NULL, NULL, NULL }
111	173	};
112	174
113	175	#define DefaultMode "line"

121	183	/* Function Prototypes */
122	184	static void usage(char *name);
123	185
124		static int ParseAggregateMask( char arg, uint32_t AggregateMasks );
	186	static int ParseAggregateMask( char arg, uint64_t AggregateMasks, uint16_t *Aggregate_bits );
125	187
126	188	static int ParseCryptoPAnKey ( char s, char key );
127	189
128		static uint32_t process_data(char *wfile, int element_stat, int flow_stat, int sort_flows,
	190	static void PrintSummary(stat_record_t *stat_record);
	191
	192	static stat_record_t process_data(char *wfile, int element_stat, int flow_stat, int sort_flows,
129	193	printer_t print_header, printer_t print_record, time_t twin_start, time_t twin_end,
130		uint64_t limitflows, uint32_t *AggregateMasks, int anon);
	194	uint64_t limitflows, uint64_t *AggregateMasks, int anon, int zero_flows);
131	195
132	196	/* Functions */
133	197	static void usage(char *name) {

135	199	"-h\t\tthis text you see right here\n"
136	200	"-V\t\tPrint version and exit.\n"
137	201	"-a\t\tAggregate netflow data.\n"
138		"-A <expr>\tWhat to aggregate: ',' sep list of 'srcip dstip srcport dstport'\n"
	202	"-A <expr>[/net]\tHow to aggregate: ',' sep list of 'srcip dstip srcport dstport'\n"
	203	"\t\tor subnet aggregation: srcip4/24, srcip6/64.\n"
139	204	"-r\t\tread input from file\n"
140	205	"-w\t\twrite output to file\n"
141	206	"-f\t\tread netflow filter from file\n"
142	207	"-n\t\tDefine number of top N. \n"
143	208	"-c\t\tLimit number of records to display\n"
144	209	"-S\t\tGenerate netflow statistics info.\n"
145		"-s\t\tGenerate SRC IP statistics.\n"
146		"-s <expr>\tGenerate statistics for <expr>: srcip, dstip, ip.\n"
	210	"-s <expr>[/<order>]\tGenerate statistics for <expr>: \n"
	211	"\t\tsrcip, dstip, ip, srcport, dstport, port, srcas, dstas, as, inif, outif, proto\n"
	212	"\t\tand ordered by <order>: packets, bytes, flows, bps pps and bpp.\n"
	213	"-q\t\tQuiet: Do not print the header and bottom stat lines.\n"
	214	"-z\t\tZero flows - dumpfile contains only statistics record.\n"
147	215	"-l <expr>\tSet limit on packets for line and packed output format.\n"
148	216	"-K <key>\tAnonymize IP addressses using CryptoPAn with key <key>.\n"
149	217	"\t\tkey: 32 character string or 64 digit hex string starting with 0x.\n"
150	218	"-L <expr>\tSet limit on bytes for line and packed output format.\n"
151	219	"-M <expr>\tRead input from multiple directories.\n"
	220	"-I \t\tPrint netflow summary statistics info from file, specified by -r.\n"
152	221	"\t\t/dir/dir1:dir2:dir3 Read the same files from '/dir/dir1' '/dir/dir2' and '/dir/dir3'.\n"
153	222	"\t\treqquests either -r filename or -R firstfile:lastfile without pathnames\n"
154	223	"-m\t\tPrint netflow data date sorted. Only useful with -M\n"

162	231	"\t\t long Standard output line format with additional fields.\n"
163	232	"\t\t extended Even more information.\n"
164	233	"\t\t pipe '\|' separated, machine parseable output format.\n"
	234	"\t\t\tmode may be extended by '6' for full IPv6 listing. e.g.long6, extended6.\n"
165	235	"-X\t\tDump Filtertable and exit (debug option).\n"
166	236	"-Z\t\tCheck filter syntax and exit.\n"
167	237	"-t <time>\ttime window for filtering packets\n"
168	238	"\t\tyyyy/MM/dd.hh:mm:ss[-yyyy/MM/dd.hh:mm:ss]\n", name);
169	239	} /* usage */
170	240
171		static int ParseAggregateMask( char arg, uint32_t AggregateMasks ) {
172		char *p;
173
	241	static int ParseAggregateMask( char arg, uint64_t AggregateMasks, uint16_t *Aggregate_bits ) {
	242	char p, q;
	243	uint32_t subnet;
	244
	245	// preset Aggregate Mask array
	246	AggregateMasks[0] = ~( MaskSrcPort \| MaskDstPort );
	247	*Aggregate_bits = 0;
	248
	249	subnet = 0;
174	250	p = strtok(arg, ",");
175	251	while ( p ) {
176		if ( strcasecmp(p, "srcip" ) == 0 ) {
177		AggregateMasks[0] = 0xffffffff;
	252	q = strchr(p, '/');
	253	if ( q ) {
	254	*q = 0;
	255	subnet = atoi(q+1);
	256	}
	257	if ( strcasecmp(p, "srcip4" ) == 0 ) {
	258	AggregateMasks[1] = 0xffffffffffffffffLL;
	259	AggregateMasks[2] = 0xffffffffffffffffLL;
	260	if ( q ) {
	261	if ( subnet < 1 \|\| subnet > 32 ) {
	262	fprintf(stderr, "Subnet specifier '%s' out of range for IPv4\n", q+1);
	263	return 0;
	264	}
	265	AggregateMasks[2] = 0xffffffffffffffffLL << ( 32 - subnet );
	266	}
	267	*Aggregate_bits \|= Aggregate_SRCIP;
	268	} else if ( strcasecmp(p, "dstip4" ) == 0 ) {
	269	AggregateMasks[3] = 0xffffffffffffffffLL;
	270	AggregateMasks[4] = 0xffffffffffffffffLL;
	271	if ( q ) {
	272	if ( subnet < 1 \|\| subnet > 32 ) {
	273	fprintf(stderr, "Subnet specifier '%s' out of range for IPv4\n", q+1);
	274	return 0;
	275	}
	276	AggregateMasks[4] = 0xffffffffffffffffLL << ( 32 - subnet );
	277	}
	278	*Aggregate_bits \|= Aggregate_DSTIP;
	279	} else if ( strcasecmp(p, "srcip6" ) == 0 ) {
	280	AggregateMasks[1] = 0xffffffffffffffffLL;
	281	AggregateMasks[2] = 0xffffffffffffffffLL;
	282	if ( q ) {
	283	if ( subnet < 1 \|\| subnet > 128 ) {
	284	fprintf(stderr, "Subnet specifier '%s' out of range for IPv6\n", q+1);
	285	return 0;
	286	}
	287	if ( subnet > 64 ) {
	288	AggregateMasks[2] = 0xffffffffffffffffLL << ( 128 - subnet );
	289	} else {
	290	AggregateMasks[1] = 0xffffffffffffffffLL << ( 64 - subnet );
	291	AggregateMasks[2] = 0;
	292	}
	293	}
	294	*Aggregate_bits \|= Aggregate_SRCIP;
	295	} else if ( strcasecmp(p, "dstip6" ) == 0 ) {
	296	AggregateMasks[3] = 0xffffffffffffffffLL;
	297	AggregateMasks[4] = 0xffffffffffffffffLL;
	298	if ( q ) {
	299	if ( subnet < 1 \|\| subnet > 128 ) {
	300	fprintf(stderr, "Subnet specifier '%s' out of range for IPv6\n", q+1);
	301	return 0;
	302	}
	303	if ( subnet > 64 ) {
	304	AggregateMasks[4] = 0xffffffffffffffffLL << ( 128 - subnet );
	305	} else {
	306	AggregateMasks[3] = 0xffffffffffffffffLL << ( 64 - subnet );
	307	AggregateMasks[4] = 0;
	308	}
	309	}
	310	*Aggregate_bits \|= Aggregate_DSTIP;
	311	} else if ( q ) {
	312	*q = '/';
	313	fprintf(stderr, "Subnet specifier not allowed here: '%s'\n", p);
	314	return 0;
	315	} else if ( strcasecmp(p, "srcip" ) == 0 ) {
	316	AggregateMasks[1] = 0xffffffffffffffffLL;
	317	AggregateMasks[2] = 0xffffffffffffffffLL;
	318	*Aggregate_bits \|= Aggregate_SRCIP;
178	319	} else if ( strcasecmp(p, "dstip" ) == 0 ) {
179		AggregateMasks[1] = 0xffffffff;
180		} else if ( strcasecmp(p, "srcport" ) == 0 ) {
181		AggregateMasks[2] = 0xffffffff;
	320	AggregateMasks[3] = 0xffffffffffffffffLL;
	321	AggregateMasks[4] = 0xffffffffffffffffLL;
	322	*Aggregate_bits \|= Aggregate_DSTIP;
	323	} else if ( strcasecmp(p, "srcport" ) == 0 ) {
	324	AggregateMasks[0] \|= MaskSrcPort;
	325	*Aggregate_bits \|= Aggregate_SRCPORT;
182	326	} else if ( strcasecmp(p, "dstport" ) == 0 ) {
183		AggregateMasks[3] = 0xffffffff;
	327	AggregateMasks[0] \|= MaskDstPort;
	328	*Aggregate_bits \|= Aggregate_DSTPORT;
	329
	330
184	331	} else {
185	332	fprintf(stderr, "Unknown aggregate field: '%s'\n", p);
186	333	return 0;

188	335	p = strtok(NULL, ",");
189	336	}
190	337	return 1;
	338
191	339	} /* End of ParseAggregateMask */
192	340
193	341	static int ParseCryptoPAnKey ( char s, char key ) {

205	353	if ( strlen(s) == 66 && s[0] == '0' && s[1] == 'x' ) {
206	354	j = 2;
207	355	for ( i=0; i<32; i++ ) {
208		if ( !isxdigit(s[j]) \|\| !isxdigit(s[j+1]) )
	356	if ( !isxdigit((int)s[j]) \|\| !isxdigit((int)s[j+1]) )
209	357	return 0;
210	358	numstr[0] = s[j++];
211	359	numstr[1] = s[j++];

219	367
220	368	} // End of ParseCryptoPAnKey
221	369
222		uint32_t process_data(char *wfile, int element_stat, int flow_stat, int sort_flows,
	370	static void PrintSummary(stat_record_t *stat_record) {
	371	static double duration;
	372	uint64_t bps, pps, bpp;
	373	char byte_str[32], packet_str[32], bps_str[32], pps_str[32], bpp_str[32];
	374
	375	bps = pps = bpp = 0;
	376	duration = stat_record->last_seen - stat_record->first_seen;
	377	duration += ((double)stat_record->msec_last - (double)stat_record->msec_first) / 1000.0;
	378	if ( duration > 0 && stat_record->last_seen > 0 ) {
	379	bps = ( stat_record->numbytes << 3 ) / duration; // bits per second. ( >> 3 ) -> * 8 to convert octets into bits
	380	pps = stat_record->numpackets / duration; // packets per second
	381	bpp = stat_record->numbytes / stat_record->numpackets; // Bytes per Packet
	382	}
	383	format_number(stat_record->numbytes, byte_str, VAR_LENGTH);
	384	format_number(stat_record->numpackets, packet_str, VAR_LENGTH);
	385	format_number(bps, bps_str, VAR_LENGTH);
	386	format_number(pps, pps_str, VAR_LENGTH);
	387	format_number(bpp, bpp_str, VAR_LENGTH);
	388	printf("Summary: total flows: %llu, total bytes: %s, total packets: %s, avg bps: %s, avg pps: %s, avg bpp: %s\n",
	389	stat_record->numflows, byte_str, packet_str, bps_str, pps_str, bpp_str );
	390
	391	} // End of PrintSummary
	392
	393	stat_record_t process_data(char *wfile, int element_stat, int flow_stat, int sort_flows,
223	394	printer_t print_header, printer_t print_record, time_t twin_start, time_t twin_end,
224		uint64_t limitflows, uint32_t *AggregateMasks, int anon) {
225		flow_header_t flow_header;
226		flow_record_t flow_record, record_buffer;
227		uint16_t cnt;
228		uint32_t NumRecords;
229		time_t win_start, win_end, first_seen, last_seen;
230		uint64_t numflows, numbytes, numpackets;
231		int i, rfd, wfd, nffd, done, ret, request_size, *ftrue, do_stat, has_aggregate_mask, old_format;
232		uint64_t numflows_tcp, numflows_udp, numflows_icmp, numflows_other;
233		uint64_t numbytes_tcp, numbytes_udp, numbytes_icmp, numbytes_other;
234		uint64_t numpackets_tcp, numpackets_udp, numpackets_icmp, numpackets_other;
235		char *string, sfile[255], tmpstring[64];
236
237		if ( wfile && ( strcmp(wfile, "-") != 0 )) { // if we write a new file
238		// write a new stat file
239		snprintf(sfile, 254, "%s.stat", wfile);
240		sfile[254] = 0;
241		} else {
242		sfile[0] = 0; // no new stat file
243		}
	395	uint64_t limitflows, uint64_t *AggregateMasks, int anon, int zero_flows) {
	396	data_block_header_t in_flow_header, *out_flow_header;
	397	common_record_t flow_record, in_buff, *out_buff;
	398	master_record_t master_record;
	399	void *writeto;
	400	stat_record_t stat_record;
	401	uint32_t NumReadRecords, file_blocks, buffer_size;
	402	int i, rfd, wfd, done, ret, do_stat, write_file, has_aggregate_mask;
	403	char *string;
	404
	405	#ifdef COMPAT14
	406	extern int Format14;
	407	#endif
	408
	409	out_flow_header = NULL;
	410	out_buff = NULL;
	411	writeto = NULL;
	412
	413	// if we write to a file other than stdout
	414	write_file = wfile && ( strcmp(wfile, "-") != 0 );
244	415
245	416	// print flows later, when all records are processed and sorted
246	417	if ( sort_flows ) {

248	419	limitflows = 0;
249	420	}
250	421
	422	memset((void *)&stat_record, 0, sizeof(stat_record_t));
	423
	424	// time window of all matched flows
	425	stat_record.first_seen = 0x7fffffff;
	426	stat_record.msec_first = 999;
	427
251	428	// time window of all processed flows
252		win_start = 0x7fffffff;
253		win_end = 0;
254		SetSeenTwin(0, 0);
255
256		// time window of all matched flows
257		first_seen = 0x7fffffff;
258		last_seen = 0;
259
260		numflows = numbytes = numpackets = 0;
261		numflows_tcp = numflows_udp = numflows_icmp = numflows_other = 0;
262		numbytes_tcp = numbytes_udp = numbytes_icmp = numbytes_other = 0;
263		numpackets_tcp = numpackets_udp = numpackets_icmp = numpackets_other = 0;
	429	t_first_flow = 0x7fffffff;
	430	t_last_flow = 0;
	431
	432	file_blocks = 0;
264	433
265	434	do_stat = element_stat \|\| flow_stat;
266	435
267	436	// check for a special aggregate mask
268		has_aggregate_mask = 0;
269		for ( i=0; i<4; i++ )
270		if ( AggregateMasks[i] )
271		has_aggregate_mask = 1;
	437	has_aggregate_mask = AggregateMasks != NULL;
272	438
273	439	// Get the first file handle
274		rfd = GetNextFile(0, twin_start, twin_end);
	440	rfd = GetNextFile(0, twin_start, twin_end, NULL);
275	441	if ( rfd < 0 ) {
276		if ( errno )
	442	if ( rfd == FILE_ERROR )
277	443	perror("Can't open input file for reading");
278		return numflows;
279		}
	444	return stat_record;
	445	}
	446
	447	#ifdef COMPAT14
	448	if ( Format14 )
	449	fprintf(stderr, "Reading nfdump <= v1.4 old style file format!\n");
	450	#endif
280	451
281	452	if ( wfile ) {
282		wfd = strcmp(wfile, "-") == 0 ? STDOUT_FILENO :
283		open(wfile, O_CREAT \| O_RDWR \| O_TRUNC , S_IRUSR \| S_IWUSR \| S_IRGRP \| S_IROTH );
	453	wfd = strcmp(wfile, "-") == 0 ? STDOUT_FILENO : OpenNewFile(wfile, &string);
	454	if ( strcmp(wfile, "-") == 0 ) { // output to stdout
	455	file_header_t *file_header;
	456	size_t len;
	457
	458	wfd = STDOUT_FILENO;
	459
	460	len = sizeof(file_header_t) + sizeof(stat_record_t);
	461	file_header = (file_header_t *)malloc(len);
	462	memset((void *)file_header, 0, len);
	463	file_header->magic = MAGIC;
	464	file_header->version = VERSION;
	465	strncpy(file_header->ident, "none", IDENT_SIZE);
	466	write(STDOUT_FILENO, (void *)file_header, len) ;
	467
	468	} else {
	469	wfd = OpenNewFile(wfile, &string);
	470
	471	}
284	472	if ( wfd < 0 ) {
285		perror("Can't open output file for writing");
	473	if ( string != NULL )
	474	fprintf(stderr, "%s\n", string);
286	475	if ( rfd )
287	476	close(rfd);
288		return numflows;
289		}
	477	return stat_record;
	478	}
	479	out_buff = malloc(OUTPUT_BUFF_SIZE);
	480	if ( !out_buff ) {
	481	fprintf(stderr, "Buffer allocation error: %s", strerror(errno));
	482	return stat_record;
	483	}
	484	out_flow_header = (data_block_header_t *)out_buff;
	485	out_flow_header->size = 0;
	486	out_flow_header->NumBlocks = 0;
	487	out_flow_header->id = DATA_BLOCK_TYPE_1;
	488	out_flow_header->pad = 0;
	489	writeto = (void *)((pointer_addr_t)out_flow_header + sizeof(data_block_header_t));
290	490	} else
291	491	wfd = 0;
292	492
293	493	// allocate buffer suitable for netflow version
294		record_buffer = (flow_record_t *) calloc(BuffNumRecords , FLOW_RECORD_LENGTH);
295
296		ftrue = (int *) calloc(BuffNumRecords , sizeof(int));
297		if ( !record_buffer \|\| !ftrue ) {
	494	buffer_size = BUFFSIZE;
	495	in_buff = (common_record_t *) malloc(buffer_size);
	496
	497	if ( !in_buff ) {
298	498	perror("Memory allocation error");
299	499	close(rfd);
300		if (wfd )
301		close(wfd);
302		return numflows;
303		}
	500	if ( write_file ) {
	501	/* Write stat info and close file */
	502	CloseUpdateFile(wfd, &stat_record, file_blocks, GetIdent(), &string );
	503	if ( string != NULL )
	504	fprintf(stderr, "%s\n", string);
	505	}
	506	return stat_record;
	507	}
	508
	509	// setup Filter Engine to point to master_record, as any record read from file
	510	// is expanded into this record
	511	Engine->nfrecord = (uint64_t *)&master_record;
304	512
305	513	done = 0;
306	514	while ( !done ) {
307		ret = read(rfd, &flow_header, FLOW_HEADER_LENGTH);
	515
	516	#ifdef COMPAT14
	517	if ( Format14 )
	518	ret = Compat14_ReadHeader(rfd, &in_flow_header);
	519	else
	520	ret = read(rfd, &in_flow_header, sizeof(data_block_header_t));
	521	#else
	522	ret = read(rfd, &in_flow_header, sizeof(data_block_header_t));
	523	#endif
	524
308	525	if ( ret == 0 ) {
309		rfd = GetNextFile(rfd, twin_start, twin_end);
	526	// EOF of rfd
	527	rfd = GetNextFile(rfd, twin_start, twin_end, NULL);
310	528	if ( rfd < 0 ) {
311		if ( errno )
312		perror("Can't open input file for reading");
	529	if ( rfd == FILE_ERROR )
	530	fprintf(stderr, "Can't read from file '%s': %s\n",GetCurrentFilename(), strerror(errno) );
313	531	done = 1;
314	532	}
315	533	continue;
316	534	} else if ( ret == -1 ) {
317		perror("Error reading data");
318		close(rfd);
319		if ( wfd )
320		close(wfd);
321		return numflows;
	535	fprintf(stderr, "Can't read from file '%s': %s\n",GetCurrentFilename(), strerror(errno) );
	536	break;
322	537	}
323	538	total_bytes += ret;
324		if ( flow_header.version != FLOW_VERSION ) {
325		fprintf(stdout, "Not a netflow v%i header\n", FLOW_VERSION);
326		close(rfd);
327		if ( wfd )
328		close(wfd);
329		return numflows;
330		}
331		if ( flow_header.count > MAX_RECORDS ) {
332		fprintf(stderr, "Too many records %u ( > MAX_RECORDS )\n", flow_header.count);
333		break;
334		}
335
336		NumRecords = flow_header.count;
337		old_format = flow_header.layout_version != 1;
338
339		request_size = NumRecords * FLOW_RECORD_LENGTH;
340		ret = read(rfd, record_buffer, request_size);
	539
	540	if ( in_flow_header.id != DATA_BLOCK_TYPE_1 ) {
	541	fprintf(stderr, "Can't process block type %u\n", in_flow_header.id);
	542	skipped_flows++;
	543	continue;
	544	}
	545
	546	NumReadRecords = in_flow_header.NumBlocks;
	547
	548	if ( in_flow_header.size > buffer_size ) {
	549	void *tmp;
	550	// Actually, this should never happen, but catch it anyway
	551	if ( in_flow_header.size > MAX_BUFFER_SIZE ) {
	552	// this is most likely corrupt
	553	fprintf(stderr, "Corrupt data file: Requested buffer size %u exceeds max. buffer size.\n", in_flow_header.size);
	554	break;
	555	}
	556	// make it at least the requested size
	557	buffer_size = in_flow_header.size;
	558	tmp = realloc((void *)in_buff, buffer_size);
	559	if ( !tmp ) {
	560	fprintf(stderr, "Can't reallocate buffer to %u bytes: %s\n", buffer_size, strerror(errno));
	561	break;
	562	}
	563	in_buff = (common_record_t *)tmp;
	564	}
	565
	566	#ifdef COMPAT14
	567	if ( Format14 )
	568	ret = Compat14_ReadRecords(rfd, in_buff, &in_flow_header);
	569	else
	570	ret = read(rfd, in_buff, in_flow_header.size);
	571	#else
	572	ret = read(rfd, in_buff, in_flow_header.size);
	573	#endif
341	574	if ( ret == 0 ) {
342	575	done = 1;
343	576	break;
344	577	} else if ( ret == -1 ) {
345	578	perror("Error reading data");
346	579	close(rfd);
347		if ( wfd )
348		close(wfd);
349		return numflows;
350		}
351		if ( request_size != ret ) {
352		fprintf(stderr, "Short read for netflow records: Expected %i, got %i bytes!\n",request_size, ret );
	580	if ( write_file ) {
	581	/* Write stat info and close file */
	582	CloseUpdateFile(wfd, &stat_record, file_blocks, GetIdent(), &string );
	583	if ( string != NULL )
	584	fprintf(stderr, "%s\n", string);
	585	}
353	586	break;
354	587	}
	588	if ( in_flow_header.size != ret ) {
	589	// Ups - this was a short read - most likely reading from the stdin pipe
	590	// loop until we have requested size
	591	size_t request_size, total_size;
	592	void *read_ptr;
	593
	594	total_size = ret;
	595	request_size = in_flow_header.size - total_size;
	596	read_ptr = (void *)((pointer_addr_t)in_buff + total_size);
	597	do {
	598	ret = read(rfd, read_ptr, request_size);
	599	if ( ret == 0 ) {
	600	break;
	601	} else if ( ret < 0 ) {
	602	perror("Error reading data");
	603	break;
	604	}
	605	total_size += ret;
	606	if ( total_size < in_flow_header.size ) {
	607	request_size = in_flow_header.size - ret;
	608	read_ptr = (void *)((pointer_addr_t)in_buff + total_size);
	609	request_size = in_flow_header.size - total_size;
	610	}
	611	} while ( ret > 0 && ( total_size < in_flow_header.size ));
	612
	613	if ( total_size != in_flow_header.size ) {
	614	// still unsuccessful
	615	#ifdef HAVE_SIZE_T_Z_FORMAT
	616	fprintf(stderr, "Short read for netflow records: Expected %i, got %zu bytes!\n",
	617	in_flow_header.size, total_size );
	618	#else
	619	fprintf(stderr, "Short read for netflow records: Expected %i, got %lu bytes!\n",
	620	in_flow_header.size, (unsigned long)total_size );
	621	#endif
	622	continue;
	623	} else {
	624	// continue
	625	ret = in_flow_header.size;
	626	}
	627	}
355	628	total_bytes += ret;
356	629
357		// cnt is the number of blocks, which survived the filter
358		// ftrue is an array of flags of the filter result
359		cnt = 0;
360		flow_record = record_buffer;
361		for ( i=0; i < NumRecords; i++ ) {
	630	flow_record = in_buff;
	631	for ( i=0; i < NumReadRecords; i++ ) {
362	632	total_flows++;
363
364		/* may be removed when old format died out */
365		if ( old_format ) {
366		flow_record->msec_first = 0;
367		flow_record->msec_last = 0;
368		}
	633	ExpandRecord( flow_record, &master_record);
	634
	635	// Update global time span window
	636	if ( master_record.first < t_first_flow )
	637	t_first_flow = master_record.first;
	638	if ( master_record.last > t_last_flow )
	639	t_last_flow = master_record.last;
369	640
370	641	// Time based filter
371	642	// if no time filter is given, the result is always true
372		ftrue[i] = twin_start && (flow_record->First < twin_start \|\| flow_record->Last > twin_end) ? 0 : 1;
373		ftrue[i] &= limitflows ? numflows < limitflows : 1;
374		Engine->nfrecord = (uint32_t *)flow_record;
	643	flow_record->mark = twin_start && (master_record.first < twin_start \|\| master_record.last > twin_end) ? 0 : 1;
	644	flow_record->mark &= limitflows ? stat_record.numflows < limitflows : 1;
375	645
376	646	// filter netflow record with user supplied filter
377		if ( ftrue[i] )
378		ftrue[i] = (*Engine->FilterEngine)(Engine);
379
380		if ( ftrue[i] ) {
381		// Update statistics
382		switch (flow_record->prot) {
383		case 1:
384		numflows_icmp++;
385		numpackets_icmp += flow_record->dPkts;
386		numbytes_icmp += flow_record->dOctets;
387		break;
388		case 6:
389		numflows_tcp++;
390		numpackets_tcp += flow_record->dPkts;
391		numbytes_tcp += flow_record->dOctets;
392		break;
393		case 17:
394		numflows_udp++;
395		numpackets_udp += flow_record->dPkts;
396		numbytes_udp += flow_record->dOctets;
397		break;
398		default:
399		numflows_other++;
400		numpackets_other += flow_record->dPkts;
401		numbytes_other += flow_record->dOctets;
402		}
403		numflows++;
404		numpackets += flow_record->dPkts;
405		numbytes += flow_record->dOctets;
406		cnt++;
407
408		if ( flow_record->First < first_seen )
409		first_seen = flow_record->First;
410		if ( flow_record->Last > last_seen )
411		last_seen = flow_record->Last;
412
413		}
414		if ( flow_record->First < win_start )
415		win_start = flow_record->First;
416		if ( flow_record->Last > win_end )
417		win_end = flow_record->Last;
418
419		// increment pointer by number of bytes for netflow record
420		flow_record = (flow_record_t *)((pointer_addr_t)flow_record + (pointer_addr_t)FLOW_RECORD_LENGTH);
421
422		} // for all records
423
424		// check if we are done, due to -c option
425		if ( limitflows )
426		done = numflows >= limitflows;
427
428		// if no records are left after filtering, continue the read loop
429		if ( cnt == 0 )
430		continue;
431
432		// Else we can process the header and any filtered records
433
434		// set new count in v5 header
435		flow_header.count = cnt;
436
437		// write binary output if requested
438		if ( wfd ) {
439		ret = write(wfd, &flow_header, FLOW_HEADER_LENGTH);
440		if ( ret < 0 ) {
441		perror("Error writing data");
442		close(rfd);
443		if ( wfd )
444		close(wfd);
445		return numflows;
446		}
447
448		flow_record = record_buffer;
449		for ( i=0; i < NumRecords; i++ ) {
450		if ( ftrue[i] ) {
	647	if ( flow_record->mark )
	648	flow_record->mark = (*Engine->FilterEngine)(Engine);
	649
	650	if ( flow_record->mark == 0 ) { // record failed to pass all filters
	651	// increment pointer by number of bytes for netflow record
	652	flow_record = (common_record_t *)((pointer_addr_t)flow_record + flow_record->size);
	653	// go to next record
	654	continue;
	655	}
	656
	657	// Records passed filter -> continue record processing
	658	flow_record->mark = 0;
	659
	660	// Update statistics
	661	switch (master_record.prot) {
	662	case 1:
	663	stat_record.numflows_icmp++;
	664	stat_record.numpackets_icmp += master_record.dPkts;
	665	stat_record.numbytes_icmp += master_record.dOctets;
	666	break;
	667	case 6:
	668	stat_record.numflows_tcp++;
	669	stat_record.numpackets_tcp += master_record.dPkts;
	670	stat_record.numbytes_tcp += master_record.dOctets;
	671	break;
	672	case 17:
	673	stat_record.numflows_udp++;
	674	stat_record.numpackets_udp += master_record.dPkts;
	675	stat_record.numbytes_udp += master_record.dOctets;
	676	break;
	677	default:
	678	stat_record.numflows_other++;
	679	stat_record.numpackets_other += master_record.dPkts;
	680	stat_record.numbytes_other += master_record.dOctets;
	681	}
	682	stat_record.numflows++;
	683	stat_record.numpackets += master_record.dPkts;
	684	stat_record.numbytes += master_record.dOctets;
	685
	686	if ( master_record.first < stat_record.first_seen ) {
	687	stat_record.first_seen = master_record.first;
	688	stat_record.msec_first = master_record.msec_first;
	689	}
	690	if ( master_record.first == stat_record.first_seen &&
	691	master_record.msec_first < stat_record.msec_first )
	692	stat_record.msec_first = master_record.msec_first;
	693
	694	if ( master_record.last > stat_record.last_seen ) {
	695	stat_record.last_seen = master_record.last;
	696	stat_record.msec_last = master_record.msec_last;
	697	}
	698	if ( master_record.last == stat_record.last_seen &&
	699	master_record.msec_last > stat_record.msec_last )
	700	stat_record.msec_last = master_record.msec_last;
	701
	702
	703	if ( wfd != 0 ) {
	704	if ( (out_flow_header->size + flow_record->size) < OUTPUT_BUFF_SIZE && !zero_flows) {
	705	memcpy(writeto, (void *)flow_record, flow_record->size);
	706
451	707	if ( anon ) {
452		flow_record->srcaddr = anonymize(flow_record->srcaddr);
453		flow_record->dstaddr = anonymize(flow_record->dstaddr);
	708	pointer_addr_t size = sizeof(common_record_t) - sizeof(uint8_t[4]);
	709	if ( (flow_record->flags & FLAG_IPV6_ADDR ) == 0 ) {
	710	uint32_t ip = (uint32_t )((pointer_addr_t)writeto + size);
	711	ip[0] = anonymize(ip[0]);
	712	ip[1] = anonymize(ip[1]);
	713	} else {
	714	ipv6_block_t ip = (ipv6_block_t )((pointer_addr_t)writeto + size);
	715	uint64_t anon_ip[2];
	716	anonymize_v6(ip->srcaddr, anon_ip);
	717	ip->srcaddr[0] = anon_ip[0];
	718	ip->srcaddr[1] = anon_ip[1];
	719
	720	anonymize_v6(ip->dstaddr, anon_ip);
	721	ip->dstaddr[0] = anon_ip[0];
	722	ip->dstaddr[1] = anon_ip[1];
	723	}
	724	}
	725
	726	out_flow_header->NumBlocks++;
	727	out_flow_header->size += flow_record->size;
	728	writeto = (void *)((pointer_addr_t)writeto + flow_record->size);
	729
	730	// flush current buffer to disc
	731	if ( out_flow_header->size > OUTPUT_FLUSH_LIMIT ) {
	732	if ( write(wfd, out_buff, sizeof(data_block_header_t) + out_flow_header->size) <= 0 ) {
	733	fprintf(stderr, "Failed to write output buffer to disk: '%s'" , strerror(errno));
	734	} else {
	735	out_flow_header->size = 0;
	736	out_flow_header->NumBlocks = 0;
	737	writeto = (void *)((pointer_addr_t)out_flow_header + sizeof(data_block_header_t) );
	738	file_blocks++;
	739	}
454	740	}
455		ret = write(wfd, flow_record, FLOW_RECORD_LENGTH);
456		if ( ret < 0 ) {
457		perror("Error writing data");
458		close(rfd);
459		return numflows;
	741	}
	742
	743	} else if ( do_stat ) {
	744	// we need to add this record to the stat hash
	745	if ( has_aggregate_mask ) {
	746	if ( (flow_record->flags & FLAG_IPV6_ADDR ) == 0 ) { // only v4 at the moment
	747	uint64_t array64 = (uint64_t )&master_record;
	748	// mask ports
	749	array64[3] &= AggregateMasks[0];
	750	// mask IP addresses
	751	array64[5] &= AggregateMasks[1];
	752	array64[6] &= AggregateMasks[2];
	753	array64[7] &= AggregateMasks[3];
	754	array64[8] &= AggregateMasks[4];
	755	}
460	756	}
461		}
462		// increment pointer by number of bytes for netflow record
463		flow_record = (flow_record_t *)((pointer_addr_t)flow_record + (pointer_addr_t)FLOW_RECORD_LENGTH);
464		}
465
466		} else if ( do_stat ) {
467		// Add records to netflow statistic hash
468		flow_record = record_buffer;
469		for ( i=0; i< NumRecords; i++ ) {
470		if ( ftrue[i] ) {
471		if ( has_aggregate_mask ) {
472		flow_record->srcaddr &= AggregateMasks[0];
473		flow_record->dstaddr &= AggregateMasks[1];
474		flow_record->srcport &= AggregateMasks[2];
475		flow_record->dstport &= AggregateMasks[3];
476		}
477		AddStat(&flow_header, flow_record, flow_stat, element_stat);
478		}
479		// increment pointer by number of bytes for netflow record
480		flow_record = (flow_record_t *)((pointer_addr_t)flow_record + (pointer_addr_t)FLOW_RECORD_LENGTH);
481		}
482
483		} else {
484		// We print out the records somehow
485
486		if ( print_header ) {
487		print_header(&flow_header, 0, 0, 0, &string, anon);
488		printf("%s", string);
489		}
490
491		flow_record = record_buffer;
492		for ( i=0; i< NumRecords; i++ ) {
493		if ( ftrue[i] ) {
494
	757	AddStat(&in_flow_header, &master_record, flow_stat, element_stat);
	758	} else {
495	759	// if we need tp print out this record
496	760	if ( print_record ) {
497		print_record(flow_record, 1, (uint64_t)flow_record->dPkts, (uint64_t)flow_record->dOctets, &string, anon);
	761	print_record(&master_record, 1, &string, anon);
498	762	if ( string ) {
499	763	if ( limitflows ) {
500		if ( (numflows <= limitflows) )
	764	if ( (stat_record.numflows <= limitflows) )
501	765	printf("%s\n", string);
502	766	} else
503	767	printf("%s\n", string);

507	771	// if we need to sort the flows first -> insert into hash table
508	772	// they get
509	773	if ( sort_flows )
510		InsertFlow(flow_record);
511		}
512
513		// increment pointer by number of bytes for netflow record
514		flow_record = (flow_record_t *)((pointer_addr_t)flow_record + (pointer_addr_t)FLOW_RECORD_LENGTH);
515		}
516		}
	774	InsertFlow(&master_record);
	775	}
	776
	777	// Advance pointer by number of bytes for netflow record
	778	flow_record = (common_record_t *)((pointer_addr_t)flow_record + flow_record->size);
	779
	780	} // for all records
	781
	782	// check if we are done, due to -c option
	783	if ( limitflows )
	784	done = stat_record.numflows >= limitflows;
517	785
518	786	} // while
519	787
520		if ( wfd )
521		close(wfd);
522
523		/* Statfile */
524		if ( sfile[0] != 0 ) {
525		nffd = open(sfile, O_CREAT \| O_RDWR \| O_TRUNC, S_IRUSR \| S_IWUSR \| S_IRGRP \| S_IROTH );
526		if ( nffd == -1 ) {
527		perror("Can't open stat file: ");
528		return numflows;
529		}
530
531		tmpstring[63] = 0;
532		snprintf(tmpstring, 63, "Time: %u\n", GetStatTime());
533		write(nffd, tmpstring, strlen(tmpstring));
534		snprintf(tmpstring, 63, "Ident: %s\n", GetIdent());
535		write(nffd, tmpstring, strlen(tmpstring));
536		snprintf(tmpstring, 63, "Flows: %llu\n", numflows);
537		write(nffd, tmpstring, strlen(tmpstring));
538		snprintf(tmpstring, 63, "Flows_tcp: %llu\n", numflows_tcp);
539		write(nffd, tmpstring, strlen(tmpstring));
540		snprintf(tmpstring, 63, "Flows_udp: %llu\n", numflows_udp);
541		write(nffd, tmpstring, strlen(tmpstring));
542		snprintf(tmpstring, 63, "Flows_icmp: %llu\n", numflows_icmp);
543		write(nffd, tmpstring, strlen(tmpstring));
544		snprintf(tmpstring, 63, "Flows_other: %llu\n", numflows_other);
545		write(nffd, tmpstring, strlen(tmpstring));
546		snprintf(tmpstring, 63, "Packets: %llu\n", numpackets);
547		write(nffd, tmpstring, strlen(tmpstring));
548		snprintf(tmpstring, 63, "Packets_tcp: %llu\n", numpackets_tcp);
549		write(nffd, tmpstring, strlen(tmpstring));
550		snprintf(tmpstring, 63, "Packets_udp: %llu\n", numpackets_udp);
551		write(nffd, tmpstring, strlen(tmpstring));
552		snprintf(tmpstring, 63, "Packets_icmp: %llu\n", numpackets_icmp);
553		write(nffd, tmpstring, strlen(tmpstring));
554		snprintf(tmpstring, 63, "Packets_other: %llu\n", numpackets_other);
555		write(nffd, tmpstring, strlen(tmpstring));
556		snprintf(tmpstring, 63, "Bytes: %llu\n", numbytes);
557		write(nffd, tmpstring, strlen(tmpstring));
558		snprintf(tmpstring, 63, "Bytes_tcp: %llu\n", numbytes_tcp);
559		write(nffd, tmpstring, strlen(tmpstring));
560		snprintf(tmpstring, 63, "Bytes_udp: %llu\n", numbytes_udp);
561		write(nffd, tmpstring, strlen(tmpstring));
562		snprintf(tmpstring, 63, "Bytes_icmp: %llu\n", numbytes_icmp);
563		write(nffd, tmpstring, strlen(tmpstring));
564		snprintf(tmpstring, 63, "Bytes_other: %llu\n", numbytes_other);
565		write(nffd, tmpstring, strlen(tmpstring));
566		#if defined __OpenBSD__ \|\| defined __FreeBSD__
567		snprintf(tmpstring, 63, "First: %u\n", first_seen);
568		write(nffd, tmpstring, strlen(tmpstring));
569		snprintf(tmpstring, 63, "Last: %u\n", last_seen);
570		#else
571		snprintf(tmpstring, 63, "First: %lu\n", first_seen);
572		write(nffd, tmpstring, strlen(tmpstring));
573		snprintf(tmpstring, 63, "Last: %lu\n", last_seen);
574		#endif
575		write(nffd, tmpstring, strlen(tmpstring));
576
577		close(nffd);
578		}
579
580		free((void *)record_buffer);
581		free((void *)ftrue);
582		SetSeenTwin(win_start, win_end);
583
584		return numflows;
	788	if ( rfd > 0 )
	789	close(rfd);
	790
	791	// flush output file
	792	if ( wfd ) {
	793	// flush current buffer to disc
	794	if ( out_flow_header->NumBlocks && !zero_flows) {
	795	if ( write(wfd, out_buff, sizeof(data_block_header_t) + out_flow_header->size) <= 0 ) {
	796	fprintf(stderr, "Failed to write output buffer to disk: '%s'" , strerror(errno));
	797	} else {
	798	file_blocks++;
	799	}
	800	}
	801
	802	/* Stat info */
	803	if ( write_file ) {
	804	/* Write stat info and close file */
	805	CloseUpdateFile(wfd, &stat_record, file_blocks, GetIdent(), &string );
	806	if ( string != NULL )
	807	fprintf(stderr, "%s\n", string);
	808	} // else stdout
	809	}
	810
	811	free((void *)in_buff);
	812	return stat_record;
585	813
586	814	} // End of process_data
587	815
588	816
589	817	int main( int argc, char **argv ) {
590	818	struct stat stat_buff;
	819	stat_record_t sum_stat, *sr;
591	820	printer_t print_header, print_record;
592	821	nfprof_t profile_data;
593	822	char c, rfile, Rfile, Mdirs, wfile, ffile, filter, tstring, stat_type;
594	823	char byte_limit_string, packet_limit_string, print_mode, record_header;
595	824	char *order_by, CryptoPAnKey[32];
596	825	int ffd, ret, element_stat, fdump;
597		int i, flow_stat, topN, aggregate, syntax_only, date_sorted, do_anonymize;
	826	int i, user_format, quiet, flow_stat, topN, aggregate, aggregate_mask;
	827	int print_stat, syntax_only, date_sorted, do_anonymize, zero_flows;
598	828	time_t t_start, t_end;
599		uint32_t limitflows, matched_flows, AggregateMasks[4];
	829	uint16_t Aggregate_Bits;
	830	uint32_t limitflows;
	831	uint64_t AggregateMasks[AGGR_SIZE];
600	832
601	833	rfile = Rfile = Mdirs = wfile = ffile = filter = tstring = stat_type = NULL;
602	834	byte_limit_string = packet_limit_string = NULL;
603	835	fdump = aggregate = 0;
	836	aggregate_mask = 0;
604	837	t_start = t_end = 0;
605	838	syntax_only = 0;
606	839	topN = 10;
607	840	flow_stat = 0;
	841	print_stat = 0;
608	842	element_stat = 0;
609	843	limitflows = 0;
610		matched_flows = 0;
611	844	date_sorted = 0;
612	845	total_bytes = 0;
613	846	total_flows = 0;
	847	skipped_flows = 0;
614	848	do_anonymize = 0;
	849	quiet = 0;
	850	user_format = 0;
	851	zero_flows = 0;
615	852
616	853	print_mode = NULL;
617	854	print_header = NULL;
618	855	print_record = NULL;
619	856	record_header = "";
	857	Aggregate_Bits = 0xFFFF; // set all bits
620	858
621	859	SetStat_DefaultOrder("flows");
622	860
623		for ( i=0; i<4; AggregateMasks[i++] = 0 ) ;
624
625		while ((c = getopt(argc, argv, "aA:c:Ss:hn:f:r:w:K:M:mO:R:XZt:Vv:l:L:o:")) != EOF) {
	861	for ( i=0; i<AGGR_SIZE; AggregateMasks[i++] = 0 ) ;
	862
	863	while ((c = getopt(argc, argv, "6aA:c:Ss:hn:f:qzr:w:K:M:ImO:R:XZt:Vv:l:L:o:")) != EOF) {
626	864	switch (c) {
627	865	case 'h':
628	866	usage(argv[0]);

632	870	aggregate = 1;
633	871	break;
634	872	case 'A':
635		if ( !ParseAggregateMask(optarg, AggregateMasks) ) {
636		fprintf(stderr, "Option -A requires a ',' separated list out of 'srcip dstip srcport dstport'\n");
	873	if ( !ParseAggregateMask(optarg, AggregateMasks, &Aggregate_Bits) ) {
	874	fprintf(stderr, "Option -A requires a ',' separated list out of 'srcip dstip srcport dstport srcip4[/netbits], dstip4[/netbits], srcip6[/netbits], dstip6[/netbits] '\n");
637	875	exit(255);
638	876	}
	877	aggregate_mask = 1;
639	878	break;
640	879	case 'X':
641	880	fdump = 1;
642	881	break;
643	882	case 'Z':
644	883	syntax_only = 1;
	884	break;
	885	case 'q':
	886	quiet = 1;
	887	break;
	888	case 'z':
	889	zero_flows = 1;
645	890	break;
646	891	case 'c':
647	892	limitflows = atoi(optarg);

652	897	break;
653	898	case 's':
654	899	stat_type = optarg;
655		if ( !SetStat(stat_type, &element_stat, &flow_stat) ) {
656		fprintf(stderr, "Stat '%s' unknown!\n", stat_type);
657		exit(255);
658		}
	900	if ( !SetStat(stat_type, &element_stat, &flow_stat) ) {
	901	exit(255);
	902	}
659	903	break;
660	904	case 'V':
661	905	printf("%s: Version: %s %s\n%s\n",argv[0], nfdump_version, nfdump_date, rcsid);

690	934	break;
691	935	case 'M':
692	936	Mdirs = optarg;
	937	break;
	938	case 'I':
	939	print_stat++;
693	940	break;
694	941	case 'o': // output mode
695	942	print_mode = optarg;

718	965	}
719	966	break;
720	967	case 'S': // Compatibility with pre 1.4 -S option
721		if ( !SetStat("record/packets/bytes", &element_stat, &flow_stat) ) {
722		// Should never happen
723		fprintf(stderr, "Software Error!\n");
724		exit(255);
725		}
	968	stat_type = "record/packets/bytes";
	969	if ( !SetStat(stat_type, &element_stat, &flow_stat) ) {
	970	// Should never happen
	971	exit(255);
	972	}
	973	break;
	974	case '6': // print long IPv6 addr
	975	Setv6Mode(1);
726	976	break;
727	977	default:
728	978	usage(argv[0]);

736	986	/* user specified a pcap filter */
737	987	filter = argv[optind];
738	988	}
	989
	990	if ( aggregate_mask && stat_type ) {
	991	char *stat_conflict = VerifyStat(Aggregate_Bits);
	992	if ( stat_conflict ) {
	993	fprintf(stderr, "Selected aggregation masks stat '%s'!\n", stat_conflict);
	994	exit(255);
	995	}
	996	}
739	997
740	998	if ( rfile && Rfile ) {
741	999	fprintf(stderr, "-r and -R are mutually exclusive. Plase specify either -r or -R\n");

746	1004	exit(255);
747	1005	}
748	1006
	1007	SetupInputFileSequence(Mdirs, rfile, Rfile);
	1008
	1009	if ( print_stat ) {
	1010	if ( !rfile && !Rfile && !Mdirs) {
	1011	fprintf(stderr, "Expect data file(s).\n");
	1012	exit(255);
	1013	}
	1014
	1015	memset((void *)&sum_stat, 0, sizeof(stat_record_t));
	1016	sum_stat.first_seen = 0x7fffffff;
	1017	sum_stat.msec_first = 999;
	1018	ffd = GetNextFile(0, 0, 0, &sr);
	1019	if ( ffd <= 0 ) {
	1020	if ( ffd == FILE_ERROR )
	1021	fprintf(stderr, "Error open file: %s\n", strerror(errno));
	1022	exit(250);
	1023	}
	1024	while ( ffd > 0 ) {
	1025	SumStatRecords(&sum_stat, sr);
	1026	ffd = GetNextFile(ffd, 0, 0, &sr);
	1027	}
	1028	PrintStat(&sum_stat);
	1029	exit(0);
	1030	}
749	1031
750	1032	// handle print mode
751	1033	if ( !print_mode )
752	1034	print_mode = DefaultMode;
753	1035
754		i = 0;
755		while ( printmap[i].printmode ) {
756		if ( strncasecmp(print_mode, printmap[i].printmode, MAXMODELEN) == 0 ) {
757		print_record = printmap[i].func;
758		record_header = printmap[i].HeaderLine;
759		if ( date_sorted && ( printmap[i].sorted == 0 ) ) {
760		date_sorted = 0;
761		fprintf(stderr, "Option -m does not make sense with output mode '%s'\n", print_mode);
762		}
763		break;
764		}
765		i++;
	1036
	1037	if ( strncasecmp(print_mode, "fmt:", 4) == 0 ) {
	1038	// special user defined output format
	1039	char *format = &print_mode[4];
	1040	if ( strlen(format) ) {
	1041	if ( !ParseOutputFormat(format) )
	1042	exit(255);
	1043	print_record = format_special;
	1044	record_header = format_special_header();
	1045	user_format = 1;
	1046	} else {
	1047	fprintf(stderr, "Missing format description for user defined output format!\n");
	1048	exit(255);
	1049	}
	1050	} else {
	1051	// predefined output format
	1052
	1053	// Check for long_v6 mode
	1054	i = strlen(print_mode);
	1055	if ( i > 2 ) {
	1056	if ( print_mode[i-1] == '6' ) {
	1057	Setv6Mode(1);
	1058	print_mode[i-1] = '\0';
	1059	} else
	1060	Setv6Mode(0);
	1061	}
	1062
	1063	i = 0;
	1064	while ( printmap[i].printmode ) {
	1065	if ( strncasecmp(print_mode, printmap[i].printmode, MAXMODELEN) == 0 ) {
	1066	if ( printmap[i].Format ) {
	1067	if ( !ParseOutputFormat(printmap[i].Format) )
	1068	exit(255);
	1069	// predefined custom format
	1070	print_record = printmap[i].func;
	1071	record_header = format_special_header();
	1072	user_format = 1;
	1073	} else {
	1074	// predefined static format
	1075	print_record = printmap[i].func;
	1076	record_header = printmap[i].HeaderLine;
	1077	user_format = 0;
	1078	}
	1079	break;
	1080	}
	1081	i++;
	1082	}
766	1083	}
767	1084
768	1085	if ( !print_record ) {

772	1089
773	1090	// this is the only case, where headers are printed.
774	1091	if ( strncasecmp(print_mode, "raw", 16) == 0 )
775		print_header = flow_header_raw;
	1092	print_header = format_file_block_header;
776	1093
777	1094	if ( aggregate && (flow_stat \|\| element_stat) ) {
778	1095	aggregate = 0;

822	1139	if ( syntax_only )
823	1140	exit(0);
824	1141
825		if ((aggregate \|\| flow_stat) && ( topN > 1000) ) {
826		printf("Topn N > 1000 only allowed for IP statistics");
	1142	if ((aggregate \|\| flow_stat) && ( topN > 1000 \|\| topN == 0) ) {
	1143	printf("TopN for record statistic: 0 < topN < 1000 only allowed for IP statistics\n");
827	1144	exit(255);
828	1145	}
829
830	1146
831	1147	if ((aggregate \|\| flow_stat \|\| date_sorted) && !Init_FlowTable(HashBits, NumPrealloc) )
832	1148	exit(250);

835	1151	exit(250);
836	1152
837	1153	SetLimits(element_stat \|\| aggregate \|\| flow_stat, packet_limit_string, byte_limit_string);
838
839		SetupInputFileSequence(Mdirs, rfile, Rfile);
840	1154
841	1155	if ( tstring ) {
842	1156	if ( !ScanTimeFrame(tstring, &t_start, &t_end) )

844	1158	}
845	1159
846	1160
847		if ( !(flow_stat \|\| element_stat \|\| wfile ) && record_header )
848		printf("%s\n", record_header);
	1161	if ( !(flow_stat \|\| element_stat \|\| wfile \|\| quiet ) && record_header ) {
	1162	if ( user_format ) {
	1163	printf("%s\n", record_header);
	1164	} else {
	1165	// static format - no static format with header any more, but keep code anyway
	1166	if ( Getv6Mode() ) {
	1167	printf("%s\n", record_header);
	1168	} else
	1169	printf("%s\n", record_header);
	1170	}
	1171	}
849	1172
850	1173	if (do_anonymize)
851	1174	PAnonymizer_Init((uint8_t *)CryptoPAnKey);
852	1175
853	1176	nfprof_start(&profile_data);
854		matched_flows = process_data(wfile, element_stat, aggregate \|\| flow_stat, date_sorted,
	1177	sum_stat = process_data(wfile, element_stat, aggregate \|\| flow_stat, date_sorted,
855	1178	print_header, print_record, t_start, t_end,
856		limitflows, AggregateMasks, do_anonymize);
	1179	limitflows, aggregate_mask ? AggregateMasks : NULL, do_anonymize, zero_flows);
857	1180	nfprof_end(&profile_data, total_flows);
	1181
	1182	if ( total_bytes == 0 )
	1183	exit(0);
858	1184
859	1185	if (aggregate) {
860	1186	ReportAggregated(print_record, limitflows, date_sorted, do_anonymize);
861		Dispose_Tables(1, 0); // Free the FlowTable
	1187	Dispose_Tables(1, 0); // Free FlowTable
862	1188	}
863	1189
864	1190	if (flow_stat \|\| element_stat) {

868	1194
869	1195	if ( date_sorted && !(aggregate \|\| flow_stat \|\| element_stat) ) {
870	1196	PrintSortedFlows(print_record, limitflows, do_anonymize);
871		Dispose_Tables(1, 0); // Free the FlowTable
872		}
873
874		if ( !wfile ) {
	1197	Dispose_Tables(1, 0); // Free FlowTable
	1198	}
	1199
	1200	if ( !wfile && !quiet ) {
875	1201	if (do_anonymize)
876	1202	printf("IP addresses anonymized\n");
877		printf("Time window: %s\n", TimeString());
878		printf("Flows analysed: %u matched: %u, Bytes read: %llu\n", total_flows, matched_flows, total_bytes);
	1203	PrintSummary(&sum_stat);
	1204	printf("Time window: %s\n", TimeString(t_first_flow, t_last_flow));
	1205	printf("Total flows processed: %u, skipped: %u, Bytes read: %llu\n",
	1206	total_flows, skipped_flows, total_bytes);
879	1207	nfprof_print(&profile_data, stdout);
880	1208	}
881	1209	return 0;

-92

nfdump.h less more

29	29	*
30	30	* $Author: peter $
31	31	*
32		* $Id: nfdump.h 34 2005-08-22 12:01:31Z peter $
	32	* $Id: nfdump.h 70 2006-05-17 08:38:01Z peter $
33	33	*
34		* $LastChangedRevision: 34 $
	34	* $LastChangedRevision: 70 $
35	35	*
36	36	*/
37	37
38	38	#define BuffNumRecords 1024
39	39
40	40	/*
41		* Defines for offsets into netflow record
	41	* Offset definitions for filter engine. Offsets must agree with the defined
	42	* flow record definition data_block_record_t in nffile.h
42	43	*/
43
44		#include "config.h"
45
46		#define AnyMask (uint32_t)0xffffffff
47
48		#ifdef WORDS_BIGENDIAN
49
50		#define OffsetSrcIP 0
51		#define OffsetDstIP 1
52		#define OffsetNext 2
53		#define MaskIP (uint32_t)0xffffffff
54
55		#define OffsetInOut 3
56		#define MaskInput (uint32_t)0xffff0000
57		#define MaskOutput (uint32_t)0x0000ffff
58		#define ShiftInput 16
59		#define ShiftOutput 0
60
61		#define OffsetPackets 4
62		#define OffsetBytes 5
63		#define MaskSize (uint32_t)0xffffffff
64
65		#define OffsetPort 8
66		#define MaskDstPort (uint32_t)0x0000ffff
67		#define MaskSrcPort (uint32_t)0xffff0000
68		#define ShiftDstPort 0
69		#define ShiftSrcPort 16
70
71		#define OffsetTos 9
72		#define MaskTos (uint32_t)0x000000ff
73		#define ShiftTos 0
74
75		#define OffsetProto 9
76		#define MaskProto (uint32_t)0x0000ff00
77		#define ShiftProto 8
78
79		#define OffsetFlags 9
80		#define MaskFlags (uint32_t)0x00ff0000
81		#define ShiftFlags 16
82
83		#define OffsetAS 10
84		#define MaskDstAS (uint32_t)0x0000ffff
85		#define MaskSrcAS (uint32_t)0xffff0000
86		#define ShiftSrcAS 16
87		#define ShiftDstAS 0
88
89		#else
90
91		#define OffsetSrcIP 0
92		#define OffsetDstIP 1
93		#define OffsetNext 2
94		#define MaskIP (uint32_t)0xffffffff
95
96		#define OffsetInOut 3
97		#define MaskInput (uint32_t)0x0000ffff
98		#define MaskOutput (uint32_t)0xffff0000
99		#define ShiftInput 0
100		#define ShiftOutput 16
101
102		#define OffsetPackets 4
103		#define OffsetBytes 5
104		#define MaskSize (uint32_t)0xffffffff
105
106		#define OffsetTos 9
107		#define MaskTos (uint32_t)0xff000000
108		#define ShiftTos 24
109
110		#define OffsetProto 9
111		#define MaskProto (uint32_t)0x00ff0000
112		#define ShiftProto 16
113
114		#define OffsetFlags 9
115		#define MaskFlags (uint32_t)0x0000ff00
116		#define ShiftFlags 8
117
118		#define OffsetPort 8
119		#define MaskDstPort (uint32_t)0xffff0000
120		#define MaskSrcPort (uint32_t)0x0000ffff
121		#define ShiftSrcPort 0
122		#define ShiftDstPort 16
123
124		#define OffsetAS 10
125		#define MaskDstAS (uint32_t)0xffff0000
126		#define MaskSrcAS (uint32_t)0x0000ffff
127		#define ShiftSrcAS 0
128		#define ShiftDstAS 16
129
130		#endif
131	44
132	45	typedef struct FilterParam {
133	46	uint32_t scale;

135	48	uint16_t direction;
136	49	uint16_t proto;
137	50	uint32_t data;
138		uint32_t ip;
	51	uint64_t ip[2];
139	52	uint32_t netmask;
140	53	uint32_t netbits;
141	54	uint32_t self;

+22

-17

nfdump.test.out less more

0		Date flow start Duration Proto Src IP Addr:Port Dst IP Addr:Port Flags Tos Packets Bytes pps bps Bpp Flows
1		2004-07-11 10:30:00.010 10.010 TCP 172.16.1.66:1024 -> 172.16.19.18:25 ...... 0 101 101 10 80 1 1
2		2004-07-11 10:30:10.110 20.010 TCP 172.16.1.66:1024 -> 172.16.19.18:25 ...... 0 101 101 5 40 1 1
3		2004-07-11 10:30:20.210 30.010 TCP 172.16.1.66:1024 -> 172.16.19.18:25 ...... 0 101 101 3 26 1 1
4		2004-07-11 10:30:30.310 40.010 UDP 172.16.2.66:2024 -> 172.16.18.18:25 .....F 1 1001 1001 25 200 1 1
5		2004-07-11 10:30:40.410 50.010 AH 172.16.3.66:3024 -> 172.16.17.18:25 ....S. 2 10001 10001 199 1599 1 1
6		2004-07-11 10:30:50.510 60.010 TCP 172.16.4.66:4024 -> 172.16.16.18:25 ...R.. 3 100001 100001 1666 13331 1 1
7		2004-07-11 10:31:00.610 70.010 TCP 172.16.5.66:5024 -> 172.16.15.18:25 ..P... 4 1000001 1000001 14283 114269 1 1
8		2004-07-11 10:31:10.710 80.010 TCP 172.16.5.66:5024 -> 172.16.15.18:25 .....F 4 9.5 M 1001 124984 100 0 1
9		2004-07-11 10:31:20.810 90.010 TCP 172.16.6.66:6024 -> 172.16.14.18:25 .A.... 5 500 9.5 M 5 888790 20000 1
10		2004-07-11 10:31:30.910 100.010 TCP 172.16.6.66:6024 -> 172.16.14.18:25 .A.... 5 500 9.5 M 4 799920 20000 1
11		2004-07-11 10:31:40.010 110.010 TCP 172.16.7.66:7024 -> 172.16.13.18:25 U..... 255 5000 95.4 M 45 6.9 M 20000 1
12		2004-07-11 10:31:50.110 120.010 TCP 172.16.8.66:8024 -> 172.16.12.18:25 UAPRSF 0 5000 953.7 M 41 63.6 M 200000 1
13		2004-07-11 10:32:00.210 130.010 ICMP 172.16.2.66:0 -> 172.16.18.18:8 ...... 0 50000 50000 384 3076 1 1
14		2004-07-11 10:32:10.310 140.010 TCP 172.160.160.166:10024 -> 172.160.160.180:25000 ...... 0 500000 500000 3571 28569 1 1
15		Time window: Jul 11 2004 10:30:00 - Jul 11 2004 10:34:30
16		Flows analysed: 14 matched: 14, Bytes read: 696
	0	2004-07-11 10:30:00.010 10.010 TCP 172.16.1.66:1024 -> 192.168.170.100:25 ...... 0 101 101 10 80 1 1
	1	2004-07-11 10:30:10.110 20.010 TCP 172.16.2.66:1024 -> 192.168.170.101:25 ...... 0 101 101 5 40 1 1
	2	2004-07-11 10:30:20.210 30.010 TCP 172.16.3.66:1024 -> 192.168.170.102:25 ...... 0 101 101 3 26 1 1
	3	2004-07-11 10:30:30.310 40.010 UDP 172.16.4.66:2024 -> 192.168.170.103:25 .....F 1 1001 1001 25 200 1 1
	4	2004-07-11 10:30:40.410 50.010 AH 172.16.5.66:3024 -> 192.168.170.104:25 ....S. 2 10001 10001 199 1599 1 1
	5	2004-07-11 10:30:50.510 60.010 TCP 172.16.6.66:4024 -> 192.168.170.105:25 ...R.. 3 100001 100001 1666 13331 1 1
	6	2004-07-11 10:31:00.610 70.010 TCP 172.16.7.66:5024 -> 192.168.170.106:25 ..P... 4 1000001 1000001 14283 114269 1 1
	7	2004-07-11 10:31:10.710 80.010 TCP 172.16.8.66:5024 -> 192.168.170.107:25 .....F 4 9.5 M 1001 124984 100 0 1
	8	2004-07-11 10:31:20.810 90.010 TCP 172.16.9.66:6024 -> 192.168.170.108:25 .A.... 5 500 9.5 M 5 888790 20000 1
	9	2004-07-11 10:31:30.910 100.010 TCP 172.16.10.66:6024 -> 192.168.170.109:25 .A.... 5 500 9.5 M 4 799920 20000 1
	10	2004-07-11 10:31:40.010 110.010 TCP 172.16.11.66:7024 -> 192.168.170.110:25 U..... 255 5000 95.4 M 45 6.9 M 20000 1
	11	2004-07-11 10:31:50.110 120.010 TCP 172.16.12.66:8024 -> 192.168.170.111:25 UAPRSF 0 5000 953.7 M 41 63.6 M 200000 1
	12	2004-07-11 10:32:00.210 130.010 ICMP 172.16.13.66:0 -> 192.168.170.112:8 ...... 0 50000 50000 384 3076 1 1
	13	2004-07-11 10:32:10.310 140.010 TCP 172.160.160.166:10024 -> 172.160.160.180:25000 ...... 0 500000 500000 3571 28569 1 1
	14	2004-07-11 10:32:20.410 150.010 TCP fe80::2..:1234:0.1024 -> fe80::2..35:4321.25 .AP.SF 0 10 15100 0 805 1510 1
	15	2004-07-11 10:32:30.510 160.010 TCP 2001:23..80:d01e.10240 -> 2001:62..52:38e5.52345 .AP.SF 0 10100 14.3 M 63 749953 1485 1
	16	2004-07-11 10:32:40.610 170.010 TCP 2001:23..80:d01e.10240 -> 2001:62..52:38e5.52345 .AP.SF 0 9.6 M 4.0 G 59408 192.7 M 425 1
	17	2004-07-11 10:32:50.710 180.010 TCP 2001:23..80:d01e.10240 -> 2001:62..52:38e5.52345 .AP.SF 0 4.0 G 14.3 M 22.8 M 666629 0 1
	18	2004-07-11 10:33:00.810 190.010 TCP 2001:23..80:d01e.10240 -> 2001:62..52:38e5.52345 .AP.SF 0 4.0 G 8.0 G 21.6 M 344.9 M 2 1
	19	2004-07-11 10:33:10.910 200.010 TCP 172.16.14.18:10240 -> 192.168.170.113:52345 .AP.SF 0 9.6 M 4.0 G 50497 163.8 M 425 1
	20	2004-07-11 10:33:20.010 210.010 TCP 172.16.15.18:10240 -> 192.168.170.114:52345 .AP.SF 0 4.0 G 14.3 M 19.5 M 571401 0 1
	21	2004-07-11 10:33:30.110 220.010 TCP 172.16.16.18:10240 -> 192.168.170.115:52345 .AP.SF 0 4.0 G 8.0 G 18.6 M 297.9 M 2 1

+521

-0

nffile.c less more

	0	/*
	1	* This file is part of the nfdump project.
	2	*
	3	* Copyright (c) 2004, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
	4	* All rights reserved.
	5	*
	6	* Redistribution and use in source and binary forms, with or without
	7	* modification, are permitted provided that the following conditions are met:
	8	*
	9	* * Redistributions of source code must retain the above copyright notice,
	10	* this list of conditions and the following disclaimer.
	11	* * Redistributions in binary form must reproduce the above copyright notice,
	12	* this list of conditions and the following disclaimer in the documentation
	13	* and/or other materials provided with the distribution.
	14	* * Neither the name of SWITCH nor the names of its contributors may be
	15	* used to endorse or promote products derived from this software without
	16	* specific prior written permission.
	17	*
	18	* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
	19	* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	20	* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	21	* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
	22	* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
	23	* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
	24	* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
	25	* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
	26	* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
	27	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
	28	* POSSIBILITY OF SUCH DAMAGE.
	29	*
	30	* $Author: peter $
	31	*
	32	* $Id: nffile.c 70 2006-05-17 08:38:01Z peter $
	33	*
	34	* $LastChangedRevision: 70 $
	35	*
	36	*/
	37
	38	#include <sys/types.h>
	39	#include <sys/uio.h>
	40	#include <unistd.h>
	41	#include <string.h>
	42	#include <fcntl.h>
	43	#include <sys/stat.h>
	44	#include <stdio.h>
	45	#include <errno.h>
	46	#include <unistd.h>
	47	#include <stdlib.h>
	48
	49	#include "config.h"
	50
	51	#ifdef HAVE_STDINT_H
	52	#include <stdint.h>
	53	#endif
	54
	55	#include "nffile.h"
	56
	57	const uint16_t MAGIC = 0xA50C;
	58	const uint16_t VERSION = 1;
	59
	60	char *CurrentIdent;
	61
	62	/* local vars */
	63	static file_header_t FileHeader;
	64	static stat_record_t NetflowStat;
	65
	66	#define ERR_SIZE 256
	67	static char error_string[ERR_SIZE];
	68
	69	#ifdef COMPAT14
	70	int Format14;
	71	uint32_t tstamp;
	72
	73	static void Compat14_ReadStat(char *sfile);
	74	#endif
	75
	76	#if ( SIZEOF_VOID_P == 8 )
	77	typedef uint64_t pointer_addr_t;
	78	#else
	79	typedef uint32_t pointer_addr_t;
	80	#endif
	81
	82	/* function prototypes */
	83
	84	static void ZeroStat(void);
	85
	86
	87	/* function definitions */
	88
	89	static void ZeroStat() {
	90
	91	FileHeader.NumBlocks = 0;
	92	strncpy(FileHeader.ident, IdentNone, IdentLen);
	93
	94	NetflowStat.first_seen = 0;
	95	NetflowStat.last_seen = 0;
	96	NetflowStat.msec_first = 0;
	97	NetflowStat.msec_last = 0;
	98
	99	CurrentIdent = FileHeader.ident;
	100
	101	} // End of ZeroStat
	102
	103	char *GetIdent(void) {
	104
	105	return CurrentIdent;
	106
	107	} // End of GetIdent
	108
	109
	110	int OpenFile(char filename, stat_record_t stat_record, char *err){
	111	struct stat stat_buf;
	112	int fd;
	113
	114	*err = NULL;
	115	if ( stat_record )
	116	*stat_record = &NetflowStat;
	117
	118	if ( filename == NULL ) {
	119	// stdin
	120	ZeroStat();
	121	fd = STDIN_FILENO;
	122	} else {
	123	// regular file
	124	if ( stat(filename, &stat_buf) ) {
	125	snprintf(error_string, ERR_SIZE, "Can't stat '%s': %s\n", filename, strerror(errno));
	126	error_string[ERR_SIZE-1] = 0;
	127	*err = error_string;
	128	ZeroStat();
	129	return -1;
	130	}
	131
	132	if (!S_ISREG(stat_buf.st_mode) ) {
	133	snprintf(error_string, ERR_SIZE, "'%s' is not a file\n", filename);
	134	error_string[ERR_SIZE-1] = 0;
	135	*err = error_string;
	136	ZeroStat();
	137	return -1;
	138	}
	139
	140	// printf("Statfile %s\n",filename);
	141	fd = open(filename, O_RDONLY);
	142	if ( fd < 0 ) {
	143	snprintf(error_string, ERR_SIZE, "Error open file: %s\n", strerror(errno));
	144	error_string[ERR_SIZE-1] = 0;
	145	*err = error_string;
	146	ZeroStat();
	147	return fd;
	148	}
	149
	150	}
	151
	152	#ifdef COMPAT14
	153	Format14 = 0;
	154	#endif
	155	read(fd, (void *)&FileHeader, sizeof(FileHeader));
	156	if ( FileHeader.magic != MAGIC ) {
	157	#ifdef COMPAT14
	158	if ( FileHeader.magic == 5 ) {
	159	Format14 = 1;
	160	lseek(fd, 0, SEEK_SET);
	161
	162	FileHeader.magic = MAGIC;
	163	FileHeader.version = 0;
	164	FileHeader.flags = 0;
	165	FileHeader.NumBlocks = 0;
	166	Compat14_ReadStat(filename);
	167	CurrentIdent = FileHeader.ident;
	168	return fd;
	169	}
	170	#endif
	171	snprintf(error_string, ERR_SIZE, "Open file: bad magic: 0x%X\n", FileHeader.magic );
	172	error_string[ERR_SIZE-1] = 0;
	173	*err = error_string;
	174	ZeroStat();
	175	close(fd);
	176	return -1;
	177	}
	178	if ( FileHeader.version != VERSION ) {
	179	snprintf(error_string, ERR_SIZE,"Open file: bad version: %u\n", FileHeader.version );
	180	error_string[ERR_SIZE-1] = 0;
	181	*err = error_string;
	182	ZeroStat();
	183	close(fd);
	184	return -1;
	185	}
	186	read(fd, (void *)&NetflowStat, sizeof(NetflowStat));
	187
	188	// for debugging:
	189	/*
	190	printf("Magic: 0x%X\n", FileHeader.magic);
	191	printf("Version: %i\n", FileHeader.version);
	192	printf("Flags: %i\n", FileHeader.flags);
	193	printf("NumBlocks: %i\n", FileHeader.NumBlocks);
	194	printf("Ident: %s\n\n", FileHeader.ident);
	195
	196	printf("Flows: %llu\n", NetflowStat.numflows);
	197	printf("Flows_tcp: %llu\n", NetflowStat.numflows_tcp);
	198	printf("Flows_udp: %llu\n", NetflowStat.numflows_udp);
	199	printf("Flows_icmp: %llu\n", NetflowStat.numflows_icmp);
	200	printf("Flows_other: %llu\n", NetflowStat.numflows_other);
	201	printf("Packets: %llu\n", NetflowStat.numpackets);
	202	printf("Packets_tcp: %llu\n", NetflowStat.numpackets_tcp);
	203	printf("Packets_udp: %llu\n", NetflowStat.numpackets_udp);
	204	printf("Packets_icmp: %llu\n", NetflowStat.numpackets_icmp);
	205	printf("Packets_other: %llu\n", NetflowStat.numpackets_other);
	206	printf("Bytes: %llu\n", NetflowStat.numbytes);
	207	printf("Bytes_tcp: %llu\n", NetflowStat.numbytes_tcp);
	208	printf("Bytes_udp: %llu\n", NetflowStat.numbytes_udp);
	209	printf("Bytes_icmp: %llu\n", NetflowStat.numbytes_icmp);
	210	printf("Bytes_other: %llu\n", NetflowStat.numbytes_other);
	211	printf("First: %u\n", NetflowStat.first_seen);
	212	printf("Last: %u\n", NetflowStat.last_seen);
	213	printf("msec_first: %u\n", NetflowStat.msec_first);
	214	printf("msec_last: %u\n", NetflowStat.msec_last);
	215	*/
	216	CurrentIdent = FileHeader.ident;
	217	return fd;
	218
	219	} // End of OpenFile
	220
	221	void PrintStat(stat_record_t *s) {
	222
	223	if ( s == NULL )
	224	s = &NetflowStat;
	225
	226	#ifdef COMPAT14
	227	if ( Format14 )
	228	printf("Time: %u\n", tstamp);
	229	#endif
	230	printf("Ident: %s\n", FileHeader.ident);
	231	printf("Flows: %llu\n", s->numflows);
	232	printf("Flows_tcp: %llu\n", s->numflows_tcp);
	233	printf("Flows_udp: %llu\n", s->numflows_udp);
	234	printf("Flows_icmp: %llu\n", s->numflows_icmp);
	235	printf("Flows_other: %llu\n", s->numflows_other);
	236	printf("Packets: %llu\n", s->numpackets);
	237	printf("Packets_tcp: %llu\n", s->numpackets_tcp);
	238	printf("Packets_udp: %llu\n", s->numpackets_udp);
	239	printf("Packets_icmp: %llu\n", s->numpackets_icmp);
	240	printf("Packets_other: %llu\n", s->numpackets_other);
	241	printf("Bytes: %llu\n", s->numbytes);
	242	printf("Bytes_tcp: %llu\n", s->numbytes_tcp);
	243	printf("Bytes_udp: %llu\n", s->numbytes_udp);
	244	printf("Bytes_icmp: %llu\n", s->numbytes_icmp);
	245	printf("Bytes_other: %llu\n", s->numbytes_other);
	246	printf("First: %u\n", s->first_seen);
	247	printf("Last: %u\n", s->last_seen);
	248	#ifdef COMPAT14
	249	if ( !Format14 ) {
	250	#endif
	251	printf("msec_first: %u\n", s->msec_first);
	252	printf("msec_last: %u\n", s->msec_last);
	253	printf("Sequence failures: %u\n", s->sequence_failure);
	254	#ifdef COMPAT14
	255	}
	256	#endif
	257	} // End of PrintStat
	258
	259	int OpenNewFile(char filename, char *err) {
	260	file_header_t *file_header;
	261	size_t len;
	262	int nffd;
	263
	264	*err = NULL;
	265	nffd = open(filename, O_CREAT \| O_RDWR \| O_TRUNC, S_IRUSR \| S_IWUSR \| S_IRGRP \| S_IROTH );
	266	if ( nffd < 0 ) {
	267	return -1;
	268	}
	269
	270	len = sizeof(file_header_t) + sizeof(stat_record_t);
	271	file_header = (file_header_t *)malloc(len);
	272	memset((void *)file_header, 0, len);
	273
	274	/* magic set, version = 0 and flags = 0 => file open for writing */
	275	file_header->magic = MAGIC;
	276	if ( write(nffd, (void *)file_header, len) < len ) {
	277	snprintf(error_string, ERR_SIZE, "Failed to write file header: '%s'" , strerror(errno));
	278	error_string[ERR_SIZE-1] = 0;
	279	*err = error_string;
	280	close(nffd);
	281	return -1;
	282	}
	283
	284	return nffd;
	285
	286	} /* End of OpenNewFile */
	287
	288	void CloseUpdateFile(int fd, stat_record_t stat_record, uint32_t record_count, char ident, char **err ) {
	289	file_header_t file_header;
	290
	291	*err = NULL;
	292
	293	file_header.magic = MAGIC;
	294	file_header.version = VERSION;
	295	file_header.flags = 0;
	296	file_header.NumBlocks = record_count;
	297	strncpy(file_header.ident, ident, IdentLen);
	298	file_header.ident[IdentLen - 1] = 0;
	299
	300	if ( lseek(fd, 0, SEEK_SET) < 0 ) {
	301	snprintf(error_string, ERR_SIZE,"lseek failed: '%s'\n" , strerror(errno));
	302	error_string[ERR_SIZE-1] = 0;
	303	*err = error_string;
	304	close(fd);
	305	return;
	306	}
	307
	308	write(fd, (void *)&file_header, sizeof(file_header_t));
	309	write(fd, (void *)stat_record, sizeof(stat_record_t));
	310	if ( close(fd) < 0 ) {
	311	snprintf(error_string, ERR_SIZE,"close failed: '%s'" , strerror(errno));
	312	error_string[ERR_SIZE-1] = 0;
	313	*err = error_string;
	314	}
	315
	316	close(fd);
	317	return;
	318
	319	} /* End of CloseUpdateFile */
	320
	321	/*
	322	* Expand file record into master record for further processing
	323	* LP64 CPUs need special 32bit operations as it is not guarateed, that 64bit
	324	* values are aligned
	325	*/
	326	inline void ExpandRecord(common_record_t input_record,master_record_t output_record ) {
	327	uint32_t *u;
	328	size_t size;
	329	void p = (void )input_record;
	330
	331	// Copy common data block
	332	size = sizeof(common_record_t) - sizeof(uint8_t[4]);
	333	memcpy((void *)output_record, p, size);
	334	p = (void *)input_record->data;
	335
	336	output_record->fill = 0;
	337
	338	if ( (input_record->flags & FLAG_IPV6_ADDR) != 0 ) { // IPv6
	339	// IPv6
	340	memcpy((void *)output_record->v6.srcaddr, p, sizeof(ipv6_block_t));
	341	p = (void *)((pointer_addr_t)p + sizeof(ipv6_block_t));
	342	} else {
	343	// IPv4
	344	u = (uint32_t *)p;
	345	output_record->v6.srcaddr[0] = 0;
	346	output_record->v6.srcaddr[1] = 0;
	347	output_record->v4.srcaddr = u[0];
	348
	349	output_record->v6.dstaddr[0] = 0;
	350	output_record->v6.dstaddr[1] = 0;
	351	output_record->v4.dstaddr = u[1];
	352	p = (void )((pointer_addr_t)p + 2 sizeof(uint32_t));
	353	}
	354
	355	// packet counter
	356	if ( (input_record->flags & FLAG_PKG_64 ) != 0 ) {
	357	// 64bit packet counter
	358	value64_t l, v = (value64_t )p;
	359	l.val.val32[0] = v->val.val32[0];
	360	l.val.val32[1] = v->val.val32[1];
	361	output_record->dPkts = l.val.val64;
	362	p = (void *)((pointer_addr_t)p + sizeof(uint64_t));
	363	} else {
	364	// 32bit packet counter
	365	output_record->dPkts = ((uint32_t )p);
	366	p = (void *)((pointer_addr_t)p + sizeof(uint32_t));
	367	}
	368
	369	// byte counter
	370	if ( (input_record->flags & FLAG_BYTES_64 ) != 0 ) {
	371	// 64bit byte counter
	372	value64_t l, v = (value64_t )p;
	373	l.val.val32[0] = v->val.val32[0];
	374	l.val.val32[1] = v->val.val32[1];
	375	output_record->dOctets = l.val.val64;
	376	p = (void *)((pointer_addr_t)p + sizeof(uint64_t));
	377	} else {
	378	// 32bit bytes counter
	379	output_record->dOctets = ((uint32_t )p);
	380	p = (void *)((pointer_addr_t)p + sizeof(uint32_t));
	381	}
	382
	383	} // End of ExpandRecord
	384
	385	#ifdef COMPAT14
	386
	387	/*
	388	* Everything below is code to read nfdump <= v1.4 binary data files.
	389	* This code will be removed in furture
	390	*/
	391
	392	#ifdef __SUNPRO_C
	393	extern
	394	#endif
	395	inline ssize_t Compat14_ReadHeader(int fd, data_block_header_t *flow_header) {
	396	compat14_flow_header_t compat14_header;
	397	ssize_t num;
	398
	399	num = read(fd, (void *)&compat14_header, sizeof(compat14_flow_header_t));
	400	if ( num <= 0 ) {
	401	flow_header->NumBlocks = 0;
	402	flow_header->size = 0;
	403	flow_header->id = DATA_BLOCK_TYPE_1;
	404	flow_header->pad = 0;
	405	return num;
	406	}
	407
	408	flow_header->NumBlocks = compat14_header.count;
	409	flow_header->size = compat14_header.count * sizeof(comapt14_flow_record_t);
	410	flow_header->id = DATA_BLOCK_TYPE_1;
	411	flow_header->pad = 0;
	412
	413	return num;
	414
	415	} // End of Compat14_ReadHeader
	416
	417	#ifdef __SUNPRO_C
	418	extern
	419	#endif
	420	inline ssize_t Compat14_ReadRecords(int fd, void buffer, data_block_header_t flow_header) {
	421	comapt14_flow_record_t compat14_records[30];
	422	common_record_t *record_buffer;
	423	uint32_t *val, record_size;
	424
	425	ssize_t num;
	426	int i;
	427
	428	record_buffer = (common_record_t *)buffer;
	429	val = (uint32_t *)record_buffer->data;
	430	record_size = (pointer_addr_t)&val[4] - (pointer_addr_t)buffer;
	431
	432	if ( flow_header->NumBlocks > 30 \|\| flow_header->NumBlocks == 0 )
	433	return 0;
	434
	435	num = read(fd, (void )compat14_records, flow_header->NumBlocks sizeof(comapt14_flow_record_t));
	436	if ( num <= 0 )
	437	return num;
	438
	439	if ( num != flow_header->NumBlocks * sizeof(comapt14_flow_record_t) )
	440	return -1;
	441
	442	for ( i=0; i<flow_header->NumBlocks; i++ ) {
	443	record_buffer->flags = 0;
	444	record_buffer->size = record_size;
	445	record_buffer->mark = 0;
	446	record_buffer->first = compat14_records[i].First;
	447	record_buffer->last = compat14_records[i].Last;
	448	record_buffer->msec_first = compat14_records[i].msec_first;
	449	record_buffer->msec_last = compat14_records[i].msec_last;
	450
	451	record_buffer->dir = 0;
	452	record_buffer->tcp_flags = compat14_records[i].tcp_flags;
	453	record_buffer->prot = compat14_records[i].prot;
	454	record_buffer->tos = compat14_records[i].tos;
	455	record_buffer->input = compat14_records[i].input;
	456	record_buffer->output = compat14_records[i].output;
	457	record_buffer->srcas = compat14_records[i].src_as;
	458	record_buffer->dstas = compat14_records[i].dst_as;
	459	record_buffer->srcport = compat14_records[i].srcport;
	460	record_buffer->dstport = compat14_records[i].dstport;
	461
	462	val[0] = compat14_records[i].srcaddr;
	463	val[1] = compat14_records[i].dstaddr;
	464	val[2] = compat14_records[i].dPkts;
	465	val[3] = compat14_records[i].dOctets;
	466
	467	record_buffer = (common_record_t *)((pointer_addr_t)record_buffer + record_size);
	468	val = (uint32_t *)record_buffer->data;
	469	}
	470	return num;
	471
	472	} // End of Compat14_ReadRecords
	473
	474	static void Compat14_ReadStat(char *sfile){
	475	FILE *fd;
	476	char stat_filename[256];
	477
	478	ZeroStat();
	479	if ( sfile == NULL )
	480	return;
	481
	482	strncpy(stat_filename, sfile, 256);
	483	sfile[255] = 0;
	484	strncat(stat_filename, ".stat", 256);
	485	sfile[255] = 0;
	486
	487	fd = fopen(stat_filename, "r");
	488	if ( !fd ) {
	489	return;
	490	}
	491
	492	fscanf(fd, "Time: %u\n", &tstamp);
	493	fscanf(fd, "Ident: %s\n", FileHeader.ident);
	494	fscanf(fd, "Flows: %llu\n", &NetflowStat.numflows);
	495	fscanf(fd, "Flows_tcp: %llu\n", &NetflowStat.numflows_tcp);
	496	fscanf(fd, "Flows_udp: %llu\n", &NetflowStat.numflows_udp);
	497	fscanf(fd, "Flows_icmp: %llu\n", &NetflowStat.numflows_icmp);
	498	fscanf(fd, "Flows_other: %llu\n", &NetflowStat.numflows_other);
	499	fscanf(fd, "Packets: %llu\n", &NetflowStat.numpackets);
	500	fscanf(fd, "Packets_tcp: %llu\n", &NetflowStat.numpackets_tcp);
	501	fscanf(fd, "Packets_udp: %llu\n", &NetflowStat.numpackets_udp);
	502	fscanf(fd, "Packets_icmp: %llu\n", &NetflowStat.numpackets_icmp);
	503	fscanf(fd, "Packets_other: %llu\n", &NetflowStat.numpackets_other);
	504	fscanf(fd, "Bytes: %llu\n", &NetflowStat.numbytes);
	505	fscanf(fd, "Bytes_tcp: %llu\n", &NetflowStat.numbytes_tcp);
	506	fscanf(fd, "Bytes_udp: %llu\n", &NetflowStat.numbytes_udp);
	507	fscanf(fd, "Bytes_icmp: %llu\n", &NetflowStat.numbytes_icmp);
	508	fscanf(fd, "Bytes_other: %llu\n", &NetflowStat.numbytes_other);
	509	fscanf(fd, "First: %u\n", &NetflowStat.first_seen);
	510	fscanf(fd, "Last: %u\n", &NetflowStat.last_seen);
	511	NetflowStat.msec_first = 0;
	512	NetflowStat.msec_last = 0;
	513	NetflowStat.sequence_failure = 0;
	514
	515	fclose(fd);
	516
	517	} // End of Compat14_ReadStat
	518
	519	#endif
	520

+447

-0

nffile.h less more

	0	/*
	1	* This file is part of the nfdump project.
	2	*
	3	* Copyright (c) 2004, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
	4	* All rights reserved.
	5	*
	6	* Redistribution and use in source and binary forms, with or without
	7	* modification, are permitted provided that the following conditions are met:
	8	*
	9	* * Redistributions of source code must retain the above copyright notice,
	10	* this list of conditions and the following disclaimer.
	11	* * Redistributions in binary form must reproduce the above copyright notice,
	12	* this list of conditions and the following disclaimer in the documentation
	13	* and/or other materials provided with the distribution.
	14	* * Neither the name of SWITCH nor the names of its contributors may be
	15	* used to endorse or promote products derived from this software without
	16	* specific prior written permission.
	17	*
	18	* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
	19	* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	20	* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	21	* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
	22	* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
	23	* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
	24	* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
	25	* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
	26	* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
	27	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
	28	* POSSIBILITY OF SUCH DAMAGE.
	29	*
	30	* $Author: peter $
	31	*
	32	* $Id: nffile.h 70 2006-05-17 08:38:01Z peter $
	33	*
	34	* $LastChangedRevision: 70 $
	35	*
	36	*/
	37
	38	#include "config.h"
	39
	40	#include <sys/types.h>
	41
	42	#ifdef HAVE_STDINT_H
	43	#include <stdint.h>
	44	#endif
	45
	46
	47	#define IdentLen 128
	48	#define IdentNone "none"
	49
	50	/*
	51	*
	52	* bit 0:
	53	*/
	54	typedef struct file_header_s {
	55	uint16_t magic; // magic to recognize endian type
	56	uint16_t version; // version of binary file layout, incl. magic
	57	uint32_t flags; //
	58	uint32_t NumBlocks; // number of blocks in file
	59	char ident[IdentLen]; // identifies this data
	60	} file_header_t;
	61
	62	typedef struct stat_record_s {
	63	// overall stat
	64	uint64_t numflows;
	65	uint64_t numbytes;
	66	uint64_t numpackets;
	67	// flow stat
	68	uint64_t numflows_tcp;
	69	uint64_t numflows_udp;
	70	uint64_t numflows_icmp;
	71	uint64_t numflows_other;
	72	// bytes stat
	73	uint64_t numbytes_tcp;
	74	uint64_t numbytes_udp;
	75	uint64_t numbytes_icmp;
	76	uint64_t numbytes_other;
	77	// packet stat
	78	uint64_t numpackets_tcp;
	79	uint64_t numpackets_udp;
	80	uint64_t numpackets_icmp;
	81	uint64_t numpackets_other;
	82	// time window
	83	uint32_t first_seen;
	84	uint32_t last_seen;
	85	uint16_t msec_first;
	86	uint16_t msec_last;
	87	// other
	88	uint32_t sequence_failure;
	89	} stat_record_t;
	90
	91	typedef struct data_block_header_s {
	92	uint32_t NumBlocks; // number of data records in data block
	93	uint32_t size; // number of this block in bytes without this header
	94	uint16_t id; // flags for this block
	95	uint16_t pad;
	96	} data_block_header_t;
	97
	98	#define DATA_BLOCK_TYPE_1 1
	99
	100	typedef struct common_record_s {
	101	// the head of each data record
	102	uint32_t flags;
	103	uint16_t size;
	104	uint16_t mark;
	105	uint16_t msec_first;
	106	uint16_t msec_last;
	107	uint32_t first;
	108	uint32_t last;
	109
	110	uint8_t dir;
	111	uint8_t tcp_flags;
	112	uint8_t prot;
	113	uint8_t tos;
	114	uint16_t input;
	115	uint16_t output;
	116	uint16_t srcport;
	117	uint16_t dstport;
	118	uint16_t srcas;
	119	uint16_t dstas;
	120	uint8_t data[4]; // .. more data below
	121	} common_record_t;
	122
	123	#define BYTE_OFFSET_first 12
	124
	125	// number of netflow elements in data_block_record_t: first .. dOctets
	126	#define NumElements 18
	127
	128	typedef struct ipv4_block_s {
	129	#ifdef WORDS_BIGENDIAN
	130	uint32_t fill1[3];
	131	uint32_t srcaddr;
	132	uint32_t fill2[3];
	133	uint32_t dstaddr;
	134	#else
	135	uint32_t fill1[2];
	136	uint32_t srcaddr;
	137	uint32_t fill2;
	138	uint32_t fill3[2];
	139	uint32_t dstaddr;
	140	uint32_t fill4;
	141	#endif
	142	} ipv4_block_t;
	143
	144	typedef struct ipv6_block_s {
	145	uint64_t srcaddr[2];
	146	uint64_t dstaddr[2];
	147	} ipv6_block_t;
	148
	149	typedef struct ip_block_s {
	150	union {
	151	ipv4_block_t _v4;
	152	ipv6_block_t _v6;
	153	} ip_union;
	154	#define v4 ip_union._v4
	155	#define v6 ip_union._v6
	156	uint8_t data[4]; // .. more data below
	157	} ip_block_t;
	158
	159	typedef struct value32_s {
	160	uint32_t val;
	161	uint8_t data[4]; // .. more data below
	162	} value32_t;
	163
	164	typedef struct value64_s {
	165	union val_s {
	166	uint64_t val64;
	167	uint32_t val32[2];
	168	} val;
	169	uint8_t data[4]; // .. more data below
	170	} value64_t;
	171
	172	/* the master record contains all possible records unpacked */
	173	typedef struct master_record_s {
	174	// common information from all netflow versions
	175	// // interpreted as uint64_t[]
	176	uint32_t flags; // index 0 0xffff'ffff'0000'0000
	177	uint16_t size; // index 0 0x0000'0000'ffff'0000
	178	uint16_t mark; // index 0 0x0000'0000'0000'ffff
	179	uint16_t msec_first; // index 1 0xffff'0000'0000'0000
	180	uint16_t msec_last; // index 1 0x0000'ffff'0000'0000
	181	uint32_t first; // index 1 0x0000'0000'ffff'ffff
	182	uint32_t last; // index 2 0xffff'ffff'0000'0000
	183	uint8_t dir; // index 2 0x0000'0000'ff00'0000
	184	uint8_t tcp_flags; // index 2 0x0000'0000'00ff'0000
	185	uint8_t prot; // index 2 0x0000'0000'0000'ff00
	186	uint8_t tos; // index 2 0x0000'0000'0000'00ff
	187	uint16_t input; // index 3 0xffff'0000'0000'0000
	188	uint16_t output; // index 3 0x0000'ffff'0000'0000
	189	uint16_t srcport; // index 3 0x0000'0000'ffff'0000
	190	uint16_t dstport; // index 3 0x0000'0000'0000'ffff
	191	uint16_t srcas; // index 4 0xffff'0000'0000'0000
	192	uint16_t dstas; // index 4 0x0000'ffff'0000'0000
	193	// align 64bit
	194	uint32_t fill; // index 4 0x0000'0000'ffff'ffff
	195
	196	// IP address block
	197	union {
	198	ipv4_block_t _v4; // srcaddr index 6 0x0000'0000'ffff'ffff
	199	// dstaddr index 8 0x0000'0000'ffff'ffff
	200	ipv6_block_t _v6; // srcaddr[0-1] index 5 0xffff'ffff'ffff'ffff
	201	// srcaddr[2-3] index 6 0xffff'ffff'ffff'ffff
	202	// dstaddr[0-1] index 7 0xffff'ffff'ffff'ffff
	203	// dstaddr[2-3] index 8 0xffff'ffff'ffff'ffff
	204	} ip_union;
	205
	206	// counter block - expanded to 8 bytes
	207	uint64_t dPkts; // index 9 0xffff'ffff'ffff'ffff
	208	uint64_t dOctets; // index 10 0xffff'ffff'ffff'ffff
	209
	210	} master_record_t;
	211
	212	typedef struct type_mask_s {
	213	union {
	214	uint8_t val8[8];
	215	uint16_t val16[4];
	216	uint32_t val32[2];
	217	uint64_t val64;
	218	} val;
	219	} type_mask_t;
	220
	221
	222	#define AnyMask 0xffffffffffffffffLL
	223
	224	#ifdef WORDS_BIGENDIAN
	225
	226	#define OffsetRecordFlags 0
	227	#define MaskRecordFlags 0xffffffff00000000LL
	228	#define ShiftRecordFlags 32
	229
	230	#define OffsetDir 2
	231	#define MaskDir 0x00000000ff000000LL
	232	#define ShiftDir 24
	233
	234	#define OffsetFlags 2
	235	#define MaskFlags 0x0000000000ff0000LL
	236	#define ShiftFlags 16
	237
	238	#define OffsetProto 2
	239	#define MaskProto 0x000000000000ff00LL
	240	#define ShiftProto 8
	241
	242	#define OffsetTos 2
	243	#define MaskTos 0x00000000000000ffLL
	244	#define ShiftTos 0
	245
	246	#define OffsetInOut 3
	247	#define MaskInput 0xffff000000000000LL
	248	#define MaskOutput 0x0000ffff00000000LL
	249	#define ShiftInput 48
	250	#define ShiftOutput 32
	251
	252	#define OffsetPort 3
	253	#define MaskDstPort 0x000000000000ffffLL
	254	#define MaskSrcPort 0x00000000ffff0000LL
	255	#define ShiftDstPort 0
	256	#define ShiftSrcPort 16
	257
	258	#define OffsetAS 4
	259	#define MaskDstAS 0x0000ffff00000000LL
	260	#define MaskSrcAS 0xffff000000000000LL
	261	#define ShiftSrcAS 48
	262	#define ShiftDstAS 32
	263
	264	#define OffsetSrcIPv4 6
	265	#define MaskSrcIPv4 0x00000000ffffffffLL
	266	#define ShiftSrcIPv4 0
	267
	268	#define OffsetDstIPv4 8
	269	#define MaskDstIPv4 0x00000000ffffffffLL
	270	#define ShiftDstIPv4 0
	271
	272	#define OffsetSrcIPv6a 5
	273	#define OffsetSrcIPv6b 6
	274	#define OffsetDstIPv6a 7
	275	#define OffsetDstIPv6b 8
	276	#define MaskIPv6 0xffffffffffffffffLL
	277	#define ShiftIPv6 0
	278
	279	#define OffsetPackets 9
	280	#define MaskPackets 0xffffffffffffffffLL
	281	#define ShiftPackets 0
	282
	283	#define OffsetBytes 10
	284	#define MaskBytes 0xffffffffffffffffLL
	285	#define ShiftBytes 0
	286
	287	#else
	288
	289	#define OffsetRecordFlags 0
	290	#define MaskRecordFlags 0x00000000ffffffffLL
	291	#define ShiftRecordFlags 0
	292
	293	#define OffsetDir 2
	294	#define MaskDir 0x000000ff00000000LL
	295	#define ShiftDir 32
	296
	297	#define OffsetFlags 2
	298	#define MaskFlags 0x0000ff0000000000LL
	299	#define ShiftFlags 40
	300
	301	#define OffsetProto 2
	302	#define MaskProto 0x00ff000000000000LL
	303	#define ShiftProto 48
	304
	305	#define OffsetTos 2
	306	#define MaskTos 0xff00000000000000LL
	307	#define ShiftTos 56
	308
	309	#define OffsetInOut 3
	310	#define MaskInput 0x000000000000ffffLL
	311	#define MaskOutput 0x00000000ffff0000LL
	312	#define ShiftInput 0
	313	#define ShiftOutput 16
	314
	315	#define OffsetPort 3
	316	#define MaskDstPort 0xffff000000000000LL
	317	#define MaskSrcPort 0x0000ffff00000000LL
	318	#define ShiftDstPort 48
	319	#define ShiftSrcPort 32
	320
	321	#define OffsetAS 4
	322	#define MaskDstAS 0x00000000ffff0000LL
	323	#define MaskSrcAS 0x000000000000ffffLL
	324	#define ShiftSrcAS 0
	325	#define ShiftDstAS 16
	326
	327	#define OffsetSrcIPv4 6
	328	#define MaskSrcIPv4 0xffffffff00000000LL
	329	#define ShiftSrcIPv4 32
	330
	331	#define OffsetDstIPv4 8
	332	#define MaskDstIPv4 0xffffffff00000000LL
	333	#define ShiftDstIPv4 32
	334
	335	#define OffsetSrcIPv6a 5
	336	#define OffsetSrcIPv6b 6
	337	#define OffsetDstIPv6a 7
	338	#define OffsetDstIPv6b 8
	339	#define MaskIPv6 0xffffffffffffffffLL
	340	#define ShiftIPv6 0
	341
	342	#define OffsetPackets 9
	343	#define MaskPackets 0xffffffffffffffffLL
	344	#define ShiftPackets 0
	345
	346	#define OffsetBytes 10
	347	#define MaskBytes 0xffffffffffffffffLL
	348	#define ShiftBytes 0
	349
	350	#endif
	351
	352
	353	/*
	354	* flags:
	355	* bit 0: 0: IPv4 1: IPv6
	356	* bit 1: 0: 32bit dPkts 1: 64bit dPkts
	357	* bit 2: 0: 32bit dOctets 1: 64bit dOctets
	358	* more to come ...
	359	*
	360	* bit 31: reserved for future use. must be 0.
	361	*/
	362
	363	#define FLAG_IPV6_ADDR 1
	364	#define FLAG_PKG_64 2
	365	#define FLAG_BYTES_64 4
	366
	367	/*
	368	* offset translation table
	369	* In netflow v9 values may have a different length, and may or may nor be present.
	370	* The commmon information ( see data_block_record_t ) is expected to be present
	371	* unconditionally, and has a fixed size. IP addrs as well as counters for packets and
	372	* bytes are expexted to exist as well, but may be variable in size. Further information
	373	* may or may not be present, according the flags. See flags
	374	* To cope with this situation, the offset translation table gives the offset into an
	375	* uint32_t array at which offset the requested value start.
	376	*
	377	* index:
	378	* 0: dstip
	379	* for IPv4 netflow v5/v7 10
	380	* 1: dPkts
	381	* for IPv4 netflow v5/v7 11
	382	* 2: dOctets
	383	* for IPv4 netflow v5/v7 12
	384	*/
	385
	386	int OpenFile(char filename, stat_record_t stat_record, char *err);
	387
	388	int OpenNewFile(char filename, char *err);
	389
	390	void PrintStat(stat_record_t *s);
	391
	392	void CloseUpdateFile(int fd, stat_record_t stat_record, uint32_t record_count, char ident, char **err );
	393
	394	char *GetIdent(void);
	395
	396	void ExpandRecord(common_record_t input_record,master_record_t output_record );
	397
	398	#ifdef COMPAT14
	399
	400	/*
	401	* compatibility code with data files - nfdump 1.4
	402	* This code will be removed some time.
	403	*/
	404
	405	/* compat definitions */
	406	typedef struct compat14_flow_header_s {
	407	uint16_t version;
	408	uint16_t count;
	409	uint32_t SysUptime;
	410	uint32_t unix_secs;
	411	uint32_t unix_nsecs;
	412	uint32_t flow_sequence;
	413	uint8_t engine_type;
	414	uint8_t engine_id;
	415	uint16_t layout_version; /* binary layout version */
	416	} compat14_flow_header_t;
	417
	418	typedef struct compat14_flow_record_s {
	419	uint32_t srcaddr;
	420	uint32_t dstaddr;
	421	uint32_t nexthop;
	422	uint16_t input;
	423	uint16_t output;
	424	uint32_t dPkts;
	425	uint32_t dOctets;
	426	uint32_t First; /* First seen timestamp in UNIX time format. msec offset at end of record */
	427	uint32_t Last; /* Last seen timestamp in UNIX time format. msec offset at end of record */
	428	uint16_t srcport;
	429	uint16_t dstport;
	430	uint8_t pad;
	431	uint8_t tcp_flags;
	432	uint8_t prot;
	433	uint8_t tos;
	434	uint16_t src_as;
	435	uint16_t dst_as;
	436	uint16_t msec_first; /* msec offset from First */
	437	uint16_t msec_last; /* msec offset from Last */
	438	} comapt14_flow_record_t;
	439
	440	/* compat functions */
	441	inline ssize_t Compat14_ReadHeader(int fd, data_block_header_t *flow_header);
	442
	443	inline ssize_t Compat14_ReadRecords(int fd, void buffer, data_block_header_t flow_header);
	444
	445	#endif
	446

+175

-58

nfgen.c less more

30	30	*
31	31	* $Author: peter $
32	32	*
33		* $Id: nfgen.c 34 2005-08-22 12:01:31Z peter $
	33	* $Id: nfgen.c 70 2006-05-17 08:38:01Z peter $
34	34	*
35		* $LastChangedRevision: 34 $
	35	* $LastChangedRevision: 70 $
36	36	*
37	37	*/
38	38

55	55	#include <stdint.h>
56	56	#endif
57	57
58
59		#include "netflow_v5.h"
60		#include "netflow_v7.h"
	58	#include "nffile.h"
	59	#include "nfnet.h"
61	60	#include "nf_common.h"
	61	#include "util.h"
	62	#include "launch.h"
	63	#include "netflow_v5_v7.h"
	64
	65	const uint16_t MAGIC = 0xA50C;
	66	const uint16_t VERSION = 1;
62	67
63	68	static time_t when = 1089534600;
64	69	uint32_t offset = 10;
65	70	uint32_t msecs = 10;
66	71
67
68		void SendHeader (int numrecords) {
69		flow_header_t nf_header;
70
71		nf_header.version = 5;
72		nf_header.count = numrecords;
73		nf_header.SysUptime = when;
74		nf_header.unix_secs = 0;
75		nf_header.unix_nsecs = 0;
76		nf_header.flow_sequence = 0;
77		nf_header.layout_version = 1;
78
79		write(1, &nf_header, FLOW_HEADER_LENGTH);
80
81		} // End of SendHeader
82
83		void GenRecord(flow_record_t nf_record, char src_ip, char *dst_ip, int src_port, int dst_port,
84		int proto, int tcp_flags, int tos, int packets, int bytes, int src_as, int dst_as) {
85
86		nf_record->srcaddr = ntohl(inet_addr(src_ip));
87		nf_record->dstaddr = ntohl(inet_addr(dst_ip));
88		nf_record->nexthop = 0;
89		nf_record->input = 0;
	72	uint32_t byte_limit, packet_limit;
	73	int byte_mode, packet_mode, failed;
	74
	75	void GenRecord(int af, void writeto, char src_ip, char dst_ip, int src_port, int dst_port,
	76	int proto, int tcp_flags, int tos, uint64_t packets, uint64_t bytes, int src_as, int dst_as);
	77
	78	void GenRecord(int af, void writeto, char src_ip, char dst_ip, int src_port, int dst_port,
	79	int proto, int tcp_flags, int tos, uint64_t packets, uint64_t bytes, int src_as, int dst_as) {
	80	common_record_t nf_record = (common_record_t )writeto;
	81	void *val;
	82
	83	nf_record->flags = 0;
	84	nf_record->mark = 0;
	85	nf_record->first = when;
	86	nf_record->last = when + offset;
	87	nf_record->msec_first = msecs;
	88	nf_record->msec_last = msecs + 10;
	89
	90	nf_record->input = 0;
90	91	nf_record->output = 255;
91		nf_record->dPkts = packets;
92		nf_record->dOctets = bytes;
93		nf_record->First = when;
94		nf_record->Last = when + offset;
95	92	nf_record->srcport = src_port;
96	93	nf_record->dstport = dst_port;
97		nf_record->pad = 0;
98		nf_record->tcp_flags = tcp_flags;
	94	nf_record->dir = 0;
	95	nf_record->tcp_flags = tcp_flags;
99	96	nf_record->prot = proto;
100	97	nf_record->tos = tos;
101		nf_record->src_as = src_as;
102		nf_record->dst_as = dst_as;
103		nf_record->msec_first = msecs;
104		nf_record->msec_last = msecs + 10;
	98	nf_record->srcas = src_as;
	99	nf_record->dstas = dst_as;
	100
	101	if ( af == PF_INET6 ) {
	102	ipv6_block_t addr;
	103	nf_record->flags = 1;
	104	inet_pton(PF_INET6, src_ip, &addr.srcaddr );
	105	inet_pton(PF_INET6, dst_ip, &addr.dstaddr );
	106	addr.srcaddr[0] = ntohll(addr.srcaddr[0]);
	107	addr.srcaddr[1] = ntohll(addr.srcaddr[1]);
	108	addr.dstaddr[0] = ntohll(addr.dstaddr[0]);
	109	addr.dstaddr[1] = ntohll(addr.dstaddr[1]);
	110	memcpy((void )nf_record->data, (void )&addr, sizeof(ipv6_block_t));
	111	val = (void *)((pointer_addr_t)nf_record->data + sizeof(ipv6_block_t));
	112	fprintf(stderr, "IPv6 ");
	113	} else {
	114	uint32_t v4addr = (uint32_t )nf_record->data;
	115	inet_pton(PF_INET, src_ip, &v4addr[0] );
	116	inet_pton(PF_INET, dst_ip, &v4addr[1] );
	117	v4addr[0] = ntohl(v4addr[0]);
	118	v4addr[1] = ntohl(v4addr[1]);
	119	val = (void )((pointer_addr_t)nf_record->data + 2 sizeof(uint32_t));
	120	fprintf(stderr, "IPv4 ");
	121	}
	122
	123	if ( packets > 0xffffffffLL ) {
	124	/* 64bit access to potentially unaligned output buffer. use 2 x 32bit for _LP64 CPUs */
	125	uint32_t outbuffer = (uint32_t )val;
	126	value64_t v;
	127
	128	v.val.val64 = packets;
	129	outbuffer[0] = v.val.val32[0];
	130	outbuffer[1] = v.val.val32[1];
	131	val = (void *)&outbuffer[2];
	132	nf_record->flags \|= FLAG_PKG_64;
	133	fprintf(stderr, "packets 64bit ");
	134	} else {
	135	uint32_t v = (uint32_t )val;
	136	*v++ = packets;
	137	val = (void *)v;
	138	fprintf(stderr, "packets 32bit ");
	139	}
	140
	141	if ( bytes > 0xffffffffLL ) {
	142	/* 64bit access to potentially unaligned output buffer. use 2 x 32bit for _LP64 CPUs */
	143	uint32_t outbuffer = (uint32_t )val;
	144	value64_t v;
	145
	146	v.val.val64 = bytes;
	147	outbuffer[0] = v.val.val32[0];
	148	outbuffer[1] = v.val.val32[1];
	149	val = (void *)&outbuffer[2];
	150	nf_record->flags \|= FLAG_BYTES_64;
	151	fprintf(stderr, "bytes 64bit ");
	152	} else {
	153	uint32_t v = (uint32_t )val;
	154	*v++ = bytes;
	155	val = (void *)v;
	156
	157	fprintf(stderr, "bytes 32bit ");
	158	}
	159	fprintf(stderr, "Flags: %x\n", nf_record->flags);
	160
	161	nf_record->size = (pointer_addr_t)val - (pointer_addr_t)nf_record;
	162
105	163	offset += 10;
106	164	when += 10;
107	165

109	167	if ( msecs > 1000 )
110	168	msecs = msecs - 1000;
111	169
112
113		} // End of GenRecord
	170	return (void *)((pointer_addr_t)writeto + nf_record->size);
	171
	172	} // End of Gen_v6_Record
	173
114	174
115	175	int main( int argc, char **argv ) {
116	176	char c;
117		flow_record_t flow_record[128];
	177	data_block_header_t *nf_header;
	178	file_header_t *file_header;
	179	size_t len;
	180	void writeto, buffer, *records;
	181	uint32_t numrecords;
118	182
119	183	while ((c = getopt(argc, argv, "h")) != EOF) {
120	184	switch(c) {

126	190	}
127	191	}
128	192
129		SendHeader(14);
130		// src_ip dst_ip, src_port, dst_port, proto, tcp_flags, tos, packets, bytes, src_as, dst_as
131		GenRecord(&flow_record[0], "172.16.1.66", "172.16.19.18", 1024, 25, 6, 0, 0, 101, 101, 775, 8404);
132		GenRecord(&flow_record[1], "172.16.1.66", "172.16.19.18", 1024, 25, 6, 0, 0, 101, 101, 775, 8404);
133		GenRecord(&flow_record[2], "172.16.1.66", "172.16.19.18", 1024, 25, 6, 0, 0, 101, 101, 775, 8404);
134		GenRecord(&flow_record[3], "172.16.2.66", "172.16.18.18", 2024, 25, 17, 1, 1, 1001, 1001, 775, 8404);
135		GenRecord(&flow_record[4], "172.16.3.66", "172.16.17.18", 3024, 25, 51, 2, 2, 10001, 10001, 775, 8404);
136		GenRecord(&flow_record[5], "172.16.4.66", "172.16.16.18", 4024, 25, 6, 4, 3, 100001, 100001, 775, 8404);
137		GenRecord(&flow_record[6], "172.16.5.66", "172.16.15.18", 5024, 25, 6, 8, 4, 1000001, 1000001, 775, 8404);
138		GenRecord(&flow_record[7], "172.16.5.66", "172.16.15.18", 5024, 25, 6, 1, 4, 10000010, 1001, 775, 8404);
139		GenRecord(&flow_record[8], "172.16.6.66", "172.16.14.18", 6024, 25, 6, 16, 5, 500, 10000001, 775, 8404);
140		GenRecord(&flow_record[9], "172.16.6.66", "172.16.14.18", 6024, 25, 6, 16, 5, 500, 10000001, 775, 8404);
141		GenRecord(&flow_record[10], "172.16.7.66", "172.16.13.18", 7024, 25, 6, 32, 255, 5000, 100000001, 775, 8404);
142		GenRecord(&flow_record[11], "172.16.8.66", "172.16.12.18", 8024, 25, 6, 63, 0, 5000, 1000000001, 775, 8404);
143		GenRecord(&flow_record[12], "172.16.2.66", "172.16.18.18", 0, 8, 1, 0, 0, 50000, 50000, 775, 8404);
144		GenRecord(&flow_record[13], "172.160.160.166", "172.160.160.180", 10024, 25000, 6, 0, 0, 500000, 500000, 775, 8404);
145		write(1, &flow_record[0], 14 * FLOW_RECORD_LENGTH);
	193	buffer = malloc(1024*1024);
	194	nf_header = (data_block_header_t *)buffer;
	195	nf_header->pad = 0;
	196	records = writeto = (void *)((pointer_addr_t)buffer + sizeof(data_block_header_t));
	197
	198	// initialize file header and dummy stat record
	199	len = sizeof(file_header_t) + sizeof(stat_record_t);
	200	file_header = (file_header_t *)malloc(len);
	201	memset((void *)file_header, 0, len);
	202	file_header->magic = MAGIC;
	203	file_header->version = VERSION;
	204	strncpy(file_header->ident, "none", IDENT_SIZE);
	205	write(STDOUT_FILENO, (void *)file_header, len) ;
	206
	207	numrecords = 0;
	208	// src_ip dst_ip, src_port, dst_port, proto, tcp_flags, tos, packets, bytes, src_as, dst_as
	209	writeto = GenRecord(PF_INET, writeto, "172.16.1.66", "192.168.170.100", 1024, 25, 6, 0, 0, 101, 101, 775, 8404);
	210	numrecords++;
	211	writeto = GenRecord(PF_INET, writeto, "172.16.2.66", "192.168.170.101", 1024, 25, 6, 0, 0, 101, 101, 775, 8404);
	212	numrecords++;
	213	writeto = GenRecord(PF_INET, writeto, "172.16.3.66", "192.168.170.102", 1024, 25, 6, 0, 0, 101, 101, 775, 8404);
	214	numrecords++;
	215	writeto = GenRecord(PF_INET, writeto, "172.16.4.66", "192.168.170.103", 2024, 25, 17, 1, 1, 1001, 1001, 775, 8404);
	216	numrecords++;
	217	writeto = GenRecord(PF_INET, writeto, "172.16.5.66", "192.168.170.104", 3024, 25, 51, 2, 2, 10001, 10001, 775, 8404);
	218	numrecords++;
	219	writeto = GenRecord(PF_INET, writeto, "172.16.6.66", "192.168.170.105", 4024, 25, 6, 4, 3, 100001, 100001, 775, 8404);
	220	numrecords++;
	221	writeto = GenRecord(PF_INET, writeto, "172.16.7.66", "192.168.170.106", 5024, 25, 6, 8, 4, 1000001, 1000001, 775, 8404);
	222	numrecords++;
	223	writeto = GenRecord(PF_INET, writeto, "172.16.8.66", "192.168.170.107", 5024, 25, 6, 1, 4, 10000010, 1001, 775, 8404);
	224	numrecords++;
	225	writeto = GenRecord(PF_INET, writeto, "172.16.9.66", "192.168.170.108", 6024, 25, 6, 16, 5, 500, 10000001, 775, 8404);
	226	numrecords++;
	227	writeto = GenRecord(PF_INET, writeto, "172.16.10.66", "192.168.170.109", 6024, 25, 6, 16, 5, 500, 10000001, 775, 8404);
	228	numrecords++;
	229	writeto = GenRecord(PF_INET, writeto, "172.16.11.66", "192.168.170.110", 7024, 25, 6, 32, 255, 5000, 100000001, 775, 8404);
	230	numrecords++;
	231	writeto = GenRecord(PF_INET, writeto, "172.16.12.66", "192.168.170.111", 8024, 25, 6, 63, 0, 5000, 1000000001, 775, 8404);
	232	numrecords++;
	233	writeto = GenRecord(PF_INET, writeto, "172.16.13.66", "192.168.170.112", 0, 8, 1, 0, 0, 50000, 50000, 775, 8404);
	234	numrecords++;
	235	writeto = GenRecord(PF_INET, writeto, "172.160.160.166", "172.160.160.180", 10024, 25000, 6, 0, 0, 500000, 500000, 775, 8404);
	236	numrecords++;
	237
	238	writeto = GenRecord(PF_INET6, writeto, "fe80::2110:abcd:1234:0", "fe80::2110:abcd:1235:4321", 1024, 25, 6, 27, 0, 10, 15100, 775, 8404);
	239	numrecords++;
	240	writeto = GenRecord(PF_INET6, writeto, "2001:234:aabb::211:24ff:fe80:d01e", "2001:620::8:203:baff:fe52:38e5", 10240, 52345, 6, 27, 0, 10100, 15000000, 775, 8404);
	241	numrecords++;
	242
	243	// flows with 64 bit counters
	244	writeto = GenRecord(PF_INET6, writeto, "2001:234:aabb::211:24ff:fe80:d01e", "2001:620::8:203:baff:fe52:38e5", 10240, 52345, 6, 27, 0, 10100000, 0x100000000LL, 775, 8404);
	245	numrecords++;
	246	writeto = GenRecord(PF_INET6, writeto, "2001:234:aabb::211:24ff:fe80:d01e", "2001:620::8:203:baff:fe52:38e5", 10240, 52345, 6, 27, 0, 0x100000000LL, 15000000, 775, 8404);
	247	numrecords++;
	248	writeto = GenRecord(PF_INET6, writeto, "2001:234:aabb::211:24ff:fe80:d01e", "2001:620::8:203:baff:fe52:38e5", 10240, 52345, 6, 27, 0, 0x100000000LL, 0x200000000LL, 775, 8404);
	249	numrecords++;
	250
	251	writeto = GenRecord(PF_INET, writeto, "172.16.14.18", "192.168.170.113", 10240, 52345, 6, 27, 0, 10100000, 0x100000000LL, 775, 8404);
	252	numrecords++;
	253	writeto = GenRecord(PF_INET, writeto, "172.16.15.18", "192.168.170.114", 10240, 52345, 6, 27, 0, 0x100000000LL, 15000000, 775, 8404);
	254	numrecords++;
	255	writeto = GenRecord(PF_INET, writeto, "172.16.16.18", "192.168.170.115", 10240, 52345, 6, 27, 0, 0x100000000LL, 0x200000000LL, 775, 8404);
	256	numrecords++;
	257
	258	nf_header->NumBlocks = numrecords;
	259	nf_header->size = (pointer_addr_t)writeto - (pointer_addr_t)records;
	260	nf_header->id = DATA_BLOCK_TYPE_1;
	261	write(1, nf_header, sizeof(data_block_header_t));
	262	write(1, records, (pointer_addr_t)writeto - (pointer_addr_t)records);
146	263
147	264	return 0;
148	265	}

+553

-0

nfnet.c less more

	0	/*
	1	* Copyright (c) 2004, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
	2	* All rights reserved.
	3	*
	4	* Redistribution and use in source and binary forms, with or without
	5	* modification, are permitted provided that the following conditions are met:
	6	*
	7	* * Redistributions of source code must retain the above copyright notice,
	8	* this list of conditions and the following disclaimer.
	9	* * Redistributions in binary form must reproduce the above copyright notice,
	10	* this list of conditions and the following disclaimer in the documentation
	11	* and/or other materials provided with the distribution.
	12	* * Neither the name of SWITCH nor the names of its contributors may be
	13	* used to endorse or promote products derived from this software without
	14	* specific prior written permission.
	15	*
	16	* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
	17	* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	18	* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	19	* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
	20	* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
	21	* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
	22	* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
	23	* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
	24	* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
	25	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
	26	* POSSIBILITY OF SUCH DAMAGE.
	27	*
	28	* $Author: peter $
	29	*
	30	* $Id: nfnet.c 70 2006-05-17 08:38:01Z peter $
	31	*
	32	* $LastChangedRevision: 70 $
	33	*
	34	*
	35	*/
	36
	37	#include <sys/types.h>
	38	#include <sys/socket.h>
	39	#include <netdb.h>
	40	#include <stdio.h>
	41	#include <errno.h>
	42	#include <unistd.h>
	43	#include <stdlib.h>
	44	#include <time.h>
	45	#include <netinet/in.h>
	46	#include <syslog.h>
	47	#include <string.h>
	48
	49	#include "config.h"
	50
	51	#ifdef HAVE_STDINT_H
	52	#include <stdint.h>
	53	#endif
	54
	55	#include "nfnet.h"
	56
	57	/* at least this number of byytes required, if we change the socket buffer */
	58	#define Min_SOCKBUFF_LEN 65536
	59
	60	const int LISTEN_QUEUE = 128;
	61
	62	/* local function prototypes */
	63	static int isMulticast(struct sockaddr_storage *addr);
	64
	65	static int joinGroup(int sockfd, int loopBack, int mcastTTL, struct sockaddr_storage *addr);
	66
	67	/* function definitions */
	68
	69	int Unicast_receive_socket(const char bindhost, const char listenport, int family, int sockbuflen ) {
	70	struct addrinfo hints, res, ressave;
	71	socklen_t optlen;
	72	int error, p, sockfd;
	73
	74
	75	if ( !listenport ) {
	76	fprintf(stderr, "listen port required!\n");
	77	syslog(LOG_ERR, "listen port required!\n");
	78	return -1;
	79	}
	80
	81	// if nothing specified on command line, prefer IPv4 over IPv6, for compatibility
	82	if ( bindhost == NULL && family == AF_UNSPEC )
	83	family = AF_INET;
	84
	85	memset(&hints, 0, sizeof(struct addrinfo));
	86
	87	/*
	88	AI_PASSIVE flag: we use the resulting address to bind
	89	to a socket for accepting incoming connections.
	90	So, when the hostname==NULL, getaddrinfo function will
	91	return one entry per allowed protocol family containing
	92	the unspecified address for that family.
	93	*/
	94
	95	hints.ai_flags = AI_PASSIVE;
	96	hints.ai_family = family;
	97	hints.ai_socktype = SOCK_DGRAM;
	98
	99	error = getaddrinfo(bindhost, listenport, &hints, &res);
	100	if ( error) {
	101	fprintf(stderr, "getaddrinfo error: [%s]\n", gai_strerror(error));
	102	syslog(LOG_ERR, "getaddrinfo error: [%s]\n", gai_strerror(error));
	103	return -1;
	104	}
	105
	106	/*
	107	Try open socket with each address getaddrinfo returned,
	108	until we get a valid listening socket.
	109	*/
	110	ressave = res;
	111	sockfd = -1;
	112	while ( res ) {
	113	// we listen only on IPv4 or IPv6
	114	if ( res->ai_family != AF_INET && res->ai_family != AF_INET6 )
	115	continue;
	116
	117	sockfd = socket(res->ai_family, res->ai_socktype, res->ai_protocol);
	118
	119	if ( !( sockfd < 0 ) ) {
	120	// socket call was successfull
	121
	122	if (bind(sockfd, res->ai_addr, res->ai_addrlen) == 0) {
	123	if ( res->ai_family == AF_INET )
	124	syslog(LOG_DEBUG, "Bound to IPv4 host/IP: %s Port: %s",
	125	bindhost == NULL ? "any" : bindhost, listenport);
	126	if ( res->ai_family == AF_INET6 )
	127	syslog(LOG_DEBUG, "Bound to IPv6 host/IP: %s Port: %s",
	128	bindhost == NULL ? "any" : bindhost, listenport);
	129
	130	// we are done
	131	break;
	132	}
	133
	134	// bind was unsuccessful :(
	135	close(sockfd);
	136	sockfd = -1;
	137	}
	138	res = res->ai_next;
	139	}
	140
	141	if (sockfd < 0) {
	142	freeaddrinfo(ressave);
	143	fprintf(stderr, "Socket error: could not open the requested socket\n");
	144	syslog(LOG_ERR, "Socket error: could not open the requested socket\n");
	145	return -1;
	146	}
	147
	148	listen(sockfd, LISTEN_QUEUE);
	149
	150	freeaddrinfo(ressave);
	151
	152	if ( sockbuflen ) {
	153	if ( sockbuflen < Min_SOCKBUFF_LEN ) {
	154	sockbuflen = Min_SOCKBUFF_LEN;
	155	syslog(LOG_INFO,"I want at least %i bytes as socket buffer", sockbuflen);
	156	}
	157	optlen = sizeof(p);
	158	getsockopt(sockfd,SOL_SOCKET,SO_RCVBUF,&p,&optlen);
	159	syslog(LOG_INFO,"Standard setsockopt, SO_RCVBUF is %i Requested length is %i bytes",p, sockbuflen);
	160	if ((setsockopt(sockfd, SOL_SOCKET, SO_RCVBUF, &sockbuflen, sizeof(sockbuflen)) != 0) ) {
	161	fprintf (stderr, "setsockopt(SO_RCVBUF,%d): %s\n", sockbuflen, strerror (errno));
	162	syslog (LOG_ERR, "setsockopt(SO_RCVBUF,%d): %s", sockbuflen, strerror (errno));
	163	close(sockfd);
	164	return -1;
	165	} else {
	166	getsockopt(sockfd,SOL_SOCKET,SO_RCVBUF,&p,&optlen);
	167	syslog(LOG_INFO,"System set setsockopt, SO_RCVBUF to %d bytes", p);
	168	}
	169	}
	170
	171	return sockfd;
	172
	173	} /* End of Unicast_receive_socket */
	174
	175	int Unicast_send_socket (const char hostname, const char sendport, int family,
	176	unsigned int wmem_size, struct sockaddr_storage addr, int addrlen) {
	177	struct addrinfo hints, res, ressave;
	178	int n, sockfd;
	179	unsigned int wmem_actual;
	180	socklen_t optlen;
	181
	182	if ( !hostname \|\| !sendport ) {
	183	fprintf(stderr, "hostname and listen port required!\n");
	184	syslog(LOG_ERR, "hostname and listen port required!\n");
	185	return -1;
	186	}
	187
	188	// create socket
	189	memset(&hints, 0, sizeof(struct addrinfo));
	190
	191	hints.ai_family = family;
	192	hints.ai_socktype = SOCK_DGRAM;
	193
	194	n = getaddrinfo(hostname, sendport, &hints, &res);
	195
	196	if ( n < 0 ) {
	197	fprintf(stderr, "getaddrinfo error: [%s]\n", strerror(errno));
	198	return -1;
	199	}
	200
	201	ressave = res;
	202	sockfd = -1;
	203	while (res) {
	204	sockfd = socket(res->ai_family, res->ai_socktype, res->ai_protocol);
	205
	206	if ( !(sockfd < 0) ) {
	207	// socket call was successsful
	208	if (connect(sockfd, res->ai_addr, res->ai_addrlen) == 0) {
	209	// connect successful - we are done
	210	close(sockfd);
	211	// ok - we need now an unconnected socket
	212	sockfd = socket(res->ai_family, res->ai_socktype, res->ai_protocol);
	213	break;
	214	}
	215	// unsuccessful connect :(
	216	close(sockfd);
	217	sockfd = -1;
	218	}
	219	res=res->ai_next;
	220	}
	221
	222	if (sockfd < 0) {
	223	freeaddrinfo(ressave);
	224	fprintf(stderr, "Socket error: could not open the requested socket\n");
	225	syslog(LOG_ERR, "Socket error: could not open the requested socket\n");
	226	return -1;
	227	}
	228
	229	*addrlen = res->ai_addrlen;
	230	memcpy(addr, res->ai_addr, res->ai_addrlen);
	231	freeaddrinfo(ressave);
	232
	233	// Set socket write buffer. Need to be root!
	234	if ( wmem_size > 0 ) {
	235	if ( geteuid() == 0 ) {
	236	setsockopt(sockfd, SOL_SOCKET, SO_SNDBUF, &wmem_size, sizeof(wmem_size));
	237
	238	// check what was set (e.g. linux 2.4.20 sets twice of what was requested)
	239	getsockopt(sockfd, SOL_SOCKET, SO_SNDBUF, &wmem_actual, &optlen);
	240
	241	if (wmem_size != wmem_actual) {
	242	printf("Warning: Socket write buffer size requested: %u set: %u\n",
	243	wmem_size, wmem_actual);
	244	}
	245	} else {
	246	printf("Warning: Socket buffer size can only be changed by root!\n");
	247	}
	248	}
	249
	250	return sockfd;
	251
	252	} // End of Unicast_send_socket
	253
	254
	255	int Multicast_receive_socket (const char hostname, const char listenport, int family, int sockbuflen ) {
	256	struct addrinfo hints, res, ressave;
	257	socklen_t optlen;
	258	int p, error, sockfd;
	259
	260	if ( !listenport ) {
	261	fprintf(stderr, "listen port required!\n");
	262	syslog(LOG_ERR, "listen port required!\n");
	263	return -1;
	264	}
	265
	266	memset(&hints, 0, sizeof(struct addrinfo));
	267	hints.ai_family = family;
	268	hints.ai_socktype = SOCK_DGRAM;
	269
	270	error = getaddrinfo(hostname, listenport, &hints, &res);
	271
	272	if ( error ) {
	273	fprintf(stderr, "getaddrinfo error:: [%s]\n", gai_strerror(error));
	274	syslog(LOG_ERR, "getaddrinfo error:: [%s]\n", gai_strerror(error));
	275	return -1;
	276	}
	277
	278
	279	/*
	280	Try open socket with each address getaddrinfo returned,
	281	until we get a valid listening socket.
	282	*/
	283
	284	sockfd = -1;
	285	ressave = res;
	286	while (res) {
	287	sockfd = socket(res->ai_family, res->ai_socktype, res->ai_protocol);
	288	if ( sockfd < 0 ) {
	289	res=res->ai_next;
	290	continue;
	291	}
	292
	293	// we found a valid socket and are done in this loop
	294	break;
	295	}
	296
	297	if ( sockfd < 0 ) {
	298	// nothing found - bye bye
	299	fprintf(stderr, "Could not create a socket for [%s:%s]\n", hostname, listenport);
	300	syslog(LOG_ERR, "Could not create a socket for [%s:%s]\n", hostname, listenport);
	301	freeaddrinfo(ressave);
	302	return -1;
	303	}
	304
	305
	306	if ( isMulticast((struct sockaddr_storage *)res->ai_addr) < 0 ) {
	307	fprintf(stderr, "Not a multicast address [%s]\n", hostname);
	308	syslog(LOG_ERR, "Not a multicast address [%s]\n", hostname);
	309	freeaddrinfo(ressave);
	310	return -1;
	311	}
	312
	313	close(sockfd);
	314
	315
	316	sockfd = socket(res->ai_family, SOCK_DGRAM, 0);
	317	if (bind(sockfd, res->ai_addr, res->ai_addrlen) < 0) {
	318	fprintf(stderr, "bind: %s\n", strerror (errno));
	319	syslog(LOG_ERR, "bind: %s\n", strerror (errno));
	320	close(sockfd);
	321	freeaddrinfo(ressave);
	322	return -1;
	323	}
	324
	325	if (joinGroup(sockfd, 1 , 1, (struct sockaddr_storage *)res->ai_addr) <0) {
	326	close(sockfd);
	327	freeaddrinfo(ressave);
	328	return -1;
	329	}
	330
	331	if ( res->ai_family == AF_INET )
	332	syslog(LOG_DEBUG, "Joined IPv4 multicast group: %s Port: %s", hostname, listenport);
	333	if ( res->ai_family == AF_INET6 )
	334	syslog(LOG_DEBUG, "Joined IPv6 multicat group: %s Port: %s", hostname, listenport);
	335
	336
	337	freeaddrinfo(ressave);
	338
	339	if ( sockbuflen ) {
	340	if ( sockbuflen < Min_SOCKBUFF_LEN ) {
	341	sockbuflen = Min_SOCKBUFF_LEN;
	342	syslog(LOG_INFO,"I want at least %i bytes as socket buffer", sockbuflen);
	343	}
	344	optlen = sizeof(p);
	345	getsockopt(sockfd,SOL_SOCKET,SO_RCVBUF,&p,&optlen);
	346	syslog(LOG_INFO,"Standard setsockopt, SO_RCVBUF is %i Requested length is %i bytes",p, sockbuflen);
	347	if ((setsockopt(sockfd, SOL_SOCKET, SO_RCVBUF, &sockbuflen, sizeof(sockbuflen)) != 0) ) {
	348	fprintf (stderr, "setsockopt(SO_RCVBUF,%d): %s\n", sockbuflen, strerror (errno));
	349	syslog (LOG_ERR, "setsockopt(SO_RCVBUF,%d): %s", sockbuflen, strerror (errno));
	350	close(sockfd);
	351	return -1;
	352	} else {
	353	getsockopt(sockfd,SOL_SOCKET,SO_RCVBUF,&p,&optlen);
	354	syslog(LOG_INFO,"System set setsockopt, SO_RCVBUF to %d bytes", p);
	355	}
	356	}
	357
	358	return sockfd;
	359
	360	} /* End of Multicast_receive_socket */
	361
	362	int Multicast_send_socket (const char hostname, const char listenport, int family,
	363	unsigned int wmem_size, struct sockaddr_storage addr, int addrlen) {
	364	struct addrinfo hints, res, ressave;
	365	int error, sockfd;
	366
	367	if ( !listenport \|\| !hostname ) {
	368	fprintf(stderr, "hostname and listen port required!\n");
	369	syslog(LOG_ERR, "hostname and listen port required!\n");
	370	return -1;
	371	}
	372
	373	memset(&hints, 0, sizeof(struct addrinfo));
	374	hints.ai_family = family;
	375	hints.ai_socktype = SOCK_DGRAM;
	376
	377	error = getaddrinfo(hostname, listenport, &hints, &res);
	378
	379	if ( error ) {
	380	fprintf(stderr, "getaddrinfo error:: [%s]\n", gai_strerror(error));
	381	syslog(LOG_ERR, "getaddrinfo error:: [%s]\n", gai_strerror(error));
	382	return -1;
	383	}
	384
	385
	386	/*
	387	Try open socket with each address getaddrinfo returned,
	388	until we get a valid listening socket.
	389	*/
	390
	391	sockfd=-1;
	392	ressave = res;
	393	while (res) {
	394	sockfd = socket(res->ai_family, res->ai_socktype, res->ai_protocol);
	395	if ( sockfd < 0 ) {
	396	res=res->ai_next;
	397	continue;
	398	}
	399
	400	// we found a valid socket and are done in this loop
	401	break;
	402	}
	403
	404	if ( sockfd < 0 ) {
	405	// nothing found - bye bye
	406	fprintf(stderr, "Could not create a socket for [%s:%s]\n", hostname, listenport);
	407	syslog(LOG_ERR, "Could not create a socket for [%s:%s]\n", hostname, listenport);
	408	freeaddrinfo(ressave);
	409	return -1;
	410	}
	411
	412	if ( isMulticast((struct sockaddr_storage *)res->ai_addr) < 0 ) {
	413	fprintf(stderr, "Not a multicast address [%s]\n", hostname);
	414	syslog(LOG_ERR, "Not a multicast address [%s]\n", hostname);
	415	freeaddrinfo(ressave);
	416	return -1;
	417	}
	418
	419	close(sockfd);
	420	sockfd = socket(res->ai_family, SOCK_DGRAM, 0);
	421
	422	*addrlen = res->ai_addrlen;
	423	memcpy(addr, res->ai_addr, res->ai_addrlen);
	424
	425	/*
	426	((struct sockaddr_in *)addr)->sin_family=AF_INET;
	427	addr.sin_addr.s_addr=inet_addr(HELLO_GROUP);
	428	addr.sin_port=htons(HELLO_PORT);
	429	*/
	430
	431	if (joinGroup(sockfd, 1 , 1, (struct sockaddr_storage *)res->ai_addr) <0) {
	432	close(sockfd);
	433	freeaddrinfo(ressave);
	434	return -1;
	435	}
	436
	437	freeaddrinfo(ressave);
	438
	439	/*
	440	message = "hello world\n";
	441	printf("Send %lu bytes, Message: %s\n", strlen(message)+1, message);
	442	while (1) {
	443	ret = sendto(sockfd,message,strlen(message)+1,0,(struct sockaddr ) addr, addrlen);
	444	if ( ret != strlen(message)+1 ) {
	445	perror("sendto");
	446	exit(1);
	447	}
	448	printf("sleep\n");
	449	sleep(1);
	450	}
	451	exit(255);
	452	*/
	453	return sockfd;
	454
	455	} /* End of Multicast_send_socket */
	456
	457	static int joinGroup(int sockfd, int loopBack, int mcastTTL, struct sockaddr_storage *addr) {
	458	int ret, err;
	459
	460	ret=-1;
	461
	462	switch (addr->ss_family) {
	463	case AF_INET: {
	464	struct ip_mreq mreq;
	465
	466	mreq.imr_multiaddr.s_addr = ((struct sockaddr_in *)addr)->sin_addr.s_addr;
	467	mreq.imr_interface.s_addr = INADDR_ANY;
	468
	469	err = setsockopt(sockfd, IPPROTO_IP, IP_ADD_MEMBERSHIP, (const void *)&mreq, sizeof(mreq));
	470	if ( err ) {
	471	fprintf(stderr, "setsockopt IP_ADD_MEMBERSHIP: %s\n", strerror (errno));
	472	syslog(LOG_ERR, "setsockopt IP_ADD_MEMBERSHIP: %s\n", strerror (errno));
	473	break;
	474	}
	475	ret = 0;
	476	/*
	477	// These two setsockopt may fail because not implemented on every OS, but harmless
	478	err = setsockopt(sockfd, IPPROTO_IP, IP_MULTICAST_LOOP, &loopBack, sizeof(loopBack));
	479	if ( err ) {
	480	fprintf(stderr, "setsockopt IP_MULTICAST_LOOP: %s\n", strerror (errno));
	481	syslog(LOG_ERR, "setsockopt IP_MULTICAST_LOOP: %s\n", strerror (errno));
	482	}
	483
	484	err = setsockopt(sockfd, IPPROTO_IP, IP_MULTICAST_TTL, &mcastTTL, sizeof(mcastTTL));
	485	if ( err ) {
	486	fprintf(stderr, "setsockopt IP_MULTICAST_LOOP: %s\n", strerror (errno));
	487	syslog(LOG_ERR, "setsockopt IP_MULTICAST_LOOP: %s\n", strerror (errno));
	488	}
	489	*/
	490	} break;
	491
	492	case AF_INET6: {
	493	struct ipv6_mreq mreq6;
	494
	495	memcpy(&mreq6.ipv6mr_multiaddr, &(((struct sockaddr_in6 *)addr)->sin6_addr), sizeof(struct in6_addr));
	496	mreq6.ipv6mr_interface= 0;
	497
	498	err = setsockopt(sockfd, IPPROTO_IPV6, IPV6_JOIN_GROUP, &mreq6, sizeof(mreq6));
	499	if ( err ) {
	500	fprintf(stderr, "setsockopt IPV6_JOIN_GROUP: %s\n", strerror (errno));
	501	syslog(LOG_ERR, "setsockopt IPV6_JOIN_GROUP: %s\n", strerror (errno));
	502	break;
	503	}
	504	ret = 0;
	505	/*
	506	// These two setsockopt may fail because not implemented on every OS, but harmless
	507	err = setsockopt(sockfd, IPPROTO_IPV6, IPV6_MULTICAST_LOOP, &loopBack, sizeof(loopBack));
	508	if ( err ) {
	509	fprintf(stderr, "setsockopt IPV6_MULTICAST_LOOP: %s\n", strerror (errno));
	510	syslog(LOG_ERR, "setsockopt IPV6_MULTICAST_LOOP: %s\n", strerror (errno));
	511	}
	512
	513	err = setsockopt(sockfd, IPPROTO_IPV6, IPV6_MULTICAST_HOPS, &mcastTTL, sizeof(mcastTTL));
	514	if ( err ) {
	515	fprintf(stderr, "setsockopt IPV6_MULTICAST_HOPS: %s\n", strerror (errno));
	516	syslog(LOG_ERR, "setsockopt IPV6_MULTICAST_HOPS: %s\n", strerror (errno));
	517	}
	518	*/
	519	} break;
	520
	521	default:
	522	;
	523	}
	524
	525	return ret;
	526	} /* joinGroup */
	527
	528
	529	static int isMulticast(struct sockaddr_storage *addr) {
	530	int ret;
	531
	532	ret = -1;
	533	switch (addr->ss_family) {
	534	case AF_INET: {
	535	struct sockaddr_in addr4=(struct sockaddr_in )addr;
	536	ret = IN_MULTICAST(ntohl(addr4->sin_addr.s_addr));
	537	}
	538	break;
	539
	540	case AF_INET6: {
	541	struct sockaddr_in6 addr6=(struct sockaddr_in6 )addr;
	542	ret = IN6_IS_ADDR_MULTICAST(&addr6->sin6_addr);
	543	}
	544	break;
	545	default:
	546	;
	547	}
	548
	549	return ret;
	550	} /* End of isMulticast */
	551
	552

+67

-0

nfnet.h less more

	0	/*
	1	* This file is part of the nfdump project.
	2	*
	3	* Copyright (c) 2004, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
	4	* All rights reserved.
	5	*
	6	* Redistribution and use in source and binary forms, with or without
	7	* modification, are permitted provided that the following conditions are met:
	8	*
	9	* * Redistributions of source code must retain the above copyright notice,
	10	* this list of conditions and the following disclaimer.
	11	* * Redistributions in binary form must reproduce the above copyright notice,
	12	* this list of conditions and the following disclaimer in the documentation
	13	* and/or other materials provided with the distribution.
	14	* * Neither the name of SWITCH nor the names of its contributors may be
	15	* used to endorse or promote products derived from this software without
	16	* specific prior written permission.
	17	*
	18	* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
	19	* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	20	* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	21	* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
	22	* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
	23	* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
	24	* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
	25	* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
	26	* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
	27	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
	28	* POSSIBILITY OF SUCH DAMAGE.
	29	*
	30	* $Author: peter $
	31	*
	32	* $Id: nfnet.h 55 2006-01-13 10:04:34Z peter $
	33	*
	34	* $LastChangedRevision: 55 $
	35	*
	36	*/
	37
	38	/* Definitions */
	39
	40	#define UDP_PACKET_SIZE 1472
	41
	42	typedef struct send_peer_s {
	43	struct sockaddr_storage addr;
	44	int addrlen;
	45	int sockfd;
	46	int family;
	47	char *port;
	48	char *hostname;
	49	int mcast;
	50	int flush;
	51	void *send_buffer;
	52	void *writeto;
	53	void *endp;
	54	} send_peer_t;
	55
	56	/* Function prototypes */
	57
	58	int Unicast_receive_socket(const char bindhost, const char listenport, int family, int sockbuflen );
	59
	60	int Multicast_receive_socket (const char hostname, const char listenport, int family, int sockbuflen);
	61
	62	int Unicast_send_socket (const char hostname, const char listenport, int family,
	63	unsigned int wmem_size, struct sockaddr_storage addr, int addrlen);
	64
	65	int Multicast_send_socket (const char hostname, const char listenport, int family,
	66	unsigned int wmem_size, struct sockaddr_storage addr, int addrlen);

+15

-5

nfprof.c less more

27	27	*
28	28	* $Author: peter $
29	29	*
30		* $Id: nfprof.c 53 2005-11-17 07:45:34Z peter $
	30	* $Id: nfprof.c 70 2006-05-17 08:38:01Z peter $
31	31	*
32		* $LastChangedRevision: 53 $
	32	* $LastChangedRevision: 70 $
33	33	*
34	34	*/
35	35

84	84	*/
85	85	void nfprof_print(nfprof_t profile_data, FILE std) {
86	86	u_long usec, sec;
87		double fps;
	87	double fps, allsecs;
88	88
89	89	usec = profile_data->used.ru_utime.tv_usec + profile_data->used.ru_stime.tv_usec;
90	90	sec = profile_data->used.ru_utime.tv_sec + profile_data->used.ru_stime.tv_sec;

92	92	if (usec > 1000000)
93	93	usec -= 1000000, ++sec;
94	94
95		fps = (double)profile_data->numflows / ((double)sec + ((double)usec/1000000));
	95
	96	allsecs = (double)sec + ((double)usec/1000000);
	97	if ( allsecs == 0.0 )
	98	fps = 0;
	99	else
	100	fps = (double)profile_data->numflows / ((double)sec + ((double)usec/1000000));
96	101
97	102	fprintf(std, "Sys: %lu.%-3.3lus flows/second: %-10.1f ", sec, usec/1000, fps);
98	103

101	106
102	107	usec = profile_data->tend.tv_usec - profile_data->tstart.tv_usec;
103	108	sec = profile_data->tend.tv_sec - profile_data->tstart.tv_sec;
104		fps = (double)profile_data->numflows / ((double)sec + ((double)usec/1000000));
	109
	110	if ( usec == 0 && sec == 0 )
	111	// acctually should never happen, but catch it anyway
	112	fps = 0;
	113	else
	114	fps = (double)profile_data->numflows / ((double)sec + ((double)usec/1000000));
105	115
106	116	fprintf(std, "Wall: %lu.%-3.3lus flows/second: %-10.1f\n", sec, usec/1000, fps);
107	117	/*

-6

nfprof.h less more

27	27	*
28	28	* $Author: peter $
29	29	*
30		* $Id: nfprof.h 53 2005-11-17 07:45:34Z peter $
	30	* $Id: nfprof.h 70 2006-05-17 08:38:01Z peter $
31	31	*
32		* $LastChangedRevision: 53 $
	32	* $LastChangedRevision: 70 $
33	33	*
34	34	*/
	35
	36	#include "config.h"
35	37
36	38	#include <stdio.h>
37	39	#include <stdlib.h>
38	40	#include <sys/types.h>
39	41	#include <string.h>
40
41		#include "config.h"
	42	#include <sys/time.h>
	43	#include <sys/resource.h>
42	44
43	45	#ifdef HAVE_STDINT_H
44	46	#include <stdint.h>
45	47	#endif
46	48
47		#include <sys/time.h>
48		#include <sys/resource.h>
49	49
50	50	typedef struct nfprof_s {
51	51	struct timeval tstart; /* start time */

+226

-140

nfprofile.c less more

31	31	*
32	32	* $Author: peter $
33	33	*
34		* $Id: nfprofile.c 34 2005-08-22 12:01:31Z peter $
	34	* $Id: nfprofile.c 70 2006-05-17 08:38:01Z peter $
35	35	*
36		* $LastChangedRevision: 34 $
	36	* $LastChangedRevision: 70 $
37	37	*
38	38	*/
39	39

53	53	#include <stdint.h>
54	54	#endif
55	55
56
57	56	#include "version.h"
58	57	#include "nf_common.h"
59	58	#include "nftree.h"
60	59	#include "nfdump.h"
	60	#include "nffile.h"
61	61	#include "nfstat.h"
62	62	#include "util.h"
63	63	#include "profile.h"
64	64
65		/* hash parameters */
66		#define HashBits 20
67		#define NumPrealloc 128000
68
69	65	/* Global Variables */
70	66	uint32_t byte_limit, packet_limit;
71	67	int byte_mode, packet_mode;
72	68
73	69	/* Local Variables */
74		static char const *rcsid = "$Id: nfprofile.c 34 2005-08-22 12:01:31Z peter $";
75
76		#define NETFLOW_VERSION 5
	70	static char const *rcsid = "$Id: nfprofile.c 70 2006-05-17 08:38:01Z peter $";
77	71
78	72	/* Function Prototypes */
79	73	static void usage(char *name);
80	74
81		static void process_data(profileinfo_t *profiles, unsigned int num_profiles, time_t twin_start, time_t twin_end);
	75	static void process_data(profile_channel_info_t *profiles, unsigned int num_profiles, time_t twin_start, time_t twin_end, int zero_flows);
82	76
83	77
84	78	/* Functions */

86	80	printf("usage %s [options] \n"
87	81	"-h\t\tthis text you see right here\n"
88	82	"-V\t\tPrint version and exit.\n"
	83	"-M <expr>\tRead input from multiple directories.\n"
89	84	"-r\t\tread input from file\n"
90	85	"-f\t\tfilename with filter syntaxfile\n"
91	86	"-p\t\tprofile dir.\n"
92	87	"-s\t\tprofile subdir.\n"
93	88	"-Z\t\tCheck filter syntax and exit.\n"
94	89	"-q\t\tSuppress profile working info\n"
	90	"-z\t\tZero flows - dumpfile contains only statistics record.\n"
95	91	"-t <time>\ttime window for filtering packets\n"
96	92	"\t\tyyyy/MM/dd.hh:mm:ss[-yyyy/MM/dd.hh:mm:ss]\n", name);
97	93	} /* usage */
98	94
99		static void process_data(profileinfo_t *profiles, unsigned int num_profiles, time_t twin_start, time_t twin_end) {
100		flow_header_t flow_header;
101		flow_record_t flow_record, record_buffer;
	95	static void process_data(profile_channel_info_t *profiles, unsigned int num_profiles, time_t twin_start, time_t twin_end, int zero_flows) {
	96	data_block_header_t in_flow_header;
	97	common_record_t flow_record, in_buff;
	98	master_record_t master_record;
102	99	FilterEngine_data_t *engine;
103		uint32_t NumRecords;
	100	uint32_t NumRecords, buffer_size;
104	101	int i, j, rfd, done, ret ;
105		short *ftrue;
106
107		rfd = GetNextFile(0, twin_start, twin_end);
	102
	103	#ifdef COMPAT14
	104	extern int Format14;
	105	#endif
	106
	107	rfd = GetNextFile(0, twin_start, twin_end, NULL);
108	108	if ( rfd < 0 ) {
109		if ( errno )
110		perror("Can't open file for reading");
	109	if ( rfd == FILE_ERROR )
	110	fprintf(stderr, "Can't open file for reading: %s\n", strerror(errno));
111	111	return;
112	112	}
113	113
	114	#ifdef COMPAT14
	115	if ( Format14 ) {
	116	fprintf(stderr, "nfprofile does not support nfdump <= v1.4 old style file format!\n");
	117	return;
	118	}
	119	#endif
	120
114	121	// allocate buffer suitable for netflow version
115		record_buffer = (flow_record_t *) calloc(BuffNumRecords , FLOW_RECORD_LENGTH);
116		ftrue = (short ) malloc(BuffNumRecords sizeof(short));
117		if ( !record_buffer \|\| !ftrue ) {
118		perror("Memory allocation error");
	122	buffer_size = BUFFSIZE;
	123	in_buff = (common_record_t *) malloc(buffer_size);
	124	if ( !in_buff ) {
	125	fprintf(stderr, "Memory allocation error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
119	126	close(rfd);
120	127	return;
121	128	}
122	129
123		SetSeenTwin(0, 0);
124	130	for ( j=0; j < num_profiles; j++ ) {
125		profiles[j].first_seen = 0x7fffffff;
126		profiles[j].last_seen = 0;
	131	profiles[j].stat_record.first_seen = 0x7fffffff;
	132	profiles[j].stat_record.last_seen = 0;
	133	profiles[j].flow_header = (data_block_header_t *)malloc(OUTPUT_BUFF_SIZE);
	134	if ( !profiles[j].flow_header ) {
	135	fprintf(stderr, "Buffer allocation error: %s", strerror(errno));
	136	return;
	137	}
	138	profiles[j].flow_header->size = 0;
	139	profiles[j].flow_header->NumBlocks = 0;
	140	profiles[j].flow_header->pad = 0;
	141	profiles[j].flow_header->id = DATA_BLOCK_TYPE_1;
	142	profiles[j].writeto = (void *)((pointer_addr_t)profiles[j].flow_header + sizeof(data_block_header_t));
	143
	144	(profiles[j].engine)->nfrecord = (uint64_t *)&master_record;
127	145	}
128	146
129	147	done = 0;
130	148	while ( !done ) {
131		ret = read(rfd, &flow_header, FLOW_HEADER_LENGTH);
	149	ret = read(rfd, &in_flow_header, sizeof(data_block_header_t));
132	150	if ( ret == 0 ) {
133		done = 1;
134		if ( rfd ) // unless stdin
135		close(rfd);
136		break;
	151	// EOF of rfd
	152	rfd = GetNextFile(rfd, twin_start, twin_end, NULL);
	153	if ( rfd < 0 ) {
	154	if ( rfd == FILE_ERROR )
	155	fprintf(stderr, "Can't read from file '%s': %s\n",GetCurrentFilename(), strerror(errno) );
	156	done = 1;
	157	}
	158	continue;
	159
137	160	} else if ( ret == -1 ) {
138		perror("Error reading data");
	161	fprintf(stderr, "Can't read from file '%s': %s\n",GetCurrentFilename(), strerror(errno) );
139	162	close(rfd);
140	163	return;
141	164	}
142		if ( flow_header.version != NETFLOW_VERSION ) {
143		fprintf(stdout, "Not a netflow v5 header\n");
144		close(rfd);
145		return;
146		}
147		if ( flow_header.count > BuffNumRecords ) {
148		fprintf(stderr, "Too many records %u ( > BuffNumRecords )\n", flow_header.count);
149		break;
150		}
151
152		NumRecords = flow_header.count;
153
154		ret = read(rfd, record_buffer, NumRecords * FLOW_RECORD_LENGTH);
	165
	166	if ( in_flow_header.id != DATA_BLOCK_TYPE_1 ) {
	167	fprintf(stderr, "Can't process block type %u\n", in_flow_header.id);
	168	continue;
	169	}
	170
	171	NumRecords = in_flow_header.NumBlocks;
	172
	173	if ( in_flow_header.size > buffer_size ) {
	174	void *tmp;
	175	// Actually, this should never happen, but catch it anyway
	176	if ( in_flow_header.size > MAX_BUFFER_SIZE ) {
	177	// this is most likely corrupt
	178	fprintf(stderr, "Corrupt data file: Requested buffer size %u exceeds max. buffer size.\n", in_flow_header.size);
	179	break;
	180	}
	181	// make it at least the requested size
	182	buffer_size = in_flow_header.size;
	183	tmp = realloc((void *)in_buff, buffer_size);
	184	if ( !tmp ) {
	185	fprintf(stderr, "Can't reallocate buffer to %u bytes: %s\n", buffer_size, strerror(errno));
	186	break;
	187	}
	188	in_buff = (common_record_t *)tmp;
	189	}
	190	ret = read(rfd, in_buff, in_flow_header.size );
155	191	if ( ret == 0 ) {
156	192	done = 1;
157	193	break;
158	194	} else if ( ret == -1 ) {
159		perror("Error reading data");
	195	fprintf(stderr, "Can't read from file '%s': %s\n",GetCurrentFilename(), strerror(errno) );
160	196	close(rfd);
161	197	return;
162	198	}
163	199
164		for ( j=0; j < num_profiles; j++ ) {
165		// cnt is the number of blocks, which survived the filter
166		// ftrue is an array of flags of the filter result
167
168		// preset profile specific vars
169		// set flow_record to the beginn of all records
170		flow_record = record_buffer;
171		profiles[j].cnt = 0;
172		engine = profiles[j].engine;
173		for ( i=0; i < NumRecords; i++ ) {
174		// Time filter
175		// if no time filter is given, the result is always true
176		ftrue[i] = twin_start && (flow_record->First < twin_start \|\| flow_record->Last > twin_end) ? 0 : 1;
177
178		// netflow record filter
179		if ( ftrue[i] ) {
180		engine->nfrecord = (uint32_t *)flow_record;
181		ftrue[i] = (*engine->FilterEngine)(engine);
182		}
183
184		if ( ftrue[i] ) {
185		switch (flow_record->prot) {
186		case 1:
187		profiles[j].numflows_icmp++;
188		profiles[j].numpackets_icmp += flow_record->dPkts;
189		profiles[j].numbytes_icmp += flow_record->dOctets;
190		break;
191		case 6:
192		profiles[j].numflows_tcp++;
193		profiles[j].numpackets_tcp += flow_record->dPkts;
194		profiles[j].numbytes_tcp += flow_record->dOctets;
195		break;
196		case 17:
197		profiles[j].numflows_udp++;
198		profiles[j].numpackets_udp += flow_record->dPkts;
199		profiles[j].numbytes_udp += flow_record->dOctets;
200		break;
201		default:
202		profiles[j].numflows_other++;
203		profiles[j].numpackets_other += flow_record->dPkts;
204		profiles[j].numbytes_other += flow_record->dOctets;
205		}
206		profiles[j].numflows++;
207		profiles[j].numpackets += flow_record->dPkts;
208		profiles[j].numbytes += flow_record->dOctets;
209		profiles[j].cnt++;
210
211		if ( flow_record->First < profiles[j].first_seen )
212		profiles[j].first_seen = flow_record->First;
213		if ( flow_record->Last > profiles[j].last_seen )
214		profiles[j].last_seen = flow_record->Last;
215		}
216		// increment pointer by number of bytes for netflow record
217		flow_record = (void *)((pointer_addr_t)flow_record + FLOW_RECORD_LENGTH);
	200
	201	flow_record = in_buff;
	202	for ( i=0; i < NumRecords; i++ ) {
	203	ExpandRecord( flow_record, &master_record);
	204
	205	// Time filter
	206	// if no time filter is given, the result is always true
	207	flow_record->mark = twin_start && (flow_record->first < twin_start \|\| flow_record->last > twin_end) ? 0 : 1;
	208
	209	// if outside given time window
	210	if ( !flow_record->mark ) {
	211	flow_record = (void *)((pointer_addr_t)flow_record + flow_record->size);
	212	continue;
218	213	}
219	214
220		// set new count in header
221		flow_header.count = profiles[j].cnt;
222
223		// dump header and records only, if any block is left
224		if ( profiles[j].cnt ) {
225		/* write to file */
226		ret = write(profiles[j].wfd, &flow_header, FLOW_HEADER_LENGTH);
227		if ( ret < 0 ) {
228		perror("Error writing data");
	215	for ( j=0; j < num_profiles; j++ ) {
	216
	217	// apply profile filter
	218	engine = profiles[j].engine;
	219	flow_record->mark = (*engine->FilterEngine)(engine);
	220
	221	// if profile filter failed -> next profile
	222	if ( !flow_record->mark )
	223	continue;
	224
	225	// filter was successful -> continue record processing
	226	flow_record->mark = 0;
	227
	228	if ( (profiles[j].flow_header->size + flow_record->size) > OUTPUT_BUFF_SIZE ) {
	229	// this should really never happen
	230	fprintf(stderr, "Record size overflow in %s line %d: skip record.\n", __FILE__, __LINE__);
229	231	continue;
230	232	}
231		flow_record = record_buffer;
232		for ( i=0; i < NumRecords; i++ ) {
233		if ( ftrue[i] ) {
234		ret = write(profiles[j].wfd, flow_record, FLOW_RECORD_LENGTH);
235		if ( ret < 0 ) {
236		perror("Error writing data");
237		continue;
238		}
	233
	234	// update statistics
	235	switch (master_record.prot) {
	236	case 1:
	237	profiles[j].stat_record.numflows_icmp++;
	238	profiles[j].stat_record.numpackets_icmp += master_record.dPkts;
	239	profiles[j].stat_record.numbytes_icmp += master_record.dOctets;
	240	break;
	241	case 6:
	242	profiles[j].stat_record.numflows_tcp++;
	243	profiles[j].stat_record.numpackets_tcp += master_record.dPkts;
	244	profiles[j].stat_record.numbytes_tcp += master_record.dOctets;
	245	break;
	246	case 17:
	247	profiles[j].stat_record.numflows_udp++;
	248	profiles[j].stat_record.numpackets_udp += master_record.dPkts;
	249	profiles[j].stat_record.numbytes_udp += master_record.dOctets;
	250	break;
	251	default:
	252	profiles[j].stat_record.numflows_other++;
	253	profiles[j].stat_record.numpackets_other += master_record.dPkts;
	254	profiles[j].stat_record.numbytes_other += master_record.dOctets;
	255	}
	256	profiles[j].stat_record.numflows++;
	257	profiles[j].stat_record.numpackets += master_record.dPkts;
	258	profiles[j].stat_record.numbytes += master_record.dOctets;
	259
	260	if ( master_record.first < profiles[j].stat_record.first_seen ) {
	261	profiles[j].stat_record.first_seen = master_record.first;
	262	profiles[j].stat_record.msec_first = master_record.msec_first;
	263	}
	264	if ( master_record.first == profiles[j].stat_record.first_seen &&
	265	master_record.msec_first < profiles[j].stat_record.msec_first )
	266	profiles[j].stat_record.msec_first = master_record.msec_first;
	267
	268	if ( master_record.last > profiles[j].stat_record.last_seen ) {
	269	profiles[j].stat_record.last_seen = master_record.last;
	270	profiles[j].stat_record.msec_last = master_record.msec_last;
	271	}
	272	if ( master_record.last == profiles[j].stat_record.last_seen &&
	273	master_record.msec_last > profiles[j].stat_record.msec_last )
	274	profiles[j].stat_record.msec_last = master_record.msec_last;
	275
	276	// write record to output buffer
	277	memcpy(profiles[j].writeto, (void *)flow_record, flow_record->size);
	278	profiles[j].flow_header->NumBlocks++;
	279	profiles[j].flow_header->size += flow_record->size;
	280	profiles[j].writeto = (void *)((pointer_addr_t)profiles[j].writeto + flow_record->size);
	281
	282	// check if we need to flush the output buffer
	283	if ( (profiles[j].flow_header->size + flow_record->size) > OUTPUT_FLUSH_LIMIT && !zero_flows) {
	284	if ( write(profiles[j].wfd, (void *)profiles[j].flow_header,
	285	sizeof(data_block_header_t) + profiles[j].flow_header->size) <= 0 ) {
	286	fprintf(stderr, "Failed to write output buffer to disk: '%s'" , strerror(errno));
	287	} else {
	288	profiles[j].flow_header->size = 0;
	289	profiles[j].flow_header->NumBlocks = 0;
	290	profiles[j].writeto = (void *)((pointer_addr_t)profiles[j].flow_header + sizeof(data_block_header_t));
	291	profiles[j].file_blocks++;
239	292	}
240		// increment pointer by number of bytes for netflow record
241		flow_record = (void *)((pointer_addr_t)flow_record + FLOW_RECORD_LENGTH);
242		}
243		} // if cnt
244		} // for j all profiles
245		} // while read file
246
247		free((void *)record_buffer);
	293	}
	294
	295	} // End of for all profiles
	296
	297	flow_record = (common_record_t *)((pointer_addr_t)flow_record + flow_record->size);
	298
	299	} // End of for all NumRecords
	300	} // End of while !done
	301
	302	for ( j=0; j < num_profiles; j++ ) {
	303	// flush output buffer
	304	if ( profiles[j].flow_header->NumBlocks && !zero_flows) {
	305	if ( write(profiles[j].wfd, (void *)profiles[j].flow_header,
	306	sizeof(data_block_header_t) + profiles[j].flow_header->size) <= 0 ) {
	307	fprintf(stderr, "Failed to write output buffer to disk: '%s'" , strerror(errno));
	308	} else {
	309	free((void *)profiles[j].flow_header);
	310	profiles[j].writeto = NULL;
	311	profiles[j].file_blocks++;
	312	}
	313	}
	314	}
	315	free((void *)in_buff);
248	316
249	317	} // End of process_data
250	318
251	319
252	320	int main( int argc, char **argv ) {
253		unsigned int num_profiles, quiet;
	321	unsigned int num_profiles, quiet, zero_flows;
254	322	struct stat stat_buf;
255		char c, rfile, ffile, filename, p, tstring, profiledir, *subdir;
	323	char c, rfile, ffile, filename, Mdirs, tstring, profiledir, *subdir;
256	324	int syntax_only;
257	325	time_t t_start, t_end;
258	326
259	327	tstring = NULL;
260		profiledir = subdir = NULL;
	328	profiledir = subdir = Mdirs = NULL;
261	329	t_start = t_end = 0;
262	330	syntax_only = 0;
263	331	quiet = 0;
	332	zero_flows = 0;
264	333
265	334	// default file names
266	335	ffile = "filter.txt";
267		rfile = "nfcapd";
268		while ((c = getopt(argc, argv, "p:s:hf:r:Zt:Vq")) != EOF) {
	336	rfile = NULL;
	337	while ((c = getopt(argc, argv, "p:s:hf:r:M:Zt:Vqz")) != EOF) {
269	338	switch (c) {
270	339	case 'h':
271	340	usage(argv[0]);

290	359	case 't':
291	360	tstring = optarg;
292	361	break;
	362	case 'M':
	363	Mdirs = optarg;
	364	break;
293	365	case 'r':
294	366	rfile = optarg;
295	367	break;
296	368	case 'q':
297	369	quiet = 1;
	370	break;
	371	case 'z':
	372	zero_flows = 1;
298	373	break;
299	374	default:
300	375	usage(argv[0]);

315	390	exit(255);
316	391	}
317	392
318		p = strrchr(rfile, '/');
319		filename = p == NULL ? rfile : ++p;
320
321		if ( strlen(filename) == 0 ) {
322		fprintf(stderr, "Filename error");
323		exit(254);
324		}
	393	if ( syntax_only ) {
	394	filename = NULL;
	395	rfile = NULL;
	396	} else {
	397	char *p;
	398	p = strrchr(rfile, '/');
	399	filename = p == NULL ? rfile : ++p;
	400	if ( strlen(filename) == 0 ) {
	401	fprintf(stderr, "Filename error");
	402	exit(254);
	403	}
	404	}
325	405
326	406	num_profiles = InitProfiles(profiledir, subdir, ffile, filename, syntax_only, quiet);
	407
327	408	if ( !num_profiles )
328	409	exit(254);
329	410
330	411	if ( syntax_only )
331	412	exit(0);
332	413
333		SetupInputFileSequence(NULL,rfile, NULL);
	414	if ( !rfile ) {
	415	fprintf(stderr, "Input file (-r) required!\n");
	416	exit(255);
	417	}
	418
	419	SetupInputFileSequence(Mdirs,rfile, NULL);
334	420
335	421	if ( tstring ) {
336	422	if ( !ScanTimeFrame(tstring, &t_start, &t_end) )
337	423	exit(255);
338	424	}
339	425
340		process_data(GetProfiles(), num_profiles, t_start, t_end);
	426	process_data(GetProfiles(), num_profiles, t_start, t_end, zero_flows);
341	427
342	428	CloseProfiles();
343	429

+25

-6

nfreplay.1 less more

6	6	.SH DESCRIPTION
7	7	.B nfreplay
8	8	is the netflow replay program of the nfdump tool set.
9		It reads the netflow data from the files stored by nfcapd. The filter
10		syntax is equivalent to nfdump. If a filter is supplied, only the
11		matching flows are sent. See the nfdump(1) man page for a detailed
12		description of the filter syntax. All records are sent as netflow version 5.
	9	It reads data from files stored by nfcapd and sents the netflow
	10	data to a host or a multicat group. The filter syntax is equivalent
	11	to nfdump. If a filter is supplied, only the matching flows are sent.
	12	See the nfdump(1) man page for a detailed description of the filter
	13	syntax. All records are sent as netflow version 5.
13	14
14	15	.SH OPTIONS
15	16	.TP 3
16		.B -i \fIIP address
17		Send all flows to this IP address. Default is localhost 127.0.0.1
	17	.B -H \fIremotehost
	18	Send all flows to this remote host. Accepts a symbolic name or a IPv4/IPv6
	19	IP address. Defaults to IPv4 localhost 127.0.0.1.
	20	.TP 3
	21	.B -j \fImcastgroup
	22	Join this multicast group and send all flows to this group host. Accepts a
	23	symbolic name or multicast IPv4/IPv6 IP address.
18	24	.TP 3
19	25	.B -p \fIport
20	26	Send all fows to this port on the remote side. Default is 9995.
	27	.TP 3
	28	.B -4
	29	Forces nfreplay to send flows to a IPv4 address only. Can be used together with -i
	30	if the remote host has an IPv4 and IPv6 address record.
	31	.TP 3
	32	.B -6
	33	Forces nfreplay to send flows to a IPv6 address only. Can be used together with -i
	34	if the remote host has an IPv4 and IPv6 address record.
	35	.TP 3
	36	.B -v \fInum
	37	Send flows as netflow version \fInum\fR. \fB5\fR and \fB9\fR are supported. The
	38	default is sending the flows as netflow version 5. In version 5 mode, IPv6 flows,
	39	are skipped and 64bit counters are truncated to 32bit.
21	40	.TP 3
22	41	.B -d \fIusec
23	42	Delay each record bei \fIusec\fR mirco seconds, to avoid overrun on the remote

+392

-205

nfreplay.c less more

30	30	*
31	31	* $Author: peter $
32	32	*
33		* $Id: nfreplay.c 42 2005-08-24 12:32:39Z peter $
	33	* $Id: nfreplay.c 70 2006-05-17 08:38:01Z peter $
34	34	*
35		* $LastChangedRevision: 42 $
	35	* $LastChangedRevision: 70 $
36	36	*
37	37	*/
38	38

42	42	#include <errno.h>
43	43	#include <time.h>
44	44	#include <string.h>
	45	#include <ctype.h>
45	46	#include <sys/types.h>
46	47	#include <sys/stat.h>
47	48	#include <fcntl.h>
48	49	#include <sys/socket.h>
	50	#include <netdb.h>
49	51	#include <netinet/in.h>
50	52	#include <arpa/inet.h>
	53	#include <sys/resource.h>
51	54
52	55	#include "config.h"
53	56

55	58	#include <stdint.h>
56	59	#endif
57	60
58
59		#include "netflow_v5.h"
60	61	#include "version.h"
	62	#include "nffile.h"
61	63	#include "nf_common.h"
62	64	#include "nftree.h"
63	65	#include "nfdump.h"
	66	#include "nfnet.h"
	67	#include "netflow_v5_v7.h"
	68	#include "netflow_v9.h"
	69	#include "nfprof.h"
64	70	#include "util.h"
65	71	#include "grammar.h"
66
67		#define BuffNumRecords 1024
68
69		// all records should be version 5
70		#define NETFLOW_VERSION 5
	72	#include "panonymizer.h"
	73
	74	#define DEFAULTCISCOPORT "9995"
	75	#define DEFAULTHOSTNAME "127.0.0.1"
71	76
72	77	/* Externals */
73	78	extern int yydebug;
74	79
75	80	/* Global Variables */
76	81	FilterEngine_data_t *Engine;
77		int byte_mode, packet_mode;
	82	int byte_mode, packet_mode, verbose;
78	83	uint32_t byte_limit, packet_limit; // needed for linking purpose only
79	84
80	85	/* Local Variables */
81		static char const *rcsid = "$Id: nfreplay.c 42 2005-08-24 12:32:39Z peter $";
	86	static char const *rcsid = "$Id: nfreplay.c 70 2006-05-17 08:38:01Z peter $";
	87
	88	send_peer_t peer;
82	89
83	90	/* Function Prototypes */
84	91	static void usage(char *name);
85	92
86		static int create_send_socket(unsigned int wmem_size);
87
88		static void send_data(char rfile, int socket, char send_ip, int send_port, char *filter,
89		time_t twin_start, time_t twin_end, uint32_t count, unsigned int delay);
90
	93	static int ParseCryptoPAnKey ( char s, char key );
	94
	95	static void send_blast(unsigned int delay );
	96
	97	static void send_data(char *rfile, time_t twin_start, time_t twin_end,
	98	uint32_t count, unsigned int delay, int anon, int netflow_version);
	99
	100	static int FlushBuffer(void);
91	101
92	102	/* Functions */
93	103	static void usage(char *name) {
94	104	printf("usage %s [options] [\"filter\"]\n"
95	105	"-h\t\tthis text you see right here\n"
96	106	"-V\t\tPrint version and exit.\n"
97		"-i <ip>\t\tTarget IP address default: 127.0.0.1\n"
	107	"-H <Host/ip>\tTarget IP address default: 127.0.0.1\n"
	108	"-j <mcast>\tSend packets to multcast group\n"
	109	"-4\t\tForce IPv4 protocol.\n"
	110	"-6\t\tForce IPv6 protocol.\n"
98	111	"-p <port>\tTarget port default 9995\n"
99	112	"-d <usec>\tDelay in usec between packets. default 10\n"
100	113	"-c <cnt>\tPacket count. default send all packets\n"

106	119	, name);
107	120	} /* usage */
108	121
109		static int create_send_socket(unsigned int wmem_size) {
110		int send_socket;
111		unsigned int wmem_actual;
112		socklen_t optlen;
113
114		// create socket
115		send_socket = socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP);
116		if (send_socket == -1) {
117		perror("Opening output socket failed");
118		return 0;
119		}
120
121		// Set socket write buffer. Need to be root!
122		if ( wmem_size > 0 ) {
123		if ( geteuid() == 0 ) {
124		setsockopt(send_socket, SOL_SOCKET, SO_SNDBUF, &wmem_size, sizeof(wmem_size));
125
126		// check what was set (e.g. linux 2.4.20 sets twice of what was requested)
127		getsockopt(send_socket, SOL_SOCKET, SO_SNDBUF, &wmem_actual, &optlen);
128
129		if (wmem_size != wmem_actual) {
130		printf("Warning: Socket write buffer size requested: %u set: %u\n",
131		wmem_size, wmem_actual);
132		}
133		} else {
134		printf("Warning: Socket buffer size can only be changed by root!\n");
135		}
136		}
137
138
139		return send_socket;
140
141		} // End of create_send_socket
142
143		static void send_data(char rfile, int socket, char send_ip, int send_port, char *filter,
144		time_t twin_start, time_t twin_end, uint32_t count, unsigned int delay) {
145		netflow_v5_header_t *nf_header;
146		netflow_v5_record_t nf_record, record_buffer;
147		flow_record_t *flow_record;
148		void sendptr, sendbuff;
149		struct sockaddr_in send_to;
150		int i, rfd, done, ret, *ftrue, old_format;
151		uint32_t NumRecords, numflows, cnt, flow_sequence;
152		uint64_t boot_time;
153
154		// address descriptor
155		send_to.sin_family = AF_INET;
156		send_to.sin_port = htons(send_port);
157		// if ( inet_aton(send_ip, &(send_to.sin_addr)) == 0 ) {
158		if ((send_to.sin_addr.s_addr = inet_addr(send_ip)) == -1) {
159		perror("Invalid target IP address ");
	122	static int FlushBuffer(void) {
	123	size_t len = (pointer_addr_t)peer.writeto - (pointer_addr_t)peer.send_buffer;
	124
	125	peer.flush = 0;
	126	peer.writeto = peer.send_buffer;
	127	return sendto(peer.sockfd, peer.send_buffer, len, 0, (struct sockaddr *)&(peer.addr), peer.addrlen);
	128	} // End of FlushBuffer
	129
	130	static int ParseCryptoPAnKey ( char s, char key ) {
	131	int i, j;
	132	char numstr[3];
	133
	134	if ( strlen(s) == 32 ) {
	135	// Key is a string
	136	strncpy(key, s, 32);
	137	return 1;
	138	}
	139
	140	tolower(s[1]);
	141	numstr[2] = 0;
	142	if ( strlen(s) == 66 && s[0] == '0' && s[1] == 'x' ) {
	143	j = 2;
	144	for ( i=0; i<32; i++ ) {
	145	if ( !isxdigit((int)s[j]) \|\| !isxdigit((int)s[j+1]) )
	146	return 0;
	147	numstr[0] = s[j++];
	148	numstr[1] = s[j++];
	149	key[i] = strtol(numstr, NULL, 16);
	150	}
	151	return 1;
	152	}
	153
	154	// It's an invalid key
	155	return 0;
	156
	157	} // End of ParseCryptoPAnKey
	158
	159	static void send_blast(unsigned int delay ) {
	160	common_flow_header_t *header;
	161	nfprof_t profile_data;
	162	int i, ret;
	163	u_long usec, sec;
	164	double fps;
	165
	166	peer.send_buffer = malloc(1400);
	167	if ( !peer.send_buffer ) {
	168	perror("Memory allocation error");
	169	close(peer.sockfd);
160	170	return;
161	171	}
162
163		rfd = GetNextFile(0, twin_start, twin_end);
	172	header = (common_flow_header_t *)peer.send_buffer;
	173	header->version = htons(255);
	174	nfprof_start(&profile_data);
	175	for ( i = 0; i < 65535; i++ ) {
	176	header->count = htons(i);
	177	ret = sendto(peer.sockfd, peer.send_buffer, 1400, 0, (struct sockaddr *)&peer.addr, peer.addrlen);
	178	if ( ret < 0 \|\| ret != 1400 ) {
	179	perror("Error sending data");
	180	}
	181
	182	if ( delay ) {
	183	// sleep as specified
	184	usleep(delay);
	185	}
	186	}
	187	nfprof_end(&profile_data, 8655351400);
	188
	189
	190	usec = profile_data.used.ru_utime.tv_usec + profile_data.used.ru_stime.tv_usec;
	191	sec = profile_data.used.ru_utime.tv_sec + profile_data.used.ru_stime.tv_sec;
	192
	193	if (usec > 1000000)
	194	usec -= 1000000, ++sec;
	195
	196	if (profile_data.tend.tv_usec < profile_data.tstart.tv_usec)
	197	profile_data.tend.tv_usec += 1000000, --profile_data.tend.tv_sec;
	198
	199	usec = profile_data.tend.tv_usec - profile_data.tstart.tv_usec;
	200	sec = profile_data.tend.tv_sec - profile_data.tstart.tv_sec;
	201	fps = (double)profile_data.numflows / ((double)sec + ((double)usec/1000000));
	202
	203	fprintf(stdout, "Wall: %lu.%-3.3lus bps: %-10.1f\n", sec, usec/1000, fps);
	204
	205
	206	} // End of send_blast
	207
	208	static void send_data(char *rfile, time_t twin_start,
	209	time_t twin_end, uint32_t count, unsigned int delay, int anon, int netflow_version) {
	210	data_block_header_t flow_header;
	211	master_record_t master_record;
	212	common_record_t flow_record, in_buff;
	213	stat_record_t *stat_record;
	214	int i, rfd, done, ret, again, old_format;
	215	uint32_t NumRecords, numflows, cnt, buffer_size;
	216
	217	#ifdef COMPAT14
	218	extern int Format14;
	219	#endif
	220
	221	rfd = GetNextFile(0, twin_start, twin_end, &stat_record);
164	222	if ( rfd < 0 ) {
165		if ( errno )
166		perror("Can't open file for reading");
	223	if ( rfd == FILE_ERROR )
	224	fprintf(stderr, "Can't open file for reading: %s\n", strerror(errno));
167	225	return;
168	226	}
169	227
	228	#ifdef COMPAT14
	229	if ( Format14 ) {
	230	fprintf(stderr, "nfreplay does not support nfdump <= v1.4 old style file format!\n");
	231	return;
	232	}
	233	#endif
	234
170	235	// prepare read and send buffer
171		record_buffer = (netflow_v5_record_t *) calloc(BuffNumRecords , NETFLOW_V5_RECORD_LENGTH);
172		sendbuff = calloc(BuffNumRecords , NETFLOW_V5_RECORD_LENGTH) + NETFLOW_V5_HEADER_LENGTH;
173		ftrue = (int *) calloc(BuffNumRecords , sizeof(int));
174		if ( !record_buffer \|\| !sendbuff \|\| !ftrue ) {
	236	buffer_size = BUFFSIZE;
	237	in_buff = (common_record_t *) malloc(buffer_size);
	238	peer.send_buffer = malloc(UDP_PACKET_SIZE);
	239	peer.flush = 0;
	240	if ( !in_buff \|\| !peer.send_buffer ) {
175	241	perror("Memory allocation error");
176	242	close(rfd);
177	243	return;
178	244	}
179
180		numflows = 0;
181		done = 0;
182		flow_sequence = 0;
183		nf_header = (netflow_v5_header_t *)sendbuff;
184
	245	peer.writeto = peer.send_buffer;
	246	peer.endp = (void *)((pointer_addr_t)peer.send_buffer + UDP_PACKET_SIZE - 1);
	247
	248	if ( netflow_version == 5 )
	249	Init_v5_v7_output(&peer);
	250	else
	251	Init_v9_output(&peer);
	252
	253	numflows = 0;
	254	done = 0;
	255
	256	// setup Filter Engine to point to master_record, as any record read from file
	257	// is expanded into this record
	258	Engine->nfrecord = (uint64_t *)&master_record;
	259
	260	cnt = 0;
185	261	while ( !done ) {
186		ret = read(rfd, nf_header, NETFLOW_V5_HEADER_LENGTH);
	262	ret = read(rfd, &flow_header, sizeof(data_block_header_t));
187	263	if ( ret == 0 ) {
188	264	done = 1;
189	265	break;

192	268	close(rfd);
193	269	return;
194	270	}
195		if ( nf_header->version != NETFLOW_VERSION ) {
196		fprintf(stdout, "Not a netflow v5 header\n");
197		close(rfd);
198		return;
199		}
200		if ( nf_header->count > BuffNumRecords ) {
201		fprintf(stderr, "Too many records %u ( > BuffNumRecords )\n", nf_header->count);
202		break;
203		}
204
205		NumRecords = nf_header->count;
206		old_format = nf_header->reserved != 1;
207
208		ret = read(rfd, record_buffer, NumRecords * NETFLOW_V5_RECORD_LENGTH);
	271
	272	if ( flow_header.id != DATA_BLOCK_TYPE_1 ) {
	273	fprintf(stderr, "Can't process block type %u\n", flow_header.id);
	274	continue;
	275	}
	276
	277	NumRecords = flow_header.NumBlocks;
	278	old_format = 0;
	279
	280	if ( flow_header.size > buffer_size ) {
	281	void *tmp;
	282	// Actually, this should never happen, but catch it anyway
	283	if ( flow_header.size > MAX_BUFFER_SIZE ) {
	284	// this is most likely corrupt
	285	fprintf(stderr, "Corrupt data file: Requested buffer size %u exceeds max. buffer size.\n", flow_header.size);
	286	break;
	287	}
	288	// make it at least the requested size
	289	buffer_size = flow_header.size;
	290	tmp = realloc((void *)in_buff, buffer_size);
	291	if ( !tmp ) {
	292	fprintf(stderr, "Can't reallocate buffer to %u bytes: %s\n", buffer_size, strerror(errno));
	293	break;
	294	}
	295	in_buff = (common_record_t *)tmp;
	296	}
	297	ret = read(rfd, in_buff, flow_header.size);
209	298	if ( ret == 0 ) {
210	299	done = 1;
211	300	break;

215	304	return;
216	305	}
217	306
	307	if ( flow_header.size != ret ) {
	308	// Ups - this was a short read - most likely reading from the stdin pipe
	309	// loop until we have requested size
	310	size_t request_size, total_size;
	311	void *read_ptr;
	312
	313	total_size = ret;
	314	request_size = flow_header.size - total_size;
	315	read_ptr = (void *)((pointer_addr_t)in_buff + total_size);
	316	do {
	317	ret = read(rfd, read_ptr, request_size);
	318	if ( ret == 0 ) {
	319	break;
	320	} else if ( ret < 0 ) {
	321	perror("Error reading data");
	322	break;
	323	}
	324	total_size += ret;
	325	if ( total_size < flow_header.size ) {
	326	request_size = flow_header.size - ret;
	327	read_ptr = (void *)((pointer_addr_t)in_buff + total_size);
	328	request_size = flow_header.size - total_size;
	329	}
	330	} while ( ret > 0 && ( total_size < flow_header.size ));
	331
	332	if ( total_size != flow_header.size ) {
	333	// still unsuccessful
	334	#ifdef HAVE_SIZE_T_Z_FORMAT
	335	fprintf(stderr, "Short read for netflow records: Expected %i, got %zu bytes!\n",
	336	flow_header.size, total_size );
	337	#else
	338	fprintf(stderr, "Short read for netflow records: Expected %i, got %lu bytes!\n",
	339	flow_header.size, (unsigned long)total_size );
	340	#endif
	341	break;
	342	} else {
	343	// continue
	344	ret = flow_header.size;
	345	}
	346	}
	347
218	348	// cnt is the number of blocks, which survived the filter
219		// ftrue is an array of flags of the filter result
220		cnt = 0;
	349	// and added to the output buffer
	350	flow_record = in_buff;
	351
221	352	for ( i=0; i < NumRecords && numflows < count; i++ ) {
222		nf_record = &(record_buffer[i]);
223	353	// if no filter is given, the result is always true
224		ftrue[i] = twin_start ? nf_record->First >= twin_start && nf_record->Last <= twin_end : 1;
225		Engine->nfrecord = (uint32_t *)nf_record;
226
227		if ( filter && ftrue[i] )
228		ftrue[i] = (*Engine->FilterEngine)(Engine);
229
230		if ( ftrue[i] ) {
	354	ExpandRecord( flow_record, &master_record);
	355
	356	flow_record->mark = twin_start && (master_record.first < twin_start \|\| master_record.last > twin_end) ? 0 : 1;
	357
	358	// filter netflow record with user supplied filter
	359	if ( flow_record->mark )
	360	flow_record->mark = (*Engine->FilterEngine)(Engine);
	361
	362	if ( flow_record->mark == 0 ) { // record failed to pass all filters
	363	// increment pointer by number of bytes for netflow record
	364	flow_record = (common_record_t *)((pointer_addr_t)flow_record + flow_record->size);
	365	// go to next record
	366	continue;
	367	}
	368
	369	if ( anon ) {
	370	if ( (flow_record->flags & FLAG_IPV6_ADDR ) == 0 ) {
	371	master_record.v4.srcaddr = anonymize(master_record.v4.srcaddr);
	372	master_record.v4.dstaddr = anonymize(master_record.v4.dstaddr);
	373	} else {
	374	uint64_t anon_ip[2];
	375	anonymize_v6(master_record.v6.srcaddr, anon_ip);
	376	master_record.v6.srcaddr[0] = anon_ip[0];
	377	master_record.v6.srcaddr[1] = anon_ip[1];
	378
	379	anonymize_v6(master_record.v6.dstaddr, anon_ip);
	380	master_record.v6.dstaddr[0] = anon_ip[0];
	381	master_record.v6.dstaddr[1] = anon_ip[1];
	382	}
	383	}
	384
	385	if ( netflow_version == 5 )
	386	again = Add_v5_output_record(&master_record, &peer);
	387	else
	388	again = Add_v9_output_record(&master_record, &peer);
	389
	390	cnt++;
	391	numflows++;
	392
	393	if ( peer.flush ) {
	394	ret = FlushBuffer();
	395
	396	if ( ret < 0 ) {
	397	perror("Error sending data");
	398	close(rfd);
	399	return;
	400	}
	401
	402	if ( delay ) {
	403	// sleep as specified
	404	usleep(delay);
	405	}
	406	cnt = 0;
	407	}
	408
	409	if ( again ) {
	410	if ( netflow_version == 5 )
	411	Add_v5_output_record(&master_record, &peer);
	412	else
	413	Add_v9_output_record(&master_record, &peer);
231	414	cnt++;
232		numflows++;
233	415	}
234		}
235
236		// set new count in v5 header
237		nf_header->count = cnt;
238
239		// dump header and records only, if any block is left
240		if ( cnt ) {
241		sendptr = (void *)((pointer_addr_t)sendbuff + NETFLOW_V5_HEADER_LENGTH);
242		boot_time = ((uint64_t)(nf_header->unix_secs)*1000 +
243		((uint64_t)(nf_header->unix_nsecs) / 1000000) ) - (uint64_t)(nf_header->SysUptime);
244
245		nf_header->version = htons(nf_header->version);
246		nf_header->count = htons(nf_header->count);
247		nf_header->SysUptime = htonl(nf_header->SysUptime);
248		nf_header->unix_secs = htonl(nf_header->unix_secs);
249		nf_header->unix_nsecs = htonl(nf_header->unix_nsecs);
250		nf_header->flow_sequence = flow_sequence;
251		flow_sequence += cnt;
252
253		for ( i=0; i < NumRecords; i++ ) {
254		nf_record = &(record_buffer[i]);
255		if ( ftrue[i] ) {
256		/* the use of the flow_record cast is a bit ugly, but needs to be rewritten anyway when v9 comes */
257		flow_record = (flow_record_t *)nf_record;
258
259		/* may be removed when old format died out */
260		if ( old_format ) {
261		flow_record->msec_first = 0;
262		flow_record->msec_last = 0;
263		}
264
265		nf_record->First = (uint32_t)(1000LL * (uint64_t)flow_record->First + flow_record->msec_first - boot_time);
266		nf_record->Last = (uint32_t)(1000LL * (uint64_t)flow_record->Last + flow_record->msec_last - boot_time);
267
268		nf_record->srcaddr = htonl(nf_record->srcaddr);
269		nf_record->dstaddr = htonl(nf_record->dstaddr);
270		nf_record->nexthop = htonl(nf_record->nexthop);
271		nf_record->input = htons(nf_record->input);
272		nf_record->output = htons(nf_record->output);
273		nf_record->dPkts = htonl(nf_record->dPkts);
274		nf_record->dOctets = htonl(nf_record->dOctets);
275		nf_record->First = htonl(nf_record->First);
276		nf_record->Last = htonl(nf_record->Last);
277		nf_record->srcport = htons(nf_record->srcport);
278		nf_record->dstport = htons(nf_record->dstport);
279		nf_record->src_as = htons(nf_record->src_as);
280		nf_record->dst_as = htons(nf_record->dst_as);
281		nf_record->src_mask = 0;
282		nf_record->dst_mask = 0;
283		nf_record->pad2 = 0;
284
285		memcpy(sendptr, nf_record, NETFLOW_V5_RECORD_LENGTH);
286		sendptr = (void *)((pointer_addr_t)sendptr + NETFLOW_V5_RECORD_LENGTH);
287		}
288		// increment pointer by number of bytes for netflow record
289		nf_record = (void *)((pointer_addr_t)nf_record + NETFLOW_V5_RECORD_LENGTH);
290
291		}
292
293		ret = sendto(socket, (void )sendbuff, cnt NETFLOW_V5_RECORD_LENGTH + NETFLOW_V5_HEADER_LENGTH, 0,
294		(struct sockaddr *)&send_to, sizeof(send_to));
295
296		if ( ret < 0 ) {
297		perror("Error writing data");
298		close(rfd);
299		return;
300		}
301
302		if ( delay ) {
303		// sleep as specified
304		usleep(delay);
305		}
306
307		if ( numflows >= count )
308		done = 1;
309
310		} // if cnt
	416
	417	// Advance pointer by number of bytes for netflow record
	418	flow_record = (common_record_t *)((pointer_addr_t)flow_record + flow_record->size);
	419	}
311	420	} // while
	421
	422	// flush still remaining records
	423	if ( cnt ) {
	424	ret = FlushBuffer();
	425
	426	if ( ret < 0 ) {
	427	perror("Error sending data");
	428	}
	429
	430	} // if cnt
	431
	432	if ( rfd )
	433	close(rfd);
	434
	435	close(peer.sockfd);
	436
	437	return;
312	438
313	439	} // End of send_data
314	440
315	441
316	442	int main( int argc, char **argv ) {
317	443	struct stat stat_buff;
318		char c, rfile, ffile, filter, tstring;
319		char *send_ip;
320		int send_port, ffd, ret, sockfd;
321		unsigned int delay, count, sockbuff;
	444	char rfile, ffile, filter, tstring;
	445	char CryptoPAnKey[32];
	446	int c, do_anonymize, ffd, ret, blast, netflow_version;
	447	unsigned int delay, count, sockbuff_size;
322	448	time_t t_start, t_end;
323	449
324	450	rfile = ffile = filter = tstring = NULL;
325	451	t_start = t_end = 0;
326		send_ip = "127.0.0.1";
327		send_port = 9995;
328		delay = 10;
	452
	453	peer.hostname = DEFAULTHOSTNAME;
	454	peer.port = DEFAULTCISCOPORT;
	455	peer.mcast = 0;
	456	peer.family = AF_UNSPEC;
	457	peer.sockfd = 0;
	458
	459	delay = 1;
329	460	count = 0xFFFFFFFF;
330		sockbuff = 0;
331		while ((c = getopt(argc, argv, "hi:p:d:c:b:r:f:t:V")) != EOF) {
	461	sockbuff_size = 0;
	462	netflow_version = 5;
	463	do_anonymize = 0;
	464	blast = 0;
	465	verbose = 0;
	466	while ((c = getopt(argc, argv, "46BhH:i:K:p:d:c:b:j:r:f:t:v:V")) != EOF) {
332	467	switch (c) {
333	468	case 'h':
334	469	usage(argv[0]);
335	470	exit(0);
336	471	break;
	472	case 'B':
	473	blast = 1;
	474	break;
337	475	case 'V':
338	476	printf("%s: Version: %s %s\n%s\n",argv[0], nfdump_version, nfdump_date, rcsid);
339	477	exit(0);
340	478	break;
341		case 'i':
342		send_ip = optarg;
	479	case 'H':
	480	case 'i': // compatibility with old version
	481	peer.hostname = strdup(optarg);
	482	peer.mcast = 0;
	483	break;
	484	case 'j':
	485	if ( peer.hostname == NULL ) {
	486	peer.hostname = strdup(optarg);
	487	peer.mcast = 1;
	488	} else {
	489	fprintf(stderr, "ERROR, -H(-i) and -j are mutually exclusive!!\n");
	490	exit(255);
	491	}
	492	break;
	493	case 'K':
	494	if ( !ParseCryptoPAnKey(optarg, CryptoPAnKey) ) {
	495	fprintf(stderr, "Invalid key '%s' for CryptoPAn!\n", optarg);
	496	exit(255);
	497	}
	498	do_anonymize = 1;
343	499	break;
344	500	case 'p':
345		send_port = atoi(optarg);
346		if ( send_port <= 0 \|\| send_port > 65535 ) {
347		fprintf(stderr, "Send port out of range\n");
348		exit(255);
349		}
	501	peer.port = strdup(optarg);
350	502	break;
351	503	case 'd':
352	504	delay = atoi(optarg);
353	505	break;
	506	case 'v':
	507	netflow_version = atoi(optarg);
	508	if ( netflow_version != 5 && netflow_version != 9 ) {
	509	fprintf(stderr, "Invalid netflow version: %s. Accept only 5 or 9!\n", optarg);
	510	exit(255);
	511	}
	512	break;
354	513	case 'c':
355	514	count = atoi(optarg);
356	515	break;
357	516	case 'b':
358		sockbuff = atoi(optarg);
	517	sockbuff_size = atoi(optarg);
359	518	break;
360	519	case 'f':
361	520	ffile = optarg;

365	524	break;
366	525	case 'r':
367	526	rfile = optarg;
	527	break;
	528	case '4':
	529	if ( peer.family == AF_UNSPEC )
	530	peer.family = AF_INET;
	531	else {
	532	fprintf(stderr, "ERROR, Accepts only one protocol IPv4 or IPv6!\n");
	533	exit(255);
	534	}
	535	break;
	536	case '6':
	537	if ( peer.family == AF_UNSPEC )
	538	peer.family = AF_INET6;
	539	else {
	540	fprintf(stderr, "ERROR, Accepts only one protocol IPv4 or IPv6!\n");
	541	exit(255);
	542	}
368	543	break;
369	544	default:
370	545	usage(argv[0]);

410	585	if ( !Engine )
411	586	exit(254);
412	587
413		sockfd = create_send_socket(sockbuff);
414		if ( sockfd <= 0 ) {
	588	if ( peer.mcast )
	589	peer.sockfd = Multicast_send_socket (peer.hostname, peer.port, peer.family, sockbuff_size,
	590	&peer.addr, &peer.addrlen );
	591	else
	592	peer.sockfd = Unicast_send_socket (peer.hostname, peer.port, peer.family, sockbuff_size,
	593	&peer.addr, &peer.addrlen );
	594	if ( peer.sockfd <= 0 ) {
415	595	exit(255);
416	596	}
417	597
	598	if ( blast ) {
	599	send_blast(delay );
	600	exit(0);
	601	}
418	602
419	603	SetupInputFileSequence(NULL,rfile, NULL);
420	604

423	607	exit(255);
424	608	}
425	609
426		send_data(rfile, sockfd, send_ip, send_port, filter, t_start, t_end, count, delay);
	610	if (do_anonymize)
	611	PAnonymizer_Init((uint8_t *)CryptoPAnKey);
	612
	613	send_data(rfile, t_start, t_end, count, delay, do_anonymize, netflow_version);
427	614
428	615	return 0;
429	616	}

+464

-347

nfstat.c less more

28	28	*
29	29	* $Author: peter $
30	30	*
31		* $Id: nfstat.c 53 2005-11-17 07:45:34Z peter $
	31	* $Id: nfstat.c 75 2006-05-21 15:32:48Z peter $
32	32	*
33		* $LastChangedRevision: 53 $
	33	* $LastChangedRevision: 75 $
34	34	*
35	35	*/
36	36

51	51	#endif
52	52
53	53	#include "nfdump.h"
54		#include "netflow_v5.h"
	54	#include "nffile.h"
	55	#include "nfnet.h"
	56	#include "netflow_v5_v7.h"
55	57	#include "nf_common.h"
56	58	#include "util.h"
57	59	#include "panonymizer.h"
58	60	#include "nfstat.h"
59	61
60	62	struct flow_element_s {
61		uint32_t offset; // set in the netflow record block
62		uint32_t mask; // mask for value in 32bit word
	63	uint32_t offset0;
	64	uint32_t offset1; // set in the netflow record block
	65	uint64_t mask; // mask for value in 64bit word
63	66	uint32_t shift; // number of bits to shift right to get final value
64	67	};
65	68

70	73	// need 2 elements to be able to get src/dst stats in one stat record
71	74	uint8_t num_elem; // number of elements used. 1 ord 2
72	75	uint8_t ipconv; // is an IP address: Convert number to readable IP Addr
	76	uint16_t aggregate_bits; // These bits must be set in aggregate mask
73	77	} StatParameters[] ={
74	78	// flow record stst
75		{ "record", "", { {0,0, 0}, {0,0,0} }, 1, 0},
	79	{ "record", "",
	80	{ {0,0, 0}, {0,0,0} },
	81	1, 0, 0},
76	82
77	83	// 9 possible flow element stats
78		{ "srcip", "Src IP Addr", { {OffsetSrcIP, MaskIP, 0}, {0,0,0} }, 1, 1},
79		{ "dstip", "Dst IP Addr", { {OffsetDstIP, MaskIP, 0}, {0,0,0} }, 1, 1},
80		{ "ip", " IP Addr", { {OffsetSrcIP, MaskIP, 0}, {OffsetDstIP, MaskIP, 0} }, 2, 1},
81		{ "srcport", " Src Port", { {OffsetPort, MaskSrcPort, ShiftSrcPort}, {0,0,0} }, 1, 0},
82		{ "dstport", " Dst Port", { {OffsetPort, MaskDstPort, ShiftDstPort}, {0,0,0} }, 1, 0},
83		{ "port", " Port", { {OffsetPort, MaskSrcPort, ShiftSrcPort}, {OffsetPort, MaskDstPort, ShiftDstPort} }, 2, 0},
84		{ "srcas", " Src AS", { {OffsetAS, MaskSrcAS, ShiftSrcAS}, {0,0,0} }, 1, 0},
85		{ "dstas", " Dst AS", { {OffsetAS, MaskDstAS, ShiftDstAS}, {0,0,0} }, 1, 0},
86		{ "as", " AS", { {OffsetAS, MaskSrcAS, ShiftSrcAS}, {OffsetAS, MaskDstAS, ShiftDstAS} }, 2, 0},
87
88		{ NULL, NULL, { {0,0, 0}, {0,0,0} }, 1, 0}
	84	{ "srcip", "Src IP Addr",
	85	{ {OffsetSrcIPv6a, OffsetSrcIPv6b, MaskIPv6, 0}, {0,0,0,0} },
	86	1, 1, Aggregate_SRCIP},
	87
	88	{ "dstip", "Dst IP Addr",
	89	{ {OffsetDstIPv6a, OffsetDstIPv6b, MaskIPv6, 0}, {0,0,0,0} },
	90	1, 1, Aggregate_DSTIP},
	91
	92	{ "ip", " IP Addr",
	93	{ {OffsetSrcIPv6a, OffsetSrcIPv6b, MaskIPv6, 0}, {OffsetDstIPv6a, OffsetDstIPv6b, MaskIPv6} },
	94	2, 1, Aggregate_SRCIP \| Aggregate_DSTIP },
	95
	96	{ "srcport", " Src Port",
	97	{ {0, OffsetPort, MaskSrcPort, ShiftSrcPort}, {0,0,0,0} },
	98	1, 0, Aggregate_SRCPORT},
	99
	100	{ "dstport", " Dst Port",
	101	{ {0, OffsetPort, MaskDstPort, ShiftDstPort}, {0,0,0,0} },
	102	1, 0, Aggregate_DSTPORT},
	103
	104	{ "port", " Port",
	105	{ {0, OffsetPort, MaskSrcPort, ShiftSrcPort}, {0, OffsetPort, MaskDstPort, ShiftDstPort}},
	106	2, 0, Aggregate_SRCPORT \| Aggregate_DSTPORT},
	107
	108	{ "proto", " Protocol",
	109	{ {0, OffsetProto, MaskProto, ShiftProto}, {0,0,0,0} },
	110	1, 0, 0},
	111
	112	{ "srcas", " Src AS",
	113	{ {0, OffsetAS, MaskSrcAS, ShiftSrcAS}, {0,0,0,0} },
	114	1, 0, 0},
	115
	116	{ "dstas", " Dst AS",
	117	{ {0, OffsetAS, MaskDstAS, ShiftDstAS}, {0,0,0,0} },
	118	1, 0, 0},
	119
	120	{ "as", " AS",
	121	{ {0, OffsetAS, MaskSrcAS, ShiftSrcAS}, {0, OffsetAS, MaskDstAS, ShiftDstAS} },
	122	2, 0, 0},
	123
	124	{ "inif", " Input If",
	125	{ {0, OffsetInOut, MaskInput, ShiftInput}, {0,0,0,0} },
	126	1, 0, 0},
	127
	128	{ "outif", " Output If",
	129	{ {0, OffsetInOut, MaskOutput, ShiftOutput}, {0,0,0,0} },
	130	1, 0, 0},
	131
	132	{ "if", " In/Out If",
	133	{ {0, OffsetInOut, MaskInput, ShiftInput}, {0, OffsetInOut, MaskOutput, ShiftOutput} },
	134	1, 0, 0},
	135
	136	{ NULL, NULL,
	137	{ {0,0,0,0}, {0,0,0,0} },
	138	1, 0, 0}
89	139	};
90	140
91	141	static const uint32_t NumOrders = 6; // Number of Stats in enum StatTypes
	142	// StatType max 32767
92	143	enum StatTypes { FLOWS = 0, PACKETS, BYTES, PPS, BPS, BPP };
93	144
	145	#define MaxStats 16
94	146	struct StatRequest_s {
95		int16_t StatType; // value out of enum StatTypes
96		uint16_t order_bits; // bits 0: flows 1: packets 2: bytes 3: pps 4: bps, 5 bpp
97		} StatRequest[9]; // 9 = number of possible flow element stats
	147	uint16_t order_bits; // bits 0: flows 1: packets 2: bytes 3: pps 4: bps, 5 bpp
	148	int16_t StatType; // value out of enum StatTypes
	149	uint8_t order_proto; // protocol separated statistics
	150	} StatRequest[MaxStats]; // This number should do it for a single run
98	151
99	152	uint32_t flow_stat_order;
100	153	/*

132	185	#define MaxMemBlocks 256
133	186
134	187	/* function prototypes */
135		static int ParseStatString(char str, int16_t StatType, uint16_t order_bits, int flow_record_stat);
136
137		static inline FlowTableRecord_t hash_lookup_FlowTable(uint32_t index_cache, uint8_t proto,
138		uint32_t addr, uint32_t dstaddr, uint16_t port, uint16_t dstport);
139
140		static inline FlowTableRecord_t *hash_insert_FlowTable(uint32_t index_cache,
141		uint32_t addr, uint32_t dstaddr, uint16_t port, uint16_t dstport);
142
143		static inline StatRecord_t *stat_hash_lookup(uint32_t addr, int hash_num);
144
145		static inline StatRecord_t *stat_hash_insert(uint32_t addr, int hash_num);
	188	static int ParseStatString(char str, int16_t StatType, uint16_t order_bits, int flow_record_stat, uint16_t *order_proto);
	189
	190	static inline FlowTableRecord_t hash_lookup_FlowTable(uint32_t index_cache, master_record_t *flow_record);
	191
	192	static inline FlowTableRecord_t hash_insert_FlowTable(uint32_t index_cache, master_record_t flow_record);
	193
	194	static inline StatRecord_t stat_hash_lookup(uint64_t value, uint8_t prot, int hash_num);
	195
	196	static inline StatRecord_t stat_hash_insert(uint64_t value, uint8_t prot, int hash_num);
146	197
147	198	static void Expand_FlowTable_Blocks(void);
148	199
149	200	static void Expand_StatTable_Blocks(int hash_num);
150	201
151		static inline void MapRecord(flow_record_t flow_record, void record);
152
153		static void PrintStatLine(StatRecord_t *StatData, int ipconv, int anon);
	202	static inline void MapRecord(master_record_t flow_record, void record);
	203
	204	static void PrintStatLine(StatRecord_t *StatData, int ipconv, int anon, int order_proto);
154	205
155	206	static void Create_topN_FlowStat(SortElement_t *topN_lists, int order, int topN, uint32_t count );
156	207
	208	static inline int TimeMsec_CMP(time_t t1, uint16_t offset1, time_t t2, uint16_t offset2 );
	209
157	210	// static SortElement_t Make_TopN_packets(int topN, uint32_t count);
158	211
159	212	// static SortElement_t Make_TopN_bytes(int topN, uint32_t count);

164	217
165	218	static void heapSort(SortElement_t *SortElement, uint32_t array_size, int topN);
166	219
167		static void siftDown(SortElement_t *SortElement, uint32_t root, uint32_t bottom);
	220	static inline void siftDown(SortElement_t *SortElement, uint32_t root, uint32_t bottom);
168	221
169	222	/* locals */
170	223	#ifndef __SUNPRO_C

178	231	hash_StatTable *StatTable;
179	232
180	233	static int NumStats = 0, DefaultOrder;
181
182		#define mix(a,b,c) { \
	234	#define mix64(a,b,c) \
	235	{ \
	236	a=a-b; a=a-c; a=a^(c>>43); \
	237	b=b-c; b=b-a; b=b^(a<<9); \
	238	c=c-a; c=c-b; c=c^(b>>8); \
	239	a=a-b; a=a-c; a=a^(c>>38); \
	240	b=b-c; b=b-a; b=b^(a<<23); \
	241	c=c-a; c=c-b; c=c^(b>>5); \
	242	a=a-b; a=a-c; a=a^(c>>35); \
	243	b=b-c; b=b-a; b=b^(a<<49); \
	244	c=c-a; c=c-b; c=c^(b>>11); \
	245	a=a-b; a=a-c; a=a^(c>>12); \
	246	b=b-c; b=b-a; b=b^(a<<18); \
	247	c=c-a; c=c-b; c=c^(b>>22); \
	248	}
	249
	250	#define mix32(a,b,c) { \
183	251	a -= b; a -= c; a ^= (c>>13); \
184	252	b -= c; b -= a; b ^= (a<<8); \
185	253	c -= a; c -= b; c ^= (b>>13); \

222	290
223	291	} // End of bpp_function
224	292
	293	static inline int TimeMsec_CMP(time_t t1, uint16_t offset1, time_t t2, uint16_t offset2 ) {
	294	if ( t1 > t2 )
	295	return 1;
	296	if ( t2 > t1 )
	297	return 2;
	298	// else t1 == t2 - offset is now relevant
	299	if ( offset1 > offset2 )
	300	return 1;
	301	if ( offset2 > offset1 )
	302	return 2;
	303	else
	304	// both times are the same
	305	return 0;
	306	} // End of TimeMsec_CMP
	307
225	308	int Init_FlowTable(uint16_t NumBits, uint32_t Prealloc) {
226	309	uint32_t maxindex;
227	310

332	415
333	416	} // End of Dispose_Tables
334	417
	418	char *VerifyStat(uint16_t Aggregate_Bits) {
	419	int16_t i, StatType;
	420
	421	for ( i=0; i<NumStats ; i++ ) {
	422	StatType = StatRequest[i].StatType;
	423	if ( (StatParameters[StatType].aggregate_bits & Aggregate_Bits) != StatParameters[StatType].aggregate_bits ) {
	424	return StatParameters[StatType].statname;
	425	}
	426	}
	427	return NULL;
	428
	429	} // End of VerifyStat
	430
335	431	int SetStat(char str, int element_stat, int *flow_stat) {
336	432	int flow_record_stat = 0;
337		int16_t StatType = 0;
338		uint16_t order_bits = 0;
339
340		if ( ParseStatString(str, &StatType, &order_bits, &flow_record_stat) ) {
	433	int16_t StatType = 0;
	434	uint16_t order_bits = 0;
	435	uint16_t order_proto = 0;
	436
	437	if ( NumStats == MaxStats ) {
	438	fprintf(stderr, "Too many stat options! Stats are limited to %i stats per single run!\n", MaxStats);
	439	return 0;
	440	}
	441	if ( ParseStatString(str, &StatType, &order_bits, &flow_record_stat, &order_proto) ) {
341	442	if ( flow_record_stat ) {
342	443	flow_stat_order = order_bits;
343	444	*flow_stat = 1;
344	445	} else {
345		StatRequest[NumStats].StatType = StatType;
346		StatRequest[NumStats].order_bits = order_bits;
	446	StatRequest[NumStats].StatType = StatType;
	447	StatRequest[NumStats].order_bits = order_bits;
	448	StatRequest[NumStats].order_proto = order_proto;
347	449	NumStats++;
348	450	*element_stat = 1;
349	451	}
350	452	return 1;
351	453	} else {
	454	fprintf(stderr, "Unknown stat: '%s'!\n", str);
352	455	return 0;
353	456	}
354	457
355	458	} // End of SetStat
356	459
357		static int ParseStatString(char str, int16_t StatType, uint16_t order_bits, int flow_record_stat) {
358		char s, q, *r;
	460	static int ParseStatString(char str, int16_t StatType, uint16_t order_bits, int flow_record_stat, uint16_t *order_proto) {
	461	char s, p, q, r;
359	462	int i=0;
360	463
361		if ( NumStats >= 9 )
	464	if ( NumStats >= MaxStats )
362	465	return 0;
363	466
364	467	s = strdup(str);
365	468	q = strchr(s, '/');
366	469	if ( q )
367	470	*q = 0;
	471
	472	*order_proto = 0;
	473	p = strchr(s, ':');
	474	if ( p ) {
	475	*p = 0;
	476	*order_proto = 1;
	477	}
368	478
369	479	i = 0;
370	480	// check for a valid stat name

381	491	if ( StatParameters[i].statname ) {
382	492	*StatType = i;
383	493	*order_bits = 0;
	494	if ( strncasecmp(StatParameters[i].statname, "proto", 16) == 0 )
	495	*order_proto = 1;
384	496	} else {
385	497	return 0;
386	498	}

435	547
436	548	} // End of SetStat_DefaultOrder
437	549
438		static inline FlowTableRecord_t hash_lookup_FlowTable(uint32_t index_cache, uint8_t proto,
439		uint32_t addr, uint32_t dstaddr, uint16_t port, uint16_t dstport ) {
	550	static inline FlowTableRecord_t hash_lookup_FlowTable(uint32_t index_cache, master_record_t *flow_record) {
	551	// uint64_t index, a1, a2;
440	552	uint32_t index, a1, a2;
441	553	FlowTableRecord_t *record;
442	554
443		index = port ^ dstport;
444		a1 = addr; a2 = dstaddr;
445		mix(a1, a2, index);
	555	index = (uint32_t)flow_record->srcport << 16 \| (uint32_t)flow_record->dstport;
	556	a1 = flow_record->v4.srcaddr;
	557	a2 = flow_record->v4.dstaddr;
	558	mix32(a1, a2, index);
	559
	560	// a1 = flow_record->v6.srcaddr[1];
	561	// a2 = flow_record->v6.dstaddr[1];
	562	// mix64(a1, a2, index);
446	563
447	564	index = (index ^ ( index >> ( 32 - FlowTable.NumBits ))) & FlowTable.IndexMask;
	565
448	566	*index_cache = index;
449	567
450	568	if ( FlowTable.bucket[index] == NULL )

452	570
453	571	record = FlowTable.bucket[index];
454	572	while ( record ) {
455		if ( ( record->ip1 == addr && record->ip2 == dstaddr &&
456		record->port1 == port && record->port2 == dstport &&
457		record->proto == proto ) )
	573	if ( record->srcport == flow_record->srcport && record->dstport == flow_record->dstport &&
	574	record->prot == flow_record->prot &&
	575	record->ip.v6.srcaddr[1] == flow_record->v6.srcaddr[1] && record->ip.v6.srcaddr[0] == flow_record->v6.srcaddr[0] )
458	576	return record;
459	577	record = record->next;
460	578	}

462	580
463	581	} // End of hash_lookup_FlowTable
464	582
465		static inline StatRecord_t *stat_hash_lookup(uint32_t addr, int hash_num) {
	583	static inline StatRecord_t stat_hash_lookup(uint64_t value, uint8_t prot, int hash_num) {
466	584	uint32_t index;
467	585	StatRecord_t *record;
468	586
469		index = addr & StatTable[hash_num].IndexMask;
	587	index = value[1] & StatTable[hash_num].IndexMask;
	588
470	589	if ( StatTable[hash_num].bucket[index] == NULL )
471	590	return NULL;
472	591
473	592	record = StatTable[hash_num].bucket[index];
474		while ( record && ( record->stat_key != addr ) ) {
475		record = record->next;
	593	if ( StatRequest[hash_num].order_proto ) {
	594	while ( record && ( record->stat_key[1] != value[1] \|\| record->stat_key[0] != value[0] \|\| prot != record->prot ) ) {
	595	record = record->next;
	596	}
	597	} else {
	598	while ( record && ( record->stat_key[1] != value[1] \|\| record->stat_key[0] != value[0] ) ) {
	599	record = record->next;
	600	}
476	601	}
477	602	return record;
478	603

524	649
525	650	} // End of Expand_StatTable_Blocks
526	651
527		inline static FlowTableRecord_t *hash_insert_FlowTable(uint32_t index_cache,
528		uint32_t addr, uint32_t dstaddr, uint16_t port, uint16_t dstport) {
	652	inline static FlowTableRecord_t hash_insert_FlowTable(uint32_t index_cache, master_record_t flow_record) {
529	653	FlowTableRecord_t *record;
530	654
531	655	if ( FlowTable.NextElem >= FlowTable.Prealloc )

533	657
534	658	record = &(FlowTable.memblock[FlowTable.NextBlock][FlowTable.NextElem]);
535	659	FlowTable.NextElem++;
536		record->next = NULL;
537		record->ip1 = addr;
538		record->ip2 = dstaddr;
539		record->port1 = port;
540		record->port2 = dstport;
	660	record->next = NULL;
	661	record->prot = flow_record->prot;
	662	record->srcport = flow_record->srcport;
	663	record->dstport = flow_record->dstport;
	664	record->ip.v6.srcaddr[0] = flow_record->v6.srcaddr[0];
	665	record->ip.v6.srcaddr[1] = flow_record->v6.srcaddr[1];
	666	record->ip.v6.dstaddr[0] = flow_record->v6.dstaddr[0];
	667	record->ip.v6.dstaddr[1] = flow_record->v6.dstaddr[1];
541	668
542	669	if ( FlowTable.bucket[index_cache] == NULL )
543	670	FlowTable.bucket[index_cache] = record;

549	676
550	677	} // End of hash_insert_FlowTable
551	678
552		void InsertFlow(flow_record_t *flow_record) {
	679	void InsertFlow(master_record_t *flow_record) {
553	680	FlowTableRecord_t *record;
554	681
555	682	if ( FlowTable.NextElem >= FlowTable.Prealloc )

559	686	FlowTable.NextElem++;
560	687
561	688	record->next = NULL;
562		record->ip1 = flow_record->srcaddr;
563		record->ip2 = flow_record->dstaddr;
564		record->port1 = flow_record->srcport;
565		record->port2 = flow_record->dstport;
	689	record->ip.v6.srcaddr[0] = flow_record->v6.srcaddr[0];
	690	record->ip.v6.srcaddr[1] = flow_record->v6.srcaddr[1];
	691	record->ip.v6.dstaddr[0] = flow_record->v6.dstaddr[0];
	692	record->ip.v6.dstaddr[1] = flow_record->v6.dstaddr[1];
	693	record->srcport = flow_record->srcport;
	694	record->dstport = flow_record->dstport;
566	695	record->counter[BYTES] = flow_record->dOctets;
567	696	record->counter[PACKETS] = flow_record->dPkts;
568		record->first = flow_record->First;
	697	record->first = flow_record->first;
569	698	record->msec_first = flow_record->msec_first;
570		record->last = flow_record->Last;
	699	record->last = flow_record->last;
571	700	record->msec_last = flow_record->msec_last;
572		record->proto = flow_record->prot;
	701	record->prot = flow_record->prot;
573	702	record->tcp_flags = flow_record->tcp_flags;
574	703	record->tos = flow_record->tos;
	704	record->srcas = flow_record->srcas;
	705	record->dstas = flow_record->dstas;
	706	record->input = flow_record->input;
	707	record->output = flow_record->output;
575	708	record->counter[FLOWS] = 1;
576	709
577	710	} // End of InsertFlow
578	711
579	712
580		inline static StatRecord_t *stat_hash_insert(uint32_t addr, int hash_num) {
	713	static inline StatRecord_t stat_hash_insert(uint64_t value, uint8_t prot, int hash_num) {
581	714	uint32_t index;
582	715	StatRecord_t *record;
583	716

586	719
587	720	record = &(StatTable[hash_num].memblock[StatTable[hash_num].NextBlock][StatTable[hash_num].NextElem]);
588	721	StatTable[hash_num].NextElem++;
589		record->next = NULL;
590		record->stat_key = addr;
591
592		index = addr & StatTable[hash_num].IndexMask;
	722	record->next = NULL;
	723	record->stat_key[0] = value[0];
	724	record->stat_key[1] = value[1];
	725	record->prot = prot;
	726
	727	index = value[1] & StatTable[hash_num].IndexMask;
593	728	if ( StatTable[hash_num].bucket[index] == NULL )
594	729	StatTable[hash_num].bucket[index] = record;
595	730	else

600	735
601	736	} // End of stat_hash_insert
602	737
603		int AddStat(flow_header_t flow_header, flow_record_t flow_record, int flow_stat, int element_stat ) {
	738	int AddStat(data_block_header_t flow_header, master_record_t flow_record, int flow_stat, int element_stat ) {
604	739	FlowTableRecord_t *FlowTableRecord;
605	740	StatRecord_t *stat_record;
606		uint32_t index_cache, value;
	741	uint32_t index_cache;
	742	uint64_t value[2];
607	743	int j, i;
608	744
609	745	if ( flow_stat ) {
610	746	// Update netflow statistics
611		FlowTableRecord = hash_lookup_FlowTable(&index_cache, flow_record->prot,
612		flow_record->srcaddr, flow_record->dstaddr, flow_record->srcport, flow_record->dstport);
	747	FlowTableRecord = hash_lookup_FlowTable(&index_cache, flow_record);
613	748	if ( FlowTableRecord ) {
614	749	FlowTableRecord->counter[BYTES] += flow_record->dOctets;
615	750	FlowTableRecord->counter[PACKETS] += flow_record->dPkts;
616	751
617		if ( TimeMsec_CMP(flow_record->First, flow_record->msec_first, FlowTableRecord->first, FlowTableRecord->msec_first) == 2) {
618		FlowTableRecord->first = flow_record->First;
	752	if ( TimeMsec_CMP(flow_record->first, flow_record->msec_first, FlowTableRecord->first, FlowTableRecord->msec_first) == 2) {
	753	FlowTableRecord->first = flow_record->first;
619	754	FlowTableRecord->msec_first = flow_record->msec_first;
620	755	}
621		if ( TimeMsec_CMP(flow_record->Last, flow_record->msec_last, FlowTableRecord->last, FlowTableRecord->msec_last) == 1) {
622		FlowTableRecord->last = flow_record->Last;
	756	if ( TimeMsec_CMP(flow_record->last, flow_record->msec_last, FlowTableRecord->last, FlowTableRecord->msec_last) == 1) {
	757	FlowTableRecord->last = flow_record->last;
623	758	FlowTableRecord->msec_last = flow_record->msec_last;
624	759	}
625	760
626	761	FlowTableRecord->counter[FLOWS]++;
627	762
628	763	} else {
629		FlowTableRecord = hash_insert_FlowTable(index_cache,
630		flow_record->srcaddr, flow_record->dstaddr, flow_record->srcport, flow_record->dstport);
	764	FlowTableRecord = hash_insert_FlowTable(index_cache, flow_record);
631	765	if ( !FlowTableRecord )
632	766	return -1;
633	767
	768	FlowTableRecord->record_flags = flow_record->flags & 1LL;
	769	FlowTableRecord->first = flow_record->first;
	770	FlowTableRecord->msec_first = flow_record->msec_first;
	771	FlowTableRecord->last = flow_record->last;
	772	FlowTableRecord->msec_last = flow_record->msec_last;
	773	FlowTableRecord->tcp_flags = flow_record->tcp_flags;
	774	FlowTableRecord->tos = flow_record->tos;
	775	FlowTableRecord->srcas = flow_record->srcas;
	776	FlowTableRecord->dstas = flow_record->dstas;
	777	FlowTableRecord->input = flow_record->input;
	778	FlowTableRecord->output = flow_record->output;
634	779	FlowTableRecord->counter[BYTES] = flow_record->dOctets;
635	780	FlowTableRecord->counter[PACKETS] = flow_record->dPkts;
636		FlowTableRecord->first = flow_record->First;
637		FlowTableRecord->msec_first = flow_record->msec_first;
638		FlowTableRecord->last = flow_record->Last;
639		FlowTableRecord->msec_last = flow_record->msec_last;
640		FlowTableRecord->tos = flow_record->tos;
641		FlowTableRecord->tcp_flags = flow_record->tcp_flags;
642		FlowTableRecord->proto = flow_record->prot;
643	781	FlowTableRecord->counter[FLOWS] = 1;
644	782	}
645	783	}

651	789	int stat = StatRequest[j].StatType;
652	790	// for the number of elements in this stat type
653	791	for ( i=0; i<StatParameters[stat].num_elem; i++ ) {
654		uint32_t offset = StatParameters[stat].element[i].offset;
655		uint32_t mask = StatParameters[stat].element[i].mask;
	792	uint32_t offset = StatParameters[stat].element[i].offset1;
	793	uint64_t mask = StatParameters[stat].element[i].mask;
656	794	uint32_t shift = StatParameters[stat].element[i].shift;
657	795
658		value = ((uint32_t *)flow_record)[offset] & mask;
659		value = value >> shift;
660		stat_record = stat_hash_lookup(value, j);
	796	value[1] = (((uint64_t *)flow_record)[offset] & mask) >> shift;
	797	offset = StatParameters[stat].element[i].offset0;
	798	value[0] = offset ? ((uint64_t *)flow_record)[offset] : 0;
	799
	800	stat_record = stat_hash_lookup(value, flow_record->prot, j);
661	801	if ( stat_record ) {
662	802	stat_record->counter[BYTES] += flow_record->dOctets;
663	803	stat_record->counter[PACKETS] += flow_record->dPkts;
664	804
665		if ( TimeMsec_CMP(flow_record->First, flow_record->msec_first, stat_record->first, stat_record->msec_first) == 2) {
666		stat_record->first = flow_record->First;
	805	if ( TimeMsec_CMP(flow_record->first, flow_record->msec_first, stat_record->first, stat_record->msec_first) == 2) {
	806	stat_record->first = flow_record->first;
667	807	stat_record->msec_first = flow_record->msec_first;
668	808	}
669		if ( TimeMsec_CMP(flow_record->Last, flow_record->msec_last, stat_record->last, stat_record->msec_last) == 1) {
670		stat_record->last = flow_record->Last;
	809	if ( TimeMsec_CMP(flow_record->last, flow_record->msec_last, stat_record->last, stat_record->msec_last) == 1) {
	810	stat_record->last = flow_record->last;
671	811	stat_record->msec_last = flow_record->msec_last;
672	812	}
673	813	stat_record->counter[FLOWS]++;
674	814
675	815	} else {
676		stat_record = stat_hash_insert(value, j);
	816	stat_record = stat_hash_insert(value, flow_record->prot, j);
677	817	if ( !stat_record )
678	818	return -1;
679	819
680	820	stat_record->counter[BYTES] = flow_record->dOctets;
681	821	stat_record->counter[PACKETS] = flow_record->dPkts;
682		stat_record->first = flow_record->First;
	822	stat_record->first = flow_record->first;
683	823	stat_record->msec_first = flow_record->msec_first;
684		stat_record->last = flow_record->Last;
	824	stat_record->last = flow_record->last;
685	825	stat_record->msec_last = flow_record->msec_last;
	826	stat_record->record_flags = flow_record->flags & 0x1;
686	827	stat_record->counter[FLOWS] = 1;
	828	/*
	829	* srcas, dstas, inout and output interface: these values are assumed to be constant
	830	* and are set when this flow is seen the first time
	831	*/
687	832	}
688	833	} // for the number of elements in this stat type
689	834	} // for every requested -s stat

693	838
694	839	} // End of AddStat
695	840
696		static inline void MapRecord(flow_record_t flow_record, void record) {
697		/* This function is needed to normalize the data the feed a flow_record_t for printing */
698
699		flow_record->srcaddr = ((FlowTableRecord_t *)record)->ip1;
700		flow_record->dstaddr = ((FlowTableRecord_t *)record)->ip2;
701		flow_record->srcport = ((FlowTableRecord_t *)record)->port1;
702		flow_record->dstport = ((FlowTableRecord_t *)record)->port2;
	841	static inline void MapRecord(master_record_t flow_record, void record) {
	842	/* This function normalizes the data for printing */
	843
	844	flow_record->flags = ((FlowTableRecord_t *)record)->record_flags;
	845
	846	flow_record->v6.srcaddr[0] = ((FlowTableRecord_t *)record)->ip.v6.srcaddr[0];
	847	flow_record->v6.srcaddr[1] = ((FlowTableRecord_t *)record)->ip.v6.srcaddr[1];
	848	flow_record->v6.dstaddr[0] = ((FlowTableRecord_t *)record)->ip.v6.dstaddr[0];
	849	flow_record->v6.dstaddr[1] = ((FlowTableRecord_t *)record)->ip.v6.dstaddr[1];
	850	flow_record->srcport = ((FlowTableRecord_t *)record)->srcport;
	851	flow_record->dstport = ((FlowTableRecord_t *)record)->dstport;
703	852	flow_record->dOctets = ((FlowTableRecord_t *)record)->counter[BYTES];
704	853	flow_record->dPkts = ((FlowTableRecord_t *)record)->counter[PACKETS];
705	854
706		flow_record->First = ((FlowTableRecord_t *)record)->first;
707		flow_record->msec_first = ((FlowTableRecord_t *)record)->msec_first;
708		flow_record->Last = ((FlowTableRecord_t *)record)->last;
	855	flow_record->first = ((FlowTableRecord_t *)record)->first;
	856	flow_record->msec_first = ((FlowTableRecord_t *)record)->msec_first;
	857	flow_record->last = ((FlowTableRecord_t *)record)->last;
709	858	flow_record->msec_last = ((FlowTableRecord_t *)record)->msec_last;
710	859
711		flow_record->prot = ((FlowTableRecord_t *)record)->proto;
	860	flow_record->prot = ((FlowTableRecord_t *)record)->prot;
712	861	flow_record->tcp_flags = ((FlowTableRecord_t *)record)->tcp_flags;
713	862	flow_record->tos = ((FlowTableRecord_t *)record)->tos;
714	863
	864	flow_record->srcas = ((FlowTableRecord_t *)record)->srcas;
	865	flow_record->dstas = ((FlowTableRecord_t *)record)->dstas;
	866	flow_record->input = ((FlowTableRecord_t *)record)->input;
	867	flow_record->output = ((FlowTableRecord_t *)record)->output;
	868
715	869	} // End of MapRecord
716	870
717		static void PrintStatLine(StatRecord_t *StatData, int ipconv, int anon) {
718		char *str, valstr[32], datestr[64], flows_str[32], byte_str[32], packets_str[32], pps_str[32], bps_str[32];
	871	static void PrintStatLine(StatRecord_t *StatData, int ipconv, int anon, int order_proto) {
	872	char proto[16], valstr[40], datestr[64], flows_str[32], byte_str[32], packets_str[32], pps_str[32], bps_str[32];
719	873	double duration;
720	874	uint32_t pps, bps, bpp;
721		time_t First;
	875	time_t first;
722	876	struct tm *tbuff;
723		struct in_addr a;
724	877
725	878	if ( ipconv ) {
726		a.s_addr = htonl(StatData->stat_key);
727		if ( anon ) {
728		a.s_addr = anonymize(a.s_addr);
729		}
730		str = inet_ntoa(a);
731		strncpy(valstr, str, 15);
732		valstr[15] = 0;
	879	if ( (StatData->record_flags & 0x1) != 0 ) { // IPv6
	880	if ( anon ) {
	881	uint64_t anon_ip[2];
	882	anonymize_v6(StatData->stat_key, anon_ip);
	883	StatData->stat_key[0] = anon_ip[0];
	884	StatData->stat_key[1] = anon_ip[1];
	885	}
	886	StatData->stat_key[0] = htonll(StatData->stat_key[0]);
	887	StatData->stat_key[1] = htonll(StatData->stat_key[1]);
	888	inet_ntop(AF_INET6, StatData->stat_key, valstr, sizeof(valstr));
	889	if ( ! Getv6Mode() )
	890	condense_v6(valstr);
	891
	892	} else { // IPv4
	893	uint32_t ipv4 = StatData->stat_key[1];
	894	if ( anon ) {
	895	ipv4 = anonymize(ipv4);
	896	}
	897	ipv4 = htonl(ipv4);
	898	inet_ntop(AF_INET, &ipv4, valstr, sizeof(valstr));
	899	}
733	900	} else {
734		snprintf(valstr, 15, "%u", StatData->stat_key);
735		valstr[31] = 0;
736		}
737
738		format_number(StatData->counter[FLOWS], flows_str);
739		format_number(StatData->counter[PACKETS], packets_str);
740		format_number(StatData->counter[BYTES], byte_str);
	901	snprintf(valstr, 40, "%llu", StatData->stat_key[1]);
	902	}
	903	valstr[39] = 0;
	904
	905	format_number(StatData->counter[FLOWS], flows_str, FIXED_WIDTH);
	906	format_number(StatData->counter[PACKETS], packets_str, FIXED_WIDTH);
	907	format_number(StatData->counter[BYTES], byte_str, FIXED_WIDTH);
741	908
742	909	duration = StatData->last - StatData->first;
743	910	duration += ((double)StatData->msec_last - (double)StatData->msec_first) / 1000.0;

749	916	pps = bps = 0;
750	917	}
751	918	bpp = StatData->counter[BYTES] / StatData->counter[PACKETS];
752		format_number(pps, pps_str);
753		format_number(bps, bps_str);
754
755		First = StatData->first;
756		tbuff = localtime(&First);
	919	format_number(pps, pps_str, FIXED_WIDTH);
	920	format_number(bps, bps_str, FIXED_WIDTH);
	921
	922	first = StatData->first;
	923	tbuff = localtime(&first);
757	924	if ( !tbuff ) {
758	925	perror("Error time convert");
759	926	exit(250);
760	927	}
761	928	strftime(datestr, 63, "%Y-%m-%d %H:%M:%S", tbuff);
762	929
763		printf("%s.%03u %8.3f %15s %8s %8s %8s %8s %8s %5u\n", datestr, StatData->msec_first, duration,
764		valstr, flows_str, packets_str, byte_str, pps_str, bps_str, bpp );
	930	if ( order_proto ) {
	931	Proto_string(StatData->prot, proto);
	932	} else {
	933	snprintf(proto, 15, "any ");
	934	proto[15] = 0;
	935	}
	936
	937	if ( Getv6Mode() && ipconv )
	938	printf("%s.%03u %9.3f %s %39s %8s %8s %8s %8s %8s %5u\n", datestr, StatData->msec_first, duration, proto,
	939	valstr, flows_str, packets_str, byte_str, pps_str, bps_str, bpp );
	940	else
	941	printf("%s.%03u %9.3f %s %16s %8s %8s %8s %8s %8s %5u\n", datestr, StatData->msec_first, duration, proto,
	942	valstr, flows_str, packets_str, byte_str, pps_str, bps_str, bpp );
765	943
766	944	} // End of PrintStatLine
767	945
768	946	void ReportAggregated(printer_t print_record, uint32_t limitflows, int date_sorted, int anon) {
769	947	FlowTableRecord_t *r;
770		flow_record_t flow_record;
	948	master_record_t flow_record;
771	949	SortElement_t *SortList;
772	950	uint32_t i, j, tmp;
773	951	uint32_t maxindex, c;
774	952	char *string;
775	953
776	954	c = 0;
	955	flow_record.flags = 0;
	956
777	957	maxindex = ( FlowTable.NextBlock * FlowTable.Prealloc ) + FlowTable.NextElem;
778	958	if ( date_sorted ) {
779	959	// Sort according the date
780		SortList = (SortElement_t *)calloc(maxindex+1, sizeof(SortElement_t));
	960	SortList = (SortElement_t *)calloc(maxindex, sizeof(SortElement_t));
781	961
782	962	if ( !SortList ) {
783	963	perror("Can't allocate Top N lists: \n");

812	992	}
813	993	}
814	994
815		SortList[maxindex].count = 0;
816		// heapSort(SortList, maxindex, 0);
817	995	if ( c >= 2 )
818	996	heapSort(SortList, c, 0);
819	997

821	999	maxindex = limitflows;
822	1000	for ( i = 0; i < maxindex; i++ ) {
823	1001
824		flow_record.srcaddr = ((FlowTableRecord_t *)(SortList[i].record))->ip1;
825		flow_record.dstaddr = ((FlowTableRecord_t *)(SortList[i].record))->ip2;
826		flow_record.srcport = ((FlowTableRecord_t *)(SortList[i].record))->port1;
827		flow_record.dstport = ((FlowTableRecord_t *)(SortList[i].record))->port2;
828		flow_record.dOctets = ((FlowTableRecord_t *)(SortList[i].record))->counter[BYTES];
829		flow_record.dPkts = ((FlowTableRecord_t *)(SortList[i].record))->counter[PACKETS];
830
831		flow_record.First = ((FlowTableRecord_t *)(SortList[i].record))->first;
832		flow_record.msec_first = ((FlowTableRecord_t *)(SortList[i].record))->msec_first;
833		flow_record.Last = ((FlowTableRecord_t *)(SortList[i].record))->last;
834		flow_record.msec_last = ((FlowTableRecord_t *)(SortList[i].record))->msec_last;
835
836		flow_record.prot = ((FlowTableRecord_t *)(SortList[i].record))->proto;
837		flow_record.tcp_flags = ((FlowTableRecord_t *)(SortList[i].record))->tcp_flags;
838		flow_record.tos = ((FlowTableRecord_t *)(SortList[i].record))->tos;
839
840		print_record((void )&flow_record, ((FlowTableRecord_t )(SortList[i].record))->counter[FLOWS],
841		((FlowTableRecord_t *)(SortList[i].record))->counter[PACKETS],
842		((FlowTableRecord_t *)(SortList[i].record))->counter[BYTES],
843		&string, anon);
	1002	flow_record.flags = ((FlowTableRecord_t *)(SortList[i].record))->record_flags;
	1003
	1004	flow_record.v6.srcaddr[0] = ((FlowTableRecord_t *)(SortList[i].record))->ip.v6.srcaddr[0];
	1005	flow_record.v6.srcaddr[1] = ((FlowTableRecord_t *)(SortList[i].record))->ip.v6.srcaddr[1];
	1006	flow_record.v6.dstaddr[0] = ((FlowTableRecord_t *)(SortList[i].record))->ip.v6.dstaddr[0];
	1007	flow_record.v6.dstaddr[1] = ((FlowTableRecord_t *)(SortList[i].record))->ip.v6.dstaddr[1];
	1008	flow_record.srcport = ((FlowTableRecord_t *)(SortList[i].record))->srcport;
	1009	flow_record.dstport = ((FlowTableRecord_t *)(SortList[i].record))->dstport;
	1010	flow_record.dOctets = ((FlowTableRecord_t *)(SortList[i].record))->counter[BYTES];
	1011	flow_record.dPkts = ((FlowTableRecord_t *)(SortList[i].record))->counter[PACKETS];
	1012
	1013	flow_record.first = ((FlowTableRecord_t *)(SortList[i].record))->first;
	1014	flow_record.msec_first = ((FlowTableRecord_t *)(SortList[i].record))->msec_first;
	1015	flow_record.last = ((FlowTableRecord_t *)(SortList[i].record))->last;
	1016	flow_record.msec_last = ((FlowTableRecord_t *)(SortList[i].record))->msec_last;
	1017
	1018	flow_record.prot = ((FlowTableRecord_t *)(SortList[i].record))->prot;
	1019	flow_record.tcp_flags = ((FlowTableRecord_t *)(SortList[i].record))->tcp_flags;
	1020	flow_record.tos = ((FlowTableRecord_t *)(SortList[i].record))->tos;
	1021
	1022	flow_record.srcas = ((FlowTableRecord_t *)(SortList[i].record))->srcas;
	1023	flow_record.dstas = ((FlowTableRecord_t *)(SortList[i].record))->dstas;
	1024	flow_record.input = ((FlowTableRecord_t *)(SortList[i].record))->input;
	1025	flow_record.output = ((FlowTableRecord_t *)(SortList[i].record))->output;
	1026
	1027	print_record((void )&flow_record, ((FlowTableRecord_t )(SortList[i].record))->counter[FLOWS], &string, anon);
844	1028	printf("%s\n", string);
845	1029
846	1030	}

870	1054	}
871	1055	}
872	1056
873		flow_record.srcaddr = r->ip1;
874		flow_record.dstaddr = r->ip2;
875		flow_record.srcport = r->port1;
876		flow_record.dstport = r->port2;
877		flow_record.dOctets = r->counter[BYTES];
878		flow_record.dPkts = r->counter[PACKETS];
879		flow_record.First = r->first;
880		flow_record.msec_first = r->msec_first;
881		flow_record.Last = r->last;
882		flow_record.msec_last = r->msec_last;
883		flow_record.prot = r->proto;
884		flow_record.tcp_flags = r->tcp_flags;
885		flow_record.tos = r->tos;
886
887		print_record((void *)&flow_record, r->counter[FLOWS], r->counter[PACKETS], r->counter[BYTES], &string, anon);
	1057	flow_record.flags = r->record_flags;
	1058	flow_record.v6.srcaddr[0] = r->ip.v6.srcaddr[0];
	1059	flow_record.v6.srcaddr[1] = r->ip.v6.srcaddr[1];
	1060	flow_record.v6.dstaddr[0] = r->ip.v6.dstaddr[0];
	1061	flow_record.v6.dstaddr[1] = r->ip.v6.dstaddr[1];
	1062	flow_record.srcport = r->srcport;
	1063	flow_record.dstport = r->dstport;
	1064	flow_record.dOctets = r->counter[BYTES];
	1065	flow_record.dPkts = r->counter[PACKETS];
	1066	flow_record.first = r->first;
	1067	flow_record.msec_first = r->msec_first;
	1068	flow_record.last = r->last;
	1069	flow_record.msec_last = r->msec_last;
	1070	flow_record.prot = r->prot;
	1071	flow_record.tcp_flags = r->tcp_flags;
	1072	flow_record.tos = r->tos;
	1073	flow_record.srcas = r->srcas;
	1074	flow_record.dstas = r->dstas;
	1075	flow_record.input = r->input;
	1076	flow_record.output = r->output;
	1077
	1078	print_record((void *)&flow_record, r->counter[FLOWS], &string, anon);
888	1079	printf("%s\n", string);
889	1080
890	1081	c++;

898	1089	void ReportStat(char *record_header, printer_t print_record, int topN, int flow_stat, int element_stat, int anon) {
899	1090	SortElement_t *topN_flow_list[NumOrders];
900	1091	SortElement_t *topN_element_list;
901		flow_record_t flow_record;
	1092	master_record_t flow_record;
902	1093	uint32_t numflows, maxindex;
903	1094	int32_t i, j, hash_num, order_index, order_bit;
904	1095	char *string;
905	1096
906
	1097	flow_record.flags = 0;
	1098	numflows = 0;
907	1099	if ( flow_stat ) {
908	1100	for ( i=0; i<NumOrders; i++ ) {
909	1101	topN_flow_list[i] = (SortElement_t *)calloc(topN, sizeof(SortElement_t));

926	1118	for ( i=topN-1; i>=0; i--) {
927	1119	if ( !topN_flow_list[order_index][i].count )
928	1120	break;
929
930	1121	MapRecord(&flow_record, topN_flow_list[order_index][i].record);
931
932		print_record((void *)&flow_record,
933		((FlowTableRecord_t *)(topN_flow_list[order_index][i].record))->counter[FLOWS],
934		((FlowTableRecord_t *)(topN_flow_list[order_index][i].record))->counter[PACKETS],
935		((FlowTableRecord_t *)(topN_flow_list[order_index][i].record))->counter[BYTES],
936		&string, anon);
	1122	print_record((void )&flow_record, ((FlowTableRecord_t )(topN_flow_list[order_index][i].record))->counter[FLOWS], &string, anon);
937	1123	printf("%s\n", string);
938
939	1124	}
940	1125	printf("\n");
941	1126	}

949	1134	for ( i=topN-1; i>=0; i--) {
950	1135	if ( !topN_flow_list[PACKETS][i].count )
951	1136	break;
952
953	1137	MapRecord(&flow_record, topN_flow_list[PACKETS][i].record);
954
955	1138	print_record((void )&flow_record, ((FlowTableRecord_t )(topN_flow_list[PACKETS][i].record))->counter[FLOWS], &string, anon);
956	1139	printf("%s\n", string);
957	1140

964	1147	for ( i=topN-1; i>=0; i--) {
965	1148	if ( !topN_flow_list[BYTES][i].count )
966	1149	break;
967
968	1150	MapRecord(&flow_record, topN_flow_list[BYTES][i].record);
969
970	1151	print_record((void )&flow_record, ((FlowTableRecord_t )(topN_flow_list[BYTES][i].record))->counter[FLOWS], &string, anon);
971	1152	printf("%s\n", string);
972	1153

980	1161	int stat = StatRequest[hash_num].StatType;
981	1162	int order = StatRequest[hash_num].order_bits;
982	1163	int ipconv = StatParameters[stat].ipconv;
983
984	1164	for ( order_index=0; order_index<NumOrders; order_index++ ) {
985	1165	order_bit = 1 << order_index;
986	1166	if ( order & order_bit ) {
987	1167	topN_element_list = StatTopN(topN, &numflows, hash_num, order_index);
988	1168	printf("Top %i %s ordered by %s:\n", topN, StatParameters[stat].HeaderInfo, order_mode[order_index].string);
989		// 2005-07-26 20:08:59.197 1553.730 ss 65255 203435 52.2 M 130 281636 268
990		printf("Date first seen Duration %s Flows Packets Bytes pps bps bpp\n",
991		StatParameters[stat].HeaderInfo);
	1169	// 2005-07-26 20:08:59.197 1553.730 ss 65255 203435 52.2 M 130 281636 268
	1170	if ( Getv6Mode() && ipconv )
	1171	printf("Date first seen Duration Proto %39s Flows Packets Bytes pps bps bpp\n",
	1172	StatParameters[stat].HeaderInfo);
	1173	else
	1174	printf("Date first seen Duration Proto %16s Flows Packets Bytes pps bps bpp\n",
	1175	StatParameters[stat].HeaderInfo);
992	1176
993	1177	maxindex = ( StatTable[hash_num].NextBlock * StatTable[hash_num].Prealloc ) + StatTable[hash_num].NextElem;
994	1178	j = numflows - topN;
995	1179	j = j < 0 ? 0 : j;
996	1180	if ( topN == 0 )
997	1181	j = 0;
998
999	1182	for ( i=numflows-1; i>=j ; i--) {
1000	1183	if ( !topN_element_list[i].count )
1001	1184	break;
1002		PrintStatLine((StatRecord_t *)topN_element_list[i].record, ipconv, anon);
	1185	PrintStatLine((StatRecord_t *)topN_element_list[i].record, ipconv, anon, StatRequest[hash_num].order_proto);
1003	1186	}
1004	1187	free((void *)topN_element_list);
1005	1188	printf("\n");

1065	1248	void PrintSortedFlows(printer_t print_record, uint32_t limitflows, int anon) {
1066	1249	FlowTableRecord_t *r;
1067	1250	SortElement_t *SortList;
1068		flow_record_t flow_record;
	1251	master_record_t flow_record;
1069	1252	unsigned int i, j, tmp;
1070	1253	uint32_t maxindex, c;
1071	1254	char *string;
1072	1255
	1256	flow_record.flags = 0;
1073	1257	maxindex = ( FlowTable.NextBlock * FlowTable.Prealloc ) + FlowTable.NextElem;
1074		SortList = (SortElement_t *)calloc(maxindex+1, sizeof(SortElement_t));
	1258	SortList = (SortElement_t *)calloc(maxindex, sizeof(SortElement_t));
1075	1259
1076	1260	if ( !SortList ) {
1077	1261	perror("Can't allocate Top N lists: \n");

1092	1276	}
1093	1277	}
1094	1278
1095		SortList[maxindex].count = 0;
1096	1279	heapSort(SortList, maxindex, 0);
1097	1280
1098	1281	if ( limitflows && limitflows < maxindex )

1100	1283	for ( i=0; i<maxindex; i++ ) {
1101	1284	r = SortList[i].record;
1102	1285
1103		flow_record.srcaddr = r->ip1;
1104		flow_record.dstaddr = r->ip2;
1105		flow_record.srcport = r->port1;
1106		flow_record.dstport = r->port2;
1107		flow_record.dOctets = r->counter[BYTES];
1108		flow_record.dPkts = r->counter[PACKETS];
1109		flow_record.First = r->first;
1110		flow_record.msec_first = r->msec_first;
1111		flow_record.Last = r->last;
1112		flow_record.msec_last = r->msec_last;
1113		flow_record.prot = r->proto;
1114		flow_record.tcp_flags = r->tcp_flags;
1115		flow_record.tos = r->tos;
1116
1117		print_record((void *)&flow_record, 1, r->counter[PACKETS], r->counter[BYTES], &string, anon);
	1286	flow_record.flags = r->record_flags;
	1287	flow_record.v6.srcaddr[0] = r->ip.v6.srcaddr[0];
	1288	flow_record.v6.srcaddr[1] = r->ip.v6.srcaddr[1];
	1289	flow_record.v6.dstaddr[0] = r->ip.v6.dstaddr[0];
	1290	flow_record.v6.dstaddr[1] = r->ip.v6.dstaddr[1];
	1291	flow_record.srcport = r->srcport;
	1292	flow_record.dstport = r->dstport;
	1293	flow_record.dOctets = r->counter[BYTES];
	1294	flow_record.dPkts = r->counter[PACKETS];
	1295	flow_record.first = r->first;
	1296	flow_record.msec_first = r->msec_first;
	1297	flow_record.last = r->last;
	1298	flow_record.msec_last = r->msec_last;
	1299	flow_record.prot = r->prot;
	1300	flow_record.tcp_flags = r->tcp_flags;
	1301	flow_record.tos = r->tos;
	1302	flow_record.srcas = r->srcas;
	1303	flow_record.dstas = r->dstas;
	1304	flow_record.input = r->input;
	1305	flow_record.output = r->output;
	1306
	1307	print_record((void *)&flow_record, 1, &string, anon);
1118	1308
1119	1309	if ( string )
1120	1310	printf("%s\n", string);

1123	1313	free(SortList);
1124	1314
1125	1315	} // End of PrintSortedFlows
1126
1127		/*
1128		* Some different implementation of statistics
1129		*
1130		static SortElement_t Make_TopN_packets(int topN, uint32_t count) {
1131		FlowTableRecord_t *r;
1132		SortElement_t *topN_SortList;
1133		unsigned int i;
1134		uint32_t maxindex, c;
1135
1136		maxindex = ( FlowTable.NextBlock * FlowTable.Prealloc ) + FlowTable.NextElem;
1137		topN_SortList = (SortElement_t *)calloc(maxindex+1, sizeof(SortElement_t));
1138
1139		if ( !topN_SortList ) {
1140		perror("Can't allocate Top N lists: \n");
1141		return NULL;
1142		}
1143
1144		// preset topN_SortList table - still unsorted
1145		c = 0;
1146		// Iterate through all buckets
1147		for ( i=0; i <= FlowTable.IndexMask; i++ ) {
1148		r = FlowTable.bucket[i];
1149		// foreach elem in this bucket
1150		while ( r ) {
1151		// next elem in bucket
1152		topN_SortList[c].count = r->pkts;
1153		topN_SortList[c].record = (void *)r;
1154		r = r->next;
1155		c++;
1156		}
1157		}
1158		*count = c;
1159
1160		topN_SortList[maxindex].count = 0;
1161		heapSort(topN_SortList, maxindex, topN);
1162		return topN_SortList;
1163
1164		} // End of Make_TopN_packets
1165
1166		static SortElement_t Make_TopN_bytes(int topN, uint32_t count) {
1167		SortElement_t *topN_bytes_list;
1168		FlowTableRecord_t *r;
1169		unsigned int i;
1170		uint32_t maxindex, c;
1171
1172		maxindex = ( FlowTable.NextBlock * FlowTable.Prealloc ) + FlowTable.NextElem;
1173		topN_bytes_list = (SortElement_t *)calloc(maxindex+1, sizeof(SortElement_t));
1174
1175		if ( !topN_bytes_list ) {
1176		perror("Can't allocate Top N lists: \n");
1177		return NULL;
1178		}
1179
1180		// preset topN_SortList table - still unsorted
1181		c = 0;
1182		// Iterate through all buckets
1183		for ( i=0; i <= FlowTable.IndexMask; i++ ) {
1184		r = FlowTable.bucket[i];
1185		// foreach elem in this bucket
1186		while ( r ) {
1187		// next elem in bucket
1188		topN_bytes_list[c].count = r->pkts;
1189		topN_bytes_list[c].record = (void *)r;
1190		r = r->next;
1191		c++;
1192		}
1193		}
1194		*count = c;
1195
1196		topN_bytes_list[maxindex].count = 0;
1197		heapSort(topN_bytes_list, maxindex, topN);
1198		return topN_bytes_list;
1199
1200		} // End of Make_TopN_bytes
1201
1202		*/
1203	1316
1204	1317	static SortElement_t StatTopN(int topN, uint32_t count, int hash_num, int order ) {
1205	1318	SortElement_t *topN_list;

1208	1321	uint32_t c, maxindex;
1209	1322
1210	1323	maxindex = ( StatTable[hash_num].NextBlock * StatTable[hash_num].Prealloc ) + StatTable[hash_num].NextElem;
1211		topN_list = (SortElement_t *)calloc(maxindex+1, sizeof(SortElement_t)); // +1 for heapsort bug
	1324	topN_list = (SortElement_t *)calloc(maxindex, sizeof(SortElement_t));
1212	1325
1213	1326	if ( !topN_list ) {
1214	1327	perror("Can't allocate Top N lists: \n");

1253	1366	*count = c;
1254	1367	// printf ("Sort %u flows\n", c);
1255	1368
1256		topN_list[maxindex].count = 0;
	1369	/*
	1370	for ( i = 0; i < maxindex; i++ )
	1371	printf("%i, %llu %llu\n", i, topN_list[i].count, topN_list[i].record);
	1372	*/
1257	1373
1258	1374	// Sorting makes only sense, when 2 or more flows are left
1259	1375	if ( c >= 2 )
1260		heapSort(topN_list, c, topN > c ? c : topN);
1261
1262		/*
	1376	heapSort(topN_list, c, topN);
	1377
	1378	/*
1263	1379	for ( i = 0; i < maxindex; i++ )
1264	1380	printf("%i, %llu %llu\n", i, topN_list[i].count, topN_list[i].record);
1265		*/
	1381	*/
	1382
1266	1383	return topN_list;
1267	1384
1268	1385	} // End of StatTopN

1273	1390	uint64_t c1, c2;
1274	1391	int j;
1275	1392
1276		if ( val > (topN_list)[0].count ) {
	1393
	1394	if ( val >= (topN_list)[0].count ) {
1277	1395	/* element value is bigger than smallest value in topN */
1278	1396	c1 = val;
1279	1397	r1 = r;
1280	1398	for (j=topN-1; j>=0; j-- ) {
1281		if ( c1 > topN_list[j].count ) {
	1399	if ( c1 >= topN_list[j].count ) {
1282	1400	c2 = topN_list[j].count;
1283	1401	r2 = topN_list[j].record;
1284	1402	topN_list[j].count = c1;

1291	1409	} // End of RankValue
1292	1410
1293	1411	static void heapSort(SortElement_t *SortElement, uint32_t array_size, int topN) {
1294		int32_t i;
1295		uint32_t top_count;
1296		SortElement_t temp;
1297
1298		for (i = (array_size >> 1)-1; i >= 0; i--)
1299		siftDown(SortElement, i, array_size);
1300
1301		top_count = 1;
1302		for (i = array_size-1; i >= 1; i--) {
1303		temp = SortElement[0];
	1412	int32_t i, maxindex;
	1413
	1414	for(i = array_size - 1; i >= 0; i--)
	1415	siftDown(SortElement,array_size,i);
	1416
	1417	/*
	1418	* we are only interested in the first top N => skip sorting the rest
	1419	* For topN == 0 -> all flows gets sorted
	1420	*/
	1421	if ( (topN >= (array_size - 1)) \|\| topN == 0 )
	1422	maxindex = 0;
	1423	else
	1424	maxindex = array_size - 1 - topN;
	1425
	1426	for(i = array_size-1; i > maxindex; i-- ) {
	1427	SortElement_t temp = SortElement[0];
1304	1428	SortElement[0] = SortElement[i];
1305	1429	SortElement[i] = temp;
1306		siftDown(SortElement, 0, i-1);
1307		/*
1308		* if we need to know only the first top N skip
1309		* the sorting of the rest. For topN == 0 -> all gets sorted
1310		* as top_count is never 0
1311		*/
1312		if ( top_count == topN ) {
1313		return;
1314		}
1315		top_count++;
	1430	siftDown(SortElement,i,0);
1316	1431	}
1317	1432
1318	1433	} // End of heapSort
1319	1434
1320		static void siftDown(SortElement_t *SortElement, uint32_t root, uint32_t bottom) {
1321		uint32_t maxChild;
1322		SortElement_t temp;
1323		int done;
1324
1325		done = 0;
1326		while (((root << 1) <= bottom) && (!done)) {
1327		if ((root << 1) == bottom)
1328		maxChild = root << 1;
1329		else if (SortElement[root << 1].count > SortElement[(root << 1) + 1].count)
1330		maxChild = root << 1;
1331		else
1332		maxChild = (root << 1) + 1;
1333
1334		if (SortElement[root].count < SortElement[maxChild].count) {
1335		temp = SortElement[root];
1336		SortElement[root] = SortElement[maxChild];
1337		SortElement[maxChild] = temp;
1338		root = maxChild;
1339		} else
1340		done = 1;
1341		}
	1435	static inline void siftDown(SortElement_t *SortElement, uint32_t numbersSize, uint32_t node) {
	1436	uint32_t i, parent, child;
	1437
	1438	parent = node;
	1439	i = parent + 1;
	1440	while( i != parent ) {
	1441	i = parent;
	1442
	1443	// Compare with left child node
	1444	child = 2*i+1;
	1445	if( (child) < numbersSize && SortElement[child].count > SortElement[parent].count)
	1446	parent = child;
	1447
	1448	// Compare with right child node
	1449	child = 2*i+2;
	1450	if( (child) < numbersSize && SortElement[child].count > SortElement[parent].count)
	1451	parent = child;
	1452
	1453	if ( i != parent ) {
	1454	SortElement_t temp = SortElement[i];
	1455	SortElement[i] = SortElement[parent];
	1456	SortElement[parent] = temp;
	1457	}
	1458	}
1342	1459	} // End of siftDown
1343	1460

+25

-12

nfstat.h less more

29	29	*
30	30	* $Author: peter $
31	31	*
32		* $Id: nfstat.h 47 2005-08-25 12:58:27Z peter $
	32	* $Id: nfstat.h 60 2006-02-14 08:49:30Z peter $
33	33	*
34		* $LastChangedRevision: 47 $
	34	* $LastChangedRevision: 60 $
35	35	*
36	36	*/
37	37

72	72	uint16_t msec_last;
73	73
74	74	// more flow parameters
75		uint8_t pad1;
	75	uint8_t record_flags;
76	76	uint8_t tcp_flags;
77		uint8_t proto;
	77	uint8_t prot;
78	78	uint8_t tos;
79	79
	80	uint16_t srcas;
	81	uint16_t dstas;
	82	uint16_t input;
	83	uint16_t output;
	84
80	85	// elements used for hash generation
81		uint32_t ip1;
82		uint32_t ip2;
83		uint16_t port1;
84		uint16_t port2;
	86	uint16_t srcport;
	87	uint16_t dstport;
	88	ip_block_t ip;
85	89	} FlowTableRecord_t;
86	90
87	91	typedef struct hash_FlowTable {

117	121	uint32_t last;
118	122	uint16_t msec_first;
119	123	uint16_t msec_last;
	124	uint8_t record_flags;
	125	uint8_t tcp_flags;
	126	uint8_t tos;
120	127	// key
121		uint32_t stat_key;
	128	uint8_t prot;
	129	uint64_t stat_key[2];
122	130	} StatRecord_t;
123	131
124	132	typedef struct hash_StatTable {

144	152	uint64_t count;
145	153	} SortElement_t;
146	154
	155	#define Aggregate_SRCIP 1
	156	#define Aggregate_DSTIP 2
	157	#define Aggregate_SRCPORT 4
	158	#define Aggregate_DSTPORT 8
147	159
148	160	/* Function prototypes */
149	161	int Init_FlowTable(uint16_t NumBits, uint32_t Prealloc);

152	164
153	165	void Dispose_Tables(int flow_stat, int ip_stat);
154	166
	167	char *VerifyStat(uint16_t Aggregate_Bits);
	168
155	169	int SetStat(char str, int element_stat, int *flow_stat);
156	170
157	171	int SetStat_DefaultOrder(char *order);
158	172
159		void InsertFlow(flow_record_t *flow_record);
	173	void InsertFlow(master_record_t *flow_record);
160	174
161		int AddStat(flow_header_t flow_header, flow_record_t flow_record, int flow_stat, int element_stat);
	175	int AddStat(data_block_header_t flow_header, master_record_t flow_record, int flow_stat, int element_stat);
162	176
163	177	void ReportAggregated(printer_t print_record, uint32_t limitflows, int date_sorted, int anon);
164	178
165	179	void ReportStat(char *record_header, printer_t print_record, int topN, int flow_stat, int ip_stat, int anon);
166	180
167	181	void PrintSortedFlows(printer_t print_record, uint32_t limitflows, int anon);
168

+522

-0

nftest.c less more

	0	/*
	1	* This file is part of the nfdump project.
	2	*
	3	* Copyright (c) 2004, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
	4	* All rights reserved.
	5	*
	6	* Redistribution and use in source and binary forms, with or without
	7	* modification, are permitted provided that the following conditions are met:
	8	*
	9	* * Redistributions of source code must retain the above copyright notice,
	10	* this list of conditions and the following disclaimer.
	11	* * Redistributions in binary form must reproduce the above copyright notice,
	12	* this list of conditions and the following disclaimer in the documentation
	13	* and/or other materials provided with the distribution.
	14	* * Neither the name of SWITCH nor the names of its contributors may be
	15	* used to endorse or promote products derived from this software without
	16	* specific prior written permission.
	17	*
	18	* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
	19	* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	20	* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	21	* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
	22	* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
	23	* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
	24	* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
	25	* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
	26	* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
	27	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
	28	* POSSIBILITY OF SUCH DAMAGE.
	29	*
	30	* $Author: peter $
	31	*
	32	* $Id: nftest.c 70 2006-05-17 08:38:01Z peter $
	33	*
	34	* $LastChangedRevision: 70 $
	35	*
	36	*/
	37
	38	#include <stdio.h>
	39	#include <sys/types.h>
	40	#include <sys/socket.h>
	41	#include <netinet/in.h>
	42	#include <arpa/inet.h>
	43
	44	#include <string.h>
	45	#include <stdlib.h>
	46
	47	#include "config.h"
	48
	49	#ifdef HAVE_STDINT_H
	50	#include <stdint.h>
	51	#endif
	52
	53	#include "nfdump.h"
	54	#include "nftree.h"
	55	#include "nffile.h"
	56	#include "nf_common.h"
	57	#include "util.h"
	58
	59	/* Global Variables */
	60	extern char *CurrentIdent;
	61
	62	FilterEngine_data_t *Engine;
	63	uint32_t byte_limit, packet_limit;
	64	int byte_mode, packet_mode;
	65
	66	#define TCP 6
	67	#define UDP 17
	68
	69	int check_filter_block(char filter, master_record_t flow_record, int expect);
	70
	71	void check_offset(char *text, pointer_addr_t offset, pointer_addr_t expect);
	72
	73	int check_filter_block(char filter, master_record_t flow_record, int expect) {
	74	int ret, i;
	75	uint64_t block = (uint64_t )flow_record;
	76
	77	Engine = CompileFilter(filter);
	78	if ( !Engine ) {
	79	exit(254);
	80	}
	81
	82	Engine->nfrecord = (uint64_t *)flow_record;
	83	ret = (*Engine->FilterEngine)(Engine);
	84	if ( ret == expect ) {
	85	printf("Success: Startnode: %i Numblocks: %i Extended: %i Filter: '%s'\n", Engine->StartNode, nblocks(), Engine->Extended, filter);
	86	} else {
	87	printf("** FAILED ** Startnode: %i Numblocks: %i Extended: %i Filter: '%s'\n", Engine->StartNode, nblocks(), Engine->Extended, filter);
	88	DumpList(Engine);
	89	printf("Expected: %i, Found: %i\n", expect, ret);
	90	printf("Record:\n");
	91	for(i=0; i<=10; i++) {
	92	printf("%3i %.16llx\n", i, block[i]);
	93	}
	94	if ( Engine->IdentList ) {
	95	printf("Current Ident: %s, Ident 0 %s\n", CurrentIdent, Engine->IdentList[0]);
	96	}
	97	exit(255);
	98	}
	99	return (ret == expect);
	100	}
	101
	102	void check_offset(char *text, pointer_addr_t offset, pointer_addr_t expect) {
	103
	104	if ( offset == expect ) {
	105	printf("Success: %s: %u\n", text, expect);
	106	} else {
	107	printf("** FAILED ** %s expected %u, evaluated %u\n", text, expect, offset);
	108	// useless to continue
	109	exit(255);
	110	}
	111	}
	112
	113	int main(int argc, char **argv) {
	114	master_record_t flow_record;
	115	uint64_t *blocks, l;
	116	uint32_t size, in[2];
	117	time_t now;
	118	int ret;
	119	value64_t v;
	120
	121	if ( sizeof(struct in_addr) != sizeof(uint32_t) ) {
	122	#ifdef HAVE_SIZE_T_Z_FORMAT
	123	printf("** FAILED ** Size struct in_addr %zu != sizeof(uint32_t)\n", sizeof(struct in_addr));
	124	#else
	125	printf("** FAILED ** Size struct in_addr %lu != sizeof(uint32_t)\n", (unsigned long)sizeof(struct in_addr));
	126	#endif
	127	exit(255);
	128	}
	129
	130	l = 0x200000000LL;
	131	v.val.val64 = l;
	132	in[0] = v.val.val32[0];
	133	in[1] = v.val.val32[1];
	134	ret = memcmp(in, &l, sizeof(uint64_t));
	135	if ( ret != 0 ) {
	136	printf("** FAILED ** val32/64 union check failed!\n" );
	137	exit(255);
	138	}
	139	size = sizeof(common_record_t) - sizeof(uint8_t[4]);
	140	memset((void *)&flow_record, 0, sizeof(master_record_t));
	141	blocks = (uint64_t *)&flow_record;
	142
	143	check_offset("First Offset", (unsigned int)((pointer_addr_t)&flow_record.first - (pointer_addr_t)blocks), BYTE_OFFSET_first);
	144	check_offset("Common Offset", (unsigned int)((pointer_addr_t)&flow_record.fill - (pointer_addr_t)blocks), size);
	145	check_offset("Src AS Offset", (unsigned int)((pointer_addr_t)&flow_record.srcas - (pointer_addr_t)&blocks[OffsetAS]), 0);
	146	check_offset("Dst AS Offset", (unsigned int)((pointer_addr_t)&flow_record.dstas - (pointer_addr_t)&blocks[OffsetAS]), 2);
	147	check_offset("Src Port Offset", (unsigned int)((pointer_addr_t)&flow_record.srcport - (pointer_addr_t)&blocks[OffsetPort]), 4);
	148	check_offset("Dst Port Offset", (unsigned int)((pointer_addr_t)&flow_record.dstport - (pointer_addr_t)&blocks[OffsetPort]), 6);
	149	check_offset("Dir Offset", (unsigned int)((pointer_addr_t)&flow_record.dir - (pointer_addr_t)&blocks[OffsetDir]), 4);
	150	check_offset("Flags Offset", (unsigned int)((pointer_addr_t)&flow_record.tcp_flags - (pointer_addr_t)&blocks[OffsetFlags]), 5);
	151	check_offset("Protocol Offset", (unsigned int)((pointer_addr_t)&flow_record.prot - (pointer_addr_t)&blocks[OffsetProto]), 6);
	152	check_offset("tos Offset", (unsigned int)((pointer_addr_t)&flow_record.tos - (pointer_addr_t)&blocks[OffsetTos]), 7);
	153
	154	#ifdef HAVE_SIZE_T_Z_FORMAT
	155	printf("Pointer Size : %zu\n", sizeof(blocks));
	156	printf("Time_t Size : %zu\n", sizeof(now));
	157	printf("int Size : %zu\n", sizeof(int));
	158	printf("long Size : %zu\n", sizeof(long));
	159	#else
	160	printf("Pointer Size : %lu\n", (unsigned long)sizeof(blocks));
	161	printf("Time_t Size : %lu\n", (unsigned long)sizeof(now));
	162	printf("int Size : %lu\n", (unsigned long)sizeof(int));
	163	printf("long Size : %lu\n", (unsigned long)sizeof(long));
	164	#endif
	165
	166	flow_record.flags = 0;
	167	ret = check_filter_block("ipv4", &flow_record, 1);
	168	flow_record.flags = 2;
	169	ret = check_filter_block("ipv4", &flow_record, 1);
	170	ret = check_filter_block("ipv6", &flow_record, 0);
	171	flow_record.flags = 1;
	172	ret = check_filter_block("ipv4", &flow_record, 0);
	173	ret = check_filter_block("ipv6", &flow_record, 1);
	174	flow_record.flags = 7;
	175	ret = check_filter_block("ipv4", &flow_record, 0);
	176	ret = check_filter_block("ipv6", &flow_record, 1);
	177
	178
	179	flow_record.prot = TCP;
	180	ret = check_filter_block("any", &flow_record, 1);
	181	ret = check_filter_block("not any", &flow_record, 0);
	182	ret = check_filter_block("tcp", &flow_record, 1);
	183	ret = check_filter_block("proto tcp", &flow_record, 1);
	184	ret = check_filter_block("proto udp", &flow_record, 0);
	185	flow_record.prot = UDP;
	186	ret = check_filter_block("proto tcp", &flow_record, 0);
	187	ret = check_filter_block("proto udp", &flow_record, 1);
	188	flow_record.prot = 50;
	189	ret = check_filter_block("proto esp", &flow_record, 1);
	190	ret = check_filter_block("proto ah", &flow_record, 0);
	191	flow_record.prot = 51;
	192	ret = check_filter_block("proto ah", &flow_record, 1);
	193	flow_record.prot = 46;
	194	ret = check_filter_block("proto rsvp", &flow_record, 1);
	195	flow_record.prot = 47;
	196	ret = check_filter_block("proto gre", &flow_record, 1);
	197	ret = check_filter_block("proto 47", &flow_record, 1);
	198	ret = check_filter_block("proto 42", &flow_record, 0);
	199
	200	inet_pton(PF_INET6, "fe80::2110:abcd:1234:5678", flow_record.v6.srcaddr);
	201	inet_pton(PF_INET6, "fe80::1104:fedc:4321:8765", flow_record.v6.dstaddr);
	202	flow_record.v6.srcaddr[0] = ntohll(flow_record.v6.srcaddr[0]);
	203	flow_record.v6.srcaddr[1] = ntohll(flow_record.v6.srcaddr[1]);
	204	flow_record.v6.dstaddr[0] = ntohll(flow_record.v6.dstaddr[0]);
	205	flow_record.v6.dstaddr[1] = ntohll(flow_record.v6.dstaddr[1]);
	206	ret = check_filter_block("src ip fe80::2110:abcd:1234:5678", &flow_record, 1);
	207	ret = check_filter_block("src ip fe80::2110:abcd:1234:5679", &flow_record, 0);
	208	ret = check_filter_block("src ip fe80::2111:abcd:1234:5678", &flow_record, 0);
	209	ret = check_filter_block("dst ip fe80::1104:fedc:4321:8765", &flow_record, 1);
	210	ret = check_filter_block("dst ip fe80::1104:fedc:4321:8766", &flow_record, 0);
	211	ret = check_filter_block("dst ip fe80::1105:fedc:4321:8765", &flow_record, 0);
	212	ret = check_filter_block("ip fe80::2110:abcd:1234:5678", &flow_record, 1);
	213	ret = check_filter_block("ip fe80::1104:fedc:4321:8765", &flow_record, 1);
	214	ret = check_filter_block("ip fe80::2110:abcd:1234:5679", &flow_record, 0);
	215	ret = check_filter_block("ip fe80::1104:fedc:4321:8766", &flow_record, 0);
	216	ret = check_filter_block("not ip fe80::2110:abcd:1234:5678", &flow_record, 0);
	217	ret = check_filter_block("not ip fe80::2110:abcd:1234:5679", &flow_record, 1);
	218
	219	inet_pton(PF_INET6, "fe80::2110:abcd:1234:0", flow_record.v6.srcaddr);
	220	flow_record.v6.srcaddr[0] = ntohll(flow_record.v6.srcaddr[0]);
	221	flow_record.v6.srcaddr[1] = ntohll(flow_record.v6.srcaddr[1]);
	222	ret = check_filter_block("src net fe80::2110:abcd:1234:0/112", &flow_record, 1);
	223
	224	inet_pton(PF_INET6, "fe80::2110:abcd:1234:ffff", flow_record.v6.srcaddr);
	225	flow_record.v6.srcaddr[0] = ntohll(flow_record.v6.srcaddr[0]);
	226	flow_record.v6.srcaddr[1] = ntohll(flow_record.v6.srcaddr[1]);
	227	ret = check_filter_block("src net fe80::2110:abcd:1234:0/112", &flow_record, 1);
	228
	229	inet_pton(PF_INET6, "fe80::2110:abcd:1235:ffff", flow_record.v6.srcaddr);
	230	flow_record.v6.srcaddr[0] = ntohll(flow_record.v6.srcaddr[0]);
	231	flow_record.v6.srcaddr[1] = ntohll(flow_record.v6.srcaddr[1]);
	232	ret = check_filter_block("src net fe80::2110:abcd:1234:0/112", &flow_record, 0);
	233	ret = check_filter_block("src net fe80::0/16", &flow_record, 1);
	234	ret = check_filter_block("src net fe81::0/16", &flow_record, 0);
	235
	236	flow_record.v6.srcaddr[0] = 0;
	237	flow_record.v6.srcaddr[1] = 0;
	238
	239	inet_pton(PF_INET6, "fe80::2110:abcd:1234:0", flow_record.v6.dstaddr);
	240	flow_record.v6.dstaddr[0] = ntohll(flow_record.v6.dstaddr[0]);
	241	flow_record.v6.dstaddr[1] = ntohll(flow_record.v6.dstaddr[1]);
	242	ret = check_filter_block("dst net fe80::2110:abcd:1234:0/112", &flow_record, 1);
	243
	244	inet_pton(PF_INET6, "fe80::2110:abcd:1234:ffff", flow_record.v6.dstaddr);
	245	flow_record.v6.dstaddr[0] = ntohll(flow_record.v6.dstaddr[0]);
	246	flow_record.v6.dstaddr[1] = ntohll(flow_record.v6.dstaddr[1]);
	247	ret = check_filter_block("dst net fe80::2110:abcd:1234:0/112", &flow_record, 1);
	248
	249	inet_pton(PF_INET6, "fe80::2110:abcd:1235:ffff", flow_record.v6.dstaddr);
	250	flow_record.v6.dstaddr[0] = ntohll(flow_record.v6.dstaddr[0]);
	251	flow_record.v6.dstaddr[1] = ntohll(flow_record.v6.dstaddr[1]);
	252	ret = check_filter_block("dst net fe80::2110:abcd:1234:0/112", &flow_record, 0);
	253	ret = check_filter_block("dst net fe80::0/16", &flow_record, 1);
	254	ret = check_filter_block("not dst net fe80::0/16", &flow_record, 0);
	255	ret = check_filter_block("dst net fe81::0/16", &flow_record, 0);
	256	ret = check_filter_block("not dst net fe81::0/16", &flow_record, 1);
	257
	258
	259	/* 172.32.7.16 => 0xac200710
	260	* 10.10.10.11 => 0x0a0a0a0b
	261	*/
	262	flow_record.v6.srcaddr[0] = 0;
	263	flow_record.v6.srcaddr[1] = 0;
	264	flow_record.v6.dstaddr[0] = 0;
	265	flow_record.v6.dstaddr[1] = 0;
	266	flow_record.v4.srcaddr = 0xac200710;
	267	flow_record.v4.dstaddr = 0x0a0a0a0b;
	268	ret = check_filter_block("src ip 172.32.7.16", &flow_record, 1);
	269	ret = check_filter_block("src ip 172.32.7.15", &flow_record, 0);
	270	ret = check_filter_block("dst ip 10.10.10.11", &flow_record, 1);
	271	ret = check_filter_block("dst ip 10.10.10.10", &flow_record, 0);
	272	ret = check_filter_block("ip 172.32.7.16", &flow_record, 1);
	273	ret = check_filter_block("ip 10.10.10.11", &flow_record, 1);
	274	ret = check_filter_block("ip 172.32.7.17", &flow_record, 0);
	275	ret = check_filter_block("ip 10.10.10.12", &flow_record, 0);
	276	ret = check_filter_block("not ip 172.32.7.16", &flow_record, 0);
	277	ret = check_filter_block("not ip 172.32.7.17", &flow_record, 1);
	278
	279	ret = check_filter_block("src host 172.32.7.16", &flow_record, 1);
	280	ret = check_filter_block("src host 172.32.7.15", &flow_record, 0);
	281	ret = check_filter_block("dst host 10.10.10.11", &flow_record, 1);
	282	ret = check_filter_block("dst host 10.10.10.10", &flow_record, 0);
	283	ret = check_filter_block("host 172.32.7.16", &flow_record, 1);
	284	ret = check_filter_block("host 10.10.10.11", &flow_record, 1);
	285	ret = check_filter_block("host 172.32.7.17", &flow_record, 0);
	286	ret = check_filter_block("host 10.10.10.12", &flow_record, 0);
	287	ret = check_filter_block("not host 172.32.7.16", &flow_record, 0);
	288	ret = check_filter_block("not host 172.32.7.17", &flow_record, 1);
	289
	290	flow_record.srcport = 63;
	291	flow_record.dstport = 255;
	292	ret = check_filter_block("src port 63", &flow_record, 1);
	293	ret = check_filter_block("dst port 255", &flow_record, 1);
	294	ret = check_filter_block("port 63", &flow_record, 1);
	295	ret = check_filter_block("port 255", &flow_record, 1);
	296	ret = check_filter_block("src port 64", &flow_record, 0);
	297	ret = check_filter_block("dst port 258", &flow_record, 0);
	298	ret = check_filter_block("port 64", &flow_record, 0);
	299	ret = check_filter_block("port 258", &flow_record, 0);
	300
	301	ret = check_filter_block("src port = 63", &flow_record, 1);
	302	ret = check_filter_block("src port == 63", &flow_record, 1);
	303	ret = check_filter_block("src port eq 63", &flow_record, 1);
	304	ret = check_filter_block("src port > 62", &flow_record, 1);
	305	ret = check_filter_block("src port gt 62", &flow_record, 1);
	306	ret = check_filter_block("src port > 63", &flow_record, 0);
	307	ret = check_filter_block("src port < 64", &flow_record, 1);
	308	ret = check_filter_block("src port lt 64", &flow_record, 1);
	309	ret = check_filter_block("src port < 63", &flow_record, 0);
	310
	311	ret = check_filter_block("dst port = 255", &flow_record, 1);
	312	ret = check_filter_block("dst port == 255", &flow_record, 1);
	313	ret = check_filter_block("dst port eq 255", &flow_record, 1);
	314	ret = check_filter_block("dst port > 254", &flow_record, 1);
	315	ret = check_filter_block("dst port gt 254", &flow_record, 1);
	316	ret = check_filter_block("dst port > 255", &flow_record, 0);
	317	ret = check_filter_block("dst port < 256", &flow_record, 1);
	318	ret = check_filter_block("dst port lt 256", &flow_record, 1);
	319	ret = check_filter_block("dst port < 255", &flow_record, 0);
	320
	321	flow_record.srcas = 123;
	322	flow_record.dstas = 456;
	323	ret = check_filter_block("src as 123", &flow_record, 1);
	324	ret = check_filter_block("dst as 456", &flow_record, 1);
	325	ret = check_filter_block("as 123", &flow_record, 1);
	326	ret = check_filter_block("as 456", &flow_record, 1);
	327	ret = check_filter_block("src as 124", &flow_record, 0);
	328	ret = check_filter_block("dst as 457", &flow_record, 0);
	329	ret = check_filter_block("as 124", &flow_record, 0);
	330	ret = check_filter_block("as 457", &flow_record, 0);
	331
	332	ret = check_filter_block("src net 172.32/16", &flow_record, 1);
	333	ret = check_filter_block("src net 172.32.7/24", &flow_record, 1);
	334	ret = check_filter_block("src net 172.32.7.0/27", &flow_record, 1);
	335	ret = check_filter_block("src net 172.32.7.0/28", &flow_record, 0);
	336	ret = check_filter_block("src net 172.32.7.0 255.255.255.0", &flow_record, 1);
	337	ret = check_filter_block("src net 172.32.7.0 255.255.255.240", &flow_record, 0);
	338
	339	ret = check_filter_block("dst net 10.10/16", &flow_record, 1);
	340	ret = check_filter_block("dst net 10.10.10/24", &flow_record, 1);
	341	ret = check_filter_block("dst net 10.10.10.0/28", &flow_record, 1);
	342	ret = check_filter_block("dst net 10.10.10.0/29", &flow_record, 0);
	343	ret = check_filter_block("dst net 10.10.10.0 255.255.255.240", &flow_record, 1);
	344	ret = check_filter_block("dst net 10.10.10.0 255.255.255.248", &flow_record, 0);
	345
	346	ret = check_filter_block("net 172.32/16", &flow_record, 1);
	347	ret = check_filter_block("net 172.32.7/24", &flow_record, 1);
	348	ret = check_filter_block("net 172.32.7.0/27", &flow_record, 1);
	349	ret = check_filter_block("net 172.32.7.0/28", &flow_record, 0);
	350	ret = check_filter_block("net 172.32.7.0 255.255.255.0", &flow_record, 1);
	351	ret = check_filter_block("net 172.32.7.0 255.255.255.240", &flow_record, 0);
	352
	353	ret = check_filter_block("net 10.10/16", &flow_record, 1);
	354	ret = check_filter_block("net 10.10.10/24", &flow_record, 1);
	355	ret = check_filter_block("net 10.10.10.0/28", &flow_record, 1);
	356	ret = check_filter_block("net 10.10.10.0/29", &flow_record, 0);
	357	ret = check_filter_block("net 10.10.10.0 255.255.255.240", &flow_record, 1);
	358	ret = check_filter_block("net 10.10.10.0 255.255.255.240", &flow_record, 1);
	359	ret = check_filter_block("net 10.10.10.0 255.255.255.248", &flow_record, 0);
	360
	361	ret = check_filter_block("src ip 172.32.7.16 or src ip 172.32.7.15", &flow_record, 1);
	362	ret = check_filter_block("src ip 172.32.7.15 or src ip 172.32.7.16", &flow_record, 1);
	363	ret = check_filter_block("src ip 172.32.7.15 or src ip 172.32.7.14", &flow_record, 0);
	364	ret = check_filter_block("src ip 172.32.7.16 and dst ip 10.10.10.11", &flow_record, 1);
	365	ret = check_filter_block("src ip 172.32.7.15 and dst ip 10.10.10.11", &flow_record, 0);
	366	ret = check_filter_block("src ip 172.32.7.16 and dst ip 10.10.10.12", &flow_record, 0);
	367
	368	flow_record.tcp_flags = 1;
	369	ret = check_filter_block("flags F", &flow_record, 1);
	370	ret = check_filter_block("flags S", &flow_record, 0);
	371	ret = check_filter_block("flags R", &flow_record, 0);
	372	ret = check_filter_block("flags P", &flow_record, 0);
	373	ret = check_filter_block("flags A", &flow_record, 0);
	374	ret = check_filter_block("flags U", &flow_record, 0);
	375	ret = check_filter_block("flags X", &flow_record, 0);
	376
	377	flow_record.tcp_flags = 2; // flags S
	378	ret = check_filter_block("flags S", &flow_record, 1);
	379	flow_record.tcp_flags = 4;
	380	ret = check_filter_block("flags R", &flow_record, 1);
	381	flow_record.tcp_flags = 8;
	382	ret = check_filter_block("flags P", &flow_record, 1);
	383	flow_record.tcp_flags = 16;
	384	ret = check_filter_block("flags A", &flow_record, 1);
	385	flow_record.tcp_flags = 32;
	386	ret = check_filter_block("flags U", &flow_record, 1);
	387	flow_record.tcp_flags = 63;
	388	ret = check_filter_block("flags X", &flow_record, 1);
	389
	390	ret = check_filter_block("not flags RF", &flow_record, 0);
	391
	392	flow_record.tcp_flags = 3; // flags SF
	393	ret = check_filter_block("flags SF", &flow_record, 1);
	394	ret = check_filter_block("flags 3", &flow_record, 1);
	395	ret = check_filter_block("flags S and not flags AR", &flow_record, 1);
	396	flow_record.tcp_flags = 7;
	397	ret = check_filter_block("flags SF", &flow_record, 1);
	398	ret = check_filter_block("flags R", &flow_record, 1);
	399	ret = check_filter_block("flags P", &flow_record, 0);
	400	ret = check_filter_block("flags A", &flow_record, 0);
	401	ret = check_filter_block("flags = 7 ", &flow_record, 1);
	402	ret = check_filter_block("flags > 7 ", &flow_record, 0);
	403	ret = check_filter_block("flags > 6 ", &flow_record, 1);
	404	ret = check_filter_block("flags < 7 ", &flow_record, 0);
	405	ret = check_filter_block("flags < 8 ", &flow_record, 1);
	406
	407	flow_record.tos = 5;
	408	ret = check_filter_block("tos 5", &flow_record, 1);
	409	ret = check_filter_block("tos = 5", &flow_record, 1);
	410	ret = check_filter_block("tos > 5", &flow_record, 0);
	411	ret = check_filter_block("tos < 5", &flow_record, 0);
	412	ret = check_filter_block("tos > 4", &flow_record, 1);
	413	ret = check_filter_block("tos < 6", &flow_record, 1);
	414
	415	ret = check_filter_block("tos 10", &flow_record, 0);
	416
	417	flow_record.input = 5;
	418	ret = check_filter_block("in if 5", &flow_record, 1);
	419	ret = check_filter_block("in if 6", &flow_record, 0);
	420	ret = check_filter_block("out if 6", &flow_record, 0);
	421	flow_record.output = 6;
	422	ret = check_filter_block("out if 6", &flow_record, 1);
	423
	424	/*
	425	* 172.32.7.17 => 0xac200711
	426	*/
	427	flow_record.dPkts = 1000;
	428	ret = check_filter_block("packets 1000", &flow_record, 1);
	429	ret = check_filter_block("packets = 1000", &flow_record, 1);
	430	ret = check_filter_block("packets 1010", &flow_record, 0);
	431	ret = check_filter_block("packets < 1010", &flow_record, 1);
	432	ret = check_filter_block("packets > 110", &flow_record, 1);
	433
	434	flow_record.dOctets = 2000;
	435	ret = check_filter_block("bytes 2000", &flow_record, 1);
	436	ret = check_filter_block("bytes = 2000", &flow_record, 1);
	437	ret = check_filter_block("bytes 2010", &flow_record, 0);
	438	ret = check_filter_block("bytes < 2010", &flow_record, 1);
	439	ret = check_filter_block("bytes > 210", &flow_record, 1);
	440
	441	flow_record.dOctets = 2048;
	442	ret = check_filter_block("bytes 2k", &flow_record, 1);
	443	ret = check_filter_block("bytes < 2k", &flow_record, 0);
	444	ret = check_filter_block("bytes > 2k", &flow_record, 0);
	445	flow_record.dOctets *= 1024;
	446	ret = check_filter_block("bytes 2m", &flow_record, 1);
	447	ret = check_filter_block("bytes < 2m", &flow_record, 0);
	448	ret = check_filter_block("bytes > 2m", &flow_record, 0);
	449	flow_record.dOctets *= 1024;
	450	ret = check_filter_block("bytes 2g", &flow_record, 1);
	451	ret = check_filter_block("bytes < 2g", &flow_record, 0);
	452	ret = check_filter_block("bytes > 2g", &flow_record, 0);
	453
	454	/*
	455	* Function tests
	456	*/
	457	flow_record.first = 1089534600; /* 2004-07-11 10:30:00 */
	458	flow_record.last = 1089534600; /* 2004-07-11 10:30:00 */
	459	flow_record.msec_first = 10;
	460	flow_record.msec_last = 20;
	461
	462	/* duration 10ms */
	463	ret = check_filter_block("duration == 10", &flow_record, 1);
	464	ret = check_filter_block("duration < 11", &flow_record, 1);
	465	ret = check_filter_block("duration > 9", &flow_record, 1);
	466	ret = check_filter_block("not duration == 10", &flow_record, 0);
	467	ret = check_filter_block("duration > 10", &flow_record, 0);
	468	ret = check_filter_block("duration < 10", &flow_record, 0);
	469
	470	flow_record.first = 1089534600; /* 2004-07-11 10:30:00 */
	471	flow_record.last = 1089534610; /* 2004-07-11 10:30:10 */
	472	flow_record.msec_first = 0;
	473	flow_record.msec_last = 0;
	474
	475	/* duration 10s */
	476	flow_record.dPkts = 1000;
	477	ret = check_filter_block("duration == 10000", &flow_record, 1);
	478	ret = check_filter_block("duration < 10001", &flow_record, 1);
	479	ret = check_filter_block("duration > 9999", &flow_record, 1);
	480	ret = check_filter_block("not duration == 10000", &flow_record, 0);
	481	ret = check_filter_block("duration > 10000", &flow_record, 0);
	482	ret = check_filter_block("duration < 10000", &flow_record, 0);
	483
	484	ret = check_filter_block("pps == 100", &flow_record, 1);
	485	ret = check_filter_block("pps < 101", &flow_record, 1);
	486	ret = check_filter_block("pps > 99", &flow_record, 1);
	487	ret = check_filter_block("not pps == 100", &flow_record, 0);
	488	ret = check_filter_block("pps > 100", &flow_record, 0);
	489	ret = check_filter_block("pps < 100", &flow_record, 0);
	490
	491	flow_record.dOctets = 1000;
	492	ret = check_filter_block("bps == 800", &flow_record, 1);
	493	ret = check_filter_block("bps < 801", &flow_record, 1);
	494	ret = check_filter_block("bps > 799", &flow_record, 1);
	495	ret = check_filter_block("not bps == 800", &flow_record, 0);
	496	ret = check_filter_block("bps > 800", &flow_record, 0);
	497	ret = check_filter_block("bps < 800", &flow_record, 0);
	498
	499	flow_record.dOctets = 20000;
	500	ret = check_filter_block("bps > 1k", &flow_record, 1);
	501	ret = check_filter_block("bps > 15k", &flow_record, 1);
	502	ret = check_filter_block("bps > 16k", &flow_record, 0);
	503
	504	ret = check_filter_block("bpp == 20", &flow_record, 1);
	505	ret = check_filter_block("bpp < 21", &flow_record, 1);
	506	ret = check_filter_block("bpp > 19", &flow_record, 1);
	507	ret = check_filter_block("not bpp == 20", &flow_record, 0);
	508	ret = check_filter_block("bpp > 20", &flow_record, 0);
	509	ret = check_filter_block("bpp < 20", &flow_record, 0);
	510
	511	// ident checks
	512	CurrentIdent = "channel1";
	513	ret = check_filter_block("ident channel1", &flow_record, 1);
	514	ret = check_filter_block("ident channel", &flow_record, 0);
	515	ret = check_filter_block("ident channel11", &flow_record, 0);
	516	ret = check_filter_block("not ident channel1", &flow_record, 0);
	517	ret = check_filter_block("ident none", &flow_record, 0);
	518	ret = check_filter_block("not ident none", &flow_record, 1);
	519
	520	return 0;
	521	}

+91

-34

nftree.c less more

29	29	*
30	30	* $Author: peter $
31	31	*
32		* $Id: nftree.c 48 2005-08-26 08:23:31Z peter $
	32	* $Id: nftree.c 70 2006-05-17 08:38:01Z peter $
33	33	*
34		* $LastChangedRevision: 48 $
	34	* $LastChangedRevision: 70 $
35	35	*
36	36	*/
37	37

39	39	#include <stdlib.h>
40	40	#include <sys/types.h>
41	41	#include <string.h>
	42	#include <errno.h>
42	43
43	44	#include "config.h"
44	45

47	48	#endif
48	49
49	50	#include "nfdump.h"
	51	#include "nffile.h"
50	52	#include "nf_common.h"
51	53	#include "nftree.h"
52	54	#include "grammar.h"

55	57	*
56	58	*/
57	59
	60	extern char *CurrentIdent;
	61
58	62	#define MAXBLOCKS 1024
59	63
60	64	static FilterBlock_t *FilterTree;

62	66
63	67	static uint32_t NumBlocks = 1; /* index 0 reserved */
64	68
	69	#define IdentNumBlockSize 32
	70	static uint16_t MaxIdents;
	71	static uint16_t NumIdents;
	72	static char **IdentList;
	73
65	74	static void UpdateList(uint32_t a, uint32_t b);
66	75
67	76	/* flow processing functions */
68		static inline uint32_t pps_function(uint32_t *data);
69		static inline uint32_t bps_function(uint32_t *data);
70		static inline uint32_t bpp_function(uint32_t *data);
71		static inline uint32_t duration_function(uint32_t *data);
	77	static inline uint64_t pps_function(uint64_t *data);
	78	static inline uint64_t bps_function(uint64_t *data);
	79	static inline uint64_t bpp_function(uint64_t *data);
	80	static inline uint64_t duration_function(uint64_t *data);
72	81
73	82	/*
74	83	* flow processing function table:

93	102	memblocks = 1;
94	103	FilterTree = (FilterBlock_t )malloc(MAXBLOCKS sizeof(FilterBlock_t));
95	104	if ( !FilterTree ) {
96		perror("Memory error: ");
	105	fprintf(stderr, "Memory allocation error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
97	106	exit(255);
98	107	}
99	108	ClearFilter();

106	115
107	116	NumBlocks = 1;
108	117	Extended = 0;
	118	MaxIdents = 0;
	119	NumIdents = 0;
	120	IdentList = NULL;
109	121	memset((void )FilterTree, 0, MAXBLOCKS sizeof(FilterBlock_t));
110	122
111	123	} /* End of ClearFilter */

126	138	}
127	139	engine = malloc(sizeof(FilterEngine_data_t));
128	140	if ( !engine ) {
129		perror("Memory error: ");
	141	fprintf(stderr, "Memory allocation error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
130	142	exit(255);
131	143	}
132	144	engine->nfrecord = NULL;
133	145	engine->StartNode = StartNode;
134	146	engine->Extended = Extended;
	147	engine->IdentList = IdentList;
135	148	engine->filter = FilterTree;
136	149	if ( Extended )
137	150	engine->FilterEngine = RunExtendedFilter;

152	165	/*
153	166	* Returns next free slot in blocklist
154	167	*/
155		uint32_t NewBlock(uint32_t offset, uint32_t mask, uint32_t value, uint16_t comp, uint32_t function) {
	168	uint32_t NewBlock(uint32_t offset, uint64_t mask, uint64_t value, uint16_t comp, uint32_t function) {
156	169	uint32_t n = NumBlocks;
157	170
158	171	if ( n >= ( memblocks * MAXBLOCKS ) ) {
159	172	memblocks++;
160	173	FilterTree = realloc(FilterTree, memblocks * MAXBLOCKS * sizeof(FilterBlock_t));
161	174	if ( !FilterTree ) {
162		perror("Memory error: ");
	175	fprintf(stderr, "Memory allocation error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
163	176	exit(255);
164	177	}
165	178	}

277	290	s = FilterTree[a].numblocks + FilterTree[b].numblocks;
278	291	FilterTree[a].blocklist = (uint32_t )realloc(FilterTree[a].blocklist, s sizeof(uint32_t));
279	292	if ( !FilterTree[a].blocklist ) {
280		perror("Memory error: ");
	293	fprintf(stderr, "Memory allocation error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
281	294	exit(250);
282	295	}
283	296

309	322
310	323	for (i=1; i<NumBlocks; i++ ) {
311	324	if ( args->filter[i].invert )
312		printf("Index: %u, Offset: %u, Mask: %x, Value: %x, Superblock: %u, Numblocks: %u, !OnTrue: %u, !OnFalse: %u Comp: %u Function: %s\n",
	325	printf("Index: %u, Offset: %u, Mask: %.16llx, Value: %.16llx, Superblock: %u, Numblocks: %u, !OnTrue: %u, !OnFalse: %u Comp: %u Function: %s\n",
313	326	i, args->filter[i].offset, args->filter[i].mask, args->filter[i].value, args->filter[i].superblock,
314	327	args->filter[i].numblocks, args->filter[i].OnTrue, args->filter[i].OnFalse, args->filter[i].comp, args->filter[i].fname);
315	328	else
316		printf("Index: %u, Offset: %u, Mask: %x, Value: %x, Superblock: %u, Numblocks: %u, OnTrue: %u, OnFalse: %u Comp: %u Function: %s\n",
	329	printf("Index: %u, Offset: %u, Mask: %.16llx, Value: %.16llx, Superblock: %u, Numblocks: %u, OnTrue: %u, OnFalse: %u Comp: %u Function: %s\n",
317	330	i, args->filter[i].offset, args->filter[i].mask, args->filter[i].value, args->filter[i].superblock,
318	331	args->filter[i].numblocks, args->filter[i].OnTrue, args->filter[i].OnFalse, args->filter[i].comp, args->filter[i].fname);
319	332	printf("\tBlocks: ");

345	358
346	359	/* extended filter engine */
347	360	int RunExtendedFilter(FilterEngine_data_t *args) {
348		uint32_t index, offset, value;
	361	uint32_t index, offset;
	362	uint64_t value;
349	363	int evaluate, invert;
350	364
351	365	index = args->StartNode;

366	380	evaluate = value > args->filter[index].value;
367	381	else if ( args->filter[index].comp == CMP_LT )
368	382	evaluate = value < args->filter[index].value;
	383	else if ( args->filter[index].comp == CMP_IDENT )
	384	evaluate = strncmp(CurrentIdent, args->IdentList[value], IdentLen) == 0 ;
	385	else if ( args->filter[index].comp == CMP_FLAGS ) {
	386	if ( invert )
	387	evaluate = value > 0;
	388	else
	389	evaluate = value == args->filter[index].value;
	390	}
369	391
370	392	index = evaluate ? args->filter[index].OnTrue : args->filter[index].OnFalse;
371	393	}

373	395
374	396	} /* End of RunExtendedFilter */
375	397
	398	uint32_t AddIdent(char *Ident) {
	399	uint32_t num;
	400
	401	if ( MaxIdents == 0 ) {
	402	// allocate first array block
	403	MaxIdents = IdentNumBlockSize;
	404	IdentList = (char *)malloc( MaxIdents sizeof(char *));
	405	if ( !IdentList ) {
	406	fprintf(stderr, "Memory allocation error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
	407	exit(254);
	408	}
	409	memset((void )IdentList, 0, MaxIdents sizeof(char *));
	410	NumIdents = 0;
	411	} else if ( NumIdents == MaxIdents ) {
	412	// extend array block
	413	MaxIdents += IdentNumBlockSize;
	414	IdentList = realloc((void )IdentList, MaxIdents sizeof(char *));
	415	if ( !IdentList ) {
	416	fprintf(stderr, "Memory allocation error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
	417	exit(254);
	418	}
	419	}
	420
	421	num = NumIdents++;
	422	IdentList[num] = strdup(Ident);
	423	if ( !IdentList[num] ) {
	424	fprintf(stderr, "Memory allocation error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
	425	exit(254);
	426	}
	427
	428	return num;
	429
	430	} // End of AddIdent
	431
376	432	/* record processing functions */
377	433
378		static inline uint32_t duration_function(uint32_t *data) {
379		flow_record_t *record;
	434	static inline uint64_t duration_function(uint64_t *data) {
	435	master_record_t *record;
380	436	uint64_t duration;
381	437
382		record = (flow_record_t *)data;
	438	record = (master_record_t *)data;
383	439	/* duration in msec */
384		duration = 1000*(record->Last - record->First) + record->msec_last - record->msec_first;
385
386		return (uint32_t)duration;
	440	duration = 1000*(record->last - record->first) + record->msec_last - record->msec_first;
	441
	442	return duration;
387	443
388	444	} // End of duration_function
389	445
390		static inline uint32_t pps_function(uint32_t *data) {
391		flow_record_t *record;
	446	static inline uint64_t pps_function(uint64_t *data) {
	447	master_record_t *record;
392	448	uint64_t duration;
393	449
394		record = (flow_record_t *)data;
	450	record = (master_record_t *)data;
395	451	/* duration in msec */
396		duration = 1000*(record->Last - record->First) + record->msec_last - record->msec_first;
	452	duration = 1000*(record->last - record->first) + record->msec_last - record->msec_first;
397	453	if ( duration == 0 )
398	454	return 0;
399	455	else
400		return ( 1000LL * (uint64_t)record->dPkts ) / duration;
	456	return ( 1000LL * record->dPkts ) / duration;
401	457
402	458	} // End of pps_function
403	459
404		static inline uint32_t bps_function(uint32_t *data) {
405		flow_record_t *record;
	460	static inline uint64_t bps_function(uint64_t *data) {
	461	master_record_t *record;
406	462	uint64_t duration;
407	463
408		record = (flow_record_t *)data;
	464	record = (master_record_t *)data;
409	465	/* duration in msec */
410		duration = 1000*(record->Last - record->First) + record->msec_last - record->msec_first;
	466	duration = 1000*(record->last - record->first) + record->msec_last - record->msec_first;
411	467	if ( duration == 0 )
412	468	return 0;
413	469	else
414		return ( 8000LL * (uint64_t)record->dOctets ) / duration; /* 8 bits per Octet - x 1000 for msec */
	470	return ( 8000LL * record->dOctets ) / duration; /* 8 bits per Octet - x 1000 for msec */
415	471
416	472	} // End of bps_function
417	473
418		static inline uint32_t bpp_function(uint32_t *data) {
419		flow_record_t *record;
420
421		record = (flow_record_t *)data;
	474	static inline uint64_t bpp_function(uint64_t *data) {
	475	master_record_t *record;
	476
	477	record = (master_record_t *)data;
422	478	return record->dPkts ? record->dOctets / record->dPkts : 0;
423	479
424	480	} // End of bpp_function
425	481
	482

+15

-9

nftree.h less more

29	29	*
30	30	* $Author: peter $
31	31	*
32		* $Id: nftree.h 34 2005-08-22 12:01:31Z peter $
	32	* $Id: nftree.h 64 2006-03-20 07:41:15Z peter $
33	33	*
34		* $LastChangedRevision: 34 $
	34	* $LastChangedRevision: 64 $
35	35	*
36	36	*/
37	37

40	40	* type definitions for nf tree
41	41	*/
42	42
43		typedef uint32_t (flow_proc_t)(uint32_t );
	43	typedef uint64_t (flow_proc_t)(uint64_t );
44	44
45	45	typedef struct FilterBlock {
46	46	/* Filter specific data */
47	47	uint32_t offset;
48		uint32_t mask;
49		uint32_t value;
	48	uint64_t mask;
	49	uint64_t value;
50	50
51	51	/* Internal block info for tree setup */
52	52	uint32_t superblock; /* Index of superblock */

63	63	typedef struct FilterEngine_data_s {
64	64	FilterBlock_t *filter;
65	65	uint32_t StartNode;
66		uint16_t Extended;
67		uint32_t *nfrecord;
	66	uint32_t Extended;
	67	char **IdentList;
	68	uint64_t *nfrecord;
68	69	int (FilterEngine)(struct FilterEngine_data_s );
69	70	} FilterEngine_data_t;
70	71
71	72	/*
72	73	* Definitions
73	74	*/
74		enum { CMP_EQ = 0, CMP_GT, CMP_LT };
	75	enum { CMP_EQ = 0, CMP_GT, CMP_LT, CMP_IDENT, CMP_FLAGS };
75	76
76	77	/*
77	78	* filter functions:

116	117	/*
117	118	* Returns next free slot in blocklist
118	119	*/
119		uint32_t NewBlock(uint32_t offset, uint32_t mask, uint32_t value, uint16_t comp, uint32_t function);
	120	uint32_t NewBlock(uint32_t offset, uint64_t mask, uint64_t value, uint16_t comp, uint32_t function);
120	121
121	122	/*
122	123	* Connects the to blocks b1 and b2 ( AND ) and returns index of superblock

133	134	*/
134	135	uint32_t Invert(uint32_t a );
135	136
	137	/*
	138	* Add Ident to Identlist
	139	*/
	140	uint32_t AddIdent(char *Ident);
	141
136	142	/*
137	143	* Dump Filterlist
138	144	*/

-395

~~nftree_check.c~~ less more

0		/*
1		* This file is part of the nfdump project.
2		*
3		* Copyright (c) 2004, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
4		* All rights reserved.
5		*
6		* Redistribution and use in source and binary forms, with or without
7		* modification, are permitted provided that the following conditions are met:
8		*
9		* * Redistributions of source code must retain the above copyright notice,
10		* this list of conditions and the following disclaimer.
11		* * Redistributions in binary form must reproduce the above copyright notice,
12		* this list of conditions and the following disclaimer in the documentation
13		* and/or other materials provided with the distribution.
14		* * Neither the name of SWITCH nor the names of its contributors may be
15		* used to endorse or promote products derived from this software without
16		* specific prior written permission.
17		*
18		* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
19		* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
20		* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
21		* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
22		* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
23		* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
24		* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
25		* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
26		* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
27		* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
28		* POSSIBILITY OF SUCH DAMAGE.
29		*
30		* $Author: peter $
31		*
32		* $Id: nftree_check.c 51 2005-08-26 10:58:13Z peter $
33		*
34		* $LastChangedRevision: 51 $
35		*
36		*/
37
38		#include <stdio.h>
39		#include <sys/types.h>
40		#include <string.h>
41		#include <stdlib.h>
42
43		#include "config.h"
44
45		#ifdef HAVE_STDINT_H
46		#include <stdint.h>
47		#endif
48
49		#include "nfdump.h"
50		#include "nftree.h"
51		#include "nf_common.h"
52
53		/* Global Variables */
54		FilterEngine_data_t *Engine;
55		uint32_t byte_limit, packet_limit;
56		int byte_mode, packet_mode, failed;
57
58		#define TCP 6
59		#define UDP 17
60
61		int check(char filter, struct flow_record flow_record, int expect);
62
63		int check(char filter, struct flow_record flow_record, int expect) {
64		int ret;
65
66		Engine = CompileFilter(filter);
67		if ( !Engine ) {
68		exit(254);
69		}
70
71		Engine->nfrecord = (uint32_t *)flow_record;
72		ret = (*Engine->FilterEngine)(Engine);
73		if ( ret == expect ) {
74		printf("Success: Startnode: %i Numblocks: %i Extended: %i Filter: '%s'\n", Engine->StartNode, nblocks(), Engine->Extended, filter);
75		} else {
76		printf("** FAILED ** Startnode: %i Numblocks: %i Extended: %i Filter: '%s'\n", Engine->StartNode, nblocks(), Engine->Extended, filter);
77		DumpList(Engine);
78		printf("Expected: %i, Found: %i\n", expect, ret);
79		failed = 1;
80		}
81		return (ret == expect);
82		}
83
84		int main(int argc, char **argv) {
85		struct flow_record flow_record;
86		uint32_t *blocks;
87		time_t now;
88		int ret;
89
90		failed = 0;
91		memset((void *)&flow_record, 0, FLOW_RECORD_LENGTH);
92		blocks = (uint32_t *)&flow_record;
93
94		printf("Src AS Offset: %u\n", (unsigned int)((pointer_addr_t)&flow_record.src_as - (pointer_addr_t)&blocks[10]));
95		printf("Dst AS Offset: %u\n", (unsigned int)((pointer_addr_t)&flow_record.dst_as - (pointer_addr_t)&blocks[10]));
96		printf("Src Port Offset: %u\n", (unsigned int)((pointer_addr_t)&flow_record.srcport - (pointer_addr_t)&blocks[8]));
97		printf("Dst Port Offset: %u\n", (unsigned int)((pointer_addr_t)&flow_record.dstport - (pointer_addr_t)&blocks[8]));
98		printf("Protocol Offset: %u\n", (unsigned int)((pointer_addr_t)&flow_record.prot - (pointer_addr_t)&blocks[9]));
99
100		#if defined __OpenBSD__ \|\| defined __FreeBSD__
101		printf("Pointer Size : %u\n", sizeof(blocks));
102		printf("Time_t Size : %u\n", sizeof(now));
103		printf("int Size : %u\n", sizeof(int));
104		printf("long Size : %u\n", sizeof(long));
105		if ( FLOW_HEADER_LENGTH != 24 ) {
106		printf("** FAILED ** Header length reported %u, expected 24\n", FLOW_HEADER_LENGTH);
107		}
108		if ( FLOW_RECORD_LENGTH != 48 ) {
109		printf("** FAILED ** Record length reported %u, expected 48\n", FLOW_RECORD_LENGTH);
110		}
111		#else
112		printf("Pointer Size : %u\n", sizeof(blocks));
113		printf("Time_t Size : %u\n", sizeof(now));
114		printf("int Size : %u\n", sizeof(int));
115		printf("long Size : %u\n", sizeof(long));
116		if ( FLOW_HEADER_LENGTH != 24 ) {
117		printf("** FAILED ** Header length reported %u, expected 24\n", FLOW_HEADER_LENGTH);
118		}
119		if ( FLOW_RECORD_LENGTH != 48 ) {
120		printf("** FAILED ** Record length reported %u, expected 48\n", FLOW_RECORD_LENGTH);
121		}
122		#endif
123
124		flow_record.prot = TCP;
125		ret = check("any", &flow_record, 1);
126		ret = check("not any", &flow_record, 0);
127		ret = check("tcp", &flow_record, 1);
128		ret = check("udp", &flow_record, 0);
129		flow_record.prot = UDP;
130		ret = check("tcp", &flow_record, 0);
131		ret = check("udp", &flow_record, 1);
132		flow_record.prot = 50;
133		ret = check("esp", &flow_record, 1);
134		ret = check("ah", &flow_record, 0);
135		flow_record.prot = 51;
136		ret = check("ah", &flow_record, 1);
137		flow_record.prot = 46;
138		ret = check("rsvp", &flow_record, 1);
139		flow_record.prot = 47;
140		ret = check("gre", &flow_record, 1);
141		ret = check("proto 47", &flow_record, 1);
142		ret = check("proto 42", &flow_record, 0);
143
144		/* 172.32.7.16 => 0xac200710
145		* 10.10.10.11 => 0x0a0a0a0b
146		*/
147		flow_record.srcaddr = 0xac200710;
148		flow_record.dstaddr = 0x0a0a0a0b;
149		ret = check("src ip 172.32.7.16", &flow_record, 1);
150		ret = check("src ip 172.32.7.15", &flow_record, 0);
151		ret = check("dst ip 10.10.10.11", &flow_record, 1);
152		ret = check("dst ip 10.10.10.10", &flow_record, 0);
153		ret = check("ip 172.32.7.16", &flow_record, 1);
154		ret = check("ip 10.10.10.11", &flow_record, 1);
155		ret = check("ip 172.32.7.17", &flow_record, 0);
156		ret = check("ip 10.10.10.12", &flow_record, 0);
157		ret = check("not ip 172.32.7.16", &flow_record, 0);
158		ret = check("not ip 172.32.7.17", &flow_record, 1);
159
160		ret = check("src host 172.32.7.16", &flow_record, 1);
161		ret = check("src host 172.32.7.15", &flow_record, 0);
162		ret = check("dst host 10.10.10.11", &flow_record, 1);
163		ret = check("dst host 10.10.10.10", &flow_record, 0);
164		ret = check("host 172.32.7.16", &flow_record, 1);
165		ret = check("host 10.10.10.11", &flow_record, 1);
166		ret = check("host 172.32.7.17", &flow_record, 0);
167		ret = check("host 10.10.10.12", &flow_record, 0);
168		ret = check("not host 172.32.7.16", &flow_record, 0);
169		ret = check("not host 172.32.7.17", &flow_record, 1);
170
171		flow_record.srcport = 63;
172		flow_record.dstport = 255;
173		ret = check("src port 63", &flow_record, 1);
174		ret = check("dst port 255", &flow_record, 1);
175		ret = check("port 63", &flow_record, 1);
176		ret = check("port 255", &flow_record, 1);
177		ret = check("src port 64", &flow_record, 0);
178		ret = check("dst port 258", &flow_record, 0);
179		ret = check("port 64", &flow_record, 0);
180		ret = check("port 258", &flow_record, 0);
181
182		ret = check("src port = 63", &flow_record, 1);
183		ret = check("src port == 63", &flow_record, 1);
184		ret = check("src port eq 63", &flow_record, 1);
185		ret = check("src port > 62", &flow_record, 1);
186		ret = check("src port gt 62", &flow_record, 1);
187		ret = check("src port > 63", &flow_record, 0);
188		ret = check("src port < 64", &flow_record, 1);
189		ret = check("src port lt 64", &flow_record, 1);
190		ret = check("src port < 63", &flow_record, 0);
191
192		ret = check("dst port = 255", &flow_record, 1);
193		ret = check("dst port == 255", &flow_record, 1);
194		ret = check("dst port eq 255", &flow_record, 1);
195		ret = check("dst port > 254", &flow_record, 1);
196		ret = check("dst port gt 254", &flow_record, 1);
197		ret = check("dst port > 255", &flow_record, 0);
198		ret = check("dst port < 256", &flow_record, 1);
199		ret = check("dst port lt 256", &flow_record, 1);
200		ret = check("dst port < 255", &flow_record, 0);
201
202		flow_record.src_as = 123;
203		flow_record.dst_as = 456;
204		ret = check("src as 123", &flow_record, 1);
205		ret = check("dst as 456", &flow_record, 1);
206		ret = check("as 123", &flow_record, 1);
207		ret = check("as 456", &flow_record, 1);
208		ret = check("src as 124", &flow_record, 0);
209		ret = check("dst as 457", &flow_record, 0);
210		ret = check("as 124", &flow_record, 0);
211		ret = check("as 457", &flow_record, 0);
212
213		ret = check("src net 172.32/16", &flow_record, 1);
214		ret = check("src net 172.32.7/24", &flow_record, 1);
215		ret = check("src net 172.32.7/27", &flow_record, 1);
216		ret = check("src net 172.32.7/28", &flow_record, 0);
217		ret = check("src net 172.32.7.0 255.255.255.0", &flow_record, 1);
218		ret = check("src net 172.32.7.0 255.255.255.240", &flow_record, 0);
219
220		ret = check("dst net 10.10/16", &flow_record, 1);
221		ret = check("dst net 10.10.10/24", &flow_record, 1);
222		ret = check("dst net 10.10.10/28", &flow_record, 1);
223		ret = check("dst net 10.10.10/29", &flow_record, 0);
224		ret = check("dst net 10.10.10.0 255.255.255.240", &flow_record, 1);
225		ret = check("dst net 10.10.10.0 255.255.255.248", &flow_record, 0);
226
227		ret = check("net 172.32/16", &flow_record, 1);
228		ret = check("net 172.32.7/24", &flow_record, 1);
229		ret = check("net 172.32.7/27", &flow_record, 1);
230		ret = check("net 172.32.7/28", &flow_record, 0);
231		ret = check("net 172.32.7.0 255.255.255.0", &flow_record, 1);
232		ret = check("net 172.32.7.0 255.255.255.240", &flow_record, 0);
233
234		ret = check("net 10.10/16", &flow_record, 1);
235		ret = check("net 10.10.10/24", &flow_record, 1);
236		ret = check("net 10.10.10/28", &flow_record, 1);
237		ret = check("net 10.10.10/29", &flow_record, 0);
238		ret = check("net 10.10.10.0 255.255.255.240", &flow_record, 1);
239		ret = check("net 10.10.10.0 255.255.255.240", &flow_record, 1);
240		ret = check("net 10.10.10.0 255.255.255.248", &flow_record, 0);
241
242		ret = check("src ip 172.32.7.16 or src ip 172.32.7.15", &flow_record, 1);
243		ret = check("src ip 172.32.7.15 or src ip 172.32.7.16", &flow_record, 1);
244		ret = check("src ip 172.32.7.15 or src ip 172.32.7.14", &flow_record, 0);
245		ret = check("src ip 172.32.7.16 and dst ip 10.10.10.11", &flow_record, 1);
246		ret = check("src ip 172.32.7.15 and dst ip 10.10.10.11", &flow_record, 0);
247		ret = check("src ip 172.32.7.16 and dst ip 10.10.10.12", &flow_record, 0);
248
249		flow_record.tcp_flags = 1;
250		ret = check("flags F", &flow_record, 1);
251		ret = check("flags S", &flow_record, 0);
252		ret = check("flags R", &flow_record, 0);
253		ret = check("flags P", &flow_record, 0);
254		ret = check("flags A", &flow_record, 0);
255		ret = check("flags U", &flow_record, 0);
256		ret = check("flags X", &flow_record, 0);
257
258		flow_record.tcp_flags = 2;
259		ret = check("flags S", &flow_record, 1);
260		flow_record.tcp_flags = 4;
261		ret = check("flags R", &flow_record, 1);
262		flow_record.tcp_flags = 8;
263		ret = check("flags P", &flow_record, 1);
264		flow_record.tcp_flags = 16;
265		ret = check("flags A", &flow_record, 1);
266		flow_record.tcp_flags = 32;
267		ret = check("flags U", &flow_record, 1);
268		flow_record.tcp_flags = 63;
269		ret = check("flags X", &flow_record, 1);
270
271		flow_record.tcp_flags = 3;
272		ret = check("flags SF", &flow_record, 1);
273		ret = check("flags 3", &flow_record, 1);
274		flow_record.tcp_flags = 7;
275		ret = check("flags SF", &flow_record, 1);
276		ret = check("flags R", &flow_record, 1);
277		ret = check("flags P", &flow_record, 0);
278		ret = check("flags A", &flow_record, 0);
279		ret = check("flags = 7 ", &flow_record, 1);
280		ret = check("flags > 7 ", &flow_record, 0);
281		ret = check("flags > 6 ", &flow_record, 1);
282		ret = check("flags < 7 ", &flow_record, 0);
283		ret = check("flags < 8 ", &flow_record, 1);
284
285		flow_record.tos = 5;
286		ret = check("tos 5", &flow_record, 1);
287		ret = check("tos = 5", &flow_record, 1);
288		ret = check("tos > 5", &flow_record, 0);
289		ret = check("tos < 5", &flow_record, 0);
290		ret = check("tos > 4", &flow_record, 1);
291		ret = check("tos < 6", &flow_record, 1);
292
293		ret = check("tos 10", &flow_record, 0);
294
295		flow_record.input = 5;
296		ret = check("in if 5", &flow_record, 1);
297		ret = check("in if 6", &flow_record, 0);
298		ret = check("out if 6", &flow_record, 0);
299		flow_record.output = 6;
300		ret = check("out if 6", &flow_record, 1);
301
302		/*
303		* 172.32.7.17 => 0xac200711
304		*/
305		flow_record.nexthop = 0xac200711;
306		ret = check("next 172.32.7.17", &flow_record, 1);
307		ret = check("next 172.32.7.16", &flow_record, 0);
308
309		flow_record.dPkts = 1000;
310		ret = check("packets 1000", &flow_record, 1);
311		ret = check("packets = 1000", &flow_record, 1);
312		ret = check("packets 1010", &flow_record, 0);
313		ret = check("packets < 1010", &flow_record, 1);
314		ret = check("packets > 110", &flow_record, 1);
315
316		flow_record.dOctets = 2000;
317		ret = check("bytes 2000", &flow_record, 1);
318		ret = check("bytes = 2000", &flow_record, 1);
319		ret = check("bytes 2010", &flow_record, 0);
320		ret = check("bytes < 2010", &flow_record, 1);
321		ret = check("bytes > 210", &flow_record, 1);
322
323		flow_record.dOctets = 2048;
324		ret = check("bytes 2k", &flow_record, 1);
325		ret = check("bytes < 2k", &flow_record, 0);
326		ret = check("bytes > 2k", &flow_record, 0);
327		flow_record.dOctets *= 1024;
328		ret = check("bytes 2m", &flow_record, 1);
329		ret = check("bytes < 2m", &flow_record, 0);
330		ret = check("bytes > 2m", &flow_record, 0);
331		flow_record.dOctets *= 1024;
332		ret = check("bytes 2g", &flow_record, 1);
333		ret = check("bytes < 2g", &flow_record, 0);
334		ret = check("bytes > 2g", &flow_record, 0);
335
336		/*
337		* Function tests
338		*/
339		flow_record.First = 1089534600; /* 2004-07-11 10:30:00 */
340		flow_record.Last = 1089534600; /* 2004-07-11 10:30:01 */
341		flow_record.msec_first = 10;
342		flow_record.msec_last = 20;
343
344		/* duration 10ms */
345		ret = check("duration == 10", &flow_record, 1);
346		ret = check("duration < 11", &flow_record, 1);
347		ret = check("duration > 9", &flow_record, 1);
348		ret = check("not duration == 10", &flow_record, 0);
349		ret = check("duration > 10", &flow_record, 0);
350		ret = check("duration < 10", &flow_record, 0);
351
352		flow_record.First = 1089534600; /* 2004-07-11 10:30:00 */
353		flow_record.Last = 1089534610; /* 2004-07-11 10:30:01 */
354		flow_record.msec_first = 0;
355		flow_record.msec_last = 0;
356
357		/* duration 10s */
358		flow_record.dPkts = 1000;
359		ret = check("duration == 10000", &flow_record, 1);
360		ret = check("duration < 10001", &flow_record, 1);
361		ret = check("duration > 9999", &flow_record, 1);
362		ret = check("not duration == 10000", &flow_record, 0);
363		ret = check("duration > 10000", &flow_record, 0);
364		ret = check("duration < 10000", &flow_record, 0);
365
366		ret = check("pps == 100", &flow_record, 1);
367		ret = check("pps < 101", &flow_record, 1);
368		ret = check("pps > 99", &flow_record, 1);
369		ret = check("not pps == 100", &flow_record, 0);
370		ret = check("pps > 100", &flow_record, 0);
371		ret = check("pps < 100", &flow_record, 0);
372
373		flow_record.dOctets = 1000;
374		ret = check("bps == 800", &flow_record, 1);
375		ret = check("bps < 801", &flow_record, 1);
376		ret = check("bps > 799", &flow_record, 1);
377		ret = check("not bps == 800", &flow_record, 0);
378		ret = check("bps > 800", &flow_record, 0);
379		ret = check("bps < 800", &flow_record, 0);
380
381		flow_record.dOctets = 20000;
382		ret = check("bps > 1k", &flow_record, 1);
383		ret = check("bps > 15k", &flow_record, 1);
384		ret = check("bps > 16k", &flow_record, 0);
385
386		ret = check("bpp == 20", &flow_record, 1);
387		ret = check("bpp < 21", &flow_record, 1);
388		ret = check("bpp > 19", &flow_record, 1);
389		ret = check("not bpp == 20", &flow_record, 0);
390		ret = check("bpp > 20", &flow_record, 0);
391		ret = check("bpp < 20", &flow_record, 0);
392
393		return failed;
394		}

+45

-2

panonymizer.c less more

30	30	*
31	31	* $Author: peter $
32	32	*
33		* $Id: panonymizer.c 53 2005-11-17 07:45:34Z peter $
	33	* $Id: panonymizer.c 70 2006-05-17 08:38:01Z peter $
34	34	*
35		* $LastChangedRevision: 53 $
	35	* $LastChangedRevision: 70 $
36	36	*
37	37	*/
38	38

145	145	//XOR the orginal address with the pseudorandom one-time-pad
146	146	return result ^ orig_addr;
147	147	}
	148
	149	/* little endian CPU's are boring! - but give it a try
	150	* orig_addr is a ptr to memory, return by inet_pton for IPv6
	151	* anon_addr return the result in the same order
	152	*/
	153	void anonymize_v6(const uint64_t orig_addr[2], uint64_t *anon_addr) {
	154	uint8_t rin_output[16], orig_bytes, result;
	155	uint8_t rin_input[16];
	156
	157	int pos, i, bit_num, left_byte;
	158
	159	anon_addr[0] = anon_addr[1] = 0;
	160	result = (uint8_t *)anon_addr;
	161	orig_bytes = (uint8_t *)orig_addr;
	162
	163	// For each prefixes with length from 0 to 127, generate a bit using the Rijndael cipher,
	164	// which is used as a pseudorandom function here. The bits generated in every rounds
	165	// are combineed into a pseudorandom one-time-pad.
	166	for (pos = 0; pos <= 127 ; pos++) {
	167	bit_num = pos & 0x7;
	168	left_byte = (pos >> 3);
	169
	170	for ( i=0; i<left_byte; i++ ) {
	171	rin_input[i] = orig_bytes[i];
	172	}
	173	rin_input[left_byte] = orig_bytes[left_byte] >> (7-bit_num) << (7-bit_num) \| (m_pad[left_byte]<<bit_num) >> bit_num;
	174	for ( i=left_byte+1; i<16; i++ ) {
	175	rin_input[i] = m_pad[i];
	176	}
	177
	178	//Encryption: The Rijndael cipher is used as pseudorandom function. During each
	179	//round, only the first bit of rin_output is used.
	180	Rijndael_blockEncrypt(rin_input, 128, rin_output);
	181
	182	//Combination: the bits are combined into a pseudorandom one-time-pad
	183	result[left_byte] \|= (rin_output[0] >> 7) << bit_num;
	184
	185	}
	186	//XOR the orginal address with the pseudorandom one-time-pad
	187	anon_addr[0] ^= orig_addr[0];
	188	anon_addr[1] ^= orig_addr[1];
	189
	190	}

-2

panonymizer.h less more

30	30	*
31	31	* $Author: peter $
32	32	*
33		* $Id: panonymizer.h 53 2005-11-17 07:45:34Z peter $
	33	* $Id: panonymizer.h 55 2006-01-13 10:04:34Z peter $
34	34	*
35		* $LastChangedRevision: 53 $
	35	* $LastChangedRevision: 55 $
36	36	*
37	37	*/
38	38

89	89
90	90	uint32_t anonymize( const uint32_t orig_addr);
91	91
	92	void anonymize_v6(const uint64_t orig_addr[2], uint64_t *anon_addr);
	93
92	94	#endif //_PANONYMIZER_H_

+180

-105

profile.c less more

31	31	*
32	32	* $Author: peter $
33	33	*
34		* $Id: profile.c 17 2005-03-04 09:06:48Z peter $
	34	* $Id: profile.c 70 2006-05-17 08:38:01Z peter $
35	35	*
36		* $LastChangedRevision: 17 $
	36	* $LastChangedRevision: 70 $
37	37	*
38	38	*/
39	39

55	55	#endif
56	56
57	57	#include "nfdump.h"
	58	#include "nffile.h"
58	59	#include "util.h"
59	60	#include "nftree.h"
60	61	#include "profile.h"
61	62
62		static profileinfo_t *profile;
63		static unsigned int num_profiles;
	63	static profile_channel_info_t *profile_channel;
	64	static unsigned int num_channels;
64	65
65	66	static void SetupProfile(char profiledir, char profilename, char subdir, char filterfile, char *filename, int veryfy_only, int quiet );
66	67
67		profileinfo_t *GetProfiles(void) {
68		return profile;
	68	static void SetupProfileChannels(char profiledir, char profilename, char filterfile, char filename, int veryfy_only, int quiet );
	69
	70	profile_channel_info_t *GetProfiles(void) {
	71	return profile_channel;
69	72	} // End of GetProfiles
70	73
71	74	int InitProfiles(char profiledir, char subdir, char filterfile, char filename, int veryfy_only, int quiet ) {

74	77	struct stat stat_buf;
75	78	char stringbuf[1024];
76	79
77		profile = NULL;
78		num_profiles = 0;
	80	profile_channel = NULL;
	81	num_channels = 0;
79	82	PDIR = opendir(profiledir);
80	83	if ( !PDIR ) {
81		perror("Can't read profiledir: ");
	84	fprintf(stderr, "Can't read profiledir '%s': %s\n",profiledir, strerror(errno) );
82	85	return 0;
83	86	}
84	87
85	88	while ( ( entry = readdir(PDIR)) != NULL ) {
86	89	snprintf(stringbuf, 1023, "%s/%s", profiledir, entry->d_name);
87	90	if ( stat(stringbuf, &stat_buf) ) {
88		perror("Can't stat entry: ");
	91	fprintf(stderr, "Can't stat '%s': %s\n",stringbuf, strerror(errno) );
89	92	continue;
90	93	}
91	94	if ( !S_ISDIR(stat_buf.st_mode) )

95	98	if ( entry->d_name[0] == '.' )
96	99	continue;
97	100
98		SetupProfile(profiledir, entry->d_name, subdir, filterfile, filename, veryfy_only, quiet);
	101	if ( subdir )
	102	SetupProfile(profiledir, entry->d_name, subdir, filterfile, filename, veryfy_only, quiet);
	103	else
	104	SetupProfileChannels(profiledir, entry->d_name, filterfile, filename, veryfy_only, quiet);
99	105	}
100	106	closedir(PDIR);
101	107
102		return num_profiles;
	108	return num_channels;
103	109
104	110	} // End of InitProfiles
105	111

107	113	FilterEngine_data_t *engine;
108	114	struct stat stat_buf;
109	115	char *filter;
110		char stringbuf[1024];
	116	char stringbuf[1024], *string;
111	117	int ffd, wfd, ret;
112	118
113	119	// check if subdir exists if defined
114		if ( subdir ) {
115		snprintf(stringbuf, 1023, "%s/%s/%s", profiledir, profilename, subdir);
116		if ( stat(stringbuf, &stat_buf) ) {
117		if ( !quiet )
118		fprintf(stderr, "Skipping profile '%s'\n", profilename);
119		return;
120		}
121		if ( !S_ISDIR(stat_buf.st_mode) )
122		return;
123		}
	120	snprintf(stringbuf, 1023, "%s/%s/%s", profiledir, profilename, subdir);
	121	if ( stat(stringbuf, &stat_buf) ) {
	122	if ( !quiet )
	123	fprintf(stderr, "Skipping profile '%s'\n", profilename);
	124	return;
	125	}
	126	if ( !S_ISDIR(stat_buf.st_mode) )
	127	return;
124	128
125	129
126	130	// Try to read filter

134	138	// stringbuf contains filter file
135	139	filter = (char *)malloc(stat_buf.st_size+1);
136	140	if ( !filter ) {
137		perror("Memory error: ");
	141	fprintf(stderr, "Memory allocation error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
138	142	exit(255);
139	143	}
140	144	ffd = open(stringbuf, O_RDONLY);
141	145	if ( ffd < 0 ) {
142		fprintf(stderr, "Error opening file '%s': %s\n",stringbuf, strerror(errno) );
143		perror("Can't open file");
	146	fprintf(stderr, "Can't open file '%s' for reading: %s\n",stringbuf, strerror(errno) );
144	147	return;
145	148	}
146	149
147	150	ret = read(ffd, (void *)filter, stat_buf.st_size);
148	151	if ( ret < 0 ) {
149		perror("Error reading file");
	152	fprintf(stderr, "Can't read from file '%s': %s\n",stringbuf, strerror(errno) );
150	153	close(ffd);
151	154	return;
152	155	}

154	157	filter[stat_buf.st_size] = 0;
155	158
156	159	if ( !quiet )
157		printf("Setup Profile %s\n", profilename);
	160	printf("Setup Profile %s static channel %s\n", profilename, subdir);
158	161	// compile profile filter
159	162	if ( veryfy_only && !quiet )
160		printf("Check profile '%s': ", profilename);
	163	printf("Check profile %s static channel %s: ", profilename, subdir);
161	164
162	165	engine = CompileFilter(filter);
163	166	free(filter);

173	176	}
174	177
175	178	// prepare output file
176		if ( subdir )
177		snprintf(stringbuf, 1023, "%s/%s/%s/%s", profiledir, profilename, subdir, filename);
178		else
179		snprintf(stringbuf, 1023, "%s/%s/%s", profiledir, profilename, filename);
180
181		wfd = open(stringbuf, O_CREAT \| O_RDWR \| O_TRUNC, S_IRUSR \| S_IWUSR \| S_IRGRP \| S_IROTH );
	179	snprintf(stringbuf, 1023, "%s/%s/%s/%s", profiledir, profilename, subdir, filename);
	180
	181	wfd = OpenNewFile(stringbuf, &string);
182	182
183	183	if ( wfd < 0 ) {
184		fprintf(stderr, "Error opening file '%s': %s\n",stringbuf, strerror(errno) );
185		perror("Can't open file for writing");
	184	if ( string != NULL )
	185	fprintf(stderr, "%s\n", string);
	186	return;
	187	}
	188
	189	if ( wfd < 0 ) {
	190	fprintf(stderr, "Can't open file '%s' for writing: %s\n",stringbuf, strerror(errno) );
186	191	return;
187	192	}
188	193
189	194	// collect all profile info
190		num_profiles++;
191		profile = realloc(profile, num_profiles * sizeof(profileinfo_t) );
192		if ( !profile ) {
193		perror("Memory error: ");
	195	num_channels++;
	196	profile_channel = realloc(profile_channel, num_channels * sizeof(profile_channel_info_t) );
	197	if ( !profile_channel ) {
	198	fprintf(stderr, "Memory allocation error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
194	199	exit(255);
195	200	}
196	201
197		memset(&profile[num_profiles-1], 0, sizeof(profileinfo_t));
198
199		profile[num_profiles-1].engine = engine;
200		profile[num_profiles-1].name = strdup(profilename);
201		profile[num_profiles-1].wfile = strdup(stringbuf);
202		profile[num_profiles-1].wfd = wfd;
203		profile[num_profiles-1].first_seen = 0xffffffff;
204		profile[num_profiles-1].last_seen = 0;
	202	memset(&profile_channel[num_channels-1], 0, sizeof(profile_channel_info_t));
	203
	204	profile_channel[num_channels-1].engine = engine;
	205	profile_channel[num_channels-1].profile = strdup(profilename);
	206	profile_channel[num_channels-1].channel = strdup(subdir);
	207	profile_channel[num_channels-1].wfile = strdup(stringbuf);
	208	profile_channel[num_channels-1].wfd = wfd;
	209	memset((void *)&profile_channel[num_channels-1].stat_record, 0, sizeof(stat_record_t));
	210	profile_channel[num_channels-1].stat_record.first_seen = 0xffffffff;
	211	profile_channel[num_channels-1].stat_record.last_seen = 0;
205	212
206	213	return;
207	214
208		} // End of InitProfiles
209
	215	} // End of SetupProfile
	216
	217
	218	static void SetupProfileChannels(char profiledir, char profilename, char filterfile, char filename, int veryfy_only, int quiet ) {
	219	DIR *PDIR;
	220	struct dirent *entry;
	221	FilterEngine_data_t *engine;
	222	struct stat stat_buf;
	223	char *filter;
	224	char stringbuf[1024], *string;
	225	int ffd, wfd, ret;
	226
	227	snprintf(stringbuf, 1023, "%s/%s", profiledir, profilename);
	228
	229	PDIR = opendir(stringbuf);
	230	if ( !PDIR ) {
	231	fprintf(stderr, "Can't open directory '%s': %s\n",stringbuf, strerror(errno) );
	232	return;
	233	}
	234
	235	while ( ( entry = readdir(PDIR)) != NULL ) {
	236	snprintf(stringbuf, 1023, "%s/%s/%s", profiledir, profilename, entry->d_name);
	237
	238	if ( stat(stringbuf, &stat_buf) ) {
	239	fprintf(stderr, "Can't stat directory entry '%s': %s\n",stringbuf, strerror(errno) );
	240	continue;
	241	}
	242	if ( !S_ISDIR(stat_buf.st_mode) )
	243	continue;
	244
	245	// skip all '.' entries -> make .anything invisible to nfprofile
	246	if ( entry->d_name[0] == '.' )
	247	continue;
	248
	249	// Try to read filter
	250	snprintf(stringbuf, 1023, "%s/%s/%s/%s", profiledir, profilename, entry->d_name, filterfile);
	251	if ( stat(stringbuf, &stat_buf) \|\| !S_ISREG(stat_buf.st_mode) ) {
	252	if ( !quiet )
	253	fprintf(stderr, "Skipping channel %s in profile '%s'\n", entry->d_name, profilename);
	254	continue;
	255	}
	256
	257	// stringbuf contains filter filename
	258	filter = (char *)malloc(stat_buf.st_size+1);
	259	if ( !filter ) {
	260	fprintf(stderr, "Memory allocation error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
	261	exit(255);
	262	}
	263	ffd = open(stringbuf, O_RDONLY);
	264	if ( ffd < 0 ) {
	265	fprintf(stderr, "Can't open file '%s' for reading: %s\n",stringbuf, strerror(errno) );
	266	return;
	267	}
	268
	269	ret = read(ffd, (void *)filter, stat_buf.st_size);
	270	if ( ret < 0 ) {
	271	fprintf(stderr, "Can't read from file '%s': %s\n",stringbuf, strerror(errno) );
	272	close(ffd);
	273	continue;
	274	}
	275	close(ffd);
	276	filter[stat_buf.st_size] = 0;
	277
	278	if ( !quiet )
	279	printf("Setup profile %s channel %s \n", profilename, entry->d_name);
	280
	281	// compile profile filter
	282	if ( veryfy_only && !quiet )
	283	printf("Check profile %s channel '%s': ", profilename, entry->d_name);
	284
	285	engine = CompileFilter(filter);
	286	free(filter);
	287
	288	if ( !engine ) {
	289	printf("\n");
	290	exit(254);
	291	}
	292
	293	if ( veryfy_only && !quiet ) {
	294	printf("Done.\n");
	295	continue;
	296	}
	297
	298	// prepare output file
	299	snprintf(stringbuf, 1023, "%s/%s/%s/%s", profiledir, profilename, entry->d_name, filename);
	300
	301	wfd = OpenNewFile(stringbuf, &string);
	302
	303	if ( wfd < 0 ) {
	304	if ( string != NULL )
	305	fprintf(stderr, "%s\n", string);
	306	continue;
	307	}
	308
	309	// collect all channel info
	310	num_channels++;
	311	profile_channel = realloc(profile_channel, num_channels * sizeof(profile_channel_info_t) );
	312	if ( !profile_channel ) {
	313	fprintf(stderr, "Memory allocation error in %s line %d: %s\n", __FILE__, __LINE__, strerror(errno) );
	314	exit(255);
	315	}
	316
	317	memset(&profile_channel[num_channels-1], 0, sizeof(profile_channel_info_t));
	318
	319	profile_channel[num_channels-1].engine = engine;
	320	profile_channel[num_channels-1].profile = strdup(profilename);
	321	profile_channel[num_channels-1].channel = strdup(entry->d_name);
	322	profile_channel[num_channels-1].wfile = strdup(stringbuf);
	323	profile_channel[num_channels-1].wfd = wfd;
	324	memset((void *)&profile_channel[num_channels-1].stat_record, 0, sizeof(stat_record_t));
	325	profile_channel[num_channels-1].stat_record.first_seen = 0xffffffff;
	326	profile_channel[num_channels-1].stat_record.last_seen = 0;
	327
	328	}
	329
	330	return;
	331
	332	} // End of SetupProfileChannels
210	333
211	334	void CloseProfiles (void) {
212		char sfile[255], tmpstring[64];
213	335	unsigned int num;
214		int fd;
215
216		for ( num = 0; num < num_profiles; num++ ) {
217
218		close(profile[num].wfd);
219
220		snprintf(sfile, 254, "%s.stat", profile[num].wfile);
221		fd = open(sfile, O_CREAT \| O_RDWR \| O_TRUNC, S_IRUSR \| S_IWUSR \| S_IRGRP \| S_IROTH );
222		if ( fd == -1 ) {
223		perror("Can't open stat file: ");
224		continue;
225		}
226
227		snprintf(tmpstring, 64, "Time: %u\n", GetStatTime());
228		write(fd, tmpstring, strlen(tmpstring));
229		snprintf(tmpstring, 64, "Ident: %s\n", GetIdent());
230		write(fd, tmpstring, strlen(tmpstring));
231		snprintf(tmpstring, 64, "Flows: %llu\n", profile[num].numflows);
232		write(fd, tmpstring, strlen(tmpstring));
233		snprintf(tmpstring, 64, "Flows_tcp: %llu\n", profile[num].numflows_tcp);
234		write(fd, tmpstring, strlen(tmpstring));
235		snprintf(tmpstring, 64, "Flows_udp: %llu\n", profile[num].numflows_udp);
236		write(fd, tmpstring, strlen(tmpstring));
237		snprintf(tmpstring, 64, "Flows_icmp: %llu\n", profile[num].numflows_icmp);
238		write(fd, tmpstring, strlen(tmpstring));
239		snprintf(tmpstring, 64, "Flows_other: %llu\n", profile[num].numflows_other);
240		write(fd, tmpstring, strlen(tmpstring));
241		snprintf(tmpstring, 64, "Packets: %llu\n", profile[num].numpackets);
242		write(fd, tmpstring, strlen(tmpstring));
243		snprintf(tmpstring, 64, "Packets_tcp: %llu\n", profile[num].numpackets_tcp);
244		write(fd, tmpstring, strlen(tmpstring));
245		snprintf(tmpstring, 64, "Packets_udp: %llu\n", profile[num].numpackets_udp);
246		write(fd, tmpstring, strlen(tmpstring));
247		snprintf(tmpstring, 64, "Packets_icmp: %llu\n", profile[num].numpackets_icmp);
248		write(fd, tmpstring, strlen(tmpstring));
249		snprintf(tmpstring, 64, "Packets_other: %llu\n", profile[num].numpackets_other);
250		write(fd, tmpstring, strlen(tmpstring));
251		snprintf(tmpstring, 64, "Bytes: %llu\n", profile[num].numbytes);
252		write(fd, tmpstring, strlen(tmpstring));
253		snprintf(tmpstring, 64, "Bytes_tcp: %llu\n", profile[num].numbytes_tcp);
254		write(fd, tmpstring, strlen(tmpstring));
255		snprintf(tmpstring, 64, "Bytes_udp: %llu\n", profile[num].numbytes_udp);
256		write(fd, tmpstring, strlen(tmpstring));
257		snprintf(tmpstring, 64, "Bytes_icmp: %llu\n", profile[num].numbytes_icmp);
258		write(fd, tmpstring, strlen(tmpstring));
259		snprintf(tmpstring, 64, "Bytes_other: %llu\n", profile[num].numbytes_other);
260		write(fd, tmpstring, strlen(tmpstring));
261		snprintf(tmpstring, 64, "First: %u\n", profile[num].first_seen);
262		write(fd, tmpstring, strlen(tmpstring));
263		snprintf(tmpstring, 64, "Last: %u\n", profile[num].last_seen);
264		write(fd, tmpstring, strlen(tmpstring));
265
266		close(fd);
	336	char *s;
	337
	338	for ( num = 0; num < num_channels; num++ ) {
	339	CloseUpdateFile(profile_channel[num].wfd, &(profile_channel[num].stat_record), profile_channel[num].file_blocks, GetIdent(), &s );
	340	if ( s != NULL )
	341	fprintf(stderr, "%s\n", s);
267	342	}
268	343
269	344	} // End of CloseProfiles

+13

-14

profile.h less more

32	32	*
33	33	* $Author: peter $
34	34	*
35		* $Id: profile.h 17 2005-03-04 09:06:48Z peter $
	35	* $Id: profile.h 55 2006-01-13 10:04:34Z peter $
36	36	*
37		* $LastChangedRevision: 17 $
	37	* $LastChangedRevision: 55 $
38	38	*
39	39	*/
40	40
41		typedef struct profileinfo_s {
	41	typedef struct profile_channel_info_s {
42	42	FilterEngine_data_t *engine;
43		char *name;
44		char *wfile;
45		uint64_t numflows, numbytes, numpackets;
46		uint32_t first_seen, last_seen;
47		uint64_t numflows_tcp, numflows_udp, numflows_icmp, numflows_other;
48		uint64_t numbytes_tcp, numbytes_udp, numbytes_icmp, numbytes_other;
49		uint64_t numpackets_tcp, numpackets_udp, numpackets_icmp, numpackets_other;
50		uint16_t cnt;
51		int wfd;
52		} profileinfo_t;
	43	char *profile;
	44	char *channel;
	45	char *wfile;
	46	data_block_header_t *flow_header;
	47	void *writeto;
	48	stat_record_t stat_record;
	49	uint32_t file_blocks;
	50	int wfd;
	51	} profile_channel_info_t;
53	52
54		profileinfo_t *GetProfiles(void);
	53	profile_channel_info_t *GetProfiles(void);
55	54
56	55	int InitProfiles(char profiledir, char subdir, char filterfile, char filename, int veryfy_only, int quiet );
57	56

+10

-12

rijndael.c less more

29	29	*
30	30	* $Nfdump Author:$
31	31	*
32		* $Id: rijndael.c 53 2005-11-17 07:45:34Z peter $
	32	* $Id: rijndael.c 70 2006-05-17 08:38:01Z peter $
33	33	*
34		* $LastChangedRevision: 53 $
	34	* $LastChangedRevision: 70 $
35	35	*
36	36	*/
37	37

108	108	#include <stdio.h>
109	109	#include <stdlib.h>
110	110	#include <sys/types.h>
	111	#include <string.h>
111	112
112	113	#include "config.h"
113	114
114	115	#ifdef HAVE_STDINT_H
115	116	#include <stdint.h>
116	117	#endif
117
118		#include <string.h>
119
120	118
121	119	#include "rijndael.h"
122	120

1038	1036	// Internal vars
1039	1037	enum State { Valid , Invalid };
1040	1038
1041		static uint8_t m_state = Invalid;
1042		static uint8_t m_mode;
1043		static uint8_t m_direction;
1044		static uint8_t m_initVector[MAX_IV_SIZE];
1045		static uint32_t m_uRounds;
1046		static uint8_t m_expandedKey[_MAX_ROUNDS+1][4][4];
	1039	static uint8_t m_state = Invalid;
	1040	static uint8_t m_mode;
	1041	static uint8_t m_direction;
	1042	static uint8_t m_initVector[MAX_IV_SIZE];
	1043	static uint32_t m_uRounds;
	1044	static uint8_t m_expandedKey[_MAX_ROUNDS+1][4][4];
1047	1045
1048	1046	static void keySched(uint8_t key[_MAX_KEY_COLUMNS][4]);
1049	1047
1050		static void keyEncToDec();
	1048	static void keyEncToDec(void);
1051	1049
1052	1050	static void encrypt(const uint8_t a[16], uint8_t b[16]);
1053	1051

-4

scandir.c less more

6	6	have this function in their system libraries */
7	7
8	8	int scandir(const char dir, struct dirent **namelist,
9		const int (select)(struct dirent ),
10		const int (compar)(const void , const void *))
	9	int (select)(struct dirent ),
	10	int (compar)(const void , const void *))
11	11	{
12	12	DIR *d;
13	13	struct dirent *entry;

25	25	namelist=(struct dirent )realloc((void )(*namelist),
26	26	(size_t)((i+1)sizeof(struct dirent )));
27	27	if (*namelist == NULL) return(-1);
28		entrysize=sizeof(struct
29		dirent)-sizeof(entry->d_name)+strlen(entry->d_name)+1;
	28	entrysize=sizeof(struct dirent)-sizeof(entry->d_name)+strlen(entry->d_name)+1;
30	29	(namelist)[i]=(struct dirent )malloc(entrysize);
31	30	if ((*namelist)[i] == NULL) return(-1);
32	31	memcpy((*namelist)[i], entry, entrysize);

-2

scandir.h less more

0	0	int scandir(const char dir, struct dirent **namelist,
1		const int (select)(const struct dirent ),
2		const int (compar)(const void , const void *));
	1	int (select)(const struct dirent ),
	2	int (compar)(const void , const void *));
3	3
4	4	int alphasort(const void a, const void b);

+36

-25

scanner.l less more

29	29	*
30	30	* $Author: peter $
31	31	*
32		* $Id: scanner.l 34 2005-08-22 12:01:31Z peter $
	32	* $Id: scanner.l 75 2006-05-21 15:32:48Z peter $
33	33	*
34		* $LastChangedRevision: 34 $
	34	* $LastChangedRevision: 75 $
35	35	*
36	36	*
37	37	*

40	40	%{
41	41
42	42	#include <sys/types.h>
	43	#include <stdio.h>
43	44	#include <stdlib.h>
44	45	#include <string.h>
45	46

48	49	#ifdef HAVE_STDINT_H
49	50	#include <stdint.h>
50	51	#endif
51
52	52
53	53	#include "nfdump.h"
54	54	#include "grammar.h"

64	64	#define getc(fp) (in_buffer == 0 ? EOF : in_buffer++)
65	65	#endif
66	66
	67	static void ProtoDepreciated(char *s);
	68
67	69	%}
68	70
69	71	N [0-9]+

71	73
72	74	%%
73	75
74		({N}{1,3}\.{N}{1,3})\|({N}{1,3}\.{N}{1,3}\.{N}{1,3}) {
75		yylval.s = strdup(yytext);
76		return QUADDOT;
	76	[0-9]+ {
	77	yylval.value = atoi(yytext);
	78	return NUMBER;
77	79	}
78		{N}{1,3}\.{N}{1,3}\.{N}{1,3}\.{N}{1,3} {
	80	[0-9A-Fa-f:][0-9A-Fa-f\.:]+[0-9A-Fa-f:] {
79	81	yylval.s = strdup(yytext);
80		return QUADDOT;
	82	return IPSTRING;
	83	}
	84	ident[ \t]+[a-zA-Z0-9_]+ {
	85	char *p = &(yytext[5]);
	86	while ( p == ' ' \|\| p == '\t' ) p++;
	87	yylval.s = strdup(p);
	88	return IDENT;
	89	}
	90	proto[ \t]+[a-zA-Z0-9_+\-:]+ {
	91	char *p = &(yytext[5]);
	92	while ( p == ' ' \|\| p == '\t' ) p++;
	93	yylval.s = strdup(p);
	94	return PROTOSTR;
81	95	}
82	96	any { return ANY; }
83		ip { return IP; }
	97	ip\|host { return IP; }
84	98	if { return IF; }
85		tcp { return TCP; }
86		udp { return UDP; }
87		gre { return GRE; }
88		esp { return ESP; }
89		ah { return AH; }
90		rsvp { return RSVP; }
91		proto { return PROTO; }
	99	in { return IN; }
	100	out { return OUT; }
	101
	102	tcp\|udp\|icmp { yylval.s = strdup(yytext); ProtoDepreciated(yylval.s); return PROTOSTR; }
	103	gre\|esp\|ah\|rsvp { yylval.s = strdup(yytext); ProtoDepreciated(yylval.s); return PROTOSTR; }
	104
92	105	flags { return FLAGS; }
93	106	tos { return TOS; }
94		host { return HOST; }
95	107	net { return NET; }
96		icmp { return ICMP; }
97	108	port { return PORT; }
98	109	as { return AS; }
99		in { return IN; }
100		out { return OUT; }
101		next { return NEXT; }
102	110	packets { return PACKETS; }
103	111	bytes { return BYTES; }
104	112	bpp { return BPP; }
105	113	bps { return BPS; }
106	114	pps { return PPS; }
107	115	duration { return DURATION; }
	116	ipv4\|inet { return IPV4; }
	117	ipv6\|inet6 { return IPV6; }
108	118	and\|"&&" { return AND; }
109	119	or\|"\|\|" { return OR; }
110	120	not\|"!" { return NOT; }

118	128	[FSRPAUX]+ {
119	129	yylval.s = strdup(yytext);
120	130	return ALPHA_FLAGS;
121		}
122		[0-9]+ {
123		yylval.value = atoi(yytext);
124		return NUMBER;
125	131	}
126	132	\n { lineno++; }
127	133	. { return yytext[0]; }

150	156	int yywrap(void) {
151	157	return 1;
152	158	}
	159
	160	static void ProtoDepreciated(char *s) {
	161	fprintf(stderr, "Syntax '%s' depreciated. Use 'proto %s' instead\n", s, s);
	162	} // End of Protodepreciated
	163

+123

-0

sfcapd.1 less more

	0	.TH sfcapd 1 2006-03-28 "" ""
	1	.SH NAME
	2	sfcapd \- sflow capture daemon
	3	.SH SYNOPSIS
	4	.HP 5
	5	.B sfcapd [options]
	6	.SH DESCRIPTION
	7	.B sfcapd
	8	is the sflow capture daemon of the nfdump tools. It reads sflow
	9	data from the network and stores it into nfcapd compatible files.
	10	The output file is automatically rotated and renamed every n
	11	minutes - typically 5 min - according the timestamp YYYYMMddhhmm
	12	of the interval e.g. nfcapd.200407110845 contains the data from
	13	July 11th 2004 08:45 onward. sfcapd supports sFlow version 4 and
	14	5 datagrams.
	15	.P
	16	Sflow is an industry standard developed by InMon Corporation.
	17	For more information see http://sflow.org.
	18
	19	.SH OPTIONS
	20	.TP 3
	21	.B -p \fIportnum
	22	Specifies the port number to listen. Default port is 6343
	23	.TP 3
	24	.B -b \fIbindhost
	25	Specifies the hostname/IPv4/IPv6 address to bind for listening. Can be an IP
	26	address or a hostname, resolving to an IP address attached to an interface.
	27	Defaults to any available IPv4 interface, if not specified.
	28	.TP 3
	29	.B -4
	30	Forces sfcapd to listen on IPv4 addresses only. Can be used together with -b
	31	if a hostname has an IPv4 and IPv6 address record.
	32	.TP 3
	33	.B -6
	34	Forces sfcapd to listen on IPv6 addresses only. Can be used together with -b
	35	if a hostname has an IPv4 and IPv6 address record.
	36	.TP 3
	37	.B -j \fIMulticastGroup
	38	Join the specified IPv6 or IPv6 multicast group for listening.
	39	.TP 3
	40	.B -l \fIdirectory
	41	Specifies the directory to store the output files. Default is /var/tmp
	42	.TP 3
	43	.B -t \fIinterval
	44	Specifies the time interval in seconds to rotate files. The default value
	45	is 300s ( 5min ).
	46	.TP 3
	47	.B -w
	48	Align file rotation with next n minute ( specified by -t ) interval.
	49	Example: If interval is 5 min, sync at 0,5,10... wall clock minutes
	50	Default: no alignment.
	51	.TP 3
	52	.B -x \fIcmd
	53	Run command \fIcmd\fR at the end of every interval, when a new file
	54	becomes available. The following command expansion is available:
	55	.PD 0
	56	.RS 4
	57	%f Replaced by the file name e.g nfcapd.200407110845.
	58	.P
	59	%d Replaced by the directory where the file is located.
	60	.P
	61	%t Replaced by the time ISO format e.g. 200407110845.
	62	.P
	63	%u Replaced by the UNIX time format.
	64	.P
	65	%i Replaced ident string given by -I
	66	.RE
	67	.PD
	68	.TP 3
	69	.B -P \fIpidfile
	70	Specify name of pidfile. Default is no pidfile.
	71	.TP 3
	72	.B -D
	73	Daemon mode: fork to background and detach from terminal.
	74	Nfcapd terminates on signal TERM, INT and HUP.
	75	.TP 3
	76	.B -I \fIIdentString
	77	Specifies an ident string, which describes the source e.g. the
	78	name of the router. This string is put into the stat file to identify
	79	the source. Default is 'none'.
	80	.TP 3
	81	.B -B \fIbufflen
	82	Specifies the socket input buffer length in bytes. For high volume traffic
	83	( near GB traffic ) it is recommended to set this value as high as possible
	84	( typically > 100k ), otherwise you risk to lose packets. The default
	85	is OS ( and kernel ) dependent.
	86	.TP 3
	87	.B -E
	88	Print data records in nfdump raw format to stdout. This option is for
	89	debugging purpose only, to see how incoming sflow data is processed and stored.
	90	.TP 3
	91	.B -V
	92	Print sfcapd version and exit.
	93	.TP 3
	94	.B -h
	95	Print help text to stdout with all options and exit.
	96	.SH "RETURN VALUE"
	97	Returns 0 on success, or 255 if initialization failed.
	98	.SH "LOGGING"
	99	sfcapd logs to syslog with SYSLOG_FACILITY LOG_DAEMON
	100	For normal operation level 'warning' should be fine.
	101	More information is reported at level 'info' and 'debug'.
	102	.P
	103	A small statistic about the collected flows, as well as errors
	104	are reported at the end of every interval to syslog with level 'info'.
	105	.SH "EXAMPLES"
	106	sfcapd -w -D -l /data/spool/router1 -p 6343 -B 128000 -I router1 -x '/path/nfprofile -p /to/profile/dir -s router1 -r %d/%f' -P /var/run/sfcapd/sfcapd.router1
	107	.SH NOTES
	108	sfcapd automatically scales the packets and bytes according the sampling rate.
	109	.P
	110	Even if sflow version 4 and 5 are supported, not all sflow available elements
	111	are stored in the data files. As of this version, sfcpad supports the the same
	112	fields, as it's netflow companion nfcapd for netflow version v9. See nfcapd(1).
	113	More fields will be supported in future.
	114	.P
	115	The format of the data files is version independant and compatible nfcapd collected data.
	116	.P
	117	Socket buffer: Setting the socket buffer size is system dependent.
	118	When starting up, sfcapd returns the number of bytes the buffer was
	119	actually set. This is done by reading back the buffer size and may
	120	differ from what you requested.
	121	.SH "SEE ALSO"
	122	nfcapd(1), nfdump(1), nfprofile(1), nfreplay(1)

+740

-0

sfcapd.c less more

	0	/*
	1	* sfcapd : Reads sflow data from socket and saves the
	2	* data into a file. The file gets automatically rotated
	3	*
	4	* Copyright (c) 2004, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
	5	* All rights reserved.
	6	*
	7	* Redistribution and use in source and binary forms, with or without
	8	* modification, are permitted provided that the following conditions are met:
	9	*
	10	* * Redistributions of source code must retain the above copyright notice,
	11	* this list of conditions and the following disclaimer.
	12	* * Redistributions in binary form must reproduce the above copyright notice,
	13	* this list of conditions and the following disclaimer in the documentation
	14	* and/or other materials provided with the distribution.
	15	* * Neither the name of SWITCH nor the names of its contributors may be
	16	* used to endorse or promote products derived from this software without
	17	* specific prior written permission.
	18	*
	19	* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
	20	* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	21	* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	22	* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
	23	* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
	24	* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
	25	* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
	26	* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
	27	* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
	28	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
	29	* POSSIBILITY OF SUCH DAMAGE.
	30	*
	31	* $Author: peter $
	32	*
	33	* $Id: sfcapd.c 75 2006-05-21 15:32:48Z peter $
	34	*
	35	* $LastChangedRevision: 75 $
	36	*
	37	*
	38	*/
	39
	40	#include <stdio.h>
	41	#include <stdlib.h>
	42	#include <string.h>
	43	#include <errno.h>
	44	#include <sys/types.h>
	45	#include <sys/socket.h>
	46	#include <sys/param.h>
	47	#include <netdb.h>
	48	#include <pwd.h>
	49	#include <grp.h>
	50	#include <unistd.h>
	51	#include <sys/wait.h>
	52	#include <sys/stat.h>
	53	#include <netinet/in.h>
	54	#include <arpa/inet.h>
	55	#include <time.h>
	56	#include <fcntl.h>
	57	#include <signal.h>
	58	#include <syslog.h>
	59	#include <sys/mman.h>
	60	#include <string.h>
	61	#include <dirent.h>
	62
	63	#include "config.h"
	64
	65	#ifdef HAVE_STDINT_H
	66	#include <stdint.h>
	67	#endif
	68
	69	#include "version.h"
	70	#include "nffile.h"
	71	#include "nf_common.h"
	72	#include "nfnet.h"
	73	#include "launch.h"
	74
	75	#include "sflow.h"
	76
	77	/* default path to store data - not really attractive, but anyway ... */
	78	#define DEFAULT_DIR "/var/tmp"
	79
	80	#define NF_DUMPFILE "nfcapd.current"
	81
	82	#define DEFAULTCISCOPORT "6343"
	83
	84	/* Default time window in seconds to rotate files */
	85	#define TIME_WINDOW 300
	86
	87	/* overdue time:
	88	* if sfcapd does not get any data, wake up the receive system call
	89	* at least after OVERDUE_TIME seconds after the time window
	90	*/
	91	#define OVERDUE_TIME 20
	92
	93	#define SYSLOG_FACILITY LOG_DAEMON
	94
	95	/* Global Variables */
	96	uint32_t byte_limit, packet_limit; // needed for linking purpose only
	97	int byte_mode, packet_mode;
	98	caddr_t shmem;
	99
	100	/* globals */
	101	int verbose = 0;
	102
	103
	104	/* module limited globals */
	105	static int done, launcher_alive, rename_trigger, launcher_pid;
	106
	107	static char Ident[IdentLen];
	108
	109	static char const *rcsid = "$Id: sfcapd.c 75 2006-05-21 15:32:48Z peter $";
	110
	111	/* Local function Prototypes */
	112	static void IntHandler(int signal);
	113
	114	static void usage(char *name);
	115
	116	static void SetPriv(char userid, char groupid );
	117
	118	static void kill_launcher(int pid);
	119
	120	void kill_launcher(int pid) {
	121	int stat;
	122
	123	if ( pid == 0 )
	124	return;
	125
	126	if ( launcher_alive ) {
	127	kill(pid, SIGTERM);
	128	waitpid (pid, &stat, 0);
	129	syslog(LOG_INFO, "laucher terminated: %i", stat);
	130	} else {
	131	waitpid (pid, &stat, 0);
	132	syslog(LOG_ERR, "Can't terminate laucher: process already did: %i", stat);
	133	}
	134
	135	} // End of kill_launcher
	136
	137
	138	static void run(int socket, time_t twin, time_t t_begin, int report_seq);
	139
	140	/* Functions */
	141	static void usage(char *name) {
	142	printf("usage %s [options] \n"
	143	"-h\t\tthis text you see right here\n"
	144	"-u userid\tChange user to userid\n"
	145	"-g groupid\tChange group to groupid\n"
	146	"-w\t\tSync file rotation with next 5min (default) interval\n"
	147	"-t interval\tset the interval to rotate sfcapd files\n"
	148	"-b host\tbind socket to host/IP addr\n"
	149	"-j mcastgroup\tJoin multicast group <mcastgroup>\n"
	150	"-p portnum\tlisten on port portnum\n"
	151	"-l logdir \tset the output directory. (default /var/tmp) \n"
	152	"-I Ident\tset the ident string for stat file. (default 'none')\n"
	153	"-P pidfile\tset the PID file\n"
	154	"-x process\tlauch process after a new file becomes available\n"
	155	"-B bufflen\tSet socket buffer to bufflen bytes\n"
	156	"-D\t\tFork to background\n"
	157	"-E\t\tPrint extended format of sflow data. for debugging purpose only.\n"
	158	"-4\t\tListen on IPv4 (default).\n"
	159	"-6\t\tListen on IPv6.\n"
	160	"-V\t\tPrint version and exit.\n"
	161	, name);
	162	} /* usage */
	163
	164	static void IntHandler(int signal) {
	165
	166	switch (signal) {
	167	case SIGALRM:
	168	rename_trigger = 1;
	169	break;
	170	case SIGHUP:
	171	case SIGINT:
	172	case SIGTERM:
	173	done = 1;
	174	break;
	175	case SIGCHLD:
	176	launcher_alive = 0;
	177	break;
	178	default:
	179	// ignore everything we don't know
	180	break;
	181	}
	182
	183	} /* End of IntHandler */
	184
	185	static void SetPriv(char userid, char groupid ) {
	186	struct passwd *pw_entry;
	187	struct group *gr_entry;
	188	uid_t myuid, newuid, newgid;
	189	int err;
	190
	191	if ( userid == 0 && groupid == 0 )
	192	return;
	193
	194	newuid = newgid = 0;
	195	myuid = getuid();
	196	if ( myuid != 0 ) {
	197	syslog(LOG_ERR, "Only root wants to change uid/gid");
	198	fprintf(stderr, "ERROR: Only root wants to change uid/gid\n");
	199	exit(255);
	200	}
	201
	202	if ( userid ) {
	203	pw_entry = getpwnam(userid);
	204	newuid = pw_entry ? pw_entry->pw_uid : atol(userid);
	205
	206	if ( newuid == 0 ) {
	207	fprintf (stderr,"Invalid user '%s'\n", userid);
	208	exit(255);
	209	}
	210	}
	211
	212	if ( groupid ) {
	213	gr_entry = getgrnam(groupid);
	214	newgid = gr_entry ? gr_entry->gr_gid : atol(groupid);
	215
	216	if ( newgid == 0 ) {
	217	fprintf (stderr,"Invalid group '%s'\n", groupid);
	218	exit(255);
	219	}
	220
	221	err = setgid(newgid);
	222	if ( err ) {
	223	syslog(LOG_ERR, "Can't set group id %ld for group '%s': %s", (long)newgid, groupid, strerror(errno));
	224	fprintf (stderr,"Can't set group id %ld for group '%s': %s\n", (long)newgid, groupid, strerror(errno));
	225	exit(255);
	226	}
	227
	228	}
	229
	230	if ( newuid ) {
	231	err = setuid(newuid);
	232	if ( err ) {
	233	syslog(LOG_ERR, "Can't set user id %ld for user '%s': %s", (long)newuid, userid, strerror(errno));
	234	fprintf (stderr,"Can't set user id %ld for user '%s': %s\n", (long)newuid, userid, strerror(errno));
	235	exit(255);
	236	}
	237	}
	238
	239	} // End of SetPriv
	240
	241	static void run(int socket, time_t twin, time_t t_begin, int report_seq) {
	242	common_flow_header_t *nf_header;
	243	data_block_header_t *data_header;
	244	stat_record_t stat_record;
	245	time_t t_start, t_now;
	246	uint64_t first_seen, last_seen, export_packets;
	247	uint32_t bad_packets, file_blocks, blast_cnt, blast_failures;
	248	struct tm *now;
	249	ssize_t cnt;
	250	void in_buff, out_buff, *writeto;
	251	int err, nffd, first;
	252	char *string, sfcapd_filename[64], dumpfile[64];
	253	srecord_t *commbuff;
	254
	255	Init_sflow();
	256
	257	in_buff = malloc(NETWORK_INPUT_BUFF_SIZE);
	258	out_buff = malloc(OUTPUT_BUFF_SIZE);
	259	if ( !in_buff \|\| !out_buff ) {
	260	syslog(LOG_ERR, "Buffer allocation error: %s", strerror(errno));
	261	return;
	262	}
	263
	264	// init vars
	265	commbuff = (srecord_t *)shmem;
	266	nf_header = (common_flow_header_t *)in_buff;
	267
	268	// Init header
	269	data_header = (data_block_header_t *)out_buff;
	270	data_header->NumBlocks = 0;
	271	data_header->size = 0;
	272	data_header->id = DATA_BLOCK_TYPE_1;
	273	data_header->pad = 0;
	274	writeto = (void *)((pointer_addr_t)data_header + sizeof(data_block_header_t) );
	275
	276	cnt = 0;
	277	export_packets = blast_cnt = blast_failures = 0;
	278
	279	snprintf(dumpfile, 63, "%s.%u",NF_DUMPFILE, getpid());
	280	dumpfile[63] = 0;
	281	nffd = OpenNewFile(dumpfile, &string);
	282	if ( string != NULL ) {
	283	syslog(LOG_ERR, "%s", string);
	284	return;
	285	}
	286
	287	bad_packets = 0;
	288
	289	// init sequence check vars
	290	first = 1;
	291	file_blocks = 0;
	292
	293	first_seen = (uint64_t)0xffffffffffffLL;
	294	last_seen = 0;
	295	t_start = t_begin;
	296	memset((void *)&stat_record, 0, sizeof(stat_record_t));
	297
	298	rename_trigger = 0;
	299	alarm(t_start + twin + OVERDUE_TIME - time(NULL));
	300	/*
	301	* Main processing loop:
	302	* this loop, continues until done = 1, set by the signal handler
	303	* The while loop will be breaked by the periodic file renaming code
	304	* for proper cleanup
	305	*/
	306	while ( 1 ) {
	307
	308	/* read next bunch of data into beginn of input buffer */
	309	if ( !done) {
	310	cnt = recvfrom (socket, in_buff, NETWORK_INPUT_BUFF_SIZE , 0, NULL, 0);
	311	if ( cnt < 0 && errno != EINTR ) {
	312	syslog(LOG_ERR, "ERROR: recvfrom: %s", strerror(errno));
	313	continue;
	314	}
	315	}
	316
	317	/* Periodic file renaming, if time limit reached or we are done. */
	318	t_now = time(NULL);
	319	if ( ((t_now - t_start) >= twin) \|\| done ) {
	320	alarm(0);
	321	now = localtime(&t_start);
	322
	323	if ( verbose ) {
	324	// Dump to stdout
	325	format_file_block_header(out_buff, 0, &string, 0);
	326	}
	327
	328	if ( data_header->NumBlocks ) {
	329	// flush current buffer to disc
	330	if ( write(nffd, out_buff, sizeof(data_block_header_t) + data_header->size) <= 0 )
	331	syslog(LOG_ERR, "Failed to write output buffer to disk: '%s'" , strerror(errno));
	332	else
	333	// update successful written blocks
	334	file_blocks++;
	335	}
	336
	337	// Initialize header and write pointer
	338	data_header->NumBlocks = 0;
	339	data_header->size = 0;
	340	writeto = (void *)((pointer_addr_t)data_header + sizeof(data_block_header_t) );
	341
	342	snprintf(sfcapd_filename, 64, "nfcapd.%i%02i%02i%02i%02i",
	343	now->tm_year + 1900, now->tm_mon + 1, now->tm_mday, now->tm_hour, now->tm_min);
	344
	345	t_start += twin;
	346	alarm(t_start + twin + OVERDUE_TIME - t_now);
	347
	348	stat_record.first_seen = first_seen/1000;
	349	stat_record.msec_first = first_seen - stat_record.first_seen*1000;
	350	stat_record.last_seen = last_seen/1000;
	351	stat_record.msec_last = last_seen - stat_record.last_seen*1000;
	352
	353	/* Write Stat Info */
	354	CloseUpdateFile(nffd, &stat_record, file_blocks, Ident, &string );
	355	if ( string != NULL ) {
	356	syslog(LOG_ERR, "%s", string);
	357	}
	358
	359	err = rename(dumpfile, sfcapd_filename);
	360	if ( err ) {
	361	syslog(LOG_ERR, "Can't rename dump file: %s", strerror(errno));
	362	if (done) break; else continue;
	363	}
	364
	365	if ( launcher_pid ) {
	366	// Signal launcher
	367	strncpy(commbuff->fname, sfcapd_filename, FNAME_SIZE);
	368	commbuff->fname[FNAME_SIZE-1] = 0;
	369	snprintf(commbuff->tstring, 16, "%i%02i%02i%02i%02i",
	370	now->tm_year + 1900, now->tm_mon + 1, now->tm_mday, now->tm_hour, now->tm_min);
	371	commbuff->tstring[15] = 0;
	372	commbuff->tstamp = t_start;
	373
	374	if ( launcher_alive ) {
	375	syslog(LOG_DEBUG, "Signal launcher");
	376	kill(launcher_pid, SIGHUP);
	377	} else {
	378	syslog(LOG_ERR, "ERROR: Launcher did unexpectedly!");
	379	}
	380	}
	381
	382
	383	syslog(LOG_INFO,"Ident: '%s' Flows: %llu, Packets: %llu, Bytes: %llu, Sequence Errors: %u, Bad Packets: %u",
	384	Ident, stat_record.numflows, stat_record.numpackets, stat_record.numbytes, stat_record.sequence_failure, bad_packets);
	385
	386	memset((void *)&stat_record, 0, sizeof(stat_record_t));
	387	bad_packets = 0;
	388	first_seen = 0xffffffffffffLL;
	389	last_seen = 0;
	390	file_blocks = 0;
	391
	392	if ( done )
	393	break;
	394
	395	nffd = OpenNewFile(dumpfile, &string);
	396	if ( string != NULL ) {
	397	syslog(LOG_ERR, "%s", string);
	398	return;
	399	}
	400
	401	}
	402
	403	/* check for error condition or done . errno may only be EINTR */
	404	if ( cnt < 0 ) {
	405	if ( rename_trigger ) {
	406	rename_trigger = 0;
	407	continue;
	408	}
	409	if ( done )
	410	continue;
	411	else {
	412	/* this should never be executed as it should be caught in other places */
	413	syslog(LOG_ERR, "error condition in '%s', line '%d', cnt: %i", __FILE__, __LINE__ ,cnt);
	414	if (done) break; else continue;
	415	}
	416	}
	417
	418	/* enough data? */
	419	if ( cnt == 0 )
	420	continue;
	421
	422
	423	/* Process data - have a look at the common header */
	424	writeto = Process_sflow(in_buff, cnt, data_header, writeto, &stat_record, &first_seen, &last_seen);
	425
	426	// each Process_xx function has to process the entire input buffer, therefore it's empty now.
	427	export_packets++;
	428
	429	// flush current buffer to disc
	430	if ( data_header->size > OUTPUT_FLUSH_LIMIT ) {
	431	if ( write(nffd, out_buff, sizeof(data_block_header_t) + data_header->size) <= 0 ) {
	432	syslog(LOG_ERR, "Failed to write output buffer to disk: '%s'" , strerror(errno));
	433	} else {
	434	data_header->size = 0;
	435	data_header->NumBlocks = 0;
	436	writeto = (void *)((pointer_addr_t)data_header + sizeof(data_block_header_t) );
	437	file_blocks++;
	438	}
	439	}
	440	}
	441
	442	if ( verbose && blast_failures ) {
	443	fprintf(stderr, "Total missed packets: %u\n", blast_failures);
	444	}
	445	free(in_buff);
	446	free(out_buff);
	447	close(nffd);
	448	unlink(dumpfile);
	449
	450	} /* End of run */
	451
	452	int main(int argc, char **argv) {
	453
	454	char bindhost, filter, datadir, pidstr[32], lauch_process;
	455	char userid, groupid, checkptr, listenport, *mcastgroup;
	456	char pidfile[MAXNAMLEN];
	457	struct stat fstat;
	458	srecord_t *commbuff;
	459	struct sigaction act;
	460	int family, bufflen;
	461	time_t twin, t_start, t_tmp;
	462	int sock, pidf, fd, err, synctime, daemonize, report_sequence;
	463	char c;
	464	pid_t pid;
	465
	466	verbose = synctime = daemonize = 0;
	467	bufflen = 0;
	468	family = AF_UNSPEC;
	469	launcher_pid = 0;
	470	launcher_alive = 0;
	471	report_sequence = 0;
	472	listenport = DEFAULTCISCOPORT;
	473	bindhost = NULL;
	474	mcastgroup = NULL;
	475	pidfile[0] = 0;
	476	filter = NULL;
	477	lauch_process = NULL;
	478	userid = groupid = NULL;
	479	twin = TIME_WINDOW;
	480	datadir = DEFAULT_DIR;
	481	strncpy(Ident, "none", IDENT_SIZE);
	482	while ((c = getopt(argc, argv, "46whEVI:DB:b:j:l:p:P:t:x:ru:g:")) != EOF) {
	483	switch (c) {
	484	case 'h':
	485	usage(argv[0]);
	486	exit(0);
	487	break;
	488	case 'u':
	489	userid = optarg;
	490	break;
	491	case 'g':
	492	groupid = optarg;
	493	break;
	494	case 'E':
	495	verbose = 1;
	496	break;
	497	case 'V':
	498	printf("%s: Version: %s %s\n%s\n",argv[0], nfdump_version, nfdump_date, rcsid);
	499	exit(0);
	500	break;
	501	case 'D':
	502	daemonize = 1;
	503	break;
	504	case 'I':
	505	strncpy(Ident, optarg, IDENT_SIZE);
	506	Ident[IDENT_SIZE - 1] = 0;
	507	if ( strchr(Ident, ' ') ) {
	508	fprintf(stderr,"Ident must not contain spaces\n");
	509	exit(255);
	510	}
	511	break;
	512	case 'w':
	513	synctime = 1;
	514	break;
	515	case 'B':
	516	bufflen = strtol(optarg, &checkptr, 10);
	517	if ( (checkptr != NULL && *checkptr == 0) && bufflen > 0 )
	518	break;
	519	fprintf(stderr,"Argument error for -B\n");
	520	exit(255);
	521	case 'b':
	522	bindhost = optarg;
	523	break;
	524	case 'j':
	525	mcastgroup = optarg;
	526	break;
	527	case 'p':
	528	listenport = optarg;
	529	break;
	530	case 'P':
	531	if ( optarg[0] == '/' ) { // absolute path given
	532	strncpy(pidfile, optarg, MAXNAMLEN-1);
	533	} else { // path relative to current working directory
	534	char tmp[MAXPATHLEN];
	535	if ( !getcwd(tmp, MAXPATHLEN-1) ) {
	536	fprintf(stderr, "Failed to get current working directory: %s\n", strerror(errno));
	537	exit(255);
	538	}
	539	tmp[MAXPATHLEN-1] = 0;
	540	snprintf(pidfile, MAXPATHLEN - 1 - strlen(tmp), "%s/%s", tmp, optarg);
	541	}
	542	// pidfile now absolute path
	543	pidfile[MAXNAMLEN-1] = 0;
	544	break;
	545	case 'r':
	546	report_sequence = 1;
	547	break;
	548	case 'l':
	549	datadir = optarg;
	550	err = stat(datadir, &fstat);
	551	if ( !(fstat.st_mode & S_IFDIR) ) {
	552	fprintf(stderr, "No such directory: %s\n", datadir);
	553	break;
	554	}
	555	break;
	556	case 't':
	557	twin = atoi(optarg);
	558	if ( twin <= 0 ) {
	559	fprintf(stderr, "ERROR: time frame <= 0\n");
	560	exit(255);
	561	}
	562	if (twin < 60) {
	563	fprintf(stderr, "WARNING, Very small time frame - < 60s!\n");
	564	}
	565	break;
	566	case 'x':
	567	lauch_process = optarg;
	568	break;
	569	case '4':
	570	if ( family == AF_UNSPEC )
	571	family = AF_INET;
	572	else {
	573	fprintf(stderr, "ERROR, Accepts only one protocol IPv4 or IPv6!\n");
	574	exit(255);
	575	}
	576	break;
	577	case '6':
	578	if ( family == AF_UNSPEC )
	579	family = AF_INET6;
	580	else {
	581	fprintf(stderr, "ERROR, Accepts only one protocol IPv4 or IPv6!\n");
	582	exit(255);
	583	}
	584	break;
	585	default:
	586	usage(argv[0]);
	587	exit(255);
	588	}
	589	}
	590
	591	if ( bindhost && mcastgroup ) {
	592	fprintf(stderr, "ERROR, -b and -j are mutually exclusive!!\n");
	593	exit(255);
	594	}
	595
	596	openlog(argv[0] , LOG_CONS\|LOG_PID, SYSLOG_FACILITY);
	597
	598	SetPriv(userid, groupid);
	599
	600	if ( strlen(pidfile) ) {
	601	pidf = open(pidfile, O_CREAT\|O_RDWR, 0644);
	602	if ( pidf == -1 ) {
	603	fprintf(stderr, "Error opening pid file '%s': %s\n", pidfile, strerror(errno));
	604	exit(255);
	605	}
	606	pid = getpid();
	607	snprintf(pidstr,31,"%i\n", pid);
	608	write(pidf, pidstr, strlen(pidstr));
	609	close(pidf);
	610	}
	611
	612	if ( lauch_process ) {
	613	// for efficiency reason, the process collecting the data
	614	// and the process launching processes, when a new file becomes
	615	// available are separated. Communication is done using signals
	616	// as well as shared memory
	617	// prepare shared memory
	618	shmem = mmap(0, sizeof(srecord_t), PROT_READ \| PROT_WRITE, MAP_ANON \| MAP_SHARED, -1, 0);
	619	if ( shmem == (caddr_t)-1 ) {
	620	perror("mmap error");
	621	exit(255);
	622	}
	623
	624	commbuff = (srecord_t *)shmem;
	625	strncpy(commbuff->ident, Ident, IDENT_SIZE );
	626	commbuff->ident[IDENT_SIZE - 1] = 0;
	627
	628	if ((launcher_pid = fork()) == -1) {
	629	syslog(LOG_ERR, "Can't fork: %s", strerror(errno));
	630	perror("Can't fork()");
	631	} else if ( launcher_pid == 0 ) { // child
	632
	633	// close stdin, stdout and stderr
	634	close(0);
	635	close(1);
	636	close(2);
	637	fd = open("/dev/null",O_RDWR); /* open stdin */
	638	dup(fd); /* stdout */
	639	dup(fd); /* stdout */
	640
	641	launcher((char *)shmem, datadir, lauch_process);
	642	exit(0);
	643	} else {
	644	launcher_alive = 1;
	645	syslog(LOG_DEBUG, "Launcher forked");
	646	}
	647	// parent continues
	648	}
	649
	650	if (argc - optind > 1) {
	651	usage(argv[0]);
	652	kill_launcher(launcher_pid);
	653	exit(255);
	654	} else {
	655	/* user specified a pcap filter */
	656	filter = argv[optind];
	657	}
	658
	659	if ( mcastgroup )
	660	sock = Multicast_receive_socket (mcastgroup, listenport, family, bufflen);
	661	else
	662	sock = Unicast_receive_socket(bindhost, listenport, family, bufflen );
	663
	664	if ( sock == -1 ) {
	665	kill_launcher(launcher_pid);
	666	fprintf(stderr,"Terminated due to errors.\n");
	667	exit(255);
	668	}
	669
	670	if ( synctime ) {
	671	t_tmp = time(NULL);
	672	t_start = t_tmp - ( t_tmp % twin);
	673	} else
	674	t_start = time(NULL);
	675
	676	if ( daemonize ) {
	677	verbose = 0;
	678	if ((pid = fork()) < 0 ) {
	679	perror("Can't fork()");
	680	} else if (pid) {
	681	if (strlen(pidfile)) {
	682	pidf = open(pidfile, O_CREAT\|O_RDWR, 0644);
	683	if ( pidf == -1 ) {
	684	syslog(LOG_ERR, "Error opening pid file: '%s' %s", pidfile, strerror(errno));
	685	perror("Error opening pid file:");
	686	kill_launcher(launcher_pid);
	687	exit(255);
	688	}
	689	snprintf(pidstr,31,"%i\n", pid);
	690	write(pidf, pidstr, strlen(pidstr));
	691	close(pidf);
	692	}
	693	exit (0); /* parent */
	694	} // else -> child continues
	695
	696	if (setsid() < 0) {
	697	syslog(LOG_ERR, "Can't create new session: '%s'", strerror(errno));
	698	exit(255);
	699	}
	700	// close stdin, stdout and stderr
	701	close(0);
	702	close(1);
	703	close(2);
	704	fd = open("/dev/null",O_RDWR); /* open stdin */
	705	dup(fd); /* stdout */
	706	dup(fd); /* stderr */
	707	}
	708
	709	if ( chdir(datadir)) {
	710	syslog(LOG_ERR, "Error can't chdir to '%s': %s", datadir, strerror(errno));
	711	kill_launcher(launcher_pid);
	712	exit(255);
	713	}
	714	done = 0;
	715
	716	/* Signal handling */
	717	memset((void *)&act,0,sizeof(struct sigaction));
	718	act.sa_handler = IntHandler;
	719	sigemptyset(&act.sa_mask);
	720	act.sa_flags = 0;
	721	sigaction(SIGTERM, &act, NULL);
	722	sigaction(SIGINT, &act, NULL);
	723	sigaction(SIGHUP, &act, NULL);
	724	sigaction(SIGALRM, &act, NULL);
	725	sigaction(SIGCHLD, &act, NULL);
	726
	727	syslog(LOG_INFO, "Startup.");
	728	run(sock, twin, t_start, report_sequence);
	729	close(sock);
	730	syslog(LOG_INFO, "Terminating sfcapd.");
	731
	732	if ( strlen(pidfile) )
	733	unlink(pidfile);
	734
	735	closelog();
	736	kill_launcher(launcher_pid);
	737	return 0;
	738
	739	} /* End of main */

+2295

-0

sflow.c less more

	0	/*
	1	* Copyright (c) 2004, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
	2	* All rights reserved.
	3	*
	4	* Redistribution and use in source and binary forms, with or without
	5	* modification, are permitted provided that the following conditions are met:
	6	*
	7	* * Redistributions of source code must retain the above copyright notice,
	8	* this list of conditions and the following disclaimer.
	9	* * Redistributions in binary form must reproduce the above copyright notice,
	10	* this list of conditions and the following disclaimer in the documentation
	11	* and/or other materials provided with the distribution.
	12	* * Neither the name of SWITCH nor the names of its contributors may be
	13	* used to endorse or promote products derived from this software without
	14	* specific prior written permission.
	15	*
	16	* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
	17	* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	18	* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	19	* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
	20	* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
	21	* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
	22	* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
	23	* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
	24	* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
	25	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
	26	* POSSIBILITY OF SUCH DAMAGE.
	27	*
	28	* $Author: peter $
	29	*
	30	* $Id: sflow.c 75 2006-05-21 15:32:48Z peter $
	31	*
	32	* $LastChangedRevision: 75 $
	33	*
	34	*
	35	*/
	36
	37	/*
	38	* sfcapd makes use of code originated from sflowtool by InMon Corp.
	39	* Those parts of the code are distributed under the InMon Public License below.
	40	* All other/additional code is pubblished under BSD license.
	41	*/
	42
	43
	44	/*
	45	* -----------------------------------------------------------------------
	46	* Copyright (c) 2001-2002 InMon Corp. All rights reserved.
	47	* -----------------------------------------------------------------------
	48	*
	49	* Redistribution and use in source and binary forms, with or without
	50	* modification, are permitted provided that the following conditions
	51	* are met:
	52	*
	53	* 1. Redistributions of source code must retain the above copyright
	54	* notice, this list of conditions and the following disclaimer.
	55	*
	56	* 2. Redistributions in binary form must reproduce the above
	57	* copyright notice, this list of conditions and the following
	58	* disclaimer in the documentation and/or other materials provided
	59	* with the distribution.
	60	*
	61	* 3. Redistributions of any form whatsoever must retain the following
	62	* acknowledgment:
	63	* "This product includes sFlow(TM), freely available from
	64	* http://www.inmon.com/".
	65	*
	66	* 4. All advertising materials mentioning features or use of this
	67	* software must display the following acknowledgment:
	68	* "This product includes sFlow(TM), freely available from
	69	* http://www.inmon.com/".
	70	*
	71	* 5. InMon Corp. may publish revised and/or new versions
	72	* of the license from time to time. Each version will be given a
	73	* distinguishing version number. Once covered code has been
	74	* published under a particular version of the license, you may
	75	* always continue to use it under the terms of that version. You
	76	* may also choose to use such covered code under the terms of any
	77	* subsequent version of the license published by InMon Corp.
	78	* No one other than the InMon Corp. has the right to modify the terms
	79	* applicable to covered code created under this License.
	80	*
	81	* 6. The name "sFlow" must not be used to endorse or promote products
	82	* derived from this software without prior written permission
	83	* from InMon Corp. This does not apply to add-on libraries or tools
	84	* that work in conjunction with sFlow. In such a case the sFlow name
	85	* may be used to indicate that the product supports sFlow.
	86	*
	87	* 7. Products derived from this software may not be called "sFlow",
	88	* nor may "sFlow" appear in their name, without prior written
	89	* permission of InMon Corp.
	90	*
	91	*
	92	* THIS SOFTWARE IS PROVIDED BY INMON CORP. ``AS IS'' AND
	93	* ANY EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
	94	* THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
	95	* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
	96	* INMON CORP. OR ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
	97	* INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
	98	* (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
	99	* SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
	100	* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
	101	* STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
	102	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
	103	* OF THE POSSIBILITY OF SUCH DAMAGE.
	104	*
	105	* --------------------------------------------------------------------
	106	*
	107	* This software consists of voluntary contributions made by many
	108	* individuals on behalf of InMon Corp.
	109	*
	110	* InMon Corp. can be contacted via Email at info@inmon.com.
	111	*
	112	* For more information on InMon Corp. and sFlow,
	113	* please see http://www.inmon.com/.
	114	*
	115	* InMon Public License Version 1.0 written May 31, 2001
	116	*
	117	*/
	118
	119	#include <stdio.h>
	120	#include <stdarg.h>
	121	#include <stdlib.h>
	122	#include <string.h>
	123	#include <errno.h>
	124	#include <sys/types.h>
	125	#include <time.h>
	126	#include <setjmp.h>
	127
	128	#include <unistd.h>
	129	#include <netdb.h>
	130	#include <sys/socket.h>
	131	#include <netinet/in.h>
	132	#include <sys/time.h>
	133	#include <syslog.h>
	134
	135	#include "config.h"
	136
	137	#ifdef HAVE_STDINT_H
	138	#include <stdint.h>
	139	#endif
	140
	141	#include "nf_common.h"
	142	#include "nffile.h"
	143	#include "sflow.h"
	144	#include "sflow_proto.h" // sFlow v5
	145
	146	/*
	147	#ifdef DARWIN
	148	#include <architecture/byte_order.h>
	149	#define bswap_16(x) NXSwapShort(x)
	150	#define bswap_32(x) NXSwapInt(x)
	151	#else
	152	#include <byteswap.h>
	153	#endif
	154	*/
	155
	156	/*
	157	* unused
	158	//
	159	static uint32_t MyByteSwap32(uint32_t n) {
	160	return (((n & 0x000000FF)<<24) +
	161	((n & 0x0000FF00)<<8) +
	162	((n & 0x00FF0000)>>8) +
	163	((n & 0xFF000000)>>24));
	164	}
	165
	166	static uint16_t MyByteSwap16(uint16_t n) {
	167	return ((n >> 8) \| (n << 8));
	168	}
	169	*/
	170
	171	#define YES 1
	172	#define NO 0
	173
	174	/* define my own IP header struct - to ease portability */
	175	struct myiphdr {
	176	uint8_t version_and_headerLen;
	177	uint8_t tos;
	178	uint16_t tot_len;
	179	uint16_t id;
	180	uint16_t frag_off;
	181	uint8_t ttl;
	182	uint8_t protocol;
	183	uint16_t check;
	184	uint32_t saddr;
	185	uint32_t daddr;
	186	};
	187
	188	/* same for tcp */
	189	struct mytcphdr {
	190	uint16_t th_sport; /* source port */
	191	uint16_t th_dport; /* destination port */
	192	uint32_t th_seq; /* sequence number */
	193	uint32_t th_ack; /* acknowledgement number */
	194	uint8_t th_off_and_unused;
	195	uint8_t th_flags;
	196	uint16_t th_win; /* window */
	197	uint16_t th_sum; /* checksum */
	198	uint16_t th_urp; /* urgent pointer */
	199	};
	200
	201	/* and UDP */
	202	struct myudphdr {
	203	uint16_t uh_sport; /* source port */
	204	uint16_t uh_dport; /* destination port */
	205	uint16_t uh_ulen; /* udp length */
	206	uint16_t uh_sum; /* udp checksum */
	207	};
	208
	209	/* and ICMP */
	210	struct myicmphdr {
	211	uint8_t type; /* message type */
	212	uint8_t code; /* type sub-code */
	213	/* ignore the rest */
	214	};
	215
	216	typedef struct _SFForwardingTarget {
	217	struct _SFForwardingTarget *nxt;
	218	struct in_addr host;
	219	uint32_t port;
	220	struct sockaddr_in addr;
	221	int sock;
	222	} SFForwardingTarget;
	223
	224	typedef enum { SFLFMT_FULL=0, SFLFMT_PCAP, SFLFMT_LINE } EnumSFLFormat;
	225
	226	typedef struct _SFConfig {
	227	uint16_t netFlowPeerAS;
	228	int disableNetFlowScale;
	229	} SFConfig;
	230
	231	/* make the options structure global to the program */
	232	static SFConfig sfConfig;
	233
	234	typedef struct _SFSample {
	235	struct in_addr sourceIP;
	236	SFLAddress agent_addr;
	237	uint32_t agentSubId;
	238
	239	/* the raw pdu */
	240	u_char *rawSample;
	241	uint32_t rawSampleLen;
	242	u_char *endp;
	243
	244	/* decode cursor */
	245	uint32_t *datap;
	246
	247	uint32_t datagramVersion;
	248	uint32_t sampleType;
	249	uint32_t ds_class;
	250	uint32_t ds_index;
	251
	252	/* generic interface counter sample */
	253	SFLIf_counters ifCounters;
	254
	255	/* sample stream info */
	256	uint32_t sysUpTime;
	257	uint32_t sequenceNo;
	258	uint32_t sampledPacketSize;
	259	uint32_t samplesGenerated;
	260	uint32_t meanSkipCount;
	261	uint32_t samplePool;
	262	uint32_t dropEvents;
	263
	264	/* the sampled header */
	265	uint32_t packet_data_tag;
	266	uint32_t headerProtocol;
	267	u_char *header;
	268	int headerLen;
	269	uint32_t stripped;
	270
	271	/* header decode */
	272	int gotIPV4;
	273	int offsetToIPV4;
	274	int gotIPV6;
	275	int offsetToIPV6;
	276	struct in_addr dcd_srcIP;
	277	struct in_addr dcd_dstIP;
	278	uint32_t dcd_ipProtocol;
	279	uint32_t dcd_ipTos;
	280	uint32_t dcd_ipTTL;
	281	uint32_t dcd_sport;
	282	uint32_t dcd_dport;
	283	uint32_t dcd_tcpFlags;
	284	uint32_t ip_fragmentOffset;
	285	uint32_t udp_pduLen;
	286
	287	/* ports */
	288	uint32_t inputPortFormat;
	289	uint32_t outputPortFormat;
	290	uint32_t inputPort;
	291	uint32_t outputPort;
	292
	293	/* ethernet */
	294	uint32_t eth_type;
	295	uint32_t eth_len;
	296	u_char eth_src[8];
	297	u_char eth_dst[8];
	298
	299	/* vlan */
	300	uint32_t in_vlan;
	301	uint32_t in_priority;
	302	uint32_t internalPriority;
	303	uint32_t out_vlan;
	304	uint32_t out_priority;
	305
	306	/* extended data fields */
	307	uint32_t num_extended;
	308	uint32_t extended_data_tag;
	309	#define SASAMPLE_EXTENDED_DATA_SWITCH 1
	310	#define SASAMPLE_EXTENDED_DATA_ROUTER 4
	311	#define SASAMPLE_EXTENDED_DATA_GATEWAY 8
	312	#define SASAMPLE_EXTENDED_DATA_USER 16
	313	#define SASAMPLE_EXTENDED_DATA_URL 32
	314	#define SASAMPLE_EXTENDED_DATA_MPLS 64
	315	#define SASAMPLE_EXTENDED_DATA_NAT 128
	316	#define SASAMPLE_EXTENDED_DATA_MPLS_TUNNEL 256
	317	#define SASAMPLE_EXTENDED_DATA_MPLS_VC 512
	318	#define SASAMPLE_EXTENDED_DATA_MPLS_FTN 1024
	319	#define SASAMPLE_EXTENDED_DATA_MPLS_LDP_FEC 2048
	320	#define SASAMPLE_EXTENDED_DATA_VLAN_TUNNEL 4096
	321
	322	/* IP forwarding info */
	323	SFLAddress nextHop;
	324	uint32_t srcMask;
	325	uint32_t dstMask;
	326
	327	/* BGP info */
	328	SFLAddress bgp_nextHop;
	329	uint32_t my_as;
	330	uint32_t src_as;
	331	uint32_t src_peer_as;
	332	uint32_t dst_as_path_len;
	333	uint32_t *dst_as_path;
	334	/* note: version 4 dst as path segments just get printed, not stored here, however
	335	* the dst_peer and dst_as are filled in, since those are used for netflow encoding
	336	*/
	337	uint32_t dst_peer_as;
	338	uint32_t dst_as;
	339
	340	uint32_t communities_len;
	341	uint32_t *communities;
	342	uint32_t localpref;
	343
	344	/* user id */
	345	#define SA_MAX_EXTENDED_USER_LEN 200
	346	uint32_t src_user_charset;
	347	uint32_t src_user_len;
	348	char src_user[SA_MAX_EXTENDED_USER_LEN+1];
	349	uint32_t dst_user_charset;
	350	uint32_t dst_user_len;
	351	char dst_user[SA_MAX_EXTENDED_USER_LEN+1];
	352
	353	/* url */
	354	#define SA_MAX_EXTENDED_URL_LEN 200
	355	#define SA_MAX_EXTENDED_HOST_LEN 200
	356	uint32_t url_direction;
	357	uint32_t url_len;
	358	char url[SA_MAX_EXTENDED_URL_LEN+1];
	359	uint32_t host_len;
	360	char host[SA_MAX_EXTENDED_HOST_LEN+1];
	361
	362	/* mpls */
	363	SFLAddress mpls_nextHop;
	364
	365	/* nat */
	366	SFLAddress nat_src;
	367	SFLAddress nat_dst;
	368
	369	/* counter blocks */
	370	uint32_t statsSamplingInterval;
	371	uint32_t counterBlockVersion;
	372
	373	/* exception handler context */
	374	jmp_buf env;
	375
	376	#define SFABORT(s, r) longjmp((s)->env, (r))
	377	#define SF_ABORT_EOS 1
	378	#define SF_ABORT_DECODE_ERROR 2
	379	#define SF_ABORT_LENGTH_ERROR 3
	380
	381	SFLAddress ipsrc;
	382	SFLAddress ipdst;
	383	} SFSample;
	384
	385	typedef struct nf_buffer_s {
	386	void *writeto;
	387	stat_record_t *stat_record;
	388	data_block_header_t *data_header;
	389	uint64_t first_seen;
	390	uint64_t last_seen;
	391	uint32_t size;
	392	uint32_t count;
	393	} nf_buffer_t;
	394
	395	static int printHex(const u_char a, int len, char buf, int bufLen, int marker, int bytesPerOutputLine);
	396
	397	static char IP_to_a(uint32_t ipaddr, char buf, int buflen);
	398
	399	static inline uint32_t getData32(SFSample *sample);
	400
	401	static inline uint32_t getData32_nobswap(SFSample *sample);
	402
	403	static inline uint64_t getData64(SFSample *sample);
	404
	405	static void writeCountersLine(SFSample *sample);
	406
	407	static void receiveError(SFSample sample, char errm, int hexdump) __attribute__ ((noreturn));
	408
	409	static inline void skipBytes(SFSample *sample, int skip);
	410
	411	static inline uint32_t sf_log_next32(SFSample sample, char fieldName);
	412
	413	static inline uint64_t sf_log_next64(SFSample sample, char fieldName);
	414
	415	static inline void sf_log_percentage(SFSample sample, char fieldName);
	416
	417	static inline uint32_t getString(SFSample sample, char buf, int bufLen);
	418
	419	static inline uint32_t getAddress(SFSample sample, SFLAddress address);
	420
	421	static inline char printTag(uint32_t tag, char buf, int bufLen);
	422
	423	static inline void skipTLVRecord(SFSample sample, uint32_t tag, uint32_t len, char description);
	424
	425	static inline void readSFlowDatagram(SFSample sample, nf_buffer_t output_buffer);
	426
	427	static inline void readFlowSample(SFSample sample, int expanded, nf_buffer_t output_buffer);
	428
	429	static inline void readCountersSample(SFSample sample, int expanded, nf_buffer_t output_buffer);
	430
	431	static inline void readFlowSample_v2v4(SFSample sample, nf_buffer_t output_buffer);
	432
	433	static inline void readCountersSample_v2v4(SFSample sample, nf_buffer_t output_buffer);
	434
	435	static inline void StoreSflowRecord(SFSample sample, nf_buffer_t output_buffer);
	436
	437	extern int verbose;
	438
	439
	440	/*_________________---------------------------__________________
	441	_________________ sf_log __________________
	442	-----------------___________________________------------------
	443	*/
	444
	445	#ifdef DEBUG
	446	void sf_log(char *fmt, ...);
	447
	448	void sf_log(char *fmt, ...) {
	449	if ( verbose ) {
	450	va_list args;
	451	va_start(args, fmt);
	452	vprintf(fmt, args);
	453	}
	454	} // End of sf_log
	455	#endif
	456
	457	#ifndef DEBUG
	458	# define sf_log(...) /* sf_log(...) */
	459	#endif
	460
	461	/*_________________---------------------------__________________
	462	_________________ printHex __________________
	463	-----------------___________________________------------------
	464	*/
	465
	466	static u_char bin2hex(int nib) { return (nib < 10) ? ('0' + nib) : ('A' - 10 + nib); }
	467
	468	static int printHex(const u_char a, int len, char buf, int bufLen, int marker, int bytesPerOutputLine) {
	469	int b = 0, i = 0;
	470	for(; i < len; i++) {
	471	u_char byte;
	472	if(b > (bufLen - 10)) break;
	473	if(marker > 0 && i == marker) {
	474	buf[b++] = '<';
	475	buf[b++] = '*';
	476	buf[b++] = '>';
	477	buf[b++] = '-';
	478	}
	479	byte = a[i];
	480	buf[b++] = bin2hex(byte >> 4);
	481	buf[b++] = bin2hex(byte & 0x0f);
	482	if(i > 0 && (i % bytesPerOutputLine) == 0) buf[b++] = '\n';
	483	else {
	484	// separate the bytes with a dash
	485	if (i < (len - 1)) buf[b++] = '-';
	486	}
	487	}
	488	buf[b] = '\0';
	489	return b;
	490	}
	491
	492	/*_________________---------------------------__________________
	493	_________________ IP_to_a __________________
	494	-----------------___________________________------------------
	495	*/
	496
	497	static char IP_to_a(uint32_t ipaddr, char buf, int buflen) {
	498	u_char ip = (u_char )&ipaddr;
	499	snprintf(buf, buflen, "%u.%u.%u.%u", ip[0], ip[1], ip[2], ip[3]);
	500	buf[buflen-1] = '\0';
	501	return buf;
	502	}
	503
	504	static char printAddress(SFLAddress address, char *buf, int bufLen) {
	505	if(address->type == SFLADDRESSTYPE_IP_V4)
	506	IP_to_a(address->address.ip_v4.s_addr, buf, bufLen);
	507	else {
	508	u_char *b = address->address.ip_v6.s6_addr;
	509	snprintf(buf, bufLen, "%02x%02x:%02x%02x:%02x%02x:%02x%02x:%02x%02x:%02x%02x:%02x%02x:%02x%02x",
	510	b[0],b[1],b[2],b[3],b[4],b[5],b[6],b[7],b[8],b[9],b[10],b[11],b[12],b[13],b[14],b[15]);
	511	}
	512	return buf;
	513	}
	514
	515	/*_________________---------------------------__________________
	516	_________________ writeFlowLine __________________
	517	-----------------___________________________------------------
	518	*/
	519
	520	static void writeFlowLine(SFSample *sample) {
	521	char agentIP[51], srcIP[51], dstIP[51];
	522	// source
	523	printf("FLOW,%s,%d,%d,",
	524	printAddress(&sample->agent_addr, agentIP, 50),
	525	sample->inputPort,
	526	sample->outputPort);
	527	// layer 2
	528	printf("%02x%02x%02x%02x%02x%02x,%02x%02x%02x%02x%02x%02x,0x%04x,%d,%d",
	529	sample->eth_src[0],
	530	sample->eth_src[1],
	531	sample->eth_src[2],
	532	sample->eth_src[3],
	533	sample->eth_src[4],
	534	sample->eth_src[5],
	535	sample->eth_dst[0],
	536	sample->eth_dst[1],
	537	sample->eth_dst[2],
	538	sample->eth_dst[3],
	539	sample->eth_dst[4],
	540	sample->eth_dst[5],
	541	sample->eth_type,
	542	sample->in_vlan,
	543	sample->out_vlan);
	544	// layer 3/4
	545	printf(",IP: %s,%s,%d,0x%02x,%d,%d,%d,0x%02x",
	546	IP_to_a(sample->dcd_srcIP.s_addr, srcIP, 51),
	547	IP_to_a(sample->dcd_dstIP.s_addr, dstIP, 51),
	548	sample->dcd_ipProtocol,
	549	sample->dcd_ipTos,
	550	sample->dcd_ipTTL,
	551	sample->dcd_sport,
	552	sample->dcd_dport,
	553	sample->dcd_tcpFlags);
	554	// bytes
	555	printf(",%d,%d,%d\n",
	556	sample->sampledPacketSize,
	557	sample->sampledPacketSize - sample->stripped - sample->offsetToIPV4,
	558	sample->meanSkipCount);
	559	}
	560
	561	/*_________________---------------------------__________________
	562	_________________ writeCountersLine __________________
	563	-----------------___________________________------------------
	564	*/
	565
	566	static void writeCountersLine(SFSample *sample)
	567	{
	568	// source
	569	char agentIP[51];
	570	printf("CNTR,%s,", printAddress(&sample->agent_addr, agentIP, 50));
	571	printf("%u,%u,%llu,%u,%u,%llu,%u,%u,%u,%u,%u,%u,%llu,%u,%u,%u,%u,%u,%u\n",
	572	sample->ifCounters.ifIndex,
	573	sample->ifCounters.ifType,
	574	sample->ifCounters.ifSpeed,
	575	sample->ifCounters.ifDirection,
	576	sample->ifCounters.ifStatus,
	577	sample->ifCounters.ifInOctets,
	578	sample->ifCounters.ifInUcastPkts,
	579	sample->ifCounters.ifInMulticastPkts,
	580	sample->ifCounters.ifInBroadcastPkts,
	581	sample->ifCounters.ifInDiscards,
	582	sample->ifCounters.ifInErrors,
	583	sample->ifCounters.ifInUnknownProtos,
	584	sample->ifCounters.ifOutOctets,
	585	sample->ifCounters.ifOutUcastPkts,
	586	sample->ifCounters.ifOutMulticastPkts,
	587	sample->ifCounters.ifOutBroadcastPkts,
	588	sample->ifCounters.ifOutDiscards,
	589	sample->ifCounters.ifOutErrors,
	590	sample->ifCounters.ifPromiscuousMode);
	591	}
	592
	593	/*_________________---------------------------__________________
	594	_________________ receiveError __________________
	595	-----------------___________________________------------------
	596	*/
	597
	598	static void receiveError(SFSample sample, char errm, int hexdump)
	599	{
	600	char ipbuf[51];
	601	char scratch[6000];
	602	char *msg = "";
	603	char *hex = "";
	604	uint32_t markOffset = (u_char *)sample->datap - sample->rawSample;
	605	if(errm) msg = errm;
	606	if(hexdump) {
	607	printHex(sample->rawSample, sample->rawSampleLen, scratch, 6000, markOffset, 16);
	608	hex = scratch;
	609	}
	610	syslog(LOG_ERR, "%s (source IP = %s) %s\n", msg, IP_to_a(sample->sourceIP.s_addr, ipbuf, 51), hex);
	611
	612	SFABORT(sample, SF_ABORT_DECODE_ERROR);
	613
	614	}
	615
	616	/*_________________---------------------------__________________
	617	_________________ lengthCheck __________________
	618	-----------------___________________________------------------
	619	*/
	620
	621	static void lengthCheck(SFSample sample, char description, u_char *start, int len) {
	622	uint32_t actualLen = (u_char *)sample->datap - start;
	623	if(actualLen != len) {
	624	syslog(LOG_ERR, "%s length error (expected %d, found %d)\n", description, len, actualLen);
	625	SFABORT(sample, SF_ABORT_LENGTH_ERROR);
	626	}
	627	}
	628
	629	/*_________________---------------------------__________________
	630	_________________ decodeLinkLayer __________________
	631	-----------------___________________________------------------
	632	store the offset to the start of the ipv4 header in the sequence_number field
	633	or -1 if not found. Decode the 802.1d if it's there.
	634	*/
	635
	636	#define NFT_ETHHDR_SIZ 14
	637	#define NFT_8022_SIZ 3
	638	#define NFT_MAX_8023_LEN 1500
	639
	640	#define NFT_MIN_SIZ (NFT_ETHHDR_SIZ + sizeof(struct myiphdr))
	641
	642	static void decodeLinkLayer(SFSample *sample)
	643	{
	644	u_char start = (u_char )sample->header;
	645	u_char *end = start + sample->headerLen;
	646	u_char *ptr = start;
	647	uint16_t type_len;
	648
	649	/* assume not found */
	650	sample->gotIPV4 = NO;
	651
	652	if(sample->headerLen < NFT_ETHHDR_SIZ) return; /* not enough for an Ethernet header */
	653
	654	sf_log("dstMAC %02x%02x%02x%02x%02x%02x\n", ptr[0], ptr[1], ptr[2], ptr[3], ptr[4], ptr[5]);
	655	memcpy(sample->eth_dst, ptr, 6);
	656	ptr += 6;
	657
	658	sf_log("srcMAC %02x%02x%02x%02x%02x%02x\n", ptr[0], ptr[1], ptr[2], ptr[3], ptr[4], ptr[5]);
	659	memcpy(sample->eth_src, ptr, 6);
	660	ptr += 6;
	661	type_len = (ptr[0] << 8) + ptr[1];
	662	ptr += 2;
	663
	664	if(type_len == 0x8100) {
	665	/* VLAN - next two bytes */
	666	uint32_t vlanData = (ptr[0] << 8) + ptr[1];
	667	uint32_t vlan = vlanData & 0x0fff;
	668	#ifdef DEBUG
	669	uint32_t priority = vlanData >> 13;
	670	#endif
	671	ptr += 2;
	672	/* _____________________________________ */
	673	/* \| pri \| c \| vlan-id \| */
	674	/* ------------------------------------- */
	675	/* [priority = 3bits] [Canonical Format Flag = 1bit] [vlan-id = 12 bits] */
	676	sf_log("decodedVLAN %lu\n", vlan);
	677	sf_log("decodedPriority %lu\n", priority);
	678	sample->in_vlan = vlan;
	679	/* now get the type_len again (next two bytes) */
	680	type_len = (ptr[0] << 8) + ptr[1];
	681	ptr += 2;
	682	}
	683
	684	/* now we're just looking for IP */
	685	if(sample->headerLen < NFT_MIN_SIZ) return; /* not enough for an IPv4 header */
	686
	687	/* peek for IPX */
	688	if(type_len == 0x0200 \|\| type_len == 0x0201 \|\| type_len == 0x0600) {
	689	#define IPX_HDR_LEN 30
	690	#define IPX_MAX_DATA 546
	691	int ipxChecksum = (ptr[0] == 0xff && ptr[1] == 0xff);
	692	int ipxLen = (ptr[2] << 8) + ptr[3];
	693	if(ipxChecksum &&
	694	ipxLen >= IPX_HDR_LEN &&
	695	ipxLen <= (IPX_HDR_LEN + IPX_MAX_DATA))
	696	/* we don't do anything with IPX here */
	697	return;
	698	}
	699
	700	if(type_len <= NFT_MAX_8023_LEN) {
	701	/* assume 802.3+802.2 header */
	702	/* check for SNAP */
	703	if(ptr[0] == 0xAA &&
	704	ptr[1] == 0xAA &&
	705	ptr[2] == 0x03) {
	706	ptr += 3;
	707	if(ptr[0] != 0 \|\|
	708	ptr[1] != 0 \|\|
	709	ptr[2] != 0) {
	710	sf_log("VSNAP_OUI %02X-%02X-%02X\n", ptr[0], ptr[1], ptr[2]);
	711	return; /* no further decode for vendor-specific protocol */
	712	}
	713	ptr += 3;
	714	/* OUI == 00-00-00 means the next two bytes are the ethernet type (RFC 2895) */
	715	type_len = (ptr[0] << 8) + ptr[1];
	716	ptr += 2;
	717	}
	718	else {
	719	if (ptr[0] == 0x06 &&
	720	ptr[1] == 0x06 &&
	721	(ptr[2] & 0x01)) {
	722	/* IP over 8022 */
	723	ptr += 3;
	724	/* force the type_len to be IP so we can inline the IP decode below */
	725	type_len = 0x0800;
	726	}
	727	else return;
	728	}
	729	}
	730
	731	/* assume type_len is an ethernet-type now */
	732	sample->eth_type = type_len;
	733
	734	if(type_len == 0x0800) {
	735	/* IPV4 */
	736	if((end - ptr) < sizeof(struct myiphdr)) return;
	737	/* look at first byte of header.... */
	738	/* ___________________________ */
	739	/* \| version \| hdrlen \| */
	740	/* --------------------------- */
	741	if((ptr >> 4) != 4) return; / not version 4 */
	742	if((ptr & 15) < 5) return; / not IP (hdr len must be 5 quads or more) */
	743	/* survived all the tests - store the offset to the start of the ip header */
	744	sample->gotIPV4 = YES;
	745	sample->offsetToIPV4 = (ptr - start);
	746	}
	747
	748	if(type_len == 0x86DD) {
	749	/* IPV6 */
	750	/* look at first byte of header.... */
	751	if((ptr >> 4) != 6) return; / not version 6 */
	752	/* survived all the tests - store the offset to the start of the ip6 header */
	753	sample->gotIPV6 = YES;
	754	sample->offsetToIPV6 = (ptr - start);
	755	}
	756	}
	757
	758
	759	/*_________________---------------------------__________________
	760	_________________ decodeIPLayer4 __________________
	761	-----------------___________________________------------------
	762	*/
	763
	764	static void decodeIPLayer4(SFSample sample, u_char ptr, uint32_t ipProtocol) {
	765	u_char *end = sample->header + sample->headerLen;
	766	if(ptr > (end - 8)) return; // not enough header bytes left
	767	switch(ipProtocol) {
	768	case 1: /* ICMP */
	769	{
	770	struct myicmphdr icmp;
	771	memcpy(&icmp, ptr, sizeof(icmp));
	772	sf_log("ICMPType %u\n", icmp.type);
	773	sf_log("ICMPCode %u\n", icmp.code);
	774	sample->dcd_sport = icmp.type;
	775	sample->dcd_dport = icmp.code;
	776	}
	777	break;
	778	case 6: /* TCP */
	779	{
	780	struct mytcphdr tcp;
	781	memcpy(&tcp, ptr, sizeof(tcp));
	782	sample->dcd_sport = ntohs(tcp.th_sport);
	783	sample->dcd_dport = ntohs(tcp.th_dport);
	784	sample->dcd_tcpFlags = tcp.th_flags;
	785	sf_log("TCPSrcPort %u\n", sample->dcd_sport);
	786	sf_log("TCPDstPort %u\n",sample->dcd_dport);
	787	sf_log("TCPFlags %u\n", sample->dcd_tcpFlags);
	788	if(sample->dcd_dport == 80) {
	789	int bytesLeft;
	790	int headerBytes = (tcp.th_off_and_unused >> 4) * 4;
	791	ptr += headerBytes;
	792	bytesLeft = sample->header + sample->headerLen - ptr;
	793	}
	794	}
	795	break;
	796	case 17: /* UDP */
	797	{
	798	struct myudphdr udp;
	799	memcpy(&udp, ptr, sizeof(udp));
	800	sample->dcd_sport = ntohs(udp.uh_sport);
	801	sample->dcd_dport = ntohs(udp.uh_dport);
	802	sample->udp_pduLen = ntohs(udp.uh_ulen);
	803	sf_log("UDPSrcPort %u\n", sample->dcd_sport);
	804	sf_log("UDPDstPort %u\n", sample->dcd_dport);
	805	sf_log("UDPBytes %u\n", sample->udp_pduLen);
	806	}
	807	break;
	808	default: /* some other protcol */
	809	break;
	810	}
	811	}
	812
	813	/*_________________---------------------------__________________
	814	_________________ decodeIPV4 __________________
	815	-----------------___________________________------------------
	816	*/
	817
	818	static void decodeIPV4(SFSample *sample)
	819	{
	820	if(sample->gotIPV4) {
	821	#ifdef DEBUG
	822	char buf[51];
	823	#endif
	824	u_char *ptr = sample->header + sample->offsetToIPV4;
	825	/* Create a local copy of the IP header (cannot overlay structure in case it is not quad-aligned...some
	826	platforms would core-dump if we tried that). It's OK coz this probably performs just as well anyway. */
	827	struct myiphdr ip;
	828	memcpy(&ip, ptr, sizeof(ip));
	829	/* Value copy all ip elements into sample */
	830	sample->dcd_srcIP.s_addr = ip.saddr;
	831	sample->dcd_dstIP.s_addr = ip.daddr;
	832	sample->dcd_ipProtocol = ip.protocol;
	833	sample->dcd_ipTos = ip.tos;
	834	sample->dcd_ipTTL = ip.ttl;
	835	sf_log("ip.tot_len %d\n", ntohs(ip.tot_len));
	836	/* Log out the decoded IP fields */
	837	sf_log("srcIP %s\n", IP_to_a(sample->dcd_srcIP.s_addr, buf, 51));
	838	sf_log("dstIP %s\n", IP_to_a(sample->dcd_dstIP.s_addr, buf, 51));
	839	sf_log("IPProtocol %u\n", sample->dcd_ipProtocol);
	840	sf_log("IPTOS %u\n", sample->dcd_ipTos);
	841	sf_log("IPTTL %u\n", sample->dcd_ipTTL);
	842	/* check for fragments */
	843	sample->ip_fragmentOffset = ntohs(ip.frag_off) & 0x1FFF;
	844	if(sample->ip_fragmentOffset > 0) {
	845	sf_log("IPFragmentOffset %u\n", sample->ip_fragmentOffset);
	846	}
	847	else {
	848	/* advance the pointer to the next protocol layer */
	849	/* ip headerLen is expressed as a number of quads */
	850	ptr += (ip.version_and_headerLen & 0x0f) * 4;
	851	decodeIPLayer4(sample, ptr, ip.protocol);
	852	}
	853	}
	854	}
	855
	856	/*_________________---------------------------__________________
	857	_________________ decodeIPV6 __________________
	858	-----------------___________________________------------------
	859	*/
	860
	861	static void decodeIPV6(SFSample *sample)
	862	{
	863	uint16_t payloadLen;
	864	uint32_t label;
	865	uint32_t nextHeader;
	866	u_char *end = sample->header + sample->headerLen;
	867
	868	if(sample->gotIPV6) {
	869	u_char *ptr = sample->header + sample->offsetToIPV6;
	870
	871	// check the version
	872	{
	873	int ipVersion = (*ptr >> 4);
	874	if(ipVersion != 6) {
	875	sf_log("header decode error: unexpected IP version: %d\n", ipVersion);
	876	return;
	877	}
	878	}
	879
	880	// get the tos (priority)
	881	sample->dcd_ipTos = *ptr++ & 15;
	882	sf_log("IPTOS %u\n", sample->dcd_ipTos);
	883	// 24-bit label
	884	label = *ptr++;
	885	label <<= 8;
	886	label += *ptr++;
	887	label <<= 8;
	888	label += *ptr++;
	889	sf_log("IP6_label 0x%lx\n", label);
	890	// payload
	891	payloadLen = (ptr[0] << 8) + ptr[1];
	892	ptr += 2;
	893	// if payload is zero, that implies a jumbo payload
	894	if(payloadLen == 0) sf_log("IPV6_payloadLen <jumbo>\n");
	895	else sf_log("IPV6_payloadLen %u\n", payloadLen);
	896
	897	// next header
	898	nextHeader = *ptr++;
	899
	900	// TTL
	901	sample->dcd_ipTTL = *ptr++;
	902	sf_log("IPTTL %u\n", sample->dcd_ipTTL);
	903
	904	{// src and dst address
	905	#ifdef DEBUG
	906	char buf[101];
	907	#endif
	908	sample->ipsrc.type = SFLADDRESSTYPE_IP_V6;
	909	memcpy(&sample->ipsrc.address, ptr, 16);
	910	ptr +=16;
	911	sf_log("srcIP6 %s\n", printAddress(&sample->ipsrc, buf, 100));
	912	sample->ipdst.type = SFLADDRESSTYPE_IP_V6;
	913	memcpy(&sample->ipdst.address, ptr, 16);
	914	ptr +=16;
	915	sf_log("dstIP6 %s\n", printAddress(&sample->ipdst, buf, 100));
	916	}
	917
	918	// skip over some common header extensions...
	919	// http://searchnetworking.techtarget.com/originalContent/0,289142,sid7_gci870277,00.html
	920	while(nextHeader == 0 \|\| // hop
	921	nextHeader == 43 \|\| // routing
	922	nextHeader == 44 \|\| // fragment
	923	// nextHeader == 50 \|\| // encryption - don't bother coz we'll not be able to read any further
	924	nextHeader == 51 \|\| // auth
	925	nextHeader == 60) { // destination options
	926	uint32_t optionLen, skip;
	927	sf_log("IP6HeaderExtension: %d\n", nextHeader);
	928	nextHeader = ptr[0];
	929	optionLen = 8 * (ptr[1] + 1); // second byte gives option len in 8-byte chunks, not counting first 8
	930	skip = optionLen - 2;
	931	ptr += skip;
	932	if(ptr > end) return; // ran off the end of the header
	933	}
	934
	935	// now that we have eliminated the extension headers, nextHeader should have what we want to
	936	// remember as the ip protocol...
	937	sample->dcd_ipProtocol = nextHeader;
	938	sf_log("IPProtocol %u\n", sample->dcd_ipProtocol);
	939	decodeIPLayer4(sample, ptr, sample->dcd_ipProtocol);
	940	}
	941	}
	942
	943	/*_________________---------------------------__________________
	944	_________________ StoreSflowRecord __________________
	945	-----------------___________________________------------------
	946	*/
	947
	948	static inline void StoreSflowRecord(SFSample sample, nf_buffer_t output_buffer) {
	949	common_record_t nf_record = (common_record_t )output_buffer->writeto;
	950	stat_record_t stat_record = (stat_record_t )output_buffer->stat_record;
	951	struct timeval now;
	952	void *val;
	953	uint32_t bytes, *v;
	954	uint64_t _bytes, _packets, _t; // tmp buffers
	955
	956	gettimeofday(&now, NULL);
	957
	958	// ignore fragments
	959	if( sample->ip_fragmentOffset > 0 )
	960	return;
	961
	962	// count the bytes from the start of IP header, with the exception that
	963	// for udp packets we use the udp_pduLen. This is because the udp_pduLen
	964	// can be up tp 65535 bytes, which causes fragmentation at the IP layer.
	965	// Since the sampled fragments are discarded, we have to use this field
	966	// to get the total bytes estimates right.
	967	if(sample->udp_pduLen > 0)
	968	bytes = sample->udp_pduLen;
	969	else
	970	bytes = sample->sampledPacketSize - sample->stripped - sample->offsetToIPV4;
	971
	972	nf_record->flags = 0;
	973	nf_record->mark = 0;
	974
	975	nf_record->first = now.tv_sec;
	976	nf_record->last = nf_record->first;
	977	nf_record->msec_first = now.tv_usec / 1000;
	978	nf_record->msec_last = nf_record->msec_first;
	979	_t = 1000*now.tv_sec + nf_record->msec_first; // tmp buff for first_seen
	980
	981	nf_record->input = (uint16_t)sample->inputPort;
	982	nf_record->output = (uint16_t)sample->outputPort;
	983	nf_record->srcport = (uint16_t)sample->dcd_sport;
	984	nf_record->dstport = (uint16_t)sample->dcd_dport;
	985	nf_record->dir = 0;
	986	nf_record->tcp_flags = sample->dcd_tcpFlags;
	987	nf_record->prot = sample->dcd_ipProtocol;
	988	nf_record->tos = sample->dcd_ipTos;
	989
	990	/*
	991	if(sfConfig.netFlowPeerAS) {
	992	pkt.flow.srcAS = htons((uint16_t)sample->src_peer_as);
	993	pkt.flow.dstAS = htons((uint16_t)sample->dst_peer_as);
	994	}
	995	else {
	996	pkt.flow.srcAS = htons((uint16_t)sample->src_as);
	997	pkt.flow.dstAS = htons((uint16_t)sample->dst_as);
	998	}
	999	*/
	1000	nf_record->srcas = (uint16_t)sample->src_as;
	1001	nf_record->dstas = (uint16_t)sample->dst_as;
	1002
	1003	if(sample->gotIPV6) {
	1004	ipv6_block_t addr = (ipv6_block_t )nf_record->data;
	1005	nf_record->flags = 1;
	1006	memcpy(&addr->srcaddr, &sample->ipsrc.address, 16);
	1007	memcpy(&addr->dstaddr, &sample->ipdst.address, 16);
	1008
	1009	val = (void *)((pointer_addr_t)nf_record->data + sizeof(ipv6_block_t));
	1010	} else {
	1011	uint32_t v4addr = (uint32_t )nf_record->data;
	1012	v4addr[0] = sample->dcd_srcIP.s_addr;
	1013	v4addr[1] = sample->dcd_dstIP.s_addr;
	1014	val = (void )((pointer_addr_t)nf_record->data + 2 sizeof(uint32_t));
	1015	}
	1016
	1017	// packets
	1018	v = (uint32_t *)val;
	1019	_packets = sample->meanSkipCount;
	1020	*v++ = _packets;
	1021
	1022	// bytes
	1023	_bytes = sample->meanSkipCount * bytes;
	1024	*v++ = _bytes;
	1025	val = (void *)v;
	1026
	1027	nf_record->size = (pointer_addr_t)val - (pointer_addr_t)nf_record;
	1028
	1029	// update first_seen, last_seen
	1030	if ( _t < output_buffer->first_seen ) // the very first time stamp need to be set
	1031	output_buffer->first_seen = _t;
	1032	output_buffer->last_seen = _t;
	1033
	1034	// Update stats
	1035	switch (nf_record->prot) {
	1036	case 1:
	1037	stat_record->numflows_icmp++;
	1038	stat_record->numpackets_icmp += _packets;
	1039	stat_record->numbytes_icmp += _bytes;
	1040	break;
	1041	case 6:
	1042	stat_record->numflows_tcp++;
	1043	stat_record->numpackets_tcp += _packets;
	1044	stat_record->numbytes_tcp += _bytes;
	1045	break;
	1046	case 17:
	1047	stat_record->numflows_udp++;
	1048	stat_record->numpackets_udp += _packets;
	1049	stat_record->numbytes_udp += _bytes;
	1050	break;
	1051	default:
	1052	stat_record->numflows_other++;
	1053	stat_record->numpackets_other += _packets;
	1054	stat_record->numbytes_other += _bytes;
	1055	}
	1056	stat_record->numflows++;
	1057	stat_record->numpackets += _packets;
	1058	stat_record->numbytes += _bytes;
	1059
	1060	if ( verbose ) {
	1061	master_record_t master_record;
	1062	char *string;
	1063	ExpandRecord((common_record_t *)output_buffer->writeto, &master_record);
	1064	format_file_block_record(&master_record, 1, &string, 0);
	1065	printf("%s\n", string);
	1066	}
	1067
	1068	output_buffer->writeto = (void *)((pointer_addr_t)output_buffer->writeto + nf_record->size);
	1069	output_buffer->size += nf_record->size;
	1070	output_buffer->count++;
	1071
	1072	}
	1073
	1074	/*_________________---------------------------__________________
	1075	_________________ read data fns __________________
	1076	-----------------___________________________------------------
	1077	*/
	1078
	1079	static inline uint32_t getData32(SFSample *sample) {
	1080	if ((u_char *)sample->datap > sample->endp)
	1081	SFABORT(sample, SF_ABORT_EOS);
	1082	return ntohl(*(sample->datap)++);
	1083	} // End of getData32
	1084
	1085	static inline uint32_t getData32_nobswap(SFSample *sample) {
	1086	if ((u_char *)sample->datap > sample->endp)
	1087	SFABORT(sample, SF_ABORT_EOS);
	1088	return *(sample->datap)++;
	1089	} // End of getData32_nobswap
	1090
	1091	static inline uint64_t getData64(SFSample *sample) {
	1092	uint64_t tmpLo, tmpHi;
	1093
	1094	tmpHi = getData32(sample);
	1095	tmpLo = getData32(sample);
	1096	return (tmpHi << 32) + tmpLo;
	1097	} // End of getData64
	1098
	1099	static inline void skipBytes(SFSample *sample, int skip) {
	1100	int quads = (skip + 3) / 4;
	1101
	1102	sample->datap += quads;
	1103	if ( (u_char *)sample->datap > sample->endp)
	1104	SFABORT(sample, SF_ABORT_EOS);
	1105	} // End of skipBytes
	1106
	1107	static inline uint32_t sf_log_next32(SFSample sample, char fieldName) {
	1108	uint32_t val = getData32(sample);
	1109
	1110	sf_log("%s %lu\n", fieldName, val);
	1111	return val;
	1112	} // End of sf_log_next32
	1113
	1114	static inline uint64_t sf_log_next64(SFSample sample, char fieldName) {
	1115	uint64_t val64 = getData64(sample);
	1116
	1117	sf_log("%s %llu\n", fieldName, val64);
	1118	return val64;
	1119	} // End of sf_log_next64
	1120
	1121	static inline void sf_log_percentage(SFSample sample, char fieldName) {
	1122	uint32_t hundredths = getData32(sample);
	1123
	1124	if ( hundredths == (uint32_t)-1)
	1125	sf_log("%s unknown\n", fieldName);
	1126	else {
	1127	#ifdef DEBUG
	1128	float percent = (float)hundredths / 10.0;
	1129	#endif
	1130	sf_log("%s %.1f\n", fieldName, percent);
	1131	}
	1132	} // End of sf_log_percentage
	1133
	1134
	1135	static inline uint32_t getString(SFSample sample, char buf, int bufLen) {
	1136	uint32_t len, read_len;
	1137
	1138	len = getData32(sample);
	1139	// truncate if too long
	1140	read_len = (len >= bufLen) ? (bufLen - 1) : len;
	1141	memcpy(buf, sample->datap, read_len);
	1142	buf[read_len] = '\0'; // null terminate
	1143	skipBytes(sample, len);
	1144	return len;
	1145	} // End of getString
	1146
	1147	static inline uint32_t getAddress(SFSample sample, SFLAddress address) {
	1148
	1149	address->type = getData32(sample);
	1150	if(address->type == SFLADDRESSTYPE_IP_V4)
	1151	address->address.ip_v4.s_addr = getData32_nobswap(sample);
	1152	else {
	1153	memcpy(&address->address.ip_v6.s6_addr, sample->datap, 16);
	1154	skipBytes(sample, 16);
	1155	}
	1156	return address->type;
	1157	} // End of getAddress
	1158
	1159	static inline char printTag(uint32_t tag, char buf, int bufLen) {
	1160	snprintf(buf, bufLen, "%u:%u", (tag >> 12), (tag & 0x00000FFF));
	1161	return buf;
	1162	} // End of printTag
	1163
	1164	static inline void skipTLVRecord(SFSample sample, uint32_t tag, uint32_t len, char description) {
	1165	#ifdef DEBUG
	1166	char buf[51];
	1167	#endif
	1168
	1169	sf_log("skipping unknown %s: %s len=%d\n", description, printTag(tag, buf, 50), len);
	1170	skipBytes(sample, len);
	1171	} // End of skipTLVRecord
	1172
	1173	/*_________________---------------------------__________________
	1174	_________________ readExtendedSwitch __________________
	1175	-----------------___________________________------------------
	1176	*/
	1177
	1178	static void readExtendedSwitch(SFSample *sample)
	1179	{
	1180	sf_log("extendedType SWITCH\n");
	1181	sample->in_vlan = getData32(sample);
	1182	sample->in_priority = getData32(sample);
	1183	sample->out_vlan = getData32(sample);
	1184	sample->out_priority = getData32(sample);
	1185
	1186	sample->extended_data_tag \|= SASAMPLE_EXTENDED_DATA_SWITCH;
	1187
	1188	sf_log("in_vlan %lu\n", sample->in_vlan);
	1189	sf_log("in_priority %lu\n", sample->in_priority);
	1190	sf_log("out_vlan %lu\n", sample->out_vlan);
	1191	sf_log("out_priority %lu\n", sample->out_priority);
	1192	}
	1193
	1194	/*_________________---------------------------__________________
	1195	_________________ readExtendedRouter __________________
	1196	-----------------___________________________------------------
	1197	*/
	1198
	1199	static void readExtendedRouter(SFSample *sample)
	1200	{
	1201	#ifdef DEBUG
	1202	char buf[51];
	1203	#endif
	1204
	1205	sf_log("extendedType ROUTER\n");
	1206	getAddress(sample, &sample->nextHop);
	1207	sample->srcMask = getData32(sample);
	1208	sample->dstMask = getData32(sample);
	1209
	1210	sample->extended_data_tag \|= SASAMPLE_EXTENDED_DATA_ROUTER;
	1211
	1212	sf_log("nextHop %s\n", printAddress(&sample->nextHop, buf, 50));
	1213	sf_log("srcSubnetMask %lu\n", sample->srcMask);
	1214	sf_log("dstSubnetMask %lu\n", sample->dstMask);
	1215	}
	1216
	1217	/*_________________---------------------------__________________
	1218	_________________ readExtendedGateway_v2 __________________
	1219	-----------------___________________________------------------
	1220	*/
	1221
	1222	static void readExtendedGateway_v2(SFSample *sample)
	1223	{
	1224	sf_log("extendedType GATEWAY\n");
	1225
	1226	sample->my_as = getData32(sample);
	1227	sample->src_as = getData32(sample);
	1228	sample->src_peer_as = getData32(sample);
	1229	sample->dst_as_path_len = getData32(sample);
	1230	/* just point at the dst_as_path array */
	1231	if(sample->dst_as_path_len > 0) {
	1232	sample->dst_as_path = sample->datap;
	1233	/* and skip over it in the input */
	1234	skipBytes(sample, sample->dst_as_path_len * 4);
	1235	// fill in the dst and dst_peer fields too
	1236	sample->dst_peer_as = ntohl(sample->dst_as_path[0]);
	1237	sample->dst_as = ntohl(sample->dst_as_path[sample->dst_as_path_len - 1]);
	1238	}
	1239
	1240	sample->extended_data_tag \|= SASAMPLE_EXTENDED_DATA_GATEWAY;
	1241
	1242	sf_log("my_as %lu\n", sample->my_as);
	1243	sf_log("src_as %lu\n", sample->src_as);
	1244	sf_log("src_peer_as %lu\n", sample->src_peer_as);
	1245	sf_log("dst_as %lu\n", sample->dst_as);
	1246	sf_log("dst_peer_as %lu\n", sample->dst_peer_as);
	1247	sf_log("dst_as_path_len %lu\n", sample->dst_as_path_len);
	1248	if(sample->dst_as_path_len > 0) {
	1249	uint32_t i = 0;
	1250	for(; i < sample->dst_as_path_len; i++) {
	1251	if(i == 0) sf_log("dst_as_path ");
	1252	else sf_log("-");
	1253	sf_log("%lu", ntohl(sample->dst_as_path[i]));
	1254	}
	1255	sf_log("\n");
	1256	}
	1257	}
	1258
	1259	/*_________________---------------------------__________________
	1260	_________________ readExtendedGateway __________________
	1261	-----------------___________________________------------------
	1262	*/
	1263
	1264	static void readExtendedGateway(SFSample *sample)
	1265	{
	1266	#ifdef DEBUG
	1267	char buf[51];
	1268	#endif
	1269	uint32_t segments;
	1270	int seg;
	1271
	1272	sf_log("extendedType GATEWAY\n");
	1273
	1274	if(sample->datagramVersion >= 5) {
	1275	getAddress(sample, &sample->bgp_nextHop);
	1276	sf_log("bgp_nexthop %s\n", printAddress(&sample->bgp_nextHop, buf, 50));
	1277	}
	1278
	1279	sample->my_as = getData32(sample);
	1280	sample->src_as = getData32(sample);
	1281	sample->src_peer_as = getData32(sample);
	1282	sf_log("my_as %lu\n", sample->my_as);
	1283	sf_log("src_as %lu\n", sample->src_as);
	1284	sf_log("src_peer_as %lu\n", sample->src_peer_as);
	1285	segments = getData32(sample);
	1286	if(segments > 0) {
	1287	sf_log("dst_as_path ");
	1288	for(seg = 0; seg < segments; seg++) {
	1289	uint32_t seg_type;
	1290	uint32_t seg_len;
	1291	int i;
	1292	seg_type = getData32(sample);
	1293	seg_len = getData32(sample);
	1294	for(i = 0; i < seg_len; i++) {
	1295	uint32_t asNumber;
	1296	asNumber = getData32(sample);
	1297	/* mark the first one as the dst_peer_as */
	1298	if(i == 0 && seg == 0) sample->dst_peer_as = asNumber;
	1299	else sf_log("-");
	1300	/* make sure the AS sets are in parentheses */
	1301	if(i == 0 && seg_type == SFLEXTENDED_AS_SET) sf_log("(");
	1302	sf_log("%lu", asNumber);
	1303	/* mark the last one as the dst_as */
	1304	if(seg == (segments - 1) && i == (seg_len - 1)) sample->dst_as = asNumber;
	1305	}
	1306	if(seg_type == SFLEXTENDED_AS_SET) sf_log(")");
	1307	}
	1308	sf_log("\n");
	1309	}
	1310	sf_log("dst_as %lu\n", sample->dst_as);
	1311	sf_log("dst_peer_as %lu\n", sample->dst_peer_as);
	1312
	1313	sample->communities_len = getData32(sample);
	1314	/* just point at the communities array */
	1315	if(sample->communities_len > 0) sample->communities = sample->datap;
	1316	/* and skip over it in the input */
	1317	skipBytes(sample, sample->communities_len * 4);
	1318
	1319	sample->extended_data_tag \|= SASAMPLE_EXTENDED_DATA_GATEWAY;
	1320	if(sample->communities_len > 0) {
	1321	int j = 0;
	1322	for(; j < sample->communities_len; j++) {
	1323	if(j == 0) sf_log("BGP_communities ");
	1324	else sf_log("-");
	1325	sf_log("%lu", ntohl(sample->communities[j]));
	1326	}
	1327	sf_log("\n");
	1328	}
	1329
	1330	sample->localpref = getData32(sample);
	1331	sf_log("BGP_localpref %lu\n", sample->localpref);
	1332
	1333	}
	1334
	1335	/*_________________---------------------------__________________
	1336	_________________ readExtendedUser __________________
	1337	-----------------___________________________------------------
	1338	*/
	1339
	1340	static void readExtendedUser(SFSample *sample)
	1341	{
	1342	sf_log("extendedType USER\n");
	1343
	1344	if(sample->datagramVersion >= 5) {
	1345	sample->src_user_charset = getData32(sample);
	1346	sf_log("src_user_charset %d\n", sample->src_user_charset);
	1347	}
	1348
	1349	sample->src_user_len = getString(sample, sample->src_user, SA_MAX_EXTENDED_USER_LEN);
	1350
	1351	if(sample->datagramVersion >= 5) {
	1352	sample->dst_user_charset = getData32(sample);
	1353	sf_log("dst_user_charset %d\n", sample->dst_user_charset);
	1354	}
	1355
	1356	sample->dst_user_len = getString(sample, sample->dst_user, SA_MAX_EXTENDED_USER_LEN);
	1357
	1358	sample->extended_data_tag \|= SASAMPLE_EXTENDED_DATA_USER;
	1359
	1360	sf_log("src_user %s\n", sample->src_user);
	1361	sf_log("dst_user %s\n", sample->dst_user);
	1362	}
	1363
	1364	/*_________________---------------------------__________________
	1365	_________________ readExtendedUrl __________________
	1366	-----------------___________________________------------------
	1367	*/
	1368
	1369	static void readExtendedUrl(SFSample *sample)
	1370	{
	1371	sf_log("extendedType URL\n");
	1372
	1373	sample->url_direction = getData32(sample);
	1374	sf_log("url_direction %lu\n", sample->url_direction);
	1375	sample->url_len = getString(sample, sample->url, SA_MAX_EXTENDED_URL_LEN);
	1376	sf_log("url %s\n", sample->url);
	1377	if(sample->datagramVersion >= 5) {
	1378	sample->host_len = getString(sample, sample->host, SA_MAX_EXTENDED_HOST_LEN);
	1379	sf_log("host %s\n", sample->host);
	1380	}
	1381	sample->extended_data_tag \|= SASAMPLE_EXTENDED_DATA_URL;
	1382	}
	1383
	1384
	1385	/*_________________---------------------------__________________
	1386	_________________ mplsLabelStack __________________
	1387	-----------------___________________________------------------
	1388	*/
	1389
	1390	static void mplsLabelStack(SFSample sample, char fieldName)
	1391	{
	1392	SFLLabelStack lstk;
	1393	uint32_t lab;
	1394	lstk.depth = getData32(sample);
	1395	/* just point at the lablelstack array */
	1396	if(lstk.depth > 0)
	1397	lstk.stack = (uint32_t *)sample->datap;
	1398	else
	1399	lstk.stack = NULL;
	1400	/* and skip over it in the input */
	1401	skipBytes(sample, lstk.depth * 4);
	1402
	1403	if(lstk.depth > 0) {
	1404	int j = 0;
	1405	for(; j < lstk.depth; j++) {
	1406	if(j == 0) sf_log("%s ", fieldName);
	1407	else sf_log("-");
	1408	lab = ntohl(lstk.stack[j]);
	1409	sf_log("%lu.%lu.%lu.%lu",
	1410	(lab >> 12), // label
	1411	(lab >> 9) & 7, // experimental
	1412	(lab >> 8) & 1, // bottom of stack
	1413	(lab & 255)); // TTL
	1414	}
	1415	sf_log("\n");
	1416	}
	1417	}
	1418
	1419	/*_________________---------------------------__________________
	1420	_________________ readExtendedMpls __________________
	1421	-----------------___________________________------------------
	1422	*/
	1423
	1424	static void readExtendedMpls(SFSample *sample)
	1425	{
	1426	#ifdef DEBUG
	1427	char buf[51];
	1428	#endif
	1429	sf_log("extendedType MPLS\n");
	1430	getAddress(sample, &sample->mpls_nextHop);
	1431	sf_log("mpls_nexthop %s\n", printAddress(&sample->mpls_nextHop, buf, 50));
	1432
	1433	mplsLabelStack(sample, "mpls_input_stack");
	1434	mplsLabelStack(sample, "mpls_output_stack");
	1435
	1436	sample->extended_data_tag \|= SASAMPLE_EXTENDED_DATA_MPLS;
	1437	}
	1438
	1439	/*_________________---------------------------__________________
	1440	_________________ readExtendedNat __________________
	1441	-----------------___________________________------------------
	1442	*/
	1443
	1444	static void readExtendedNat(SFSample *sample)
	1445	{
	1446	#ifdef DEBUG
	1447	char buf[51];
	1448	#endif
	1449	sf_log("extendedType NAT\n");
	1450	getAddress(sample, &sample->nat_src);
	1451	sf_log("nat_src %s\n", printAddress(&sample->nat_src, buf, 50));
	1452	getAddress(sample, &sample->nat_dst);
	1453	sf_log("nat_dst %s\n", printAddress(&sample->nat_dst, buf, 50));
	1454	sample->extended_data_tag \|= SASAMPLE_EXTENDED_DATA_NAT;
	1455	}
	1456
	1457
	1458	/*_________________---------------------------__________________
	1459	_________________ readExtendedMplsTunnel __________________
	1460	-----------------___________________________------------------
	1461	*/
	1462
	1463	static void readExtendedMplsTunnel(SFSample *sample)
	1464	{
	1465	#define SA_MAX_TUNNELNAME_LEN 100
	1466	char tunnel_name[SA_MAX_TUNNELNAME_LEN+1];
	1467	uint32_t tunnel_id, tunnel_cos;
	1468
	1469	if(getString(sample, tunnel_name, SA_MAX_TUNNELNAME_LEN) > 0)
	1470	sf_log("mpls_tunnel_lsp_name %s\n", tunnel_name);
	1471	tunnel_id = getData32(sample);
	1472	sf_log("mpls_tunnel_id %lu\n", tunnel_id);
	1473	tunnel_cos = getData32(sample);
	1474	sf_log("mpls_tunnel_cos %lu\n", tunnel_cos);
	1475	sample->extended_data_tag \|= SASAMPLE_EXTENDED_DATA_MPLS_TUNNEL;
	1476	}
	1477
	1478	/*_________________---------------------------__________________
	1479	_________________ readExtendedMplsVC __________________
	1480	-----------------___________________________------------------
	1481	*/
	1482
	1483	static void readExtendedMplsVC(SFSample *sample)
	1484	{
	1485	#define SA_MAX_VCNAME_LEN 100
	1486	char vc_name[SA_MAX_VCNAME_LEN+1];
	1487	uint32_t vll_vc_id, vc_cos;
	1488	if(getString(sample, vc_name, SA_MAX_VCNAME_LEN) > 0)
	1489	sf_log("mpls_vc_name %s\n", vc_name);
	1490	vll_vc_id = getData32(sample);
	1491	sf_log("mpls_vll_vc_id %lu\n", vll_vc_id);
	1492	vc_cos = getData32(sample);
	1493	sf_log("mpls_vc_cos %lu\n", vc_cos);
	1494	sample->extended_data_tag \|= SASAMPLE_EXTENDED_DATA_MPLS_VC;
	1495	}
	1496
	1497	/*_________________---------------------------__________________
	1498	_________________ readExtendedMplsFTN __________________
	1499	-----------------___________________________------------------
	1500	*/
	1501
	1502	static void readExtendedMplsFTN(SFSample *sample)
	1503	{
	1504	#define SA_MAX_FTN_LEN 100
	1505	char ftn_descr[SA_MAX_FTN_LEN+1];
	1506	uint32_t ftn_mask;
	1507	if(getString(sample, ftn_descr, SA_MAX_FTN_LEN) > 0)
	1508	sf_log("mpls_ftn_descr %s\n", ftn_descr);
	1509	ftn_mask = getData32(sample);
	1510	sf_log("mpls_ftn_mask %lu\n", ftn_mask);
	1511	sample->extended_data_tag \|= SASAMPLE_EXTENDED_DATA_MPLS_FTN;
	1512	}
	1513
	1514	/*_________________---------------------------__________________
	1515	_________________ readExtendedMplsLDP_FEC __________________
	1516	-----------------___________________________------------------
	1517	*/
	1518
	1519	static void readExtendedMplsLDP_FEC(SFSample *sample)
	1520	{
	1521	#ifdef DEBUG
	1522	uint32_t fec_addr_prefix_len = getData32(sample);
	1523	#endif
	1524	sf_log("mpls_fec_addr_prefix_len %lu\n", fec_addr_prefix_len);
	1525	sample->extended_data_tag \|= SASAMPLE_EXTENDED_DATA_MPLS_LDP_FEC;
	1526	}
	1527
	1528	/*_________________---------------------------__________________
	1529	_________________ readExtendedVlanTunnel __________________
	1530	-----------------___________________________------------------
	1531	*/
	1532
	1533	static void readExtendedVlanTunnel(SFSample *sample)
	1534	{
	1535	uint32_t lab;
	1536	SFLLabelStack lstk;
	1537	lstk.depth = getData32(sample);
	1538	/* just point at the lablelstack array */
	1539	if(lstk.depth > 0)
	1540	lstk.stack = (uint32_t *)sample->datap;
	1541	else
	1542	lstk.stack = NULL;
	1543	/* and skip over it in the input */
	1544	skipBytes(sample, lstk.depth * 4);
	1545
	1546	if(lstk.depth > 0) {
	1547	int j = 0;
	1548	for(; j < lstk.depth; j++) {
	1549	if(j == 0) sf_log("vlan_tunnel ");
	1550	else sf_log("-");
	1551	lab = ntohl(lstk.stack[j]);
	1552	sf_log("0x%04x.%lu.%lu.%lu",
	1553	(lab >> 16), // TPI
	1554	(lab >> 13) & 7, // priority
	1555	(lab >> 12) & 1, // CFI
	1556	(lab & 4095)); // VLAN
	1557	}
	1558	sf_log("\n");
	1559	}
	1560	sample->extended_data_tag \|= SASAMPLE_EXTENDED_DATA_VLAN_TUNNEL;
	1561	}
	1562
	1563	/*_________________---------------------------__________________
	1564	_________________ readExtendedProcess __________________
	1565	-----------------___________________________------------------
	1566	*/
	1567
	1568	static void readExtendedProcess(SFSample *sample)
	1569	{
	1570	char pname[51];
	1571	uint32_t num_processes, i;
	1572	sf_log("extendedType process\n");
	1573	num_processes = getData32(sample);
	1574	for(i = 0; i < num_processes; i++) {
	1575	#ifdef DEBUG
	1576	uint32_t pid = getData32(sample);
	1577	#endif
	1578	if(getString(sample, pname, 50) > 0) sf_log("pid %lu %s\n", pid, pname);
	1579	else sf_log("pid %lu <no_process_name>\n", pid);
	1580	}
	1581	}
	1582
	1583	/*_________________---------------------------__________________
	1584	_________________ readFlowSample_header __________________
	1585	-----------------___________________________------------------
	1586	*/
	1587
	1588	static void readFlowSample_header(SFSample *sample) {
	1589	sf_log("flowSampleType HEADER\n");
	1590	sample->headerProtocol = getData32(sample);
	1591	sf_log("headerProtocol %lu\n", sample->headerProtocol);
	1592	sample->sampledPacketSize = getData32(sample);
	1593	sf_log("sampledPacketSize %lu\n", sample->sampledPacketSize);
	1594	if(sample->datagramVersion > 4) {
	1595	// stripped count introduced in sFlow version 5
	1596	sample->stripped = getData32(sample);
	1597	sf_log("strippedBytes %lu\n", sample->stripped);
	1598	}
	1599	sample->headerLen = getData32(sample);
	1600	sf_log("headerLen %lu\n", sample->headerLen);
	1601
	1602	sample->header = (u_char )sample->datap; / just point at the header */
	1603	skipBytes(sample, sample->headerLen);
	1604	{
	1605	char scratch[2000];
	1606	printHex(sample->header, sample->headerLen, scratch, 2000, 0, 2000);
	1607	sf_log("headerBytes %s\n", scratch);
	1608	}
	1609
	1610	switch(sample->headerProtocol) {
	1611	/* the header protocol tells us where to jump into the decode */
	1612	case SFLHEADER_ETHERNET_ISO8023:
	1613	decodeLinkLayer(sample);
	1614	break;
	1615	case SFLHEADER_IPv4:
	1616	sample->gotIPV4 = YES;
	1617	sample->offsetToIPV4 = 0;
	1618	break;
	1619	case SFLHEADER_ISO88024_TOKENBUS:
	1620	case SFLHEADER_ISO88025_TOKENRING:
	1621	case SFLHEADER_FDDI:
	1622	case SFLHEADER_FRAME_RELAY:
	1623	case SFLHEADER_X25:
	1624	case SFLHEADER_PPP:
	1625	case SFLHEADER_SMDS:
	1626	case SFLHEADER_AAL5:
	1627	case SFLHEADER_AAL5_IP:
	1628	case SFLHEADER_IPv6:
	1629	case SFLHEADER_MPLS:
	1630	sf_log("NO_DECODE headerProtocol=%d\n", sample->headerProtocol);
	1631	break;
	1632	default:
	1633	syslog(LOG_ERR, "undefined headerProtocol = %d\n", sample->headerProtocol);
	1634	exit(-12);
	1635	}
	1636
	1637	if(sample->gotIPV4) {
	1638	// report the size of the original IPPdu (including the IP header)
	1639	sf_log("IPSize %d\n", sample->sampledPacketSize - sample->stripped - sample->offsetToIPV4);
	1640	decodeIPV4(sample);
	1641	}
	1642	else if(sample->gotIPV6) {
	1643	// report the size of the original IPPdu (including the IP header)
	1644	sf_log("IPSize %d\n", sample->sampledPacketSize - sample->stripped - sample->offsetToIPV6);
	1645	decodeIPV6(sample);
	1646	}
	1647
	1648	}
	1649
	1650	/*_________________---------------------------__________________
	1651	_________________ readFlowSample_ethernet __________________
	1652	-----------------___________________________------------------
	1653	*/
	1654
	1655	static void readFlowSample_ethernet(SFSample *sample)
	1656	{
	1657	u_char *p;
	1658	sf_log("flowSampleType ETHERNET\n");
	1659	sample->eth_len = getData32(sample);
	1660	memcpy(sample->eth_src, sample->datap, 6);
	1661	skipBytes(sample, 6);
	1662	memcpy(sample->eth_dst, sample->datap, 6);
	1663	skipBytes(sample, 6);
	1664	sample->eth_type = getData32(sample);
	1665	sf_log("ethernet_type %lu\n", sample->eth_type);
	1666	sf_log("ethernet_len %lu\n", sample->eth_len);
	1667	p = sample->eth_src;
	1668	sf_log("ethernet_src %02x%02x%02x%02x%02x%02x\n", p[0], p[1], p[2], p[3], p[4], p[5]);
	1669	p = sample->eth_dst;
	1670	sf_log("ethernet_dst %02x%02x%02x%02x%02x%02x\n", p[0], p[1], p[2], p[3], p[4], p[5]);
	1671	}
	1672
	1673
	1674	/*_________________---------------------------__________________
	1675	_________________ readFlowSample_IPv4 __________________
	1676	-----------------___________________________------------------
	1677	*/
	1678
	1679	static void readFlowSample_IPv4(SFSample *sample)
	1680	{
	1681	sf_log("flowSampleType IPV4\n");
	1682	sample->headerLen = sizeof(SFLSampled_ipv4);
	1683	sample->header = (u_char )sample->datap; / just point at the header */
	1684	skipBytes(sample, sample->headerLen);
	1685	{
	1686	#ifdef DEBUG
	1687	char buf[51];
	1688	#endif
	1689	SFLSampled_ipv4 nfKey;
	1690	memcpy(&nfKey, sample->header, sizeof(nfKey));
	1691	sample->sampledPacketSize = ntohl(nfKey.length);
	1692	sf_log("sampledPacketSize %lu\n", sample->sampledPacketSize);
	1693	sf_log("IPSize %d\n", sample->sampledPacketSize);
	1694	sample->dcd_srcIP = nfKey.src_ip;
	1695	sample->dcd_dstIP = nfKey.dst_ip;
	1696	sample->dcd_ipProtocol = ntohl(nfKey.protocol);
	1697	sample->dcd_ipTos = ntohl(nfKey.tos);
	1698	sf_log("srcIP %s\n", IP_to_a(sample->dcd_srcIP.s_addr, buf, 51));
	1699	sf_log("dstIP %s\n", IP_to_a(sample->dcd_dstIP.s_addr, buf, 51));
	1700	sf_log("IPProtocol %u\n", sample->dcd_ipProtocol);
	1701	sf_log("IPTOS %u\n", sample->dcd_ipTos);
	1702	sample->dcd_sport = ntohl(nfKey.src_port);
	1703	sample->dcd_dport = ntohl(nfKey.dst_port);
	1704	switch(sample->dcd_ipProtocol) {
	1705	case 1: /* ICMP */
	1706	sf_log("ICMPType %u\n", sample->dcd_dport);
	1707	/* not sure about the dest port being icmp type
	1708	- might be that src port is icmp type and dest
	1709	port is icmp code. Still, have seen some
	1710	implementations where src port is 0 and dst
	1711	port is the type, so it may be safer to
	1712	assume that the destination port has the type */
	1713	break;
	1714	case 6: /* TCP */
	1715	sf_log("TCPSrcPort %u\n", sample->dcd_sport);
	1716	sf_log("TCPDstPort %u\n", sample->dcd_dport);
	1717	sample->dcd_tcpFlags = ntohl(nfKey.tcp_flags);
	1718	sf_log("TCPFlags %u\n", sample->dcd_tcpFlags);
	1719	break;
	1720	case 17: /* UDP */
	1721	sf_log("UDPSrcPort %u\n", sample->dcd_sport);
	1722	sf_log("UDPDstPort %u\n", sample->dcd_dport);
	1723	break;
	1724	default: /* some other protcol */
	1725	break;
	1726	}
	1727	}
	1728	}
	1729
	1730	/*_________________---------------------------__________________
	1731	_________________ readFlowSample_IPv6 __________________
	1732	-----------------___________________________------------------
	1733	*/
	1734
	1735	static void readFlowSample_IPv6(SFSample *sample)
	1736	{
	1737	sf_log("flowSampleType IPV6\n");
	1738	sample->header = (u_char )sample->datap; / just point at the header */
	1739	sample->headerLen = sizeof(SFLSampled_ipv6);
	1740	skipBytes(sample, sample->headerLen);
	1741	{
	1742	SFLSampled_ipv6 nfKey6;
	1743	memcpy(&nfKey6, sample->header, sizeof(nfKey6));
	1744	sample->sampledPacketSize = ntohl(nfKey6.length);
	1745	sf_log("sampledPacketSize %lu\n", sample->sampledPacketSize);
	1746	}
	1747	/* bug: more decode to do here */
	1748	}
	1749
	1750	/*_________________---------------------------__________________
	1751	_________________ readFlowSample_v2v4 __________________
	1752	-----------------___________________________------------------
	1753	*/
	1754
	1755	static void readFlowSample_v2v4(SFSample sample, nf_buffer_t output_buffer) {
	1756	sf_log("sampleType FLOWSAMPLE\n");
	1757
	1758	sample->samplesGenerated = getData32(sample);
	1759	sf_log("sampleSequenceNo %lu\n", sample->samplesGenerated);
	1760	{
	1761	uint32_t samplerId = getData32(sample);
	1762	sample->ds_class = samplerId >> 24;
	1763	sample->ds_index = samplerId & 0x00ffffff;
	1764	sf_log("sourceId %lu:%lu\n", sample->ds_class, sample->ds_index);
	1765	}
	1766
	1767	sample->meanSkipCount = getData32(sample);
	1768	sample->samplePool = getData32(sample);
	1769	sample->dropEvents = getData32(sample);
	1770	sample->inputPort = getData32(sample);
	1771	sample->outputPort = getData32(sample);
	1772	sf_log("meanSkipCount %lu\n", sample->meanSkipCount);
	1773	sf_log("samplePool %lu\n", sample->samplePool);
	1774	sf_log("dropEvents %lu\n", sample->dropEvents);
	1775	sf_log("inputPort %lu\n", sample->inputPort);
	1776	if(sample->outputPort & 0x80000000) {
	1777	uint32_t numOutputs = sample->outputPort & 0x7fffffff;
	1778	if(numOutputs > 0) sf_log("outputPort multiple %d\n", numOutputs);
	1779	else sf_log("outputPort multiple >1\n");
	1780	}
	1781	else sf_log("outputPort %lu\n", sample->outputPort);
	1782
	1783	sample->packet_data_tag = getData32(sample);
	1784
	1785	switch(sample->packet_data_tag) {
	1786
	1787	case INMPACKETTYPE_HEADER: readFlowSample_header(sample); break;
	1788	case INMPACKETTYPE_IPV4: readFlowSample_IPv4(sample); break;
	1789	case INMPACKETTYPE_IPV6: readFlowSample_IPv6(sample); break;
	1790	default: receiveError(sample, "unexpected packet_data_tag", YES); break;
	1791	}
	1792
	1793	sample->extended_data_tag = 0;
	1794	{
	1795	uint32_t x;
	1796	sample->num_extended = getData32(sample);
	1797	for(x = 0; x < sample->num_extended; x++) {
	1798	uint32_t extended_tag;
	1799	extended_tag = getData32(sample);
	1800	switch(extended_tag) {
	1801	case INMEXTENDED_SWITCH: readExtendedSwitch(sample); break;
	1802	case INMEXTENDED_ROUTER: readExtendedRouter(sample); break;
	1803	case INMEXTENDED_GATEWAY:
	1804	if(sample->datagramVersion == 2) readExtendedGateway_v2(sample);
	1805	else readExtendedGateway(sample);
	1806	break;
	1807	case INMEXTENDED_USER: readExtendedUser(sample); break;
	1808	case INMEXTENDED_URL: readExtendedUrl(sample); break;
	1809	default: receiveError(sample, "unrecognized extended data tag", YES); break;
	1810	}
	1811	}
	1812	}
	1813
	1814	if(sample->gotIPV4 \|\| sample->gotIPV6)
	1815	StoreSflowRecord(sample, output_buffer);
	1816
	1817	/* if we are writing tcpdump format, write the next packet record now */
	1818	/* or line-by-line output... */
	1819	if ( verbose )
	1820	writeFlowLine(sample);
	1821	}
	1822
	1823	/*_________________---------------------------__________________
	1824	_________________ readFlowSample __________________
	1825	-----------------___________________________------------------
	1826	*/
	1827
	1828	static void readFlowSample(SFSample sample, int expanded, nf_buffer_t output_buffer) {
	1829	uint32_t num_elements, sampleLength;
	1830	u_char *sampleStart;
	1831
	1832	sf_log("sampleType FLOWSAMPLE\n");
	1833	sampleLength = getData32(sample);
	1834	sampleStart = (u_char *)sample->datap;
	1835	sample->samplesGenerated = getData32(sample);
	1836	sf_log("sampleSequenceNo %lu\n", sample->samplesGenerated);
	1837	if(expanded) {
	1838	sample->ds_class = getData32(sample);
	1839	sample->ds_index = getData32(sample);
	1840	}
	1841	else {
	1842	uint32_t samplerId = getData32(sample);
	1843	sample->ds_class = samplerId >> 24;
	1844	sample->ds_index = samplerId & 0x00ffffff;
	1845	}
	1846	sf_log("sourceId %lu:%lu\n", sample->ds_class, sample->ds_index);
	1847
	1848	sample->meanSkipCount = getData32(sample);
	1849	sample->samplePool = getData32(sample);
	1850	sample->dropEvents = getData32(sample);
	1851	sf_log("meanSkipCount %lu\n", sample->meanSkipCount);
	1852	sf_log("samplePool %lu\n", sample->samplePool);
	1853	sf_log("dropEvents %lu\n", sample->dropEvents);
	1854	if(expanded) {
	1855	sample->inputPortFormat = getData32(sample);
	1856	sample->inputPort = getData32(sample);
	1857	sample->outputPortFormat = getData32(sample);
	1858	sample->outputPort = getData32(sample);
	1859	}
	1860	else {
	1861	uint32_t inp, outp;
	1862	inp = getData32(sample);
	1863	outp = getData32(sample);
	1864	sample->inputPortFormat = inp >> 30;
	1865	sample->outputPortFormat = outp >> 30;
	1866	sample->inputPort = inp & 0x3fffffff;
	1867	sample->outputPort = outp & 0x3fffffff;
	1868	}
	1869	if(sample->inputPortFormat == 3) sf_log("inputPort format==3 %lu\n", sample->inputPort);
	1870	else if(sample->inputPortFormat == 2) sf_log("inputPort multiple %lu\n", sample->inputPort);
	1871	else if(sample->inputPortFormat == 1) sf_log("inputPort dropCode %lu\n", sample->inputPort);
	1872	else if(sample->inputPortFormat == 0) sf_log("inputPort %lu\n", sample->inputPort);
	1873	if(sample->outputPortFormat == 3) sf_log("outputPort format==3 %lu\n", sample->outputPort);
	1874	else if(sample->outputPortFormat == 2) sf_log("outputPort multiple %lu\n", sample->outputPort);
	1875	else if(sample->outputPortFormat == 1) sf_log("outputPort dropCode %lu\n", sample->outputPort);
	1876	else if(sample->outputPortFormat == 0) sf_log("outputPort %lu\n", sample->outputPort);
	1877
	1878	num_elements = getData32(sample);
	1879	{
	1880	int el;
	1881	for(el = 0; el < num_elements; el++) {
	1882	#ifdef DEBUG
	1883	char buf[51];
	1884	#endif
	1885	uint32_t tag, length;
	1886	u_char *start;
	1887	tag = getData32(sample);
	1888	sf_log("flowBlock_tag %s\n", printTag(tag, buf, 50));
	1889	length = getData32(sample);
	1890	start = (u_char *)sample->datap;
	1891
	1892	switch(tag) {
	1893	case SFLFLOW_HEADER: readFlowSample_header(sample); break;
	1894	case SFLFLOW_ETHERNET: readFlowSample_ethernet(sample); break;
	1895	case SFLFLOW_IPV4: readFlowSample_IPv4(sample); break;
	1896	case SFLFLOW_IPV6: readFlowSample_IPv6(sample); break;
	1897	case SFLFLOW_EX_SWITCH: readExtendedSwitch(sample); break;
	1898	case SFLFLOW_EX_ROUTER: readExtendedRouter(sample); break;
	1899	case SFLFLOW_EX_GATEWAY: readExtendedGateway(sample); break;
	1900	case SFLFLOW_EX_USER: readExtendedUser(sample); break;
	1901	case SFLFLOW_EX_URL: readExtendedUrl(sample); break;
	1902	case SFLFLOW_EX_MPLS: readExtendedMpls(sample); break;
	1903	case SFLFLOW_EX_NAT: readExtendedNat(sample); break;
	1904	case SFLFLOW_EX_MPLS_TUNNEL: readExtendedMplsTunnel(sample); break;
	1905	case SFLFLOW_EX_MPLS_VC: readExtendedMplsVC(sample); break;
	1906	case SFLFLOW_EX_MPLS_FTN: readExtendedMplsFTN(sample); break;
	1907	case SFLFLOW_EX_MPLS_LDP_FEC: readExtendedMplsLDP_FEC(sample); break;
	1908	case SFLFLOW_EX_VLAN_TUNNEL: readExtendedVlanTunnel(sample); break;
	1909	case SFLFLOW_EX_PROCESS: readExtendedProcess(sample); break;
	1910	default: skipTLVRecord(sample, tag, length, "flow_sample_element"); break;
	1911	}
	1912	lengthCheck(sample, "flow_sample_element", start, length);
	1913	}
	1914	}
	1915	lengthCheck(sample, "flow_sample", sampleStart, sampleLength);
	1916
	1917	if ( sample->gotIPV4 \|\| sample->gotIPV6 )
	1918	StoreSflowRecord(sample, output_buffer);
	1919
	1920	/* or line-by-line output... */
	1921	if ( verbose )
	1922	writeFlowLine(sample);
	1923	}
	1924
	1925	/*_________________---------------------------__________________
	1926	_________________ readCounters_generic __________________
	1927	-----------------___________________________------------------
	1928	*/
	1929
	1930	static void readCounters_generic(SFSample *sample)
	1931	{
	1932	/* the first part of the generic counters block is really just more info about the interface. */
	1933	sample->ifCounters.ifIndex = sf_log_next32(sample, "ifIndex");
	1934	sample->ifCounters.ifType = sf_log_next32(sample, "networkType");
	1935	sample->ifCounters.ifSpeed = sf_log_next64(sample, "ifSpeed");
	1936	sample->ifCounters.ifDirection = sf_log_next32(sample, "ifDirection");
	1937	sample->ifCounters.ifStatus = sf_log_next32(sample, "ifStatus");
	1938	/* the generic counters always come first */
	1939	sample->ifCounters.ifInOctets = sf_log_next64(sample, "ifInOctets");
	1940	sample->ifCounters.ifInUcastPkts = sf_log_next32(sample, "ifInUcastPkts");
	1941	sample->ifCounters.ifInMulticastPkts = sf_log_next32(sample, "ifInMulticastPkts");
	1942	sample->ifCounters.ifInBroadcastPkts = sf_log_next32(sample, "ifInBroadcastPkts");
	1943	sample->ifCounters.ifInDiscards = sf_log_next32(sample, "ifInDiscards");
	1944	sample->ifCounters.ifInErrors = sf_log_next32(sample, "ifInErrors");
	1945	sample->ifCounters.ifInUnknownProtos = sf_log_next32(sample, "ifInUnknownProtos");
	1946	sample->ifCounters.ifOutOctets = sf_log_next64(sample, "ifOutOctets");
	1947	sample->ifCounters.ifOutUcastPkts = sf_log_next32(sample, "ifOutUcastPkts");
	1948	sample->ifCounters.ifOutMulticastPkts = sf_log_next32(sample, "ifOutMulticastPkts");
	1949	sample->ifCounters.ifOutBroadcastPkts = sf_log_next32(sample, "ifOutBroadcastPkts");
	1950	sample->ifCounters.ifOutDiscards = sf_log_next32(sample, "ifOutDiscards");
	1951	sample->ifCounters.ifOutErrors = sf_log_next32(sample, "ifOutErrors");
	1952	sample->ifCounters.ifPromiscuousMode = sf_log_next32(sample, "ifPromiscuousMode");
	1953	}
	1954
	1955	/*_________________---------------------------__________________
	1956	_________________ readCounters_ethernet __________________
	1957	-----------------___________________________------------------
	1958	*/
	1959
	1960	static void readCounters_ethernet(SFSample *sample)
	1961	{
	1962	sf_log_next32(sample, "dot3StatsAlignmentErrors");
	1963	sf_log_next32(sample, "dot3StatsFCSErrors");
	1964	sf_log_next32(sample, "dot3StatsSingleCollisionFrames");
	1965	sf_log_next32(sample, "dot3StatsMultipleCollisionFrames");
	1966	sf_log_next32(sample, "dot3StatsSQETestErrors");
	1967	sf_log_next32(sample, "dot3StatsDeferredTransmissions");
	1968	sf_log_next32(sample, "dot3StatsLateCollisions");
	1969	sf_log_next32(sample, "dot3StatsExcessiveCollisions");
	1970	sf_log_next32(sample, "dot3StatsInternalMacTransmitErrors");
	1971	sf_log_next32(sample, "dot3StatsCarrierSenseErrors");
	1972	sf_log_next32(sample, "dot3StatsFrameTooLongs");
	1973	sf_log_next32(sample, "dot3StatsInternalMacReceiveErrors");
	1974	sf_log_next32(sample, "dot3StatsSymbolErrors");
	1975	}
	1976
	1977
	1978	/*_________________---------------------------__________________
	1979	_________________ readCounters_tokenring __________________
	1980	-----------------___________________________------------------
	1981	*/
	1982
	1983	static void readCounters_tokenring(SFSample *sample)
	1984	{
	1985	sf_log_next32(sample, "dot5StatsLineErrors");
	1986	sf_log_next32(sample, "dot5StatsBurstErrors");
	1987	sf_log_next32(sample, "dot5StatsACErrors");
	1988	sf_log_next32(sample, "dot5StatsAbortTransErrors");
	1989	sf_log_next32(sample, "dot5StatsInternalErrors");
	1990	sf_log_next32(sample, "dot5StatsLostFrameErrors");
	1991	sf_log_next32(sample, "dot5StatsReceiveCongestions");
	1992	sf_log_next32(sample, "dot5StatsFrameCopiedErrors");
	1993	sf_log_next32(sample, "dot5StatsTokenErrors");
	1994	sf_log_next32(sample, "dot5StatsSoftErrors");
	1995	sf_log_next32(sample, "dot5StatsHardErrors");
	1996	sf_log_next32(sample, "dot5StatsSignalLoss");
	1997	sf_log_next32(sample, "dot5StatsTransmitBeacons");
	1998	sf_log_next32(sample, "dot5StatsRecoverys");
	1999	sf_log_next32(sample, "dot5StatsLobeWires");
	2000	sf_log_next32(sample, "dot5StatsRemoves");
	2001	sf_log_next32(sample, "dot5StatsSingles");
	2002	sf_log_next32(sample, "dot5StatsFreqErrors");
	2003	}
	2004
	2005
	2006	/*_________________---------------------------__________________
	2007	_________________ readCounters_vg __________________
	2008	-----------------___________________________------------------
	2009	*/
	2010
	2011	static void readCounters_vg(SFSample *sample)
	2012	{
	2013	sf_log_next32(sample, "dot12InHighPriorityFrames");
	2014	sf_log_next64(sample, "dot12InHighPriorityOctets");
	2015	sf_log_next32(sample, "dot12InNormPriorityFrames");
	2016	sf_log_next64(sample, "dot12InNormPriorityOctets");
	2017	sf_log_next32(sample, "dot12InIPMErrors");
	2018	sf_log_next32(sample, "dot12InOversizeFrameErrors");
	2019	sf_log_next32(sample, "dot12InDataErrors");
	2020	sf_log_next32(sample, "dot12InNullAddressedFrames");
	2021	sf_log_next32(sample, "dot12OutHighPriorityFrames");
	2022	sf_log_next64(sample, "dot12OutHighPriorityOctets");
	2023	sf_log_next32(sample, "dot12TransitionIntoTrainings");
	2024	sf_log_next64(sample, "dot12HCInHighPriorityOctets");
	2025	sf_log_next64(sample, "dot12HCInNormPriorityOctets");
	2026	sf_log_next64(sample, "dot12HCOutHighPriorityOctets");
	2027	}
	2028
	2029
	2030
	2031	/*_________________---------------------------__________________
	2032	_________________ readCounters_vlan __________________
	2033	-----------------___________________________------------------
	2034	*/
	2035
	2036	static void readCounters_vlan(SFSample *sample)
	2037	{
	2038	sample->in_vlan = getData32(sample);
	2039	sf_log("in_vlan %lu\n", sample->in_vlan);
	2040	sf_log_next64(sample, "octets");
	2041	sf_log_next32(sample, "ucastPkts");
	2042	sf_log_next32(sample, "multicastPkts");
	2043	sf_log_next32(sample, "broadcastPkts");
	2044	sf_log_next32(sample, "discards");
	2045	}
	2046
	2047	/*_________________---------------------------__________________
	2048	_________________ readCounters_processor __________________
	2049	-----------------___________________________------------------
	2050	*/
	2051
	2052	static void readCounters_processor(SFSample *sample)
	2053	{
	2054	sf_log_percentage(sample, "5s_cpu");
	2055	sf_log_percentage(sample, "1m_cpu");
	2056	sf_log_percentage(sample, "5m_cpu");
	2057	sf_log_next64(sample, "total_memory_bytes");
	2058	sf_log_next64(sample, "free_memory_bytes");
	2059	}
	2060
	2061	/*_________________---------------------------__________________
	2062	_________________ readCountersSample_v2v4 __________________
	2063	-----------------___________________________------------------
	2064	*/
	2065
	2066	static void readCountersSample_v2v4(SFSample sample, nf_buffer_t output_buffer)
	2067	{
	2068	sf_log("sampleType COUNTERSSAMPLE\n");
	2069	sample->samplesGenerated = getData32(sample);
	2070	sf_log("sampleSequenceNo %lu\n", sample->samplesGenerated);
	2071	{
	2072	uint32_t samplerId = getData32(sample);
	2073	sample->ds_class = samplerId >> 24;
	2074	sample->ds_index = samplerId & 0x00ffffff;
	2075	}
	2076	sf_log("sourceId %lu:%lu\n", sample->ds_class, sample->ds_index);
	2077
	2078
	2079	sample->statsSamplingInterval = getData32(sample);
	2080	sf_log("statsSamplingInterval %lu\n", sample->statsSamplingInterval);
	2081	/* now find out what sort of counter blocks we have here... */
	2082	sample->counterBlockVersion = getData32(sample);
	2083	sf_log("counterBlockVersion %lu\n", sample->counterBlockVersion);
	2084
	2085	/* first see if we should read the generic stats */
	2086	switch(sample->counterBlockVersion) {
	2087	case INMCOUNTERSVERSION_GENERIC:
	2088	case INMCOUNTERSVERSION_ETHERNET:
	2089	case INMCOUNTERSVERSION_TOKENRING:
	2090	case INMCOUNTERSVERSION_FDDI:
	2091	case INMCOUNTERSVERSION_VG:
	2092	case INMCOUNTERSVERSION_WAN: readCounters_generic(sample); break;
	2093	case INMCOUNTERSVERSION_VLAN: break;
	2094	default: receiveError(sample, "unknown stats version", YES); break;
	2095	}
	2096
	2097	/* now see if there are any specific counter blocks to add */
	2098	switch(sample->counterBlockVersion) {
	2099	case INMCOUNTERSVERSION_GENERIC: /* nothing more */ break;
	2100	case INMCOUNTERSVERSION_ETHERNET: readCounters_ethernet(sample); break;
	2101	case INMCOUNTERSVERSION_TOKENRING:readCounters_tokenring(sample); break;
	2102	case INMCOUNTERSVERSION_FDDI: break;
	2103	case INMCOUNTERSVERSION_VG: readCounters_vg(sample); break;
	2104	case INMCOUNTERSVERSION_WAN: break;
	2105	case INMCOUNTERSVERSION_VLAN: readCounters_vlan(sample); break;
	2106	default: receiveError(sample, "unknown INMCOUNTERSVERSION", YES); break;
	2107	}
	2108	/* line-by-line output... */
	2109	writeCountersLine(sample);
	2110	}
	2111
	2112	/*_________________---------------------------__________________
	2113	_________________ readCountersSample __________________
	2114	-----------------___________________________------------------
	2115	*/
	2116
	2117	static void readCountersSample(SFSample sample, int expanded, nf_buffer_t output_buffer) {
	2118	uint32_t sampleLength;
	2119	uint32_t num_elements;
	2120	u_char *sampleStart;
	2121	sf_log("sampleType COUNTERSSAMPLE\n");
	2122	sampleLength = getData32(sample);
	2123	sampleStart = (u_char *)sample->datap;
	2124	sample->samplesGenerated = getData32(sample);
	2125
	2126	sf_log("sampleSequenceNo %lu\n", sample->samplesGenerated);
	2127	if(expanded) {
	2128	sample->ds_class = getData32(sample);
	2129	sample->ds_index = getData32(sample);
	2130	}
	2131	else {
	2132	uint32_t samplerId = getData32(sample);
	2133	sample->ds_class = samplerId >> 24;
	2134	sample->ds_index = samplerId & 0x00ffffff;
	2135	}
	2136	sf_log("sourceId %lu:%lu\n", sample->ds_class, sample->ds_index);
	2137
	2138	num_elements = getData32(sample);
	2139	{
	2140	int el;
	2141	for(el = 0; el < num_elements; el++) {
	2142	#ifdef DEBUG
	2143	char buf[51];
	2144	#endif
	2145	uint32_t tag, length;
	2146	u_char *start;
	2147	tag = getData32(sample);
	2148	sf_log("counterBlock_tag %s\n", printTag(tag, buf, 50));
	2149	length = getData32(sample);
	2150	start = (u_char *)sample->datap;
	2151
	2152	switch(tag) {
	2153	case SFLCOUNTERS_GENERIC: readCounters_generic(sample); break;
	2154	case SFLCOUNTERS_ETHERNET: readCounters_ethernet(sample); break;
	2155	case SFLCOUNTERS_TOKENRING:readCounters_tokenring(sample); break;
	2156	case SFLCOUNTERS_VG: readCounters_vg(sample); break;
	2157	case SFLCOUNTERS_VLAN: readCounters_vlan(sample); break;
	2158	case SFLCOUNTERS_PROCESSOR: readCounters_processor(sample); break;
	2159	default: skipTLVRecord(sample, tag, length, "counters_sample_element"); break;
	2160	}
	2161	lengthCheck(sample, "counters_sample_element", start, length);
	2162	}
	2163	}
	2164	lengthCheck(sample, "counters_sample", sampleStart, sampleLength);
	2165	/* line-by-line output... */
	2166	writeCountersLine(sample);
	2167	}
	2168
	2169	/*_________________---------------------------__________________
	2170	_________________ readSFlowDatagram __________________
	2171	-----------------___________________________------------------
	2172	*/
	2173
	2174	static inline void readSFlowDatagram(SFSample sample, nf_buffer_t output_buffer) {
	2175	uint32_t samplesInPacket;
	2176	uint32_t samp = 0;
	2177	struct timeval now;
	2178	#ifdef DEBUG
	2179	char buf[51];
	2180	#endif
	2181
	2182	/* log some datagram info */
	2183	now.tv_sec = time(NULL);
	2184	now.tv_usec = 0;
	2185	sf_log("datagramSourceIP %s\n", IP_to_a(sample->sourceIP.s_addr, buf, 51));
	2186	sf_log("datagramSize %lu\n", sample->rawSampleLen);
	2187	sf_log("unixSecondsUTC %lu\n", now.tv_sec);
	2188
	2189	/* check the version */
	2190	sample->datagramVersion = getData32(sample);
	2191	sf_log("datagramVersion %d\n", sample->datagramVersion);
	2192	if(sample->datagramVersion != 2 &&
	2193	sample->datagramVersion != 4 &&
	2194	sample->datagramVersion != 5) {
	2195	receiveError(sample, "unexpected datagram version number\n", YES);
	2196	}
	2197
	2198	/* get the agent address */
	2199	getAddress(sample, &sample->agent_addr);
	2200
	2201	/* version 5 has an agent sub-id as well */
	2202	if(sample->datagramVersion >= 5) {
	2203	sample->agentSubId = getData32(sample);
	2204	sf_log("agentSubId %lu\n", sample->agentSubId);
	2205	}
	2206
	2207	sample->sequenceNo = getData32(sample); /* this is the packet sequence number */
	2208	sample->sysUpTime = getData32(sample);
	2209	samplesInPacket = getData32(sample);
	2210	sf_log("agent %s\n", printAddress(&sample->agent_addr, buf, 50));
	2211	sf_log("packetSequenceNo %lu\n", sample->sequenceNo);
	2212	sf_log("sysUpTime %lu\n", sample->sysUpTime);
	2213	sf_log("samplesInPacket %lu\n", samplesInPacket);
	2214
	2215	/* now iterate and pull out the flows and counters samples */
	2216	for(; samp < samplesInPacket; samp++) {
	2217	// just read the tag, then call the approriate decode fn
	2218	sample->sampleType = getData32(sample);
	2219	sf_log("startSample ----------------------\n");
	2220	sf_log("sampleType_tag %s\n", printTag(sample->sampleType, buf, 50));
	2221	if(sample->datagramVersion >= 5) {
	2222	switch(sample->sampleType) {
	2223	case SFLFLOW_SAMPLE: readFlowSample(sample, NO, output_buffer);
	2224	break;
	2225	case SFLCOUNTERS_SAMPLE: readCountersSample(sample, NO, output_buffer);
	2226	break;
	2227	case SFLFLOW_SAMPLE_EXPANDED: readFlowSample(sample, YES, output_buffer);
	2228	break;
	2229	case SFLCOUNTERS_SAMPLE_EXPANDED: readCountersSample(sample, YES, output_buffer);
	2230	break;
	2231	default: skipTLVRecord(sample, sample->sampleType, getData32(sample), "sample");
	2232	break;
	2233	}
	2234	} else {
	2235	switch(sample->sampleType) {
	2236	case FLOWSAMPLE: readFlowSample_v2v4(sample, output_buffer);
	2237	break;
	2238	case COUNTERSSAMPLE: readCountersSample_v2v4(sample, output_buffer);
	2239	break;
	2240	default: receiveError(sample, "unexpected sample type", YES);
	2241	break;
	2242	}
	2243	}
	2244	sf_log("endSample ----------------------\n");
	2245	}
	2246	} // readSFlowDatagram
	2247
	2248	void Init_sflow(void) {
	2249
	2250	sfConfig.disableNetFlowScale = 0;
	2251	sfConfig.netFlowPeerAS = 0;
	2252
	2253	} // End of Init_sflow
	2254
	2255	void Process_sflow(void in_buff, ssize_t in_buff_cnt, data_block_header_t data_header, void writeto,
	2256	stat_record_t stat_record, uint64_t first_seen, uint64_t *last_seen) {
	2257
	2258	SFSample sample;
	2259	nf_buffer_t output_buffer;
	2260	int exceptionVal;
	2261
	2262	memset(&sample, 0, sizeof(sample));
	2263	sample.rawSample = in_buff;
	2264	sample.rawSampleLen = in_buff_cnt;
	2265	sample.sourceIP.s_addr = 0;
	2266
	2267	output_buffer.writeto = writeto;
	2268	output_buffer.stat_record = stat_record;
	2269	output_buffer.data_header = data_header;
	2270	output_buffer.first_seen = *first_seen;
	2271	output_buffer.last_seen = *last_seen;
	2272	output_buffer.size = 0;
	2273	output_buffer.count = 0;
	2274
	2275	sf_log("startDatagram =================================\n");
	2276	if((exceptionVal = setjmp(sample.env)) == 0) {
	2277	// TRY
	2278	sample.datap = (uint32_t *)sample.rawSample;
	2279	sample.endp = (u_char *)sample.rawSample + sample.rawSampleLen;
	2280	readSFlowDatagram(&sample, &output_buffer );
	2281	} else {
	2282	// CATCH
	2283	syslog(LOG_ERR, "caught exception: %d\n", exceptionVal);
	2284	}
	2285	sf_log("endDatagram =================================\n");
	2286
	2287	data_header->size += output_buffer.size;
	2288	data_header->NumBlocks += output_buffer.count;
	2289	*first_seen = output_buffer.first_seen;
	2290	*last_seen = output_buffer.last_seen;
	2291
	2292	return output_buffer.writeto;
	2293
	2294	} // End of Process_sflow

+547

-0

sflow.h less more

	0	/*
	1	* Copyright (c) 2004, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
	2	* All rights reserved.
	3	*
	4	* Redistribution and use in source and binary forms, with or without
	5	* modification, are permitted provided that the following conditions are met:
	6	*
	7	* * Redistributions of source code must retain the above copyright notice,
	8	* this list of conditions and the following disclaimer.
	9	* * Redistributions in binary form must reproduce the above copyright notice,
	10	* this list of conditions and the following disclaimer in the documentation
	11	* and/or other materials provided with the distribution.
	12	* * Neither the name of SWITCH nor the names of its contributors may be
	13	* used to endorse or promote products derived from this software without
	14	* specific prior written permission.
	15	*
	16	* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
	17	* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	18	* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	19	* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
	20	* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
	21	* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
	22	* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
	23	* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
	24	* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
	25	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
	26	* POSSIBILITY OF SUCH DAMAGE.
	27	*
	28	* $Author: peter $
	29	*
	30	* $Id: sflow.h 75 2006-05-21 15:32:48Z peter $
	31	*
	32	* $LastChangedRevision: 75 $
	33	*
	34	*
	35	*/
	36
	37	/*
	38	* sfcapd makes use of code originated from sflowtool by InMon Corp.
	39	* Those parts of the code are distributed under the InMon Public License below.
	40	* All other/additional code is pubblished under BSD license.
	41	*/
	42
	43
	44	/*
	45	* -----------------------------------------------------------------------
	46	* Copyright (c) 2001-2002 InMon Corp. All rights reserved.
	47	* -----------------------------------------------------------------------
	48	*
	49	* Redistribution and use in source and binary forms, with or without
	50	* modification, are permitted provided that the following conditions
	51	* are met:
	52	*
	53	* 1. Redistributions of source code must retain the above copyright
	54	* notice, this list of conditions and the following disclaimer.
	55	*
	56	* 2. Redistributions in binary form must reproduce the above
	57	* copyright notice, this list of conditions and the following
	58	* disclaimer in the documentation and/or other materials provided
	59	* with the distribution.
	60	*
	61	* 3. Redistributions of any form whatsoever must retain the following
	62	* acknowledgment:
	63	* "This product includes sFlow(TM), freely available from
	64	* http://www.inmon.com/".
	65	*
	66	* 4. All advertising materials mentioning features or use of this
	67	* software must display the following acknowledgment:
	68	* "This product includes sFlow(TM), freely available from
	69	* http://www.inmon.com/".
	70	*
	71	* 5. InMon Corp. may publish revised and/or new versions
	72	* of the license from time to time. Each version will be given a
	73	* distinguishing version number. Once covered code has been
	74	* published under a particular version of the license, you may
	75	* always continue to use it under the terms of that version. You
	76	* may also choose to use such covered code under the terms of any
	77	* subsequent version of the license published by InMon Corp.
	78	* No one other than the InMon Corp. has the right to modify the terms
	79	* applicable to covered code created under this License.
	80	*
	81	* 6. The name "sFlow" must not be used to endorse or promote products
	82	* derived from this software without prior written permission
	83	* from InMon Corp. This does not apply to add-on libraries or tools
	84	* that work in conjunction with sFlow. In such a case the sFlow name
	85	* may be used to indicate that the product supports sFlow.
	86	*
	87	* 7. Products derived from this software may not be called "sFlow",
	88	* nor may "sFlow" appear in their name, without prior written
	89	* permission of InMon Corp.
	90	*
	91	*
	92	* THIS SOFTWARE IS PROVIDED BY INMON CORP. ``AS IS'' AND
	93	* ANY EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
	94	* THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
	95	* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
	96	* INMON CORP. OR ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
	97	* INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
	98	* (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
	99	* SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
	100	* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
	101	* STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
	102	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
	103	* OF THE POSSIBILITY OF SUCH DAMAGE.
	104	*
	105	* --------------------------------------------------------------------
	106	*
	107	* This software consists of voluntary contributions made by many
	108	* individuals on behalf of InMon Corp.
	109	*
	110	* InMon Corp. can be contacted via Email at info@inmon.com.
	111	*
	112	* For more information on InMon Corp. and sFlow,
	113	* please see http://www.inmon.com/.
	114	*
	115	* InMon Public License Version 1.0 written May 31, 2001
	116	*
	117	*/
	118
	119
	120	enum INMAddress_type {
	121	INMADDRESSTYPE_IP_V4 = 1,
	122	INMADDRESSTYPE_IP_V6 = 2
	123	};
	124
	125	typedef union _INMAddress_value {
	126	struct in_addr ip_v4;
	127	struct in6_addr ip_v6;
	128	} INMAddress_value;
	129
	130	typedef struct _INMAddress {
	131	uint32_t type; /* enum INMAddress_type */
	132	INMAddress_value address;
	133	} INMAddress;
	134
	135	/* Packet header data */
	136
	137	#define INM_MAX_HEADER_SIZE 256 /* The maximum sampled header size. */
	138	#define INM_DEFAULT_HEADER_SIZE 128
	139	#define INM_DEFAULT_COLLECTOR_PORT 6343
	140	#define INM_DEFAULT_SAMPLING_RATE 400
	141
	142	/* The header protocol describes the format of the sampled header */
	143	enum INMHeader_protocol {
	144	INMHEADER_ETHERNET_ISO8023 = 1,
	145	INMHEADER_ISO88024_TOKENBUS = 2,
	146	INMHEADER_ISO88025_TOKENRING = 3,
	147	INMHEADER_FDDI = 4,
	148	INMHEADER_FRAME_RELAY = 5,
	149	INMHEADER_X25 = 6,
	150	INMHEADER_PPP = 7,
	151	INMHEADER_SMDS = 8,
	152	INMHEADER_AAL5 = 9,
	153	INMHEADER_AAL5_IP = 10, /* e.g. Cisco AAL5 mux */
	154	INMHEADER_IPv4 = 11,
	155	INMHEADER_IPv6 = 12
	156	};
	157
	158	typedef struct _INMSampled_header {
	159	uint32_t header_protocol; /* (enum INMHeader_protocol) */
	160	uint32_t frame_length; /* Original length of packet before sampling */
	161	uint32_t header_length; /* length of sampled header bytes to follow */
	162	uint8_t header[INM_MAX_HEADER_SIZE]; /* Header bytes */
	163	} INMSampled_header;
	164
	165	/* Packet IP version 4 data */
	166
	167	typedef struct _INMSampled_ipv4 {
	168	uint32_t length; /* The length of the IP packet
	169	excluding lower layer encapsulations */
	170	uint32_t protocol; /* IP Protocol type (for example, TCP = 6, UDP = 17) */
	171	struct in_addr src_ip; /* Source IP Address */
	172	struct in_addr dst_ip; /* Destination IP Address */
	173	uint32_t src_port; /* TCP/UDP source port number or equivalent */
	174	uint32_t dst_port; /* TCP/UDP destination port number or equivalent */
	175	uint32_t tcp_flags; /* TCP flags */
	176	uint32_t tos; /* IP type of service */
	177	} INMSampled_ipv4;
	178
	179	/* Packet IP version 6 data */
	180
	181	typedef struct _INMSampled_ipv6 {
	182	uint32_t length; /* The length of the IP packet
	183	excluding lower layer encapsulations */
	184	uint32_t protocol; /* IP Protocol type (for example, TCP = 6, UDP = 17) */
	185	struct in6_addr src_ip; /* Source IP Address */
	186	struct in6_addr dst_ip; /* Destination IP Address */
	187	uint32_t src_port; /* TCP/UDP source port number or equivalent */
	188	uint32_t dst_port; /* TCP/UDP destination port number or equivalent */
	189	uint32_t tcp_flags; /* TCP flags */
	190	uint32_t tos; /* IP type of service */
	191	} INMSampled_ipv6;
	192
	193
	194	/* Packet data */
	195
	196	enum INMPacket_information_type {
	197	INMPACKETTYPE_HEADER = 1, /* Packet headers are sampled */
	198	INMPACKETTYPE_IPV4 = 2, /* IP version 4 data */
	199	INMPACKETTYPE_IPV6 = 3 /* IP version 4 data */
	200	};
	201
	202	typedef union _INMPacket_data_type {
	203	INMSampled_header header;
	204	INMSampled_ipv4 ipv4;
	205	INMSampled_ipv6 ipv6;
	206	} INMPacket_data_type;
	207
	208	/* Extended data types */
	209
	210	/* Extended switch data */
	211
	212	typedef struct _INMExtended_switch {
	213	uint32_t src_vlan; /* The 802.1Q VLAN id of incomming frame */
	214	uint32_t src_priority; /* The 802.1p priority */
	215	uint32_t dst_vlan; /* The 802.1Q VLAN id of outgoing frame */
	216	uint32_t dst_priority; /* The 802.1p priority */
	217	} INMExtended_switch;
	218
	219	/* Extended router data */
	220
	221	typedef struct _INMExtended_router {
	222	INMAddress nexthop; /* IP address of next hop router */
	223	uint32_t src_mask; /* Source address prefix mask bits */
	224	uint32_t dst_mask; /* Destination address prefix mask bits */
	225	} INMExtended_router;
	226
	227	/* Extended gateway data */
	228
	229	enum INMExtended_as_path_segment_type {
	230	INMEXTENDED_AS_SET = 1, /* Unordered set of ASs */
	231	INMEXTENDED_AS_SEQUENCE = 2 /* Ordered sequence of ASs */
	232	};
	233
	234	typedef struct _INMExtended_as_path_segment {
	235	uint32_t type; /* enum INMExtended_as_path_segment_type */
	236	uint32_t length; /* number of AS numbers in set/sequence */
	237	union {
	238	uint32_t *set;
	239	uint32_t *seq;
	240	} as;
	241	} INMExtended_as_path_segment;
	242
	243	/* note: the INMExtended_gateway structure has changed between v2 and v4.
	244	Here is the old version first... */
	245
	246	typedef struct _INMExtended_gateway_v2 {
	247	uint32_t as; /* AS number for this gateway */
	248	uint32_t src_as; /* AS number of source (origin) */
	249	uint32_t src_peer_as; /* AS number of source peer */
	250	uint32_t dst_as_path_length; /* number of AS numbers in path */
	251	uint32_t *dst_as_path;
	252	} INMExtended_gateway_v2;
	253
	254	/* now here is the new version... */
	255
	256	typedef struct _INMExtended_gateway_v4 {
	257	uint32_t as; /* AS number for this gateway */
	258	uint32_t src_as; /* AS number of source (origin) */
	259	uint32_t src_peer_as; /* AS number of source peer */
	260	uint32_t dst_as_path_segments; /* number of segments in path */
	261	INMExtended_as_path_segment dst_as_path; / list of seqs or sets */
	262	uint32_t communities_length; /* number of communities */
	263	uint32_t communities; / set of communities */
	264	uint32_t localpref; /* LocalPref associated with this route */
	265	} INMExtended_gateway_v4;
	266
	267	/* Extended user data */
	268	typedef struct _INMExtended_user {
	269	uint32_t src_user_len;
	270	char *src_user;
	271	uint32_t dst_user_len;
	272	char *dst_user;
	273	} INMExtended_user;
	274	enum INMExtended_url_direction {
	275	INMEXTENDED_URL_SRC = 1, /* URL is associated with source address */
	276	INMEXTENDED_URL_DST = 2 /* URL is associated with destination address */
	277	};
	278
	279	typedef struct _INMExtended_url {
	280	uint32_t direction; /* enum INMExtended_url_direction */
	281	uint32_t url_len;
	282	char *url;
	283	} INMExtended_url;
	284
	285	/* Extended data */
	286
	287	enum INMExtended_information_type {
	288	INMEXTENDED_SWITCH = 1, /* Extended switch information */
	289	INMEXTENDED_ROUTER = 2, /* Extended router information */
	290	INMEXTENDED_GATEWAY = 3, /* Extended gateway router information */
	291	INMEXTENDED_USER = 4, /* Extended TACAS/RADIUS user information */
	292	INMEXTENDED_URL = 5 /* Extended URL information */
	293	};
	294
	295	/* Format of a single sample */
	296
	297	typedef struct _INMFlow_sample {
	298	uint32_t sequence_number; /* Incremented with each flow sample
	299	generated */
	300	uint32_t source_id; /* fsSourceId */
	301	uint32_t sampling_rate; /* fsPacketSamplingRate */
	302	uint32_t sample_pool; /* Total number of packets that could have been
	303	sampled (i.e. packets skipped by sampling
	304	process + total number of samples) */
	305	uint32_t drops; /* Number of times a packet was dropped due to
	306	lack of resources */
	307	uint32_t input; /* SNMP ifIndex of input interface.
	308	0 if interface is not known. */
	309	uint32_t output; /* SNMP ifIndex of output interface,
	310	0 if interface is not known.
	311	Set most significant bit to indicate
	312	multiple destination interfaces
	313	(i.e. in case of broadcast or multicast)
	314	and set lower order bits to indicate
	315	number of destination interfaces.
	316	Examples:
	317	0x00000002 indicates ifIndex = 2
	318	0x00000000 ifIndex unknown.
	319	0x80000007 indicates a packet sent
	320	to 7 interfaces.
	321	0x80000000 indicates a packet sent to
	322	an unknown number of
	323	interfaces greater than 1.*/
	324	uint32_t packet_data_tag; /* enum INMPacket_information_type */
	325	INMPacket_data_type packet_data; /* Information about sampled packet */
	326
	327	/* in the sFlow packet spec the next field is the number of extended objects
	328	followed by the data for each one (tagged with the type). Here we just
	329	provide space for each one, and flags to enable them. The correct format
	330	is then put together by the serialization code */
	331	int gotSwitch;
	332	INMExtended_switch switchDevice;
	333	int gotRouter;
	334	INMExtended_router router;
	335	int gotGateway;
	336	union {
	337	INMExtended_gateway_v2 sf_v2; /* make the version explicit so that there is */
	338	INMExtended_gateway_v4 sf_v4; /* less danger of mistakes when upgrading code */
	339	} gateway;
	340	int gotUser;
	341	INMExtended_user user;
	342	int gotUrl;
	343	INMExtended_url url;
	344	} INMFlow_sample;
	345
	346	/* Counter types */
	347
	348	/* Generic interface counters - see RFC 1573, 2233 */
	349
	350	typedef struct _INMIf_counters {
	351	uint32_t ifIndex;
	352	uint32_t ifType;
	353	uint64_t ifSpeed;
	354	uint32_t ifDirection; /* Derived from MAU MIB (RFC 2239)
	355	0 = unknown, 1 = full-duplex,
	356	2 = half-duplex, 3 = in, 4 = out */
	357	uint32_t ifStatus; /* bit field with the following bits assigned:
	358	bit 0 = ifAdminStatus (0 = down, 1 = up)
	359	bit 1 = ifOperStatus (0 = down, 1 = up) */
	360	uint64_t ifInOctets;
	361	uint32_t ifInUcastPkts;
	362	uint32_t ifInMulticastPkts;
	363	uint32_t ifInBroadcastPkts;
	364	uint32_t ifInDiscards;
	365	uint32_t ifInErrors;
	366	uint32_t ifInUnknownProtos;
	367	uint64_t ifOutOctets;
	368	uint32_t ifOutUcastPkts;
	369	uint32_t ifOutMulticastPkts;
	370	uint32_t ifOutBroadcastPkts;
	371	uint32_t ifOutDiscards;
	372	uint32_t ifOutErrors;
	373	uint32_t ifPromiscuousMode;
	374	} INMIf_counters;
	375
	376	/* Ethernet interface counters - see RFC 2358 */
	377	typedef struct _INMEthernet_specific_counters {
	378	uint32_t dot3StatsAlignmentErrors;
	379	uint32_t dot3StatsFCSErrors;
	380	uint32_t dot3StatsSingleCollisionFrames;
	381	uint32_t dot3StatsMultipleCollisionFrames;
	382	uint32_t dot3StatsSQETestErrors;
	383	uint32_t dot3StatsDeferredTransmissions;
	384	uint32_t dot3StatsLateCollisions;
	385	uint32_t dot3StatsExcessiveCollisions;
	386	uint32_t dot3StatsInternalMacTransmitErrors;
	387	uint32_t dot3StatsCarrierSenseErrors;
	388	uint32_t dot3StatsFrameTooLongs;
	389	uint32_t dot3StatsInternalMacReceiveErrors;
	390	uint32_t dot3StatsSymbolErrors;
	391	} INMEthernet_specific_counters;
	392
	393	typedef struct _INMEthernet_counters {
	394	INMIf_counters generic;
	395	INMEthernet_specific_counters ethernet;
	396	} INMEthernet_counters;
	397
	398	/* FDDI interface counters - see RFC 1512 */
	399	typedef struct _INMFddi_counters {
	400	INMIf_counters generic;
	401	} INMFddi_counters;
	402
	403	/* Token ring counters - see RFC 1748 */
	404
	405	typedef struct _INMTokenring_specific_counters {
	406	uint32_t dot5StatsLineErrors;
	407	uint32_t dot5StatsBurstErrors;
	408	uint32_t dot5StatsACErrors;
	409	uint32_t dot5StatsAbortTransErrors;
	410	uint32_t dot5StatsInternalErrors;
	411	uint32_t dot5StatsLostFrameErrors;
	412	uint32_t dot5StatsReceiveCongestions;
	413	uint32_t dot5StatsFrameCopiedErrors;
	414	uint32_t dot5StatsTokenErrors;
	415	uint32_t dot5StatsSoftErrors;
	416	uint32_t dot5StatsHardErrors;
	417	uint32_t dot5StatsSignalLoss;
	418	uint32_t dot5StatsTransmitBeacons;
	419	uint32_t dot5StatsRecoverys;
	420	uint32_t dot5StatsLobeWires;
	421	uint32_t dot5StatsRemoves;
	422	uint32_t dot5StatsSingles;
	423	uint32_t dot5StatsFreqErrors;
	424	} INMTokenring_specific_counters;
	425
	426	typedef struct _INMTokenring_counters {
	427	INMIf_counters generic;
	428	INMTokenring_specific_counters tokenring;
	429	} INMTokenring_counters;
	430
	431	/* 100 BaseVG interface counters - see RFC 2020 */
	432
	433	typedef struct _INMVg_specific_counters {
	434	uint32_t dot12InHighPriorityFrames;
	435	uint64_t dot12InHighPriorityOctets;
	436	uint32_t dot12InNormPriorityFrames;
	437	uint64_t dot12InNormPriorityOctets;
	438	uint32_t dot12InIPMErrors;
	439	uint32_t dot12InOversizeFrameErrors;
	440	uint32_t dot12InDataErrors;
	441	uint32_t dot12InNullAddressedFrames;
	442	uint32_t dot12OutHighPriorityFrames;
	443	uint64_t dot12OutHighPriorityOctets;
	444	uint32_t dot12TransitionIntoTrainings;
	445	uint64_t dot12HCInHighPriorityOctets;
	446	uint64_t dot12HCInNormPriorityOctets;
	447	uint64_t dot12HCOutHighPriorityOctets;
	448	} INMVg_specific_counters;
	449
	450	typedef struct _INMVg_counters {
	451	INMIf_counters generic;
	452	INMVg_specific_counters vg;
	453	} INMVg_counters;
	454
	455	/* WAN counters */
	456
	457	typedef struct _INMWan_counters {
	458	INMIf_counters generic;
	459	} INMWan_counters;
	460
	461	typedef struct _INMVlan_counters {
	462	uint32_t vlan_id;
	463	uint64_t octets;
	464	uint32_t ucastPkts;
	465	uint32_t multicastPkts;
	466	uint32_t broadcastPkts;
	467	uint32_t discards;
	468	} INMVlan_counters;
	469
	470	/* Counters data */
	471
	472	enum INMCounters_version {
	473	INMCOUNTERSVERSION_GENERIC = 1,
	474	INMCOUNTERSVERSION_ETHERNET = 2,
	475	INMCOUNTERSVERSION_TOKENRING = 3,
	476	INMCOUNTERSVERSION_FDDI = 4,
	477	INMCOUNTERSVERSION_VG = 5,
	478	INMCOUNTERSVERSION_WAN = 6,
	479	INMCOUNTERSVERSION_VLAN = 7
	480	};
	481
	482	typedef union _INMCounters_type {
	483	INMIf_counters generic;
	484	INMEthernet_counters ethernet;
	485	INMTokenring_counters tokenring;
	486	INMFddi_counters fddi;
	487	INMVg_counters vg;
	488	INMWan_counters wan;
	489	INMVlan_counters vlan;
	490	} INMCounters_type;
	491
	492	typedef struct _INMCounters_sample_hdr {
	493	uint32_t sequence_number; /* Incremented with each counters sample
	494	generated by this source_id */
	495	uint32_t source_id; /* fsSourceId */
	496	uint32_t sampling_interval; /* fsCounterSamplingInterval */
	497	} INMCounters_sample_hdr;
	498
	499	typedef struct _INMCounters_sample {
	500	INMCounters_sample_hdr hdr;
	501	uint32_t counters_type_tag; /* Enum INMCounters_version */
	502	INMCounters_type counters; /* Counter set for this interface type */
	503	} INMCounters_sample;
	504
	505	/* when I turn on optimisation with the Microsoft compiler it seems to change
	506	the values of these enumerated types and break the program - not sure why */
	507	enum INMSample_types {
	508	FLOWSAMPLE = 1,
	509	COUNTERSSAMPLE = 2
	510	};
	511
	512	typedef union _INMSample_type {
	513	INMFlow_sample flowsample;
	514	INMCounters_sample counterssample;
	515	} INMSample_type;
	516
	517	/* Format of a sample datagram */
	518
	519	enum INMDatagram_version {
	520	INMDATAGRAM_VERSION2 = 2,
	521	INMDATAGRAM_VERSION4 = 4
	522	};
	523
	524	typedef struct _INMSample_datagram_hdr {
	525	uint32_t datagram_version; /* (enum INMDatagram_version) = VERSION4 */
	526	INMAddress agent_address; /* IP address of sampling agent */
	527	uint32_t sequence_number; /* Incremented with each sample datagram
	528	generated */
	529	uint32_t uptime; /* Current time (in milliseconds since device
	530	last booted). Should be set as close to
	531	datagram transmission time as possible.*/
	532	uint32_t num_samples; /* Number of flow and counters samples to follow */
	533	} INMSample_datagram_hdr;
	534
	535	#define INM_MAX_DATAGRAM_SIZE 1500
	536	#define INM_MIN_DATAGRAM_SIZE 200
	537	#define INM_DEFAULT_DATAGRAM_SIZE 1400
	538
	539	#define INM_DATA_PAD 400
	540
	541	void Init_sflow(void);
	542
	543	void Process_sflow(void in_buff, ssize_t in_buff_cnt, data_block_header_t data_header, void writeto,
	544	stat_record_t stat_record, uint64_t first_seen, uint64_t *last_seen);
	545
	546

+700

-0

sflow_proto.h less more

	0	/*
	1	* Copyright (c) 2004, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
	2	* All rights reserved.
	3	*
	4	* Redistribution and use in source and binary forms, with or without
	5	* modification, are permitted provided that the following conditions are met:
	6	*
	7	* * Redistributions of source code must retain the above copyright notice,
	8	* this list of conditions and the following disclaimer.
	9	* * Redistributions in binary form must reproduce the above copyright notice,
	10	* this list of conditions and the following disclaimer in the documentation
	11	* and/or other materials provided with the distribution.
	12	* * Neither the name of SWITCH nor the names of its contributors may be
	13	* used to endorse or promote products derived from this software without
	14	* specific prior written permission.
	15	*
	16	* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
	17	* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	18	* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	19	* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
	20	* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
	21	* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
	22	* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
	23	* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
	24	* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
	25	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
	26	* POSSIBILITY OF SUCH DAMAGE.
	27	*
	28	* $Author: peter $
	29	*
	30	* $Id: sflow_proto.h 75 2006-05-21 15:32:48Z peter $
	31	*
	32	* $LastChangedRevision: 75 $
	33	*
	34	*
	35	*/
	36
	37	/*
	38	* sfcapd makes use of code originated from sflowtool by InMon Corp.
	39	* Those parts of the code are distributed under the InMon Public License below.
	40	* All other/additional code is pubblished under BSD license.
	41	*/
	42
	43
	44	/*
	45	* -----------------------------------------------------------------------
	46	* Copyright (c) 2001-2002 InMon Corp. All rights reserved.
	47	* -----------------------------------------------------------------------
	48	*
	49	* Redistribution and use in source and binary forms, with or without
	50	* modification, are permitted provided that the following conditions
	51	* are met:
	52	*
	53	* 1. Redistributions of source code must retain the above copyright
	54	* notice, this list of conditions and the following disclaimer.
	55	*
	56	* 2. Redistributions in binary form must reproduce the above
	57	* copyright notice, this list of conditions and the following
	58	* disclaimer in the documentation and/or other materials provided
	59	* with the distribution.
	60	*
	61	* 3. Redistributions of any form whatsoever must retain the following
	62	* acknowledgment:
	63	* "This product includes sFlow(TM), freely available from
	64	* http://www.inmon.com/".
	65	*
	66	* 4. All advertising materials mentioning features or use of this
	67	* software must display the following acknowledgment:
	68	* "This product includes sFlow(TM), freely available from
	69	* http://www.inmon.com/".
	70	*
	71	* 5. InMon Corp. may publish revised and/or new versions
	72	* of the license from time to time. Each version will be given a
	73	* distinguishing version number. Once covered code has been
	74	* published under a particular version of the license, you may
	75	* always continue to use it under the terms of that version. You
	76	* may also choose to use such covered code under the terms of any
	77	* subsequent version of the license published by InMon Corp.
	78	* No one other than the InMon Corp. has the right to modify the terms
	79	* applicable to covered code created under this License.
	80	*
	81	* 6. The name "sFlow" must not be used to endorse or promote products
	82	* derived from this software without prior written permission
	83	* from InMon Corp. This does not apply to add-on libraries or tools
	84	* that work in conjunction with sFlow. In such a case the sFlow name
	85	* may be used to indicate that the product supports sFlow.
	86	*
	87	* 7. Products derived from this software may not be called "sFlow",
	88	* nor may "sFlow" appear in their name, without prior written
	89	* permission of InMon Corp.
	90	*
	91	*
	92	* THIS SOFTWARE IS PROVIDED BY INMON CORP. ``AS IS'' AND
	93	* ANY EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
	94	* THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
	95	* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
	96	* INMON CORP. OR ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
	97	* INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
	98	* (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
	99	* SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
	100	* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
	101	* STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
	102	* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
	103	* OF THE POSSIBILITY OF SUCH DAMAGE.
	104	*
	105	* --------------------------------------------------------------------
	106	*
	107	* This software consists of voluntary contributions made by many
	108	* individuals on behalf of InMon Corp.
	109	*
	110	* InMon Corp. can be contacted via Email at info@inmon.com.
	111	*
	112	* For more information on InMon Corp. and sFlow,
	113	* please see http://www.inmon.com/.
	114	*
	115	* InMon Public License Version 1.0 written May 31, 2001
	116	*
	117	*/
	118
	119	/////////////////////////////////////////////////////////////////////////////////
	120	/////////////////////// sFlow Sampling Packet Data Types ////////////////////////
	121	/////////////////////////////////////////////////////////////////////////////////
	122
	123	#ifndef SFLOW_H
	124	#define SFLOW_H 1
	125
	126	#if defined(__cplusplus)
	127	extern "C" {
	128	#endif
	129
	130	enum SFLAddress_type {
	131	SFLADDRESSTYPE_IP_V4 = 1,
	132	SFLADDRESSTYPE_IP_V6 = 2
	133	};
	134
	135	typedef union _SFLAddress_value {
	136	struct in_addr ip_v4;
	137	struct in6_addr ip_v6;
	138	} SFLAddress_value;
	139
	140	typedef struct _SFLAddress {
	141	uint32_t type; /* enum SFLAddress_type */
	142	SFLAddress_value address;
	143	} SFLAddress;
	144
	145	/* Packet header data */
	146
	147	#define SFL_DEFAULT_HEADER_SIZE 128
	148	#define SFL_DEFAULT_COLLECTOR_PORT 6343
	149	#define SFL_DEFAULT_SAMPLING_RATE 400
	150
	151	/* The header protocol describes the format of the sampled header */
	152	enum SFLHeader_protocol {
	153	SFLHEADER_ETHERNET_ISO8023 = 1,
	154	SFLHEADER_ISO88024_TOKENBUS = 2,
	155	SFLHEADER_ISO88025_TOKENRING = 3,
	156	SFLHEADER_FDDI = 4,
	157	SFLHEADER_FRAME_RELAY = 5,
	158	SFLHEADER_X25 = 6,
	159	SFLHEADER_PPP = 7,
	160	SFLHEADER_SMDS = 8,
	161	SFLHEADER_AAL5 = 9,
	162	SFLHEADER_AAL5_IP = 10, /* e.g. Cisco AAL5 mux */
	163	SFLHEADER_IPv4 = 11,
	164	SFLHEADER_IPv6 = 12,
	165	SFLHEADER_MPLS = 13
	166	};
	167
	168	/* raw sampled header */
	169
	170	typedef struct _SFLSampled_header {
	171	uint32_t header_protocol; /* (enum SFLHeader_protocol) */
	172	uint32_t frame_length; /* Original length of packet before sampling */
	173	uint32_t stripped; /* header/trailer bytes stripped by sender */
	174	uint32_t header_length; /* length of sampled header bytes to follow */
	175	uint8_t header_bytes; / Header bytes */
	176	} SFLSampled_header;
	177
	178	/* decoded ethernet header */
	179
	180	typedef struct _SFLSampled_ethernet {
	181	uint32_t eth_len; /* The length of the MAC packet excluding
	182	lower layer encapsulations */
	183	uint8_t src_mac[8]; /* 6 bytes + 2 pad */
	184	uint8_t dst_mac[8];
	185	uint32_t eth_type;
	186	} SFLSampled_ethernet;
	187
	188	/* decoded IP version 4 header */
	189
	190	typedef struct _SFLSampled_ipv4 {
	191	uint32_t length; /* The length of the IP packet
	192	excluding lower layer encapsulations */
	193	uint32_t protocol; /* IP Protocol type (for example, TCP = 6, UDP = 17) */
	194	struct in_addr src_ip; /* Source IP Address */
	195	struct in_addr dst_ip; /* Destination IP Address */
	196	uint32_t src_port; /* TCP/UDP source port number or equivalent */
	197	uint32_t dst_port; /* TCP/UDP destination port number or equivalent */
	198	uint32_t tcp_flags; /* TCP flags */
	199	uint32_t tos; /* IP type of service */
	200	} SFLSampled_ipv4;
	201
	202	/* decoded IP version 6 data */
	203
	204	typedef struct _SFLSampled_ipv6 {
	205	uint32_t length; /* The length of the IP packet
	206	excluding lower layer encapsulations */
	207	uint32_t protocol; /* IP Protocol type (for example, TCP = 6, UDP = 17) */
	208	struct in6_addr src_ip; /* Source IP Address */
	209	struct in6_addr dst_ip; /* Destination IP Address */
	210	uint32_t src_port; /* TCP/UDP source port number or equivalent */
	211	uint32_t dst_port; /* TCP/UDP destination port number or equivalent */
	212	uint32_t tcp_flags; /* TCP flags */
	213	uint32_t priority; /* IP priority */
	214	} SFLSampled_ipv6;
	215
	216	/* Extended data types */
	217
	218	/* Extended switch data */
	219
	220	typedef struct _SFLExtended_switch {
	221	uint32_t src_vlan; /* The 802.1Q VLAN id of incomming frame */
	222	uint32_t src_priority; /* The 802.1p priority */
	223	uint32_t dst_vlan; /* The 802.1Q VLAN id of outgoing frame */
	224	uint32_t dst_priority; /* The 802.1p priority */
	225	} SFLExtended_switch;
	226
	227	/* Extended router data */
	228
	229	typedef struct _SFLExtended_router {
	230	SFLAddress nexthop; /* IP address of next hop router */
	231	uint32_t src_mask; /* Source address prefix mask bits */
	232	uint32_t dst_mask; /* Destination address prefix mask bits */
	233	} SFLExtended_router;
	234
	235	/* Extended gateway data */
	236	enum SFLExtended_as_path_segment_type {
	237	SFLEXTENDED_AS_SET = 1, /* Unordered set of ASs */
	238	SFLEXTENDED_AS_SEQUENCE = 2 /* Ordered sequence of ASs */
	239	};
	240
	241	typedef struct _SFLExtended_as_path_segment {
	242	uint32_t type; /* enum SFLExtended_as_path_segment_type */
	243	uint32_t length; /* number of AS numbers in set/sequence */
	244	union {
	245	uint32_t *set;
	246	uint32_t *seq;
	247	} as;
	248	} SFLExtended_as_path_segment;
	249
	250	typedef struct _SFLExtended_gateway {
	251	SFLAddress nexthop; /* Address of the border router that should
	252	be used for the destination network */
	253	uint32_t as; /* AS number for this gateway */
	254	uint32_t src_as; /* AS number of source (origin) */
	255	uint32_t src_peer_as; /* AS number of source peer */
	256	uint32_t dst_as_path_segments; /* number of segments in path */
	257	SFLExtended_as_path_segment dst_as_path; / list of seqs or sets */
	258	uint32_t communities_length; /* number of communities */
	259	uint32_t communities; / set of communities */
	260	uint32_t localpref; /* LocalPref associated with this route */
	261	} SFLExtended_gateway;
	262
	263	typedef struct _SFLString {
	264	uint32_t len;
	265	char *str;
	266	} SFLString;
	267
	268	/* Extended user data */
	269
	270	typedef struct _SFLExtended_user {
	271	uint32_t src_charset; /* MIBEnum value of character set used to encode a string - See RFC 2978
	272	Where possible UTF-8 encoding (MIBEnum=106) should be used. A value
	273	of zero indicates an unknown encoding. */
	274	SFLString src_user;
	275	uint32_t dst_charset;
	276	SFLString dst_user;
	277	} SFLExtended_user;
	278
	279	/* Extended URL data */
	280
	281	enum SFLExtended_url_direction {
	282	SFLEXTENDED_URL_SRC = 1, /* URL is associated with source address */
	283	SFLEXTENDED_URL_DST = 2 /* URL is associated with destination address */
	284	};
	285
	286	typedef struct _SFLExtended_url {
	287	uint32_t direction; /* enum SFLExtended_url_direction */
	288	SFLString url; /* URL associated with the packet flow.
	289	Must be URL encoded */
	290	SFLString host; /* The host field from the HTTP header */
	291	} SFLExtended_url;
	292
	293	/* Extended MPLS data */
	294
	295	typedef struct _SFLLabelStack {
	296	uint32_t depth;
	297	uint32_t stack; / first entry is top of stack - see RFC 3032 for encoding */
	298	} SFLLabelStack;
	299
	300	typedef struct _SFLExtended_mpls {
	301	SFLAddress nextHop; /* Address of the next hop */
	302	SFLLabelStack in_stack;
	303	SFLLabelStack out_stack;
	304	} SFLExtended_mpls;
	305
	306	/* Extended NAT data
	307	Packet header records report addresses as seen at the sFlowDataSource.
	308	The extended_nat structure reports on translated source and/or destination
	309	addesses for this packet. If an address was not translated it should
	310	be equal to that reported for the header. */
	311
	312	typedef struct _SFLExtended_nat {
	313	SFLAddress src; /* Source address */
	314	SFLAddress dst; /* Destination address */
	315	} SFLExtended_nat;
	316
	317	/* additional Extended MPLS stucts */
	318
	319	typedef struct _SFLExtended_mpls_tunnel {
	320	SFLString tunnel_lsp_name; /* Tunnel name */
	321	uint32_t tunnel_id; /* Tunnel ID */
	322	uint32_t tunnel_cos; /* Tunnel COS value */
	323	} SFLExtended_mpls_tunnel;
	324
	325	typedef struct _SFLExtended_mpls_vc {
	326	SFLString vc_instance_name; /* VC instance name */
	327	uint32_t vll_vc_id; /* VLL/VC instance ID */
	328	uint32_t vc_label_cos; /* VC Label COS value */
	329	} SFLExtended_mpls_vc;
	330
	331	/* Extended MPLS FEC
	332	- Definitions from MPLS-FTN-STD-MIB mplsFTNTable */
	333
	334	typedef struct _SFLExtended_mpls_FTN {
	335	SFLString mplsFTNDescr;
	336	uint32_t mplsFTNMask;
	337	} SFLExtended_mpls_FTN;
	338
	339	/* Extended MPLS LVP FEC
	340	- Definition from MPLS-LDP-STD-MIB mplsFecTable
	341	Note: mplsFecAddrType, mplsFecAddr information available
	342	from packet header */
	343
	344	typedef struct _SFLExtended_mpls_LDP_FEC {
	345	uint32_t mplsFecAddrPrefixLength;
	346	} SFLExtended_mpls_LDP_FEC;
	347
	348	/* Extended VLAN tunnel information
	349	Record outer VLAN encapsulations that have
	350	been stripped. extended_vlantunnel information
	351	should only be reported if all the following conditions are satisfied:
	352	1. The packet has nested vlan tags, AND
	353	2. The reporting device is VLAN aware, AND
	354	3. One or more VLAN tags have been stripped, either
	355	because they represent proprietary encapsulations, or
	356	because switch hardware automatically strips the outer VLAN
	357	encapsulation.
	358	Reporting extended_vlantunnel information is not a substitute for
	359	reporting extended_switch information. extended_switch data must
	360	always be reported to describe the ingress/egress VLAN information
	361	for the packet. The extended_vlantunnel information only applies to
	362	nested VLAN tags, and then only when one or more tags has been
	363	stripped. */
	364
	365	typedef SFLLabelStack SFLVlanStack;
	366	typedef struct _SFLExtended_vlan_tunnel {
	367	SFLVlanStack stack; /* List of stripped 802.1Q TPID/TCI layers. Each
	368	TPID,TCI pair is represented as a single 32 bit
	369	integer. Layers listed from outermost to
	370	innermost. */
	371	} SFLExtended_vlan_tunnel;
	372
	373
	374	////////////////// InMon Extension structs //////////////////////////
	375
	376	typedef struct _SFLProcess {
	377	uint32_t pid;
	378	SFLString command;
	379	} SFLProcess;
	380
	381	#define SFL_MAX_PROCESSES 10
	382	typedef struct _SFLExtended_process {
	383	uint32_t num_processes;
	384	SFLProcess processes[SFL_MAX_PROCESSES];
	385	} SFLExtended_process;
	386
	387	//////////////////////////////////////////////////////////////////////
	388
	389	enum SFLFlow_type_tag {
	390	/* enterprise = 0, format = ... */
	391	SFLFLOW_HEADER = 1, /* Packet headers are sampled */
	392	SFLFLOW_ETHERNET = 2, /* MAC layer information */
	393	SFLFLOW_IPV4 = 3, /* IP version 4 data */
	394	SFLFLOW_IPV6 = 4, /* IP version 6 data */
	395	SFLFLOW_EX_SWITCH = 1001, /* Extended switch information */
	396	SFLFLOW_EX_ROUTER = 1002, /* Extended router information */
	397	SFLFLOW_EX_GATEWAY = 1003, /* Extended gateway router information */
	398	SFLFLOW_EX_USER = 1004, /* Extended TACAS/RADIUS user information */
	399	SFLFLOW_EX_URL = 1005, /* Extended URL information */
	400	SFLFLOW_EX_MPLS = 1006, /* Extended MPLS information */
	401	SFLFLOW_EX_NAT = 1007, /* Extended NAT information */
	402	SFLFLOW_EX_MPLS_TUNNEL = 1008, /* additional MPLS information */
	403	SFLFLOW_EX_MPLS_VC = 1009,
	404	SFLFLOW_EX_MPLS_FTN = 1010,
	405	SFLFLOW_EX_MPLS_LDP_FEC = 1011,
	406	SFLFLOW_EX_VLAN_TUNNEL = 1012, /* VLAN stack */
	407	/* enterprise = 4300 (inmon)...*/
	408	SFLFLOW_EX_PROCESS = (4300 << 12) + 3, /* =17612803 Extended Process information */
	409	};
	410
	411	typedef union _SFLFlow_type {
	412	SFLSampled_header header;
	413	SFLSampled_ethernet ethernet;
	414	SFLSampled_ipv4 ipv4;
	415	SFLSampled_ipv6 ipv6;
	416	SFLExtended_switch sw;
	417	SFLExtended_router router;
	418	SFLExtended_gateway gateway;
	419	SFLExtended_user user;
	420	SFLExtended_url url;
	421	SFLExtended_mpls mpls;
	422	SFLExtended_nat nat;
	423	SFLExtended_mpls_tunnel mpls_tunnel;
	424	SFLExtended_mpls_vc mpls_vc;
	425	SFLExtended_mpls_FTN mpls_ftn;
	426	SFLExtended_mpls_LDP_FEC mpls_ldp_fec;
	427	SFLExtended_vlan_tunnel vlan_tunnel;
	428	// extensions
	429	SFLExtended_process process;
	430	} SFLFlow_type;
	431
	432	typedef struct _SFLFlow_sample_element {
	433	struct _SFLFlow_sample_element *nxt;
	434	uint32_t tag; /* SFLFlow_type_tag */
	435	uint32_t length;
	436	SFLFlow_type flowType;
	437	} SFLFlow_sample_element;
	438
	439	enum SFL_sample_tag {
	440	SFLFLOW_SAMPLE = 1, /* enterprise = 0 : format = 1 */
	441	SFLCOUNTERS_SAMPLE = 2, /* enterprise = 0 : format = 2 */
	442	SFLFLOW_SAMPLE_EXPANDED = 3, /* enterprise = 0 : format = 3 */
	443	SFLCOUNTERS_SAMPLE_EXPANDED = 4 /* enterprise = 0 : format = 4 */
	444	};
	445
	446	/* Format of a single flow sample */
	447
	448	typedef struct _SFLFlow_sample {
	449	/* uint32_t tag; / / SFL_sample_tag -- enterprise = 0 : format = 1 */
	450	/* uint32_t length; */
	451	uint32_t sequence_number; /* Incremented with each flow sample
	452	generated */
	453	uint32_t source_id; /* fsSourceId */
	454	uint32_t sampling_rate; /* fsPacketSamplingRate */
	455	uint32_t sample_pool; /* Total number of packets that could have been
	456	sampled (i.e. packets skipped by sampling
	457	process + total number of samples) */
	458	uint32_t drops; /* Number of times a packet was dropped due to
	459	lack of resources */
	460	uint32_t input; /* SNMP ifIndex of input interface.
	461	0 if interface is not known. */
	462	uint32_t output; /* SNMP ifIndex of output interface,
	463	0 if interface is not known.
	464	Set most significant bit to indicate
	465	multiple destination interfaces
	466	(i.e. in case of broadcast or multicast)
	467	and set lower order bits to indicate
	468	number of destination interfaces.
	469	Examples:
	470	0x00000002 indicates ifIndex = 2
	471	0x00000000 ifIndex unknown.
	472	0x80000007 indicates a packet sent
	473	to 7 interfaces.
	474	0x80000000 indicates a packet sent to
	475	an unknown number of
	476	interfaces greater than 1.*/
	477	uint32_t num_elements;
	478	SFLFlow_sample_element *elements;
	479	} SFLFlow_sample;
	480
	481	/* same thing, but the expanded version (for full 32-bit ifIndex numbers) */
	482
	483	typedef struct _SFLFlow_sample_expanded {
	484	/* uint32_t tag; / / SFL_sample_tag -- enterprise = 0 : format = 1 */
	485	/* uint32_t length; */
	486	uint32_t sequence_number; /* Incremented with each flow sample
	487	generated */
	488	uint32_t ds_class; /* EXPANDED */
	489	uint32_t ds_index; /* EXPANDED */
	490	uint32_t sampling_rate; /* fsPacketSamplingRate */
	491	uint32_t sample_pool; /* Total number of packets that could have been
	492	sampled (i.e. packets skipped by sampling
	493	process + total number of samples) */
	494	uint32_t drops; /* Number of times a packet was dropped due to
	495	lack of resources */
	496	uint32_t inputFormat; /* EXPANDED */
	497	uint32_t input; /* SNMP ifIndex of input interface.
	498	0 if interface is not known. */
	499	uint32_t outputFormat; /* EXPANDED */
	500	uint32_t output; /* SNMP ifIndex of output interface,
	501	0 if interface is not known. */
	502	uint32_t num_elements;
	503	SFLFlow_sample_element *elements;
	504	} SFLFlow_sample_expanded;
	505
	506	/* Counter types */
	507
	508	/* Generic interface counters - see RFC 1573, 2233 */
	509
	510	typedef struct _SFLIf_counters {
	511	uint32_t ifIndex;
	512	uint32_t ifType;
	513	uint64_t ifSpeed;
	514	uint32_t ifDirection; /* Derived from MAU MIB (RFC 2668)
	515	0 = unknown, 1 = full-duplex,
	516	2 = half-duplex, 3 = in, 4 = out */
	517	uint32_t ifStatus; /* bit field with the following bits assigned:
	518	bit 0 = ifAdminStatus (0 = down, 1 = up)
	519	bit 1 = ifOperStatus (0 = down, 1 = up) */
	520	uint64_t ifInOctets;
	521	uint32_t ifInUcastPkts;
	522	uint32_t ifInMulticastPkts;
	523	uint32_t ifInBroadcastPkts;
	524	uint32_t ifInDiscards;
	525	uint32_t ifInErrors;
	526	uint32_t ifInUnknownProtos;
	527	uint64_t ifOutOctets;
	528	uint32_t ifOutUcastPkts;
	529	uint32_t ifOutMulticastPkts;
	530	uint32_t ifOutBroadcastPkts;
	531	uint32_t ifOutDiscards;
	532	uint32_t ifOutErrors;
	533	uint32_t ifPromiscuousMode;
	534	} SFLIf_counters;
	535
	536	/* Ethernet interface counters - see RFC 2358 */
	537	typedef struct _SFLEthernet_counters {
	538	uint32_t dot3StatsAlignmentErrors;
	539	uint32_t dot3StatsFCSErrors;
	540	uint32_t dot3StatsSingleCollisionFrames;
	541	uint32_t dot3StatsMultipleCollisionFrames;
	542	uint32_t dot3StatsSQETestErrors;
	543	uint32_t dot3StatsDeferredTransmissions;
	544	uint32_t dot3StatsLateCollisions;
	545	uint32_t dot3StatsExcessiveCollisions;
	546	uint32_t dot3StatsInternalMacTransmitErrors;
	547	uint32_t dot3StatsCarrierSenseErrors;
	548	uint32_t dot3StatsFrameTooLongs;
	549	uint32_t dot3StatsInternalMacReceiveErrors;
	550	uint32_t dot3StatsSymbolErrors;
	551	} SFLEthernet_counters;
	552
	553	/* Token ring counters - see RFC 1748 */
	554
	555	typedef struct _SFLTokenring_counters {
	556	uint32_t dot5StatsLineErrors;
	557	uint32_t dot5StatsBurstErrors;
	558	uint32_t dot5StatsACErrors;
	559	uint32_t dot5StatsAbortTransErrors;
	560	uint32_t dot5StatsInternalErrors;
	561	uint32_t dot5StatsLostFrameErrors;
	562	uint32_t dot5StatsReceiveCongestions;
	563	uint32_t dot5StatsFrameCopiedErrors;
	564	uint32_t dot5StatsTokenErrors;
	565	uint32_t dot5StatsSoftErrors;
	566	uint32_t dot5StatsHardErrors;
	567	uint32_t dot5StatsSignalLoss;
	568	uint32_t dot5StatsTransmitBeacons;
	569	uint32_t dot5StatsRecoverys;
	570	uint32_t dot5StatsLobeWires;
	571	uint32_t dot5StatsRemoves;
	572	uint32_t dot5StatsSingles;
	573	uint32_t dot5StatsFreqErrors;
	574	} SFLTokenring_counters;
	575
	576	/* 100 BaseVG interface counters - see RFC 2020 */
	577
	578	typedef struct _SFLVg_counters {
	579	uint32_t dot12InHighPriorityFrames;
	580	uint64_t dot12InHighPriorityOctets;
	581	uint32_t dot12InNormPriorityFrames;
	582	uint64_t dot12InNormPriorityOctets;
	583	uint32_t dot12InIPMErrors;
	584	uint32_t dot12InOversizeFrameErrors;
	585	uint32_t dot12InDataErrors;
	586	uint32_t dot12InNullAddressedFrames;
	587	uint32_t dot12OutHighPriorityFrames;
	588	uint64_t dot12OutHighPriorityOctets;
	589	uint32_t dot12TransitionIntoTrainings;
	590	uint64_t dot12HCInHighPriorityOctets;
	591	uint64_t dot12HCInNormPriorityOctets;
	592	uint64_t dot12HCOutHighPriorityOctets;
	593	} SFLVg_counters;
	594
	595	typedef struct _SFLVlan_counters {
	596	uint32_t vlan_id;
	597	uint64_t octets;
	598	uint32_t ucastPkts;
	599	uint32_t multicastPkts;
	600	uint32_t broadcastPkts;
	601	uint32_t discards;
	602	} SFLVlan_counters;
	603
	604	/* Processor Information */
	605	/* opaque = counter_data; enterprise = 0; format = 1001 */
	606
	607	typedef struct _SFLProcessor_counters {
	608	uint32_t five_sec_cpu; /* 5 second average CPU utilization */
	609	uint32_t one_min_cpu; /* 1 minute average CPU utilization */
	610	uint32_t five_min_cpu; /* 5 minute average CPU utilization */
	611	uint64_t total_memory; /* total memory (in bytes) */
	612	uint64_t free_memory; /* free memory (in bytes) */
	613	} SFLProcessor_counters;
	614
	615	/* Counters data */
	616
	617	enum SFLCounters_type_tag {
	618	/* enterprise = 0, format = ... */
	619	SFLCOUNTERS_GENERIC = 1,
	620	SFLCOUNTERS_ETHERNET = 2,
	621	SFLCOUNTERS_TOKENRING = 3,
	622	SFLCOUNTERS_VG = 4,
	623	SFLCOUNTERS_VLAN = 5,
	624	SFLCOUNTERS_PROCESSOR = 1001
	625	};
	626
	627	typedef union _SFLCounters_type {
	628	SFLIf_counters generic;
	629	SFLEthernet_counters ethernet;
	630	SFLTokenring_counters tokenring;
	631	SFLVg_counters vg;
	632	SFLVlan_counters vlan;
	633	SFLProcessor_counters processor;
	634	} SFLCounters_type;
	635
	636	typedef struct _SFLCounters_sample_element {
	637	struct _SFLCounters_sample_element nxt; / linked list */
	638	uint32_t tag; /* SFLCounters_type_tag */
	639	uint32_t length;
	640	SFLCounters_type counterBlock;
	641	} SFLCounters_sample_element;
	642
	643	typedef struct _SFLCounters_sample {
	644	/* uint32_t tag; / / SFL_sample_tag -- enterprise = 0 : format = 2 */
	645	/* uint32_t length; */
	646	uint32_t sequence_number; /* Incremented with each counters sample
	647	generated by this source_id */
	648	uint32_t source_id; /* fsSourceId */
	649	uint32_t num_elements;
	650	SFLCounters_sample_element *elements;
	651	} SFLCounters_sample;
	652
	653	/* same thing, but the expanded version, so ds_index can be a full 32 bits */
	654	typedef struct _SFLCounters_sample_expanded {
	655	/* uint32_t tag; / / SFL_sample_tag -- enterprise = 0 : format = 2 */
	656	/* uint32_t length; */
	657	uint32_t sequence_number; /* Incremented with each counters sample
	658	generated by this source_id */
	659	uint32_t ds_class; /* EXPANDED */
	660	uint32_t ds_index; /* EXPANDED */
	661	uint32_t num_elements;
	662	SFLCounters_sample_element *elements;
	663	} SFLCounters_sample_expanded;
	664
	665	#define SFLADD_ELEMENT(_sm, _el) do { (_el)->nxt = (_sm)->elements; (_sm)->elements = (_el); } while(0)
	666
	667	/* Format of a sample datagram */
	668
	669	enum SFLDatagram_version {
	670	SFLDATAGRAM_VERSION2 = 2,
	671	SFLDATAGRAM_VERSION4 = 4,
	672	SFLDATAGRAM_VERSION5 = 5
	673	};
	674
	675	typedef struct _SFLSample_datagram_hdr {
	676	uint32_t datagram_version; /* (enum SFLDatagram_version) = VERSION5 = 5 */
	677	SFLAddress agent_address; /* IP address of sampling agent */
	678	uint32_t sub_agent_id; /* Used to distinguishing between datagram
	679	streams from separate agent sub entities
	680	within an device. */
	681	uint32_t sequence_number; /* Incremented with each sample datagram
	682	generated */
	683	uint32_t uptime; /* Current time (in milliseconds since device
	684	last booted). Should be set as close to
	685	datagram transmission time as possible.*/
	686	uint32_t num_records; /* Number of tag-len-val flow/counter records to follow */
	687	} SFLSample_datagram_hdr;
	688
	689	#define SFL_MAX_DATAGRAM_SIZE 1500
	690	#define SFL_MIN_DATAGRAM_SIZE 200
	691	#define SFL_DEFAULT_DATAGRAM_SIZE 1400
	692
	693	#define SFL_DATA_PAD 400
	694
	695	#if defined(__cplusplus)
	696	} /* extern "C" */
	697	#endif
	698
	699	#endif /* SFLOW_H */

+87

-0

test.sh less more

	0	#!/bin/sh
	1	# This file is part of the nfdump project.
	2	#
	3	# Copyright (c) 2004, SWITCH - Teleinformatikdienste fuer Lehre und Forschung
	4	# All rights reserved.
	5	#
	6	# Redistribution and use in source and binary forms, with or without
	7	# modification, are permitted provided that the following conditions are met:
	8	#
	9	# * Redistributions of source code must retain the above copyright notice,
	10	# this list of conditions and the following disclaimer.
	11	# * Redistributions in binary form must reproduce the above copyright notice,
	12	# this list of conditions and the following disclaimer in the documentation
	13	# and/or other materials provided with the distribution.
	14	# * Neither the name of SWITCH nor the names of its contributors may be
	15	# used to endorse or promote products derived from this software without
	16	# specific prior written permission.
	17	#
	18	# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
	19	# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	20	# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	21	# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
	22	# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
	23	# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
	24	# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
	25	# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
	26	# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
	27	# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
	28	# POSSIBILITY OF SUCH DAMAGE.
	29	#
	30	# $Author: peter $
	31	#
	32	# $Id: test.sh 62 2006-03-08 12:59:51Z peter $
	33	#
	34	# $LastChangedRevision: 62 $
	35	#
	36	#
	37
	38	set -e
	39
	40	# run filter and type tests
	41	./nftest
	42
	43	# Check for correct output
	44	./nfgen \| ./nfdump -q -o extended > test1.out
	45	diff test1.out nfdump.test.out
	46
	47	./nfgen \| ./nfdump -q -w test.flows
	48	./nfdump -q -r test.flows -o extended > test2.out
	49	diff test2.out nfdump.test.out
	50
	51	rm -r test1.out test2.out
	52
	53	# create tmp dir for flow replay
	54	if [ -d tmp ]; then
	55	rm -f tmp/*
	56	rmdir tmp
	57	fi
	58	mkdir tmp
	59
	60	# Start nfcapd on localhost and replay flows
	61	echo
	62	echo -n Starting nfcapd ...
	63	./nfcapd -p 65530 -l tmp -D -P tmp/pidfile
	64	sleep 1
	65	echo done.
	66	echo -n Replay flows ...
	67	./nfreplay -r test.flows -v9 -H 127.0.0.1 -p 65530
	68	echo done.
	69	sleep 1
	70
	71	echo -n Terminate nfcapd ...
	72	kill -TERM `cat tmp/pidfile`;
	73	sleep 1
	74	echo done.
	75
	76	if [ -f tmp/pidfile ]; then
	77	echo nfcapd does not terminate
	78	exit
	79	fi
	80
	81	diff test.flows tmp/nfcapd.*
	82
	83	rm test.flows tmp/nfcapd.*
	84	rmdir tmp
	85
	86	echo All tests successful.

+131

-185

util.c less more

0		/*
	0	/*
1	1	* This file is part of the nfdump project.
2	2	*
3	3	* Copyright (c) 2004, SWITCH - Teleinformatikdienste fuer Lehre und Forschung

29	29	*
30	30	* $Author: peter $
31	31	*
32		* $Id: util.c 53 2005-11-17 07:45:34Z peter $
	32	* $Id: util.c 70 2006-05-17 08:38:01Z peter $
33	33	*
34		* $LastChangedRevision: 53 $
	34	* $LastChangedRevision: 70 $
35	35	*
36	36	*/
37	37

47	47	#include <sys/stat.h>
48	48	#include <fcntl.h>
49	49	#include <ctype.h>
	50	#include <netinet/in.h>
50	51
51	52	#include "config.h"
52	53

54	55	#include <stdint.h>
55	56	#endif
56	57
	58	#include "nffile.h"
57	59	#include "util.h"
58	60
59	61	#ifndef HAVE_SCANDIR
60	62	int scandir(const char dir, struct dirent **namelist,
61		const int (select)(struct dirent ),
62		const int (compar)(const void , const void *));
	63	int (select)(struct dirent ),
	64	int (compar)(const void , const void *));
63	65
64	66	int alphasort(const void a, const void b);
65	67
66	68	#endif
67	69
68	70	/* Global vars */
69		// uint64_t total_bytes = 0;
70	71
71	72	extern uint32_t byte_limit, packet_limit;
72	73	extern int byte_mode, packet_mode;
	74	extern char *CurrentIdent;
73	75
74	76	enum { NONE, LESS, MORE };
75	77

79	81
80	82	static int ParseTime(char s, time_t t_start);
81	83
82		static int FileFilter(const struct dirent *dir);
83
84		static void ReadNetflowStat(char *sfile);
	84	static inline int CheckTimeWindow(uint32_t t_start, uint32_t t_end, stat_record_t *stat_record);
	85
	86	static int FileFilter(struct dirent *dir);
85	87
86	88	static void GetFileList(char *path);
87	89
88	90	static void GetDirList(char *dirs);
89
90		static time_t firstseen, lastseen;
91
92		typedef struct StatInfo_s {
93		char Ident[32];
94		uint32_t tstamp;
95		uint64_t flows;
96		uint64_t flows_tcp;
97		uint64_t flows_udp;
98		uint64_t flows_icmp;
99		uint64_t flows_other;
100		uint64_t pkts;
101		uint64_t pkts_tcp;
102		uint64_t pkts_udp;
103		uint64_t pkts_icmp;
104		uint64_t pkts_other;
105		uint64_t bytes;
106		uint64_t bytes_tcp;
107		uint64_t bytes_udp;
108		uint64_t bytes_icmp;
109		uint64_t bytes_other;
110		uint32_t first_seen;
111		uint32_t last_seen;
112		} StatInfo_t;
113
114		static StatInfo_t NetflowStat;
115	91
116	92	typedef struct DirList_s {
117	93	struct DirList_s *next;

122	98	static struct dirent **namelist;
123	99	static DirList_t dirlist, current_dir;
124	100	static uint32_t numfiles, cnt;
125		static char first_file, last_file;
	101	static char first_file, last_file, current_file[255];
126	102	static uint32_t twin_first, twin_last;
127	103
128	104	/* Functions */
	105
129	106
130	107	#ifndef HAVE_SCANDIR
131	108	#include "scandir.c"

290	267	// check for delta time window
291	268	if ( tstring[0] == '-' \|\| tstring[0] == '+' ) {
292	269	if ( !twin_first \|\| !twin_last ) {
293		fprintf(stderr,"Time Window error: Check if stat files available\n");
	270	fprintf(stderr,"Time Window error: No time slot information available\n");
294	271	return 0;
295	272	}
296	273

324	301
325	302	} // End of ScanTimeFrame
326	303
327		static void ReadNetflowStat(char *sfile){
328		FILE *fd;
329
330		if ( sfile == NULL ) {
331		// printf("No Statfile\n");
332		NetflowStat.Ident[0] = 0;
333		NetflowStat.first_seen = 0;
334		NetflowStat.last_seen = 0;
335		}
336
337		// printf("Statfile %s\n",sfile);
338		fd = fopen(sfile, "r");
339		if ( !fd ) {
340		NetflowStat.Ident[0] = 0;
341		NetflowStat.first_seen = 0;
342		NetflowStat.last_seen = 0;
343		// printf("No such Statfile\n");
344		return;
345		}
346		// printf("Statfile ok\n");
347		fscanf(fd, "Time: %u\n", &NetflowStat.tstamp);
348		fscanf(fd, "Ident: %s\n", NetflowStat.Ident);
349		fscanf(fd, "Flows: %llu\n", &NetflowStat.flows);
350		fscanf(fd, "Flows_tcp: %llu\n", &NetflowStat.flows_tcp);
351		fscanf(fd, "Flows_udp: %llu\n", &NetflowStat.flows_udp);
352		fscanf(fd, "Flows_icmp: %llu\n", &NetflowStat.flows_icmp);
353		fscanf(fd, "Flows_other: %llu\n", &NetflowStat.flows_other);
354		fscanf(fd, "Packets: %llu\n", &NetflowStat.pkts);
355		fscanf(fd, "Packets_tcp: %llu\n", &NetflowStat.pkts_tcp);
356		fscanf(fd, "Packets_udp: %llu\n", &NetflowStat.pkts_udp);
357		fscanf(fd, "Packets_icmp: %llu\n", &NetflowStat.pkts_icmp);
358		fscanf(fd, "Packets_other: %llu\n", &NetflowStat.pkts_other);
359		fscanf(fd, "Bytes: %llu\n", &NetflowStat.bytes);
360		fscanf(fd, "Bytes_tcp: %llu\n", &NetflowStat.bytes_tcp);
361		fscanf(fd, "Bytes_udp: %llu\n", &NetflowStat.bytes_udp);
362		fscanf(fd, "Bytes_icmp: %llu\n", &NetflowStat.bytes_icmp);
363		fscanf(fd, "Bytes_other: %llu\n", &NetflowStat.bytes_other);
364		fscanf(fd, "First: %u\n", &NetflowStat.first_seen);
365		fscanf(fd, "Last: %u\n", &NetflowStat.last_seen);
366		fclose(fd);
367
368		/*
369		printf("Time: %u\n", NetflowStat.tstamp);
370		printf("Ident: %s\n", NetflowStat.Ident);
371		printf("Flows: %llu\n", NetflowStat.flows);
372		printf("Flows_tcp: %llu\n", NetflowStat.flows_tcp);
373		printf("Flows_udp: %llu\n", NetflowStat.flows_udp);
374		printf("Flows_icmp: %llu\n", NetflowStat.flows_icmp);
375		printf("Flows_other: %llu\n", NetflowStat.flows_other);
376		printf("Packets: %llu\n", NetflowStat.pkts);
377		printf("Packets_tcp: %llu\n", NetflowStat.pkts_tcp);
378		printf("Packets_udp: %llu\n", NetflowStat.pkts_udp);
379		printf("Packets_icmp: %llu\n", NetflowStat.pkts_icmp);
380		printf("Packets_other: %llu\n", NetflowStat.pkts_other);
381		printf("Bytes: %llu\n", NetflowStat.bytes);
382		printf("Bytes_tcp: %llu\n", NetflowStat.bytes_tcp);
383		printf("Bytes_udp: %llu\n", NetflowStat.bytes_udp);
384		printf("Bytes_icmp: %llu\n", NetflowStat.bytes_icmp);
385		printf("Bytes_other: %llu\n", NetflowStat.bytes_other);
386		printf("First: %u\n", NetflowStat.first_seen);
387		printf("Last: %u\n", NetflowStat.last_seen);
388		*/
389
390		} // End of ReadNetflowStat
391
392		char *GetIdent(void) {
393
394		return strlen(NetflowStat.Ident) > 0 ? NetflowStat.Ident : "none";
395
396		} // End of GetIdent
397
398		uint32_t GetStatTime(void) {
399
400		return NetflowStat.tstamp;
401
402		} // End of GetIdent
403
404		char *TimeString(void){
	304	char *TimeString(time_t start, time_t end) {
405	305	static char datestr[255];
406	306	char t1[64], t2[64];
407	307	struct tm *tbuff;
408	308
409		if ( firstseen ) {
410		tbuff = localtime(&firstseen);
	309	if ( start ) {
	310	tbuff = localtime(&start);
411	311	if ( !tbuff ) {
412	312	perror("Error time convert");
413	313	exit(250);
414	314	}
415		strftime(t1, 63, "%b %d %Y %T", tbuff);
416
417		tbuff = localtime(&lastseen);
	315	strftime(t1, 63, "%Y-%m-%d %H:%M:%S", tbuff);
	316
	317	tbuff = localtime(&end);
418	318	if ( !tbuff ) {
419	319	perror("Error time convert");
420	320	exit(250);
421	321	}
422		strftime(t2, 63, "%b %d %Y %T", tbuff);
	322	strftime(t2, 63, "%Y-%m-%d %H:%M:%S", tbuff);
423	323
424	324	snprintf(datestr, 254, "%s - %s", t1, t2);
425	325	} else {

429	329	return datestr;
430	330	}
431	331
432		int CheckTimeWindow(uint32_t t_start, uint32_t t_end) {
	332	static inline int CheckTimeWindow(uint32_t t_start, uint32_t t_end, stat_record_t *stat_record) {
	333
433	334	/*
434	335	printf("t start %u %s", t_start, ctime(&t_start));
435	336	printf("t end %u %s", t_end, ctime(&t_end));

441	342	if ( t_start == 0 )
442	343	return 1;
443	344
444		if ( NetflowStat.first_seen == 0 )
445		return 0;
446
447		if ( t_start >= NetflowStat.first_seen && t_start <= NetflowStat.last_seen )
	345	if ( stat_record->first_seen == 0 )
	346	return 0;
	347
	348	if ( t_start >= stat_record->first_seen && t_start <= stat_record->last_seen )
448	349	return 1;
449	350
450		if ( t_end >= NetflowStat.first_seen && t_end <= NetflowStat.last_seen )
	351	if ( t_end >= stat_record->first_seen && t_end <= stat_record->last_seen )
451	352	return 1;
452	353
453		if ( t_start < NetflowStat.first_seen && t_end > NetflowStat.last_seen )
	354	if ( t_start < stat_record->first_seen && t_end > stat_record->last_seen )
454	355	return 1;
455	356
456
457	357	return 0;
458	358
459	359	} // End of CheckTimeWindow
460	360
461		void SetSeenTwin(uint32_t first_seen, uint32_t last_seen) {
462		firstseen = first_seen;
463		lastseen = last_seen;
464
465		} /* End of SetSeenTwin */
466
467	361	// file filter for scandir function
468	362
469		static int FileFilter(const struct dirent *dir) {
	363	static int FileFilter(struct dirent *dir) {
470	364	struct stat stat_buf;
471	365	char string[255];
472	366
473	367	string[254] = 0;
474	368	snprintf(string, 254, "%s/%s", dirlist->dirname, dir->d_name);
475	369	if ( stat(string, &stat_buf) ) {
476		perror("Can't stat entry: ");
	370	fprintf(stderr, "Can't stat entry for:'%s': %s\n", string, strerror(errno));
477	371	return 0;
478	372	}
479	373

724	618	} // End of GetDirList
725	619
726	620	void SetupInputFileSequence(char multiple_dirs, char single_file, char *multiple_files) {
	621	stat_record_t *stat_ptr;
727	622	char string[255];
728		char *p;
	623	char p, s;
	624	int fd;
729	625
730	626	namelist = NULL;
731	627	twin_first = 0;

741	637
742	638	// get time window spanning all the files
743	639	if ( numfiles ) {
744		snprintf(string, 254, "%s/%s.stat", dirlist->dirname, namelist[0]->d_name);
745		ReadNetflowStat(string); // read the corresponding stat file
746		twin_first = NetflowStat.first_seen;
747		snprintf(string, 254, "%s/%s.stat", dirlist->dirname, namelist[numfiles-1]->d_name);
748		ReadNetflowStat(string); // read the corresponding stat file
749		twin_last = NetflowStat.last_seen;
	640	snprintf(string, 254, "%s/%s", dirlist->dirname, namelist[0]->d_name);
	641	fd = OpenFile(string, &stat_ptr, &s); // read the stat record
	642	if ( s != NULL ) {
	643	fprintf(stderr, "%s\n", s);
	644	exit(250);
	645	}
	646	close(fd);
	647	twin_first = stat_ptr->first_seen;
	648	snprintf(string, 254, "%s/%s", dirlist->dirname, namelist[numfiles-1]->d_name);
	649	fd = OpenFile(string, &stat_ptr, &s); // read the stat record
	650	if ( s != NULL ) {
	651	fprintf(stderr, "%s\n", s);
	652	exit(250);
	653	}
	654	close(fd);
	655	twin_last = stat_ptr->last_seen;
750	656	}
751	657
752	658	} else if ( single_file ) {

764	670	perror("GetDirList failed!");
765	671	exit(250);
766	672	}
	673
767	674	namelist = ( struct dirent )malloc(sizeof(struct dirent ));
768	675	if ( !*namelist ) {
769	676	perror("GetDirList failed!");
770	677	exit(250);
771	678	}
772	679
773		/* with best regards from Solaris */
	680	/* with best regards from Solaris */
774	681	if ( sizeof((*namelist)->d_name) < 255 )
775		namelist = ( struct dirent )realloc(namelist, sizeof(struct dirent ) - sizeof((namelist)->d_name) + 255);
776
	682	namelist = ( struct dirent )realloc(namelist, sizeof(struct dirent ) - sizeof((namelist)->d_name) + 255);
	683
777	684	if ( !*namelist ) {
778	685	perror("GetDirList failed!");
779	686	exit(250);

812	719	// set first directory to current dir
813	720	cnt = 0;
814	721	current_dir = dirlist;
815
816		/*
	722
	723	/*
	724	{
	725	int i;
817	726	for ( current_dir = dirlist; current_dir != NULL; current_dir = current_dir->next ) {
818	727	fprintf(stderr, "Dirlist: '%s'\n", current_dir->dirname);
819	728	}

822	731	fprintf(stderr, "File: '%s'\n", namelist[i]->d_name);
823	732	}
824	733	current_dir = dirlist;
825		*/
	734	exit(0);
	735	}
	736	*/
826	737
827	738	} // End of SetupInputFileSequence
828	739
829		int GetNextFile(int current, time_t twin_start, time_t twin_end) {
830		struct stat stat_buf;
831		char string[255], *fname;
832		int stat_failed;
	740	char *GetCurrentFilename(void) {
	741	return current_file;
	742	} // End of GetCurrentFilename
	743
	744	int GetNextFile(int current, time_t twin_start, time_t twin_end, stat_record_t **stat_record) {
	745	stat_record_t *stat_ptr;
	746	char fname, s;
	747	int fd;
833	748
834	749	// close current file before open the next one
835	750	// stdin ( current = 0 ) is not closed
836		if ( current )
	751	if ( current ) {
837	752	close(current);
	753	current_file[0] = '\0';
	754	}
838	755
839	756	// no or no more files available
840	757	if ( (numfiles == 0) ) {
841		errno = 0;
842		return -1;
	758	if ( stat_record )
	759	*stat_record = NULL;
	760	return EMPTY_LIST;
843	761	}
844	762
845	763	if ( dirlist == NULL ) {
846	764	// use stdin
847	765	// printf("Return stdin\n");
848	766	numfiles = 0;
849		return STDIN_FILENO;
	767	snprintf(current_file, 254, "%s", "<stdin>");
	768	if ( stat_record )
	769	*stat_record = NULL;
	770	CurrentIdent = "none";
	771	// use NULL for stdin
	772	return OpenFile(NULL, &stat_ptr, &s);
850	773	}
851	774
852	775	while ( cnt < numfiles ) {
853	776	while ( current_dir ) {
854	777	fname = namelist[cnt]->d_name;
855		snprintf(string, 254, "%s/%s.stat", current_dir->dirname, fname);
856		ReadNetflowStat(string); // read the corresponding stat file
857		if ( CheckTimeWindow(twin_start, twin_end) ) {
858		snprintf(string, 254, "%s/%s", current_dir->dirname, fname);
859		stat_failed = 0;
860		if ( stat(string, &stat_buf) ) {
861		fprintf(stderr, "Can't stat '%s': %s\n", string, strerror(errno));
862		stat_failed = 1;
863		}
864		if (!S_ISREG(stat_buf.st_mode) ) {
865		fprintf(stderr, "'%s' is not a file\n", string);
866		stat_failed = 1;
867		}
868		// total_bytes += stat_buf.st_size;
	778	snprintf(current_file, 254, "%s/%s", current_dir->dirname, fname);
	779	fd = OpenFile(current_file, &stat_ptr, &s); // Open the file
	780	if ( fd > 0 && CheckTimeWindow(twin_start, twin_end, stat_ptr) ) {
869	781	// printf("Return file: %s\n", string);
870	782	current_dir = current_dir->next;
871		if ( !stat_failed )
872		return open(string, O_RDONLY);
	783	if ( stat_record )
	784	*stat_record = stat_ptr;
	785	return fd;
873	786	}
	787	close(fd);
	788	if ( s != NULL )
	789	fprintf(stderr, "%s\n", s);
874	790	// in the event of missing the stat file in the last directory
875	791	// of the directory queue current_dir is already NULL
876	792	current_dir = current_dir ? current_dir->next : NULL;

879	795	current_dir = dirlist;
880	796	}
881	797
882
883		errno = 0;
884		return -1;
	798	if ( stat_record )
	799	*stat_record = NULL;
	800	return EMPTY_LIST;
885	801
886	802	} // End of GetNextFile
887	803

999	915
1000	916	} // End of SetLimits
1001	917
1002		void UpdateTimeWindow ( time_t start, time_t end ) {
1003
1004		if ( NetflowStat.first_seen < *start )
1005		*start = NetflowStat.first_seen;
1006		if ( NetflowStat.last_seen > *end )
1007		*end = NetflowStat.last_seen;
1008
1009		} // End of UpdateTimeWindow
	918	void SumStatRecords(stat_record_t s1, stat_record_t s2) {
	919
	920	s1->numflows += s2->numflows;
	921	s1->numbytes += s2->numbytes;
	922	s1->numpackets += s2->numpackets;
	923	s1->numflows_tcp += s2->numflows_tcp;
	924	s1->numflows_udp += s2->numflows_udp;
	925	s1->numflows_icmp += s2->numflows_icmp;
	926	s1->numflows_other += s2->numflows_other;
	927	s1->numbytes_tcp += s2->numbytes_tcp;
	928	s1->numbytes_udp += s2->numbytes_udp;
	929	s1->numbytes_icmp += s2->numbytes_icmp;
	930	s1->numbytes_other += s2->numbytes_other;
	931	s1->numpackets_tcp += s2->numpackets_tcp;
	932	s1->numpackets_udp += s2->numpackets_udp;
	933	s1->numpackets_icmp += s2->numpackets_icmp;
	934	s1->numpackets_other += s2->numpackets_other;
	935	s1->sequence_failure += s2->sequence_failure;
	936
	937	if ( s2->first_seen < s1->first_seen ) {
	938	s1->first_seen = s2->first_seen;
	939	s1->msec_first = s2->msec_first;
	940	}
	941	if ( s2->first_seen == s1->first_seen &&
	942	s2->msec_first < s1->msec_first )
	943	s1->msec_first = s2->msec_first;
	944
	945	if ( s2->last_seen > s1->last_seen ) {
	946	s1->last_seen = s2->last_seen;
	947	s1->msec_last = s2->msec_last;
	948	}
	949	if ( s2->last_seen == s1->last_seen &&
	950	s2->msec_last > s1->msec_last )
	951	s1->msec_last = s2->msec_last;
	952
	953	} // End of AddStatRecords
	954
	955

+17

-14

util.h less more

29	29	*
30	30	* $Author: peter $
31	31	*
32		* $Id: util.h 34 2005-08-22 12:01:31Z peter $
	32	* $Id: util.h 70 2006-05-17 08:38:01Z peter $
33	33	*
34		* $LastChangedRevision: 34 $
	34	* $LastChangedRevision: 70 $
35	35	*
36	36	*/
37	37
38	38
39		#include <time.h>
40		#include <sys/types.h>
	39	#define FILE_ERROR -1
	40	#define EMPTY_LIST -2
	41
	42	#ifdef WORDS_BIGENDIAN
	43	# define ntohll(n) (n)
	44	# define htonll(n) (n)
	45	#else
	46	# define ntohll(n) (((uint64_t)ntohl(n)) << 32) + ntohl((n) >> 32)
	47	# define htonll(n) (((uint64_t)htonl(n)) << 32) + htonl((n) >> 32)
	48	#endif
41	49
42	50	int ScanTimeFrame(char tstring, time_t t_start, time_t *t_end);
43	51
44		char *GetIdent(void);
45
46		int CheckTimeWindow(uint32_t t_start, uint32_t t_end);
47
48		uint32_t GetStatTime(void);
49
50		char *TimeString(void);
	52	char *TimeString(time_t start, time_t end);
51	53
52	54	void SetupInputFileSequence(char multiple_dirs, char single_file, char *multiple_files);
53	55
54		int GetNextFile(int current, time_t twin_start, time_t twin_end);
	56	char *GetCurrentFilename(void);
55	57
56		void SetSeenTwin(uint32_t first_seen, uint32_t last_seen);
	58	int GetNextFile(int current, time_t twin_start, time_t twin_end, stat_record_t **stat_record);
57	59
58	60	void SetLimits(int stat, char packet_limit_string, char byte_limit_string );
59	61
60		void UpdateTimeWindow ( time_t start, time_t end );
	62	void SumStatRecords(stat_record_t s1, stat_record_t s2);
61	63
	64	void Setv6Mode(int mode);
62	65

-5

version.h less more

27	27	*
28	28	* $Author: peter $
29	29	*
30		* $Id: version.h 53 2005-11-17 07:45:34Z peter $
	30	* $Id: version.h 70 2006-05-17 08:38:01Z peter $
31	31	*
32		* $LastChangedRevision: 53 $
	32	* $LastChangedRevision: 70 $
33	33	*
34	34	*
35	35	*/
36	36
	37	const char *nfdump_version = "1.5.2";
37	38
38		const char *nfdump_version = "1.4.1";
39		const char *nfdump_date = "$LastChangedDate: 2005-11-17 08:45:34 +0100 (Thu, 17 Nov 2005) $";
40
	39	const char *nfdump_date = "$LastChangedDate: 2006-05-17 10:38:01 +0200 (Wed, 17 May 2006) $";

0	0	int scandir(const char dir, struct dirent **namelist,
1		const int (select)(const struct dirent ),
2		const int (compar)(const void , const void *));
	1	int (select)(const struct dirent ),
	2	int (compar)(const void , const void *));
3	3
4	4	int alphasort(const void a, const void b);