2 | 2 |
|
3 | 3 |
--Overview
|
4 | 4 |
|
5 | |
Squidview is a program meant to display the squid proxy server log
|
6 | |
file in a nice fashion, providing the log file is in squid's
|
7 | |
native reporting fashion. It has features such as search, report
|
8 | |
generation, monitor mode and supports three log files.
|
9 | |
|
10 | |
Thus, the program can be used to monitor Internet usage on a
|
11 | |
networked site. But please note squid has to be running first and
|
12 | |
this program is not a proactive resource controller. What it can
|
13 | |
do is tell you who and which sites are consuming the most bandwidth.
|
|
5 |
Squidview is a program meant to display the squid proxy server log file in a
|
|
6 |
nice fashion, providing the log file is in squid's native reporting fashion.
|
|
7 |
It has features such as search, report generation, monitor mode and supports
|
|
8 |
three log files.
|
|
9 |
|
|
10 |
Thus, the program can be used to monitor Internet usage on a networked site.
|
|
11 |
But please note squid has to be running first and this program is not a
|
|
12 |
proactive resource controller. What it can do is tell you who and which sites
|
|
13 |
are consuming the most bandwidth.
|
14 | 14 |
|
15 | 15 |
--Using squidview
|
16 | 16 |
|
17 | |
Squidview shows each proxy request on one line starting with the
|
18 | |
user's name (ie the name of the user on the client machine as
|
19 | |
reported by identd or similar), flags and then the target (ie
|
20 | |
destination) of the request.
|
21 | |
|
22 | |
Should an identd process not be running on the client, squidview
|
23 | |
can display the client IP address instead of "-", or if you are
|
24 | |
using an aliases file it can get a name from that. See Reports
|
25 | |
about this.
|
26 | |
|
27 | |
The target bit is truncated if necessary so as to fit the
|
28 | |
information on one line. There are two methods of truncation
|
29 | |
(discussed in Reports.)
|
30 | |
|
31 | |
Change the selected line with the cursor control keys or the number
|
32 | |
keypad. Down the bottom, on the status bar, is the time the selected
|
33 | |
request was made as well how far through the log it is (as a
|
34 | |
percentage: 0% top, 100% end). Press 'h' to get some help or 'r'
|
35 | |
for this readme file.
|
36 | |
|
37 | |
Also on the status bar will be "Mon Pri". "Mon" means the program
|
38 | |
will update the screen if new proxy requests are made. Toggle this
|
39 | |
off by pressing 'm' if you want to remain on a selected line.
|
40 | |
"Pri" is the primary log file, to switch to another log file press
|
41 | |
the appropriate key (press 'h' for keys.)
|
|
17 |
Squidview shows each proxy request on one line starting with the user's name
|
|
18 |
(ie the name of the user on the client machine as reported by identd or
|
|
19 |
similar), flags and then the target (ie destination) of the request.
|
|
20 |
|
|
21 |
Should an identd process not be running on the client, squidview can display
|
|
22 |
the client IP address instead of "-", or if you are using an aliases file it
|
|
23 |
can get a name from that. See Reports about this.
|
|
24 |
|
|
25 |
The target bit is truncated if necessary so as to fit the information on one
|
|
26 |
line. There are two methods of truncation (discussed in Reports.)
|
|
27 |
|
|
28 |
Change the selected line with the cursor control keys or the number keypad.
|
|
29 |
Down the bottom, on the status bar, is the time the selected request was made
|
|
30 |
as well how far through the log it is (as a percentage: 0% top, 100% end).
|
|
31 |
Press 'h' to get some help or 'r' for this howto file.
|
|
32 |
|
|
33 |
Also on the status bar will be "Mon Pri". "Mon" means the program will update
|
|
34 |
the screen if new proxy requests are made. Toggle this off by pressing 'm' if
|
|
35 |
you want to remain on a selected line. "Pri" is the primary log file, to switch
|
|
36 |
to another log file press the appropriate key (press 'h' for keys.)
|
42 | 37 |
|
43 | 38 |
--Flags
|
44 | 39 |
|
45 | |
Between the user and URL columns is the flags field. For example
|
46 | |
it may be "w2Rf":
|
|
40 |
Between the user and URL columns is the flags field. For example it may be
|
|
41 |
"w2Rf":
|
47 | 42 |
|
48 | 43 |
'w': a word match on the URL (see Searches below)
|
49 | 44 |
'2': bytes transferred was between 0.25MB and 1MB
|
50 | 45 |
'R': the request was a cache refresh hit
|
51 | 46 |
'f': part of the current focus
|
52 | 47 |
|
53 | |
For a bit of help about these flags select the relevant line and
|
54 | |
press 'v'.
|
|
48 |
For a bit of help about these flags select the relevant line and press 'v'.
|
55 | 49 |
|
56 | 50 |
--Searching
|
57 | 51 |
|
58 | |
A search forward is made by the right arrow key, backwards is
|
59 | |
handled by the left arrow key. But first you need something to
|
60 | |
search for. Press 'f' to find a piece of text. Both user names and
|
61 | |
http/ftp addresses can cause a match. Request lines which match
|
62 | |
are noted by a 'w' in the flags column.
|
63 | |
|
64 | |
Multiple search strings are possible, and are necessary when you
|
65 | |
use skips. Skips tell squidview not to match some requests, such
|
66 | |
as when the word "sex" is searched for but not when the target is
|
67 | |
on doubleclick.net (that advertisement server likes to use the word
|
68 | |
"sex" in URLs.) The following will accomplish that:
|
|
52 |
A search forward is made by the right arrow key, backwards is handled by the
|
|
53 |
left arrow key. But first you need something to search for. Press 'f' to find
|
|
54 |
a piece of text. Both user names and http/ftp addresses can cause a match.
|
|
55 |
Request lines which match are noted by a 'w' in the flags column.
|
|
56 |
|
|
57 |
Multiple search strings are possible, and are necessary when you use skips.
|
|
58 |
Skips tell squidview not to match some requests, such as when the word "sex"
|
|
59 |
is searched for but not when the target is on doubleclick.net (that
|
|
60 |
advertisement server likes to use the word "sex" in URLs.) The following will
|
|
61 |
accomplish that:
|
69 | 62 |
|
70 | 63 |
!doubleclick.net
|
71 | 64 |
sex
|
72 | 65 |
|
73 | |
The requests with "doubleclick.net" in them will be skipped
|
74 | |
because that piece of text is first and it is preceded with an
|
75 | |
"!". In doubleclick.net cases the flags column will have a
|
76 | |
'-' where the word match would have been.
|
77 | |
|
78 | |
Note that text you enter with 'f' is placed at the top of the
|
79 | |
search list so it has priority. Using 'F' (capital F) will add
|
80 | |
search text to the bottom.
|
81 | |
|
82 | |
Your search words can be saved from the search options menu, and
|
83 | |
you can do some other things there: pick up large requests and
|
84 | |
focus on a particular user. These two can be turned off when not
|
85 | |
needed.
|
|
66 |
The requests with "doubleclick.net" in them will be skipped because that piece
|
|
67 |
of text is first and it is preceded with an "!". In doubleclick.net cases the
|
|
68 |
flags column will have a '-' where the word match would have been.
|
|
69 |
|
|
70 |
Note that text you enter with 'f' is placed at the top of the search list so
|
|
71 |
it has priority. Using 'F' (capital F) will add search text to the bottom.
|
|
72 |
|
|
73 |
Your search words can be saved from the search options menu, and you can do
|
|
74 |
some other things there: pick up large requests and focus on a particular user.
|
|
75 |
These two can be turned off when not needed.
|
86 | 76 |
|
87 | 77 |
--Navigating the log file
|
88 | 78 |
|
89 | |
As well as jumping to the beginning or end of the log file, you
|
90 | |
can go to a certain percentage through with 'g', or to the
|
91 | |
beginning of a certain day with 'j' or 'J'. Of course 'home' and
|
92 | |
'end' work too, if you are using a remote shell and they don't,
|
93 | |
press '7' or '1' respectfully (look at your number keypad.)
|
|
79 |
As well as jumping to the beginning or end of the log file, you can go to a
|
|
80 |
certain percentage through with 'g', or to the beginning of a certain day with
|
|
81 |
'j' or 'J'. Of course 'home' and 'end' work too, if you are using a remote
|
|
82 |
shell and they don't, press '7' or '1' respectfully (look at your number
|
|
83 |
keypad.)
|
94 | 84 |
|
95 | 85 |
--User lookup
|
96 | 86 |
|
97 | |
On a selected line you can press 'v' to get a verbose description
|
98 | |
of it - this is actually a dump of the line to the screen.
|
99 | |
Squidview will try to match the user to those known in a file
|
100 | |
called "users", displaying that line in the file. For example
|
101 | |
the "users" file could have in it:-
|
|
87 |
On a selected line you can press 'v' to get a verbose description of it - this
|
|
88 |
is actually a dump of the line to the screen. Squidview will try to match the
|
|
89 |
user to those known in a file called "users", displaying that line in the file.
|
|
90 |
For example the "users" file could have in it:-
|
102 | 91 |
|
103 | 92 |
root system administrator
|
104 | 93 |
|
105 | |
The first word on each line must be the login name with no spaces
|
106 | |
between it and the real name.
|
|
94 |
The first word on each line must be the login name (with no spaces in it) and
|
|
95 |
the real name.
|
107 | 96 |
|
108 | 97 |
--Common options
|
109 | 98 |
|
110 | |
By default if no login name is available the client's IP number is
|
111 | |
displayed instead. Change this with "ip instead of null user" to
|
112 | |
get, instead, reports of bandwidth attributed to "-". The aliases
|
113 | |
file is another option here. You might specify that 192.168.0.15
|
114 | |
be displayed as "server4". You need to enable this one because
|
115 | |
it's off by default.
|
116 | |
|
117 | |
Keeping the filename of target also affects the main window. When
|
118 | |
on, the target URL is shifted left - but not over the domain - so that
|
119 | |
the type of file can be seen on one line. Otherwise the line is
|
120 | |
simply truncated to be displayed.
|
|
99 |
By default if no login name is available the client's IP number is displayed
|
|
100 |
instead. Change this with "ip instead of null user" to get, instead, reports
|
|
101 |
of bandwidth attributed to "-". The aliases file is another option here. You
|
|
102 |
might specify that 192.168.0.15 be displayed as "server4". You need to enable
|
|
103 |
this one because it's off by default.
|
|
104 |
|
|
105 |
Keeping the filename of target also affects the main window. When on, the
|
|
106 |
target URL is shifted left - but not over the domain - so that the type of
|
|
107 |
file can be seen on one line. Otherwise the line is simply truncated to be
|
|
108 |
displayed.
|
121 | 109 |
|
122 | 110 |
--Log a report
|
123 | 111 |
|
124 | |
Make a text or CSV report of search hits. A few options are there.
|
125 | |
You will need to specify a report file name to view the details.
|
126 | |
Otherwise you will just get a summary. Reports are placed in ~/.squidview.
|
127 | |
|
128 | |
To start or finish the report at a particular point in the log file
|
129 | |
highlight the line in the main window, press 'l' and then either
|
130 | |
'a' or 'b'. Press 'a' or 'b' again to toggle it.
|
131 | |
|
132 | |
User bandwidth totals can be calculated. The options are search hits
|
133 | |
or "not veto" (which is mostly everything excluding skips mentioned
|
134 | |
above.) These are sorted so you can find the heavy internet users.
|
135 | |
|
136 | |
Bandwidth totals will find the most popular sites for you. This can
|
137 | |
be done for one user specifically or for all users as a whole.
|
138 | |
|
139 | |
When a word hit is detected it can be written in the report (eg
|
140 | |
"word hit action: normal text").
|
141 | |
Text reports are good for viewing with "less"; CSV ones are intended
|
142 | |
for spreadsheets.
|
143 | |
|
144 | |
In the case of normal text reports you may or may not want to see
|
145 | |
the request size. This information takes up a column. Splitting long
|
146 | |
lines will show the details on more than one line if need be. Then
|
147 | |
again, you may only want all "hits" to be shown on just one line.
|
148 | |
|
149 | |
The other options are straight forward. Be a bit picky about the CSV
|
150 | |
field separator - they put just about any characters in URLs. Try a
|
151 | |
tab (yes, just press 'tab', 'enter') or "*".
|
152 | |
|
153 | |
To get a summary report about a particular user, say "graeme", do
|
154 | |
this:
|
|
112 |
Make a text or CSV report of search hits. A few options here.
|
|
113 |
|
|
114 |
You will need to specify a report file name to view the details. Otherwise you
|
|
115 |
will just get a summary. Reports are placed in ~/.squidview.
|
|
116 |
|
|
117 |
To start or finish the report at a particular point in the log file highlight
|
|
118 |
the line in the main window, press 'l' and then either 'a' or 'b'. Press 'a'
|
|
119 |
or 'b' again to toggle it.
|
|
120 |
|
|
121 |
User bandwidth totals can be calculated. The options are search hits or
|
|
122 |
"not veto" (which is mostly everything excluding skips mentioned above.) These
|
|
123 |
are sorted so you can find the heavy internet users.
|
|
124 |
|
|
125 |
Bandwidth totals will find the most popular sites for you. This can be done for
|
|
126 |
one user specifically or for all users as a whole.
|
|
127 |
|
|
128 |
When a word hit is detected it can be written in the report (eg "word hit
|
|
129 |
action: normal text"). Text reports are good for viewing with "less"; CSV ones
|
|
130 |
are intended for spreadsheets.
|
|
131 |
|
|
132 |
In the case of normal text reports you may or may not want to see the request
|
|
133 |
size. This information takes up a column. Splitting long lines will show the
|
|
134 |
details on more than one line if need be. Then again, you may only want all
|
|
135 |
"hits" to be shown on just one line.
|
|
136 |
|
|
137 |
The other options are straight forward. Be a bit picky about the CSV field
|
|
138 |
separator - they put just about any characters in URLs. Try a tab (yes, just
|
|
139 |
press 'tab', 'enter') or "*".
|
|
140 |
|
|
141 |
To get a summary report about a particular user, say "graeme", do this:
|
155 | 142 |
- in search options focus on "graeme"
|
156 | 143 |
- go to log a report
|
157 | 144 |
- select "domain bandwidth totals"
|
158 | 145 |
- select "only focus user graeme"
|
159 | 146 |
- and press enter on the previous screen where it says go
|
160 | 147 |
|
161 | |
You will need to unset the above options for reports to come
|
162 | |
out normally again, and for searches (cursor <-, ->) too.
|
|
148 |
You will need to unset the above options for reports to come out normally
|
|
149 |
again, and for searches (cursor <-, ->) too.
|
163 | 150 |
|
164 | 151 |
--Filtered reports
|
165 | 152 |
|
166 | |
After making a general report it is possible to filter it for
|
167 | |
just one user. That way you don't have to rescan the log file
|
168 | |
with a focus. The downside of this is the target totals the
|
169 | |
user surfed to can't be calculated.
|
|
153 |
After making a general report it is possible to filter it for just one user.
|
|
154 |
That way you don't have to rescan the log file with a focus. The downside of
|
|
155 |
this is the target totals the user surfed to can't be calculated.
|
170 | 156 |
|
171 | 157 |
--Tally Mode
|
172 | 158 |
|
173 | |
This mode tells you some statistics about each user's usage of the Web.
|
174 | |
Given any given starting point, it doesn't have to be at the begining of the
|
175 | |
log, squidview will gather the data, display it, and then keep it up to
|
176 | |
date.
|
|
159 |
This mode tells you some statistics about each user's usage of the Web. Given
|
|
160 |
any given starting point, it doesn't have to be at the begining of the log,
|
|
161 |
squidview will gather the data, display it, and then keep it up to date.
|
177 | 162 |
|
178 | 163 |
So from the main screen press T (capital) and let it work. Then you should
|
179 | 164 |
get the tally screen that has the list of users down the left hand side and
|
180 | 165 |
their statistics to the right. Most numbers are self explanatory. "Points"
|
181 | 166 |
indicate who has used the Web a lot recently. Every web byte is counted and
|
182 | 167 |
added to that user's number of points. Then after a certain period of time
|
183 | |
the points list is aged, eg multiplied by 0.75. This means big users will
|
184 | |
rise to the top quickly and then slowly progress further down the list if
|
185 | |
they stop surfing.
|
186 | |
|
187 | |
Tally mode can be set to go in monitor mode. Turn that off if you need to
|
188 | |
stay selected on one user. There are other views and options mentioned in
|
189 | |
help (h). One of these toggles what to do about requests that have been
|
190 | |
denied. You may not want to see attempts by computers (often by themselves)
|
191 | |
"phoning home" regularly. If the status line shows "-d" you won't get
|
192 | |
these cluttering up your view.
|
|
168 |
the points list is aged, eg multiplied by 0.75. This means big users will rise
|
|
169 |
to the top quickly and then slowly progress further down the list if they stop
|
|
170 |
surfing.
|
|
171 |
|
|
172 |
Tally mode can be set to go in monitor mode. Turn that off if you need to stay
|
|
173 |
selected on one user. There are other views and options mentioned in help (h).
|
|
174 |
One of these toggles what to do about requests that have been denied. You may
|
|
175 |
not want to see attempts by computers (often by themselves) "phoning home"
|
|
176 |
regularly. If the status line shows "-d" you won't get these cluttering up your
|
|
177 |
view.
|
193 | 178 |
|
194 | 179 |
--One User History
|
195 | 180 |
|
|
197 | 182 |
entries only by the specified user. It is useful to discover that user's
|
198 | 183 |
recent activity, warranted for example by a spike in his/her tally points.
|
199 | 184 |
|
200 | |
Pressing O (capital) will bring you into this mode using as the user the
|
201 | |
one currently selected. u will switch to another user.
|
|
185 |
Pressing O (capital) will bring you into this mode using as the user the one
|
|
186 |
currently selected. u will switch to another user.
|
202 | 187 |
|
203 | 188 |
The numbers on the status line deserve explanation. If they say:
|
204 | 189 |
|
|
210 | 195 |
file.
|
211 | 196 |
|
212 | 197 |
The 90.90% indicates the selected line is that far down the request entries
|
213 | |
in memory. For your information the request lines aren't actually in
|
214 | |
memory - just their positions in the log file are.
|
|
198 |
in memory. For your information the request lines aren't actually in memory -
|
|
199 |
just their positions in the log file are.
|