In this article I'll demonstrate two novel features of Marty Roesch's Daemonlogger tool.
Daemonlogger is a packet capture tool written by Snort author Marty Roesch. I first mentioned it in my blog in April 2007. You can download Daemonlogger from Marty's user page; the version hosted on the Snort download page doesn't appear current.
A look at the help switch for Daemonlogger demonstrates some of its capabilities:
freebsd70:/usr/local/daemonlogger-1.2.1/bin# ./daemonlogger -h
USAGE: daemonlogger [-options] <bpf filter="">
-c <count> Log <count> packets and exit
-d Daemonize at startup
-f <bpf file=""> Load BPF filter from <bpf file="">
-F Flush the pcap buffer for each packet
-g <group name=""> Set group ID to <group name="">
-h Show this usage statement
-i <intf> Grab packets from interface <intf>
-l <path> Log to directory <path>
-m <count> Generate <count> log files and quit
-M <pct> In ringbuffer mode log data to <pct> of
volume capacity
-n <name> Set output filename prefix to <name>
-o <outf> Disable logging, retransmit data from
<intf> to <outf>
-p <pidfile> Use <pidfile> for PID filename
-P <pidpath> Use <pidpath> for PID directory
-r Activate ringbuffer mode
-R <pcap file=""> Read packets from <pcap file="">
-s <bytes> Rollover the log file every <bytes>
-S <snaplen> Capture <snaplen> bytes per packet
-t <time> Rollover the log file on time intervals
-u <user name=""> Set user ID to <user name="">
-v Show daemonlogger version
Daemonlogger has two features of note. First, it has a user-friendly ring buffer mode. In other words, it can log packets within a specified number of files and file sizes. Why is this useful? It's the simplest worry-free network packet capture implementation you could ask for. Being able to specify a defined amount of space to capture Pcap data means you can start a tool like Daemonlogger in ring buffer mode and never touch it again. You'll always have the last 300 GB, or 100 GB, or whatever amount of hard drive space you allocate, of network traffic available for analysis.
Tcpdump 3.9.4 introduced a similar feature in 2005, but I find its implementation too simplistic. For example, the following command tells Tcpdump to save up to 5 Pcap files, each of which should be no larger than 1,000,000 bytes (approximately 977 KB).
# tcpdump -n -i le1 -s 1515 -C 1 -W 5 -w /tmp/test1.lpc -U
I produced the following directory listing while collecting packets with Tcpdump.
-rw-r--r-- 1 root wheel 373K Dec 9 20:26 test1.lpc0
-rw-r--r-- 1 root wheel 977K Dec 9 20:26 test1.lpc1
-rw-r--r-- 1 root wheel 977K Dec 9 20:26 test1.lpc2
-rw-r--r-- 1 root wheel 977K Dec 9 20:26 test1.lpc3
-rw-r--r-- 1 root wheel 978K Dec 9 20:26 test1.lpc4
Notice the first file is smaller than the others. That's because Tcpdump wrote the five files, then started overwriting the first file. When I grabbed that output it had written 373 KB in place of the original test1.lpc0 file. I don't like the way Tcpdump manages its file names in that respect. I also don't like seeing file names like NAME0, NAME1, etc.
Let's see how to implement a similar capture with Daemonlogger. Notice the different switches required, especially the -r to active ring buffer mode and the separation between the logging directory (-l /tmp) and the name of the files in the directory (-n dltest1.lpc). Also note the specification of the file size using bytes instead of MB as was the case with Tcpdump.
daemonlogger -i le1 -S 1515 -s 1000000 -m 5 -r -l /tmp/ -n dltest1.lpc
[-] Interface set to le1
[-] Logpath set to /tmp/
[-] Max files to write set to 5
[-] Log filename set to "dltest1.lpc"
[-] Pidfile configured to "daemonlogger.pid"
[-] Pidpath configured to "/var/run"
[-] Ringbuffer active
[-] Rollover size set to 1000000 bytes
[-] Snaplen set to 1515
[-] Rollover time configured for 0 seconds
[-] Pruning behavior set to oldest IN DIRECTORY
-*> DaemonLogger <*-
Version 1.2.1
By Martin Roesch
(C) Copyright 2006-2007 Sourcefire Inc., All rights reserved
Checking partition stats for log directory "/tmp//."
sniffing on interface le1
Logging packets to /tmp//dltest1.lpc.1228873098
I like that Daemonlogger tells you what it's doing. You could avoid seeing this output if you use the -d switch to daemonize at startup. Based on this output I noticed I neglected to enable packet-buffered output, which I could've activated using the -F switch to produce this notice:
[-] Packet-buffered output activated
As Daemonlogger works it reports that it's rolling over logfiles.
Size limit reached (1000506 - 1000000 = 506), rolling over!
Rolling over logfile...
Logging packets to /tmp//dltest1.lpc.1228873102
Size limit reached (1000576 - 1000000 = 576), rolling over!
Rolling over logfile...
Logging packets to /tmp//dltest1.lpc.1228873109
Size limit reached (1000386 - 1000000 = 386), rolling over!
Rolling over logfile...
Logging packets to /tmp//dltest1.lpc.1228873117
Size limit reached (1000716 - 1000000 = 716), rolling over!
Rolling over logfile...
Logging packets to /tmp//dltest1.lpc.1228873122
Size limit reached (1000574 - 1000000 = 574), rolling over!
Rolling over logfile...
[!] Ringbuffer: deleting /tmp//dltest1.lpc.1228873098
Logging packets to /tmp//dltest1.lpc.1228873127
Size limit reached (1000716 - 1000000 = 716), rolling over!
Rolling over logfile...
[!] Ringbuffer: deleting /tmp//dltest1.lpc.1228873102
Logging packets to /tmp//dltest1.lpc.1228873133
Size limit reached (1000480 - 1000000 = 480), rolling over!
Rolling over logfile...
[!] Ringbuffer: deleting /tmp//dltest1.lpc.1228873109
Logging packets to /tmp//dltest1.lpc.1228873138
If I take an arbitrary look at the directory where Daemonlogger stores its Pcap data, I'll see the following:
-rw-r--r-- 1 root wheel 977K Dec 9 20:38 dltest1.lpc.1228873117
-rw-r--r-- 1 root wheel 977K Dec 9 20:38 dltest1.lpc.1228873122
-rw-r--r-- 1 root wheel 977K Dec 9 20:38 dltest1.lpc.1228873127
-rw-r--r-- 1 root wheel 977K Dec 9 20:38 dltest1.lpc.1228873133
-rw-r--r-- 1 root wheel 884K Dec 9 20:39 dltest1.lpc.1228873138
There are only five files, as I would expect. However, I can always be sure that the newest file has the newest data. You can verify this by checking the timestamp at the end of the files. Here I check the timestamp on the first and last files. They're in Unix time.
freebsd70:/tmp# date -r 1228873117
Tue Dec 9 20:38:37 EST 2008
freebsd70:/tmp# date -r 1228873138
Tue Dec 9 20:38:58 EST 2008
You can dispense with determining the number of individual files you want to log and just use a percentage of available partition space. In the following I tell Daemonlogger to start rolling over its ring buffer when it approaches 65% utilization (via the -M 65 switch).
/daemonlogger -i le1 -S 1515 -s 1000000 -M 65 -r -l /tmp/ -n dltest1.lpc -F
[-] Packet-buffered output activated
[-] Interface set to le1
[-] Logpath set to /tmp/
[-] Log filename set to "dltest1.lpc"
[-] Pidfile configured to "daemonlogger.pid"
[-] Pidpath configured to "/var/run"
[-] Ringbuffer active
[-] Rollover size set to 1000000 bytes
[-] Snaplen set to 1515
[-] Rollover time configured for 0 seconds
[-] Pruning behavior set to oldest IN DIRECTORY
-*> DaemonLogger <*-
Version 1.2.1
By Martin Roesch
(C) Copyright 2006-2007 Sourcefire Inc., All rights reserved
Checking partition stats for log directory "/tmp//."
65% max disk utilization = 666358 blocks free (out of 1903879)
Blocksize = 2048
Rollsize = 488 blocks
sniffing on interface le1
Logging packets to /tmp//dltest1.lpc.1228874034
Size limit reached (1000722 - 1000000 = 722), rolling over!
Rolling over logfile...
Logging packets to /tmp//dltest1.lpc.1228874043
Size limit reached (1000798 - 1000000 = 798), rolling over!
Rolling over logfile...
Logging packets to /tmp//dltest1.lpc.1228874047
Size limit reached (1000215 - 1000000 = 215), rolling over!
Rolling over logfile...
Once the 65% threshold is reached Daemonlogger lets us know.
Logging packets to /tmp//dltest1.lpc.1228874206
Size limit reached (1000503 - 1000000 = 503), rolling over!
Rolling over logfile...
Disk max utilization reached, rolling over and pruning
[!] Ringbuffer: deleting /tmp//dltest1.lpc.1228874034
Logging packets to /tmp//dltest1.lpc.1228874215
Size limit reached (1001167 - 1000000 = 1167), rolling over!
Rolling over logfile...
Disk max utilization reached, rolling over and pruning
[!] Ringbuffer: deleting /tmp//dltest1.lpc.1228874043
Logging packets to /tmp//dltest1.lpc.1228874221
You can also tell Daemonlogger to drop privileges using the -u and -g switches, e.g. -u nobody -g nobody:
[-] Setting user ID to nobody
Checking ps output confirms this.
$ ps -auxww | grep daemonlogger
nobody 3332 0.0 0.3 3312 1360 p0 S+ 9:01PM 0:00.07 ./daemonlogger -i le1 -S 1515 -s 1000000 -M 65 -r -l /tmp/ -n dltest1.lpc -F -u nobody -g nobody
Dropping privileges helps limit what an intruder could do if he were able to exploit a vulnerability in Daemonlogger.
The final aspect of Daemonlogger that I'd like to demonstrate is its "soft tap" functionality. With Daemonlogger you can listen to traffic on one interface and retransmit on another. This could be useful if you wish to create a network traffic director that performs I/O intensive tasks (like filtering or splitting traffic) in tandem with other systems that take actions, like logging traffic to disk. I called this "Distributed Traffic Collection" in a 2005 blog post (http://taosecurity.blogspot.com/2005/07/distributed-traffic-collection-with-pf.html) where I showed how to achieve this with the Pf packet filter on FreeBSD.
To simulate taking traffic from one interface and sending it elsewhere, I'll create a fake interface on FreeBSD. This edsc(4) interface imitates an Ethernet device, and is intended to take the place of a real interface connected to another computer.
First I'll create the edsc0 interface.
freebsd70:/root# ifconfig edsc0 create
freebsd70:/root# ifconfig edsc0 up
freebsd70:/root# ifconfig edsc0
edsc0: flags=8843<up,broadcast,running,simplex,multicast> metric 0 mtu 1500
ether 00:00:00:00:00:00
With it created, I'll capture some traffic with Daemonlogger and send only ICMP packets over to the edsc0 interface.
daemonlogger -F -i le1 -o edsc0 icmp
[-] Packet-buffered output activated
[-] Interface set to le1
[-] Log filename set to "daemonlogger.pcap"
[-] Tap output interface set to edsc0[-] Pidfile configured to "daemonlogger.pid"
[-] Pidpath configured to "/var/run"
[-] Rollover size set to 2147483648 bytes
[-] Rollover time configured for 0 seconds
[-] Pruning behavior set to oldest IN DIRECTORY
-*> DaemonLogger <*-
Version 1.2.1
By Martin Roesch
(C) Copyright 2006-2007 Sourcefire Inc., All rights reserved
sniffing on interface le1
If I sniff on the edsc0 interface I'll only see ICMP traffic, e.g.
# tcpdump -n -i edsc0
tcpdump: WARNING: BIOCPROMISC: Invalid argument
tcpdump: WARNING: edsc0: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on edsc0, link-type EN10MB (Ethernet), capture size 96 bytes
21:13:59.694057 IP 172.16.99.131 > 64.233.169.104: ICMP echo request, id 27405, seq 0, length 64
21:13:59.694058 IP 64.233.169.104 > 172.16.99.131: ICMP echo reply, id 27405, seq 0, length 64
On a single system you might wonder why you shouldn't just have your sniffing program watch edsc0 for ICMP traffic. Remember, here edsc0 is simulating a connection to a separate computer. Imagine a directly connected second system plugged into the interface represented by edsc0. The second computer would only see ICMP traffic.
Daemonlogger only supports one invocation of the -o switch, so you can't specify multiple outputs for one input. If you need that functionality, Pf can do it.. It's possible to run more than one instance of Daemonlogger, but you could encounter performance issues. Still, this soft tap functionality probably has other uses that I haven't imagined.