Raw packet logging with iptables and Wireshark

I've been doing a lot of work with Asterisk and SIP recently and have found that viewing the raw SIP packets in Wireshark is an incredibly useful tool when debugging. The logs that Asterisk and various phones provide tend to give fairly cryptic error messages that don't really resemble the issue, plus some problems are caused by a packet not arriving at all due to the other end sending an incorrect media address etc.

We wanted to run a capture of all SIP traffic to a rotating log file, just grabbing all UDP would be a lot of data not to mention the fact that we'd be recording all of our clients calls

The first challenge is targeting just the SIP packets and not the other stuff.

There is one obvious way which is to capture UDP traffic on port 5060, in reality though different devices can use different port numbers to get around NAT or just because they feel like it I guess.

The tshark tool can do packet inspection and identify packets using a display filter

tshark -i any -Y 'sip'

Problem solved! Well only sort of, this works perfectly but has the downside of having to keep the whole capture in memory until you kill it. I'll save you the hours I spent trying to get this to work and say don't use tshark

There is another tool called dumpcap that simply writes the packets it sees to a capture file, this uses a tiny amount of memory as it doesn't do any analysis on traffic. Sounds perfect! Again only sort of, dumpcap can only apply capture filters and not display filters so we're back to basic port listening only.

What we really need is some sort of combination of these two tools; a way to apply a packet-inspection style filter before dumpcap gets them. Enter iptables and the NFLOG target. The simplest way I can think of to explain what this does is to say that it mirrors packets that the rule matches to a sort of virtual interface. That's not at all what it actually does but you hopefully get the idea.

Example time.

iptables -A INPUT -i wan -p tcp --dport 80 -j NFLOG --nflog-group 30

This will send all tcp packets aimed at port 80 to the, imaginary, logging interface which can then be targeted with dumpcap

dumpcap -i nflog:30 -w http.pcap

Any iptables rule can be used, in my case it was a string match on the SIP protocol

iptables -A INPUT -i wan -p udp -m string --algo bm --string "SIP/1.0" -j NFLOG --nflog-group 30
iptables -A INPUT -i wan -p udp -m string --algo bm --string "SIP/2.0" -j NFLOG --nflog-group 30

iptables -A OUTPUT -o wan -p udp -m string --algo bm --string "SIP/1.0" -j NFLOG --nflog-group 30
iptables -A OUTPUT -o wan -p udp -m string --algo bm --string "SIP/2.0" -j NFLOG --nflog-group 30

Problem solved! Completely! One small thing though - if you use an accept rule for whatever is being logged this must come before that and have the same matches, like this

iptables -A INPUT -i wan -p udp -s 123.234.124.214 -m string --algo bm --string "SIP/1.0" -j NFLOG --nflog-group 30
iptables -A INPUT -i wan -p udp -s 123.234.124.214 -j ACCEPT

The second challenge is getting this to run unattended

This bit is much more straight forward and has two parts; getting it to run automatically and rotating the log file.

Part 1.

As much as everyone randomly hates systemd is has made this sort of thing much easier than it used to be - you can basically run a script let is handle the whole start-stop-restart thing.

Create a .service file for the capture process at /etc/systemd/system/sip-capture.service, basically all this needs to define is the name and script path

[Unit]
Description=SIP packet logging
After=network.target

[Service]
ExecStart=/usr/bin/sip-capture

[Install]
WantedBy=multi-user.target

The cool bit about this is that if there is no stop defined systemd will just send a SIGTERM to the process - no more .pid files! It does need to be told about the new unit though

systemctl enable sip-capture.service

Part 2.

The log file for packets is going to be /var/log/sip-capture/packets.pcap which needs to be rotated fairly often to stop it getting giant. There is a tool for that already which is cool. Create a definition file for our log at /etc/logrotate.d/sip-capture

/var/log/sip-capture/packets.pcap {
    daily
    maxsize 102400k
    rotate 30
    compress
    missingok
    prerotate
        service sip-capture stop
    endscript
    postrotate
        service sip-capture start
    endscript
}

This will rotate the log file once a day or if it gets to 100mb compressing the old ones and keeping 30 copies. Before it does the rotation it will stop the capture and start it again after.

The script that run the capture, the one I referenced above but never mentioned until now, will need to rotate the log files before starting to make sure no data is lost. That's easy enough to do by calling logrotate manually although since the rotation will restart the service there is an infinite loop. A second rotation file is needed that will only be called by the script /usr/lib/sip-capture/pre-logrotate

/var/log/sip-capture/packets.pcap {
    daily
    maxsize 102400k
    rotate 30
    compress
    missingok
}

Exactly the same file as before but this one assumes the capture is not running.

Finally the script needs creating /usr/bin/sip-capture

#!/bin/bash

log_location="/var/log/sip-capture/packets.pcap"

touch $log_location
chmod 0600 $log_location

logrotate --force /ussr/lib/sip-capture/pre-logrotate

dumpcap -i nflog:30 -q -n 1 -w $log_location

All done. There is one tiny thing - the service will start itself even if you leave it disabled when the log rotation gets called, can't be bothered to solve that one at the moment.

Post a new comment