15dot4-detectiveAnalysis tool for understanding IEEE 802.15.4/6LoWPAN Captures
What is it?15dot4-detective is a Python script which interfaces to tshark. Tshark is a command-line version of the Wireshark packet analyzer.
Tshark understands all the same protocols that Wireshark does, this means for example IEEE 802.15.4, 6LoWPAN, RPL, TCP, UDP, and HTTP.
- Only IEEE 802.15.4 short addresses are understood/used. Your network can have long addresses in it, but those packets will be ignored. If you are using lots of long addresses this will throw off the resulting stats, because the tool won't be counting packets to/from those addresses.
- For IEEE 802.15.4 statistics, you must have captured with the 'zep' protocol. This will be fixed once a generic wpan.length or similar field is added to Wireshark.
How do I get it?Two options:
- You can get the script directly from the GIT repository. You will require the IPy python module, along with python.
- You can download a .exe for Windows if you don't have a python environment, see The Sourceforge Page
Where is the documentation?This page is basically everything (so far).
How it Worksdot4-detective uses TShark to dump certain fields (such as IPv6 length, protocol in use, etc) from either a .pcap file or a live capture.
As the data comes in, dot4-detective puts each packet into a list. The list contains a table of connections, for example from Node A to Node B. Over a 'minorInterval' time-frame it sums up each packet which is sent between those same two nodes, taking note of for example total IPv6 packets sent and TCP Retransmissions.
Once the minorInterval is over a total statistics are written to a file, which is the summation of all the connections in that time-frame.
Finally there is a larger majorInterval, over which the dot4-detective looks at the network and decides how many nodes are present in the network. If majorInterval is set too low, it won't count nodes which send infrequently. If majorInterval is set too high, the number of nodes may be inflated as it takes a long time before a node is considered removed.
The following diagram shows how major/minorInterval interact:
Usage: 15dot4-detective.py [options] 802.15.4 TShark Analysis Tool Options: -h, --help show this help message and exit -v, --verbose print more diagnostic messages (option can be given multiple times) -f FILENAME, --input-file=FILENAME Set the input filename to read from -o OUTNAME, --output-file=OUTNAME Set the output filename to write to -i INTERFACE, --interface=INTERFACE Set the interface to capture on -t INTERVALTIME, --interval=INTERVALTIME Interval to count nodes/connections over in seconds -r ROOTSHORT, --root=ROOTSHORT Root node short address -a ACTIVETHRESHOLD, --activity=ACTIVETHRESHOLD Activity Threshold, node must send more than this m 802.15.4 frames to appear in node counts -b TSHARK_EXEC, --tshark-bin=TSHARK_EXEC The tshark executable name/location -w TSHARK_FILTER, --filter=TSHARK_FILTER Additional capture/read filter to apply on tshark -m MINOR_INTERVAL, --stats-interval=MINOR_INTERVAL Interval statistics are calculated over -c, --connections Print connections between nodes and stats
Description in more detail:
|-i||Capture on a live interface, not functional yet|
|-f||Read from a .pcap file|
|-o||Write data about statistics, protocols, etc to a .CSV file|
|-a||Sometimes nodes will send only a single frame, or be trying many different 802.15.4 short addresses. Every time a unique short address is seen on the network, it counts it as a new node. By setting an activity threshold, the node must broadcast at least this many times to count as an active node.|
|-r||For calculating hop-counts, the root node must be known. Give the root short address with this option.|
|-s||Not actually implemented|
|-w||Not actually implemented|
|-m||set the minorInterval, defaults to 1.0 second, anything from 1-10 is a pretty reasonable value|
|-t||set the majorInterval. Good values between 30-600 seconds, depending how active all nodes are. If you set this too low, the count of number of nodes and what protocols they are running in the network will be too low.|
Analyzing a Log
A typical analysis section might look like:
15dot4-detective.py -f capture.pcap -t 600 -r 0x000E -a 5 -m 2 -o stats.csv
Note on Windows you may require a full path to tshark since it is probably not in your path. Use the -b option to pass this:
15dot4-detective.py -f capture.pcap -t 600 -r 0x000E -a 5 -b "c:\Program Files\Wireshark\tshark.exe" -m 2 -o stats.csv
Output File FormatFirst, there are several lines of text header telling you about how the program was called:
#*****PROCESSING BEGINS***** #Time: 2011-05-26 22:28:45 #1st Frame Time (relative) = 0.000023 #1st Frame Time (absolute) = May 19, 2011 11:02:38.850276000 #Interval: 120 #argv: ['C:\\Atmel\\workspace\\15dot4-tools\\detective\\15dot4-detective.py', '-f', 'E:\\Documents\\ZigBee\\Interop Events\\Dresden\\19th_darioSecondTestHTTPS\\capture.pcap', '-t', '120', '-r', '0x000E', '-a', '5', '-b', 'c:\\Program Files\\Wireshark\\tshark.exe', '-m', '10', '-o', 'stats.csv']
Next, the actual data comes in CSV format. A text header tells you what each column is:
time,HTTP Byte/sec,HTTP Packets/sec,HTTP Content-Len Byte/sec,TCP Byte/sec (not including retrans),TCP Retransmission Byte/sec,IEEE 802.15.4 (type == Data) Byte/sec,IEEE 802.15.4 (6LoWPAN) Byte/sec,IEEE 802.15.4 (type != Data) Byte/sec,IPv6 Originated Byte/sec,IPv6 Forwarded Byte/sec,HTTPS Byte/sec,RPL Byte/sec,IEEE 802.15.4 Retransmissions/sec,IEEE 802.15.4 Nodes,IPv6 Nodes,TCP Nodes,HTTP Nodes,HTTPS Nodes,1-hop IP Nodes,1-hop HTTP Nodes,1-hop HTTPS Nodes,1-hop IP Nodes,1-hop HTTP Nodes,1-hop HTTPS Nodes,2-hop IP Nodes,2-hop HTTP Nodes,2-hop HTTPS Nodes,3-hop IP Nodes,3-hop HTTP Nodes,3-hop HTTPS Nodes,4-hop IP Nodes,4-hop HTTP Nodes,4-hop HTTPS Nodes,5-hop IP Nodes,5-hop HTTP Nodes,5-hop HTTPS Nodes,6-hop IP Nodes,6-hop HTTP Nodes,6-hop HTTPS Nodes,7-hop IP Nodes,7-hop HTTP Nodes,7-hop HTTPS Nodes,8-hop IP Nodes,8-hop HTTP Nodes,8-hop HTTPS Nodes,9-hop IP Nodes,9-hop HTTP Nodes,9-hop HTTPS Nodes,10-hop IP Nodes,10-hop HTTP Nodes,10-hop HTTPS Nodes,
Followed by a line of data:
This example shows data dumped from the T=10 timestamp. At this moment in time there is 839.5 byte/second of 802.15.4 traffic (counting 802.15.4 PHY PDU, so no PHY headers) for example.
Version 0.03 (June 1, 2011)
- If ZEP protocol isn't used, still parse data. Also use frame.len as the IEEE 802.15.4 frame length if zep.length doesn't exist.
Version 0.02 (May 26, 2011)
- This version uses different command-line arguments from 0.01
- Structure changed around, now uses a sliding majorInterval to calculate number of nodes more often, results in more accurate topology.
- TODO: Work without ZEP protocol, at least check it doesn't crash
- TODO: Output ASCII-art graph of network topology
Version 0.01 (May 25, 2011)
- Initial Release