ADVANCED CONSOLE FUNCTIONALITY
Getting the Latest Versions
Make sure you get the new versions, by running, waiting for 30 seconds, then closing and running again (java webstart downloads the new version in the background when you launch, and then is what it runs the next time you launch).
Filtering/focusing/flagging
When you highlight a node or bot, the menu has filter, focus, and flag options. These are for pruning out data you don't care about, leaving data you want to analyze further. Filter will remove all matching traffic. Focus will remove all non-matching traffic. Flag will make matching traffic NOT be removed by subsequent filter/focus operations.
The "bots" are patterns generated by Deep Node's trade secret artificial life algorithm, or entered manually by the user. To hide them, hit TAB. To turn them back on, hit TAB again. When you highlight one with the mouse or by cycling through using i and o, all matching traffic highlights in blue. Within the "workspace" section of the timewell, the highlighted bot draws lines to connect to all blips which contain matching traffic.
If you have filtered out traffic and want to get rid of limbs which now have no traffic on them, hit p (prune).
Creating a bot from the command line:
If you go into the command line in the console (hit HOME to show it, q to be able to type into it, HOME to turn it back off - also hitting q turns it on automatically)...
There are commands called "bot", "mark", "filter", "focus", and "flag"; just type "mark from" and hit enter to see the format... you can create a bot for any pattern... like
bot from external|1e100.net|*|* to *|*|*|*
will create a bot that will grab all traffic to and from the 1e100.net domain. This new bot will appear in the middle of the timewell. You can highlight it, and currently there are two options:
- mark bot: mark the blips the bot has already connected to
- automark: mark the current blips and all new blips the bot connects to
- filter: get rid of all matching traffic
- focus: show only matching traffic
- delete: get rid of this bot
Every message that comes in to the timewell has a source and destination; source and destination are each four strings... for example, a tcp packet from google to a local pc might have:
source: external 1e100.net 10.20.30.123 tcp80
destination: internal internal 192.168.1.23 tcp
If you wanted to create a "bot" to highlight this traffic, you could do:
bot from external|1e100.net|10.20.30.123|tcp80 to internal|internal|192.168.1.23|tcp
But it would probably be much more useful to highlight, say, all google traffic:
bot from external|1e100.net|*|* to *|*|*|*
Or, you might want to highlight all external, tcp80 traffic:
bot from external|*|*|tcp80 to *|*|*|*
If you want the bot you create to filter out matching traffic rather than just marking it, use "filter" instead of "bot"; to show ONLY the matching traffic and filter out everything else, use "focus"; to ensure that matching traffic is never filtered, use "flag". Use "mark" to create a bot which is in "automark" mode: marking all matching blips.
You can create bots using regex also. For example this command:
mark rx from .*(tcp).* to .*
... will mark all tcp traffic. You can also add a regex expression which will be tested against the comma-delimited list of tags, like this:
mark rx from .*(tcp).* to .* with .*(e0:aa:03:7f:b3:25).*
This will mark only tcp traffic, only involving a particular mac address.
By default, filters simply cause data to not be displayed - but it is still processed and kept in memory so that if you reset a filter, it displays instantly. If you have too much data coming in to the timewell, you may want to use your filters to completely drop matching messages:
The command "gate" will cause filters to drop messages.
The command "nogate" will cause all messages to be processed regardless of filters.
This just introduced you to the command line within the console. You can enter "help" in there to get a list of available commands. Commands which are not built in are passed through to an underlying shell process; so you can do things like "ls" on linux or "dir" on windows.
_h macro in command line
When you select a path in the timewell by clicking the mouse (thus highlighting the selected path in blue, all connected paths in grey, and all corresponding blips in blue), the host level (level 3) of the path is automatically stored in the "_h" macro. This macro is usable in the command line; for example, clicking the path "external|1e100.net|10.11.12.33|tcp80", then hitting "q" to go into typing mode in the command line, and then entering "ping _h" will cause the command line to execute the command "ping 10.11.12.33" in the shell.
Labelling Nodes
The command "label <node> <string>" will add a text label to the given node. <node> is the full path of the node - like "external|1e100.net". The "_k" macro is really useful here - it gives you the path of the currently highlighted node. Try going into the command line, then highlighting a node with your mouse or gamepad, then typing "label _k Hello". The text "Hello" will appear in gold over the sphere you highlighted. Note that highlight text should not contain spaces.
Saving and Loading Rules
The command "rule dump <filepath>" will save all of your labels and filters (bots that are filtering, focusing, or flagging) to the file you specify. Leave out the filepath to save to the default file, which is automatically loaded when you start up the console.
The command "rule load <filepath>" will load labels and filters from the file you specify.
Saving and Replaying Messages
1) once you have the console loaded up with data you want to save, press "q". This will turn on the command line within the console. Then type "save <filename>" and hit enter, substituting the full path where you want to save the file - like c:\users\bob\dnsave.txt. Once done, you can turn off the command line by pressing HOME key. The console will save only unfiltered messages, to the file you specify.
2) now from some other place and time, where you have that file, you can launch the console, then launch the sniffer - but instead of pressing the "MONITOR ALL" button on the sniffer, press the "LOAD FILE" button and select your file...
3) the sniffer will pump the messages from the file into the console in rapid ingest mode, and then stop at the end... press "[" or "]" to get things moving and move around in time in the dataset you just loaded
Peering Inside a Blip
If you highlight a blip and then select "show contents" from the HUD menu, you will be taken to a view of all of the messages contained within the blip. The messages are laid out in a path from oldest to newest, with header and tag information displayed. The width of each message is proportional to the size of the message, and each message has an arrow indicating whether it was inbound (pointing toward you) with respect to the path the blip is on, or outbound (pointing away from you).
The path is laid out so that you can look down a little bit (using the down arrow) and then fly along it with the 'w' key (go backward with 's') to review the entire set of messages. Use 'v' to increase your speed.
If you are using the sniffer feed and you mouse over a message, and the message is still in the feed's cache, you will receive a histogram showing the contents of the actual packet. Use 'a' and 'd' to move sideways to get into the viewing position you want.
The packet is shown as a 3d histogram. The first byte in the packet is the top-left corner; the last byte is the bottom right corner. Each cell in the histogram has height and brightness proportional to the byte value (255 is highest, 0 is lowest). If a byte is within the ASCII range, the corresponding ASCII character is shown. If the header length is known, the header bytes will be highlighted in blue.
Press 'b' to go back to where you were before you selected "show contents". Press 'c' to clear out the view and go back to the well.
Consolidating and Separating Nodes
If a node and its descendants are taking up too much space, highlight the node and choose "consolidate" from the menu. All desendants of the node will be collapsed down (consolidated) into one line. Blips along that line will accumulate all the messages involving all the descendants.
To undo the consolidation, highlight the node and choose "separate". The descendants will be split back out into individual nodes. The timewell has configurable thresholds for auto consolidation of nodes in the second and third levels from root. Once you "separate" a node, it will no longer be subject to auto consolidation.
"Rewind to Here" Blip Menu Option
Choosing the "rewind to here" option from the menu when you have a blip highlighted will reset the timewell (clear all nodes and traffic) and start it back from the point in time represented by the blip.
Saving a PCAP With the Packets You Want (Pro Version Only)
The blip menu "mark" option will mark all messages contained in the blip. You can browse around, marking the blips you want to research further... then hover over the root node (center of the timewell) and choose the "pcap" option to make the sniffer feed save the packets corresponding to all the marked messages into a file name "dn.pcap" and located in the .deepnode directory. This is a great way to pre-filter a pcap so you aren't overwhelmed with data in other tools.
Turn on the Java Console
Lots of debugging information is written out to the "Java Console" by the Deep Node Console, but the java console is turned off by default. To change this, go into your Java control panel widget, choose the Advanced tab, and select "show console".
Make sure you get the new versions, by running, waiting for 30 seconds, then closing and running again (java webstart downloads the new version in the background when you launch, and then is what it runs the next time you launch).
Filtering/focusing/flagging
When you highlight a node or bot, the menu has filter, focus, and flag options. These are for pruning out data you don't care about, leaving data you want to analyze further. Filter will remove all matching traffic. Focus will remove all non-matching traffic. Flag will make matching traffic NOT be removed by subsequent filter/focus operations.
The "bots" are patterns generated by Deep Node's trade secret artificial life algorithm, or entered manually by the user. To hide them, hit TAB. To turn them back on, hit TAB again. When you highlight one with the mouse or by cycling through using i and o, all matching traffic highlights in blue. Within the "workspace" section of the timewell, the highlighted bot draws lines to connect to all blips which contain matching traffic.
If you have filtered out traffic and want to get rid of limbs which now have no traffic on them, hit p (prune).
Creating a bot from the command line:
If you go into the command line in the console (hit HOME to show it, q to be able to type into it, HOME to turn it back off - also hitting q turns it on automatically)...
There are commands called "bot", "mark", "filter", "focus", and "flag"; just type "mark from" and hit enter to see the format... you can create a bot for any pattern... like
bot from external|1e100.net|*|* to *|*|*|*
will create a bot that will grab all traffic to and from the 1e100.net domain. This new bot will appear in the middle of the timewell. You can highlight it, and currently there are two options:
- mark bot: mark the blips the bot has already connected to
- automark: mark the current blips and all new blips the bot connects to
- filter: get rid of all matching traffic
- focus: show only matching traffic
- delete: get rid of this bot
Every message that comes in to the timewell has a source and destination; source and destination are each four strings... for example, a tcp packet from google to a local pc might have:
source: external 1e100.net 10.20.30.123 tcp80
destination: internal internal 192.168.1.23 tcp
If you wanted to create a "bot" to highlight this traffic, you could do:
bot from external|1e100.net|10.20.30.123|tcp80 to internal|internal|192.168.1.23|tcp
But it would probably be much more useful to highlight, say, all google traffic:
bot from external|1e100.net|*|* to *|*|*|*
Or, you might want to highlight all external, tcp80 traffic:
bot from external|*|*|tcp80 to *|*|*|*
If you want the bot you create to filter out matching traffic rather than just marking it, use "filter" instead of "bot"; to show ONLY the matching traffic and filter out everything else, use "focus"; to ensure that matching traffic is never filtered, use "flag". Use "mark" to create a bot which is in "automark" mode: marking all matching blips.
You can create bots using regex also. For example this command:
mark rx from .*(tcp).* to .*
... will mark all tcp traffic. You can also add a regex expression which will be tested against the comma-delimited list of tags, like this:
mark rx from .*(tcp).* to .* with .*(e0:aa:03:7f:b3:25).*
This will mark only tcp traffic, only involving a particular mac address.
By default, filters simply cause data to not be displayed - but it is still processed and kept in memory so that if you reset a filter, it displays instantly. If you have too much data coming in to the timewell, you may want to use your filters to completely drop matching messages:
The command "gate" will cause filters to drop messages.
The command "nogate" will cause all messages to be processed regardless of filters.
This just introduced you to the command line within the console. You can enter "help" in there to get a list of available commands. Commands which are not built in are passed through to an underlying shell process; so you can do things like "ls" on linux or "dir" on windows.
_h macro in command line
When you select a path in the timewell by clicking the mouse (thus highlighting the selected path in blue, all connected paths in grey, and all corresponding blips in blue), the host level (level 3) of the path is automatically stored in the "_h" macro. This macro is usable in the command line; for example, clicking the path "external|1e100.net|10.11.12.33|tcp80", then hitting "q" to go into typing mode in the command line, and then entering "ping _h" will cause the command line to execute the command "ping 10.11.12.33" in the shell.
Labelling Nodes
The command "label <node> <string>" will add a text label to the given node. <node> is the full path of the node - like "external|1e100.net". The "_k" macro is really useful here - it gives you the path of the currently highlighted node. Try going into the command line, then highlighting a node with your mouse or gamepad, then typing "label _k Hello". The text "Hello" will appear in gold over the sphere you highlighted. Note that highlight text should not contain spaces.
Saving and Loading Rules
The command "rule dump <filepath>" will save all of your labels and filters (bots that are filtering, focusing, or flagging) to the file you specify. Leave out the filepath to save to the default file, which is automatically loaded when you start up the console.
The command "rule load <filepath>" will load labels and filters from the file you specify.
Saving and Replaying Messages
1) once you have the console loaded up with data you want to save, press "q". This will turn on the command line within the console. Then type "save <filename>" and hit enter, substituting the full path where you want to save the file - like c:\users\bob\dnsave.txt. Once done, you can turn off the command line by pressing HOME key. The console will save only unfiltered messages, to the file you specify.
2) now from some other place and time, where you have that file, you can launch the console, then launch the sniffer - but instead of pressing the "MONITOR ALL" button on the sniffer, press the "LOAD FILE" button and select your file...
3) the sniffer will pump the messages from the file into the console in rapid ingest mode, and then stop at the end... press "[" or "]" to get things moving and move around in time in the dataset you just loaded
Peering Inside a Blip
If you highlight a blip and then select "show contents" from the HUD menu, you will be taken to a view of all of the messages contained within the blip. The messages are laid out in a path from oldest to newest, with header and tag information displayed. The width of each message is proportional to the size of the message, and each message has an arrow indicating whether it was inbound (pointing toward you) with respect to the path the blip is on, or outbound (pointing away from you).
The path is laid out so that you can look down a little bit (using the down arrow) and then fly along it with the 'w' key (go backward with 's') to review the entire set of messages. Use 'v' to increase your speed.
If you are using the sniffer feed and you mouse over a message, and the message is still in the feed's cache, you will receive a histogram showing the contents of the actual packet. Use 'a' and 'd' to move sideways to get into the viewing position you want.
The packet is shown as a 3d histogram. The first byte in the packet is the top-left corner; the last byte is the bottom right corner. Each cell in the histogram has height and brightness proportional to the byte value (255 is highest, 0 is lowest). If a byte is within the ASCII range, the corresponding ASCII character is shown. If the header length is known, the header bytes will be highlighted in blue.
Press 'b' to go back to where you were before you selected "show contents". Press 'c' to clear out the view and go back to the well.
Consolidating and Separating Nodes
If a node and its descendants are taking up too much space, highlight the node and choose "consolidate" from the menu. All desendants of the node will be collapsed down (consolidated) into one line. Blips along that line will accumulate all the messages involving all the descendants.
To undo the consolidation, highlight the node and choose "separate". The descendants will be split back out into individual nodes. The timewell has configurable thresholds for auto consolidation of nodes in the second and third levels from root. Once you "separate" a node, it will no longer be subject to auto consolidation.
"Rewind to Here" Blip Menu Option
Choosing the "rewind to here" option from the menu when you have a blip highlighted will reset the timewell (clear all nodes and traffic) and start it back from the point in time represented by the blip.
Saving a PCAP With the Packets You Want (Pro Version Only)
The blip menu "mark" option will mark all messages contained in the blip. You can browse around, marking the blips you want to research further... then hover over the root node (center of the timewell) and choose the "pcap" option to make the sniffer feed save the packets corresponding to all the marked messages into a file name "dn.pcap" and located in the .deepnode directory. This is a great way to pre-filter a pcap so you aren't overwhelmed with data in other tools.
Turn on the Java Console
Lots of debugging information is written out to the "Java Console" by the Deep Node Console, but the java console is turned off by default. To change this, go into your Java control panel widget, choose the Advanced tab, and select "show console".