Command Line Tools

Tools that you run from the command line – i.e. from a terminal or shell window – are often more powerful and quicker to use than their GUI or web equivalents.

Their output is plain text so it’s much easier to copy and paste into an email or a slack conversation – sure, you can take a screenshot of a GUI tool and share that, but then the folks you’re sharing it with can’t copy the text out of it. And you can easily run them on a remote machine, which can be particularly useful when you’re diagnosing network issues, or email reputation issues that may be IP address based.

Here are some of the tools I use daily, and how to install them on your laptop. (If you’re installing these for a class I’m giving we might have an alternate way to use them if you didn’t install them ahead of time. Or we might not.)

dig

dig is a powerful tool for doing DNS queries, such as looking up MX records to work out where to send email, and TXT records for checking email authentication. It’s default output format can be a bit overwhelming, but you can use the +short flag to get a simpler format.

The full output format, though, gives you a lot of information that’s hidden by other tools and that can be useful for diagnosing more obscure issues so it’s good to have it handy.

Example: dig +short gmail.com mx

Documentation: man dig or online

Alternatives include host, drill, dnsget and web-based tools. And, as a last resort, nslookup but avoid that if you can.

netcat / nc

This tool is called "netcat" but it’s usually installed as nc. I’ve no idea why, just accept it as one of those things. At it’s simplest it lets you connect to a network server and type at it by hand. A lot of network protocols are text based, so you can often run full transactions – such as sending an email – using nc if the server is patient enough to wait for your typing.

We used to use telnet to send email by hand but telnet’s real job – to connect to a shell on a remote server – was replaced by ssh decades ago, and doing that is considered so insecure that all trace of it was removed from a lot of systems. nc can do much, much more but it’s a fine replacement for simple protocol testing.

Example: nc reject.wordtothewise.com 25 (use Ctrl-C to exit)

Documentation: man nc or online

Newer, better maintained alternatives include ncat (part of the nmap package), socat and many reimplementations that are still called nc and work much the same.

swaks

swaks describes itself as "the swiss army knife for SMTP", which is about right. It lets you send an email, and show the SMTP protocol as it does it. Almost everything you might want to customize, and many things you’d never though of, are customizable. You can send to a specific server, a non-standard port, use TLS in a variety of ways, authenticate to a smarthost, add custom headers, or abandon the delivery attempt at different points in the delivery.

Example: swaks --to discard@reject.wordtothewise.com

Documentation: swaks --help or online

whois

whois lets you identify who owns an IP address. (It used to let you find the owner of a domain name too, but that was made mostly useless some years ago).

whois works by contacting a server operated by one of the regional internet registries, sending them an IP address and printing out the response.

Example: whois 192.0.43.7

Alternatives: web based whois

curl

curl is mostly used as a command line tool for downloading files from webservers. But it lets you see bits of the http transaction you normally wouldn’t, such as when the server redirects you to somewhere else. That makes it useful for diagnosing click tracking redirectors.

If you give it the -D- flag it will fetch a web page and show you the headers the server responded with. A click tracker will usually use what’s called a "301 redirect", where it responds with the destination link in a "location:" header.

Example: curl -D- https://wttw.me/installtools

Documentation: man curl or online

Installation on Windows

Using WSL

WSL is "Windows Subsystem for Linux", Microsofts way of installing Linux tools on a Windows machine. Check out the full installation instructions but on modern Windows installations it’s often as simple as opening an administrator command prompt or powershell window and running wsl --install.

Once it’s installed you’ll need to set up a user and password, and then you’ll be able to follow the Linux/Ubuntu installation instructions.

Using native windows

Either install dig or use nslookup instead, e.g. nslookup -query=mx gmail.com.

Install statically compiled ncat.exe in place of netcat.

For whois either use a web based whois or install the sysinternals whois.

curl is available from the curl website

You can probably install swaks from CPAN after installing a native Perl from Strawberry or ActiveState. Probably. But that’s going to be painful, just install WSL.

Installation on macOS

You can get to a shell using "Terminal" in the "Utilities" subdirectory under "Applications". If you’re going to be using a shell much you should consider installing iterm – it’s much nicer.

dig, nc, whois and curl should be installed by default.

Perl is also installed by default, so you can install swaks using the "Unpackaged" or "Packaged" methods listed on the swaks installation page

Or, if you have homebrew installed you can do brew install swaks.

Installation on Linux

On Ubuntu or Debian based distributions:

sudo apt-get install dnsutils netcat-openbsd swaks whois curl

On anything RedHat-ish: sudo yum install bind-utils netcat swaks whois curl

Let someone else do the work

If you have access to a linux server on your network you can ssh in to it and run everything from there.

Related Posts

AOL converting all FBLs to ARF

AOL announced today that they are phasing out non-ARF feedback loops. As of September 2, 2008, no new non-ARF feedback loops will be created and all existing non-ARF feedback loops will be converted to ARF.
What is ARF?
ARF stands for Abuse Reporting Format. It is a standardized format intended to make processing of automated abuse reports (or feedback loop reports) easier. Word to the Wise has published tools to help recipients process ARF formatted reports and help developers create tools to handle ARF formatted reports. Abacus also supports ARF format out of the box.

Read More

Tools!

I just added a DMARC validation tool over on tools.wordtothewise.com.

You can give it a domain – such as ebay.com – and it will fetch the DMARC record, then explain and validate it. Or you can paste the DMARC record you’re planning to publish into it,  to validate it before you go live.
If you’ve not seen our tools page before, take a look. As well as DMARC we have a DKIM validator, SPF expander and optimizer, general DNS lookup tools, a bunch of RFCs covering all sorts of protocols, and base64 and quoted-printable decoders.
There’s also a widget that lets you add those little unicode pictures to your subject lines, whether you need a snowman ⛄, a forest ????, or a pig getting closer ???.
The results pages all have easily copyable URLs so they’re pretty good for sharing with co-workers or customers if you need that sort of thing.
(And if you need a cidr calculator, whois, or easy access to abuse.net & Microsoft SNDS check out Al’s xnnd.com.)

Read More

Who's publishing DMARC?

DMARC is a way for a domain owner to say “If you see this domain in a From: header and it’s not been sent straight from us, please don’t deliver the mail”. If a domain is only used for bulk and transactional mail, it can mitigate a subset of phishing attacks without causing too many problems for legitimate email.
In other cases, it can cause significant problems. Some of those problems impact discussion lists, but others cause problems for ESPs servicing small companies and individuals. ESP customers use their email addresses in the From: field; if they’re a small customer using the email address provided by their ISP, and that ISP publishes a DMARC record with p=reject, a large chunk of the mail they’re sending will bounce. When that happens recipients will stop getting their email, they’ll be removed from the mailing list due to bounces, and there’s some risk of blocks being raised against the sending IP address.
Because of that, it’s good to be able to see what consumer ISPs are doing with DMARC.
I’ve created a tool at dmarc.wordtothewise.com that regularly checks a list of large consumer ISPs and webmail providers and sees what DMARC records they’re publishing.
There are two main variants of DMARC records.
One is policy “reject” – meaning that mail that isn’t authenticated (or for which authentication has been broken in transit) will likely be rejected.
The other is policy “none” – meaning that the ISP publishing the record doesn’t want recipients to change their delivery decisions, but are asking for feedback about their mailstream, and how much of it fails authentication. That can mean that the ISP is evaluating whether or not to publish p=REJECT, or is in the process of deploying p=REJECT. Or it can just mean that they’re using DMARC to monitor where mail using their domain in the From: address is being sent from. There’s no way to tell which is the case unless they’ve made an announcement about their plans.
Hopefully this will be a useful tool to monitor DMARC deployment by consumer ISPs, and to help diagnose delivery problems that may be caused by DMARC.

Read More