troubleshooting dnssec

If you’re looking for DNSSEC troubleshooting tools, Carsten Strotmann’s article Delve deep into DNSSEC presents a new cli tool that will be available in BIND 9.10 called delv.

delv works very much like the already know dig. It is like dig with special DNSSEC validation powers. delv checks the DNSSEC validation chain using the same code that is used by the BIND 9 DNS server itself.

But keep in mind that delv, as any standard unix tool, will use the resolvers in /etc/resolv.conf to perform all lookups. If all your resolvers listed  are DNSSEC validating, delv will not be able to lookup a non-dnssec validating RR, and will not help you debug the problem.

[stsimb@jumbo ~]$ delv +multi +rtrace
 ;; fetch:
 ;; resolution failed: failure

You need to have access to some non-DNSSEC validating resolver, to kick start the trace..

[stsimb@jumbo ~]$ delv @ +multi +rtrace
 ;; fetch:
 ;; fetch: eu/DS
 ;; fetch: ./DNSKEY
 ;; fetch:
 ;; fetch: eu/DNSKEY
 ;; fetch: eu/DS
 ;; fetch: ./DNSKEY
 ;; fetch:
 ;; fetch: eu/DS
 ;; fetch: ./DNSKEY
 ;; fetch:
 ;; fetch: eu/DNSKEY
 ;; fetch: eu/DS
 ;; fetch: ./DNSKEY
 ;; no valid RRSIG resolving '':
 ;; no valid DS resolving '':
 ;; resolution failed: no valid DS

Personally I don’t consider this a bug. It’s expected behaviour, a unix tool to use the resolvers in /etc/resolv.conf for resolution.

delv cannot determine for sure if it’s your resolver’s fault, and then try another one (and if it did, which one should it choose?). It’s your job to have this in mind, and ask some other non-dncsec validating resolver that you have access to.

Nevertheless, this situation may come up many times. Troubleshooting broken trust chains is one of the reasons someone will use delv, and gives you a reason to keep a non dnssec-validating resolver available somewhere.

Update: Thanks to Peter van Dijk‘s tweet I discovered that +cdflag works in delv (although it’s not listed in delv -h).

Zimbra mailbox size per folder

It looks like Zimbra has an easy way to report message count per folder (zmmailbox getAllFolders), but not folder size.

Size information is buried in the database, and needs a few queries per folder to dig..

Here is a script to do it for you..

[zimbra@zimbra ~]$ /tmp/'s max mailbox size = 20480 MB, current mailbox size = 343.02 MB.

size (MB) msgcount unread folder
--------- --------- ---------- ----------------------------
0 0 0 /Chats
0 0 0 /Drafts
12 221 221 /Inbox
15 1001 1001 /Inbox/Tickets
310 7158 7158 /Inbox/Syslog
3 78 78 /Inbox/Wiki
0 0 0 /Junk
3 21 0 /Sent

increase ldap search results per user

By default, openldap limits search results to 500 for all users except root.

If you don’t want to increase it globally using sizelimit, then you can do it for specific users.

All you have to do is add something like the following in slapd.conf.

limits dn.exact="uid=serviceuser,ou=users,ou=domain,ou=com" size=unlimited

(manual reference)

sudoedit vs rvim

Recently I inherited some systems running Chef client, with a script that has to be called via sudo to edit some root owned Chef provisioned config files (e.g. munin.conf)..

/etc/sudoers contents looked like this

%sudoers hostname=(root) NOPASSWD: /usr/bin/chef-man-provision

and it actually executed rvim

rvim -u /root/.rvimrc /etc/munin/munin.conf

Contents of /root/.rvimrc look like this

" all following options are for securing vi for chef-man-provision script. 
" Security option: disable file opening with `:r` command
cmap r <Nop>
cmap o <Nop>

This «security option» had a pretty nasty effect… We couldn’t use characters «r» and «o» in vi command mode, even in simple search strings

Enter sudoedit! To quote the sudoedit(8) manpage

-e‘ The -e (edit) option indicates that, instead of running a command, the user wishes to edit one or more files. In lieu of a command, the string «sudoedit» is used when consulting the security policy. If the user is authorized by the policy, the following steps are taken:

1. Temporary copies are made of the files to be edited with the owner set to the invoking user.

2. The editor specified by the policy is run to edit the temporary files. The sudoers policy uses the SUDO_EDITOR, VISUAL and EDITOR environment variables (in that order). If none of SUDO_EDITOR, VISUAL or EDITOR are set, the first program listed in the editor sudoers(5) option is used.

3. If they have been modified, the temporary files are copied back to their original location and the temporary versions are removed.

If the specified file does not exist, it will be created. Note that unlike most commands run by sudo, the editor is run with the invoking user’s environment unmodified. If, for some reason, sudo is unable to update a file with its edited version, the user will receive a warning and the edited copy will remain in a temporary file.

So, /etc/sudoers was modified to look like this

%sudoers hostname=(root) NOPASSWD: sudoedit /etc/munin/munin.conf

We now type «sudoedit /etc/munin/munin.conf» and everything works as it should work in the first place, i.e. securely and we can use all the letters!

cvi going distributed

cvi was published in an earlier post: version control for important files

Going distributed instead of local can be tricky, because somebody else may have changed the file in the master repo.. but here is a cruel attempt..


if [ -z "$1" ] ; then
  echo edit a file stored in a git repo
  echo usage = $0 filename
  exit 1

git pull origin master || exit $?
vi $1
git add $1
git commit
git push

soacheck – report changes in domain serial numbers


I wanted to check how often a domain gets updated, so I quickly put together soacheck.

I assume that everybody increments the SOA serial number when they make a change or generate the zone file. But strictly speaking, they don’t have to do it (they can use some other way to sync changes to all their NSs and make them available).


List the zones you’re interested in file «zones-to-check». Execute soacheckd in your crontab (no need to run as root), as often as you like. e.g.

* * * * * /path/to/soacheckd > /dev/null

Check the results in soacheckd.log.

[stsimb@pipitsa soacheck]$ grep " gr. " soacheckd.log | tail
Fri Feb 1 04:51:01 EET 2013 soacheckd[22093] gr. serial changed from 1301312194 to 1302010099
Fri Feb 1 07:51:02 EET 2013 soacheckd[19481] gr. serial changed from 1302010099 to 1302010401
Fri Feb 1 10:52:01 EET 2013 soacheckd[18500] gr. serial changed from 1302010401 to 1302010695
Fri Feb 1 13:52:01 EET 2013 soacheckd[19079] gr. serial changed from 1302010695 to 1302011000


  • Record answers from authoritative servers (answers now are from system resolver’s cache)
  • Record answers from multiple servers
    • use a set of public dns servers
    • use some dns-lg instances around the world
  • Make multiple checks in parallel
  • Record changes of other RR types, e.g. A, AAAA, CNAME, PTR etc
  • Trigger some event on change