Java and SkyDNS

Java and SkyDNS

SkyDNS is a good option if you are looking for a valid service discovery system. Instead of using custom solutions, as stated by Old is Gold philosophy, why not using a DNS server for service discovery? Spotify use this “old” technology in their stack, and convert a name in to an URI is exactly the DNS job (yes, you can specify a port using SRV records). By the way, it is exactly what AWS Route53 does…

IMHO, DNS for service discovery has at least the following disadvantages:

  • records does not expire there is nothing like heartbeath defined in the protocol
  • changes propagates quite slowly (it tokk about 90 seconds to Route53 to propagate a change - which is really fast for a global DNS system, but quite slow for a backend service discovery - think about a broken service that is not being replaced for less than 90 seconds…)

I found SkyDNS, which is a DNS server written in GO that uses etcd as configuration directory, that also allows a TTL at record level - yes, DNS record can expire! I created a test zone, and everything was working with dig and nslookup. Then I tried a simple java class

public class Main {

    public static void main(String[] args) throws UnknownHostException {

        System.setProperty("sun.net.spi.nameservice.nameservers", "");
        System.setProperty("sun.net.spi.nameservice.provider.1", "dns,sun");

        catch (Exception e) {e.printStackTrace();}

        catch (Exception e) {e.printStackTrace();}



The properties sun.net.spi.nameservice.* are needed to tell Java to not use the system DNS (there are more settings that you need to tune, eg for avoid name caching)

But running the above class resulted in this exception:

  java.net.UnknownHostException: DNS name not found [response code 3]
  	at sun.net.spi.nameservice.dns.DNSNameService.resolve(DNSNameService.java:180)
  	at sun.net.spi.nameservice.dns.DNSNameService.lookupAllHostAddr(DNSNameService.java:291)
  	at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323)
  	at java.net.InetAddress.getAllByName0(InetAddress.java:1276)
  	at java.net.InetAddress.getAllByName(InetAddress.java:1192)
  	at java.net.InetAddress.getAllByName(InetAddress.java:1126)
  	at java.net.InetAddress.getByName(InetAddress.java:1076)
  	at ztest.dns.Main.main(Main.java:20)

Mh. Strange, since dig is working. So I tried a packet analysis using Wiresharck, and BOOM: the sun DNS implementation was doing a request for ANY jhon.services. Looking at the code of SkyDNS, we have confirmation that it is blocking request for ANY records (and by the way is The Right Thing (TM), since this is the DDOS Amplification technique used by any script kiddie).

Ok, so to fix we have to tell to the Java DNS resolver to not use ANY. Good for us, there is another great DNS implementation that we can use in Java, which is dnsjava. We just need to use this resolver instead of the one shipped with Java, by setting the right property: System.setProperty("sun.net.spi.nameservice.provider.1", "dns,dnsjava") (of course dnsjava must be in the classpath).

The ANY record is issued with the default Java resolver, that means it is not possible to use SkyDNS with libraries like Apache HTTP Commons, or RestAssured