SimpleResolver: UDP response larger than the EDNS buffer fails with WireParseException instead of falling back to TCP #418

Summary

When a UDP response is larger than the EDNS payload size the query advertised, SimpleResolver fails with WireParseException ("end of input" / "truncated record"). It does not retry over TCP. From the caller's point of view the lookup just fails (Lookup.getResult() == TRY_AGAIN, getErrorString() == "network error"), even though the same query over TCP would succeed.

This can happen with a recursive resolver that appears to return UDP responses larger than the requester's advertised payload size. A resolver that does this isn't RFC 6891-compliant. Even so, dnsjava turns it into a hard failure, and the caller can't do anything about it because it never sees the raw bytes.

Versions

• dnsjava 3.6.3 (reproduced)
• The same code paths are unchanged on current master

What happens

1. SimpleResolver sends the query over UDP with the default EDNS payload size (1280).
2. NioUdpClient.Transaction.processReadyKey() reads into ByteBuffer.allocate(max), where max is that payload size. DatagramChannel.read() silently discards the part of the datagram that doesn't fit.
3. SimpleResolver.sendAsync() parses the cut-off bytes before it checks Flags.TC. Parsing throws WireParseException, so the existing "Got truncated response ..., retrying via TCP" path is never reached.
4. Lookup maps the IOException to TRY_AGAIN / "network error".

The TC-based retry doesn't help here because the TC bit is never evaluated. A resolver that sends the full response over UDP typically doesn't set TC anyway.

Reproduction

A UDP-only stub that answers any query with ~2.7 KB of TXT records, ignores the EDNS payload size, and never sets TC:

# oversized_stub.py
import socket, struct
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.bind(("127.0.0.1", 15353))
while True:
    q, addr = s.recvfrom(4096)
    i = 12
    while q[i] != 0:
        i += q[i] + 1
    question = q[12:i + 5]
    answers = b""
    for n in range(24):
        txt = ("record-%02d=" % n + "x" * 90).encode()
        rdata = bytes([len(txt)]) + txt
        answers += b"\xc0\x0c" + struct.pack(">HHIH", 16, 1, 60, len(rdata)) + rdata
    header = q[0:2] + b"\x81\x80" + struct.pack(">HHHH", 1, 24, 0, 0)  # QR RD RA, no TC
    s.sendto(header + question + answers, addr)

// Repro.java
import org.xbill.DNS.*;

public class Repro {
  public static void main(String[] args) throws Exception {
    SimpleResolver resolver = new SimpleResolver("127.0.0.1");
    resolver.setPort(15353);
    Message query = Message.newQuery(
        org.xbill.DNS.Record.newRecord(Name.fromString("example.com."), Type.TXT, DClass.IN));
    try {
      System.out.println("answers=" + resolver.send(query).getSection(Section.ANSWER).size());
    } catch (Exception e) {
      System.out.println(e.getClass().getName() + ": " + e.getMessage());
    }
  }
}

dig +bufsize=4096 -p 15353 @127.0.0.1 example.com TXT shows the stub's full response (ANSWER: 24, MSG SIZE rcvd: 2741, no tc flag).

Output with dnsjava 3.6.3 (org.xbill.DNS.SimpleResolver at DEBUG):

DEBUG SimpleResolver - Sending example.com./TXT, id=22819 to udp/127.0.0.1:15353
org.xbill.DNS.WireParseException: end of input

There is no "retrying via TCP" line. Through Lookup, the result is records=null result=2 error=network error.

Depending on where the cut falls, the exception can also be WireParseException: truncated record.

Expected

A UDP response that didn't fit in the receive buffer should be handled like a truncated response and retried over TCP, the same as a response with TC set. The retry should respect ignoreTruncation.

Related

#240 reported the same exception, but from application code parsing a received message itself. In this case the cut-off happens inside dnsjava's own UDP client, so the caller has no way to detect or handle it.