Wes Hardaker (b-root): We have a TLS version of b-root as a production service, would like feedback. Our RSSAC002 stats include encrypted traffic (although very, very small). Would like to move to QUIC - no blocker to moving to QUIC, just haven’t done it yet. Using BIND 9.18 for TLS today (we also use Knot). Service is architected to be safe and isolated from other services and can be taken down if problems arise.
Joe Abley (Cloudflare): We don’t provide any encrypted service today on auth servers (but 12% of 1.1.1.1 traffic is encrypted today). However we can share the CDN infrastructure that already terminates TLS and QUIC and could just ’turn it on’ - we are enthusiastic about the idea but no firm timelines. In principle all 3 transports are available but would only deploy what would be used.
Sara: Straw pole of the room and webex - is there anybody interested in doing DoH recursive to auth, otherwise we will confine the following discussion to just DoT/DoQ? No responses, so assume no interest in DoH for this.
Johan Stenstam (The Swedish Internet Foundation): I have a new draft in DNSOP called Authoritative DNS Transport Signaling which is an enhancement to RFC9539 using SVCB to opportunistically signal the transports supported by the authoritative server. I have prototype implementations and would like feedback on the draft.
Sara: Thank you for making us aware - encourage people to review that document.
Shane Kerr (IBM): We are doing an internal PoC - encrypting traffic from own internal recursive resolvers to own auths with target date of Q3 this year. The hope is that after that we can open this up for opportunistic probing by opening port 853 but current benchmarks for DoT are an order of magnitude less performant. Concerned that QUIC implementation status is not mature enough yet but would consider it in future. Concern around performance cliff of enabling DoT - the traffic level will increase immediately. We already see SYN packets probing port 853.
Peter van Dijk (PowerDNS): We know Google do RFC9539 probing so those packets may be from Google. Also Quad9 enabled probing yesterday.
Sara: Last time I saw published data from Google it was a few years ago and they said they have 6% egress traffic encrypted. Puneet is unfortunately not on the call although he wanted to be here so we can hopefully follow up after the meeting with him.
Shane Kerr: He is apparently willing to on do peer-to-peer controlled experiments with partners. We would be willing to engage with other experiments beyond RFC9539 around publishing credentials.
Babak Farrokhi (Quad9 - remote): We have been doing experiments for past year. We are concerned that we have seen no increase in adoption over the last 12 months and we have a chicken and egg problem. We do see the following doing DoT on their auths: Facebook, CDN77, b-root and some smaller ccTLGD (.gr, .co, .kg). Recursives won’t enable unless more auths deploy as it is not worth it so this collaboration could really help that.
Andrew Campling (419 Consulting): Do I remember that after the probing draft was published there was outreach to auth operators and no-one was interested in deploying this?
Sara: Maybe true 18 months ago but one of the reasons for this meeting is that there are currently multiple auth operators that do want to deploy this now.
Paul Hoffman (ICANN): I have heard about several pairs of partners doing private deployments before now but they were maybe not been willing to go public yet. So connecting those sets of partners could be a good outcome of this effort. But also I would like to see people who are experimenting with both DoT and DoQ privately publishing numbers to compare the 2 transports.
Allison Mankin (PCH - remote): We are looking very actively at this because customers keep asking us for it and we want to give a timeline. We have been doing measurements and looking at the technology and we want to report transparently to our clients. We have done experiments with Quad9 but we want a production service. We like this group because want to be able to share tips and experience on performance challenges between operators as we go forward. We have a wide variety of customers and they have been asking us about enabling this - it seems to be in their business plans or regulatory framework.
Josh Aas (ISRG/Let’s Encrypt - remote): Interested in encrypting to auths. We are investing in the development of Hickory - a recursive resolver DNS (written in Rust). If we can make encrypted connections to top 8 auth servers that we connect to we can protect traffic to half of the zones we query. If we can the top 10-12 auth servers, it gets us 70% of queries. So we only need a small number of auth operators to encrypt to have a big impact. Our plan is put together a list of auths that support any encrypted transport and publish this as a JSON file in a github repo ref. We will configure our recursive to use that list and if we need to query one of the auths we will try to use encrypted transport with no fallback to clear text. We are talking to the big ones such as Cloudlfare so we can get them in the list and start experimenting with encrypted transports and move that to production relatively quickly. The list is public but doesn’t have any entries yet but we see it as removing any blocker to immediate deployment and want it to be a collaborative effort. We want to move to something standards based from e.g. DELEG long term but would be open to other standards based solutions in the interim.
Jim Reid (RTFM): Sense of deja vu with DNSSEC adoption. I think that failed because the participant had up front costs with no real benefit and so the business case was not made. Are we making the same mistake again with this? More technical data is good but information on costs from auth operators would be great to so we can understand how to sell the business case to upper management. Can we find out what e.g. Comcast or Deutsche Telekom are doing from a business perspective?
Joe Abley: We should also not forget that while we might do this because privacy is nice…. UDP is awful. It is very cheap but it has baggage - having to worry about source address spoofing is an unpleasant side effect of the performance benefit. There are benefits in moving to stateful transports if we can understand how to manage the performance issues. If we can make UDP go away in the long term that would be a glorious future! [Note from Sara added after the meeting: See Do Not Accommodate Classic DNS over UDP and discussion].
Paul Hoffman: I have learnt that the web people giggle at how great QUIC is and how the DNS folks don’t understand this. QUIC for round trips is always as good or better than TLS for performance. And it is better in production for lossy environments but with identical security to TLS.
Sara: Someone expressed a concern that initial experiments with DoT where the performance isn’t as good as desired will deter people from even trying to DoQ and the way to counter that is if we can share information here on experiments with both.
Paul: With DoQ the knobs are not in the kernel (as they are with TLS) they are in the transport implementation so are easier to experiment with.
Sara: It is true that there is more experience today with DoT than DoQ, both in terms of implementation and operationally so it would be nice to see a shift in that.
Nick Williams (Infoblox but speak for self): We have been tracking the discussion on this internally for a long time and wanted to echo 2 sentiments. Most of our customers in are NOT requesting this, instead this is industry bringing it to others first. Some of our customers use DNSSEC and we have a lot of interested in experimenting with encrypted transports (we use BIND and CoreDNS) but we will need to take that to our customers, not the other way round.
Sara: Can you talk about what categories of customers you have that are not seeing the value of this? For example banks and governments are potential early adopters as with DNSSEC and I am interested in what business cases we can make to e.g. corporations, enterprises that are not seeing the value of this today?
Nick: Large service providers and major strategic organizations have well founded performance concerns about enabling this. But from a US centric stand point government organizations need to collaboration so we can bring them expertise on this, and for service providers and enterprise customers this is not even an agenda item today.
Sara: Thank you - that is very useful feedback.
Alister Woodman (NetDEF / FRR / EEF): I was in the EU Stakeholders meeting yesterday and one of their 4 categories for baseline operational standards is DNS and there were not many DNS people in the room. For EU corporations with large regulatory departments I expect they will start paying attention to this next year. NISA2 and CRA say if an company is following best practices and have an incident they are OK, if they were not there will be questions asked… You have an opportunity to get involved in that initiative early and lay out this as a best practice and influence adoption within the EU.
Sara: Thank you for raising that point. With that we need to wrap up - thank you to everyone who contributed and shared their plans.