Skip to main content

DNS

Overview​

The Edge Enforces starts its own DNS name server on each host in an edge. The name server provides three important functions:

  • Act as an authoritative name server for the application container instances and their names in the application network domain.
  • Act as a recursive resolver for application containers (and hosts on the same network as the edge enforcers).
  • Act as an authoritative name server for application container instances that has ingress IP addresses.

The dns server is configured to:

  • Answer both UDP and TCP requests (on the default dns port, 53)
  • Perform recursive lookups for requests coming from application containers
    • In case the host was configured to use an upstream name server it will be used as a forwarder
    • Recursive lookups from external sources (other than the locally attached networks) are rejected
  • Accept queries for its local site domain, both from applications and external queries (and answer authoritatively)

The application network domain, ".internal"​

As explained in Application networking section each service instance is connected to an application network. It is shared between all services of an application (and can be shared with other applications when using the shared-application-network configuration). Each application forms its own domain using the application name and .internal as the top-level domain, for example if the application is named theater-room-manager (as in this example) its domain is theater-room-manager.internal. Application domains are only visible to applications that share the application network.

When a container is started it will get an /etc/resolv.conf file pointing to the local name server. The .internal domain, as well as the application name plus .internal are configured as the default domains to search in, for example:

nameserver 172.25.252.1
search theater-room-manager.internal internal

The effect of the search ... configuration is that all containers, in all services in the same application can use "short names" (i.e. just the service name, for example curtain-controller) to refer to each other. However, note that if there are several applications using a shared application network, you must use at least "service dot application" to lookup another application. For example, if the theater-room-manager applications used a shared network together with the Redis application (which has a single service called "redis") you would have to use redis.redis, to refer to Redis from e.g. the Curtain Controller, and curtain-controller.theater-room-manager to do vice versa. (It is of course always possible to use the full domain name redis.redis.internal and curtain-controller.theater-room-manager.internal.)

Static records in the application network domain​

In order to make it easier to communicate over the application network each service instance is added as an A record in the applications domain. These records are added as soon as the service instance is started, and resolve to the service instance IP address on the application network. Using the theater-room-manager example: the application has two services (theater-operations and curtain-controller) each running with one replica. As soon as these service instances are started the names curtain-controller-1.theater-room-manager.internal and theater-operations-1.theater-room-manager.internal are added.

It is possible to see the records added for an instance, for example using the supctl command line tool:

% supctl show --site stockholm-sergel applications theater-room-manager service-instances theater-operations-1 --fields application-network
application-network:
  ips:
    - 172.26.0.2/16
  dns-records:
    - theater-operations-1.theater-room-manager.internal. 15 IN A 172.26.0.2
    - theater-operations.theater-room-manager.internal. 15 IN A 172.26.0.2

% supctl show --site stockholm-sergel applications theater-room-manager service-instances curtain-controller-1 --fields application-network
application-network:
  ips:
    - 172.26.0.2/16
  dns-records:
    - curtain-controller-1.theater-room-manager.internal. 15 IN A 172.26.0.1
    - curtain-controller.theater-room-manager.internal. 15 IN A 172.26.0.1

Dynamic records in the application network domain​

Using the instance name (as in curtain-controller-1) is useful when we know which instance we would like to connect to. When you have several instances you may want to connect to "any one of the instances that are ready". This is possible using the service name without the instance number, i.e. curtain-controller, this is how it works:

When a service instance is ready (see the ready state and probe concepts explained in Application probes) an A record with just the service name is added. This means that using the service name (without the instance number) will always return as many IP addresses as there are instances ready. When this name is looked up, the returned addresses are rotated in a round-robin fashion, thus trying to connect to the instance name will evenly distribute connections between instances.

Again, let us use the theater-room-manager application as an example. If the curtain-controller replicas is increased to 3, then, apart from the curtain-controller-1, curtain-controller-2, and curtain-controller-3 entries there will be three A records with the name curtain-controller.theater-room-manager.internal added (when the three services are all ready). Using nslookup in one of the containers illustrates this:

% supctl --site stockholm-sergel applications theater-room-manager service-instances theater-operations-1 containers projector-operations exec "nslookup curtain-controller.theater-room-manager.internal."
Server:         172.25.253.1
Address:        172.25.253.1:53

Name:   curtain-controller.theater-room-manager.internal
Address: 172.26.0.4
Name:   curtain-controller.theater-room-manager.internal
Address: 172.26.0.3
Name:   curtain-controller.theater-room-manager.internal
Address: 172.26.0.1

If an instance becomes unready its A record is removed, so when:

% supctl show --site stockholm-sergel applications theater-room-manager service-instances curtain-controller-3 --fields name,ready
ready
name: curtain-controller-3
ready: false

it is reflected like so:

% supctl --site stockholm-sergel applications theater-room-manager service-instances theater-operations-1 containers projector-operations exec "nslookup curtain-controller.theater-room-manager.internal."
Server:         172.25.253.1
Address:        172.25.253.1:53

Name:   curtain-controller.theater-room-manager.internal
Address: 172.26.0.4
Name:   curtain-controller.theater-room-manager.internal
Address: 172.26.0.1

DNS and ingress networks​

When a service instance has an address in the ingress network (configured in the application spec using ingress-ip-per-instance) that address is also added to the name server in a similar, but slightly different, manner as in the application network domain. The difference is the domain used, and the types of records that can be added.

Default site domain​

Each site in an environment is given its own domain by combining the site name with the global domain of the environment. For example if the environment is called example.acme.avassa.net and the site is called stockholm-sergel the site domain will be stockholm-sergel.site.example.acme.avassa.net. The name server running on each host will respond authoritatively to queries for this domain from external addresses as well (but it will not do recursive lookups unless the source is from a local network).

Application specific records​

Just like in the application network domain, A records are added for each service instance (and when the service is ready for the service name itself). The difference is that the domain that the record is added to is <TENANT>.<SITE-NAME>.<GLOBAL-DOMAIN>, i.e. each tenant on a site will have a unique subdomain in which its applications are registered. The full record per instance will be <SERVICE>-<INSTANCE-ID>.<APPLICATION>.<TENANT>.<SITE-NAME>.<GLOBAL-DOMAIN>. For example, if the curtain-controller service (from the example above) was deployed to the site stockholm-sergel by the tenant curtainco, in the environment called example and the organization acme, it could look something like this:

% supctl show --site stockholm-sergel applications theater-room-manager service-instances curtain-controller-1 --fields ingress
ingress:
  ips:
    - 192.168.100.33
  dns-records:
    - curtain-controller-1.theater-room-manager.curtainco.stockholm-sergel.site.example.acme.avassa.net. 15 IN A 192.168.100.33
    - curtain-controller.theater-room-manager.curtainco.stockholm-sergel.site.example.acme.avassa.net. 15 IN A 192.168.100.33

To reference this domain from another application, the FQDN can be paramaterized as curtain-controller.theater-room-manager.${SYS_TENANT}.${SYS_SITE}.${SYS_GLOBAL_DOMAIN}.

In addition to the A records added for each service, it is possible to add SRV records to the application specification, each instance, when ready, will add a SRV record pointing to an instance.

Extending the curtain controller ingress configuration with a dns-records: section:

    network:
      ingress-ip-per-instance:
        protocols:
          - name: tcp
            port-ranges: 80,443
        dns-records:
          domains:
            - domain: default
              srv:
                - name: _http._tcp
                  priority: 20
                  port: 80
                - name: _https._tcp
                  priority: 10
                  port: 443

Yields the following:

% supctl show --site stockholm-sergel applications theater-room-manager service-instances curtain-controller-1 --fields ready,ingress
ready: true
ingress:
  ips:
    - 192.168.100.33
  dns-records:
    - _https._tcp.theater-room-manager.curtainco.stockholm-sergel.site.example.acme.avassa.net. 15 IN SRV 10 100 443 curtain-controller-1.theater-room-manager.curtainco.stockholm-sergel.site.example.acme.avassa.net.
    - _http._tcp.theater-room-manager.curtainco.stockholm-sergel.site.example.acme.avassa.net. 15 IN SRV 20 100 80 curtain-controller-1.theater-room-manager.curtainco.stockholm-sergel.site.example.acme.avassa.net.
    - curtain-controller-1.theater-room-manager.curtainco.stockholm-sergel.site.example.acme.avassa.net. 15 IN A 192.168.100.33
    - curtain-controller.theater-room-manager.curtainco.stockholm-sergel.site.example.acme.avassa.net. 15 IN A 192.168.100.33

when the curtain-controller-1 service instance is ready, and:

% supctl show --site stockholm-sergel applications theater-room-manager service-instances curtain-controller-1 --fields ready,ingress
ready: false
ingress:
  ips:
    - 192.168.100.33
  dns-records:
    - curtain-controller-1.theater-room-manager.curtainco.stockholm-sergel.site.example.acme.avassa.net. 15 IN A 192.168.100.33

when the service instance isn't.

Application specific site records​

In addition to the SRV records that can be added when a service instance is ready, it is also possible to add SRV, CNAME, and NAPTR records as soon as an application is started on a site. These records are configured in the application spec under site-dns-records.

For example, if we would like a CNAME called cc and point it to the curtain-controller service, then extend the service in the theater-room-manager application with:

    site-dns-records:
      domains:
        - domain: default
          cname:
            - name: cc
              cname: curtain-controller.theater-room-manager.${SYS_DNS_ZONES[default]}

To see all name server records that are installed in a site use the state endpoint of the dns object. Note that it is site specific, to select a particular site, include the --site parameter to supctl.

% supctl show --site stockholm-sergel dns
zones:
  - name: default
    domain: curtainco.stockholm-sergel.site.example.acme.avassa.net
    records:
      - rr: _https._tcp.theater-room-manager 15 IN SRV 10 100 443 curtain-controller-1.theater-room-manager.curtainco.stockholm-sergel.site.example.acme.avassa.net.
      - rr: _http._tcp.theater-room-manager 15 IN SRV 20 100 80 curtain-controller-1.theater-room-manager.curtainco.stockholm-sergel.site.example.acme.avassa.net.
      - rr: curtain-controller-1.theater-room-manager 15 IN A 192.168.100.33
      - rr: curtain-controller.theater-room-manager 15 IN A 192.168.100.33
      - rr: cc.theater-room-manager 15 IN CNAME curtain-controller.theater-room-manager.curtainco.stockholm-sergel.site.example.acme.avassa.net

Custom site domains​

In addition to the Avassa provided domains that each environment has, it is possible to define custom domains and have the same type of records as described above for applications automatically added to these domains as well.

Custom domains are configured per tenant, but only a site-provider tenant can add these domains. A tenant can only add application specific records that are in theses configured domains (trying to add records to other domains will fail, with a warning).

To add custom domains, edit the dns object for the tenant that should have the domain. The dns object has a list called zones and each zone is given a name (to be able to refer to it) and a domain. The domain can be "static", i.e. it will be the same for all sites, for example:

% supctl show tenants curtainco dns --config
zones:
  - name: example
    domain: example.com

But it is more common that one would like to have a unique domain per site. In order to do so it is possible to refer to variables, e.g. it is possible to access the site name using the ${SYS_SITE} variable, or labels using the ${SYS_SITE_LABELS[label]} variable. The variables available are:

  • SYS_SITE the name of the current site.
  • SYS_TENANT the name of the tenant for which this domain is added.
  • SYS_GLOBAL_DOMAIN the name of the top level domain used by all sites in the environment.
  • SYS_SITE_LABELS an array of all the system labels and site provider tenant labels. I.e. if a tenant has their own labels they will not be included here.

The default domain can then be described with the following expression: ${SYS_TENANT}.${SYS_SITE}.${SYS_GLOBAL_DOMAINS}.

Application records in custom domains​

To use the custom domain in an application it needs to be explicitly included in the dns-records or site-dns-records elements in the application specification. To just get the default records in the custom domain, just include it in the dns-records list:

    network:
      ingress-ip-per-instance:
        protocols:
          - name: tcp
            port-ranges: 80,443
        dns-records:
          domains:
            - domain: default
            - domain: ${SYS_DNS_ZONES[example]}

Delegation and custom domains​

The name server running on each host in the site will consider itself authoritative for all custom domains, and will install SOA and NS records to match it. However, in order for delegation to work the owner of the domain needs to add NS records pointing to at least one of the hosts in the site.

note

In most places the default name server will block private addresses to protect against DNS rebinding attacks, thus if your sites only have private addresses delegation may not work.

Here is an example on how to setup delegation. Consider the following setup (we are using private addresses here to illustrate how the setup works but see the note above).

Custom Domains

There are two sites: a and b, the custom domain is setup under edge.example.com.

In control tower configure the custom domain:

zones:
  - name: example
  - domain: ${SYS_SITE}.edge.example.com

In the name server add:

$ORIGIN example.com
$TTL 1h

ns1-a.edge.example.com. $TTL IN A 10.10.10.23
a.edge.example.com      $TTL IN NS ns1-a.edge.example.com.

ns1-b.edge.example.com. $TTL IN A 192.168.14.40
b.edge.example.com      $TTL IN NS ns1-b.edge.example.com.

The name servers on each site will automatically pick up the "glue" NS records that the authoritative example.com name server has, and add SOA and NS records accordingly:

supctl show --site a dns
zones:
...
  - name: example
    domain: a.edge.example.com
    records:
      - rr: @ 300 IN NS ns1-a.edge.example.com.
      - rr: @ 900 IN SOA ns1-a.edge.example.com. hostmaster.edge.example.com. 2023011637 3600 1800 604800 60