Crate adds Unicast Host Discovery via DNS and the AWS EC2 API

Chris Ward
Filed under
July 28, 2015

A well functioning Crate cluster relies on its nodes being able to utilize service discovery to become aware of each other.

Until now the options available have been multicast, and when this is unavailable, specifying unicast hosts.

To help with Unicast host discovery we are pleased to announce support for discovery via DNS and the AWS EC2 API.

DNS Discovery

In your crate.yml configuration file set discovery.type to srv, this setting cannot be set at runtime.

Configure the DNS query that will be used to look up SRV records with the discovery.srv.query option, this will be typically be in the format _service._protocol.fqdn and it needs to be configured for the service discovery to function.

discovery.srv.resolver sets the hostname or IP address of the DNS server used to resolve the DNS records, if nothing is set or is unresolvable, then the default system resolver is used.

Here is an example section of a crate.yml file we use for our own test cluster:

  type: srv

All being well, the resolver and query will return a series of discovery nodes ordered by the SRV record weights and priorities, for example a query that returns: 3600 IN SRV 2 20 4300 3600 IN SRV 1 10 4300 3600 IN SRV 2 10 4300

will result in the following series of discovery nodes:,,


Cloud hosting such as Amazon's EC2 typically disable multicast and thus forces Crate to use Unicast host discovery. Whilst this is not difficult, it lacks flexibility for dynamic infastructures.

Our new EC2 discovery feature uses the EC2 API to look up other EC2 hosts that are then used as unicast hosts for node discovery. This allows for flexibility and the usage of other AWS features, such as filtering by tags or location.

For instructions on how to use the EC2 Discovery API read our best practice guide.

Need Help?

That's it! Read more about all our discovery options here and if you have any problems or questions then please get in touch and we will be happy to help.

Back to topAll posts