Apache Mesos Framework

Apache Mesos (Mesos) abstracts CPU, memory, storage, and other compute resources away from machines (physical or virtual), enabling fault-tolerant and elastic distributed systems to easily be built and run effectively.

Mesos is a perfect fit for a scalable database like Crate, and our framework allows you to launch and manage the nodes in your cluster as one abstract machine.

Quick Guide

The easiest way to run the Crate Mesos Framework is by scheduling it using Marathon, e.g. on a DCOS cluster.

The current version of the framework is 0.2.0, here's an example of a Marathon.json file:

{
    "id": "crate-framework",
    "instances": 1,
    "cpus": 0.25,
    "mem": 128,
    "portDefinitions": [
        {
            "port": 4040,
            "protocol": "tcp",
            "name": "api"
        }
    ],
    "requirePorts": true,
    "env": {
        "CRATE_CLUSTER_NAME": "my-crate-cluster",
        "CRATE_VERSION": "0.54.9",
        "CRATE_HTTP_PORT": "4200",
        "CRATE_TRANSPORT_PORT": "4300",
        "ZOOKEEPER": "master.mesos:2181"
    },
    "fetch": [
        {
            "uri": "https://cdn.crate.io/downloads/releases/crate-mesos-0.2.0.tar.gz",
            "extract": true,
            "executable": false,
            "cache": false
        },
        {
            "uri": "https://cdn.crate.io/downloads/openjdk/jre-7u80-linux.tar.gz",
            "extract": true,
            "executable": false,
            "cache": false
        }
    ],
    "cmd": "env && $(pwd)/jre/bin/java $JAVA_OPTS -jar $(pwd)/crate-mesos-0.2.0.jar --crate-cluster-name $CRATE_CLUSTER_NAME --crate-version $CRATE_VERSION --api-port $PORT0 --crate-http-port $CRATE_HTTP_PORT --crate-transport-port $CRATE_TRANSPORT_PORT --zookeeper $ZOOKEEPER",
    "healthChecks": [
        {
            "protocol": "HTTP",
            "path": "/cluster",
            "gracePeriodSeconds": 3,
            "intervalSeconds": 10,
            "portIndex": 0,
            "timeoutSeconds": 10,
            "maxConsecutiveFailures": 3
        }
    ]
}

You can use curl to submit the file to Marathon:

curl -sXPOST "http://master.mesos:8080/v2/apps" -d@crate.json -H "Content-Type: application/json" | jq .

Once deployed you can use the framework API to launch a Crate cluster. You will need to find out on which host the framework is scheduled and execute the resize command:

curl -sXPOST -H "Content-Type: application/json" $FRAMEWORK_HOST:4040/cluster/resize -d '{"instances": $NUM_INSTANCE}'

More Details

Full documentation for this project can be found here.