{"metadata":{"image":[],"title":"","description":""},"api":{"url":"","auth":"required","params":[],"results":{"codes":[]}},"next":{"description":"","pages":[]},"title":"Orchestrator Development","type":"basic","slug":"orchestrator-development","excerpt":"","body":"## Using `fig`\n\nEnsure you have fig installed (http://www.fig.sh/install.html). And boot2docker if you're on a mac (http://docs.docker.com/installation/mac/). Install `etcdctl` and `fleetctl` using brew (we won't use the latter now but you'll need it).\n\nCheck if `boot2docker` is running:\n```\n$ boot2docker status\n```\n\nIf it's not up, bring it up:\n```\n$ boot2docker up\n```\n\nIt should stay up until you poweroff. Sometimes you'll lose it after a sleep. If fig gives you errors about \"Connection to 192.168.59.103 timed out\" then you know boot2docker is down.\n\nFor each new terminal session type the following (so fig and Docker know where to find boot2docker):\n```\n$ export DOCKER_HOST=tcp://192.168.59.103:2375\n```\n\nIf you forget this, you'll get the following error message:\n```\n2014/09/02 17:15:06 Get http:///var/run/docker.sock/v1.13/containers/json: dial unix /var/run/docker.sock: no such file or directory\n```\n\nNow to bring up the Orchestrator so you can test against it, grab the repo from Github, `cd` to root dir, and run:\n```\n$ fig up -d\nRecreating orchestrator_etcd_1...\nRecreating orchestrator_svcdir_1...\nRecreating orchestrator_scheduler_1...\nRecreating orchestrator_orchestrator_1...\n```\n\nThe `-d` means \"run in the background\" so you get your cmd-prompt back. As you can see, it has brought up all its dependencies. To check if it's running, and what ports it's bound to:\n```\n$ fig ps\n           Name                 Command      State               Ports\n-----------------------------------------------------------------------------------\norchestrator_etcd_1           /bin/bash      Up      7001->7001/tcp, 4001->4001/tcp\norchestrator_svcdir_1         ./bin/server   Up      9001->9001/tcp\norchestrator_scheduler_1      ./bin/server   Up      9002->9002/tcp\norchestrator_orchestrator_1   ./bin/server   Up      9000->9000/tcp\n```\n\nThis is telling up that these services are \"Up\" and their ports.\n\nHere is a little script to set up some useful environment variables to aid some tasks with curl I'll introduce later:\n```\n$ cat > paz-envvars.sh\n#!/bin/bash\nexport ORCHESTRATOR_URL=192.168.59.103:9000\nexport SCHEDULER_URL=192.168.59.103:9002\nexport SVCDOC='{\"name\":\"demo-api\",\"description\":\"Very simple HTTP Hello World server\",\"dockerRepository\":\"lukebond/demo-api\",\"loadBalanced\":false,\"publicFacing\":false}'\nexport DEPLOY_DOC=\"{\\\"serviceName\\\":\\\"demo-api\\\",\\\"dockerRepository\\\":\\\"lukebond/demo-api\\\",\\\"pushedAt\\\":`date +%s`}\"\n^D\n$ chmod +x paz-envvars.sh\n```\n\nThe above is just creating a text file. If it doesn't work just use vim or whatever you like to create the file.\n\nRun the above script with this special syntax so that it exports the env vars:\n```\n$ . paz-envvars.sh\n```\n\nTo ensure the export worked, try:\n```\n$ echo $ORCHESTRATOR_URL\n172.17.8.101:49164\n```\n\nIf you get any errors, ensure `fig ps` above is working as per the example.\n\nTo add a service via the orchestrator, and test that it worked:\n```\n$ curl -i -XPOST -H \"Content-Type: application/json\" -d \"$SVCDOC\" $ORCHESTRATOR_URL/services\n$ curl -i $ORCHESTRATOR_URL/services\n```\nLook out for non-20X response codes.\n\nTo delete a service:\n```\n$ curl -i -XDELETE $ORCHESTRATOR_URL/services/demo-api\n```\n\nTo patch a service (in this case to set an environment var- demo-api can optionally use a \"MESSAGE\" envvar):\n```\n$ curl -i -XPATCH -H \"Content-Type: application/json\" -d '{\"env\": {\"MESSAGE\": \"so platform; many service; wow\"}}' $ORCHESTRATOR_URL/services/demo-api/config/next\n```\n\nNow to deploy that service via the scheduler:\n```\n$ curl -i -XPOST -H \"Content-Type: application/json\" -d \"$DEPLOY_DOC\" $SCHEDULER_URL/hooks/deploy\n```\n\nRefer to the Orchestrator API docs for all the things you can do but this should get you started.\n\nNow let's try running a service on the platform. We'll use a basic \"hello world\" web server. On your local machine, create a file called `contrived-service-1.json` with the following contents:\n```\n{\n  \"name\": \"contrived-service-1\",\n  \"description\": \"Test service\",\n  \"dockerImage\": \"lukebond/contrived-service-1\",\n  \"ports\": [\n    {\n      \"container\": 9000,\n      \"host\": 80\n    }\n  ],\n  \"autoDeploy\": \"always\",\n  \"numInstances\": 1,\n  \"conflictsWith\": \"contrived-service-*\"\n}\n```\n\nFind out which host and port the service-directory is running on:\n```\n$ etcdctl --peers=172.17.8.101:4001 get /paz/services/paz-service-directory\n172.17.8.103:49153\n```\n\nNow post the above JSON file to the service directory API:\n```\n$ curl -XPOST -H \"Content-Type: application/json\" -d :::at:::contrived-service-1.json 172.17.8.103:49153/services\n{\"meta\":{\"statusCode\":201,\"ok\":true,\"uuid\":\"8n373x\",\"name\":\"contrived-service-1\"}}\n```\n\nNow get the host and port of the scheduler:\n```\n$ etcdctl --peers=172.17.8.101:4001 get /paz/services/paz-scheduler\n172.17.8.103:49154\n```\n\nNow post to the scheduler to trigger the deployment of this service:\n```\n$ curl -XPOST -H \"Content-Type: application/json\" -d '{\"serviceName\": \"contrived-service-1\", \"dockerRepository\": \"lukebond/contrived-service-1:0.0.1\", \"pushedAt\": 0}' 172.17.8.103:49154/hooks/deploy\n{\"statusCode\":200}\n```\n\nWatch the logs of this service and wait until you see that it is up (it's a Harp web server):\n```\n$ fleetctl journal -f contrived-service-1-1.service\n-- Logs begin at Tue 2014-07-15 10:30:37 UTC. --\nJul 15 12:00:15 core-03 systemd[1]: Starting Test service...\nJul 15 12:00:15 core-03 etcdctl[5131]: running\nJul 15 12:00:15 core-03 docker[5130]: Unable to find image 'lukebond/contrived-service-1' locally\nJul 15 12:00:15 core-03 systemd[1]: Started Test service.\nJul 15 12:00:15 core-03 docker[5130]: Pulling repository lukebond/contrived-service-1\nJul 15 12:01:59 core-03 docker[5130]: ------------\nJul 15 12:01:59 core-03 docker[5130]: Harp v0.12.1 – Chloi Inc. 2012–2014\nJul 15 12:01:59 core-03 docker[5130]: Your server is listening at http://localhost:9000/\nJul 15 12:01:59 core-03 docker[5130]: Press Ctl+C to stop the server\nJul 15 12:01:59 core-03 docker[5130]: ------------\n```\n\nHit up port 80 on the box containing this new service to see if it works.\n```\n$ fleetctl list-units\nUNIT                                    MACHINE                   ACTIVE  SUB\npaz-orchestrator-announce.service       7c83517a.../172.17.8.101  active  running\npaz-orchestrator.service                7c83517a.../172.17.8.101  active  running\npaz-scheduler-announce.service          83fe3a48.../172.17.8.102  active  running\npaz-scheduler.service                   83fe3a48.../172.17.8.102  active  running\npaz-service-directory-announce.service  5c3e57b1.../172.17.8.103  active  running\npaz-service-directory.service           5c3e57b1.../172.17.8.103  active  running\npaz-web-announce.service                5c3e57b1.../172.17.8.103  active  running\npaz-web.service                         5c3e57b1.../172.17.8.103  active  running\ncontrived-service-1-1.service           5c3e57b1.../172.17.8.103  active  running\n```\n\nAs you can see here, it's on 172.17.8.103, so paste that into your browser and you should see \"Hello World 1\".","updates":[],"order":5,"isReference":false,"hidden":false,"sync_unique":"","link_url":"","link_external":false,"_id":"54fadd895c41173700ecf2a2","category":{"sync":{"isSync":false,"url":""},"pages":["54fada07961fea21009206fb","54fada3d5c41173700ecf29c","54fadd49f63cf9210041c66a","54fadd895c41173700ecf2a2","54faddd35c41173700ecf2a4","54fadea0f63cf9210041c66d"],"title":"Contributing","slug":"contributing","order":4,"from_sync":false,"reference":false,"_id":"54fad9d5f63cf9210041c669","project":"54fa004ff63cf9210041c3be","__v":6,"createdAt":"2015-03-07T10:58:29.385Z","version":"54fa0050f63cf9210041c3c1"},"user":"54fa0018e7a0ba2f00306211","version":{"version":"1.0","version_clean":"1.0.0","codename":"","is_stable":true,"is_beta":false,"is_hidden":false,"is_deprecated":false,"categories":["54fa0050f63cf9210041c3c2","54fa42b3961fea210092068e","54facefd5c41173700ecf296","54fad9d5f63cf9210041c669","54fae9ed5c41173700ecf2ad"],"_id":"54fa0050f63cf9210041c3c1","__v":5,"createdAt":"2015-03-06T19:30:24.258Z","project":"54fa004ff63cf9210041c3be","releaseDate":"2015-03-06T19:30:24.258Z"},"__v":0,"createdAt":"2015-03-07T11:14:17.991Z","project":"54fa004ff63cf9210041c3be","githubsync":""}

Getting Started

Integrations

APIs

Videos

Contributing

Orchestrator Development

## Using `fig` Ensure you have fig installed (http://www.fig.sh/install.html). And boot2docker if you're on a mac (http://docs.docker.com/installation/mac/). Install `etcdctl` and `fleetctl` using brew (we won't use the latter now but you'll need it). Check if `boot2docker` is running: ``` $ boot2docker status ``` If it's not up, bring it up: ``` $ boot2docker up ``` It should stay up until you poweroff. Sometimes you'll lose it after a sleep. If fig gives you errors about "Connection to 192.168.59.103 timed out" then you know boot2docker is down. For each new terminal session type the following (so fig and Docker know where to find boot2docker): ``` $ export DOCKER_HOST=tcp://192.168.59.103:2375 ``` If you forget this, you'll get the following error message: ``` 2014/09/02 17:15:06 Get http:///var/run/docker.sock/v1.13/containers/json: dial unix /var/run/docker.sock: no such file or directory ``` Now to bring up the Orchestrator so you can test against it, grab the repo from Github, `cd` to root dir, and run: ``` $ fig up -d Recreating orchestrator_etcd_1... Recreating orchestrator_svcdir_1... Recreating orchestrator_scheduler_1... Recreating orchestrator_orchestrator_1... ``` The `-d` means "run in the background" so you get your cmd-prompt back. As you can see, it has brought up all its dependencies. To check if it's running, and what ports it's bound to: ``` $ fig ps Name Command State Ports ----------------------------------------------------------------------------------- orchestrator_etcd_1 /bin/bash Up 7001->7001/tcp, 4001->4001/tcp orchestrator_svcdir_1 ./bin/server Up 9001->9001/tcp orchestrator_scheduler_1 ./bin/server Up 9002->9002/tcp orchestrator_orchestrator_1 ./bin/server Up 9000->9000/tcp ``` This is telling up that these services are "Up" and their ports. Here is a little script to set up some useful environment variables to aid some tasks with curl I'll introduce later: ``` $ cat > paz-envvars.sh #!/bin/bash export ORCHESTRATOR_URL=192.168.59.103:9000 export SCHEDULER_URL=192.168.59.103:9002 export SVCDOC='{"name":"demo-api","description":"Very simple HTTP Hello World server","dockerRepository":"lukebond/demo-api","loadBalanced":false,"publicFacing":false}' export DEPLOY_DOC="{\"serviceName\":\"demo-api\",\"dockerRepository\":\"lukebond/demo-api\",\"pushedAt\":`date +%s`}" ^D $ chmod +x paz-envvars.sh ``` The above is just creating a text file. If it doesn't work just use vim or whatever you like to create the file. Run the above script with this special syntax so that it exports the env vars: ``` $ . paz-envvars.sh ``` To ensure the export worked, try: ``` $ echo $ORCHESTRATOR_URL 172.17.8.101:49164 ``` If you get any errors, ensure `fig ps` above is working as per the example. To add a service via the orchestrator, and test that it worked: ``` $ curl -i -XPOST -H "Content-Type: application/json" -d "$SVCDOC" $ORCHESTRATOR_URL/services $ curl -i $ORCHESTRATOR_URL/services ``` Look out for non-20X response codes. To delete a service: ``` $ curl -i -XDELETE $ORCHESTRATOR_URL/services/demo-api ``` To patch a service (in this case to set an environment var- demo-api can optionally use a "MESSAGE" envvar): ``` $ curl -i -XPATCH -H "Content-Type: application/json" -d '{"env": {"MESSAGE": "so platform; many service; wow"}}' $ORCHESTRATOR_URL/services/demo-api/config/next ``` Now to deploy that service via the scheduler: ``` $ curl -i -XPOST -H "Content-Type: application/json" -d "$DEPLOY_DOC" $SCHEDULER_URL/hooks/deploy ``` Refer to the Orchestrator API docs for all the things you can do but this should get you started. Now let's try running a service on the platform. We'll use a basic "hello world" web server. On your local machine, create a file called `contrived-service-1.json` with the following contents: ``` { "name": "contrived-service-1", "description": "Test service", "dockerImage": "lukebond/contrived-service-1", "ports": [ { "container": 9000, "host": 80 } ], "autoDeploy": "always", "numInstances": 1, "conflictsWith": "contrived-service-*" } ``` Find out which host and port the service-directory is running on: ``` $ etcdctl --peers=172.17.8.101:4001 get /paz/services/paz-service-directory 172.17.8.103:49153 ``` Now post the above JSON file to the service directory API: ``` $ curl -XPOST -H "Content-Type: application/json" -d @contrived-service-1.json 172.17.8.103:49153/services {"meta":{"statusCode":201,"ok":true,"uuid":"8n373x","name":"contrived-service-1"}} ``` Now get the host and port of the scheduler: ``` $ etcdctl --peers=172.17.8.101:4001 get /paz/services/paz-scheduler 172.17.8.103:49154 ``` Now post to the scheduler to trigger the deployment of this service: ``` $ curl -XPOST -H "Content-Type: application/json" -d '{"serviceName": "contrived-service-1", "dockerRepository": "lukebond/contrived-service-1:0.0.1", "pushedAt": 0}' 172.17.8.103:49154/hooks/deploy {"statusCode":200} ``` Watch the logs of this service and wait until you see that it is up (it's a Harp web server): ``` $ fleetctl journal -f contrived-service-1-1.service -- Logs begin at Tue 2014-07-15 10:30:37 UTC. -- Jul 15 12:00:15 core-03 systemd[1]: Starting Test service... Jul 15 12:00:15 core-03 etcdctl[5131]: running Jul 15 12:00:15 core-03 docker[5130]: Unable to find image 'lukebond/contrived-service-1' locally Jul 15 12:00:15 core-03 systemd[1]: Started Test service. Jul 15 12:00:15 core-03 docker[5130]: Pulling repository lukebond/contrived-service-1 Jul 15 12:01:59 core-03 docker[5130]: ------------ Jul 15 12:01:59 core-03 docker[5130]: Harp v0.12.1 – Chloi Inc. 2012–2014 Jul 15 12:01:59 core-03 docker[5130]: Your server is listening at http://localhost:9000/ Jul 15 12:01:59 core-03 docker[5130]: Press Ctl+C to stop the server Jul 15 12:01:59 core-03 docker[5130]: ------------ ``` Hit up port 80 on the box containing this new service to see if it works. ``` $ fleetctl list-units UNIT MACHINE ACTIVE SUB paz-orchestrator-announce.service 7c83517a.../172.17.8.101 active running paz-orchestrator.service 7c83517a.../172.17.8.101 active running paz-scheduler-announce.service 83fe3a48.../172.17.8.102 active running paz-scheduler.service 83fe3a48.../172.17.8.102 active running paz-service-directory-announce.service 5c3e57b1.../172.17.8.103 active running paz-service-directory.service 5c3e57b1.../172.17.8.103 active running paz-web-announce.service 5c3e57b1.../172.17.8.103 active running paz-web.service 5c3e57b1.../172.17.8.103 active running contrived-service-1-1.service 5c3e57b1.../172.17.8.103 active running ``` As you can see here, it's on 172.17.8.103, so paste that into your browser and you should see "Hello World 1".