YAO-docs/setup.md

Steps to bring up the YAO components

Install docker

curl -fsSL https://get.docker.com | sh

Install nvidia driver

Install Nvidia-docker

Read NVIDIA/nvidia-docker for guidance.

Set default runtime to nvidia, see Default runtime.

Init a docker swarm cluster

# on master node
docker swarm init

# Add other nodes to the cluster
docker swarm join --token A-LONG-TOKEN-STRING-HERE 192.168.0.1:2377

Create an overlay network named yao

docker network create --driver overlay --attachable --opt encrypted yao-net

Start zk cluster

sbin/start_zks.sh

Start HDFS cluster

sbin/run_hdfs.sh

Start Kafka cluster

sbin/start_kafka.sh

Create kafka topic yao

# delete old
bin/kafka-topics.sh \
	--delete \
	--zookeeper zookeeper-node1:2181,zookeeper-node2:2181,zookeeper-node3:2181 \
	--topic yao

# create new
bin/kafka-topics.sh \
	--create \
	--zookeeper zookeeper-node1:2181,zookeeper-node2:2181,zookeeper-node3:2181 \
	--replication-factor 3 \
	--partitions 1 \
	--topic yao

Start the agents in each YAO-Worker

sbin/run_agent_helper.sh

sbin/run_agent.sh

Start the agent-master on YAO-Master

sbin/start_agent_master.sh

Start mysql

sbin/start_mysql.sh

Start yao-optimizer on Master Node

sbin/run_optimizer.sh

Start yao-scheduler

sbin/start_scheduler.sh

Start Redis

sbin/start_redis.sh

Start the web portal

sbin/start_portal.sh

Install

Visit http://YOUR_IP/install.php