diff --git a/spark/test/Dockerfile b/spark/test/Dockerfile index e4f122d..2ce2a71 100644 --- a/spark/test/Dockerfile +++ b/spark/test/Dockerfile @@ -13,11 +13,11 @@ ENV PATH $PATH:$JAVA_HOME/bin ENV SPARK_VER 2.2.1 -RUN wget https://archive.apache.org/dist/spark/spark-$SPARK_ENV/spark-$SPARK_ENV-bin-without-hadoop.tgz && \ - tar -xvf spark-$SPARK_ENV-bin-without-hadoop.tgz -C /usr/local && \ - rm spark-$SPARK_ENV-bin-without-hadoop.tgz +RUN wget https://archive.apache.org/dist/spark/spark-$SPARK_VER/spark-$SPARK_VER-bin-without-hadoop.tgz && \ + tar -xvf spark-$SPARK_VER-bin-without-hadoop.tgz -C /usr/local && \ + rm spark-$SPARK_VER-bin-without-hadoop.tgz -RUN ln -s /usr/local/spark-$SPARK_ENV-bin-without-hadoop /usr/local/spark +RUN ln -s /usr/local/spark-$SPARK_VER-bin-without-hadoop /usr/local/spark ENV SPARK_HOME /usr/local/spark diff --git a/spark/test/README.md b/spark/test/README.md index d8c6eeb..56c9a9a 100644 --- a/spark/test/README.md +++ b/spark/test/README.md @@ -2,8 +2,6 @@ ## Create a spark cluster in swarm mode -`--hostname` needs 1.13 or higher - ```bash docker service create \ --name spark-master \