1. Cloud (AWS / Azure / Google)

Install Apache Spark standalone mode on AWS Ubuntu EC2 (step by step guide)

This is the step to install standalone spark in AWS Ubuntu Ec2. Working and tested steps

Install Java
————

  • sudo apt-add-repository ppa:webupd8team/java
  • sudo apt-get update
  • sudo apt-get install oracle-java7-installer

Install Scala (www.scala-lang.org)
————-

  • sudo mkdir /usr/local/src/scala
  • sudo tar -xvf scala-2.11.7.tgz -C /usr/local/src/scala/
  • sudo vim .bashrc
  • export SCALA_HOME=/usr/local/src/scala/scala-2.11.7
  • export PATH=$SCALA_HOME/bin:$PATH
  • . .bashrc

Install Git
———–

  • sudo apt-get install git

Build Spark
———–

  • sudo tar -xvf spark-2.0.0.tgz
  • cd spark-2.0.0
  • sudo build/sbt assembly
  • sudo build/sbt package
Comments to: Install Apache Spark standalone mode on AWS Ubuntu EC2 (step by step guide)

Your email address will not be published. Required fields are marked *

Nishant Kumar Verma

Full Stack Developer, Experience in enterprise application development. Providing solution for scalable SaaS application and developing product to automate part of development efforts.

#CleanCodeArchitect #.Net #C# #NodeJS #TypeScript #React #JavaScript #DesignPattern #SOLID #WebAPI #HTML5 #Bootstrap #CSS

#AWS #Docker #Kubernetes #GIT #GitHubAction #ECS #LoadBalancer #S3 #Lambda #Azure #CI/CD Pipeline

#MongoDB #MSSQL #MySQL #JSON