Create a cluster in the cloud

Welcome to the documentation for cluster in the cloud. By the end of this you will have a fully-operational, elastically-scaling, heterogeneous Slurm cluster running on cloud resources.

In the future, the intention is that this tutorial will cover installing on all major cloud providers but for now AWS, Google Cloud Platform and Oracle Cloud Infrastructure are covered.

This tutorial and the Cluster in the Cloud software was created by Matt Williams at the ACRC in Bristol. Contributions to this tutorial document are welcome at GitHub.

If you need help

If you have any questions or issues with the system as a whole, please raise them as a GitHub issue at clusterinthecloud/support.

Prerequisites

To complete this tutorial you will need:

  • access to a command-line (i.e. Linux, MacOS Terminal or WSL)
  • an SSH key pair
  • an account with credit on AWS, Google or Oracle cloud
    • the account must have admin permissions to create infrastructure
  • local software installed
    • Terraform 0.12
    • SSH
    • Git

Alternatively, for some cloud providers (only Google so far) we have a simpler “one-click” installer which has no up-front requirements at all. That is covered in the Google chapter.

Start by creating the infrastructure.