Create a cluster in the cloud

Welcome to the documentation for Cluster in the Cloud. By the end of this you will have a fully-operational, elastically-scaling, heterogeneous Slurm cluster running on cloud resources.

In the future, the intention is that this tutorial will cover installing on all major cloud providers but for now AWS, Google Cloud Platform and Oracle Cloud Infrastructure are covered.

This tutorial and the Cluster in the Cloud software was created by Matt Williams at the ACRC in Bristol. Contributions to this tutorial document are welcome at GitHub.

If you need help

If you have any questions or issues with the system as a whole, please raise them as a GitHub issue at clusterinthecloud/support.

Note

If you use Cluster in the Cloud for any published research, please mention us in your acknowledgements.

Prerequisites

To complete this tutorial you will need:

  • access to a command-line (i.e. Linux, MacOS Terminal or WSL)
  • an SSH key pair
  • an account with credit on AWS, Google or Oracle cloud
    • the account must have admin permissions to create infrastructure
  • local software installed
    • Terraform 1.0 or newer
    • SSH
    • Git

Alternatively, for some cloud providers (only AWS and Google so far) we have a simpler “one-click” installer which has fewer or no up-front requirements at all. That is covered in the AWS and Google chapters.

Start the tutorial here by creating the infrastructure.

More info

Once you’ve got your cluster up and running, see these topics for more information:

Concepts