GPU Acceleration

​

Running Tasks on GPU

You can run any code on a cloud GPU by passing a gpu argument in your function decorator.

from beam import endpoint


@endpoint(gpu="A100-40")
def handler():
    # Prints the available GPU drivers
    import subprocess
    print(subprocess.check_output(["nvidia-smi"], shell=True))

    return {"gpu":"true"}

​

Available GPUs

Currently available GPU options are:

• T4 (16Gi)
• A10G (24Gi)
• A100-40 (40Gi)
• RTX4090 (24Gi)

​

Check GPU Availability

Run beam machine list to check whether a machine is available.

$ beam machine list

  GPU Type   Available
 ──────────────────────
  A100-40       ✅
  A10G          ✅
  RTX4090       ✅

​

Prioritizing GPU Types

You can split traffic across multiple GPUs by passing a list to the gpu parameter.

The list is ordered by priority. You can choose which GPUs to prioritize by specifying them at the front of the list.

gpu=["T4", "A10G", "A100-40"]

In this example, the T4 is prioritized over the A10G, followed by the A100-40.

​

Using Multiple GPUs

You can run workloads across multiple GPUs by using the gpu_count parameter.

from beam import endpoint


@endpoint(gpu="A10G", gpu_count=2) 
def handler():
    return {"hello": "world"}

​

GPU Regions

Beam runs on servers distributed around the world, with primary locations in the United States, Europe, and Asia. If you would like your workloads to run in a specific region of the globe, please reach out.