Queues

Beam includes a concurrency-safe distributed queue, accessible both locally and within remote containers. Serialization is done using cloudpickle, so any object that supported by that should work here. The interface is that of a standard python queue. Because this is backed by a distributed queue, it will persist between runs. In the example below, we run one function remotely on Beam and another locally. The remote function puts a value in the queue, and the local function pops it out and prints it. The output will be beam me up.

Simple Queue

from beam import Queue, function


@function()
def first():
    q = Queue(name="q")
    q.put("beam me up")
    return


@function()
def second():
    q = Queue(name="q")
    print(q.pop())
    return


if __name__ == '__main__':
    first.remote()
    second.local()

Start Here

Customizing Container Images

Managing Data

Scaling & Performance

Sandboxes

Endpoints and Web Servers

Task Queues

One-Off Functions

Hosting Containers

Advanced Topics

Self-Hosting

Resources

Security