Maps

Beam includes a concurrency-safe distributed map, accessible both locally and within remote containers. Serialization is done using cloudpickle, so any pickleable object will work. The interface is that of a standard python dictionary, but unlike a typical dicitonary it will persist between runs.

​

Example: Accessing a map locally and remotely

In the following example, we create a distributed map. Our first function is invoked remotely using .remote(), and it sets the value a key in our map. The second function is invoked locally using .local(), and it sets another value. Finally, we print the result of our third, remotely invoked function, which retrieves the values we just set.

from beam import Map, function


@function()
def first():
    m = Map(name="m")
    m["beam"] = "me up"
    return


@function()
def second():
    m = Map(name="m")
    m["speed"] = "of light"
    return


@function()
def third():
    m = Map(name="m")
    return [m["beam"], m["speed"]]


if __name__ == '__main__':
    first.remote()
    second.local()
    print(third.remote())   

You can run the example above with python app.py. The output will be:

['me up', 'of light']