Hosting a Web Server
Deploying web servers on Beam
With Beam, you can deploy web servers that use the ASGI protocol. This means that you can deploy applications built with popular frameworks like FastAPI and Django.
Multiple Endpoints Per App
In the example below, we are deploying a FastAPI web server that uses the Huggingface Transformers library to perform sentiment analysis and text generation.
We also included a warmup endpoint so that we can preemptively get our container ready for incoming requests.
This example uses Pydantic to serialize request inputs. You can read more about it here.
As shown above, the handler function for the web server must return the ASGI app object.
Launch a Preview Environment (Optional)
Just like an endpoint, you can prototype your web server using beam serve
. This command will monitor changes in your local file system, live-reload the remote environment as you work, and forward remote container logs to your local shell.
Serve sessions end automatically after 10 minutes of inactivity. The entire duration of the session is counted towards billable usage, even if the session is not receiving requests.
Deploying the Web Server
When you are ready to deploy your web server, run the following command:
You’ll see some logs in the console that show the progress of your deployment.
The container handling the app will spin down after 180 seconds of inactivity
by default, or customized with the keep_warm_seconds
parameter. The
container will be billed for the time it is active and handling requests.
Sending Requests
If we wanted to perform sentiment analysis using our deployed example from above, we would send a POST request like this:
Concurrent Requests
When building an ASGI app, you can specify the number of concurrent requests your app can handle using the concurrent_requests
parameter in the @asgi
decorator.
This allows you to increase the number of requests your app can handle at once, which can help you achieve higher throughput. For instance, if your app is doing I/O-bound work, additional requests can be handled while your I/O operations complete in the background.
We can simulate this by adding a model
endpoint that pretends to do some expensive I/O to our example from above.
Now, if you send a request to model
and then send another request to generate
, you will see that the second request will complete before the first.
Response Types
Beam supports various response types, including any FastAPI response type. You can find a list of FastAPI response types here.
Uploading Local Files
If your web server needs access to local files like model weights or other resources, you can use Beam volumes.
To add files to a volume, you can use the beam cp
command.
Then, you can define a volume and pass it into your @asgi
decorator like this: