Tower, Episode 1: your Service as a Function

, ,

When I first came across the Tower library, which is part of the Tokio stack, I noticed its title line:

1
async fn(Request) -> Result<Response, Error>

In the documentation this is explained as follows:

Tower provides a simple core abstraction, the Service trait, which represents an asynchronous function taking a request and returning either a response or an error. This abstraction can be used to model both clients and servers.

This immediately reminded the seasoned Scala developer in me of the Finagle paper titled “Your Server as a Function”, which defines that “Systems boundaries are represented by asynchronous functions called services”. It is always good to see when new projects make use of already existing knowledge.

While both Tower and Finagle services share the same core abstraction – the asynchronous function – there are two differences.

First, Tower does not necessarily interpret services as system boundaries. Instead services can also be composed “locally” which rather matches the Finagle “filters”:

Generic components, like timeouts, rate limiting, and load balancing, can be modeled as Services that wrap some inner service and apply additional behavior before or after the inner service is called. This allows implementing these components in a protocol-agnostic, composable way. Typically, such services are referred to as middleware.

Second, Tower enriches the core abstraction of an asynchronous function with the concept of readiness, as we can see form the definition of the Service trait:

1
2
3
4
5
6
7
8
9
10
11
pub trait Service<Request> {
    type Response;

    type Error;

    type Future: Future<Output = Result<Self::Response, Self::Error>>;

    fn poll_ready(&mut self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>>;

    fn call(&mut self, req: Request) -> Self::Future;
}

At its core, the contract for callers of these service methods is as follows:

  • First poll_ready must be invoked – potentially several times – until it returns Poll::Ready, then call can be invoked.
  • call might panic if the above is not the case.

Before going into the details of calling services in the next episode, let’s first take a look at a simple example for a service:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
pub struct EchoRequest(String);

pub struct EchoResponse(String);

pub struct EchoService;

/// Tower `Service` implementation for [EchoService]: always ready, never fails and responds
/// immediately with an echo, i.e. a response with the same content like the request.
impl Service<EchoRequest> for EchoService {
    type Response = EchoResponse;

    /// This service never fails.
    type Error = Infallible;

    /// This service responds immediately.
    type Future = Ready<Result<Self::Response, Self::Error>>;

    /// Always return `Poll::Ready`: this service is always ready.
    fn poll_ready(&mut self, _cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
        Poll::Ready(Ok(()))
    }

    /// Always return an echo, i.e. a response with the same content like the request.
    fn call(&mut self, req: EchoRequest) -> Self::Future {
        ready(Ok(EchoResponse(req.0)))
    }
}

As this service is always ready, it is not necessary for call to panic if poll_ready has not been called before.

The full code is available in tower-experiments on GitHub. In the next episode we will take a look at service clients.