What is gRPC load balancing?

Many web applications today operate at scale without any noticeable drop in performance. For example, a popular social networking website may support millions of users connected simultaneously, while everything else works perfectly. None of this can be achieved without load balancing.

However, load balancing doesn't only apply to users trying to access an in demand website through their browsers. Neither does it only apply to HTTP requests. Load balancing can be applied to any service accessible by any communication protocols over the network, including gRPC.

Load balancing basics

Any load-balancing setup follows the same basic principles, regardless of the technologies, programming languages, or communication protocols. These are as follows:

• The client, which is the entity that initiates the communication, sends the request to the server.

• The request is received by the load balancer.

• The load balancer routes the request to one of many server instances, which hosts the actual app that the client tried to communicate with.

This can be summarized by the following diagram:

In a setup with a load balancer, there are multiple replicas of the server-side application that clients want to access. This is because one instance would be insufficient to handle a large volume of incoming requests.

The role of the load balancer is to route an incoming request to a specific instance of the application. This way, many client connections are distributed between many different nodes, so none of the nodes get overwhelmed.

Balancer vs. resolver

There are many ways of implementing load balancers, but all the implementations apply the concepts of balancers and resolvers. They can be summarized as follows:

• Balancer: The algorithm that selects the next node to connect to.

• Resolver: The algorithm that resolves addresses of the specific nodes.

For example, the resolver may pull a list of IP addresses from a DNS endpoint. The balancer then decides which address to connect to. It may do so randomly or maybe by attempting to connect to every address in turn in a round-robinRound-robin communication is when each new request is routed to a new instance of the server. The servers are always contacted in a specific order. Once the final server has been contacted, the next request is routed to the first server and the process starts over. fashion.

Server-side load balancing in gRPC

Commonly, load balancing is done on the server. The client isn't aware of how many instances the application consists of. It can be one or it can be many. The client connects to a specific address. This would typically be the address of a proxy or another type of gateway. The gateway then uses the load balancer to connect the client to a specific application instance.

This is what we experience when we visit a high-demand website such as Facebook or YouTube. We enter a specific URL into the browser and the load balancer handles everything else for us. To us, websites that use load balancers work exactly the same as the ones that consist of a single application instance.

The same applies to gRPC. Because gRPC is a wrapper protocol that uses HTTP/2, gRPC calls can be redirected the same way as any other HTTP requests. In this case, we don't need to do anything in the client app.

Client-side load balancing in gRPC

With gRPC in particular, it's possible to skip server-side load balancing and do it directly from the client. This may give us an advantage in terms of performance, because there are fewer connections to make. The client itself will connect to a specific address directly without using any middleware services, such as proxies, for every connection that it establishes.

Different language-specific implementations of gRPC have different support for client-side load balancing, but here are some options available:

• DNS resolver: The client connects to a specific DNS endpoint and retrieves a list of addresses it can connect to.

• Static resolver: A list of addresses is already pre-configured inside the client application.

In addition to this, developers can apply their own custom logic on the client for both the resolver and balancer algorithms. Certain platforms, such as .NET, where gRPC is a first-class citizen, have components that make this process relatively easy.

Code example

Below is an example of how client-side load balancing can be applied in .NET. In this code example, we have two projects:

• WebApiGrpcClient: This is a fully interactive project that represents a web application. The application is accessible via a browser, and it uses a gRPC client with a static load balancer.

• BasicGrpcService: This is a read-only project that represents the gRPC service instances we will connect to. It is read-only because we have three separate instances of it running in the background. The project itself has been provided solely so we can check its internal code.

Our gRPC client application, which is represented by the WebApiGrpcClient project, uses the standard .NET project setup. On lines 1–5, in the Program.cs file, we reference all the namespaces that we need in the code. On lines 7–11, we register the standard dependencies that we need for the standard RESTREST stands for Representational State Transfer. It is a protocol that allows the clients to exchange data with a hosted application via HTTP. API functionality, so we can access the app via the browser.

On lines 13–18, in the Program.cs file of the WebApiGrpcClient, we register a static resolver with three addresses that represent the addresses of the three instances of the gRPC service we're running in the background. On line 24, we register the address for the client to connect to. Because we're using static:/// as the prefix, the client will expect a static load balancer. The remainder of the address is completely arbitrary.

On line 32, we specify the balancer algorithm. We use an instance of the RoundRobinConfig class, which represents the round-robin algorithm. Every new call will be made to a new address from the list. The addresses will be iterated through in a particular order.

Finally, on lines 42–53, we create a request-processing pipeline for our application, adding Swagger dependencies to the pipeline and launching our application in the background so it can be accessed by HTTP requests.

Testing the application

We can launch the application by clicking the "Run" button to see how the load balancing works. This will build and launch the application. Once the application is built, a Swagger web page will be available in the "Output" tab. Moreover, if we want to open this page in a full browser tab, we can click on the link with the heading "Your app can be found at."

Once we open the Swagger page, it will represent our gRPC client endpoint as a dropdown. The following screenshot demonstrates what it should look like:

To send a request against the endpoint, we would need to click on the dropdown. When it expands, we should see a section with parameters that looks as follows:

If we click the "Try it out" button, we'll be able to populate the request parameters and make the request by clicking the "Execute" button that will appear. For example, we can type our name as the name parameter and type Hello as the message parameter.

We can then switch back to the "Terminal" tab in the original code playground and see which specific address the request went to. If we do it multiple times, we should expect to see different ports being used in the URL, which represent different addresses from our static resolver.

Note: Due to the limitation of the host environment, the round-robin resolver may not iterate through the addresses in a specific order.