One application or many services?

Software can be built as one application or as several small ones. This lesson compares the memory each design needs and what splitting costs.

You open a food delivery app at dinner time. The restaurant list loads at once. You press Place order and the screen waits, then shows an error. You go back, and the restaurant list still works.

One program does every jobMonolith
Several programs, one job eachMicroservices
One service asks anotherNetwork call
Think first. If the app runs as 24 workers that can each do every task, and the heaviest task needs 400 MB, how much memory do the workers need in total?
The answer is 24 times 400 MB, which is 9,600 MB or about 9.4 GB. Most of that memory sits unused, because few workers ever do the heavy task. The next step shows the same effect.
More detail

The identical machines in Load balancers and scaling were a monolith running as several copies. Most systems should be a monolith. Case study: enlistment showed one candidate for splitting: browsing the catalog and reserving a slot need opposite things from the same machine.

Calls between services cross the network, and each call has a cost.

Building a receipt PDF needs 400 MB per worker, and the other three parts need 120 MB or less. Drag the load down to 40 rps and see which column uses less memory. Then drag it above 800.

Monolith

Every copy does everything

Every worker needs memory for the heaviest task.

—total RAM: —
Services

Each part gets what it needs

Only the PDF service needs 400 MB workers.

—total RAM: —
Choose the load
More detail

The four parts of this application

These profiles are illustrative. Handler time is how long the code takes to process one request.

How the columns are sized

Both columns use 4 workers per machine, as in Server capacity. The workers differ in what each one must be able to do.

In the monolith, any machine can serve any request, so the load balancer can pick any of them. The application is simple to deploy, and a failure produces one stack trace, which is the error report that shows where in the code the failure happened. Every worker needs enough memory for the heaviest task it might be given.

The restaurant list service never loads the PDF library, so it never needs 400 MB. You can add copies of the slow part without paying for extra copies of the fast parts.

Splitting can save memory at high load. It adds four costs from the first day.

Calls cross the network

A call can be slow, time out, or fail after the work is done.

Services change at different times

Services deploy separately, so a changed message must add fields and remove none.

Every call needs retry rules

Each call needs a timeout, retries and an idempotency key, so retries are safe.

One failure, several logs

Follow a failure across services with a shared trace ID on every request.

More detail

Where you have met these costs

Calls cross the network: see Round trips and latency. The same delays now happen inside your own system.

Services get out of step: see One API, many clients. Version drift now happens between your own services, because they deploy at different times.

Every call needs retry rules: see Payments and gateways. The same rules now apply to calls you wrote yourself.

One failure, several logs: in HTTP messages, you followed one request at a time. Now one request leaves entries in the logs of several services.

Showing one order screen can need the restaurant list, the customer account and the payment status. Across services, each of these is a network call. Drag the slider to 10 calls and read the bottom two figures.

One request that needs several services
Handler time
—
Chance of a slow call
—
Combined uptime
—
Unavailable per month
—
Choose how many calls one request makes
More detail

The assumptions behind the figures

Each call to another service takes about 3 ms longer than the same work done inside one program. The extra time covers opening a connection, converting data to a message and back (serialization), and the network. That figure is for a good day inside one data center.

Each service is up 99.9% of the time, which is the share of time it is working, and 1% of its calls are slow. These are good numbers for one service. The widget in this lesson shows what they become when one request needs several services.

A similar problem one layer down

The problem in The N+1 query problem has the same pattern: many small requests where one would do. A database option such as select_related cannot fix it here.

Build the monolith, with its parts in separate modules and no code that reads another module's tables. Then a boundary exists if you ever need to split. Split a part only when its needs differ and you can say why. If two services call each other on every request, treat them as one service.

You can now decide whether a part of a project should be its own service.

More detail

Distributed monolith

Splitting an application whose parts still need each other on every request produces a distributed monolith. It has all the costs of separation and all the dependence between parts, so it is slower, fails more often and is harder to debug than the single application you started with.

Split only where the parts do not need each other. That depends on your business domain.

Why most small projects stay as one application

At low load, four services use more memory than one monolith, and most small projects run at low load. Most microservices failures start with a team that split before it had a reason, and paid the four costs of splitting to solve a scaling problem it did not yet have.