How a server's memory is used

A server has a fixed amount of RAM. The operating system and the database take some of it before your application starts, and what is left decides how many workers fit.

You press Enlist at 8:00 on enlistment day. The page spins for half a minute, then shows an error and asks you to try again. Thousands of students sent requests in the same minute. One cause of such an error is a server that has run out of RAM.

Your laptopSends the request
Enlistment serverReceives thousands of requests
RAMA fixed amount per server
Think first. Before any student reaches the enlistment server, part of its RAM is already in use. What do you think is using it?
Open Task Manager (Windows) or Activity Monitor (Mac) to show how much RAM a laptop uses with no programs open. Students usually guess close to zero. Then ask what a server has running before the first request, and compare their list with the bar at the default setting in the next step.

Each request from an enlistment page is handled by a worker, and a worker needs memory of its own. Drag Total RAM and Workers. Watch the bar and the message under it.

How the memory is used
The machine you are paying for
How many workers fit
…
More detail

These two sliders and two switches change what each worker costs and where the database runs. The figures are typical for a small Django deployment and are illustrative.

Why 15% stays free

Keep 15% of the RAM free. Memory that looks unused holds the disk cache, which is recently read files kept in RAM for fast access. When no RAM is left, the system starts swapping, which means using the disk as extra memory. The disk is roughly a thousand times slower.

Who uses the memory before your application starts

The operating system, nginx (a program that passes browser requests to your application) and Postgres (a database program) use memory first.

Each worker is a separate process. Twenty workers need twenty copies of your application in memory. A thread shares the memory of its process, so many threads cost little extra. Threads suit work that mostly waits, such as waiting for the database.

More detail

What one worker is

Django is a Python framework for building web applications. A common way to run a Django application is gunicorn, a program that starts several workers. The command gunicorn --workers 8 starts eight separate processes. Each worker loads its own Python interpreter, its own copy of Django and its own copy of every library the code imports. The pandas data library alone can use 100 MB before the first request arrives.

The RAM per worker setting decides how large the machine has to be. Each worker uses that memory whether or not anyone is visiting the site.

Limits of threads

Python has a rule called the global interpreter lock, which lets only one thread run Python code at a time. Threads therefore suit work that mostly waits. Separate processes suit work that mostly computes.

The database can run on a second machine. The application machine gets its memory back, you pay for two machines, and every query now crosses a network. Drag Total RAM and compare the two columns.

One machine and two machines
 Everything on one machinePostgres on its own machine

You can now work out how many workers fit and whether a second machine pays.

More detail

What splitting changes

The application machine gets the database's memory back, so more workers fit. Postgres gets a whole machine, so it can keep frequently used data in memory and read less from disk. Both help.

Every query now crosses a network and no longer stays inside one machine. That adds about 0.4 ms to each query. At two queries per request, every request takes 0.8 ms longer.

The model charges ₱540 a month for each gigabyte of RAM. The bottom row, cost per request per second, shows whether the second machine pays for itself. The "Database on the same machine" switch under the memory bar removes the database's memory from the bar without adding a machine. This comparison charges you for the second machine.

Reasons to separate the database

The main reasons are operational. The database can be backed up, copied to a standby and restarted without touching the application. A runaway query cannot use up the application's memory. The two machines can be sized separately. Speed is a weaker reason, as the cost row shows.

For the workers' side of the same machine, see Server capacity.