Avoiding the NGINX Configuration Mistakes

Not Enough File Descriptors per Worker

The worker_connections directive sets the utmost number of simultaneous connections that an NGINX worker process can have open (the default is 512).  all kinds of connections (for example, connections with proxied servers) count against the utmost, not just client connections. But it’s important to stay in mind that ultimately there is another limit on the number of simultaneous connections per worker: the operating system limit on the maximum number of file descriptors (FDs) allocated to each process. The default limit in modern UNIX distributions is 1024.

For about the smallest NGINX deployments, a limit of 512 connections per worker is perhaps too small. It is increased to 1024 in the default Nginx. conf file that we include with NGINX Open Source binaries and NGINX Plus.

The most typical configuration mistake is failing to increase the FD limit to at least double the value of worker_connections. . The fix is to line that value with the worker_rlimit_nofile directive in the main configuration context.

A connection from an NGINX worker process to a client or upstream server uses an FD, which is why more FDs are required. When NGINX acts as an internet server, it uses one FD for the client connection and one FD per served file, for a minimum of two FDs per client (but most sites are built from many files). When it acts as a proxy server, NGINX uses one FD each for the connection to the client and upstream server, and potentially a 3rd  FD for the file used to store the server’s response temporarily. As a caching server, NGINX behaves sort of a  web server for cached responses and like a proxy server if the cache is empty or expired.

NGINX also uses an FD per log file and a pair of FDs to communicate with the master process, but usually, these numbers are small compared to the amount of FDs used for connections and files.

UNIX offers several ways  to line  the number of FDs per process:

• The ulimit command if  you begin  NGINX from a shell
• The init script or systemd service manifest variables if  you begin  NGINX as a service
• The /etc/security/limits.conf file

However,  the tactic to use depends on how you start NGINX, whereas worker_rlimit_nofile works regardless of how you start NGINX.

There is also a system‑wide limit on the number of FDs, which you'll set with the OS’s sysctl fs. file-max command.  it's usually large enough, but it's worth verifying that the maximum number of file descriptors all NGINX worker processes might use (worker_rlimit_nofile * worker_processes) is significantly less than fs. file‑max. If NGINX somehow uses all available FDs (for example, during a DoS attack), it becomes impossible even to log in to the machine to repair the issue.

Not Enabling Keepalive Connections to Upstream Servers

By default, NGINX opens a replacement connection to an upstream (backend) server for every new incoming request.  this is often safe but inefficient because NGINX  and therefore the server must exchange three packets to establish a connection and three or four to terminate it.

At high traffic volumes, opening a replacement connection for every request can exhaust system resources and make it impossible to open connections at all. Here’s why:  for every connection, the 4-tuple of source address, source port, a destination address, and destination port must be unique. For connections from NGINX to an upstream server, three of the weather  (the first, third, and fourth) are fixed, leaving only the source port as a variable. When a connection is closed, the Linux socket sits within the  TIME‑WAIT state for two minutes, which at high traffic volumes increases the likelihood of exhausting the pool of available source ports.If this happens, NGINX will be unable to establish new connections with upstream servers.

The fix is to enable keepalive connections between NGINX and upstream servers –  rather than being closed when a request completes, the connection stays hospitable be used for additional requests. This both reduces the likelihood of running out of source ports and improves performance.

To enable keepalive connections:

Include the keepalive directive in every upstream{} block,  to line the number of idle keepalive connections to upstream servers preserved in the cache of each worker process.

Note that the keepalive directive doesn't limit the total number of connections to upstream servers that an NGINX worker process can open – this is a common misconception.  therefore the parameter to keepalive does not need to be as large as you might think.

We recommend setting the parameter to twice the amount of servers listed in the upstream{} block.  this is often large enough for NGINX to maintain keepalive connections with all the servers, but sufficiently small that upstream servers can process new incoming connections as well.

Note also that once you specify a load‑balancing algorithm in the upstream{} block – with the hash, ip_hash, least_conn, least_time, or random directive – the directive must appear above the keepalive directive.  this is often one of the rare exceptions to the general rule that the order of directives in the NGINX configuration doesn’t matter.

Forgetting How Directive Inheritance Works

NGINX directives are inherited downwards, or “outside‑in”:  a toddler context – one nested within another context (its parent) – inherits the settings of directives included at the parent level.  for instance, all server{} and location{} blocks within the http{} context inherit the value of directives included at the HTTP level, and a directive during a  server{} block is inherited by all the child location{} blocks in it. However, when the identical directive is included in both a parent context and its child context, the values aren't added together – instead,  the worth in the child context overrides the parent value.

The mistake is to forget this “override rule” for array directives,  which may be included not only in multiple contexts but also multiple times within a given context. Examples include proxy_set_header and add_header – having “add”  within the name of the second makes it particularly easy to forget about the overriding rule.

The proxy_buffering off Directive

Proxy buffering is enabled by default in NGINX (the proxy_buffering directive is about to on). Proxy buffering means  NGINX stores the response from a server in internal buffers as it comes in, and doesn’t start sending data to the client until the whole response is buffered.Buffering improves performance with sluggish clients since NGINX buffers the response for as long as it takes the client to receive it all, allowing the proxied server to return its response as quickly as possible and resume serving other requests.

When proxy buffering is deactivated, NGINX buffers only the principal portion of a server's response before beginning to send it to the client, using a buffer that is one memory page in size by default (4 KB or 8 KB depending on the operating system).  this is often usually just enough space for the response header. NGINX then sends the response to the client synchronously because it receives it, forcing the server to take a seat idle as it waits until NGINX can accept the next response segment.

So we're shocked at how often proxy_buffering is turned off in setups.  Perhaps it's intended to reduce the latency experienced by clients, but the effect is negligible while the side effects are numerous: with proxy buffering disabled, rate limiting and caching don’t work whether or not configured, performance suffers, and so on.

There is only a little number of use cases where disabling proxy buffering might make sense (such as long polling), so we strongly discourage changing the default. For more information, see the NGINX Plus Admin Guide.