Giving a container its own address on the network: what you gain, what you pay

The default arrangement is this: the container shares the machine's address and you reach it through a port number. It is simple, it is safe, and for most work it is enough.

There are cases where it is not, and they are predictable.

When a port number is not enough

Two applications want the same number. You move one to another number, and then the links the application generates itself do not carry that number and things get confusing.

The application announces itself on the network. Finding printers, finding media players, finding smart home devices: these work through self-advertising messages. Those messages do not survive port mapping. The application announces itself, but the address it announces is the machine's address, and the other side cannot reach it.

You want the application in the router's device list. To give it a fixed address, to write a separate firewall rule, to see its traffic on its own. With port mapping that application does not exist as far as the network is concerned; only the machine does.

The other way: give the container its own identity

The container appears on the network with its own hardware address, takes its own address from the router, and sits in the device list like a physical device.

The first reaction on hearing this is usually "why doesn't everyone do it this way". The answer is in the price.

Price one: the machine cannot reach its own container

This is the part that surprises people most, and it is usually discovered after the work is finished.

Every device on the network can reach that container. The machine hosting it cannot.

The reason makes sense: both use the same physical interface, and the traffic does not come back from the switch. The machine sends the packet out, and the packet does not return to its own card.

The consequences are everyday ones: a health check running on the machine cannot reach the application, another container on the same machine cannot connect to it, a script you wrote does not work. All of it happens while the network looks flawless from the outside.

There is a fix, but it is an extra piece: open a second virtual interface on the machine attached to the same card, and route traffic for the container's address through it. Once installed the problem is gone, but because it is an add-on, if it gets forgotten nobody understands why.

Price two: you spend a real address

Every container consumes an address from your network. Ten applications means ten addresses. On a small network you reach the limit quickly.

Also, if the address is a lease it can change. When it changes, everything pinned to it, including a rule you wrote, quietly points at the wrong place.

Price three: the network card and the firewall

The card has to accept more than one hardware address. Most wireless connections do not, and some virtual environments come with it disabled. Do not decide without testing.

And your firewall now sees a device on the network it does not recognise. To you it is a container; to whoever writes the rules it is an ordinary device. That is not a problem, but left unstated it will confuse somebody one day.

An address is not enough, you also need a name

An application with an address and no name is half the win. Nobody wants to memorise an address.

For the name to be announced on the network, something has to do that job. The application's own image usually has no such part, and there is no reason it should: a database image has no business advertising itself on a network.

The rule

A port mapping gives a door a number. A network identity grants citizenship.

Grant citizenship only when the network genuinely has to treat that application as a device. When it does not, a port number is both simpler and safer.

What Atlas does

When Atlas sets this up it does not touch the application's own image. It opens a small helper next to the application, and the helper shares the application's network stack. The helper asks the router for the address, and the helper announces the name on the network.

That attachment is deliberate and it buys two things. The application's image does not need to contain an address client or a name announcer. And because the network stack belongs to the application, when the application stops the helper drops with it, leaving no orphan container behind.

For the machine-cannot-reach-its-own-container problem, the bridge described above is installed, and it is installed as a service, so it rebuilds itself when the machine restarts.

What is really worth telling is a bug found in that bridge during live testing.

The bridge was opening a route to the container's address. But it was not deleting the route of a container that had gone. After a restart, the route of a temporary address had been left hanging. The consequence: if another device later takes that address, traffic meant for that device gets diverted into the bridge. So the bug does not hit the container, it hits an innocent third party on the network.

The fix was to stop the bridge from being a thing that only adds routes, and make it a thing that deletes routes no longer matching a live container.

The general lesson, and it holds outside networking too: adding is the easy half. A route, a rule, a mapping is a claim you make about the world. When the world changes, a claim nobody withdrew turns into a lie, and it usually hits not you but somebody who has nothing to do with you.