I turned on high availability but the machine still stopped: what HA really promises

You turned on high availability. A node fell and your machine really did come up on the other node. But it served nothing for a few minutes in between, and you thought there would be no interruption.

Here is the misunderstanding: high availability does not promise uninterrupted operation, it promises an automatic restart.

Two different things

Migration while running is planned work. You start it, the machine moves to the other node together with its memory and barely stops at all. This is the tool of maintenance nights.

High availability is for an unplanned failure. When a node collapses, whatever was in its memory went with it; there is nothing left to copy. The only thing that can be done is to start the machine from scratch on another node. That is, it comes up the way your server comes up after a power cut.

So the measure of HA should not be "was there an interruption" but "how long did the interruption last".

Three things that set the duration

Noticing. The cluster does not decide immediately that a node has really died. Mistaking a network wobble for death and moving the machines is worse than not noticing the death. So there is a wait.

Silencing the old node. This step is the most misunderstood and it is the heart of this article: before the machine comes up on the new node, one must be sure the old one really stopped. If that cannot be established, the cluster forcibly restarts that node.

It sounds harsh, but the alternative is a disaster: if the old node is actually up and only its network is cut, the same machine runs in two places and writes to the same disk from both sides. The data is damaged beyond recovery. That is why the cluster prefers to shut down a node it is unsure about.

Booting. The machine boots from the beginning on the new node. The operating system starts, services come up, a database if there is one recovers itself. This time is your machine's normal boot time; the cluster has no way to shorten it.

The total in most setups is on the order of minutes. It is not zero, and it will not be.

Three conditions for HA to work at all

A majority. HA depends on the cluster being able to decide. With no majority, nothing moves. Turning on HA in a two node cluster means building a safety net that fails at the exact moment you need it most.

The disk being visible from the other node. If the machine is to come up on the new node, it must reach its disk. A machine that sits only on local storage cannot go anywhere.

Capacity. The remaining nodes must be able to carry the fallen node's load too. If both nodes are eighty percent full, HA will not save you, it will bring down the second node as well.

Do not turn HA on for every machine

Every machine with HA turned on becomes something the cluster watches constantly and starts or stops on its own decision when it sees fit. That is not always what you want.

The test is simple: if this machine came up on another node by itself at three in the morning, would learning that in the morning reassure me or worry me? If it reassures you, turn HA on. If it worries you, do not; make that machine's decisions yourself.

What Atlas does

Atlas marks separately the machines that have been left to high availability management. Where this helps most is maintenance: when the machines on a node are listed before an update, which ones depend on your decision and which on the cluster's is visible apart. Moving by hand and leaving to management are different things, and confusing them is expensive.

A small but meaningful behaviour: if high availability was never configured on the installation, Atlas sees that the relevant configuration files are absent and does not ask that question at all, returning an empty answer. So on a setup that does not use HA, the product does not keep you or the system busy looking for something that is not there.

And one limit deserves to be said plainly: Atlas does not make or change the cluster's HA decision. Whether a machine is left to HA is your business and Proxmox's; Atlas makes it visible, it does not decide for you.