I changed the network settings and lost access: sawing the branch you sit on

Network configuration is applied over the very connection you are using to apply it. A small mistake cuts the connection at the moment of applying, and without a prepared way back you need physical access.

AtlasPVE ·

This entry answers

  • proxmox no access after network change
  • proxmox locked myself out
  • proxmox interfaces file broken
  • proxmox ifreload hangs
  • proxmox revert network change

You corrected the address, you applied it, and the panel went silent. This is the most common way people lock themselves out of a server, and the reason is simple: the network setting is applied over the connection you are using to apply it. You are sawing the branch you sit on.

Three preparations before changing anything

One: a second way in that the change cannot affect. A physical console, the server's own out-of-band management card, or a second network card on a different bridge. If you have none of these, what you are about to do is an irreversible operation, whatever it is called.

Two: knowing what "back" looks like. Keep a copy of the current configuration. "I'll remember" is not remembered in the moment of panic.

Three: one change at a time. Change two things at once and you cannot tell which one cut the connection, and trial and error is expensive on a machine you have to walk to.

Common mistakes

Asking for an address on a bridge with no physical port. A bridge with no physical port in it cannot send packets. Ask it to obtain an address and the apply step does not finish, it hangs. A hung apply is worse than a failed one: the system is neither in the old state nor the new one.

The port name changing. Physical card names can shift when hardware changes or after certain upgrades. If the name in the configuration no longer exists on the machine, the bridge sits empty.

Changing the address and forgetting the gateway. Or the reverse. The machine comes up, sees its own network, and cannot get out.

Putting a tag on the management interface. Right in one direction, silence in the other. Because tagging produces no error, this is also the one noticed last.

If you are already locked out

The honest answer: you need physical access or an out-of-band route. Rebooting does not rescue you either, because the machine comes up with the file on disk and that file is still broken. A reboot helps only if you have already fixed that file.

What Atlas does

When Atlas applies a network configuration it starts a rollback countdown: if you do not confirm within the set time, the old configuration comes back on its own. What matters is who holds that countdown. The countdown is set on the system's own timer, not on Atlas itself, so even if the Atlas service crashes at that moment the rollback still runs. A safety net that depends on the survival of the software making the change is not a safety net.

Second, Atlas refuses up front any configuration that would make rollback impossible. If an address is requested on a bridge with no physical port, the operation stops before being applied, because that combination locks the apply step, and a locked step cannot even write the rollback record. This check sits on the server rather than in the interface, so a request sent by hand hits the same wall.

Sources

Proxmox's own documentation. In English, and it has the final word on this subject.

Related entries

How does this look inside Atlas?

Go to the product page