AVX, and the Database That Would Not Start

mongo:7 came up, ran for a moment, and stopped. Docker restarted it. It stopped again. The loop settled into a rhythm and the log had nothing in it I could use — no configuration complaint, no failed connection, no permissions error, just a process that started and did not continue.

A restart loop with an empty log looks exactly like a config fault, so that is what I spent the first stretch of the evening assuming it was.

Chasing the usual suspects

In order, because this is the order everyone chases them in:

• The compose file. Read it, diffed it against a working stack, changed nothing that mattered.
• The volume. Removed it and let Mongo initialise from empty, on the theory that a half-written data directory from an earlier attempt was poisoning startup. Same loop, faster.
• Permissions on the data path. Checked ownership, checked the container's uid. Fine.
• The image tag. Pulled it again in case I had a corrupt layer. Same.

None of that was wasted exactly, but none of it was pointed at the problem either. I was working through a list of things that commonly break at this layer, and the thing that was broken was not at this layer.

The one-line diagnosis

The VM this runs on presents its CPU to the guest as QEMU's generic kvm64 model. That model masks AVX. MongoDB has required AVX since 5.0, and a binary that needs an instruction the processor does not expose does not fail gracefully, it stops.

Which makes the whole diagnosis this:

grep -o avx /proc/cpuinfo | head -1

Nothing came back. That is the answer, and it takes a second to get, and it was well into the evening before I thought to ask for it. Everything I had checked before was a question about my configuration. This is the first question I asked about the machine underneath it.

The workaround and the actual fix

The workaround was immediate: pin mongo:4.4, the last release that runs without AVX. Mongoose 8 spans 4.4 through 8.x, so nothing in the application had to change. The stack came up and stayed up.

It is also past end of life, which is not a small footnote. I am running a database that receives no security patches because my hypervisor is lying to my guest about its own processor.

The real fix is not in this repository at all:

1. Proxmox, VM, Hardware, Processors, set Type to host (or x86-64-v2-AES).
2. Stop and start the VM. A reboot from inside the guest is not enough — CPU flags are negotiated when the VM process is created, and a warm reboot reuses it. I learned this by rebooting and finding the flags unchanged and briefly concluding that the setting had not saved.
3. grep -o avx /proc/cpuinfo again, and this time expect output.

After that, MONGO_IMAGE=mongo:7 docker compose up -d.

What the workaround costs

I am, today, shipping on an end-of-life database. That is the trade: the correct fix costs a full stop and start of the VM, which takes everything else on it down with it, and on the evening in question I wanted the site back more than I wanted to be right. A small, boring, entirely legible reason, and it is still the wrong state to be in.

What I refused to do was hardcode the workaround. The image is an environment variable with a default:

mongo:
  image: ${MONGO_IMAGE:-mongo:4.4}

So the day the CPU exposes AVX, moving to a supported release is one line in .env and a restart. No Dockerfile change, no rebuild, no archaeology to work out what the pin was protecting against — the reason is written in CLAUDE.md next to the steps.

The database is still on 4.4 as I write this. The setting is still on the list of things to do during the next window where taking the VM down is acceptable, and the note explaining why is sitting where whoever gets there next will read it.