Repository navigation
Failed to change offering of ROOT volume on VMware #13261
Description
problem
the power_state is not update immediately when vm is stopped, which cause an issue when change offering of ROOT volume on VMware.
resizing volume works well.
Steps to reproduce the issue:
- create a vm and start it on VMware
mysql> select id,name,uuid,state,power_state,power_state_update_time,host_id,last_host_id from vm_instance where id=7;
+----+----------+--------------------------------------+---------+-------------+-------------------------+---------+--------------+
| id | name | uuid | state | power_state | power_state_update_time | host_id | last_host_id |
+----+----------+--------------------------------------+---------+-------------+-------------------------+---------+--------------+
| 7 | test-wei | 07618433-1617-4ca6-869a-b8abbe4a3db6 | Running | PowerOn | 2026-05-27 07:28:43 | 1 | 1 |
+----+----------+--------------------------------------+---------+-------------+-------------------------+---------+--------------+
1 row in set (0.00 sec)
- stop the vm, the state is updated, but power_state is still
PowerOn
mysql> select id,name,uuid,state,power_state,power_state_update_time,host_id,last_host_id from vm_instance where id=7;
+----+----------+--------------------------------------+----------+-------------+-------------------------+---------+--------------+
| id | name | uuid | state | power_state | power_state_update_time | host_id | last_host_id |
+----+----------+--------------------------------------+----------+-------------+-------------------------+---------+--------------+
| 7 | test-wei | 07618433-1617-4ca6-869a-b8abbe4a3db6 | Stopping | PowerOn | 2026-05-27 07:28:43 | 1 | 1 |
+----+----------+--------------------------------------+----------+-------------+-------------------------+---------+--------------+
1 row in set (0.00 sec)
mysql> select id,name,uuid,state,power_state,power_state_update_time,host_id,last_host_id from vm_instance where id=7;
+----+----------+--------------------------------------+---------+-------------+-------------------------+---------+--------------+
| id | name | uuid | state | power_state | power_state_update_time | host_id | last_host_id |
+----+----------+--------------------------------------+---------+-------------+-------------------------+---------+--------------+
| 7 | test-wei | 07618433-1617-4ca6-869a-b8abbe4a3db6 | Stopped | PowerOn | 2026-05-27 07:28:43 | NULL | 1 |
+----+----------+--------------------------------------+---------+-------------+-------------------------+---------+--------------+
1 row in set (0.00 sec)
- change the disk offering of the ROOT volume
got an exception
- retry when the power_state is updated to
PowerOff, it worked
mysql> select id,name,uuid,state,power_state,power_state_update_time,host_id,last_host_id,update_time from vm_instance where id=7;
+----+----------+--------------------------------------+---------+-------------+-------------------------+---------+--------------+---------------------+
| id | name | uuid | state | power_state | power_state_update_time | host_id | last_host_id | update_time |
+----+----------+--------------------------------------+---------+-------------+-------------------------+---------+--------------+---------------------+
| 7 | test-wei | 07618433-1617-4ca6-869a-b8abbe4a3db6 | Stopped | PowerOff | 2026-05-27 11:24:43 | NULL | 1 | 2026-05-27 11:24:12 |
+----+----------+--------------------------------------+---------+-------------+-------------------------+---------+--------------+---------------------+
1 row in set (0.00 sec)
versions
ACS 4.22.0.0
VMware 8
it should impact other versions too
The steps to reproduce the bug
...
What to do about it?
No response
Activity
DaanHoogland commentedon May 27, 2026
@weizhouapache is this vmware 8 only, i.e. when you say "should impact other versions too” is this ACS only or vmware as well?
weizhouapache commentedon May 27, 2026
I think the VMware version does not matter
the error comes from the ACS mgmt server, before sending the command to VMware.
it should impact other ACS versions too
DaanHoogland commentedon May 27, 2026
But do you agree the root issue is the powerstate not being updated, is it? Not the error on change offering.
weizhouapache commentedon May 27, 2026
I think it is good that the powerstate is updated when mgmt server receives the vm states from agent (kvm or vmware).
the root cause is, changing offering for root volume should NOT rely on the power state as it is NOT the real-time state of VM. Instead, similar to all other vm/volume operations, it should rely on the VM state (Stopped, not power state/PowerOff).
the fix could be just 1 line of code change
github-actions commentedon Jun 11, 2026
🏷️ Issue Triaged
Hi
@weizhouapache! I've categorized this issue as bug based on the following analysis:Reasoning: This issue reports that
power_stateis not updated immediately when a VM is stopped on VMware, causing the "change offering of ROOT volume" operation to fail with an exception. It includes clear reproduction steps and database state evidence.View Triage Details
Analysis
power_state,stopped,exception,VMware,reproduce,failedpower_state=PowerOnafter VM stop, and error screenshotRecommended Next Steps
power_stateis updated synchronously, or that the disk-offering-change operation waits for the power state to be consistentpower_statematches expected state before allowing disk offering changesReferences: Triage run §27329568210
Add this agentic workflows to your repo
To install this agentic workflow, run
github-actions commentedon Jun 16, 2026
🏷️ Issue Triaged
Hi @weizhouapache! I've categorized this issue as bug based on the following analysis:
Reasoning: The
power_statefield is not updated synchronously when a VM is stopped on VMware, causing a race condition where an attempt to change the ROOT volume offering fails with an exception while the state still showsPowerOndespite the VM being inStoppedstate.View Triage Details
Analysis
power_state, VMware,Stoppedstate mismatch, ROOT volume, offering change, exceptionpower_stateupdates toPowerOffsucceeds), version info providedRecommended Next Steps
power_stateupdate for VMware can be made synchronous (or at least awaited) during the VM stop flow before allowing volume operationspower_stateconsistency before proceedingReferences: Triage run §27631885429
Add this agentic workflows to your repo
To install this agentic workflow, run
DaanHoogland commentedon Jul 6, 2026
fixed in #13271