Problem with vRouter after Livepatching from 4.20.1.0 -> 4.20.2.0

problem

The existing vRouter have been live patched from 4.20.1.0 to 4.20.2.0, everything shows up fine in the UI. The systemvm template used by the vRouter still shows 4.20.1.0, but the software version is correctly displayed with 4.20.2.0
Now, when i tried to start a vm, it failed with an error "Network is unavailable. Please contact administrator".

The cloudstack management servers log shows this

2025-12-09 11:40:35,705 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-92:[ctx-04bdbebb, job-717188]) (logid:8225e4c6) Unexpected exception while executing org.apache.cloudstack.api.command.admin.vm.StartVMCmdByAdmin com.cloud.utils.exception.CloudRuntimeException: Network is unavailable. Please contact administrator
...
...
...
Caused by: com.cloud.exception.ResourceUnavailableException: Resource [VirtualRouter:5014] is unreachable: Unable to send command. Router requires upgrade

after rebooting the vRouter everything started working again. Live patching seems to be broken here, i had a similar effect a few days ago as well. After a reboot, the vRouter also uses the newer systemvm template 4.20.2.0

we are on Ubuntu 24.04, our vRouters are redundant.

@jgotteswinter

@jgotteswinter
can you check the global setting minreq.sysvmtemplate.version ?
if it is 4.20.2.0, change it to 4.20.1.0

yes its indeed 4.20.2.0, i guess this has been set by the update routines? I did not change it

@jgotteswinter

@jgotteswinter , has the workaround @weizhouapache gave worked for you?

@jgotteswinter

@jgotteswinter , has the workaround @weizhouapache gave worked for you?

@jgotteswinter did it work?

@weizhouapache

yes it worked after switching the template version. But, as i said, the newer template version was set by the update routines (i guess, i didnt check it). This breaks livepatching, the workaround is to switch the template version back to the previous one.

Thanks @weizhouapache for the hint

@weizhouapache

@weizhouapache , I think we can close this as it won’t be an issue in 20.3 anymore, can we?

@weizhouapache

@weizhouapache @DaanHoogland is there any fix we need to add to address this issue?

@sudo87

@weizhouapache @DaanHoogland is there any fix we need to add to address this issue?

@sudo87
I think no code changes are needed.
just check if the change of minreq.sysvmtemplate.version setting is mentioned in the section of VR live-patch in cloudstack doc, update doc if not found.

@weizhouapache

Thank you @weizhouapache for your suggestion to update the document. We add note specifying what value should be set for config wrt VR template version in the following pr:
minreq.sysvmtemplate.version

Doc pr: apache/cloudstack-documentation#618

@weizhouapache

@weizhouapache has another suggestion to add an optional version check in live patch path, described as below:

add a VR versoin check during VR live-patch
throw an exception if VR template version is older than minreq.sysvmtemplate.version
the error message asks user to change the value of minreq.sysvmtemplate.version

@weizhouapache

@weizhouapache has another suggestion to add an optional version check in live patch path, described as below:

add a VR versoin check during VR live-patch

throw an exception if VR template version is older than minreq.sysvmtemplate.version

the error message asks user to change the value of minreq.sysvmtemplate.version

There is an existing version check, we need not to implement above.

cloudstack/server/src/main/java/com/cloud/network/router/NetworkHelperImpl.java

Line 200 in 1b0a036

if (!checkRouterVersion(router)) {

It logs following message:

2026-01-21 08:58:16,978 DEBUG [c.c.n.r.NetworkHelperImpl] (API-Job-Executor-1:[ctx-e3f94ae4, job-49, ctx-abe4488b]) (logid:185d8fcd) Router requires upgrade. Unable to send command to router: VM instance {"id":4,"instanceName":"r-4-VM","state":"Running","type":"DomainRouter","uuid":"602a9bcc-9767-4622-a504-2ece9e3650bb"}, router template version: Cloudstack Release 4.20.1 Wed May 28 06:10:04 AM UTC 2025, minimal required version: 4.20.2

We can close this issue as document is updated and required version check is in place. @weizhouapache @DaanHoogland

@weizhouapache

I have tried replicating the issue with 4.20.1 env and even after upgrade, live patch of VR worked successfully.

After live patch I am able to create instance without any issues.

Env with 4.20.1.0 -> upgraded to  4.20.2.0

Before upgrade:
Minreq sysvmtemplate version (minreq.sysvmtemplate.version): 4.10.0 

VR
Software version: 4.20.1.0
Template version: 4.20.1

After upgrade
Minreq sysvmtemplate version (minreq.sysvmtemplate.version): 4.20.2

Before live Patch
Software version: 4.20.1.0
Template version: 4.20.1

After live Patch
Software version: 4.20.2.0
Template version: 4.20.1

Created few instances in the network associated to same VR. cc: @weizhouapache

Repeated the upgrade (4.20.1.0 -> 4.20.2.0) -> live patch -> Instance deployment with:

minreq.sysvmtemplate.version: 4.20.2.0

Didn't face any error either with Live patch or Instance deployment.

Closing the issue as discussed above.

Navigation Menu

Repository navigation

Problem with vRouter after Livepatching from 4.20.1.0 -> 4.20.2.0 #12211

problem

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Problem with vRouter after Livepatching from 4.20.1.0 -> 4.20.2.0 #12211

Description

problem

Activity

weizhouapache commented on Dec 9, 2025

jgotteswinter commented on Dec 9, 2025

DaanHoogland commented on Dec 10, 2025

abh1sar commented on Dec 31, 2025

jgotteswinter commented on Dec 31, 2025

DaanHoogland commented on Dec 31, 2025

sudo87 commented on Jan 12, 2026

weizhouapache commented on Jan 12, 2026

sudo87 commented on Jan 13, 2026

sudo87 commented on Jan 13, 2026

sudo87 commented on Jan 21, 2026

sudo87 commented on Jan 22, 2026

abh1sar commented on Jan 23, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions