VPC restart with cleanup fails when VPC has multiple public IP ranges from different VLANs — "cidr is not formatted correctly"

problem

When restarting a VPC with cleanup=true that has public IPs from different VLAN ranges, the operation fails with:

cidr is not formatted correctly: 192.168.2.0/24,160.0.0.0/24

Expected Behavior:

VPC restarts successfully.

Actual Behavior:

VPC restart fails with error:

cidr is not formatted correctly: 192.168.2.0/24,160.0.0.0/24

versions

4.21
kvm

The steps to reproduce the bug

Steps to Reproduce:

In an existing zone, add a second public IP range on a different VLAN (e.g., if the zone already has a range like 183.9.9.0/24 on VLAN 100, add a new range like 160.0.0.0/24 on VLAN 200)
Create a VPC
Create a network tier (subnet) inside the VPC
Deploy a VM inside that tier
Acquire a public IP from the second VLAN range and assign it to the VM (so the VPC now has the default Source NAT IP from the first range and the newly acquired IP from the second range)
Go to the VPC and click Restart VPC with "Clean up" enabled

What to do about it?

No response

@jeanvetorello

@jeanvetorello
can you share some logs ?
it looks like a major issue

@weizhouapache

@weizhouapache Sure, here are the logs and reproduction details.

Why this is a major issue

I consider this a major issue because it directly causes VPC unavailability in production. The scenario is:

You add a second public IP range (on a different VLAN/VXLAN) to a VPC — this works fine, the VPC operates normally
However, if the VPC router happens to reboot (e.g., host migration, restart with cleanup, or any event that triggers router re-deployment), the router fails to come back up
The VPC becomes completely unavailable — all VMs inside lose connectivity
The only workaround to recover is to remove public IP ranges until only one remains in the VPC, then restart

This is particularly dangerous because the failure doesn't happen when adding the IP range — everything looks fine. It's a ticking time bomb that only triggers on the next router restart/redeployment.

I've applied a patch to my production environment (CloudStack 4.21.0.0) and it resolved the issue. The PR is submitted.

Reproduction (cloudstack-simulator 4.21.0.0)

Steps:

Create a VPC with a tier network
Deploy a VM in the tier (so the VPC router is created)
The networks table ends up with comma-separated CIDRs for the VPC's public/control network:

mysql> SELECT id, name, cidr, gateway FROM networks WHERE removed IS NULL;
+-----+----------+-----------------------------------+-------------------------+
| id  | name     | cidr                              | gateway                 |
+-----+----------+-----------------------------------+-------------------------+
| 200 | NULL     | 192.168.2.0/24,160.1.0.0/24       | 192.168.2.1,160.1.0.1   |
| 204 | test-sub | 10.0.0.0/24                       | 10.0.0.1                |
+-----+----------+-----------------------------------+-------------------------+

Restart the VPC with cleanup=true → FAILS

API Error Response

{
    "queryasyncjobresultresponse": {
        "cmd": "org.apache.cloudstack.api.command.user.vpc.RestartVPCCmd",
        "jobresultcode": 530,
        "jobresult": {
            "errorcode": 530,
            "errortext": "cidr is not formatted correctly: 192.168.2.0/24,160.1.0.0/24"
        },
        "jobstatus": 2
    }
}

Management Server Stack Trace (vmops.log)

2026-02-10 12:36:49,746 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-1:[ctx-a55fcee4, job-44]) (logid:036c8e2e)
Unexpected exception while executing org.apache.cloudstack.api.command.user.vpc.RestartVPCCmd
com.cloud.utils.exception.CloudRuntimeException: cidr is not formatted correctly: 192.168.2.0/24,160.1.0.0/24
	at com.cloud.utils.net.NetUtils.cidrToLong(NetUtils.java:911)
	at com.cloud.utils.net.NetUtils.isNetworkAWithinNetworkB(NetUtils.java:894)
	at com.cloud.network.dao.NetworkVO.equals(NetworkVO.java:603)
	at java.base/java.util.HashMap.getNode(HashMap.java:568)
	at java.base/java.util.LinkedHashMap.get(LinkedHashMap.java:440)
	at com.cloud.network.router.VpcNetworkHelperImpl.reallocateRouterNetworks(VpcNetworkHelperImpl.java:162)
	at com.cloud.network.router.NetworkHelperImpl.deployRouterWithTemplates(NetworkHelperImpl.java:542)
	at com.cloud.network.router.NetworkHelperImpl.deployRouter(NetworkHelperImpl.java:602)
	at org.apache.cloudstack.network.router.deployment.VpcRouterDeploymentDefinition.deployAllVirtualRouters(VpcRouterDeploymentDefinition.java:195)
	at org.apache.cloudstack.network.router.deployment.RouterDeploymentDefinition.executeDeployment(RouterDeploymentDefinition.java:393)
	at org.apache.cloudstack.network.router.deployment.RouterDeploymentDefinition.findOrDeployVirtualRouter(RouterDeploymentDefinition.java:255)
	at org.apache.cloudstack.network.router.deployment.VpcRouterDeploymentDefinition.findOrDeployVirtualRouter(VpcRouterDeploymentDefinition.java:157)
	at org.apache.cloudstack.network.router.deployment.RouterDeploymentDefinition.deployVirtualRouter(RouterDeploymentDefinition.java:221)
	at com.cloud.network.element.VpcVirtualRouterElement.implementVpc(VpcVirtualRouterElement.java:165)
	at com.cloud.network.vpc.VpcManagerImpl.startVpc(VpcManagerImpl.java:2039)
	at com.cloud.network.vpc.VpcManagerImpl.rollingRestartVpc(VpcManagerImpl.java:3633)
	at com.cloud.network.vpc.VpcManagerImpl.restartVpc(VpcManagerImpl.java:2422)
	at com.cloud.network.vpc.VpcManagerImpl.restartVpc(VpcManagerImpl.java:2382)
	...
	at org.apache.cloudstack.api.command.user.vpc.RestartVPCCmd.execute(RestartVPCCmd.java:94)

Root Cause

The fix in the PR wraps the cidr values with StringUtils.getFirstValueFromCommaSeparatedString() before passing them to NetUtils.isNetworkAWithinNetworkB() in NetworkVO.equals().

@jeanvetorello

@jeanvetorello
it looks like a regression of #11249

for Public network (id=200), the values should be NULL

        broadcast_uri: NULL
              gateway: NULL
                 cidr: NULL

can you set these to NULL, and retry ?

@weizhouapache

@weizhouapache for the pointer, you're right — it is a regression from #11249.

I confirmed that setting the Public network's cidr/gateway to NULL does make the VPC restart work:

UPDATE networks SET cidr = NULL, gateway = NULL WHERE id = 200;
-- restartVPC cleanup=true → SUCCESS

However, the values get repopulated every time a new public IP range (VLAN) is added. The problem is in ConfigurationManagerImpl.addCidrAndGatewayForIpv4() (introduced by #11249) — it's called for all network types without checking if the network is Shared:

// ConfigurationManagerImpl.java ~line 5440 — no network type check
if (vlan != null) {
    if (ipv4) {
        addCidrAndGatewayForIpv4(networkId, vlanGateway, vlanNetmask);  // called for ANY network
    }
}

So the addCidrAndGatewayForIpv4 / addCidrAndGatewayForIpv6 methods append comma-separated CIDRs to the Public network too, not just Shared networks as intended.

@jeanvetorello

@weizhouapache for the pointer, you're right — it is a regression from #11249.

I confirmed that setting the Public network's cidr/gateway to NULL does make the VPC restart work:

UPDATE networks SET cidr = NULL, gateway = NULL WHERE id = 200;
-- restartVPC cleanup=true → SUCCESS
However, the values get repopulated every time a new public IP range (VLAN) is added. The problem is in ConfigurationManagerImpl.addCidrAndGatewayForIpv4() (introduced by #11249) — it's called for all network types without checking if the network is Shared:

// ConfigurationManagerImpl.java ~line 5440 — no network type check
if (vlan != null) {
if (ipv4) {
addCidrAndGatewayForIpv4(networkId, vlanGateway, vlanNetmask); // called for ANY network
}
}
So the addCidrAndGatewayForIpv4 / addCidrAndGatewayForIpv6 methods append comma-separated CIDRs to the Public network too, not just Shared networks as intended.

@jeanvetorello
your analysis is correct.
no network type check - we do need to add a check

cc @sureshanaparti

🏷️ Issue Triaged

I've categorized this issue as bug based on the following analysis:

Reasoning: This describes a functional failure where a VPC restart with cleanup fails with a CIDR formatting error when multiple public IP ranges from different VLANs are present. This is unexpected behavior that prevents normal operations.

View Triage Details

Analysis

Keywords detected: "fails", "error", "Expected Behavior", "Actual Behavior"
Issue type indicators: Clear bug report format with problem description, expected vs actual behavior, and reproduction steps
Confidence: High

Recommended Next Steps

Investigate the CIDR validation logic in VPC restart code
Ensure proper handling of comma-separated CIDR ranges
Add test cases for multi-VLAN public IP configurations

References: Triage run §22673429317

Generated by Issue Triage Agent

To install this workflow, run gh aw add github/gh-aw/.github/workflows/issue-triage-agent.md@94662b1dee8ce96c876ba9f33b3ab8be32de82a4. View source at https://github.com/github/gh-aw/tree/94662b1dee8ce96c876ba9f33b3ab8be32de82a4/.github/workflows/issue-triage-agent.md.

Fixed in #12622

Navigation Menu

Repository navigation

VPC restart with cleanup fails when VPC has multiple public IP ranges from different VLANs — "cidr is not formatted correctly" #12621

problem

versions

The steps to reproduce the bug

What to do about it?

Why this is a major issue

Reproduction (cloudstack-simulator 4.21.0.0)

API Error Response

Management Server Stack Trace (vmops.log)

Root Cause

🏷️ Issue Triaged

Analysis

Recommended Next Steps

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

VPC restart with cleanup fails when VPC has multiple public IP ranges from different VLANs — "cidr is not formatted correctly" #12621

Description

problem

versions

The steps to reproduce the bug

What to do about it?

Activity

weizhouapache commented on Feb 10, 2026

jeanvetorello commented on Feb 10, 2026

Why this is a major issue

Reproduction (cloudstack-simulator 4.21.0.0)

API Error Response

Management Server Stack Trace (vmops.log)

Root Cause

weizhouapache commented on Feb 10, 2026

jeanvetorello commented on Feb 10, 2026

weizhouapache commented on Feb 10, 2026

github-actions commented on Mar 4, 2026

🏷️ Issue Triaged

Analysis

Recommended Next Steps

sureshanaparti commented on Apr 13, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions