Thursday, February 27, 2025

Challenges with vSphere LCM after upgrade from 6.x\7.x to 8.0GA



From Ports portal we can see that both 9084 and 9087 ports are used for version 8.x

 

Issues with ports 9087 were already documented and explained what to do for a workaround.

From our release notes for version 8.0U3GA we can see that the problem is documented: 

If you do not open port 9087 in your firewall between ESXi hosts and vCenter, compliance checks and vSphere HA might fail after vCenter update to 8.0 Update 3

Starting from 8.0 Update 3, the vSphere Lifecycle Manager downloads updates for ESXi hosts by a HTTPS connection to the vCenter instance on port 9087. If you do not open port 9087 in your firewall between ESXi hosts and vCenter, you might see compliance check errors. For example, in the lifecycle.log

 you see messages such as:

<Timestamp> In(14) lifecycle[2112988]: Downloader:373 Opening https://<VC-FQDN>:9087/vum/repository/hostupdate/__micro-depot__vendor-DEL__DEL-ESXi-8.0-Addon-cumulative_metadata__index__.xml for download

<Timestamp> Wa(12) lifecycle[2112988]: Downloader:210 Download failed: <urlopen error timed out>, 9 retry left...

In addition, vSphere HA might fail to start.

Workaround: Open port 9087 in your firewall between ESXi hosts and vCenter.

 

In 8.0U3A release notes we can see that reported “problem” was addressed by reverting the port back to how it was in previous versions, to 8084, instead of newly introduced 9087.

 

PR 3408378: If you do not open port 9087 in your firewall between ESXi hosts and vCenter, compliance checks and vSphere HA might fail after vCenter update to 8.0 Update 3

Starting from 8.0 Update 3, the vSphere Lifecycle Manager downloads updates for ESXi hosts by a HTTPS connection to the vCenter instance on port 9087. If you do not open port 9087 in your firewall between ESXi hosts and vCenter, you might see compliance check errors. For example, in the lifecycle.log

 you see messages such as:

<Timestamp> In(14) lifecycle[2112988]: Downloader:373 Opening https://<VC-FQDN>:9087/vum/repository/hostupdate/__micro-depot__vendor-DEL__DEL-ESXi-8.0-Addon-cumulative_metadata__index__.xml for download

<Timestamp> Wa(12) lifecycle[2112988]: Downloader:210 Download failed: <urlopen error timed out>, 9 retry left...

In addition, after updating vCenter to 8.0 Update 3, vSphere HA might fail to start on all ESXi hosts with messages such as:

An error occurred when vCenter Server attempted to initialize the vSphere HA Agent running on the host.

HA Agent Unreachable - The vSphere HA Agent on the host cannot be reached.

Cannot complete the configuration of the vSphere HA agent on the host. Applying HA VIBs on the cluster encountered failure.

A general system error occurred: Installing HA components failed on the cluster: domain-<ID>.

Cannot find vSphere HA master agent.

This issue is resolved in this release. The fix restores functionality on port 9084.

 

Checking further release noted of newer versions 8.0U3B, 8.0U3C and 8.0U3D we can see that there is no mention of “This issue is resolved in this release. The fix restores functionality on port 9084.”

 

On 8.0U3GA we have:A screenshot of a computer error

AI-generated content may be incorrect.

  

On 8.0U3A we have:A screenshot of a computer

AI-generated content may be incorrect.

 

On 8.0U3B, 8.0U3C and 8.0U3D we have same information like we have on 8.0U3GA: A blue and white text box

AI-generated content may be incorrect.

A screenshot of a computer error message

AI-generated content may be incorrect.

A blue and white text box

AI-generated content may be incorrect.

From checking the current documentation, we can draw the conclusion that documentation is not correct or that behavior was reverted from 8.0U3B\C\D to how it was in 8.0U3GA.

Thankfully, the issue is with the documentation, and it will be addressed soon. This was flagged to Broadcom colleagues. But please keep in mind that in upcoming new releases of VCF 9.X there will be changes in ports… so be sure to check the requirements before doing the upgrades 😊