Announcing the General Availability of VMware vCenter Server 7.0 U3g

As of July 23, 2022, VMware has officially released vSphere version 7.0 U3g. The vCenter 7.0 U3g release specifically addresses an issue seen in 7.0 U3f for users where the vCenter is or was previously joined to an Active Directory domain that used Integrated Windows Authentication (IWA) as an identity source.

VMware KB 89027 addresses the issue and associated workarounds in much more detail, but that’s the big item that is addressed in this particular release.

Please note: VMware ESXi 7.0 U3g wasn’t included as part of this release announcement. As of this writing, this U3g update applied to vCenter Server only.

vCenter Server 7.0 U3g | Build 20150588

Release Notes: https://docs.vmware.com/en/VMware-vSphere/7.0/rn/vsphere-vcenter-server-70u3g-release-notes.html
Documentation: https://docs.vmware.com/en/VMware-vSphere/index.html
Download: https://customerconnect.vmware.com/downloads/info/slug/datacenter_cloud_infrastructure/vmware_vsphere/7_0

Announcing the General Availability of VMware vSphere 7.0 U3f

Update as of July 24, 2022: If your vCenter Server is or was previously attached to an Active Directory domain that used Integrated Windows Authentication (IWA) as an Identity Source, skip this vCenter release and go to vCenter 7.0 U3g. VMware KB 89027 explains the issue in more detail.

As of July 12, 2022, VMware has officially released vSphere version 7.0 U3f. This particular release addresses a number of security patches for vCenter Server, including CVE-2022-22982. This particular CVE has a CVSSv3 score of 5.3 (Moderate) and is explained in more detail in VMware’s VMSA-2022-0018 Security Advisory.

As for ESXi, this patch release addresses a number of security advisories, PRs, as well as updating drivers and VIBs on ESXi hosts. The specific security advisories addressed in this release include: CVE-2022-23816, CVE-2022-23825, CVE-2022-26373, CVE-2022-28693, and CVE-2022-29901. These security advisories are explained in more detail in VMware’s VMSA-2022-0020 Security Advisory.

vCenter Server 7.0 U3f | Build 20051473

Release Notes: https://docs.vmware.com/en/VMware-vSphere/7.0/rn/vsphere-vcenter-server-70u3f-release-notes.html
Documentation: https://docs.vmware.com/en/VMware-vSphere/index.html
Download: https://customerconnect.vmware.com/downloads/info/slug/datacenter_cloud_infrastructure/vmware_vsphere/7_0

ESXi 7.0 U3f | Build 20036589

Release Notes: https://docs.vmware.com/en/VMware-vSphere/7.0/rn/vsphere-esxi-70u3f-release-notes.html

Announcing the General Availability of VMware vSphere 7.0 U3d

As of March 29, 2022, VMware has officially released vSphere version 7.0 U3d. The most notable thing I’ve seen from this patch release is that the vCenter Server patch addresses CVE-2022-22948. This particular CVE has a CVSSv3 score of 5.5 (Moderate) and is explained in more detail in VMware’s VMSA-2022-0009 Security Advisory. As for ESXi, this patch release addresses a number of issues and PRs, as well as updating drivers and VIBs on ESXi hosts.

vCenter Server 7.0 U3d | Build 19480866

Release Notes: https://docs.vmware.com/en/VMware-vSphere/7.0/rn/vsphere-vcenter-server-70u3d-release-notes.html
Documentation: https://docs.vmware.com/en/VMware-vSphere/index.html
Download: https://customerconnect.vmware.com/downloads/info/slug/datacenter_cloud_infrastructure/vmware_vsphere/7_0

ESXi 7.0 U3d | Build 19482537

Release Notes: https://docs.vmware.com/en/VMware-vSphere/7.0/rn/vsphere-esxi-70u3d-release-notes.html

Announcing the General Availability of VMware vSphere 7.0 U3c

Update as of March 31, 2022: VMware has since released vSphere 7.0 U3d which replaces the 7.0 U3c release. It also appears that the 7.0 U3c release is no longer available in certain online repos, as well.

As of January 27, 2022, VMware has officially released vSphere version 7.0 U3c, and this particular release resolves a number of issues that were identified in previous U3 versions. In addition, the Apache log4j components have also been updated to version 2.17 to resolve both CVE-2021-44228 and CVE-2021-45046.

Please be sure to read the release notes carefully and work with your VMware TAM or account team, as upgrade guidance can vary based on the release you’re upgrading from. If you’re upgrading from version 6.5, 6.7, 7.0 GA, or 7.0 U1 to 7.0 U3c, this should be pretty straightforward; Upgrade vCenter Server first, and then upgrade your hosts.

However, if you’re upgrading from 7.0 U2c, 7.0 U2d, or prior releases of 7.0 U3, please read through the release notes and KB 87528 in thorough detail before upgrading. As mentioned in the release notes, there’s now a pre-check script called vSphere_upgrade_assessment.py that should be used to determine if there are any ESXi hosts that require remediation before upgrading the vCenter Server.

In short, the reason behind this has to do with an Intel i40en (or i40enu) driver that was renamed between releases. In some cases, both drivers could exist on the same ESXi hosts, and this could lead to network communication errors. The aforementioned script checks to see if duplicate versions of this driver exist on the hosts. If the driver is found, it’s likely that the hosts will need to be upgraded before you can upgrade the vCenter Server. Also note that the script isn’t able to confirm or check hosts that may be in a disconnected or powered off state, so it’s best to ensure that everything is online and accessible when running this script.

vCenter Server 7.0 U3c | Build 19234570

Release notes: https://docs.vmware.com/en/VMware-vSphere/7.0/rn/vsphere-vcenter-server-70u3c-release-notes.html
Documentation: https://docs.vmware.com/en/VMware-vSphere/index.html
Download: https://customerconnect.vmware.com/web/vmware/downloads/info/slug/datacenter_cloud_infrastructure/vmware_vsphere/7_0

ESXi 7.0 U3c | Build 19193900

Release Notes: https://docs.vmware.com/en/VMware-vSphere/7.0/rn/vsphere-esxi-70u3c-release-notes.html

Other Blogs and KBs related to the vSphere 7.0 U3c release announcement

Announcing Availability of vSphere 7.0 U3c (blog post):
https://blogs.vmware.com/vsphere/2022/01/announcing-availability-of-vsphere-7-update-3c.html
vSphere 7 Update 3 – What’s New (blog post):
https://core.vmware.com/blog/vsphere-7-update-3-whats-new
Using the vSphere_upgrade_assessment.py script | VMware KB 87258: https://kb.vmware.com/s/article/87258
Important list of Knowledge base articles identified for vSphere 7.0 U3c release | VMware KB 87327: https://kb.vmware.com/s/article/87327

Cisco UCS Log Fullness due to ECC Memory Errors

Greetings, everyone! I recently had a customer who was running into an issue where they were seeing the Cisco UCS System Event Log (SEL) fullness being reported within vCenter Server.

Upon looking at the host’s SEL Logs tab in UCS Manager, we could see that the SEL had filled up due to a significant number of ECC errors on a particular set of DIMMs. Typically, we could just clear the SEL and move on, but I’ve found that following these steps can not only clear the SEL, but may reset the ECC memory error state to help determine if a DIMM truly is flaky.

  1. Open your SSH client of choice and connect to the Cisco UCS Manager.


  2. Log in to UCS Manager. In this particular environment, the customer had to logon using their domain credentials in this format:
    ucs-DOMAIN\USERID


  3. Run these set of commands to connect to the particular blade (if applicable), reset the memory errors, and clear the SEL.

    In this example, connect to Chassis #3, Blade #2:
    scope server 3/2

    Then, reset all ECC memory errors being reported in the SEL:
    reset-all-memory-errors

    Commit the changes to UCS manager:
    commit-buffer

    The next step is to reset or clear the SEL:
    clear sel

    Again, commit the changes to UCS Manager:
    commit-buffer

  4. I believe the last step is optional, but in my experience, it didn’t hurt. Reset the CIMC, just to be safe.
    reset

    As usual, commit the changes:
    commit-buffer

    Doing so will drop any connection to the CIMC for that server (including the SSH session that was established earlier in this post).


  5. Ping or try to connect to the CIMC address after a few minutes to ensure connectivity and remote management.

And that’s pretty much all there is to it! Hopefully you found this post helpful. As always, thanks for stopping by!

PowerCLI: Find Host Profiles and Versions in vCenter

As part of our planned upgrade to vSphere 6.7, we needed the ability to quickly scan the various vCenter Servers for host profiles that may be configured for version 5.5 or older. According to the vSphere 6.7 Release Notes, if these older host profiles are found, the vCenter pre-upgrade check will fail.

Continue reading “PowerCLI: Find Host Profiles and Versions in vCenter”

PowerCLI: Find BIOS-Enabled VMs

This script is an idea that spun off of my previous post, PowerCLI: Find UEFI-Enabled VMs. If you’re preparing to enable Secure Boot in a VMware environment, it may be helpful to identify the VMs that cannot be upgraded. As you might recall, enabling secure boot requires the following:

  • VMware vSphere 6.5 or higher
  • Virtual hardware version 13 or higher
  • VMs need to be configured with EFI boot firmware

Continue reading “PowerCLI: Find BIOS-Enabled VMs”

PowerCLI: Find UEFI-Enabled VMs

With all the news regarding the Spectre and Meltdown CPU vulnerabilities over the past several months, there’s been a greater focus to get VMware virtual machines to virtual hardware version 9 or higher, as noted by Andrea Mauro’s post regarding these vulnerabilities. In addition to that, several companies and organizations may be looking to enable Secure Boot, a feature first introduced with vSphere 6.5. However, in order to enable secure boot, the virtual machine needs to be configured with both EFI boot firmware AND be on virtual hardware version 13 or higher.

Continue reading “PowerCLI: Find UEFI-Enabled VMs”