TLDR; How to update CIMC 2.x/3.x to a recent version starts below at „Update Procedure“

Storytime

I support a Cisco Wireless infrastructure for a hospital with about 2.000 APs and 3 5520 WLCs. The environment was on 8.5.151 and the plan was to simply upgrade to 8.10.162 to still be supported by cisco after 2021-12-31.

I do always the same for such upgrades: Reading the release notes, searching for issues in the community and testing up- and downgrades between the two version.

As the 5520 Controller is built on a UCS C220 M4 Rackserver, they list the following bugs in the release notes

• FieldNotice – 63942 – Wireless Lightweight Access Points and WLAN Controllers Fail to Create CAPWAP/LWAPP Connections Due to Certificate Expiration
• Bug – CSCvt55829 – SSDs will experience data loss at 40k power on hours

Additionaly, the installed CIMC v2.x works only with adobe flash. You even cannot log in to the webgui without flash. The next option I saw, was upgrading CIMC over CLI. Therefore I SSHed into CIMC (ssh -l user CIMC-IP) and gave it a try. Oh, yes there was something funny again. You have to compile the CIMC and BIOS Binary manually, because Cisco is not providing them for C220 M4 servers directly.

You can compile those binaries by doing following steps (-> Don’t do it, it won’t help you anyway. Just read through it to feel the pain I had)

• Download the desired HUU ISO from software.cisco.com
• Copy it to a Ubuntu 16.04 Host (Nope, other distributions don’t work, Nope Ubuntu 20.04 and later don’t work) using scp or similar methods
• Mount the iso (mkdir /mnt/huu && mount ucs-c220m4-huu-3.0.4r.iso /mnt/huu)
• Generate the bios and cimc binary with the following command inside the GETFW folder, inside the iso: ./getfw -s ucs-c220m5-huu-4.1.1d.iso -d /tmp/
• The binaries will be placed in your /tmp folder

The next step seemed straight forward. Log into CIMC using SSH and start the update over CLI. But then I ran into bug „CSCvg36148 – Increase the size in the code for bmc update“. Versions below CIMC 4.x only support CLI Upgrades with files below 50 MB filesize. The current 3.x image is above this limit and therefore upgrading via CIM CLI is not possible. The whole part above is garbage…

Last chance is booting directly from the ISO using a USB stick, as mounting the ISO via CIMC is not possible due to the missing flash player. I used rufus to create a bootable USB stick as suggested by the Cisco guide. Unfortunately this doesn’t work and end’s up in bug CSCup62091 with the following message on the screen:

Googling „Error Code 906“ directs you directly to bug CSCup62091. After trying some things I found finally a way to create a bootable USB stick with the newest 3.x Host Upgrade Utility (HUU) to update CIMC.

Update Procedure

• First, do always a backup of your WLC configuration. DO NOT SKIP THIS STEP!
• Second, use a USB2.0 USB stick. The console update I used, doesn’t support USB3.0 sticks and doens’t let you boot from it
• Third, you need to create the USB stick using below script. There is currently no way doing it on Mac nor on Windows

Now you need to setup a Ubuntu 20.04 server. You can install it with or without gui, that doesn’t matter. I used VMware Fusion on my mac. Connect your USB stick to the VM configuring the settings

Copy the HUU ISO to the server. I did that using scp from my mac:

scp /tmp/ucs-c220m4-huu-3.0.4r.iso user@172.16.85.7:/tmp/

Download Ciscos script to create the bootable USB Stick from here (create_util_usb_v4.sh) or if you wish, you can check out if there’s an updated version in the original post at Cisco. Copy it again to your Ubuntu host using scp or similar methods.

Fire the script:

sh create_util_usb_v4.sh /dev/<usdevice> <isoimage.iso>
sh create_util_usb_v4.sh /dev/sdb ucs-c220m4-huu-3.0.4r.iso

Next, plug the USB Stick to your WLC, connect a Monitor, a Keyboard (for using F6) and a mouse (to navigate in HUU). During the boot, press F6 to get into the boot selection window. Select your usb device and press enter. If the screen shows something similar to the below picture, make sure to check your heart rate 😉

After some time, you’ll be presented with the EULA. Accept it and the Host Upgrade Utility will show it’s menu to you. Click on „Update All“ to update CIMC, the BIOS and other components automatically one after another.

After about 1 hour, you should see „PASS“ on all components. Click on „Exit“ to close the HUU

Now, wait about 30 minutes. I’m not sure what exactly happens in this phase, but I guess CIMC flashes certain chips with the new firmware. You’ll experience following things during this period:

• CIMC is not reachable
• The startup screen of the server shows that CIMC is not configured
• The server is shutdown (orange/yello power LED) and it does nothing after pressing the power button
• You can log in in CIMC, but starting the server gives you following error message
  

After some time, you can start the server again using CIMC or the power button and your WLC will boot up with the new BIOS. Do not unplug power or other things. Just. Wait.

As you see, this upgrade is pain. My coworker sent me this picture during the upgrade:

Portable Firefox with Flashplayer; A shortcut?

There is a shortcut. You could download a special browser that still supports flash. Mount the HUU-ISO over CIMC and update it from there. But be aware, that such browsers could always contain malware: https://www.panagenda.com/kbase/display/gl/Use+FireFox+Portable+and+Flash