HP Procurve Switches Crashing

We have been coming into a fairly critical issue with HP Procurve switches crashing resulting in ‘dead ports’. This seems to only affect the Procurve 1810 series – but does it for the 8, 24 and 48 port models alike.

This often takes a little while to spot, since the switch doesn’t completely stop working. Anything that was currently connected actually stays working, but new connections don’t work. Typical symptoms are as follows:

  • No DHCP for new devices
  • Setting IP address statically still won’t connect
  • Current connections are still working
  • Changing the port on the switch does nothing
  • Unable to log onto the GUI of the switch
  • Restarting the switch resolves the issue

We’ve even noticed that often the switches crash at the exact same time, most likely indicating the issue is caused by something network itself. This can potentially make troubleshooting even harder, since swapping the cable to another switch doesn’t fix the problem.

We have never identified the exactly what causes the problem, but have an easy fix – a firmware update.

HP have identified and described the issue as the following:

0000128904 - Port Hang – Randomly, a port on the switch may hang. No MAC addresses will be registered on the port and both inbound and outbound traffic will no longer be forwarded on an affected port. The only recovery method is to reset the switch.

As of writing this, there’s a different firmware versions for each model, however all of them identify and resolve the issue of the HP Procurve switches crashing.

Updating the firmware is relatively simple and only causes a few minutes of downtime whilst the switch reboots. We have found it’s best to update the backup image first, then swap over the core image and then restart the switch.

This can be done by doing the following:

  • Download the newest firmware
  • Log onto the GUI of the switch
  • Click Maintenance and then Update Manager
  • Change the dropdown to HTTP
  • Select Browse and select the firmware
  • Keep the Update Type to Code and the Image as Backup
  • Press Download
  • Wait for the firmware to update. This usually takes about a minute
  • Then to go Maintenance and then Dual Image Configuration
  • Choose Image2 and press activate
  • Go to Diagnostics and then Reboot Switch
  • Keeping the Save Configuration box ticked, press Reboot Switch
  • Wait for it to reboot
  • All done!

Download links for the firmware/release notes can be located here.