CSCur43050 APs mfg in September/October 2014 unable to join an AireOS controller

Zum Jahresenden überrascht Cisco mit einem heftigen Bug:

CSCur43050

 

Doch worum gehts beim CSCur43050?

New Aironet APs with factory installed recovery IOS are able to join the controller 8.0.100.0 and download 15.3(3)JA IOS. But after the AP reload, the APs are unable to join the controller. On the AP, logs similar to the following are seen:

*Oct 16 12:39:06.231: AP has SHA2 MIC certificate – Using SHA2 MIC certificate for DTLS.

*Oct 16 13:14:56.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: ***.***.***.*** peer_port: 5246Peer certificate verification failed FFFFFFFF

*Oct 16 13:14:56.127: DTLS_CLIENT_ERROR: ../capwap/base_capwap/capwap/base_capwap_wtp_dtls.c:496 Certificate verified failed!
*Oct 16 13:14:56.127: %DTLS-5-SEND_ALERT: Send FATAL : Bad certificate Alert to ***.***.***.***:5246
*Oct 16 13:14:56.127: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to ***.***.***.***:5246

Another symptom of this problem is that the AP may be able to join the 8.0.100.0 controller, download the IOS code, boot up and join the controller OK … but when it goes to upgrade to newer 8.x code, it gets stuck in a loop failing the download.

 

Wann tritt der Bug auf?

Seen only with APs that were manufactured in September or October, 2014 – all Aironet APs were affected EXCEPT the 700 series. Seen with WLCs running 8.0.100.0 or an 8.0.100.x special.

If the WLC was manufactured in September 2014, or later (i.e. has a SHA2 MIC), then the first symptom is seen, i.e. the AP joins the 8.0.100 WLC, downloads the image, but then fails to rejoin.

If the WLC was manufactured before September 2014 (i.e. does not have a SHA2 MIC), then the second symptom is seen, i.e. the AP can join the 8.0.100 WLC OK, but then will fail download during a subsequent upgrade.

Also seen with new APs trying to join a controller running IOS-XE 3.6.0 (15.3(3)JN k9w8 image.) (Track CSCur50946 for the IOS-XE fix)

Wie schlimm ist der CSCur43050 wirklich?

Der CSCur43050 ist ein Severity 1 Bug somit hat er die Statusbezeichnung „Catastrophic„. Kurz gesagt eine üble Sache.

 

Welche Plattformen sind betroffen?

Entgegen der erwähnten 2700 Serie sind sämtliche Aironet Accesspoints, ausser der 700er Serie betroffen, welche nach September produziert wurden. Allerdings nur im Zusammenhang mit einer WLC auf Softwarestand 8.0.100.0.

 

Welche Workarounds gibt es?

Workaround:Downgrade to AireOS 7.6.130.0, or to IOS-XE 3.3, if the APs are supported in the earlier code.

Für welche Accesspoints gibt es keinen Workaround?

Unglücklicherweise gibt es für die Aironet 1700 Serie aktuell keinen Workaround, da diese nicht von AireOS 7.6.130.0 bzw. IOS-XE 3.3 supported werden.

Cisco WLC Release Access Point IOS Release Supported Access Points
8.0.100.0 15.3(3)JAB /12.4(25e)JAP Lightweight APs: 1040, 1130, 1140, 1240, 1250, 1260, 1600, 1702, 2600, 2700, 3500e, 3500i, 3500p, 3600e, 3600i, 3600p, 3702e, 3702i, 3702p, 600 OEAP, 700, 700W, AP801, and AP802
Outdoor Mesh APs: 1522, 1524PS, 1524SB, 1532E, 1532I, 1552E, 1552H, 1552I, 1552C, 1552EU, 1552CU, and 1552S
7.6.130.0 15.2(4)JB6/12.4.25e-JAO6 Lightweight APs: 1040, 1130, 1140, 1240, 1250, 1260, 1600, 2600, 2700, 3500e, 3500i, 3500p, 3600e, 3600i, 3600p, 3702e, 3702i, 3702p, 600 OEAP, 700, 700W, AP801, and AP802
Outdoor Mesh APs: 1522, 1524PS, 1524SB, 1532E, 1532I, 1552E, 1552H, 1552I, 1552C, 1552EU, 1552CU, and 1552S

 

Was sagt Cisco TAC?

Hello,

This is Szymon from Cisco TAC Team in Brussels and I am the engineer assigned to your case.

Please find my contact details in signature below along with my working hours.

From what I understand you are hitting following bug CSCur43050 when 1700 AP fails to join to WLC running 8.0 AirOS.

The error messages seen on AP console confirms that the problem is cause by the above defect in 8.0, and unfortunately 1700 AP are not supported under 7.6 release, so you cannot workaround it in this way.

I am going to open internal BU escalation and request them to provide fixed engineering image for you.

I will keep you updated.

Best Regards,

 

Es heisst also abwarten auf 8.0 (100.5) bzw. ein Custome Fix image via TAC.

update folg…

Update 20.11.2014 Cisco TAC:

Hi Samuel,

Currently BU is completing image process and sanity check, so code process
should be done today or tomorrow.
Seems that they should be able to post image to CCO next week if they do
not encounter any issue.
I hope you can wait until next week to get fully tested image.
I let you know as soon as I will have any news about this release.

Best Regards,

 

Update 25.11.2014 Cisco TAC:

Hi Samuel,

I have a good news for you, we have fixed image and I can publish it for you.

Please find attached release notes for code 8.0.100.6 which contains the fix.

Can you please remind what is your WLC platform, so I will provide you

proper image?

Best Regards,

 

 

E voila:

 

 

Samuel Heinrich
Senior Network Engineer at Selution AG (Switzerland)
Arbeitet in Raum Basel (Switzerland) als Senior Network Engineer mit über 15 Jahren Erfahrung im Bereich Netzwerk

Schreibe einen Kommentar

Deine E-Mail-Adresse wird nicht veröffentlicht. Erforderliche Felder sind mit * markiert

Diese Website verwendet Akismet, um Spam zu reduzieren. Erfahre mehr darüber, wie deine Kommentardaten verarbeitet werden.