Cisco WLC DTLS Failed Due to Expired Manufacturer Installed Certificate (MIC)

I was troubleshooting a Cisco WLC configured as FlexConnect (or H-REAP) to a remote AP 1242 that wouldn't join. You can view WLC logs under Management > Logs > Message Logs.

*spamApTask2: Aug 26 13:37:28.876: #DTLS-3-HANDSHAKE_FAILURE: openssl_dtls.c:681 Failed to complete DTLS handshake with peer 192.168.10.14 for AP d4:6d:50:88:12:34

*spamApTask3: Aug 26 13:36:13.824: #DTLS-3-HANDSHAKE_FAILURE: openssl_dtls.c:681 Failed to complete DTLS handshake with peer 192.168.10.14 for AP d4:6d:50:88:12:34

*spamApTask2: Aug 26 13:35:03.936: #DTLS-3-HANDSHAKE_FAILURE: openssl_dtls.c:681 Failed to complete DTLS handshake with peer 192.168.10.14 for AP d4:6d:50:88:12:34

<OUTPUT TRUNCATED>

 

 

(Cisco Controller) >show ap join stats summary all

 

Number of APs.............................................. 1

 

Base Mac             AP EthernetMac       AP Name                 IP Address         Status

00:3a:99:12:ab:cd    58:8d:09:03:12:34   ap01               192.168.10.14     Not Joined

 

Since DTLS need accurate date and time, I tried to manually configure the WLC date/time, timezone under Commands > Set Time and also removed NTP but still no joy.

I also tried to upgrade the WLC AireOS 7.4.121.0 > 7.4.150.0 but still AP won't join the WLC.

My search led me to an expired WLC Manufacturer Installed Certificate (MIC) which has reached its 10 year expiration date. Below are some WLC commands as a workaround.

 

(Cisco Controller) >config ap ?

              

802.1Xuser     Configures the dot1x user for AP.

SSH            Enables/Disables SSH.

add            Adds a Foreign Access Point.

bhrate         Configures Cisco Bridge Backhaul Tx Rate.

bridgegroupname Sets/Deletes bridge group name.

bridging       Enables/Disables Ethernet-to-Ethernet bridging.

cdp            Enable/Disable CDP on Cisco AP.

cert-expiry-ignore Configures cert-expiry-ignore check operation.

core-dump      Configures the AP's memory core dump.

country        Configures the country of operation.

crash-file     Manages crash data and radio core files.

delete         Deletes a Foreign Access Point.

disable        Disables Cisco APs

enable         Enables Cisco APs

ethernet       Configures Ethernet Port of the AP

flexconnect    Enables/Disables VLAN on the flexconnect.

group-name     Configures the group name.

hotspot        Configures Hotspot configs on the AP.

image          Configure image.

led-state      Enables/Disables the LED-State or configure LED flash

link-encryption Capwap Data Link encryption options

 

--More-- or (q)uit

link-latency   Configures Link Latency Feature.

location       Configures the Location.

logging        Configures the AP logging.

mgmtuser       Configures the user for AP management.

mode           Configures an AP's  mode of operation.

monitor-mode   Configures an AP's monitor-mode channel optimization.

name           Configures the Name of an AP.

packet-dump    Configure AP Packet Capture parameters

port           Configures the port for a Foreign Access Point.

power          Configures Cisco Power over Ethernet (PoE) feature for an AP

primary-base   Configures the Primary Cisco Switch for an AP.

priority       Configure the AP's priority.

reporting-period Configures the AP's rogue/error reporting period

reset          Resets an AP

retransmit     Configures AP Control packet retransmission parameters.

role           Configures an AP's  Bridge role of operation.

rst-button     Enables or disables the Reset Button.

secondary-base Configures the Secondary Cisco Switch for an AP.

sniff          Enables/Disables sniffing on a radio.

static-IP      Enables/Disables/Changes an AP's static IP address configuration

stats-mode     Configures the mode(realtime vs normal) in which statistics are sent from an AP

stats-timer    Configures the frequency at which statistics are sent from an AP

syslog         Configures the system logging settings for an AP

 

--More-- or (q)uit

tcp-mss-adjust Configures TCP on Cisco AP.

telnet         Enables/Disables telnet.

tertiary-base  Configures the Tertiary Cisco Switch for an AP.

tftp-downgrade Initiates AP's image downgrade from a TFTP server

              

(Cisco Controller) >config ap cert-expiry-ignore ?

              

mic            Configures cert-expiry-ignore check operation for MIC.

ssc            Configures cert-expiry-ignore check operation for SSC.

              

(Cisco Controller) >config ap cert-expiry-ignore mic ?

              

enable         Enabling will ignore the lifetime-check for MIC.

disable        Disabling will do the lifetime-check for MIC.

              

(Cisco Controller) >config ap cert-expiry-ignore mic enable

 

(Cisco Controller) >config ap cert-expiry-ignore ssc ?

              

enable         Enabling will ignore the lifetime-check for SSC.

disable        Disabling will do the lifetime-check for SSC.

 

(Cisco Controller) >config ap cert-expiry-ignore ssc enable

 

(Cisco Controller) >config auth-list ?

              

add            Creates an authorized AP entry.

ap-policy      Configures an AP authorization policy.

delete         Delete an existing AP entry.

              

(Cisco Controller) >config auth-list ap-policy ?

                         

authorize-ap   Configures AP authorization policies.

authorize-lsc-ap Use auth-list to Authorize APs with Locally Significant Certificate.

ssc            Configures authorization of APs with self-signed certificate.

mic            Configures authorization of APs with manufacturing-installed certificates.

lsc            Configures authorization of APs with locally significant certificates.

              

(Cisco Controller) >config auth-list ap-policy ssc ?

              

enable         Allows APs with self-signed certificates to connect.

disable        Disallows APs with self-signed certificates to connect.

              

(Cisco Controller) >config auth-list ap-policy ssc enable

 

(Cisco Controller) >config certificate ?

              

compatibility  Configure certificate compatibility mode.

generate       Generates new certificates.

lsc            Configure Locally Significant Certificates (LSC)

ssc            Configure Self Signed Certificates (SSC)

use-device-certificate Use device certificate.

              

(Cisco Controller) >config certificate ssc ?

              

hash           Configure Self Signed Certificates hash

              

(Cisco Controller) >config certificate ssc hash ?

              

validation     Configures validation of SSC Hash

                      

(Cisco Controller) >config certificate ssc hash validation ?

 

enable         Enable hash validation of SSC certificate

              

disable        Disable hash validation of SSC certificate

              

(Cisco Controller) >config certificate ssc hash validation enable

 

(Cisco Controller) >save config

 

Are you sure you want to save? (y/n) y

 

Configuration Saved!

 

After the said commands were configured, the AP 1242 joined the WLC again.

(Cisco Controller) >show ap join stats summary all

 

Number of APs.............................................. 1

 

Base Mac             AP EthernetMac       AP Name                 IP Address         Status

00:3a:99:12:ab:cd    58:8d:09:03:12:34   ap01               192.168.10.14     Joined    

 

Manual Radio Channel Assignment in a Cisco WLC

It's rainy season here in Singapore (April 2021) and I was craving for some Japanese food. So I went to Don Don Donki in Orchard Central and they sell a wide range of Japanese products like snacks, sweets, drinks, bento set, sushi, etc.

These are the items I brought home: Dango balls (sweet soy), California maki roll, Takoyaki (octopus) balls, Katsudon pork and sweet potato. They all taste authentic and almost have the same quality from a Japanese restaurant.

I headed over to Menbaka Fire Ramen in Cineleisure Orchard, which is a famous Ramen shop in Kyoto, Japan. They use burning negi (green) onion oil which creates the fire effect when poured into the bowl. This brings out the flavor of the green onions. The restaurant allow diners to seat in front of the kitchen to experience the "theatrical" fire display.

I ordered the Shoyu Fire Ramen and Ocha tea (cold) which is perfect for the rainy weather.

Here's a nice link regarding the Radio Resource Management (RRM) in a Cisco WLC. You can manually assign or override the AP's radio channel in a Cisco WLC under Wireless > Access Points > Radios > select the specific 802.11 radio (in this case 802.11b/g/n) > click the blue arrow icon (far right) on the specific AP (ap03 in this case) > click Configure.

Under RF Channel Assignment > Assignment Method > select Custom > select the specific channel (ap03 current channel is 1 so I chose channel 6) > click Apply > Save Configuration.

 

The ranges are as follows:
802.11a - 36, 40, 44, 48, 52, 56, 60, 64, 100, 104, 108, 112, 116, 132, 136, 140, 149, 153, 157, 161, 165, 190, 196

802.11b/g - 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11

The defaults are as follows:
802.11a - 36, 40, 44, 48, 52, 56, 60, 64, 100, 104, 108, 112, 116, 132, 136, 140, 149, 153, 157, 161 

802.11b/g - 1, 6, 11

It's recommend that you use only non-overlapping channels: i.e. 1, 6, 11, and so on.

 

Cisco WLC 2504 Field Upgrade Software (FUS) 2.0

I needed to upgrade a Cisco WLC 2504 to AireOS version 8.5 and there's a note on the Cisco.com download site that a Field Upgrade Software (FUS) version 1.9 or higher is needed.

Cisco 2500 Series Wireless Controllers Release 8.5 Software.In order to use 8.4 or higher code, you must upgrade the 2504 Wireless Lan Controllers to FUS version 1.9 or higher, this must be done before installing the new AireOS version.

There's a separate link or download area for the WLC FUS firmware.

You can't find the FUS version in the WLC web GUI and the only way to get this info is via the CLI.

You use the WLC show sysinfo command to get the FUS version info. Look under the Bootloader Version (1.0.20) and Firmware Version (PIC 20.0). Since I'm running the latest FUS version on the WLC (as of this writing), I don't need to perform an FUS upgrade. 

Refer to Table 2 on this link for the FUS release notes. This link provides the steps in performing an FUS upgrade.

 

(Cisco Controller) >show sysinfo

 

Manufacturer's Name.............................. Cisco Systems Inc.

Product Name..................................... Cisco Controller

Product Version.................................. 8.2.130.0

Bootloader Version............................... 1.0.20

Field Recovery Image Version..................... 7.6.101.1

Firmware Version................................. PIC 20.0

 

Build Type....................................... DATA + WPS

 

System Name...................................... WLC01

System Location..................................

System Contact...................................

System ObjectID.................................. 1.3.6.1.4.1.9.1.1279

IP Address....................................... 192.168.7.10

IPv6 Address..................................... ::

Last Reset....................................... Software reset

System Up Time................................... 0 days 1 hrs 41 mins 35 secs

System Timezone Location.........................

System Stats Realtime Interval................... 5

System Stats Normal Interval..................... 180

 

Configured Country............................... SG  - Singapore

Operating Environment............................ Commercial (0 to 40 C)

Internal Temp Alarm Limits....................... 0 to 65 C

Internal Temperature............................. +47 C

External Temperature............................. +50 C

Fan Status....................................... 3800 rpm

 

State of 802.11b Network......................... Enabled

State of 802.11a Network......................... Enabled

Number of WLANs.................................. 0

Number of Active Clients......................... 0

 

Burned-in MAC Address............................ DC:EB:94:95:12:34

Maximum number of APs supported.................. 75

System Nas-Id.................................... WLC01

WLC MIC Certificate Types........................ SHA1/SHA2