THIS FIELD NOTICE IS PROVIDED ON AN "AS IS" BASIS AND DOES NOT IMPLY ANY KIND OF GUARANTEE OR WARRANTY, INCLUDING THE WARRANTY OF MERCHANTABILITY. YOUR USE OF THE INFORMATION ON THE FIELD NOTICE OR MATERIALS LINKED FROM THE FIELD NOTICE IS AT YOUR OWN RISK. CISCO RESERVES THE RIGHT TO CHANGE OR UPDATE THIS FIELD NOTICE AT ANY TIME.
Revision | Publish Date | Comments |
---|---|---|
1.0 |
12-Jan-17 |
Initial Release |
10.0 |
14-Nov-17 |
Migration to new field notice system |
10.1 |
04-Jun-18 |
Fixed Broken Image Links |
Affected Product ID | Comments |
---|---|
CRS-FP-X |
|
CRS-FP-X-L |
|
CRS-FP200G |
|
CRS-FP200G-L |
|
CRS-FP200G= |
|
CRS-FP400G |
|
CRS-LSP-X |
|
CRS-LSP400G |
|
CRS-MSC-X |
|
CRS-MSC-X-L |
|
CRS-MSC200G= |
|
CRS-MSC400G |
Defect ID | Headline |
---|---|
CSCvf34445 | There were no defects filed with this field notice at the time of publication. |
CRS-X line cards (CRS-MSC-X, CRS-MSC400G, CRS-FP-X, CRS-FP400G=, CRS-FP400G, CRS-LSP-X, CRS-LSP400G, CRS-MSC-X-L, CRS-MSC200G=, CRS-FP-X-L, CRS-FP200G, CRS-FP200G-L, and CRS-FP200G=) built with new DRAM memory fail to boot to the XR-Run state in XR releases of 5.3.3, 5.1.4, 5.1.3, 5.3.1, 5.3.2, 5.1.1, 5.1.2 and 5.3.4.
In the manufacture of line cards CRS-MSC-X, CRS-MSC400G, CRS-FP-X, CRS-FP400G=, CRS-FP400G, CRS-LSP-X, CRS-LSP400G, CRS-MSC-X-L, CRS-MSC200G=, CRS-FP-X-L, CRS-FP200G, CRS-FP200G-L, and CRS-FP200G=, it is required to use a new DRAM memory since the existing one is End of Life.
It was found that this new DRAM memory on the board fails to boot up due to Network Processor Unit (NPU) driver initialization errors. The unit requires a reload Software Maintenance Upgrade (SMU) in order to make the boards work properly.
The line card fails to boot to the XR-Run state. The unit goes for continuous reload and reaches the IN-Reset state.
The sample logs as the signature of this issue on XR 533 release are:
Reboot reason: 'NPU initialization failed'
The sample syslogs for issue symptoms are:
LC/0/2/CPU0:Mar 10 06:33:35.070 : npu_driver[271]: %L2-NPU-2-DRAM_INIT_FAILURE : NPU_INST 0: DRAM instance 10 initialization failed for memory type PBS at the phase control init
LC/0/2/CPU0:Mar 10 06:33:46.417 : npu_driver[271]: %L2-NPU-2-DRAM_INIT_FAILURE : NPU_INST 0: DRAM instance 10 initialization failed for memory type PBS at the phase final
LC/0/2/CPU0:Mar 10 06:33:51.144 : npu_driver[271]: %L2-NPU-3-ERROR_MSG : *** NPU0: FATAL error at {line:831}: failure in GIF block, rc=-1463657962, 'NPU Driver' detected the 'fatal' condition 'at hw lib layer: Internal object': Invalid argument
LC/0/2/CPU0:Mar 10 06:33:51.145 : npu_driver[271]: %L2-NPU-3-INIT_ERROR_MSG : NPU0: Init Failed. Caused by:'NPU Driver' detected the 'fatal' condition 'at hw lib layer: Internal object': Invalid argument
LC/0/2/CPU0:Mar 10 06:33:51.145 : npu_driver[271]: %L2-NPU-3-ERR_EXIT : Exit on error: Failed asic power-on reset init: Caused by 'NPU Driver' detected the 'fatal' condition 'at hw lib layer: Internal object': Invalid argument : pkg/bin/npu_driver : (PID=106555) : -Traceback= 420956c 420a14c 421071a df6996e cb84050
LC/0/2/CPU0:Mar 10 06:33:51.979 : npu_driver[271]: %L2-NPU-6-INFO_MSG : NPU0: *** Total Valid NPUs:2 (max=2) ***
LC/0/2/CPU0:Mar 10 06:33:58.339 : sysmgr[82]: %OS-SYSMGR-2-MANAGED_REBOOT : reboot to be managed by process (platform_mgr_common) reason (ASIC pat instance 0 in critical alarm)
LC/0/2/CPU0:Mar 10 06:33:58.522 : platform_mgr_common[286]: %PLATFORM-HFR_PM-3-COMMUNICATION_ERR : Device pse #0 has a fault=CRITICAL. action: Rebooting node
If the listed line cards show symptoms of not booting up due to NPU driver initialization errors, then install the recommended and mandatory reload SMU which initializes the register settings properly for the new DRAM memory module during initial boot up of the line card.
Release 5.1.1
If you run release 5.1.1, upgrade with this SMU:
Description: Topaz LC with Atris 30nm memory failed to boot to XR state
Release: 5.1.1
Release Date: 04/25/16
File Name: hfr-px-5.1.1.CSCuy71498.pie
Release 5.1.2
If you run release 5.1.2, upgrade with this SMU:
Description: NCS6K/CRS-X LC with Atris 30nm memory failed to boot to XR-Run state
Release:5.1.2
Release Date: 04/25/16
File Name: hfr-px-5.1.2.CSCuy71498.pie
Release 5.1.3
If you run release 5.1.3, upgrade with this SMU:
Description: NCS6K/CRS-X LC with Atris 30nm memory failed to boot to XR-Run state
Release:5.1.3
Release Date: 04/14/16
File Name: hfr-px-5.1.3.CSCuy71498.pie
Release 5.1.4
If you run release 5.1.4, upgrade with this SMU:
Description: NCS6K/CRS-X LC with Atris 30nm memory failed to boot to XR-Run state
Release:5.1.4
Release Date: 04/16/16
File Name: hfr-px-5.1.4.CSCuy71498.pie
Release 5.3.1
If you run release 5.3.1, upgrade with this SMU:
Description: Topaz LC with Atris 30nm memory failed to boot to XR state
Release:5.3.1
Release Date: 04/25/16
File Name: hfr-px-5.3.1.CSCuy71498.pie
Release 5.3.2
If you run release 5.3.2, upgrade with this SMU:
Description: NCS6K/CRS-X LC with Atris 30nm memory failed to boot to XR-Run state
Release:5.3.2
Release Date: 04/25/16
File Name: hfr-px-5.3.2.CSCuy71498.pie
Release 5.3.3
If you run release 5.3.3, upgrade with this SMU:
Description: NCS6K/CRS-X LC with Atris 30nm memory failed to boot to XR-Run state
Release:5.3.3
Release Date: 4/10/2016
File Name: hfr-px-5.3.3.CSCuy71498.pie
Release 5.3.4
If you run release 5.3.4, upgrade with this SMU:
Description: NCS6K/CRS-X LC with Atris 30nm memory failed to boot to XR-Run state
Release:5.3.4
Release Date: 12/23/2016
File Name: hfr-px-5.3.4.CSCuy71498.pie
Complete one of these steps in order to determine if your product is the updated version, Version 03:
For CRS-X line card Product IDs (PIDs), mentioned previously, the new Version ID (VID) displayed is V03.
An example of the command output is shown here:
RP/0/RP0/CPU0:rtr(admin)#sh diag 0/1/CPU0 CARD 0/1/* : Cisco CRS Series Modular Services Card 400G MAIN: board type 50006c 800-38514-05 rev A0 dev 139822 S/N CAT1744T0A5 PCA: 73-14750-09 rev A0 PID: CRS-MSC-X VID: V03
For any assistance please contact Technical Assistance Center.
If you require further assistance, or if you have any further questions regarding this field notice, please contact the Cisco Systems Technical Assistance Center (TAC) by one of the following methods:
Cisco Notification Service—Set up a profile to receive email updates about reliability, safety, network security, and end-of-sale issues for the Cisco products you specify.