Field Notice: FN74004 - Firepower Software: Some Versions Of Vulnerability Database (VDB) Might Cause Excessive Memory Consumption And Traffic Loss For Some Secure Firewall Devices - Software Upgrade Recommended
The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.
THIS FIELD NOTICE IS PROVIDED ON AN "AS IS" BASIS AND DOES NOT IMPLY ANY KIND OF GUARANTEE OR WARRANTY, INCLUDING THE WARRANTY OF MERCHANTABILITY. YOUR USE OF THE INFORMATION ON THE FIELD NOTICE OR MATERIALS LINKED FROM THE FIELD NOTICE IS AT YOUR OWN RISK. CISCO RESERVES THE RIGHT TO CHANGE OR UPDATE THIS FIELD NOTICE AT ANY TIME.
VDB 361 and 362 fail to install on SW versions less than 6.3
Some versions of Vulnerability Database (VDB) releases might cause excessive memory consumption and traffic loss for some Secure Firewall devices.
Cisco Firepower Threat Defense (FTD) Software uses the VDB updates to provide protection against known vulnerabilities to which hosts might be susceptible, as well as fingerprints for operating systems, clients, and applications. Customers are encouraged to configure the scheduling of automatic VDB updates to maximize protection against any new attack patterns.
For some models of Cisco Secure Firewall devices that are running Snort2 network intrusion detection software, the system memory may be insufficient to support the number of different fingerprints that are introduced in VDB versions 360 and later. Cisco Secure Firewall devices that are affected by this issue includes all models of the following:
Starting with VDB version 363, three conditional checks were added to the VDB package to prevent potential memory issues and traffic loss for affected Cisco Secure Firewall devices. Installation of the VDB update will intentionally fail if all conditional checks are met. This will generate a generic failure message in the Cisco Firepower Management Center (MC) UI that requires a review of the log files to confirm the failure reason.
The three conditional checks that are performed for VDB version 363 and later are as follows:
Note: VDB versions 363 and later contained a reduced fingerprint set for the Cisco Secure Firewall devices that are affected by this issue.
The reduced fingerprint set prevents Snort from consuming excess system memory for affected Cisco Secure Firewall devices. Fingerprints have been removed for applications that have more than one fingerprint. Fingerprints that trigger on behavior that is least likely to be seen by the application have also been removed. The effect on general security efficacy is low, but in some limited situations it might result in the incorrect access control rule being used for some connections.
The reduced fingerprint set affects only the access control, QoS, and SSL policies because these are the only policies that can control traffic based on application. The change does not affect the Snort Intrusion Prevention System (IPS) system.
Symptom 1: D State
To determine if the Cisco Secure Firewall device is affected by the excessive memory consumption due to a VDB update, enter /var/log/top.log.
The Snort processes will show a D state, as highlighted in the example:
top - 2022-11-05 00:01:23 up 7 days, 20:10, 0 users, load average: 3.21, 2.26,
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 3596 sfsnort 1 -19 2062m 759m 508 D 23 22.2 1108:07 snort 3594 sfsnort 1 -19 2062m 764m 360 D 23 22.3 2578:38 snort 3595 sfsnort 1 -19 2057m 754m 1248 D 22 22.0 834:58.61 snort
There might be a loss of network traffic, either entirely or partially, during this condition. The device might automatically recover from the Snort D state or a Snort restart might be required. In some cases, Snort may not reply to the restart command and a reboot may be required to fix the issue.
Symptom 2: VDB Update Failed Installation
To determine whether determine the VDB update failed installation, complete the following steps to check the log files:
SSH to the Cisco Secure Firewall device and enter expert mode.
Enter the command sudo su to elevate to root.
Run one of the following commands based on the management platform. Make sure that the VDB version in the log path (shown as vdb* below) matches the VDB version that you are trying to install. For example, for the log path /var/log/sf/vdb-4.5.0-363/pre/005_check_low_end.pl.log, the path vdb-4.5.0-363 is for VDB version is 363.
For devices managed by Cisco FMC or devices running Cisco ASA with FirePOWER Services, enter grep -H 'VDB install cancelled' /var/log/sf/vdb*/pre/005_check_low_end.pl.log
For devices managed by Cisco FDM, enter grep -H 'VDB install cancelled' /ngfw/var/log/sf/vdb*/pre/005_check_low_end.pl.log
The output of the command will show the following:
VDB install cancelled: insufficient device memory. At least one of your managed devices or for device manager, this device cannot install the full VDB. Before you install VDB 363+, upgrade the management center or device manager. This allows you to install a smaller VDB package on lower memory devices. For more information, see the VDB release notes:'' at pre/005_check_low_end.pl line 64.
Rollback to VDB version 362 to prevent memory issues for affected Cisco Secure Firewall devices. This VDB contains reduced fingerprints and is recommended only for the devices that are affected by this issue.
Cisco recommends upgrading to one of the Cisco Firepower manager software releases shown in the following table. The fixed releases will also prevent the warning message for the installation of VDB versions 363 and later for affected devices,
To avoid installation failure, remove all VDB databases that are version 363 and later before attempting to upgrade system software. For more information, see Cisco Bug ID CSCwf21682.
If the Cisco Secure Firewall device is managed using Cisco FDM or Cisco ASA with FirePOWER services and a HotFix is required, contact Cisco TAC for instructions on how to obtain the software (HotFix) to fix the issue for affected Cisco Secure Firewall devices.
For More Information
For further assistance or for more information about this field notice, contact the Cisco Technical Assistance Center (TAC) using one of the following methods:
Receive Email Notification About New Field Notices
To receive email updates about Field Notices (reliability and safety issues), Security Advisories (network security issues), and end-of-life announcements for specific Cisco products, set up a profile in My Notifications