VPN Monitoring and Troubleshooting in CDO

Monitor Remote Access VPN Sessions

The CDO Remote Access Monitoring dashboard can be used to view consolidated information about RA VPN users, including the current status of users, device types, client applications, user geolocation information, and duration of connections. You can also disconnect RA VPN sessions as needed.

Perform the following to see the VPN sessions:

  1. In the cloud-delivered Firewall Management Center page, click Return Home.

  2. In the CDO navigation pane, click VPN > Remote Access VPN Monitoring.

See Monitor Remote Access Virtual Private Network Sessions for more information.

SD-WAN Summary Dashboard

The SD-WAN Summary dashboard (Analysis > SD-WAN Summary) provides a snapshot of your WAN devices and their interfaces. This dashboard helps you to:

  • Identify issues with the underlay and overlay (VPN) topologies.

  • Troubleshoot VPN issues using the existing Health Monitoring, Device Management, and Site-to-Site Monitoring pages.

  • Monitor application performance metrics of WAN interfaces. The threat defense steers application traffic based on these metrics.

A WAN device must meet one of the following criteria:

  • The device must be a VPN peer.

  • The device must have WAN interface.

A WAN interface must meet one of the following criteria:

  • The interface has IP address-based path monitoring enabled on it.

  • The interface has a Policy Based Routing (PBR) policy with at least one application configured to monitor it.

For more information about PBR policy and path monitoring, see Policy Based Routing.

Click Uplink Decisions to view the VPN Troubleshooting page. You can view syslogs with ID: 880001. These syslogs show the threat defense interfaces through which it steers traffic based on the configured PBR policy.

To view the above syslogs and to view the data on this dashboard, ensure that you review Prerequisites for Using SD-WAN Summary Dashboard.

For clusters, this dashboard displays application performance metrics of only the control node and not the data nodes.

Prerequisites for Using SD-WAN Summary Dashboard

  • You must be an Admin, Security Analyst, or Maintenance user to view this dashboard. See Secure Firewall Management Center and Cloud-delivered Firewall Management Center User Role Mapping for more information.

  • Threat defense devices must be Version 7.2 or later.

  • Enable IP-based path monitoring and HTTP-based application monitoring on the WAN interfaces.

    1. Choose Devices > Device Management.

    2. Click the edit icon adjacent to the device that you want to edit.

    3. Click the edit icon adjacent to the interface that you want to edit.

    4. Click the Path Monitoring tab.

    5. Check the Enable IP based Monitoring check box.

    6. Check the Enable HTTP based Application Monitoring check box.

    7. Click OK.

  • Configure a PBR policy with at least one application configured to monitor it:

    1. Choose Devices > Device Management.

    2. Click the edit icon adjacent to the device that you want to edit.

    3. Click Routing.

    4. In the left pane, click Policy Based Routing.

    5. Click Add.

    6. From the Ingress Interface drop-down list, choose an interface.

    7. Click Add to configure a forwarding action.

    8. Configure the parameters.

    9. Click Save.

  • To view the application performance metrics for the WAN interfaces, you must:

    • Threat defense devices must be Version 7.4.1.

    • Enable data collection from the SD-WAN module in the health policy.

      1. Choose System > Policy.

      2. Click the Edit health policy icon.

      3. In the Health Modules tab, under SD-WAN, click the SD-WAN Monitoring toggle button.

    • Configure applications for the PBR policies.

      1. Choose Objects > Object Management > Access List > Extended.

      2. Click the edit icon adjacent to the access list and add the applications for the PBR policy.

    • Configure the forwarding action for the policy with one of the four application metrics.

      1. Choose Devices > Device Management.

      2. Click the edit icon adjacent to the device that you want to edit.

      3. Click Routing.

      4. In the left pane, click Policy Based Routing.

      5. Click the edit icon adjacent to the policy that you want to edit.

      6. In the Edit Policy Based Route dialog box, click the edit icon adjacent to the corresponding ACL.

      7. In the Edit Forwarding Actions dialog box, from the Interface Ordering drop-down list, choose one of the following options:

        • Minimal Jitter

        • Maximum Mean Opinion Score

        • Minimal Round-Trip Time

        • Minimal Packet Loss

        If you choose Interface Priority or Order, application monitoring is not enabled on the interface.

    • Configure ECMP on the WAN interfaces:

      1. Choose Devices > Device Management.

      2. Click the edit icon adjacent to the device that you want to edit.

      3. Click Routing.

      4. In the left pane, click ECMP.

      5. Click Add and specify a name for the ECMP zone.

      6. Click Add to move interfaces from Available Interfaces to Selected Interfaces.

      7. Click OK.

    • Ensure that traffic passes through the interface.

    • Enable DNS inspection on each WAN device so that the threat defense device can do DNS snooping, and configure the trusted DNS servers:

      1. Choose Devices > Platform Settings.

      2. Click the edit icon adjacent to the threat defense policy that you want to edit.

      3. In the left pane, click DNS.

      4. Click the DNS Settings tab.

      5. Check the Enable DNS name resolution by device check box.

      6. Click the Trusted DNS Servers tab.

      7. Do one of the following:

        • Click the Trust Any DNS server toggle button.

        • Under Specify DNS Servers, click Edit to add trusted DNS servers.

  • To view syslogs when you click Uplink Decisions, you must:

    • Choose Devices > Platform Settings and create or edit a threat defense policy.

    • In the left pane, click Syslog.

    • Click the Logging Setup tab.

    • Check the Enable Logging check box to turn on the data plane system logging for the threat defense device.

    • Click the All Logs radio button to enable logging of all the troubleshooting syslog messages.

      or

      Click the VPN Logs radio button to enable logging of only the VPN troubleshooting messages.

    • Click Save.

Monitor WAN Devices and Interfaces Using the SD-WAN Summary Dashboard

The SD-WAN Summary dashboard has the following widgets under the Overview tab:

WAN Connectivity

This widget provides a summary of the WAN interfaces statuses. It shows the number of WAN interfaces that are in the Online, Offline or No Data states. Note that you cannot monitor subinterfaces using this widget.

Click View All Interfaces to view more details about the interfaces in the health monitor page.

If a WAN interface is in the Offline or No Data state, you can troubleshoot it from the health monitor page:

  1. In the Monitoring pane, expand Devices.

  2. Click the corresponding WAN device to view the device-specific health details.

  3. Click the Interface tab to view the interface status and aggregate traffic statistics for a specific time.

    Alternatively, you can click View System & Troubleshoot Details. The health monitor page is displayed with all the necessary details.

VPN Topology

This widget provides a summary of the site-to-site VPN tunnel statuses. It shows the number of Active, Inactive, and No Active Data VPN tunnels.

Click View All Connections to view the VPN tunnel details in the Site-to-site VPN Monitoring dashboard.

If the tunnels are in the Inactive or No Active Data state, you can troubleshoot using the Site-to-site VPN Monitoring dashboard. In the Tunnel Status widget, hover your cursor over a topology, click the View icon and do one of the following:

  • Click the CLI Details tab to view the details of the VPN tunnels.

  • Click the Packet Tracer tab to use the packet tracer tool for the topology.

Interface Throughput

This widget monitors the throughput utilization of the WAN interfaces.

The interface throughput is classified into four bands. These details aid in cost planning and resourcing. You can choose a time range for the widget data from the Show Last drop-down list. The range is from 15 minutes to two weeks.

Click View Health Monitoring to view more details about the interface in the health monitor page.

Device Inventory

This widget lists all the managed WAN devices and groups them according to the model.

Click View Device Management to view more details about the device in the Device Management page.

WAN Device Health

This widget displays the device count according to the health of the WAN devices. You can view the number of devices with errors, warnings, or those that are in Disabled state.

Click View Health Monitoring to view the alarms, and quickly identify, isolate, and resolve issues.

If the health of a device is affected, you can troubleshoot it from the health monitor page.

  1. In the Monitoring pane, expand Devices.

  2. Click the corresponding WAN device to view the device-specific health details.

  3. Click View System & Troubleshoot Details. The health monitor page is displayed with all the necessary details.

A device can be in Disabled state for multiple reasons, including the following:

  • Management interface is disabled.

  • Device is powered off.

  • Device is being upgraded.

Monitor Application Performance Metrics of WAN Interfaces Using the SD-WAN Summary Dashboard

Under the Application Monitoring tab, you can select a WAN device and view the application performance metrics for the corresponding WAN interfaces. These metrics include Jitter, Round Trip Time (RTT), Mean Opinion Score (MOS), and Packet Loss.

By default, the metrics data is refreshed every 5 minutes. You can change the refresh time; the range is from 5 to 30 minutes. You can view the metrics in tabular and graphical formats. For each WAN interface, the latest metric value appears in the table. For graphical data, you can choose a time interval of up to 24 hours to view the metrics data for the corresponding WAN interfaces.

System Messages

The Message Center is the place to start your troubleshooting. This feature allows you to view messages that are continually generated about system activities and status. To open the Message Center, click System Status, located to the immediate right of the Deploy button in the main menu.

Debug Commands

This section explains how you use debug commands to help you diagnose and resolve VPN-related problems. The commands described here are not exhaustive, this section include commands according to their usefulness in assisting you to diagnose VPN-related problems.

Usage Guidelines

Because debugging output is assigned high priority in the CPU process, it can render the system unusable. For this reason, use debug commands only to troubleshoot specific problems or during troubleshooting sessions with the Cisco Technical Assistance Center (TAC). Moreover, it is best to use debug commands during periods of lower network traffic and fewer users. Debugging during these periods decreases the likelihood that increased debug command processing overhead will affect system use.

You can view debug output in a CLI session only. Output is directly available when connected to the Console port, or when in the diagnostic CLI (enter system support diagnostic-cli ). You can also view output from the regular Firepower Threat Defense CLI using the show console-output command.

To show debugging messages for a given feature, use the debug command. To disable the display of debug messages, use the no form of this command. Use no debug all to turn off all debugging commands.

debug feature [ subfeature] [ level]

no debug feature [ subfeature]

Syntax Description

feature

Specifies the feature for which you want to enable debugging. To see the available features, use the debug ? command for CLI help.

subfeature

(Optional) Depending on the feature, you can enable debug messages for one or more subfeatures. Use ? to see the available subfeatures.

level

(Optional) Specifies the debugging level. Use ? to see the available levels.

Command Default

The default debugging level is 1.

Example

With multiple sessions running on remote access VPN, troubleshooting can be difficult, given the size of the logs. You can use the debug webvpn condition command to set up filters to target your debug process more precisely.

debug webvpn condition {group name | p-ipaddress ip_address [{subnet subnet_mask | prefix length}] | reset | user name}

Where:

  • group name filters on a group policy (not a tunnel group or connection profile).

  • p-ipaddress ip_address [{subnet subnet_mask | prefix length}] filters on the public IP address of the client. The subnet mask (for IPv4) or prefix (for IPv6) is optional.

  • reset resets all filters. You can use the no debug webvpn condition command to turn off a specific filter.

  • user name filters by username.

If you configure more than one condition, the conditions are conjoined (ANDed), so that debugs appear only if all conditions are met.

After setting up the condition filter, use the base debug webvpn command to turn on the debug. Setting the conditions alone does not enable the debug. Use the show debug and show webvpn debug-condition commands to view the current state of debugging.

The following shows an example of enabling a conditional debug on the user jdoe.


firepower# debug webvpn condition user jdoe

firepower# show webvpn debug-condition
INFO: Webvpn conditional debug is turned ON
INFO: User name filters:
INFO: jdoe

firepower# debug webvpn
INFO: debug webvpn  enabled at level 1.

firepower# show debug
debug webvpn  enabled at level 1
INFO: Webvpn conditional debug is turned ON
INFO: User name filters:
INFO: jdoe


debug aaa

See the following commands for debugging configurations or authentication, authorization, and accounting (AAA) settings.

debug aaa [ accounting | authentication | authorization | common | internal | shim | url-redirect]

Syntax Description

aaa

Enables debugging for AAA. Use ? to see the available subfeatures.

accounting

(Optional) Enables AAA accounting debugging.

authentication

(Optional) Enables AAA authentication debugging.

authorization

(Optional) Enables AAA authorization debugging.

common

(Optional) Specifies the AAA common debug level. Use ? to see the available levels.

internal

(Optional) Enables AAA internal debugging.

shim

(Optional) Specifies the AAA shim debug level. Use ? to see the available levels.

url-redirect

(Optional) Enables AAA url-redirect debugging.

Command Default

The default debugging level is 1.

debug crypto

See the following commands for debugging configurations or settings associated with crypto.

debug crypto [ ca | condition | engine | ike-common | ikev1 | ikev2 | ipsec | ss-apic]

Syntax Description

crypto

Enables debugging for crypto . Use ? to see the available subfeatures.

ca

(Optional) Specifies the PKI debug levels. Use ? to see the available subfeatures.

condition

(Optional) Specifies the IPsec/ISAKMP debug filters. Use ? to see the available filters.

engine

(Optional) Specifies the crypto engine debug levels. Use ? to see the available levels.

ike-common

(Optional) Specifies the IKE common debug levels. Use ? to see the available levels.

ikev1

(Optional) Specifies the IKE version 1 debug levels. Use ? to see the available levels.

ikev2

(Optional) Specifies the IKE version 2 debug levels. Use ? to see the available levels.

ipsec

(Optional) Specifies the IPsec debug levels. Use ? to see the available levels.

condition

(Optional) Specifies the Crypto Secure Socket API debug levels. Use ? to see the available levels.

vpnclient

(Optional) Specifies the EasyVPN client debug levels. Use ? to see the available levels.

Command Default

The default debugging level is 1.

debug crypto ca

See the following commands for debugging configurations or settings associated with crypto ca.

debug crypto ca [ cluster | messages | periodic-authentication | scep-proxy | transactions | trustpool] [ 1-255]

Syntax Description

crypto ca

Enables debugging for crypto ca . Use ? to see the available subfeatures.

cluster

(Optional) Specifies the PKI cluster debug level. Use ? to see the available levels.

cmp

(Optional) Specifies the CMP transactions debug level. Use ? to see the available levels.

messages

(Optional) Specifies the PKI Input/Output message debug level. Use ? to see the available levels.

periodic-authentication

(Optional) Specifies the PKI periodic-authentication debug level. Use ? to see the available levels.

scep-proxy

(Optional) Specifies the SCEP proxy debug level. Use ? to see the available levels.

server

(Optional) Specifies the local CA server debug level. Use ? to see the available levels.

transactions

(Optional) Specifies the PKI transaction debug level. Use ? to see the available levels.

trustpool

(Optional) Specifies the trustpool debug level. Use ? to see the available levels.

1-255

(Optional) Specifies the debugging level.

Command Default

The default debugging level is 1.

debug crypto ikev1

See the following commands for debugging configurations or settings associated with Internet Key Exchange version 1 (IKEv1).

debug crypto ikev1 [ timers] [ 1-255]

Syntax Description

ikev1

Enables debugging for ikev1 . Use ? to see the available subfeatures.

timers

(Optional) Enables debugging for IKEv1 timers.

1-255

(Optional) Specifies the debugging level.

Command Default

The default debugging level is 1.

debug crypto ikev2

See the following commands for debugging configurations or settings associated with Internet Key Exchange version 2 (IKEv2).

debug crypto ikev2 [ ha | platform | protocol | timers]

Syntax Description

ikev2

Enables debugging ikev2 . Use ? to see the available subfeatures.

ha

(Optional) Specifies the IKEv2 HA debug level. Use ? to see the available levels.

platform

(Optional) Specifies the IKEv2 platform debug level. Use ? to see the available levels.

protocol

(Optional) Specifies the IKEv2 protocol debug level. Use ? to see the available levels.

timers

(Optional) Enables debugging for IKEv2 timers.

Command Default

The default debugging level is 1.

debug crypto ipsec

See the following commands for debugging configurations or settings associated with IPsec.

debug crypto ipsec [ 1-255]

Syntax Description

ipsec

Enables debugging for ipsec . Use ? to see the available subfeatures.

1-255

(Optional) Specifies the debugging level.

Command Default

The default debugging level is 1.

debug ldap

See the following commands for debugging configurations or settings associated with LDAP (Lightweight Directory Access Protocol).

debug ldap [ 1-255]

Syntax Description

ldap

Enables debugging for LDAP. Use ? to see the available subfeatures.

1-255

(Optional) Specifies the debugging level.

Command Default

The default debugging level is 1.

debug ssl

See the following commands for debugging configurations or settings associated with SSL sessions.

debug ssl [ cipher | device] [ 1-255]

Syntax Description

ssl

Enables debugging for SSL. Use ? to see the available subfeatures.

cipher

(Optional) Specifies the SSL cipher debug level. Use ? to see the available levels.

device

(Optional) Specifies the SSL device debug level. Use ? to see the available levels.

1-255

(Optional) Specifies the debugging level.

Command Default

The default debugging level is 1.

debug webvpn

See the following commands for debugging configurations or settings associated with WebVPN.

debug webvpn [ anyconnect | chunk | cifs | citrix | compression | condition | cstp-auth | customization | failover | html | javascript | kcd | listener | mus | nfs | request | response | saml | session | task | transformation | url | util | xml]

Syntax Description

webvpn

Enables debugging for WebVPN. Use ? to see the available subfeatures.

anyconnect

(Optional) Specifies the WebVPN Secure Client debug level. Use ? to see the available levels.

chunk

(Optional) Specifies the WebVPN chunk debug level. Use ? to see the available levels.

cifs

(Optional) Specifies the WebVPN CIFS debug level. Use ? to see the available levels.

citrix

(Optional) Specifies the WebVPN Citrix debug level. Use ? to see the available levels.

compression

(Optional) Specifies the WebVPN compression debug level. Use ? to see the available levels.

condition

(Optional) Specifies the WebVPN filter conditions debug level. Use ? to see the available levels.

cstp-auth

(Optional) Specifies the WebVPN CSTP authentication debug level. Use ? to see the available levels.

customization

(Optional) Specifies the WebVPN customization debug level. Use ? to see the available levels.

failover

(Optional) Specifies the WebVPN failover debug level. Use ? to see the available levels.

html

(Optional) Specifies the WebVPN HTML debug level. Use ? to see the available levels.

javascript

(Optional) Specifies the WebVPN Javascript debug level. Use ? to see the available levels.

kcd

(Optional) Specifies the WebVPN KCD debug level. Use ? to see the available levels.

listener

(Optional) Specifies the WebVPN listener debug level. Use ? to see the available levels.

mus

(Optional) Specifies the WebVPN MUS debug level. Use ? to see the available levels.

nfs

(Optional) Specifies the WebVPN NFS debug level. Use ? to see the available levels.

request

(Optional) Specifies the WebVPN request debug level. Use ? to see the available levels.

response

(Optional) Specifies the WebVPN response debug level. Use ? to see the available levels.

saml

(Optional) Specifies the WebVPN SAML debug level. Use ? to see the available levels.

session

(Optional) Specifies the WebVPN session debug level. Use ? to see the available levels.

task

(Optional) Specifies the WebVPN task debug level. Use ? to see the available levels.

transformation

(Optional) Specifies the WebVPN transformation debug level. Use ? to see the available levels.

url

(Optional) Specifies the WebVPN URL debug level. Use ? to see the available levels.

util

(Optional) Specifies the WebVPN utility debug level. Use ? to see the available levels.

xml

(Optional) Specifies the WebVPN XML debug level. Use ? to see the available levels.

Command Default

The default debugging level is 1.