Cisco Crosswork Data Gateway supports CLI-based data collection from the network devices. Only show commands are supported for this type of collection job.

Note

The initial status for all the collection jobs in the UI is Unknown. Upon receiving a CLI collection job, Cisco Crosswork Data Gateway performs basic validations on it. If the collection job is valid, its status changes to Successful, else it changes to Failed. Device should not have any banner configuration for CLI collection to work properly. Please refer to device documentation on how to turn this off. The value of Cadence is in seconds. It should be set either to 0 to indicate the sensor configured to be collected only once. OR It should be >= 60 (i.e. at least 1 minute) up to 2764800 seconds ( i.e. at most 32 days) max, indicating how frequently configured sensor data should be collected. When collection from a device is skipped due to previous execution still in progress, Cisco Crosswork Data Gateway raises a warning log. No alert is generated for this scenario.

Following is a sample of CLI collection job. For more information, see API documentation on Cisco DevNet.

{
  "collection_job": {
    "application_context": {
      "context_id": "collection-job1",
      "application_id": "APP1"
    },
    "collection_mode": {
      "lifetime_type": "APPLICATION_MANAGED",
      "collector_type": "CLI_COLLECTOR"
    },
    "job_device_set": {
      "device_set": {
        "devices": {
          "device_ids": [
            "658adb03-cc61-448d-972f-4fcec32cbfe8"
          ]
        }
      }
    },
    "sensor_input_configs": [
      {
        "sensor_data": {
          "cli_sensor": {
            "command": "show platform"
          }
        },
        "cadence_in_millisec": "tel:60000"
      }
    ],
    "sensor_output_configs": [
     {
        "sensor_data": {
          "cli_sensor": {
            "command": "show platform"
          }
        },
        "destination": {
          "destination_id": "1e71f2fb-ea65-4242-8efa-e33cec71b369",
          "context_id": "topic1"
        }
      }
    ]
  }
}

Cisco Crosswork Data Gateway supports SNMP-based data collection based on the OIDs supported on the devices.

The SNMP collector makes a poll request to Cisco Crosswork to get its configuration profile (a list of MIB objects to collect and a list of devices to fetch from). It determines the corresponding OIDs by looking up the pre-packaged list of MIB modules or the custom list of MIB modules.

Note

Cisco Crosswork Data Gateway enables SNMP polling on third party devices for standard MIBs already included in the system. Proprietary MIBs are required only if the collection request references MIB TABLE names or SCALAR names from a proprietary MIB. However, if the requests are OID-based, then MIBs are not required.

Once the OIDs are resolved, they are provided as input to the SNMP collectors.

The device packages can be imported into the Crosswork Data Gateway VM as described in Section Add a Custom Software Package.

The following SNMP versions are supported:

• SNMPv1

• SNMPv2c

• SNMPv3

The table below lists supported privacy protocols and the value that needs to be given in the collection payload for SNMP and SNMP Trap collection jobs:

Note

The initial status for all the collection jobs in the UI is Unknown. Upon receiving a SNMP collection job, Cisco Crosswork Data Gateway performs basic validations on it. If the collection job is valid, its status changes to Successful, else it changes to Failed. The value of Cadence is in seconds. It should be set either to 0 to indicate the sensor configured to be collected only once. OR It should be >= 60 (i.e. at least 1 minute) up to 2764800 seconds ( i.e. at most 32 days) max, indicating how frequently configured sensor data should be collected. When collection from a device is skipped due to previous execution still in progress, Cisco Crosswork Data Gateway raises a warning log. No alert is generated for this scenario. For SNMP v1/v2c, if the device details (such as host or community string) are incorrect in the payload, Cisco Crosswork Data Gateway ignores the traps received from the device and logs a WARN message. Only SNMPv1 and SNMPv2c versions are supported for SNMP traps In case of SNMP v3, if the device details (such as auth, priv, and security name details) are incorrect in the payload, Cisco Crosswork Data Gateway filters it out and hence, does not receive the trap. Thus, no WARN message is logged.

Sample Configurations on Device:

The SNMP Collector supports the following operations:

• SCALAR

• TABLE

• MIB_WALK

• TRAP

• DEVICE_PACKAGE

These operations are defined in the sensor config (see payload sample below).

Note

There is an optional deviceParams attribute snmpRequestTimeoutMillis (not shown in the sample payloads) that should be used if the device response time is very high. It’s not recommended to use snmpRequestTimeoutMillis unless you are absolutely certain that your device response time is very high. The value for snmpRequestTimeoutMillis should be specified in milliseconds: Default value is 1500 milliseconds Minimum value is 1500 milliseconds However, there is no limitation on the maximum value of this attribute.

Following is an SNMP collection job sample:

{
  "collection_job": {
    "application_context": {
      "context_id": "collection-job1",
      "application_id": "APP1"
    },
    "collection_mode": {
      "lifetime_type": "APPLICATION_MANAGED",
      "collector_type": "SNMP_COLLECTOR"
    },
    "job_device_set": {
      "device_set": {
        "devices": {
          "device_ids": [
            "c70fc034-0cbd-443f-ad3d-a30d4319f937",
            "8627c130-9127-4ed7-ace5-93d3b4321d5e",
            "c0067069-c8f6-4183-9e67-1f2e9bf56f58"
          ]
        }
      }
    },
    "sensor_input_configs": [
      {
        "sensor_data": {
          "snmp_sensor": {
            "snmp_mib": {
              "oid": "1.3.6.1.2.1.1.3.0",
              "snmp_operation": "SCALAR"
            }
          }
        },
        "cadence_in_millisec": "60000"
      },
      {
        "sensor_data": {
          "snmp_sensor": {
            "snmp_mib": {
              "oid": "1.3.6.1.2.1.31.1.1",
              "snmp_operation": "TABLE"
            }
          }
        },
        "cadence_in_millisec": "60000"
      }
    ],
    "sensor_output_configs": [
      {
        "sensor_data": {
          "snmp_sensor": {
            "snmp_mib": {
              "oid": "1.3.6.1.2.1.1.3.0",
              "snmp_operation": "SCALAR"
            }
          }
        },
        "destination": {
          "destination_id": "4c2ab662-2670-4b3c-b7d3-b94acba98c56",
          "context_id": "topic1_461cb8aa-a16a-44b8-b79f-c3daf3ea925f"
        }
      },
      {
        "sensor_data": {
          "snmp_sensor": {
            "snmp_mib": {
              "oid": "1.3.6.1.2.1.31.1.1",
              "snmp_operation": "TABLE"
            }
          }
        },
        "destination": {
          "destination_id": "4c2ab662-2670-4b3c-b7d3-b94acba98c56",
          "context_id": "topic2_e7ed6300-fc8c-47ee-8445-70e543057f8a"
        }
      }
    ]
  }
}

SNMP Traps Collection Job

SNMP traps are handled in a similar manner. Trap listeners listen on a port and then dispatch data to recipients (based on their topic of interest).

Cisco Crosswork Data Gateway supports three types of non-yang/OID based traps:

sensor path	purpose
*	To get all the traps pushed from the device without any filter.
MIB level traps	OID of one MIB notifications (Ex: 1.3.6.1.2.1.138.0 to get all the isis-mib level traps)
Specific trap	OID of the specific trap (Ex: 1.3.6.1.6.3.1.1.5.4 to get the linkUp trap)

Note

Device should have been pre-configured by the traps. Cisco Crosswork Data Gateway listens on UDP port 1062 for Traps. If the collection job is invalid, there is missing configuration on the device, or no trap is received, the status of the job remains "Unknown". For list of supported Traps and MIBs, see List of Pre-loaded Traps and MIBs for SNMP Collection

On receiving a trap, Cisco Crosswork Data Gateway does the following validations:

1. Check if any collection job is created for the device.

2. Checks the trap version and community string.

3. For SNMP v3, validates for user auth and priv protocol and credentials.

Cisco Crosswork Data Gateway filters the traps based on the trap OID mentioned in the sensor path and sends only those requested.

Cisco Crosswork Data Gateway supports the following YANG paths:

sensor path	purpose
snmp-trap-raw-oper:traps/data	To get all the traps pushed from the device without any filter.
IF-MIB:notifications	To get all the IF-MIB notifications (ex: linkUp, linkDown, etc.)
ISIS-MIB:notifications	To get all the ISIS-MIB notifications.
SNMPv2-MIB:notifications	To get all the snmpV2 Mib notifications.

Following is an SNMP-Trap collection job sample:

{
  "collection_job": {
    "application_context": {
      "context_id": "collection-job1",
      "application_id": "APP1"
    },
    "collection_mode": {
      "lifetime_type": "APPLICATION_MANAGED",
      "collector_type": "TRAP_COLLECTOR"
    },
    "job_device_set": {
      "device_set": {
        "devices": {
          "device_ids": [
            "a9b8f43d-130b-4866-a26a-4d0f9e07562a",
            "8c4431a0-f21d-452d-95a8-84323a19e0d6",
            "eaab2647-2351-40ae-bf94-6e4a3d79af3a"
          ]
        }
      }
    },
    "sensor_input_configs": [
      {
        "sensor_data": {
          "trap_sensor": {
            "path": "1.3.6.1.6.3.1.1.4"
          }
        },
        "cadence_in_millisec": "60000"
      }
    ],
    "sensor_output_configs": [
      {
        "sensor_data": {
          "trap_sensor": {
            "path": "1.3.6.1.6.3.1.1.4"
          }
        },
        "destination": {
          "destination_id": "4c2ab662-2670-4b3c-b7d3-b94acba98c56",
          "context_id": "topic1_696600ae-80ee-4a02-96cb-3a01a2415324"
        }
      }
    ]
  }
}

Enabling Traps forwarding to external applications

As per the current implementation, in case of an SNMP Trap collection job, all traps are sent to the specified data destination even if the SNMP Trap OID is not provided in the sensor path.

Note	It is also recommended to selectively enable on the device only those traps that are needed by Crosswork.

To identify the type of trap from the data received on the destination, look for oid (OBJECT_IDENTIFIER, for example, 1.3.6.1.6.3.1.1.4.1.0 ) and strValue associated to the oid in the OidRecords (application can match the OID of interest to determine the kind of trap).

Below are some sample values and a sample payload:

• Link up

  1.3.6.1.6.3.1.1.4.1.0 = 1.3.6.1.6.3.1.1.5.4

• Link Down

  1.3.6.1.6.3.1.1.4.1.0 = 1.3.6.1.6.3.1.1.5.3

• Syslog

  1.3.6.1.6.3.1.1.4.1.0 = 1.3.6.1.4.1.9.9.41.2.0.1

• Cold Start

  1.3.6.1.6.3.1.1.4.1.0 = 1.3.6.1.6.3.1.1.5.1

{
  "nodeIdStr": "BF5-XRV9K1.tr3.es",
  "nodeIdUuid": "C9tZ5lJoSJKf5OZ67+U5JQ==",
  "collectionId": "133",
  "collectionStartTime": "1580931985267",
  "msgTimestamp": "1580931985267",
  "dataGpbkv": [
    {
      "timestamp": "1580931985267",
      "name": "trapsensor.path",
      "snmpTrap": {
        "version": "V2c",
        "pduType": "TRAP",
        "v2v3Data": {
          "agentAddress": "172.70.39.227",
          "oidRecords": [
            {
              "oid": "1.3.6.1.2.1.1.3.0",
              "strValue": "7 days, 2:15:17.02"
            },
            {
              "oid": "1.3.6.1.6.3.1.1.4.1.0",  // This oid is the Object Identifier.
              "strValue": "1.3.6.1.6.3.1.1.5.3" // This is the value that determines the kind of trap.
            },
            {
              "oid": "1.3.6.1.2.1.2.2.1.1.8",
              "strValue": "8"
            },
            {
              "oid": "1.3.6.1.2.1.2.2.1.2.8",
              "strValue": "GigabitEthernet0/0/0/2"
            },
            {
              "oid": "1.3.6.1.2.1.2.2.1.3.8",
              "strValue": "6"
            },
            {
              "oid": "1.3.6.1.4.1.9.9.276.1.1.2.1.3.8",
              "strValue": "down"
            }
          ]
        }
      }
    }
  ],
  "collectionEndTime": "1580931985267",
  "collectorUuid": "YmNjZjEzMTktZjFlOS00NTE5LWI4OTgtY2Y1ZmQxZDFjNWExOlRSQVBfQ09MTEVDVE9S",
  "status": {
    "status": "SUCCESS"
  },
  "modelData": {},
  "sensorData": {
    "trapSensor": {
      "path": "1.3.6.1.6.3.1.1.5.4"
    }
  },
  "applicationContexts": [
    {
      "applicationId": "APP1",
      "contextId": "collection-job-snmp-traps"
    }
  ]
}

Crosswork Data Gateway supports data collection from network devices using Model-driven Telemetry (MDT) to consume telemetry streams directly from devices (for IOS-XR based platforms only).

Note

MDT collector retains the collection ID that comes as part of the telemetry proto for the device. This behavior is different from CLI and SNMP collectors which compute the collection ID based on the sequence number of the collection. MDT collection jobs require some configuration to be done on the device. This configuration is automatically taken care of by NSO. Ensure that NSO is integrated and properly working If there is some change (delete/update) in existing MDT jobs between backup and restore operations, Cisco Crosswork does not replay the jobs for config update on the devices as it involves Provider(NSO). You have to restore configs on provider/devices. Cisco Crosswork will just restore the jobs in database. Before using any YANG modules, check if they are supported. See Section: List of Pre-loaded YANG Modules for MDT Collection

It supports data collection for the following transport mode:

• MDT TCP Dial-out Mode

Following is a sample of MDT collection payload:

{
	"collection_job": {
		"job_device_set": {
			"device_set": {
				"device_group": "mdt"
			}
		},
		"sensor_output_configs": [{
				"sensor_data": {
					"mdt_sensor": {
						"path": "Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/generic-counters"
					}
				},
				"destination": {
					"context_id": "cw.mdt_sensor.cisco-ios-xr-infra-statsd-oper.gpb",
					"destination_id": "c2a8fba8-8363-3d22-b0c2-a9e449693fae"
				}
			},
			{
				"sensor_data": {
					"mdt_sensor": {
						"path": "Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/data-rate"
					}
				},
				"destination": {
					"context_id": "cw.mdt_sensor.cisco-ios-xr-infra-statsd-oper.gpb",
					"destination_id": "c2a8fba8-8363-3d22-b0c2-a9e449693fae"
				}
			}
		],
		"sensor_input_configs": [{
				"sensor_data": {
					"mdt_sensor": {
						"path": "Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/data-rate"
					}
				},
				"cadence_in_millisec": "70000"
			}, {
				"sensor_data": {
					"mdt_sensor": {
						"path": "Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/generic-counters"
					}
				},
				"cadence_in_millisec": "70000"
			}
		],
		"application_context": {
			"context_id": "c4",
			"application_id": "a4-mdt"
		},
		"collection_mode": {
			"lifetime_type": "APPLICATION_MANAGED",
			"collector_type": "MDT_COLLECTOR"
		}
	}
}

Cisco Crosswork supports gRPC Network Management Interface (gNMI) based telemetry data collection via Cisco Crosswork Data Gateway. It supports only gNMI Dial-In (gRPC Dial-In) streaming telemetry data based on subscription and relaying subsequent subscription response(notifications) to requested destinations.

Note	gNMI collection is supported as long as the models are supported by the target device platform. gNMI must be configured on devices before you can submit gNMI collection jobs. Check platform-specific documentation.

For sample device configuration, see Sample Device Configuration - gNMI.

In gNMI, both secure and insecure mode can co-exist on the device. Cisco Crosswork gives preference to secure mode over insecure mode based on the information passed in the inventory.

If device reloads, gNMI collector ensures that the existing subscriptions are re-subscribed to the device.

gNMI specification does not have a way to mark end of message. Hence, Destination/Dispatch cadence is not supported in gNMI collector.

Cisco Crosswork Data Gateway supports all types of stream-based subscriptions:

• SAMPLE : Cadence-based collection.

• ON_CHANGE: First response include the state of all the elements for the subscribed path, followed by subsequent updates to the changes leaf values.

• TARGET_DEFINED: Router/Device choses the mode of subscription on a per-leaf basis based on the subscribed path (i.e. one of SAMPLE or ON_CHANGE)

Note

Cisco Crosswork Data Gateway relies on the device to declare the support of one or more modes. gNMI sensor path with default values does not appear payload. This is a known protobuf behavior. For boolean default value will be false. For enum it is gnmi.proto specified. Example 1: message GNMIDeviceSetting { bool suppress_redundant = 1; bool allow_aggregation = 4; bool updates_only = 6; } Example 2: enum SubscriptionMode { TARGET_DEFINED = 0; //default value will not be printed ON_CHANGE = 1; SAMPLE = 2; }

Following is a sample gNMI collection payload:

{
  "collection_job": {
    "job_device_set": {
      "device_set": {
        "device_group": "gnmi"
      }
    },
    "sensor_input_configs": [
      {
        "sensor_data": {
          "gnmi_sensor": {
               "path":{
                  "origin":"",
                  "elem":[
                     {
                        "name":"interfaces"
                     },
                     {
                        "name":"interface",
                         "key":{
                           "name":"GigabitEthernet0/0/0/4"
                        }
                     }
                  ]
               },
               "mode" : "SAMPLE"
          }
        },
        "cadence_in_millisec": "30000"
      }
    ],
    "sensor_output_configs": [
      {
        "sensor_data": {
          "gnmi_sensor": {
               "path":{
                  "origin":"",
                  "elem":[
                     {
                        "name":"interfaces"
                     },
                     {
                        "name":"interface",
                         "key":{
                           "name":"GigabitEthernet0/0/0/4"
                        }
                     }
                  ]
               },
               "mode" : "SAMPLE"
          }
        },
        "destination": {
          "context_id": "topic_gnmi",
          "destination_id": "c2a8fba8-8363-3d22-b0c2-a9e449693fae"
        }
      }
    ],
    "application_context": {
      "context_id": "gnmi_test_context",
      "application_id": "gnmi"
    },
    "collection_mode": {
      "lifetime_type": "APPLICATION_MANAGED",
      "collector_type": "GNMI_COLLECTOR"
    }
  }
}

Cisco Crosswork Data Gateway supports Syslog-based events collection from devices. Following Syslog formats are supported:

• RFC5424 syslog format

• RFC3164 syslog format

Note	Syslog must be configured on devices before you can submit Syslog collection jobs. Please refer to the platform-specific documentation.

For sample device configuration, see Configure Syslog in RFC3164/RFC5424 format.

Following is a sample Syslog collection payload:

{
  "collection_job": {
      "job_device_set": {
      "device_set": {
        "devices": {
          "device_ids": [
            "c6f25a33-92e6-468a-ba0d-15490f1ce787"
          ]
        }
      }
    },
    "sensor_output_configs": [
      {
        "sensor_data": {
          "syslog_sensor": {
            "pris": {
                "facilities": [0, 1, 3, 23,4],
                "severities": [0, 4, 5, 6, 7]
            }
        }
        },
        "destination": {
          "context_id": "syslogtopic",
          "destination_id": "c2a8fba8-8363-3d22-b0c2-a9e449693fae"
        }
      }
    ],
    "sensor_input_configs": [
      {
        "sensor_data": {
          "syslog_sensor": {
            "pris": {
                "facilities": [0,1, 3, 23,4],
                "severities": [0,4, 5, 6, 7]
            }
        }
        },
        "cadence_in_millisec": "60000"
      }
    ],
    "application_context": {
      "context_id": "demomilesstone2syslog",
      "application_id": "SyslogDemo2"
    },
    "collection_mode": {
      "lifetime_type": "APPLICATION_MANAGED",
      "collector_type": "SYSLOG_COLLECTOR"
    }
  }
}

Based on the facilities and severities mentioned in the payload, matching Syslog events will be sent to the specified destination. All other non-matching syslog events will be dropped.