When you enter startDiagnostics.sh without specifying any argument, the diagnostics tool runs in the interactive mode and prompts you to select the statistics that you want from the RDU, the DPE, and the Network Registrar servers.

Caution

Ensure that you process statistics with caution because doing so could severely impact system performance.

Syntax Description

startDiagnostics.sh [-help ]

• startDiagnostics.sh —Runs diagnostics in the interactive mode

• -help —Displays help for the tool. You must use the -help option exclusively. Do not use this with any other option.

Examples

# ./startDiagnostics.sh

Please enter directory where to put output files [] /var/CSCObac
   Please enter the duration of the diagnostic (sec) [600]

Please select statistics you would like to gather on RDU

   CPU statistics (y/n/q)? [y]
   Process statistics (y/n/q)? [n]
   IO statistics (y/n/q)? [y]
   Memory statistics (y/n/q)? [y]
   Network statistics (y/n/q)? [y]
   RDU API traffic (y/n/q)? [y]
   RDU CNR traffic (y/n/q)? [y]
   RDU DPE traffic (y/n/q)? [y]
   RDU CNR extension traffic (y/n/q)? [y]
   RDU SNMP traffic (y/n/q)? [y]
   System Configuration (y/n/q)? [y]

Enter addition argument for RDU API traffic
   Please enter RDU Server port [49187]

Enter addition arguments for RDU DPE traffic
   Enter DPE ip addr if you want to capture traffic by ip addr [] 10.10.29.1
   Enter DPE port number if you want to capture traffic by port number [] 49186

Enter addition arguments for RDU CNR_EX traffic
   Enter Ip addr if you want to capture traffic by Cnr Extension IP addr [] 10.10.85.2
   Enter port number if you want to capture traffic by Cnr Extension port []

You could run statusDiagnostics.sh to find out the status of the diagnostics.
You could run stopDiagnostics.sh to stop the diagnostics.
You could run bundleState.sh to bundle the output when diagnostics is complete.

Note	If you do not enable statistics for the following options, the tool does not request values for additional arguments, as featured in the example: RDU-API traffic RDU-DPE traffic RDU-Network Registrar extension traffic

After you run the startDiagnostics.sh tool, output files for each statistic are created under the directory in which you run the tool. You can also bundle the output files and forward them to the Cisco Technical Assistance Center for support. To do so, enter y at the System Diagnostics Capture prompt.

For example:

System Configuration (y/n/q)? [y]

For more details on bundling server state, see Bundling Server State for Support, page 26-11.

Before running the startDiagnostics.sh tool in the noninteractive mode for the first time, you must generate the response file. Thereafter, you only need to run a single command, which collects diagnostics based on the arguments contained in the response file.

Syntax Description

startDiagnostics.sh {-g response_file | -r response_file} [-help ]

• -g —Generates the response file. You need to use this option only when generating a response file for the first time.

• -r —Runs the diagnostics tool using the response file

• response_file —Specifies the name of the response file

• -help —Displays help for the tool. You must use the -help option exclusively. Do not use this with any other option.

  Examples

This results occurs when you generate the response file.

# ./startDiagnostics.sh -g response.txt

Please enter directory where to put output files [] /var/CSCObac
   Please enter the duration of the diagnostic (sec) [600]

Please select statistics you would like to gather on RDU

   CPU statistics (y/n/q)? [y]
   Process statistics (y/n/q)? [n]
   IO statistics (y/n/q)? [y]
   Memory statistics (y/n/q)? [y]
   Network statistics (y/n/q)? [y]
   RDU API traffic (y/n/q)? [y] n
   RDU CNR traffic (y/n/q)? [y]
   RDU DPE traffic (y/n/q)? [y] n
   RDU CNR extension traffic (y/n/q)? [y] n
   RDU SNMP traffic (y/n/q)? [y]
   System Configuration (y/n/q)? [y]

Finished generate response file (response.txt).

The response.txt is generated under the directory in which you run the startDiagnostics.sh script; in this case, BPR_HOME/rdu/diagnostics/bin. A sample response file generated for RDU diagnostics is featured here:

test.bundle.direcotry=/var/CSCObac
test.bundle.duration.sec=100
test.cpu.enable=true
test.process.enable=false
test.io.enable=true
test.memory.enable=true
test.network.enable=true
test.rdu_api_traffic.enable=true
test.rdu_cnr_traffic.enable=true
test.rdu_dpe_traffic.enable=true
test.rdu_cnr_ex_traffic.enable=true
test.rdu_snmp_traffic.enable=true
test.system_config.enable=true
test.rdu.port=49187
test.dpe.port=49186
test.dpe.ip=10.10.29.1
test.cnr_ex.ip=10.10.85.2
test.cnr_ex.port=
EOF

This result occurs when you run the diagnostics tool using the response file that you have generated.

# ./startDiagnostics.sh -r response.txt

You could run statusDiagnostics.sh to find out the status of the diagnostics.
You could run stopDiagnostics.sh to stop the diagnostics.

After you run the startDiagnostics.sh tool, output files for each statistic are created under the directory in which you run the tool.

Running stopDiagnostics.sh in the interactive mode, without any argument, prompts you to specify if you want to stop diagnostics for all statistics or for specific statistics.

Syntax Description

stopDiagnostics.sh [-help ]

• stopDiagnostics.sh —Stops diagnostic collection in the interactive mode.

• -help —Displays help for the tool. You must use the -help option exclusively. Do not use this with any other option.

  Examples

  
  # ./stopDiagnostics.sh 
  
  This script allowed to stop specific diagnostic or all diagnostics.
  If you would like to stop specific diagnostics, say no to question below.
  
     Would you like to stop all diagnostics (y/n/q)? [y]
  

Running stopDiagnostics.sh in the noninteractive mode stops diagnostics for all statistics.

Syntax Description

stopDiagnostics.sh -a [-help ]

• -a —Stops diagnostics for all statistics without prompting you.

• -help —Displays help for the tool. You must use the -help option exclusively. Do not use this with any other option.

  Examples

  
  # ./stopDiagnostics.sh -a
  #
  

The eMTA is a cable modem and an MTA in one box, with a common software image. The CM and MTA each have their own MAC addresses and each performs DHCP to get its own IP address. The eMTA contains, at minimum, three certificates. One certificate is a unique MTA certificate. A second certificate identifies the MTA manufacturer. Both the device and manufacture certificates are sent by the MTA to authenticate itself to the KDC. The third certificate is a telephony root certificate used to verify the certificates sent by the KDC to the MTA. The KDC certificates will be chained from the telephony root, therefore the telephony root must reside on the MTA to validate the authenticity of the KDC certificates. The MTA portion receives its own configuration file, which it uses to identify its controlling call agent, among other things.

The DOCSIS specifications mandate that cable modems negotiate their IP addresses using DHCP. The MTA, like most CPE on a DOCSIS network, must use DHCP to obtain its IP address and other crucial information (DNS servers, PacketCable Option 122 for Kerberos realm name, provisioning server FQDN).

Note	The cable modem portion, in addition to its normally required DHCP options, also requests, and must receive, Option 122 suboption 1, which it passes to the MTA portion as the IP address of the correct DHCP server from which to accept offers.

When using Prime Cable Provisioning with PacketCable support, be aware that a correctly configured Prime Cable Provisioning will automatically populate the ToD server, DNS servers, TFTP server, as well as the Option 122 fields; these do not need to be explicitly set in the Network Registrar policy.

The Domain Name System (DNS) server is fundamental in PacketCable provisioning. The PacketCable provisioning server, which is the device provisioning engine (DPE) in a Prime Cable Provisioning architecture, must have an address (A) record in the appropriate zone, because its fully qualified domain name (FQDN) is provided to the MTA in Option 122 by the DHCP server. The KDC realm must have a zone of the same name as the realm name containing a server (SRV) record that contains the FQDN of the Kerberos server.

The Kerberos server identified in the SRV record must itself have an A record in the appropriate zone. The call management server (CMS) identified in the MTA configuration file must also have an A record in the appropriate zone. Lastly, the MTAs themselves must have A records in the appropriate zone, because the CMS reaches the MTA by resolving its FQDN. Dynamic DNS (DDNS) is the preferred method of creating A records for the MTA. See Cisco Prime Network Registrar documentation for information on configuring and troubleshooting DDNS.

The KDC is responsible for authenticating MTAs. As such, it must check the MTA certificate, and provide its own certificates so that the MTA can authenticate the KDC. It also communicates with the DPE (the provisioning server) to validate that the MTA is provisioned on the network.

The PacketCable provisioning server is responsible for communicating the location of the MTA configuration file to the MTA, and/or provisioning MTA parameters via SNMP. SNMPv3 is used for all communication between the MTA and the provisioning server. The keys used to initiate SNMPv3 communication are obtained by the MTA during its authentication phase with the KDC. Provisioning server functionality is provided by the DPE in a Prime Cable Provisioning architecture.

The call management server (CMS) is essentially a soft switch, or call-agent, with additional PacketCable functionality to control, among other things, quality of service on a cable network. The MTA sends a network call signaling (NCS) restart in progress (RSIP) message to the CMS upon successful PacketCable provisioning.

The MTA_Root.cer file contains the MTA root certificate (a certificate that is rooted in the official PacketCable MTA root).

You must know in advance what telephony root certificate is required for the MTAs you want to provision. Deployments in production networks use telephony certificates rooted in the PacketCable real root. There is also a PacketCable test root used in testing environments.

The KDC certificates used by the KDC to authenticate itself to the MTA must be rooted in the same telephony root that is stored on the MTA (PacketCable real or test root). Most MTA vendors support test images that have Telnet and/or HTTP login capabilities so that you can determine which telephony root is enabled, and change the root used (in most cases, you can only select between the PacketCable real or test root).

The most common scenario has the KDC loaded with certificates (from the BPR_HOME/kdc/<Operating System>/packetcable/certificates directory) as follows:

• CableLabs_Service_Provider_Root.cer

• Service_Provider.cer

• Local_System.cer

• KDC.cer

• MTA_Root.cer

The first four certificates comprise the telephony certificate chain. The MTA_Root.cer file contains the MTA root used by the KDC to validate the certificates sent by the MTA.

Note	See Using PKCert.sh, for information on installing and managing KDC certificates.

To determine if you are using PacketCable test root, open the CableLabs_Service_Provider_Root.cer file in Windows, and validate that the Subject OrgName entry is O = CableLabs, and/or check that the Subject Alternative name reads CN=CABLELABS GENERATED TEST ROOT FOR EQUIPMENT TEST PURPOSES ONLY.

The KDC certificate (KDC.cer) contains the realm name to use. The realm name that Prime Cable Provisioning (and the corresponding DNS zone) is configured to use must match this realm name. Additionally, the MTA configuration file realm org name must match the organization name as seen in the telephony root.

The KDC certificate has a corresponding private key that must be installed in the BPR_HOME/kdc/linux directory. Usually it is named KDC_private_key.pkcs8 or KDC_private_key_proprietary. When changing certificates, you must also change the private key.

In most scenarios, Prime Cable Provisioning is involved in processing all DHCP requests from scopes with scope-selection tags that match selection criteria specified in the DHCP Criteria page of the Prime Cable Provisioning administrator user interface. Client class can also be used to tie scopes to Prime Cable Provisioning processing; ensure you make this association before you attempt to provision devices.

The MTA configuration file contains the location of the CMS. Additionally, it must contain an entry for Realm Name. This value must match that of the certificate chain in use.

Certain table entries within the MTA configuration file are indexed by the realm name delivered to the MTA in Option 122. This realm name entry in the MTA configuration file must match that delivered in Option 122. For example, if DEF.COM was the realm name delivered in Option 122, MTA configuration file entries in the pktcMtaDevRealm table would be indexed with a suffix made up of the ASCII-coded character values (in dot-delimited decimal format when using the Cisco Broadband Configurator) of the realm name; for example 68.69.70.46.67.79.77. There are many free ASCII conversion pages available on the web to make this conversion easier.

These log files are used to maintain the following information:

• The Network Registrar has two logs (name_dhcp_1_log and name_dns_1_log), which contain the most recent logging entries from Network Registrar. Look in these files for DHCP- or DNS-related problems.

• The BPR_HOME/kdc/logs/kdc.log file shows all KDC interactions with MTAs, and KDC interactions with the DPE.

• The BPR_DATA/dpe/logs/dpe.log file shows the major steps related to SNMPv3 interaction with the MTA.

  Note

  Turning on the tracing of snmp, registration server, and registration server detail messages, using the command-line interface (CLI), helps to troubleshoot potential PacketCable problems. For information on the appropriate troubleshooting commands, see the Cisco Prime Cable Provisioning 6.1.1 DPE CLI Reference Guide.

A packet capture tool is indispensable when troubleshooting the eMTAs. The Ethereal version, as packaged by CableLabs, includes numerous packet decoders specific to PacketCable. These include the Kerberos AS and AP packets.

• If you suspect that a specific failure is DHCP-related, capture packets while filtering on packets sourced from, or destined to, the CMTS cable interface IP address and the DHCP server IP address.

• If you suspect that a specific failure is related to any of the 25 steps occurring after DHCP, filter all packets to and from the eMTA IP address. This provides a very concise, easy-to-follow trace of provisioning steps 5 through 25, as shown in Figure 1.

This certificate must be verified as part of a certificate chain containing the MTA Root Certificate, the MTA Manufacturer Certificate, and the MTA Device Certificate.

The following table lists the values relevant to the MTA Root Certificate.

This certificate must be verified as part of a certificate chain containing the MTA Root Certificate, the MTA Manufacturer Certificate, and the MTA Device Certificate. The state/province, city, and manufacturer’s facility are optional attributes. A manufacturer may have more than one manufacturer’s certificate, and there may exist one or more certificates per manufacturer. All certificates for the same manufacturer may be provided to each MTA either at manufacture time or during a field update. The MTA must select an appropriate certificate for its use by matching the issuer name in the MTA Device Certificate with the subject name in the MTA Manufacturer Certificate. If present, the authorityKeyIdentifier of the device certificate must match the subjectKeyIdentifier of the manufacturer certificate as described in RFC 2459. The CompanyName field that is present in O and CN may be different in the two instances.

The following table lists the values relevant to the MTA Manufacturer Certificate.

Code Verification Certificate (CVC) specification for eMTAs must be identical to the DOCSIS 1.1 CVC, specified in DOCSIS specification SP-BPI+-I11-040407.

Before any Kerberos key management can be performed, an MTA and a KDC need to perform mutual authentication using the PKINIT extension to the Kerberos protocol. An MTA authenticates a KDC after it receives a PKINIT Reply message containing a KDC certificate chain. In authenticating the KDC, the MTA verifies the KDC certificate chain, including the KDC’s Service Provider Certificate signed by the CableLabs Service Provider Root CA.

The following table lists the values relevant to the CableLabs Service Provider Root Certificate.

This is the certificate held by the service provider, signed by the CableLabs Service Provider Root CA. It is verified as part of a certificate chain that includes the CableLabs Service Provider Root Certificate, the Telephony Service Provider Certificate, an optional Local System Certificate, and an end-entity server certificate. The authenticating entities normally already possess the CableLabs Service Provider Root Certificate and it is not transmitted with the rest of the certificate chain.

The fact that a Service Provider CA Certificate is always explicitly included in the certificate chain allows a Service Provider the flexibility to change its certificate without requiring reconfiguration of each entity that validates this certificate chain (for example, an MTA validating a PKINIT Reply). Each time the Service Provider CA Certificate changes, its signature must be verified with the CableLabs Service Provider Root Certificate. However, a new certificate for the same Service Provider must preserve the same value of the OrganizationName attribute in the SubjectName. The Company field that is present in O and CN may be different in the two instances.

The following table lists the values relevant to the CableLabs Service Provider CA Certificate.

A Service Provider CA may delegate the issuance of certificates to a regional Certification Authority called Local System CA (with the corresponding Local System Certificate). Network servers are allowed to move freely between regional Certification Authorities of the same Service Provider. Therefore, the MTA MIB does not contain any information regarding a Local System Certificate (which might restrict an MTA to KDCs within a particular region).

The following table lists the values relevant to the Local System CA Certificate.

All these are signed by either the Local System CA or by the Service Provider CA. Other ancillary certificates may be added to this standard at a later time.

The following requirements apply to all Code Verification Certificates:

• Certificates must be DER encoded.

• Certificates must be version 3.

• Certificates must include the extensions that are specified in the following tables and must not include any additional extensions.

• The public exponent must be F4 (65537 decimal).

This certificate must be validated as part of the certificate chain containing the CableLabs Code Verification Root CA Certificate, the CableLabs Code Verification CA, and the Code Verification Certificates. See Certificate Validation, page 26-21, for additional information on how to 
validate certificates.

The following table lists the values relevant to the CableLabs Code Verification Root CA Certificate.

The CableLabs Code Verification CA Certificate must be validated as part of a certificate chain containing the CableLabs Code Verification Root CA Certificate, the CableLabs Code Verification CA Certificate, and the Code Verification Certificate. See Certificate Validation, page 26-21, for additional information on how to validate certificates. There may be more than one CableLabs Code Verification CA. An S-MTA must support one CableLabs CVC CA at a time.

The following table lists the values relevant to the CableLabs Code Verification CA Certificate.

The CableLabs Code Verification CA issues this certificate to each authorized Manufacturer. It is used in the policy set by the cable operator for secure software download.

The following table lists the values relevant to the Manufacturer Code Verification Certificate.

The Company Name in the Organization may be different than the Company Name in the 
Common Name.

The Service Provider Code Verification Certificate must be validated as part of a certificate chain containing the CableLabs Code Verification Root CA Certificate, the CableLabs Code Verification CA Certificate, and the Service Provider Code Verification Certificate. See Certificate Validation, page 26-21, for additional information on how to validate certificates.

The following table lists the values relevant to the Service Provider Code Verification Certificate.

The Company Name in the Organization may be different than the Company Name in the Common Name.

The S-MTA is not required to support Certificate Revocation Lists (CRLs) for CVCs.