- Preface
- Overview
- Troubleshooting Matrix
- Troubleshooting an Unsuccessful Installation or Update
- Troubleshooting the Configuration
- Troubleshooting Cisco APIC-EM Multi-Host
- Troubleshooting Services Using System Health
- Troubleshooting Services Using the Developer Console
- Troubleshooting Using the Logs
- Troubleshooting Passwords
- Troubleshooting Commands
- Troubleshooting Log Files
- Contacting the Cisco Technical Assistance Center
- Index
Troubleshooting Services Using the Developer Console
The following procedures may be used to troubleshoot services using the developer console:
- Grapevine Developer Console
- Logging into the Grapevine Developer Console
- Reviewing the Service Version, Status, and Logs
- Removing a Service Instance Manually
- Creating a Service Instance Manually
- Reviewing the User Logs
- Monitoring Services and Clients Using the CLI
Grapevine Developer Console
The Cisco APIC-EM creates a Platform as a Service (PaaS) environment for your network. A service in this PaaS environment is a horizontally scalable application that adds instances of itself when increasing loads occur on a client within the network. You use the Grapevine developer console to troubleshoot these services. The Grapevine developer console tools were bundled with the deployment files and installed when you first deployed the Cisco APIC-EM.
 Note | For a multi-host cluster, you do not have to log into each host to view the Grapevine developer console. In a multi-host cluster, you get a single, consolidated view of all of the services running on all three hosts. |
The Grapevine developer console provides the following windows and functionality:
-
Overview—Provides a list of services with information about their version and status. You can add or remove services in this window.
-
Clients—Provides detailed client information in this window.
-
Hosts—Provides detailed host information in this window.
-
Waiting Queue—Provides information about the waiting queue.
-
Services—Provides detailed service information. You can add or remove services in this window.
-
Logs—Provides detailed task, instance, and client logs
Note | You cannot access the Grapevine developer console as a Linux root user. You can only access the Grapevine developer console using the administrator username and password that you configured during the deployment process. |
Logging into the Grapevine Developer Console
You are able to perform the following tasks using the Grapevine developer console:
 Caution | Only advanced users should access the Grapevine developer console to perform any troubleshooting tasks for the services. |
You must have successfully deployed the Cisco APIC-EM.
What to Do Next
Proceed to review any of the service versions, status, and logs for any troubleshooting purpose.
Reviewing the Service Version, Status, and Logs
You are able to perform the following tasks using the Grapevine developer console:
Caution | Only advanced users should access the console to perform the tasks described in this procedure or attempt to troubleshoot the services. |
You must have successfully deployed the Cisco APIC-EM and it must be operational.
| Step 1 | Access
the Grapevine developer console by opening a browser window and entering the IP
address that you configured for the network adapter using the configuration
wizard.
For example, enter the following IP address with required port number: https://external network IP address:14141 | ||
| Step 2 | Enter
your administrative username and password when prompted.
The administrative username and password were configured by you using the configuration wizard. The console for the Elastic Service Platform (Grapevine) appears. | ||
| Step 3 | Review the
status of each service listed in the
Overview window in the console.
Each service is represented by a square. A green colored square represents an active instance of the service, and a red colored square represents a service with a faulty or failed instance. Squares without color represents inactive services (no instances initiated and running). In a multi-host environment, a service may be represented by two green colored squares, indicating that the service is running on two different hosts within your cluster. Place your cursor over each square to view the host that the service is running on. | ||
| Step 4 | Review the
version of each service in the
Overview window in the console.
The version is located in the header of each listed service. | ||
| Step 5 | Review the service logs by clicking a specific active instance of a service (green square icon) and then viewing the Instance or Client logs located at the bottom of the window. |
What to Do Next
When finished with the Grapevine developer console, click the Logout button to log out of the console.
Removing a Service Instance Manually
You are able to remove or harvest a service instance manually by using the Grapevine developer console. The Grapevine developer console tools are bundled within the ISO image and installed when you first deploy the Cisco APIC-EM.
Caution | Only advanced users should access the Grapevine developer console to perform the tasks described in this procedure or attempt to troubleshoot the services. |
You must have successfully deployed the Cisco APIC-EM and it must be operational.
| Step 1 | Access
the Grapevine developer console by opening a Google Chrome browser window and
entering the IP address that you configured for the network adapter using the
configuration wizard.
For example, enter the following IP address with required port number: https://external network IP address:14141 | ||
| Step 2 | Enter
your administrative username and password when prompted.
The administrative username and password were configured by you using the configuration wizard. The Grapevine developer console for the Elastic Service Platform (Grapevine) appears. | ||
| Step 3 | Review the list
of operational services in the
Overview window in the console.
Each service is represented by a square. A green colored square represents an active instance of the service, and a red colored square represents a service with a faulty or failed instance. Squares without color represents inactive services (no instances initiated and running). Placing your cursor over a square displays the IP address of the Grapevine client where the service is running. In a multi-host environment, a service may be represented by two green colored squares, indicating that the service is running on two different hosts within your cluster. Place your cursor over each square to view the host that the service is running on.
| ||
| Step 4 | Locate the
service where you want to manually harvest an instance of a service and click
the subtraction sign (-) at the lower right.
You are then prompted to confirm your action to harvest an instance. | ||
| Step 5 | Choose
Yes in the dialog box to confirm that you want to
harvest an instance of the service.
Grapevine then proceeds to spin down the instance of the service. When the process is finished, the square representing the service instance is removed. |
What to Do Next
Manage your services by manually growing additional instances or removing (harvesting) instances from the services. When finished with the Grapevine developer console, click the Logout button to log out of the console.
Creating a Service Instance Manually
You can create a service instance manually using the Grapevine developer console.
Caution | Only advanced users should access the Grapevine developer console to perform the tasks described in this procedure or attempt to troubleshoot the services. |
You must have successfully deployed the Cisco APIC-EM and it must be operational.
| Step 1 | Access
the Grapevine developer console by opening a Google Chrome browser window and
entering the IP address that you configured for the network adapter using the
configuration wizard.
For example, enter the following IP address with required port number: https://external network IP address:14141 | ||
| Step 2 | Enter
your administrative username and password when prompted.
The administrative username and password were configured by you using the configuration wizard. The Grapevine developer console for the Elastic Service Platform (Grapevine) appears. | ||
| Step 3 | Review the list
of operational services in the
Overview window in the Grapevine developer console.
Each service is represented by a square. A green colored square represents an active instance of the service, and a red colored square represents a service with a faulty or failed instance. Squares without color represents inactive services (no instances initiated and running). Placing your cursor over a square displays the IP address of the Grapevine client where the service is running. In a multi-host environment, a service may be represented by two green colored squares, indicating that the service is running on two different hosts within your cluster. Place your cursor over each square to view the host that the service is running on.
| ||
| Step 4 | Locate the
service where you want to manually grow an instance of a service and click the
addition sign (+) at the lower right.
You are then prompted to confirm your action to grow an instance. | ||
| Step 5 | Choose
Yes in the dialog box to confirm that you want to
grow an instance of the service.
Grapevine then takes a Grapevine client from the Spare Capacity Pool and spins up an instance of the service. When the process is finished, the square that represents the new service instance turns green. |
What to Do Next
Manage your services by manually growing additional instances or removing (harvesting) instances from the services. When finished with the Grapevine developer console, click the Logout button to log out of the console.
Reviewing the User Logs
You are able to review the user logs using the Elastic Service Platform (Grapevine) developer console.
Note | We recommend that only advanced users use the developer console to perform the steps described in this procedure. |
You must have successfully deployed the Cisco APIC-EM and it must be operational.
| Step 1 | Access
the Grapevine developer console by opening a Google Chrome browser window and
entering the IP address that you configured for the network adapter using the
configuration wizard.
For example, enter the following IP address with required port number: https://external network IP address:14141 | ||
| Step 2 | Enter
your administrative username and password when prompted.
The administrative username and password were configured by you using the configuration wizard. The Grapevine developer console for the Elastic Service Platform (Grapevine) appears. | ||
| Step 3 | Review the
status of each service listed in the
Overview window in the console.
Each service is represented as a square. A green square represents an active instance of the service, and a red square represents a service with a faulty or failed instance. Squares without color represents inactive services (no instances initiated and running). | ||
| Step 4 | Review the
version of each service in the
Overview window in the console.
The version is located in the header of each listed service. | ||
| Step 5 | Review the service logs by clicking a specific active instance of a service (green square icon) and then viewing the Instance logs located at the bottom of the window. | ||
| Step 6 | Search through
the logs for the keyword: USER-ACCOUNTING.
The following data is displayed for these log entry types:
|
What to Do Next
After reviewing the user data in the logs, proceed to troubleshoot any user activity.
When finished with the developer console, click Logout.
Monitoring Services and Clients Using the CLI
In addition to the developer console, a command-line interface on the host is also provided for troubleshooting purposes. This CLI allows you to monitor the health of the Cisco APIC-EM from the command line.
You have deployed the Cisco APIC-EM using the procedures described in this guide.
| Step 1 | Using a Secure
Shell (SSH) client, log into the host (physical or virtual) with the IP address
that you specified using the configuration wizard.
| ||
| Step 2 | When prompted, enter your Linux username ('grapevine') and password for SSH access. | ||
| Step 3 | Display all of
the
Cisco APIC-EM
services currently installed by entering the
grape
service display command. Key data about each service is then
displayed separated by hash marks.
$ grape service display
...
#
rabbitmq config {}
rabbitmq core_service True
rabbitmq enabled False
rabbitmq endpoint_config {u'default':
{u'backend_protocol': u'amqp', u'backend_path': u'',
u'frontend_protocol': u'', u'frontend_path': u'', u'frontend_port':
0, u'backend_port': 5672}}
rabbitmq kill_as_group True
rabbitmq max_instances 1
rabbitmq min_instances 0
rabbitmq priority 1
rabbitmq queue_config {u'queues': [],
u'bindings': [], u'exchanges': []}
rabbitmq requirements {u'template_id':
u'default', u'persistent_disk': False}
rabbitmq run_as_group grapevine
rabbitmq run_as_user grapevine
rabbitmq service_type rabbitmq
rabbitmq spare_count 0
rabbitmq start_secs 10
rabbitmq static_load 10
rabbitmq status_interval 60
rabbitmq stop_as_group True
rabbitmq stop_signal TERM
rabbitmq version 1.0.0
#
...
| ||
| Step 4 | Display
running instances of the
Cisco APIC-EM
services by entering the
grape
instance display command. Key data about running instances of
service is then displayed separated by hash marks.
$ grape instance display
...
#
4c4c83db-2da6-4a04-9af6-96c8dac692d1 client_id
4c4c83db-2da6-4a04-9af6-96c8dac692d1
4c4c83db-2da6-4a04-9af6-96c8dac692d1 endpoint_config
{u'default': {u'backend_port': 5672, u'backend_protocol': u'amqp'}}
4c4c83db-2da6-4a04-9af6-96c8dac692d1 interfaces
[{'interface': 'eth0', 'ip': '192.168.0.1', 'mac': '00:50:56:9f:6c:c4'},
{'interface': 'eth1', 'ip': '172.16.0.15', 'mac': '00:50:56:9f:71:46'}]
4c4c83db-2da6-4a04-9af6-96c8dac692d1 is_error None
4c4c83db-2da6-4a04-9af6-96c8dac692d1 service_type rabbitmq
4c4c83db-2da6-4a04-9af6-96c8dac692d1 state running
4c4c83db-2da6-4a04-9af6-96c8dac692d1 task_id None
4c4c83db-2da6-4a04-9af6-96c8dac692d1 timestamp
Fri Oct 03, 2014 02:05:43 PM (4 days ago)
4c4c83db-2da6-4a04-9af6-96c8dac692d1 version 1.0.0
#
...
| ||
| Step 5 | Display all
Grapevine clients currently running in
Cisco APIC-EM
by entering the
grape
client display command. Key data about each client is then
displayed separated by hash marks.
$ grape client display
CLIENT PROPERTY VALUE
----------------------------------------------------------------------
ae63a6c1-a946-4df7-a68d-33227eed8134 client_id ae63a6c1-a946-4df7-a68d-33227eed8134
ae63a6c1-a946-4df7-a68d-33227eed8134 client_version 0.1.0.212.dev633-gf7e21de
ae63a6c1-a946-4df7-a68d-33227eed8134 interfaces [{'interface': 'eth0', 'ip': '192.168.0.32',
'mac': '00:50:56:9f:3a:90'}]
ae63a6c1-a946-4df7-a68d-33227eed8134 is_alive True
ae63a6c1-a946-4df7-a68d-33227eed8134 last_heartbeat Wed Oct 08, 2014 10:22:50 AM (15 secs ago)
ae63a6c1-a946-4df7-a68d-33227eed8134 template_id default
ae63a6c1-a946-4df7-a68d-33227eed8134 vm_id ce0a634a-5475-4450-9dce-f3217d855ac4
#
...
All clients should have a is alive property of True. |
Feedback