Fibre Channel domains can be started disruptively or nondisruptively. If you perform a disruptive restart, reconfigure fabric (RCF) frames are sent to other switches in the fabric and data traffic is disrupted on all the switches in the VSAN (including remotely segmented ISLs). If you perform a nondisruptive restart, build fabric (BF) frames are sent to other switches in the fabric and data traffic is disrupted only on the switch.

If you are attempting to resolve a domain ID conflict, you must manually assign domain IDs. A disruptive restart is required to apply most configuration changes, including manually assigned domain IDs. Nondisruptive domain restarts are acceptable only when changing a preferred domain ID into a static one (and the actual domain ID remains the same).

Note	It is not recommended to use disruptive restart followed by VSAN suspend/no-suspend, since it is used only for recovery purpose when normal restart does not solve the problem.

Note	A static domain is specifically configured by the user and may be different from the runtime domain. If the domain IDs are different, the runtime domain ID changes to take on the static domain ID after the next restart, either disruptive or nondisruptive.

Tip	If a VSAN is in interop mode, you cannot restart the fcdomain for that VSAN disruptively.

You can apply most of the configurations to their corresponding runtime values. Each of the following sections provide further details on how the fcdomain parameters are applied to the runtime values.

The fcdomain restart command applies your changes to the runtime settings. Use the disruptive option to apply most of the configurations to their corresponding runtime values, including preferred domain IDs (see the Domain IDs).

Domain Manager All Optimization feature can be used to enable or disable all of the optimization modes.

Note	You cannot enable all the optimizations such as Selective Restart, Fast Restart, and Scale Restart in VSANs where Interop mode is enabled (non-native modes). Also you cannot move a VSAN where the optimizations are enabled into Interop mode 1 to 4.

As of Cisco MDS SAN-OS Release 3.0(2), when a principal link fails, the domain manager must select a new principal link. By default, the domain manager starts a build fabric phase, followed by a principal switch selection phase. Both of these phases involve all the switches in the VSAN and together take at least 15 seconds to complete. To reduce the time required for the domain manager to select a new principal link, you can enable the domain manager fast restart feature.

When fast restart is enabled and a backup link is available, the domain manager needs only a few milliseconds to select a new principal link to replace the one that failed. Also, the reconfiguration required to select the new principal link only affects the two switches that are directly attached to the failed link, not the entire VSAN. When a backup link is not available, the domain manager reverts to the default behavior and starts a build fabric phase, followed by a principal switch selection phase. We recommend using fast restart on most fabrics, especially those with a large number of logical ports (3200 or more), where a logical port is an instance of a physical port in a VSAN.

During fabric reconfiguration, as and when principal switch assigns a domain ID to a switch (including itself), it transmits an Exchange Fabric Parameter (EFP) request. This request basically carries domain list information of the fabric. So whenever domain list grows there will be a Exchange Fabric Parameter flooded to the fabric. With this feature optimization enabled, a single consolidated Exchange Fabric Parameter request will be flooded by the principal switch once the domain identifier allocation phase is completed. This feature optimization cannot be supported in interop mode.

Scale Restart will be enabled by default in all native VSANs. It will not be enabled in interop VSANs.

In the Fibre Channel protocol, fabric reconfiguration starts with build fabric frame flooding, which indicates to all the switches in the fabric that the fabric is changing. This process is followed by principal switch selection and domain ID allocation phases. During the build fabric flooding phase, build fabric frames are flooded on all the links. A switch may have more than one link to a peer switch. In such cases, the build fabric frame can be sent to only one of the links to the peer switch. This situation reduces the number of build fabric frames that are to be exchanged during the build fabric phase of fabric reconfiguration. Enabling this feature optimization, sends the build frame to only one of the peer switch links which benefits scaling.

Any new switch can become the principal switch when it joins a stable fabric. During the principal switch selection phase, the switch with the highest priority becomes the principal switch. If two switches have the same configured priority, the switch with the lower WWN becomes the principal switch.

The priority configuration is applied to runtime when the fcdomain is restarted (see the Domain Restart). This configuration is applicable to both disruptive and nondisruptive restarts.

By default, the fcdomain feature is enabled on each switch. If you disable the fcdomain feature in a switch, that switch can no longer participate with other switches in the fabric. The fcdomain configuration is applied to runtime through a disruptive restart.

You can choose to reject RCF request frames on a per-interface, per-VSAN basis. By default, the RCF reject option is disabled (that is, RCF request frames are not automatically rejected).

The RCF reject option takes immediate effect at runtime through a disruptive restart (see the Domain Restart)

You can configure the rcf-reject option on a per-interface, per-VSAN basis. By default, the rcf-reject option is disabled (that is, RCF request frames are not automatically rejected).

The rcf-reject option takes effect immediately. No fcdomain restart is required.

By default, the autoreconfigure option is disabled. When you join two switches belonging to two different stable fabrics that have overlapping domains, the following cases apply:

• If the autoreconfigure option is enabled on both switches, a disruptive reconfiguration phase is started.
• If the autoreconfigure option is disabled on either or both switches, the links between the two switches become isolated.
• RCF is expected only when auto-reconfigure is enabled in entire fabric.

The autoreconfigure option takes immediate effect at runtime. You do not need to restart the fcdomain. If a domain is currently isolated due to domain overlap, and you later enable the autoreconfigure option on both switches, the fabric continues to be isolated. If you enabled the autoreconfigure option on both switches before connecting the fabric, a disruptive reconfiguration (RCF) will occur. A disruptive reconfiguration may affect data traffic. You can nondisruptively reconfigure the fcdomain by changing the configured domains on the overlapping links and eliminating the domain overlap.

Domain IDs uniquely identify a switch in a VSAN. A switch may have different domain IDs in different VSANs. The domain ID is part of the overall FC ID.

The configured domain ID can be preferred or static. By default, the configured domain ID is 0 (zero) and the configured type is preferred.

Note	The 0 (zero) value can be configured only if you use the preferred option.

If you do not configure a domain ID, the local switch sends a random ID in its request. We recommend that you use static domain IDs.

When a subordinate switch requests a domain, the following process takes place (see Figure 1):

1. The local switch sends a configured domain ID request to the principal switch.

2. The principal switch assigns the requested domain ID if available. Otherwise, it assigns another available domain ID.

The behavior for a subordinate switch changes based on three factors:

• The allowed domain ID lists.

• The configured domain ID.

• The domain ID that the principal switch has assigned to the requesting switch.

In specific situations, the changes are as follows:

• When the received domain ID is not within the allowed list, the requested domain ID becomes the runtime domain ID and all interfaces on that VSAN are isolated.

• When the assigned and requested domain IDs are the same, the preferred and static options are not relevant, and the assigned domain ID becomes the runtime domain ID.

• When the assigned and requested domain IDs are different, the following cases apply:

  • If the configured type is static, the assigned domain ID is discarded, all local interfaces are isolated, and the local switch assigns itself the configured domain ID, which becomes the runtime domain ID.

  • If the configured type is preferred, the local switch accepts the domain ID assigned by the principal switch and the assigned domain ID becomes the runtime domain ID.

If you change the configured domain ID, the change is only accepted if the new domain ID is included in all the allowed domain ID lists currently configured in the VSAN. Alternatively, you can also configure zero-preferred domain ID.

Tip	When the FICON feature is enabled in a given VSAN, the domain ID for that VSAN remains in the static state. You can change the static ID value but you cannot change it to the preferred option.

Note

In an IVR without NAT configuration, if one VSAN in the IVR topology is configured with static domain IDs, then the other VSANs (edge or transit) in the topology should also be configured with static domain IDs. In an IVR NAT configuration, if one VSAN in the IVR topology is configured with static domain IDs, then the IVR domains that can be exported to that VSAN must also be assigned static domains.

Caution

You must enter the fcdomain restart command if you want to apply the configured domain changes to the runtime domain.

Caution

You must restart the fcdomain if you want to apply the configured domain changes to the runtime domain.

Note	If you have configured an allowed domain ID list, the domain IDs that you add must be in that range for the VSAN. See the Configuring Allowed Domain ID Lists.

The first action that modifies the existing configuration creates the pending configuration and locks the feature in the fabric. Once you lock the fabric, the following conditions apply:

• No other user can make any configuration changes to this feature.
• A pending configuration is created by copying the active configuration. Modifications from this point on are made to the pending configuration and remain there until you commit the changes to the active configuration (and other switches in the fabric) or discard them.

To apply the pending domain configuration changes to other MDS switches in the VSAN, you must commit the changes. The pending configuration changes are distributed and, on a successful commit, the configuration changes are applied to the active configuration in the MDS switches throughout the VSAN and the fabric lock is released.

If you have performed a domain configuration task and have not released the lock by either committing or discarding the changes, an administrator can release the lock from any switch in the fabric. If the administrator performs this task, your pending changes are discarded and the fabric lock is released.

The pending changes are only available in the volatile directory and are discarded if the switch is restarted.

When an N or NL port logs into a Cisco MDS 9000 Family switch, it is assigned an FC ID. By default, the persistent FC ID feature is enabled. If this feature is disabled, the following consequences apply:

• An N or NL port logs into a Cisco MDS 9000 Family switch. The WWN of the requesting N or NL port and the assigned FC ID are retained and stored in a volatile cache. The contents of this volatile cache are not saved across reboots.
• The switch is designed to preserve the binding FC ID to the WWN on a best-effort basis. For example, if one N port disconnects from the switch and its FC ID is requested by another device, this request is granted and the WWN with the initial FC ID association is released.
• The volatile cache stores up to 4000 entries of WWN to FC ID binding. If this cache is full, a new (more recent) entry overwrites the oldest entry in the cache. In this case, the corresponding WWN to FC ID association for the oldest entry is lost.
• The switch connection behavior differs between N ports and NL ports:
  • N ports receive the same FC IDs if disconnected and reconnected to any port within the same switch (as long as it belongs to the same VSAN).
  • NL ports receive the same FC IDs only if connected back to the same port on the switch to which they were originally connected.

To release a fabric lock, issue the clear fcdomain session vsan command in EXEC mode using a login ID that has administrative privileges.

switch# clear fcdomain session vsan 10

You can display the status of CFS distribution for allowed domain ID lists using the show fcdomain status command.

switch# show fcdomain status
CFS distribution is enabled

You can display the pending configuration changes using the show fcdomain pending command:

switch# show fcdomain pending vsan 10
Pending Configured Allowed Domains
----------------------------------
VSAN 10
Assigned or unallowed domain IDs: 1-9,24,100,231-239.
[User] configured allowed domain IDs: 10-230.

You can display the differences between the pending configuration and the current configuration using the show fcdomain pending-diff command.

switch#show fcdomain pending-diff vsan 10
Current Configured Allowed Domains
----------------------------------
VSAN 10
Assigned or unallowed domain IDs: 24,100.
[User] configured allowed domain IDs: 1-239.
Pending Configured Allowed Domains
----------------------------------
VSAN 10
Assigned or unallowed domain IDs: 1-9,24,100,231-239.
[User] configured allowed domain IDs: 10-230.

You can display the status of the distribution session using the show fcdomain session-status vsan command.

switch# show fcdomain session-status vsan 1
Last Action: Distribution Enable
Result: Success