Cisco Prime Access Registrar 9.1 User Guide

Replication Data Flow

The following sections describe data flow on the master server and the slave server:

• Master Server
• Slave Server

Master Server

The master server or primary server is the fully configured machine that is used to archive all the transactions that taken place in Prime Access Registrar.

Performing the Data Flow for the Master Server

To perform data flow for the master server:

Step 1 The administrator makes a change to the master server’s configuration using the aregcmd command line interface (CLI) and issues a save command.

Step 2 After the changes are successfully validated, the changes are stored in the Prime Access Registrar database.

Step 3 aregcmd then notifies the Prime Access Registrar server executing on the master of the configuration change.

Step 4 The Prime Access Registrar server then updates its version of the configuration stored in memory. (This is called hot-config because it happens while the server is running and processing requests.)

Step 5 The Prime Access Registrar server first copies the changes pertaining to the aregcmd save, also known as a transaction to its replication archive, then transmits the transaction to the slave server for processing.

Step 6 In aregcmd, the prompt returns indicating that the save has completed successfully, the transaction has been archived, and the transaction has been transmitted to the slaves.

Slave Server

The slave server or secondary server is a fully-installed and configured system stands ready to takeover when the primary machine fails.

Performing Data Flow for the Slave Server

To perform data flow for the slave server:

Step 1 When the slave server receives the transaction, its contents are verified.

Step 2 After verification, the changes are applied to the slave server's database.

Step 3 The changes are then applied (hot-configured) in the slave server's in-memory configuration.

Step 4 The transaction is written to the slave server's replication archive.

Security

Replication has two primary security concerns:

1. Security of the transactions transmitted to the slave server

2. Storage of transactions in the replication archive

Both of these concerns use shared secret (MD5) encryption via the shared secret specified in the replication configuration on both master and slave servers. Replication data transmitted between master and slave is encrypted at the source and decrypted at the destination the same way as standard RADIUS packets between Prime Access Registrar's clients and the Prime Access Registrar server. Transactions written to the replication archive are also encrypted in the same manner and decrypted when read from the replication archive.

Replication Archive

The replication archive serves two primary purposes:

• To provide persistent, or saved, information regarding the last successful transaction
• To persist transactions in case the slave server requires re synchronization (see Ensuring Data Integrity below for more information on re synchronization).

The replication archive is simply a directory located in../CSCOar/data/archive. Each transaction replicated by the master is written to this directory as a single file. The name of each transaction file is of the form txn########## where ########## is the unique transaction number assigned by the master server. The replication archive size, that is the number of transaction files it might contain, is configured in the Replication configuration setting of TransactionArchiveLimit. When the TransactionArchive limit is exceeded, the oldest transaction file is deleted.

Ensuring Data Integrity

Prime Access Registrar's configuration replication feature ensures data integrity through transaction data verification, transaction ordering, automatic resynchronization and manual full-resynchronization. With the single exception of a manual full-resynchronization, each of the following techniques help to automatically ensure that master and slave servers contain identical configurations. A detailed description of each technique follows. This section contains the following topics:

• Transaction Data Verification
• Transaction Order
• Automatic Resynchronization

Transaction Data Verification

When the master prepares a transaction for replication to a slave, the master calculates a 2's complement Cyclic Redundancy Check (CRC) for each element (individual configuration change) in the transaction and for the entire transaction and includes these CRC values in the transmitted transaction. When the slave receives the transaction, the slave calculates a CRC for each transaction element and for the entire transaction and compares its own calculated values with those sent with the message. If a discrepancy occurs from these comparisons, the transaction element or the entire transaction is discarded and a re-transmission of that particular transaction element or the entire transaction is requested by the slave from the master. This process is called automatic resynchronization. (described in more detail below)

Transaction Order

When the master prepares a transaction for replication, it assigns the transaction a unique transaction number. This number is used to ensure the transactions are processed by the slave in exactly the same order as they were processed on the master. Transactions are order dependent. Since the functionality of Prime Access Registrar's configuration replication feature is to maintain identical configurations between master and slave, if transaction order were not retained, master and slave would not contain identical configurations. Consider where two transactions modify the same thing (a defined client's IP address for example). If the first transaction was a mistake and the second was the desired result, the client configuration on the master would contain the second setting; however, if the transactions were processed in the reverse order on the slave, the client configuration on the slave would contain the mistaken IP Address. This example illustrates the critical need for transaction ordering to ensure data integrity.

Automatic Resynchronization

Automatic Resynchronization is the most significant feature with respect to data integrity. This feature ensures the configurations on both the master and slave are identical. If they are not, this feature automatically corrects the problem.

When the master and slave start-up, they determine the transaction number of the last replication transaction from their respective replication archives. The master immediately begins periodic transmission of a TransactionSync message to the slave. This message informs the slave of the transaction number of the transaction that the master last replicated.

If the transaction number in the TransactionSync message does not match the transaction number of the last received transaction in the slave's archive, then the slave will request resynchronization from the master. The resynchronization request sent by the slave will include the slave's last received transaction number.

The master will respond by retransmitting each transaction since the last transaction number indicated by the slave in the resynchronization request. The master obtains these transactions from its replication archive.

Should the slave's last received transaction number be less than the lowest transaction number in the master's replication archive, then automatic resynchronization cannot occur as the master's replication archive does not contain enough history to synchronize the slave. In this case, the slave must be resynchronized with a full-resynchronization.

Full Resynchronization

Full Resynchronization means that the slave has missed more transactions than are stored in the master's replication archive and cannot be resynchronized automatically. There is no automatic full-resynchronization mechanism in Prime Access Registrar's configuration replication feature. To perform a full resynchronization, see Full Resynchronization.

Understanding Hot-Configuration

Hot-Configuration is the process of reflecting configuration changes made to Prime Access Registrar's internal configuration database in the in-memory configuration of the executing Prime Access Registrar server. Hot-Configuration is accomplished without interruption of RADIUS request processing. For example, if an administrator uses aregcmd to configure a new client and issues a save command, when the prompt returns, the newly configured client can send requests to Prime Access Registrar.

Hot-Configuration minimizes the down-time associated with having to restart an Prime Access Registrar server to put configuration changes into effect. With the Hot-Configuration feature, a restart is only necessary when a Session Manager, Resource Manager or Remote Server configuration is modified. These configuration elements might not be hot-configured because they maintain state (an active session, for example) and cannot be modified without losing the state information they maintain. Changes to these configuration elements require a restart of Prime Access Registrar to put them into effect.

Hot-Configuration’s only connection to the replication feature is that when a change is replicated to the slave, the slave is hot-configured to reflect the replicated change as if an administrator had used aregcmd to make the changes directly on the slave server.

Note Any misconfiguration of replication will impact the performance of Prime Access Registrar. Hence manual resync is required to rectify the misconfiguration.

Replication’s Impact on Request Processing

The replication feature was designed to perform replication of transactions with minimal impact on RADIUS request processing. When a transaction is received by a slave, RADIUS requests are queued while the transaction is applied to the slave. After the transaction is complete, RADIUS request processing resumes.

The impact on RADIUS request processing is a direct result of the size of a transaction. The smaller the transaction the lesser the impact, and the larger the transaction, the greater the impact. In other words, when making changes to the master, frequent saves are better than making lots of changes and then saving. Each change is one transaction element and all changes involved in a save comprise a single transaction with one element per change. Since the replication feature only impacts RADIUS request processing when changes are made, the impact under normal operation (when changes are not being made) is virtually unmeasurable.

RepType

RepType indicates the type of replication. The choices available are SMDBR and NONE.

When RepType is set to NONE, replication is disabled. To enable replication, set RepType to SMDBR for Single Master DataBase Replication. RepType must be set to SMDBR on both the master and slave servers.

RepTransactionSyncInterval

Master

On the master server, RepTransactionSyncInterval is the duration between periodic transmission of the TransactionSync message expressed in milliseconds. The default is 60000 or 1 minute.

The purpose of RepTransactionSyncInterval is to indicate how frequently to check for an out-of -sync condition between the master and slave servers. When the slave received the TransactionSync message, it uses its contents to determine if it needs to resynchronize with the master.

The larger the setting for RepTransactionSyncInterval, the longer the period of time between out-of-sync detection. However, if RepTransactionSyncInterval is set too small, the slave can frequently request resynchronization when it is not really out of sync. If the duration is too small, the slave cannot completely receive a transaction before it receives the TransactionSync message. In this case, the servers will remain synchronized, but there will be unnecessary excess traffic that could affect performance.

Note We recommend that you use smaller values for the RepTransactionSyncInterval to limit the time a slave server is offline applying change sets during automatic resynchronization.

Slave

On the slave, RepTransactionSyncInterval is used to determine if the slave has lost contact with the master and to alert administrators of a possible loss of connectivity between the master an slave. If the elapsed time since the last received TransactionSync message exceeds the setting of RepTransactionSyncInterval, the slave writes a log message indicating that it might have lost contact with the master. This log message is repeated each TransactionSyncInterval until a TransactionSync message is received.

RepTransactionArchiveLimit

On both master and slave, the RepTransactionArchiveLimit setting determines how many transactions can be stored in the archive. The default setting is 100. When the limit is exceeded, the oldest transaction file is deleted. If a slave requires resynchronization and the last transaction it received is no longer in the archive, a full resynchronization will be necessary to bring the slave back in sync with the master.

Note The value set for RepTransactionArchiveLimit should be the same on the master and the slave.

An appropriate value for RepTransactionArchiveLimit depends upon how much hard disk space an administrator can provide for resynchronization. If this value is large, say 10,000, then the last 10,000 transactions will be stored in the archive. This is like saying the last 10,000 saves from aregcmd will be stored in the archive. Large values are best. The size of each transaction depends upon how many configuration changes were included in the transaction, so hard disk space usage is difficult to estimate.

Note We recommend that you use smaller values for the RepTransactionArchiveLimit to limit the time a slave server is offline applying change sets during automatic resynchronization.

If the slave should go down or otherwise be taken off line, the value of RepTransactionArchiveLimit and the frequency of aregcmd saves will determine how long the slave can be offline before a full-resynchronization will be required.

There are two reasons why a slave server should have an archive:

1. The slave must save the last received transaction for resynchronization purposes (at a minimum).

2. Should the master go down, the slave can then be configured as the master and provide resynchronization services to other slaves.

RepIPAddress

The RepIPAddress value is set to the IP Address of the machine containing the Prime Access Registrar installation.

Note The IP address format is enhanced to support both IPv4 and IPv6.

RepPort

The RepPort is the port used to receive of replication messages. In most cases, the default value (1812) is sufficient. If another port is to be used, the interfaces must exist in the machine.

RepSecret

RepSecret is the replication secret shared between the master and slave. The value of this setting must be identical on both the master and the slave.

RepIsMaster

The RepIsMaster setting indicates whether the machine is a master or a slave. On the master, set RepIsMaster to TRUE. On the slave set it to FALSE. Only the master can have this value set to TRUE and there can be only one master.

RepMasterIPAddress

RepMasterIPAddress specifies the IP Address of the master. On the master, set RepMasterIPAddress to the same value used in RepIPAddress above. On the slave, RepMasterIPAddress must be set to the IP Address of the master.

Note The IP address format is enhanced to support both IPv4 and IPv6.

RepMasterPort

RepMasterPort is the port to use to send replication messages to the master. In most cases, the default value (1812) is sufficient; however, if another is to be used, the interfaces must exist in the machine.

Rep Members Subdirectory

The Rep Members\ subdirectory contains the list of slaves to which the master will replicate transactions.

Rep Members/Slave1

Each slave is added much like a client is added. Each slave must have a configuration in the Rep Members directory to be considered part of the replication network by the master. The master will not transmit any messages or replications to servers not in this list, and any communication received by a server not in this list will be ignored.

Note Although it is possible to configure multiple slaves with the same master, we have only considered a single-master/single-slave configuration. This is the recommended configuration.

Name

This is the name of the slave. The name must be unique.

IPAddress

This is the IP Address of the slave.

Note The IP address format is enhanced to support both IPv4 and IPv6.

Port

This is the port upon which the master will send replication messages to the slave.

Configuring The Master

On the master server, RepTransactionSyncInterval is the duration between periodic transmission of the TransactionSync message expressed in milliseconds. The default is 60000 or 1 minute.

Configuring the Master Server for Replication

To configure the master server for replication:

Step 1 On the machine which is to be the master, using aregcmd, navigate to //localhost/Radius/Replication

Step 2 Set the RepType to SMDBR:

set RepType SMDBR

Step 3 Set the RepIPAddress to the IP address of the master:

set RepIPAddress 192.168.1.1

Step 4 Set the RepSecret to MySecret:

set RepSecret MySecret

Step 5 Set RepIsMaster to TRUE:

set RepIsMaster TRUE

Step 6 Set RepMasterIPAddress to the same value used in step 3:

set RepMasterIPAddress 192.168.1.1

Step 7 Change directory to /Radius/Advanced and set the MaximumNumberOfRadiusPackets property to 8192:

cd /Radius/Advanced

set MaximumNumberOfRadiusPackets 8192

Step 8 Change directory to Rep Members:

cd “rep members”

Note You must enclose Rep Members in quotes due to the space in the name.

Step 9 Add member1:

add member1

Step 10 Change directory to member1:

cd member1

Step 11 Set the IPAddress to the IP Address of the machine to be the member:

set IPAddress 192.168.1.2

Note The RepPort and RepMasterPort properties on the Master must correspond to one of the ports configured in /Radius/Advanced/Ports, if one is configured. Otherwise, the default values for the RepPort and RepMasterPort properties are sufficient.

Step 12 Save the configuration:

save

Step 13 Reload the configuration:

reload

Configuring The Member

On the slave, RepTransactionSyncInterval is used to determine if the slave has lost contact with the master and to alert administrators of a possible loss of connectivity between the master an slave.

Configuring the Member Server for Replication

To configure the member server for replication:

Step 1 On the machine which is to be the member, using aregcmd, navigate to //localhost/Radius/Replication.

Step 2 Set the RepType to SMDBR.

set RepType SMDBR

Step 3 Set the RepIPAddress to the IP address of the member.

set RepIPAddress 192.168.1.2

Step 4 Set the RepSecret to MySecret.

set RepSecret MySecret

Step 5 Set RepMasterIPAddress to IP Address of the master (the same value used in Step 3 on page 8-1).

set RepMasterIPAddress 192.168.1.1

Step 6 Change directory to /Radius/Advanced and set the MaximumNumberOfRadiusPackets property to 8192.

cd /Radius/Advanced

set MaximumNumberOfRadiusPackets 8192

Step 7 If the Master has been configured to use a port other than the well-known (and default) RADIUS ports, configure each Member to use the same port.

Note The RepPort and RepMasterPort properties on the Master must correspond to one of the ports configured in /Radius/Advanced/Ports, if one is configured. Otherwise, the default values for the RepPort and RepMasterPort properties are sufficient.

Step 8 Save the configuration:

save

Step 9 Reload the configuration:

reload

Verifying the Configuration

After both servers have successfully started, use aregcmd to make a small change to be replicated to the member server which you can easily verify. We recommend setting the description in //localhost/Radius to something like Test1. After you issue an aregcmd save and the prompt returns, run aregcmd on the member server and change directory to //localhost/Radius. Ensure that the description is set to Test1. If this was successful, then replication is properly configured and functional.

Adding a User

The Users object contains all of the information necessary to authenticate a user or authorize a user. Users in local UserLists can have multiple profiles. On the master server, use aregcmd to add a new user to the default user list.

Adding a New User

To add a new user:

Step 1 Change directory to //localhost/Radius/UserLists/Default.

Step 2 Enter the following:

add testuser

Step 3 Change directory to testuser.

cd testuser

Step 4 Set the password for testuser.

set password testuser

Step 5 Confirm the password by entering testuser again.

Step 6 Enter save to save the configuration.

Master Server’s Log

The log on the master shows the following:

Member Server’s Log

The log on the member shows the following:

Verifying Replication

You can use one of two methods to verify that the new user testuser was properly replicated to the member:

• Run aregcmd on the member and look at the default userlist to see if it is there.
• Run radclient on the member and enter simple testuser testuser to create a simple access request packet (p001).

Enter p001 send to send it. When it returns with p002, enter p002 to see if it is an Access Accept packet or an Access Reject packet. If it is an Access Accept, the user was properly replicated to the member. Using radclient is the recommended method to validate that a user was properly replicated.

On the Master, use aregcmd to delete the user from the default user list and save the user list.

Master Server’s Log

The log on the master shows the following:

Member Server’s Log

The log on the member shows the following:

Repeat the validation procedure above to ensure the user testuser is no longer present on the member.

Using aregcmd -pf Option

Prime Access Registrar's replication feature works well using aregcmd input files. An aregcmd input file contains a list of aregcmd commands. For example, if the initial configuration of Prime Access Registrar were constructed in an input file, the master and member could be configured for replication first, then the input file applied to the master will be automatically replicated to the member.

Using aregcmd -pf Option

To illustrate replication using an aregcmd input file:

Step 1 Create a text file called add5users with the following commands:

add /Radius/UserLists/Default/testuser1

cd /Radius/UserLists/Default/testuser1

set password testuser1

add /Radius/UserLists/Default/testuser2

cd /Radius/UserLists/Default/testuser2

set password testuser2

add /Radius/UserLists/Default/testuser3

cd /Radius/UserLists/Default/testuser3

set password testuser3

add /Radius/UserLists/Default/testuser4

cd /Radius/UserLists/Default/testuser4

set password testuser4

add /Radius/UserLists/Default/testuser5

cd /Radius/UserLists/Default/testuser5

set password testuser5

save

Step 2 On the master server, run the following command:

aregcmd -pf add5users

Master Server’s Log

The log on the master shows the following:

Member Server’s Log

The log on the member shows the following:

When the prompt returns, go to the member and use aregcmd to view the /radius/defaults/userlist. There should be five users there named testuser1 through testuser5.

An Automatic Resynchronization Example

This example will illustrate resynchronization of the member. This will be accomplished by stopping the member, making changes on the master, then restarting the member forcing a resynchronization.

Performing Resynchronization of the Member

To perform resynchronization of the member:

Step 1 At the member, stop the Prime Access Registrar server:

/etc/init.d/arservagt stop

At the master, run aregcmd and change directory to /radius/userlist/default.

cd /radius/userlist/default

Step 2 Enter the following:

add foouser

Step 3 Change directory to foouser.

cd foouser

Step 4 Set the password for foouser.

set password foouser

Step 5 Confirm the password by entering foouser again.

Step 6 Save the configuration:

save

Master Server’s Log

The log on the master shows the following:

On the member, run /etc/init.d/arservagt start. Notice the following log messages in the Master's log:

Member Server’s Log

The log on the member shows the following:

As the log above shows, when the member started up, it validated its last received transaction number (#3) with the master's last replicated transaction number (#4). They did not match because a replication was initiated by the master which was not received by the member (because the member was stopped). When the member detected this discrepancy, the member made a resynchronization request to the master. The master responded by transmitting the missed transaction (#4) to the member. After it received and processed the retransmitted transaction, the member determined that it was then synchronized with the master and placed itself in an online status.