This topic covers configuration considerations to reduce or eliminate output disruptions due to controller switchovers when there is inter-controller communication.
When a controller switches over to its redundant partner, it can take several seconds to re-establish communication with other controllers. The general behavior during this brief time period is for parameters residing in the controller switching over to be seen by other controllers as maintaining their previous value. Similarly, parameters residing in the other controllers are seen by the controller switching over as maintaining their value.
The precise behavior during this brief time period is a function of the configuration technique used. There is a possibility of experiencing an output disruption as a result of a controller switchover. If you are susceptible to an output disruption, simple configuration changes can be made to prevent the disruption.
Determining if you are susceptible to an output disruption
One way to tell if you are susceptible is to perform switchover testing under simulated normal operating conditions.
Another way to tell is to consider whether you have used certain configuration techniques. Refer to the How particular configuration techniques are impacted by controller switchovers section.
Preventing an output disruption due to a controller switchover
-
Select the Abort on Read Errors option in ALGO_OPTS in Condition function blocks. A read error occurs when a parameter in another controller cannot be read due to a communication error. When the Abort on Read Errors option is used and there is a read error, PRE_OUT_D of the Condition block retains its last value. The status of PRE_OUT_D and OUT_D remains GoodNonCascade during read errors when the expression aborts.
-
Alternatively, leave the Abort on Read Errors option deselected and change ERROR_OPT to Use Last. When ERROR_OPT is Use Last, OUT_D retains its last value when there is a read error, but the status of OUT_D changes to BadNoCommNUV.
-
When the interlock or tracking inputs are sourced in other controllers, avoid the use of the Passive if Bad and Track if Bad options in the Device Control and PID function blocks.
Expressions in SFC transitions and step actions do not have an option to abort. Such expressions always abort on read errors, which mean transitions will evaluate as False and step actions will not be executed.
How particular configuration techniques are impacted by controller switchovers
- Reading data
- There are several different ways to read data into a module from another module. The configuration technique is the same whether the other module is in this controller or another controller. But the runtime mechanism is different when the data originates in another controller.
At runtime if the parameter being read is in the same controller, it is read directly at the time the function block or external reference parameter executes. But if the parameter being read is in another controller, the function block or external reference parameter reads from a proxy in the local controller, which is updated asynchronously by the other controller.
- Block 1 in MODULE_B represents a Calc or Action function block. When the expression reads MODULE_A/PARAM1, it reads it locally from the proxy in Controller B and stores it as a cached variable in the block. The proxy in Controller B returns a NoComm status for a few seconds when Controller A switches over. When the expression does a read and the proxy’s status is NoComm, a read error has occurred and the expression does not update the cached value in the block for PARAM1. This is independent of the Abort on Read Errors option. Expression blocks use the last communicated value when communication with other controllers is disrupted. This applies not only to Calc and Action function blocks, but also to Condition function blocks, SFC transitions and SFC step actions.
-
Block 2 in MODULE_B represents a Condition function block. With default parameter values in the Condition block a read error results in OUT_D of the Condition block being 0. Therefore, if the expression normally evaluates to True, a switchover in Controller A or any network disruption will cause OUT_D to change from 1 to 0, which may result in an output change depending on the configuration. Selecting the ‘Abort on Read Errors’ option in ALGO_OPTS of the Condition block causes PRE_OUT_D to retain its last value when there is a read error. The status of PRE_OUT_D and OUT_D remains GoodNonCascade during read errors when the expression aborts.
Another technique to maintain OUT_D during a switchover or network disruption is to change ERROR_OPT to Use Last. In this case the status on OUT_D is BadNoCommNUV during read errors. This technique allows the output to hold value yet let downstream logic know that a read error has occurred. The Abort on Read Errors option in ALGO_OPTS takes precedence over the ERROR_OPT setting.
-
Block 3 in MODULE_B represents an external reference parameter to PARAM1 in MODULE_A. Block 4 represents any downstream function block. When communication is established between the controllers, Block 4 pulls PARAM1 from the proxy in Controller B when Block 4 executes, essentially reading through the external reference parameter to update the wired parameter in Block 4. If the proxy for PARAM1 has a NoComm status due to a switchover or other network disruption, Block 4 does not update the value of its wired parameter, but does change its status to BadNoCommNUV. External reference parameters can be said to hold last value during switchovers and network disruptions because of this behavior.Note
This may not appear to be the case when looking at the external reference parameter’s Set Value dialog in Control Studio on-line during a network disruption. If the referenced parameter is in another controller, the dialog will show a value of 0 and status of Bad. But the value is not 0. The default value of 0 on the dialog is simply not updated when the proxy’s status is NoComm. You must look at the right end of the wire to see the actual value.
-
During network interruptions SFC transitions and SFC step actions use the last communicated values for parameters read from other controllers. But there are other considerations. During a read error, an SFC transition expression always aborts. In this case the expression continues to evaluate to False even though the last communicated values may otherwise result in a True expression. SFC step actions are similar in that read errors result in an error on the action and its assignment statements are not executed.
-
Blocks 1 and 2 - Expression blocks (Calc, Action, Condition) have special behavior when they begin to execute in a newly active controller after a switchover. When the block executes after a switchover, it will not execute its expression until all references are resolved. During the few seconds it takes to create the proxy, the expression appears to maintain the last value because it doesn’t actually read a value of zero.
NoteWhile the proxy is being created, a referencing SFC transition will continue to evaluate to False and a referencing step action will not be executed.
- Block 3 - When an external reference parameter reads from a proxy being created after a switchover of the local controller, it too reads a zero. But the zero value does not get transferred to the destination parameter, so the right end of the wire maintains its value.
The impact of inter-controller communication behavior following a switchover is the same whether the controller switching over is the one doing the reading or the other controller.
- Writing data
- There are several different ways to write data to another module. As with reads, the configuration technique is the same whether the other module is in this controller or another controller, but the runtime mechanism is different.
When the parameter being written is in the same controller, it is written synchronously at the time of execution. The write is done asynchronously by another process if the parameter is in another controller. There is no opportunity for a write to cause an unintended output disruption. If the write succeeds, the parameter is updated with the intended value. If the write fails, the parameter retains its previous value.
When a write to another controller is done as a result of wiring to an external reference parameter, a write is queued up each time the module executes. During the brief time communication is being reestablished after a controller switchover, the write is essentially delayed. The same is true for a step action with a non-stored qualifier. As long as the step is active, the write will remain queued. This is also true with a pulse qualifier on the action. A pulse action is done once if it succeeds. But if the write cannot be done because the controllers are not communicating, a pulse action retries until it succeeds (provided the step remains active).