Fast Start Failover Issues

Enable/disable:

Prerequ.: Flashback needs to be enabled on primary and failover target:

SQL> select NAME, OPEN_MODE, FLASHBACK_ON from v$database;

NAME      OPEN_MODE            FLASHBACK_ON
--------- -------------------- ------------------
ITSDBTST  READ ONLY WITH APPLY YES

Enable if flashback is set to OFF:

SQL> alter database flashback ON;

Enable fast_start failover in DataGuard:

DGMGRL> enable fast_start failover
Enabled in Potential Data Loss Mode.

DGMGRL> show fast_start failover
Fast-Start Failover: Enabled in Potential Data Loss Mode
  Protection Mode:    MaxPerformance
  Lag Limit:          30 seconds
  Threshold:          60 seconds
  Active Target:      itsdbtst2
  Potential Targets:  "itsdbtst2"
    itsdbtst2  valid
  Observers:      (*) obs_two
                      obs_one
  Shutdown Primary:   TRUE
  Auto-reinstate:     TRUE
  Observer Reconnect: (none)
  Observer Override:  FALSE
Configurable Failover Conditions
  Health Conditions:
    Corrupted Controlfile          YES
    Corrupted Dictionary           YES
    Inaccessible Logfile            NO
    Stuck Archiver                  NO
    Datafile Write Errors          YES
  Oracle Error Conditions:
    (none)

Set fast_start failover target if missing (enable fails):

On primary:
DGMGRL> edit database <primary_db> set property faststartfailovertarget = '<standby_db>';

On standby:
DGMGRL> edit database <standby_db> set property faststartfailovertarget = '<primary_db>';

Update properties, e.g. failover threshold, if necessary:

DGMGRL> EDIT CONFIGURATION SET PROPERTY FastStartFailoverThreshold = '90';
Property "faststartfailoverthreshold" updated

Issue: Unexpected shutdown primary db (ORA-16830)

Primary database is shutting down during reboot of standby db during scheduled maintenance, and restarts in mount state.

Alert log primary:

Primary has heard from neither observer nor target standby within FastStartFailoverThreshold seconds.
It is likely an automatic failover has already occurred. Primary is shutting down.
...
ORA-16830: primary isolated from fast-start failover partners longer than FastStartFailoverThreshold seconds: shutting down

Root Cause

DataGuard was not able to contact observer and target standby for longer than the configured FastStartFailoverThreshold property (default 30 sec, reboot might take longer).

To avoid any brain-split scenario with both databases opened read-write, the DG broker will shutdown the primary in this case (if FastStartFailoverPmyShutdown=TRUE).

Please note (https://docs.oracle.com/cd/E11882_01/server.112/e40771/dbpropref.htm#DGBKR3440):

Setting FastStartFailoverPmyShutdown=FALSE “… will not prevent the primary database from shutting down if a fast-start failover occurred because a user configuration condition was detected or was requested by an application by calling the DBMS_DG.INITIATE_FS_FAILOVER function.”

Solution

Check/update FSFO configuration:

DGMGRL> show fast_start failover

Fast-Start Failover: Enabled in Potential Data Loss Mode

  Protection Mode:    MaxPerformance
  Lag Limit:          30 seconds
  Threshold:          30 seconds
  Active Target:      itsdbtst2
  Potential Targets:  "itsdbtst2"
    itsdbtst2  valid
  Observers:      (*) obs_one
                      obs_two
  Shutdown Primary:   TRUE
...
DGMGRL> edit configuration set property faststartfailoverthreshold=60
DGMGRL> edit configuration set property faststartfailoverlaglimit=60

For details see Oracle DataGuard Broker documentation

Check Oracle status on both nodes:

SQL> SELECT INSTANCE_NAME, STATUS, DATABASE_STATUS FROM V$INSTANCE;
SQL> SELECT NAME, DATABASE_ROLE, OPEN_MODE FROM V$DATABASE;

Usually primary was started but not opened (“Data Guard: detected that a role change has occurred or is likely to have occurred. This is an old primary in need of reinstatement. Database will not be opened.” in alert log), standby db not started.

Start standby db and wait until broker auto-reinstated old primary database.

Alternative: Check log apply status (see Verify log shipping from Primary to Standby). Shutdown/startup mount both databases. Open primary (ALTER DATABASE OPEN). Open standby (ALTER DATABASE OPEN READ ONLY), check log apply.

Comments

No comments yet. Why don’t you start the discussion?

Leave a Reply

Your email address will not be published. Required fields are marked *