data guard failover steps

In order to accommodate all load conditions, Oracle recommends having at least one more SRL group than the number of ORL groups of the same size. However failing over to a snapshot standby database will require more time because the broker must first convert it back to a physical standby database. Failover automation ensures a seamless transition from the primary database to a synchronized standby database in cases of failure, while ensuring database availability by replaying uncommitted in-flight transactions. If they are isolated from each other, then you must first disable fast-start failover by using the FORCE option, and then stop the observer. 4. How To Setup Dataguard Broker Configuration (DG Broker) In 19c The broker never automatically reinstates the former primary database if a fast-start failover was initiated because a user configuration condition was detected or was requested by an application calling the DBMS_DG.INITIATE_FS_FAILOVER function. Reinstatement is supported only after failover in a broker configuration. In these sample commands, the ellipse () signifies any other add service options you wish to supply. files are stored in subdirectories of the DG_ADMIN directory. Since a fast-start failover (automatic failover) could become a false failover when the observer or the standby database cannot connect to the primary database within a specific time, which may cost the database to lose some transactions followed by reinstating or recreating the standby database (the former primary database). In order to fully automate switchover, Broker needs SYSDBA credentials in order to restart one or both databases. Reconnect within the time specified by the FastStartFailoverThreshold property. After the restart, Redo Apply begins applying redo data from the new primary So SALESRW will start on CHICAGO (which is now the primary) and SALESRO will start on BOSTON (which is now the physical standby). $DG_ADMIN/config_ConfigurationSimpleName/callout For example: In the following example, assume the network between the primary database and the observer has failed. 11.2 rac servicefailover 2020-01-28 ORACLE ORACLE RAC/ASM RAC112. The FastStartFailoverTarget configuration property on the primary unless the new property value contains the current fast-start failover target. If the configured data loss guarantee cannot be upheld, file also declares broker configurations and defines configuration Look for the desired data in the RAM. Application calls to DBMS_DG.INITIATE_FS_FAILOVER. Always try to perform a complete failover first unless redo apply has stopped at the failover target due to an ORA-752 or ORA-600 [3020] error. If it reconnects to the primary database before the standby agrees to fail over, then the master observer will stop attempting to initiate a fast-start failover. To disable fast-start failover, use the Fast-Start Failover wizard in Cloud Control or the DGMGRL DISABLE FAST_START FAILOVER [FORCE] command. In maximum availability mode, the behavior depends on the value of the There are configuration requirements that must be met in order to publish and properly handle FAN events generated as the result of a broker-managed failover. Data Guard Failover to physical standby Tips - dba-oracle.com Although the default value of 30 seconds is typically adequate for detecting outages and failures on most configurations, you can adjust failover sensitivity with this property to decrease the probability of false failovers in a temporarily unstable environment. In the following example, a service named sales is configured to be active in the PHYSICAL_STANDBY role on the primary database NORTH. If no value is specified for the The following sections provide more information about the fast-start failover environment: When Fast-Start Failover Is Enabled and the Observer Is Running, Restrictions When Fast-Start Failover is Enabled, Shutting Down the Primary Database When Fast-Start Failover Is Enabled, Performing Manual Role Changes When Fast-Start Failover Is Enabled. In addition to setting the configuration protection mode to maximum performance, you will also need to ensure that the LogXptMode database property for both the primary and target standby database is set to ASYNC. The terminal session will appear to hang at this point. You want to prevent fast-start failover from occurring because the primary database will resume service soon. Enabling Fast-Start Failover Task 1: Determine Which of the Available Standby Databases is the Best Target for the Failover, Enabling Fast-Start Failover Task 2: Specify Target Standbys with the FastStartFailoverTarget Configuration Property, Enabling Fast-Start Failover Task 3: Determine the Protection Mode You Want, Enabling Fast-Start Failover Task 4: Set the FastStartFailoverThreshold Configuration Property, Enabling Fast-Start Failover Task 5: Set Other Properties Related to Fast-Start Failover (Optional), Enabling Fast-Start Failover Task 6: Enable Additional Fast-Start Failover Conditions (Optional), Enabling Fast-Start Failover Task 7: Using DGMGRL or Cloud Control, Enabling Fast-Start Failover Task 8: Start the Observer, Enabling Fast-Start Failover Task 9: Verify the Fast-Start Failover Environment. However, failover is attempted if the ObserverOverride configuration property is set to TRUE. Fast-start failover will not occur unless all instances comprising the Oracle RAC primary database are perceived to have failed. For example: Using DGMGRL, you can do this by examining the output of the SHOW CONFIGURATION LAG. Switching over to a logical standby database results in the snapshot and physical standby databases in the broker configuration being disabled by the broker, making these databases no longer viable as standby databases. Refer to the appropriate Oracle RAC or Oracle Restart documentation for further information. There are two types of failover operations: Graceful or "no-data-loss" failover and Forced or "minimal-data-loss" failover. It is then configured to be active in the PHYSICAL_STANDBY role on the physical standby database SOUTH. Issue the DISABLE FAST_START FAILOVER command or the DISABLE FAST_START FAILOVER FORCE command. Synopsis. Running a StatusReport on the primary should verify that the error is due to a missing observer. Transitions the target standby database into the primary role, opens the new primary database in read/write mode, and starts redo transport services. 11.2 rac servicefailover - FAN events are always published through ONS. See Troubleshooting Problems During a Switchover Operation for more information. It is important that all SRVCTL add service options be identical on all the databases so that the services behave the same way before and after a role change. Here's a one-liner observer startup for *nix. Create a unique connect alias for each database. The broker reinstates a failed primary database as a standby database of the same type (physical or logical standby database) as the old standby database. To enable fast-start failover in Cloud Control, use the Fast-Start Failover wizard. This property also affects whether the broker skips viability checks of bystander standby databases when a fast-start failover occurs. The name of the callout configuration file is fsfocallout.ora. If the client uses remote ONS subscription, the client must specify the hostname and port of the ONS daemon(s) of the primary database and each standby database. Displays when the target standby database does not have all of the primary database redo data and the configuration is operating in maximum availability mode. See Oracle Data Guard Concepts and Administration for information about tuning the log apply rate for a physical standby database. To verify the observer is started and the configuration is ready for If you want the broker to skip this viability check of bystander standby databases during a complete failover, thus decreasing the overall failover time, set the BystandersFollowRoleChange configuration property to NONE. The broker allows an immediate failover to proceed even if there are errors present on the standby database that you selected to participate in the failover. The Oracle Database 11g observer can make use of specific credentials, allowing the same wallet to be used for multiple observers with different SYS passwords. The advanced way is in the following article: Connect-Time Failover by a Dynamic Service Name. Data Guard switchover with dgmgrl - dba-oracle.com The master observer cannot connect to the target standby database, What Happens if the Observer Fails? But before enabling Flashback Database, you must enable Flash Recovery Area (FRA). This function can be called from a connection to either the primary or any standby in the configuration. Its primary job is to perform a failover when conditions permit it to do so without violating the data durability constraints set by the DBA. You can query the V$DATABASE view to verify that the observer is started and the configuration is ready for fast-start failover. disable fast-start failover with the FORCE option on the In maximum performance mode, the ability to automatically failover is restored Once the Oracle instance is transitioned into primary database status in either switchover or failover, the life of the database as the standby ends and its service as the primary database . Any database that was disabled while multiple role changes were performed cannot be reinstated. A broker configuration can belong to multiple groups. LinkedIn:https://www.linkedin.com/in/hari-prasath-aa65bb19/ If the protection mode was at maximum availability or maximum performance, it remains unchanged. For any work, queries and help. The following is a sample observer configuration file: Since the broker configuration SALES consists of three databases, Boston, Chicago, and Dallas, with a CONNECT_ID of SALES_P, the SALES_P connect identifier must be defined such that it can reach any instance of any database within the configuration. FSFO is a feature of Broker which records information about the failover target, how long to wait after a failure before triggering a failover, and other FSFO specific properties. The same thing happens if a shutdown and startup of either database occurs - the service that is started is the one that matches the role of the database being started. Database dismounted. Switchover and Failover Operations - Oracle Help Center Note that these properties only affect whether primary shutdown and automatic reinstatement are performed if a fast-start failover occurs because the primary crashed or was isolated from the observer and target standby database. Else, broker restarts the new How we create a failover group in Azure Managed Instance There are prerequisites that must be met before the broker allows you to enable fast-start failover. Performing a Manual Failover Task 1: Determine Which of the Available Standby Databases is the Best Target for the Failover, Performing a Manual Failover Task 2: Start the Failover, Performing a Manual Failover Task 3: Reset the Protection Mode, Performing a Manual Failover Task 4: Re-establish a Disaster-Recovery Configuration. The DB_ROLE_CHANGE event will fire whenever a database is opened for the first time after a role transition. To enable fast-start failover with DGMGRL, issue the ENABLE FAST_START FAILOVER command while connected to any database in the broker configuration, including on the observer computer. If the WAIT option is included in the If fast-start failover is enabled you can still perform a switchover or a manual failover as long as certain conditions are met. With FSFO enabled, Broker expects to find an observer, which we haven't started yet, so if you verify the at this point with 'show configuration', Broker will report a warning (if it doesn't, give it a minute to discover that the observer isn't there). If the primary database can be mounted, it may be possible to flush any unsent redo data from the primary database to the target standby database using the ALTER SYSTEM FLUSH REDO SQL statement. Overall commit latency is increased by the round-trip network latency. For Oracle Database Release 12.2 and higher, Oracle Enterprise Manager Cloud Control (Cloud Control) supports configuring multiple observers using the Enterprise Manager Command Line Interface (EM CLI). If it's not, DGB will not allow the failover to continue until the DBA has manually resolved any discrepancies. By choosing the standby database with the least amount of unapplied redo, you can minimize the overall time it takes to complete the switchover operation. Fast-Start Failover in Data Guard Environments on Oracle Cloud We suggest you try the following to help find what youre looking for: This document will guide you through configuringOracle Data GuardFast-Start Failover (FSFO) using a physical standby database. post-callout script, and pre-callout success file for the broker client-side broker files, the specified values are used. You can use Cloud Control or DGMGRL, to perform either a complete (recommended) or an immediate failover. Data Guard Broker - Controls the creation and monitoring of Data Guard. Syntax for Mandatory Configuration Declaration. an alias of the broker configuration name. For this reason, you should first issue this command on the target standby database. Overview of Switchover and Failover in a Broker Environment. It also requires Flashback Database to be enabled on both the primary and target standby databases. Verify the primary database instance is open. To override this behavior and allow a fast-start failover to occur if the observer is unable to contact the primary for more than FastStartFailoverThreshold seconds, set the ObserverOverride property to TRUE. We want the observer to be able to automatically reinstate the former primary as a standby after our failover tests, so before each test, make sure that Flashback Database has at least 30 minutes of history. This can be avoided by first disabling fast-start failover with the FORCE option on the target standby. If the primary database is an Oracle Real Application Clusters (Oracle RAC) database, the master observer will attempt to connect to one of the remaining primary instances. In order for Flashback Database to succeed, there must be sufficient history available in the Flashback Database logs and all of the redo generated between the restore point and the standby_became_primary_scn must be available. You can also switch the master observer hosts for a group of configurations to one specific host. REINSTATE REQUIRED is present only after fast-start failover has occurred and shows on both the new primary database and the database undergoing reinstatement. 1,000,000 block changes on a small set of blocks generates less Flashback Database history than 1,000,000 changes on a larger set of blocks. In fact, failovers are so reliable, fast, and simple that switchovers become the exception rather than the rule. Valid values are >= 100. This document only talks about switchover involving physical standby database. This is the recommended method for disabling fast-start failover. It comes with a GUI and command line interface. At a minimum, you must set db_unique_name. If you have not used the SET ObserverConfigFile command after starting the current DGMGRL client, then the result will always be: ObserverConfigFile=observer.ora. Once fast-start failover is enabled, the broker will ensure that fast-start failover Slightly less critical than making sure you've got a good primary is making sure the failed primary can be automatically reinstated. receives redo data from a far sync instance. Valid values are >= 10. Use the callout configuration file and script This property specifies the amount of data, in seconds, that the target standby database can lag behind the primary database in terms of redo applied. See Manual Failover for complete information about manual failovers. the Steps To Congure Oracle 11g Data Guard Physical Standby associate that we give here and check . Flashback Database stores its logs in the Flash Recovery Area (FRA), so the FRA must be large enough to store at least 60 minutes of Flashback Database history. present, you must start the observer manually using the following Figure 6-2 shows the observer monitoring a fast-start failover configuration. 2. This method will disable fast-start failover on all databases in the broker configuration. The previous examples dealt with setting up only one service on a database. DGMGRL can be used to manage multiple observers in a group of broker configurations. For more details about managing redo transport services using database properties, see Managing Redo Transport Services. The time interval starts when the observer first loses its connection to the primary database. Step-by-step instructions for manual reinstatement are described in Reenabling Disabled Databases After a Role Change. pre-callout configuration script and post-callout configuration script. On the new primary database STAN, perform a SWITCH LOGFILE to start sending redo data to the standby database PRIM. In such a case, no attempt is made to transmit any unsent redo from the cascader to the terminal standby. After a failover, a bystander will not automatically become the new failover target. the primary role, use the PreferredObserverHosts configuration. It may be possible to convert the old Primary into a Standby database now instead of having to do a time consuming duplicate again. alter database recover managed standby database finish; alter database activate standby database; Managed recovery process has been stopped between primary and standby database and standby becomes primary database. See the Oracle Maximum Availability Architecture technical briefs at: When setting the FastStartFailoverLagLimit configuration property, consider these tradeoffs between performance and potential data-loss: A low lag limit will minimize data loss but may impact the performance of the primary database. Figure 6-2 The Observer in the Fast-Start Failover Environment. gets enabled and then begins monitoring. In cases where Once Flashback Database has succeeded, the observer will convert the database to a standby, bounce it, and begin apply services. Each group that you define must have at least one broker configuration. For zero data loss in maximum availability mode, the FastStartFailoverLagLimit property must be set to zero. For example, perform full level 0 backups weekly and incremental level 1 backups daily. Without the credentials, Broker will complete the role transition, but will leave the databases in need of a manual restart. Disabling fast-start failover does not stop the observer. If you do not want to use the default, you can define a specific group. For Active Oracle Data Guard, it will fail to open up a connection unless its in read-only mode. Manual failover can be performed even if the pre-condition checks are not met. If you performed a failover or switchover that requires you to re-create the failed primary database or standby databases that were disabled during the role transition, then follow the procedures in the Oracle Data Guard Concepts and Administration chapter, "Creating a Physical Standby Database" and also the Oracle Data Guard Concepts and Administration chapter, "Creating a Logical Standby Database.". File. If the value is non-zero, failover is possible any time the standby database's apply This document describes how to setup clients to connect to Data Guard databases (primary and standby) and configure automatic client failover such that in case there is role change due to switchover or . See Manual Failover for information about manual failover. The behavior of the broker if the master observer fails depends on whether the broker configuration has one observer or multiple observers. If block change tracking is enabled on the primary, and the target Note that the database will not open at this point. If any errors occur during either conversion, the broker stops the switchover. Broker Configuration Has Multiple Registered Observers. Note that the new primary database does not need to be restarted. Stores files related to the observer and callout configuration. The list is empty by default. This file is stored in the This can happen for either of the following reasons: A bystander standby database has applied more redo data than the new primary database itself had applied when it was a standby database. The ObserverOverride configuration property, when set to TRUE, allows an automatic failover to occur when the observer has lost connectivity to the primary, even if the standby has a healthy connection to the primary. 2. specified, the file is stored in an appropriate directory under the broker's Stopping the Observer When There is Only One Observer. Bystander standby databases can be shut down at any time in any order without impacting fast-start failover. directory. observer immediately begins monitoring the status and connections to Note that role changes to logical standby databases always result in physical standby database bystanders being disabled. Now it will return PRIMARY. Verify the configuration from both hosts. Then the STOP OBSERVER command can be issued successfully on the former master observer. To configure fast-start failover in observe-only mode: Fast-start failover will not be triggered if the primary or standby database is shut down normally. The following sections provide information about managing observers: How the Observer Maintains Fast-Start Failover Configuration Information, Patching an Environment When the Observer Is Running and Fast-start Failover Is Enabled. To reenable broker management of these databases, you must reinstate or re-create the databases using one of the following procedures: If a database can be reinstated, the database will show the following status: Reinstate the database using the DGMGRL REINSTATE DATABASE command or the reinstate option in Cloud Control, as described in How to Reinstate a Database. Rather, fast-start failover will be enabled in accordance with the current protection mode. 1)What are the steps to do Switchover/Failover operation manually in 2-node RAC and 2-node DATAGUARD environment. Such preparation includes: Ensuring that standby redo log files are configured on the primary database. In addition, the primary database will shut down if it perceives a loss of connectivity for a period longer than FastStartFailoverThreshold seconds, if the FastStartFailoverPmyShutdown configuration property is set to TRUE. After the database has been re-created, enable broker management of the re-created standby database by using the DGMGRL ENABLE DATABASE command. In Oracle RAC configurations, the Inaccessible Logfile and Stuck Archiver health conditions may only be applicable to a single instance. When the configuration has more than one registered observer, if the primary and target standby databases stay connected but the connection to the master observer is lost, then the broker tries to nominate a backup observer as the new master observer. orapwd file=$ORACLE_HOME/dbs/orapw$ORACLE_SID. If the primary or target standby databases lose connections to all backup observers, then the broker does not try to nominate a backup observer as the new master observer, and the broker reports that the configuration is not observed. The command SHOW OBSERVER provides detailed information about registered observers. A manual failover is already in progress. 1. operation: Example 6-1 Fast-start Failover Configuration For more information, see START OBSERVER IN BACKGROUND. files to automate tasks that must be performed before and after a fast-start failover Starting Observers as Background Processes. fsfocallout.ora and they have the required permissions. To change the FastStartFailoverTarget property to point to a different standby database, disable fast-start failover, set the FastStartFailoverTarget property, and reenable fast-start failover. 3. It's good practice to use separate listeners for application connections and Data Guard connections. These are some points to consider before you begin a switchover. ensure that it has the required permissions. To avoid the overhead of recording every change to every block, Flashback Database takes a "fuzzy" snapshot every 30 minutes and only records the before-image block upon its first change since the last snapshot. fast-start failover has not occurred to the target standby database. Instead, Oracle Clusterware opens PDBs on particular instances based on The foundation of FSFO is Data Guard - a primary and at least one standby. Issue the following commands on Primary database and Standby database to find out: To prevent automatic reinstatement of the former primary database in these cases, set this configuration property to FALSE. Starts redo transport services to begin transmitting redo data to all bystander standby databases that were not disabled. Oracle Database 10g databases running versions prior to 10.2.0.4 will remain in a stalled state until aborted or signaled to remain the primary by the observer once connectivity has been restored. Database services can be configured to be active in specific database roles on Oracle RAC databases and on single-instance databases managed by Oracle Restart. In the rare event that a switchover operation fails and you are left with no primary database, retry the switchover command. A complete failover is the recommended and default failover option. After the fast-start failover completes successfully, the master observer will attempt to reinstate the former primary database as a new standby database when a connection to the former primary database is reestablished, and the FastStartFailoverAutoReinstate configuration property is set to TRUE. If groups are not defined, you can still operate on all configurations defined in the file as a whole. In an Oracle Data Guard configuration, the SRVCTL -startoption for a standby database is always set to OPEN after a switchover. $DG_ADMIN directory. The primary and target standby must have connectivity for the STOP OBSERVER command to complete successfully. If the former primary database cannot be reinstated automatically, you can manually reinstate it using either the DGMGRL REINSTATE command or Cloud Control. CONNECT command. For example, if a physical standby database was in the APPLY-OFF state, it will remain in the APPLY-OFF state. command is submitted successfully, the command-line prompt on the Use the EMCLI verb dg_configure_observers. Issue the following SRVCTL commands so that both databases in the Data Guard configuration know about the two potential services for each database: To start things up initially, you must manually start the services on the right node. time specified by maximum configured Once the observer is started, you cannot change the file's name and location. You can find detailed information about all observers, including master observers and backup observers, in the V$FS_FAILOVER_OBSERVERS view. Configure the TNSNAMES.ORA file on the observer system so that the observer is able to connect to the primary database and to the pre-selected target standby database. In the following example, ObserverReconnect is set to 30 seconds. PDF Oracle Database 19c: Data Guard Administration Workshop