目的：本博文给出11.2.0.4 oracle数据库ogg搭建过程中，备库的准备过程包括2种方式：第一种是主库rman全量备份后恢复，第二种是expdp从主库导出用户然后在备库导入。
对ogg数据同步进行测试，主要偏重类似mysql的半同步数据，即备库从主库同步部分数据。
?一、ogg相关服务器基本信息
|
配置 主机 |
源端 |
目标端 |
|
主机名 |
ogg1 |
ogg2 |
|
IP地址 |
10.117.130.231 |
10.117.130.232 |
|
内存 |
3832MB |
3832MB |
|
uid=500(oracle) gid=601(oinstall) 组=601(oinstall),603(dba) |
uid=500(oracle) gid=601(oinstall) 组=601(oinstall),603(dba) |
|
|
数据库版本 |
11.2.0.4 |
11.2.0.4 |
|
ORACLE_HOME |
/u01/oracle/app/oracle/product/11.2.0.4/db |
/u01/oracle/app/oracle/product/11.2.0.4/db |
|
NIL_LANG |
AMERICAN_AMERICA.ZHS16GBK |
AMERICAN_AMERICA.ZHS16GBK |
|
OGG_HOME |
/u01/oracle/app/ogg |
/u01/oracle/app/ogg |
|
OGG版本 |
12.3.0.1.0 |
12.3.0.1.0 |
|
OGG管理用户/密码 |
GOLDENGATE/GOLDENGATE |
GOLDENGATE/GOLDENGATE |
|
OGG同步的用户 |
HR(示例库)，ZHUL（模拟业务） |
|
二、主库调整
1、主库开启归档模式
select log_mode from v$database;
如果主库没有开启归档模式则开启归档模式
2、打开数据库force-logging及补充日志
alter database force logging;
alter database add supplemental log data;
alter database add supplemental log data (primary key) columns;
3、处理压缩表及分区压缩表，将其变为非压缩表
select 'alter table '||table_owner||'.'||table_name||' move partition '||partition_name||' nocompress update indexes;' from dba_tab_partitions where compression ='ENABLED' and table_owner='HR';
4、处理nologging的表，将其修改为logging的表
select 'alter table '||owner||'.'||table_name||' logging;' from dba_tables where owner in ('HR') and logging='NO';
5、处理nologging分区表为logging分区
select distinct 'alter table '||table_owner||'.'||table_name||' logging;' from dba_tab_partitions where table_owner in('HR') and logging='NO';
6、主库创建OGG专用用户
create tablespace goldengate datafile '$ORACLE_BASE/oradata/$ORACLE_SID/goldengate.dbf' size 100M autoextend on;
CREATE USER GOLDENGATE IDENTIFIED BY GOLDENGATE DEFAULT TABLESPACE GOLDENGATE TEMPORARY TABLESPACE TEMP;
GRANT CONNECT TO GOLDENGATE;
GRANT ALTER ANY TABLE TO GOLDENGATE;
GRANT ALTER SESSION TO GOLDENGATE;
GRANT CREATE SESSION TO GOLDENGATE;
GRANT FLASHBACK ANY TABLE TO GOLDENGATE;
GRANT SELECT ANY DICTIONARY TO GOLDENGATE;
GRANT SELECT ANY TABLE TO GOLDENGATE;
GRANT RESOURCE TO GOLDENGATE;
GRANT SELECT ANY TRANSACTION TO GOLDENGATE;
四、主库源数据备份
1、rman全备份
backup database format '/home/oracle/rman/full_%D_%d_%U.bak';
select current_scn from v$database;
alter system switch logfile;
backup archivelog all format '/home/oracle/rman/arch_%D_%d_%U.bak';
2、expdp按方案导出
2.1 测试数据来源
[oracle@ogg1 ~]$ sqlplus / as sysdba
sql*Plus: Release 11.2.0.4.0 Production on Tue Sep 26 14:28:36 2017
copyright (c) 1982, 2013, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
sql> create user zhul identified by zhul default tablespace users;
User created.
sql> grant create session to zhul;
Grant succeeded.
sql> grant resource to zhul;
Grant succeeded.
sql> create table zhul.emp as select * from scott.emp;
Table created.
sql> c/emp/dept
1* create table zhul.dept as select * from scott.emp
sql> c/emp/dept
1* create table zhul.dept as select * from scott.dept
sql> /
Table created.
sql> c/dept/SALGRADE
1* create table zhul.SALGRADE as select * from scott.dept
sql> c/dept/SALGRADE
1* create table zhul.SALGRADE as select * from scott.SALGRADE
sql> /
Table created.
sql> create directory zhul as '/home/oracle/';
Directory created.
sql>
sql> select current_scn from v$database;
CURRENT_SCN
-----------
1057694
2.2 expdp备份
[oracle@ogg1 ~]$ cat param
userid=zhul/zhul
directory=zhul
tables=emp
dumpfile=expdp_zhul_emp_270926.dmp
logfile=log_zhul_emp_270926.log
flashback_scn=1057694
query=emp:"where hiredate>'1982-01-02'"
[oracle@ogg1 ~]$
[oracle@ogg1 ~]$ expdp parfile=/home/oracle/param
Export: Release 11.2.0.4.0 - Production on Tue Sep 26 14:54:27 2017
copyright (c) 1982, 2011, Oracle and/or its affiliates. All rights reserved.
Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
Starting "ZHUL"."SYS_EXPORT_TABLE_01": zhul/******** parfile=/home/oracle/param
Estimate in progress using BLOCKS method...
Processing object type TABLE_EXPORT/TABLE/TABLE_DATA
Total estimation using BLOCKS method: 64 KB
Processing object type TABLE_EXPORT/TABLE/TABLE
. . exported "ZHUL"."EMP" 8.109 KB 3 rows
Master table "ZHUL"."SYS_EXPORT_TABLE_01" successfully loaded/unloaded
******************************************************************************
Dump file set for ZHUL.SYS_EXPORT_TABLE_01 is:
/home/oracle/expdp_zhul_emp_270926.dmp
Job "ZHUL"."SYS_EXPORT_TABLE_01" successfully completed at Tue Sep 26 14:54:30 2017 elapsed 0 00:00:01
[oracle@ogg1 ~]$
五、备库目标数据恢复
恢复前准备
如源库开启闪回，需创建同等闪回目录即$ORACLE_BASE/ fast_recovery_area，否则报错：
需要建立与主库同等的审计目录，否则报错：
mkdir -p /u01/oracle/app/oracle/admin/orcl/adump
备库创建数据目录：
mkdir –p /u01/oracle/app/oracle/oradata/orcl/
1、rman全恢复
rman target /
RMAN> startup nomount;
RMAN> restore spfile from '/home/oracle/rman/full_25_ORCL_02sfc009_1_1.bak';
RMAN> shutdown immediate;
RMAN> startup nomount;
RMAN> restore controlfile from '/home/oracle/rman/full_25_ORCL_02sfc009_1_1.bak';
RMAN> alter database mount;
RMAN> restore database;
RMAN> catalog start with '/home/oracle/rman';
RMAN> run{
set until scn 996247;
recover database;
}
--这里的SCN号是主库源数据备份时查出来的SCN号，就是第四部里边的执行绿色sql命令时查出来的，注意一定要有。
2、备库expdp备份导入
2.1 创建directory
create directory zhul as ‘/home/oracle’;
2.2 创建Ogg专用用户
CREATE USER GOLDENGATE IDENTIFIED BY GOLDENGATE DEFAULT TABLESPACE GOLDENGATE TEMPORARY TABLESPACE TEMP;
GRANT CONNECT TO GOLDENGATE;
GRANT ALTER ANY TABLE TO GOLDENGATE;
GRANT ALTER SESSION TO GOLDENGATE;
GRANT CREATE SESSION TO GOLDENGATE;
GRANT FLASHBACK ANY TABLE TO GOLDENGATE;
GRANT SELECT ANY DICTIONARY TO GOLDENGATE;
GRANT SELECT ANY TABLE TO GOLDENGATE;
GRANT RESOURCE TO GOLDENGATE;
GRANT SELECT ANY TRANSACTION TO GOLDENGATE;
2.3 数据导入
[oracle@ogg2 ~]$ impdp system/oracleoracle directory=zhul tables=zhul.emp dumpfile=expdp_zhul_emp_270926.dmp logfile=impdp_zhul_emp_270926.log
Import: Release 11.2.0.4.0 - Production on Tue Sep 26 15:08:09 2017
copyright (c) 1982, 2011, Oracle and/or its affiliates. All rights reserved.
Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
Master table "SYstem"."SYS_IMPORT_TABLE_01" successfully loaded/unloaded
Starting "SYstem"."SYS_IMPORT_TABLE_01": system/******** directory=zhul tables=zhul.emp dumpfile=expdp_zhul_emp_270926.dmp logfile=impdp_zhul_emp_270926.log
Processing object type TABLE_EXPORT/TABLE/TABLE
Processing object type TABLE_EXPORT/TABLE/TABLE_DATA
. . imported "ZHUL"."EMP" 8.109 KB 3 rows
Job "SYstem"."SYS_IMPORT_TABLE_01" successfully completed at Tue Sep 26 15:08:16 2017 elapsed 0 00:00:05
[oracle@ogg2 ~]$
六、主库安装OGG
1、创建安装目录
mkdir –p $ORACLE_BASE\ogg
2、上传并解压OGG软件
3、编辑OGG静默安装脚本
|
[oracle@ogg1 response]$ pwd /u01/oracle/app/fbo_ggs_Linux_x64_shiphome/disk1/response [oracle@ogg1 response]$ ls [oracle@ogg1 response]$ cat oggcore.rsp #################################################################### ## copyright(c) Oracle Corporation 2014. All rights reserved. ## ## ## ## Specify values for the variables listed below to customize ## ## your installation. ## ## ## ## Each variable is associated with a comment. The comment ## ## can help to populate the variables with the appropriate ## ## values. ## ## ## ## IMPORTANT NOTE: This file should be secured to have read ## ## permission only by the oracle user or an administrator who ## ## own this installation to protect any sensitive input values. ## ## ## #################################################################### #------------------------------------------------------------------------------- # Do not change the following system generated value. #------------------------------------------------------------------------------- oracle.install.responseFiLeversion=/oracle/install/rspfmt_ogginstall_response_schema_v12_1_2 ################################################################################ ## ## ## Oracle GoldenGate installation option and details ## ## ## ################################################################################ #------------------------------------------------------------------------------- # Specify the installation option. # Specify ORA12c for installing Oracle GoldenGate for Oracle Database 12c and # ORA11g for installing Oracle GoldenGate for Oracle Database 11g #------------------------------------------------------------------------------- INSTALL_OPTION=ORA11g #------------------------------------------------------------------------------- # Specify a location to install Oracle GoldenGate #------------------------------------------------------------------------------- SOFTWARE_LOCATION=/u01/oracle/app/ogg #------------------------------------------------------------------------------- # Specify true to start the manager after installation. #------------------------------------------------------------------------------- START_MANAGER=true #------------------------------------------------------------------------------- # Specify a free port within the valid range for the manager process. # required only if START_MANAGER is true. #------------------------------------------------------------------------------- MANAGER_PORT=1700 #------------------------------------------------------------------------------- # Specify the location of the Oracle Database. # required only if START_MANAGER is true. #------------------------------------------------------------------------------- DATABASE_LOCATION=/u01/oracle/app/oracle/product/11.2.0.4/db ################################################################################ ## ## ## Specify details to Create inventory for Oracle installs ## ## required only for the first Oracle product install on a system. ## ## ## ################################################################################ #------------------------------------------------------------------------------- # Specify the location which holds the install inventory files. # This is an optional parameter if installing on # Windows based Operating System. #------------------------------------------------------------------------------- INVENTORY_LOCATION=/u01/oracle/app/oraInventory #------------------------------------------------------------------------------- # Unix group to be set for the inventory directory. # This parameter is not applicable if installing on # Windows based Operating System. #------------------------------------------------------------------------------- UNIX_GROUP_NAME=oinstall [oracle@ogg1 response]$ |
[oracle@ogg1 response]$
4、OGG静默安装
[oracle@ogg1 disk1]$ pwd
/u01/oracle/app/fbo_ggs_Linux_x64_shiphome/disk1
[oracle@ogg1 disk1]$./runInstaller -silent -responseFile /u01/oracle/app/fbo_ggs_Linux_x64_shiphome/disk1/response/oggcore.rsp
5、源端配置OGG相关的视图基表
[oracle@ogg1 app]$ cd ogg
[oracle@ogg1 ogg]$ sqlplus / as sysdba
sql*Plus: Release 11.2.0.4.0 Production on Mon Sep 25 21:25:29 2017
copyright (c) 1982, 2013, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
sql> @marker_setup.sql
sql> @ ddl_setup.sql
sql> @ role_setup.sql
sql> @ddl_enable.sql
sql> sequence.sql
sql> grant GGS_GGSUSER_ROLE to GOLDENGATE;
sql> alter system set enable_goldengate_replication = true scope=both;
6、配置OGG mgr进程
edit param mgr
PORT 1700
PURGEOLDEXTRACTS /u01/oracle/app/ogg/dirdat, USECHECKPOINTS
--确定当前目录就在ogg的安装目录/u01/oracle/app/ogg，然后重启mgr进程
start mgr
dblogin userid goldengate,password goldengate;
info trandata hr.*
add trandata hr.*
确认传输日志开启
8、配置源数据抽取进程ext_hr，ext_zhul
add extract ext_hr,tranlog,begin Now
8.1 添加ext_hr的数据抽取数据存放文件
add exttrail /u01/oracle/app/ogg/dirdat/et,extract ext_hr
8.1.1 添加ext_hr的远程数据存放文件
ADD RMTTRAIL /u01/oracle/app/ogg/dirat/et, EXTRACT EXT_HR
edit param ext_hr
|
EXTRACT ext_hr setenv (NLS_LANG=AMERICAN_AMERICA.ZHS16GBK ) userid GOLDENGATE,password GOLDENGATE rmthost 10.117.130.232,mgrport 1700 rmttrail /u01/oracle/app/ogg/dirdat/et TRANlogoPTIONS EXCLUDEUSER GOLDENGATE ddl include all table hr.t,sqlPREDICATE "where salary>9000"; |
8.1.3 启动ext_hr进程
start ext_hr
8.1.4 增加ogg用户检查点
add checkpointtable GOLDENGATE.checkpoint
8.2 主库配置数据抽取进程exp_zhul
dblogin userid GOLDENGATE,password GOLDENGATE;
查看zhul下的数据表的数据传输信息
info trandata zhul.*
启用zhul下的数据表传输
add trandata zhul.*
添加备库的ogg数据抽取进程
add extract ext_zhul,tranlog,begin Now
添加备库的ogg数据抽取日志队列文件
add exttrail /u01/oracle/app/ogg/dirdat/ez,extract ext_zhul
ADD RMTTRAIL /u01/oracle/app/ogg/dirat/ez, EXTRACT ext_zhul
编辑主库的数据抽取进程ext_zhul
edit param ext_zhul
|
EXTRACT ext_zhul setenv (NLS_LANG=AMERICAN_AMERICA.ZHS16GBK ) userid GOLDENGATE,password GOLDENGATE rmthost 10.117.130.232,mgrport 1700 rmttrail /u01/oracle/app/ogg/dirdat/ez TRANlogoPTIONS EXCLUDEUSER GOLDENGATE --ddl include all table zhul.emp,sqlPREDICATE "where hiredate>'1982-01-02'"; |
8.3 启动exp_zhul进程
七、目标端安装OGG
1、创建安装目录
mkdir –p $ORACLE_BASE\ogg
2、上传并解压OGG软件
3、编辑OGG静默安装脚本
|
[oracle@ogg1 response]$ pwd /u01/oracle/app/fbo_ggs_Linux_x64_shiphome/disk1/response [oracle@ogg1 response]$ ls [oracle@ogg1 response]$ cat oggcore.rsp #################################################################### ## copyright(c) Oracle Corporation 2014. All rights reserved. ## ## ## ## Specify values for the variables listed below to customize ## ## your installation. ## ## ## ## Each variable is associated with a comment. The comment ## ## can help to populate the variables with the appropriate ## ## values. ## ## ## ## IMPORTANT NOTE: This file should be secured to have read ## ## permission only by the oracle user or an administrator who ## ## own this installation to protect any sensitive input values. ## ## ## #################################################################### #------------------------------------------------------------------------------- # Do not change the following system generated value. #------------------------------------------------------------------------------- oracle.install.responseFiLeversion=/oracle/install/rspfmt_ogginstall_response_schema_v12_1_2 ################################################################################ ## Oracle GoldenGate installation option and details ## ################################################################################ #------------------------------------------------------------------------------- # Specify the installation option. # Specify ORA12c for installing Oracle GoldenGate for Oracle Database 12c and # ORA11g for installing Oracle GoldenGate for Oracle Database 11g #------------------------------------------------------------------------------- INSTALL_OPTION=ORA11g #------------------------------------------------------------------------------- # Specify a location to install Oracle GoldenGate #------------------------------------------------------------------------------- SOFTWARE_LOCATION=/u01/oracle/app/ogg #------------------------------------------------------------------------------- # Specify true to start the manager after installation. #------------------------------------------------------------------------------- START_MANAGER=true #------------------------------------------------------------------------------- # Specify a free port within the valid range for the manager process. # required only if START_MANAGER is true. #------------------------------------------------------------------------------- MANAGER_PORT=1700 #------------------------------------------------------------------------------- # Specify the location of the Oracle Database. # required only if START_MANAGER is true. #------------------------------------------------------------------------------- DATABASE_LOCATION=/u01/oracle/app/oracle/product/11.2.0.4/db ################################################################################ ## Specify details to Create inventory for Oracle installs ## ## required only for the first Oracle product install on a system. ## ################################################################################ #------------------------------------------------------------------------------- # Specify the location which holds the install inventory files. # This is an optional parameter if installing on # Windows based Operating System. #------------------------------------------------------------------------------- INVENTORY_LOCATION=/u01/oracle/app/oraInventory #------------------------------------------------------------------------------- # Unix group to be set for the inventory directory. # This parameter is not applicable if installing on # Windows based Operating System. #------------------------------------------------------------------------------- UNIX_GROUP_NAME=oinstall [oracle@ogg1 response]$ |
4、OGG静默安装
[oracle@ogg1 disk1]$ pwd
/u01/oracle/app/fbo_ggs_Linux_x64_shiphome/disk1
[oracle@ogg1 disk1]$./runInstaller -silent -responseFile /u01/oracle/app/fbo_ggs_Linux_x64_shiphome/disk1/response/oggcore.rsp
5、源端配置OGG相关的视图基表
[oracle@ogg2 app]$ cd ogg
[oracle@ogg2 ogg]$ sqlplus / as sysdba
sql*Plus: Release 11.2.0.4.0 Production on Mon Sep 25 21:25:29 2017
copyright (c) 1982, 2013, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
sql> @marker_setup.sql
sql> @ ddl_setup.sql
sql> @ role_setup.sql
sql> @ddl_enable.sql
sql> sequence.sql
sql> grant GGS_GGSUSER_ROLE to GOLDENGATE;
sql> alter system set enable_goldengate_replication = true scope=both;
6、目标端配置MGR进程
edit param mgr
PORT 1700
PURGEOLDEXTRACTS /u01/oracle/app/ogg/dirdat, USECHECKPOINTS
start mgr
7、目标端配置复制进程rep_hr
add replicat rep_hr exttrail /u01/oracle/app/ogg/dirdat/et,checkpointtable GOLDENGATE.checkpoint
edit param rep_hr
|
--Replicat group -- replicat rep_hr SETENV (NLS_LANG=AMERICAN_AMERICA.ZHS16GBK) --source and target deFinitions ASSUMetaRGETDEFS HANDLECOLLISIONS --target database login -- userid GOLDENGATE, password GOLDENGATE --file for dicarded transaction -- discardfile /u01/oracle/app/ogg/rep_hr_discard.txt, append, megabytes 10 --ddl support DDL DDL INCLUDE ALL DDLERROR DEFAULT IGnorE RETRYOP --Specify table mapping --- MAP hr.t, TARGET hr.t, WHERE (salary > 9000); |
7.1 启动复制进程rep_hr
8、（主库expdp导出备库Impdp导入）备库配置复制进程rep_zhul
add replicat rep_zhul exttrail /u01/oracle/app/ogg/dirdat/ez,checkpointtable GOLDENGATE.checkpoint
查看备库zhul数据表的传输
info trandata zhul.*
开启备库zhul下数据表的传输
add trandata zhul.*
编辑复制进程rep_zhul参数
edit param rep_zhul
|
--Replicat group -- replicat rep_zhul SETENV (NLS_LANG=AMERICAN_AMERICA.ZHS16GBK) --source and target deFinitions ASSUMetaRGETDEFS HANDLECOLLISIONS --target database login -- userid GOLDENGATE, password GOLDENGATE --file for dicarded transaction -- discardfile /u01/oracle/app/ogg/rep_zhul_discard.txt, append, megabytes 10 --ddl support DDL --DDL INCLUDE ALL --DDLERROR DEFAULT IGnorE RETRYOP --Specify table mapping --- MAP zhul.emp, TARGET zhul.emp, WHERE (hiredate>'1982-01-02'); |
--测试中红色部分无法启动进程，否则进程启动报错
8.1 启动rep_zhul进程
八、数据同步测试
针对公司业务场景测试，hr.t表来源:create table hr.t as select * from hr.employees;
1、主库源端清空hr.t表
2、备库目标端清空hr.t表
3、主库源端hr.t表DML模拟
4、rman方式目标端查看数据同步情况
4.1 源端符合salary>9000的数据统计
结论：从数据同步看，可以在目标端控制数据同步策略
?
5、expdp方式数据同步测试
5.1 源端删除一条数据hiredate>'1982-01-02'
5.2 目标端查看数据同步
5.3 源端删除where hiredate<'1982-01-02'
5.4 目标端查看数据同步，没有受到影响
--结论：从源端控制目标端数据同步策略可以控制源端部分数据同步到目标端
?
6、expdp方式源端与目标端有差量数据DML
测试目的：在源端ogg安装完毕，extract进程配置恰当并处于启动状态，而目标端正在处理数据导入过程时，源端被同步的数据发生DML，在目标端的OGG安装完毕，目标端的replicat进程配置恰当并启动后，复制进程replicat可以应用目标端落后于源端的差量数据
6.1 删除目标端rep_zhul进程
6.2 源端被同步数据发生DML
6.3 目标端添加复制进程rep_zhul
6.4 目标端查看落后于源端的差量数据同步情况
--结论：在源端ogg安装完毕，extract进程配置恰当并处于启动状态，而目标端正在处理数据导入过程时，源端被同步的数据发生DML，在目标端的OGG安装完毕，目标端的replicat进程配置恰当并启动后，复制进程replicat可以应用目标端落后于源端的差量数据。
版权声明：本文内容由互联网用户自发贡献，该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 dio@foxmail.com 举报，一经查实，本站将立刻删除。
