标签搜索

MySQL通过goldengate单向同步到Oracle数据库

basil
2019-10-06 / 490 阅读
  1. 操作系统以及软件版本

    源端:操作系统centos7.6 数据库MYSQL5.6.45 goldengate版本122022_ggs_Linux_x64_MySQL_64bit

    目标端:操作系统win10 数据库Oracle11gR2 goldengate版本122023_ggs_Windows_x64_shiphome

  2. 配置mysql
    /var/lib/mysql目录下创建存放binlog日志的文件夹binlog

    mkdir /var/lib/mysql/binlog

    编辑my.cnf文件,在[mysqld]后面添加以下配置项

    log-bin = /usr/local/var/mysql/logs/mysql-bin.log
    expire-logs-days = 14
    max-binlog-size = 500M
    server-id = 1
    binlog_format=row

    重启mysqlservice mysql restart,登录mysql,执行fsd可以看到是否开启binlog日志,log_bin为ON表示开启成功

    | log_bin                         | ON                                    |
    | log_bin_basename                | /var/lib/mysql/binlog/mysql-bin       |
    | log_bin_index                   | /var/lib/mysql/binlog/mysql-bin.index |
    | log_bin_trust_function_creators | OFF                                   |
    | log_bin_use_v1_row_events       | OFF                                   |
    | sql_log_bin                     | ON

    同时可以看到/var/lib/mysql/binlog文件夹生成

    mysql-bin.000001  mysql-bin.index

    创建测试数据库以及数据表

    create database ogg default character set utf8 collate utf8_general_ci;
    CREATE TABLE `user` (
    `id` int(10) unsigned NOT NULL AUTO_INCREMENT COMMENT '自增ID',
    `name` varchar(10) CHARACTER SET utf8 NOT NULL DEFAULT '' COMMENT '姓名',
    `age` tinyint(3) unsigned NOT NULL DEFAULT '0' COMMENT '年龄',
    PRIMARY KEY (`id`)) ENGINE=InnoDB DEFAULT CHARSET=latin1 COMMENT='人物表';
  3. 安装和配置源端mysql版goldengate

    通过网址https://www.oracle.com/middleware/technologies/goldengate-downloads.html下载软件包 122022_ggs_Linux_x64_MySQL_64bit.zip

    切换至mysql用户

     su - mysql

    新建文件夹ogg

    mkdir ogg

    122022_ggs_Linux_x64_MySQL_64bit.zip文件通过fizezilla或其它工具上传至ogg文件夹,并更改所有者为mysql

    chown mysql:mysql 122022_ggs_Linux_x64_MySQL_64bit.zip

    切换至mysql用户并解压文件到当前文件夹

    unzip 122022_ggs_Linux_x64_MySQL_64bit.zip
    tar -xvf ggs_Linux_x64_MySQL_64bit.tar

    配置ogg参数
    进入ogg文件夹,运行ogg主程序

    ./ggsci

    执行创建子目录命令

    create subdirs

    配置管理进程,输入命令edit params mgr,创建配置文件填入以下内容后保存

    PORT 7809
    autorestart extract *,waitminutes 2,resetminutes 5

    启动管理进程,输入命令start mgr,启动成功后显示如下

    Manager started.

    输入命令info mgr可以看到管理进程运行状态

    Manager is running (IP port localhost.localdomain.7809, Process ID 3862).

    配置抽取进行e_m

    执行命令edit params e_m创建配置文件,并填入以下内容

    EXTRACT e_m
    setenv (MYSQL_HOME=/var/lib/mysql)
    TRANLOGOPTIONS ALTLOGDEST  /var/lib/mysql/binlog/mysql-bin.index
    SOURCEDB ogg@localhost:3306, USERID root, PASSWORD 123456
    REPORTCOUNT EVERY 30 MINUTES, RATE
    EXTTRAIL /var/lib/mysql/ogg/dirdat/mq,FORMAT RELEASE 12.2
    TABLE ogg.user;

    添加抽取进程

    add extract e_m,tranlog,begin now
    add exttrail /var/lib/mysql/ogg/dirdat/mq,extract e_m

    启动抽取进程

    start e_m

    使用info e_m命令查看启动状态,成功如下

    EXTRACT    E_M       Last Started 2019-10-06 04:37   Status RUNNING
    Checkpoint Lag       00:00:00 (updated 00:00:04 ago)
    Process ID           5032
    VAM Read Checkpoint  2019-10-06 04:33:54.477981

    配置推送进程p_m

    执行edit params p_m命令创建配置文件并填入以下内容

    extract p_m
    SETENV (NLS_LANG=AMERICAN_AMERICA.ZHS16GBK)
    sourcedb ogg@127.0.0.1:3306 userid root password 123456
    PASSTHRU
    numfiles 20000
    rmthost 192.168.56.1,mgrport 7809,compress
    rmttrail F:\oracle12\product\12.2.0\oggcore_1\dirdat\mq,FORMAT RELEASE 12.2
    table ogg.user;

    添加推送进程

    add extract p_m,exttrailsource /var/lib/mysql/ogg/dirdat/mq
    
    add rmttrail F:\oracle12\product\12.2.0\oggcore_1\dirdat\mq,extract p_m,megabytes 500

    启动推送进程

    start p_m

    输入info p_m命令查看启动状态,成功如下

    EXTRACT    P_M       Last Started 2019-10-06 05:01   Status RUNNING
    Checkpoint Lag       00:00:00 (updated 00:02:37 ago)
    Process ID           6315
    Log Read Checkpoint  File /var/lib/mysql/ogg/dirdat/mq000000000
                         First Record  RBA 0

    输入info all命令,可以查看mgr、e_m、p_m进程已经启动成功

    MANAGER     RUNNING                                           
    EXTRACT     RUNNING     E_M         00:00:00      00:00:08    
    EXTRACT     ABENDED     P_M         00:00:00      00:04:00 

    配置异构文件defgen.prm(异构数据库需要配置)
    输入命令edit params defgen创建配置文件并填入内容

    defsfile ./dirdef/defgen.prm
    sourcedb ogg@127.0.0.1:3306 userid root, password 123456
    table ogg.user;

    输入exit退出ogg主程序,执行defgen程序生成表定义文件defgen.prm

    ./defgen paramfile ./dirprm/defgen.prm

    将生成在/dirdef/defgen.prm文件传到目标端对应文件夹下

  4. 创建Oracle测试表

    创建测试表

    CREATE TABLE "CXWT"."user" 
     (    "id" NUMBER(10,0) DEFAULT 0 NOT NULL ENABLE, 
      "name" VARCHAR2(15) DEFAULT '' NOT NULL ENABLE, 
      "age" NUMBER(10,0) DEFAULT 0 NOT NULL ENABLE, 
       PRIMARY KEY ("id")
    USING INDEX PCTFREE 10 INITRANS 2 MAXTRANS 255 NOCOMPRESS LOGGING
    TABLESPACE "CXWT_SPACE"  ENABLE
     ) SEGMENT CREATION DEFERRED 
    PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255 NOCOMPRESS LOGGING
    TABLESPACE "CXWT_SPACE";
    CREATE UNIQUE INDEX "CXWT"."SYS_C0011103" ON "CXWT"."user" ("id") 
    PCTFREE 10 INITRANS 2 MAXTRANS 255 NOCOMPRESS LOGGING
    TABLESPACE "CXWT_SPACE";
  5. 安装目标端Oracle版goldengate

    通过网址https://www.oracle.com/middleware/technologies/goldengate-downloads.html下载软件包122023_ggs_Windows_x64_shiphome.zip
    解压软件包,并打开setup.exe文件进行安装,在弹出的安装界面第一步选择Oracle 11g数据库,安装路径选择了F:\oracle12\product\12.2.0\oggcore_1

    使用cmd打开goldengate安装目录,运行ogg主程序

    .\ggsci.exe

    配置管理进程,输入命令edit params mgr,修改配置文件,内容改为

    PORT 7809
    ACCESSRULE, PROG REPLICAT, IPADDR 192.168.56.101 ALLOW //IPADDR为源端IP地址

    输入命令start mgr启动管理进程,启动成功将弹出另外两个窗口,一个管理进程窗口,另外一个是收集进程窗口

    配置GLOBALS和checkpoint

    输入命令edit params ./GLOBALS创建配置文件并填入以下内容

    checkpointtable cxwt.checkpoint

    输入命令dblogin userid cxwt,password 123456登录Oracle数据库

    输入以下命令添加checkpoint表

     add checkpointtable cxwt.checkpoint

    创建复制进程e_m

    输入命令edit params r_m创建配置文件并填入以下内容

    replicat r_m
    setenv(NLS_LANG=AMERICAN_AMERICA.ZHS16GBK)
    setenv(ORACLE_HOME=F:\oracle12\product\11.2.0\dbhome_1)
    sourcedefs F:\oracle12\product\12.2.0\oggcore_1\dirdef\defgen.prm
    userid cxwt,password remainconfident
    discardfile F:\oracle12\product\12.2.0\oggcore_1\dirrpt\r_m.dsc,purge
    HANDLECOLLISIONS
    map ogg.user,target cxwt.user;

    添加复制进程

    add replicat r_m,exttrail F:\oracle12\product\12.2.0\oggcore_1\dirdat\mq,checkpointtable cxwt.checkpoint
  6. 启动goldengate进行同步

    拷贝源端defen.prm到目标端F:\oracle12\product\12.2.0\oggcore_1\dirdef文件夹下
    源端开启mgr、e_m、p_m进程

    start mgr
     
    start e_m
    
    start p_m

    使用info all命令查看启动状态,成功开启如下

    Program     Status      Group       Lag at Chkpt  Time Since Chkpt
    MANAGER     RUNNING                                           
    EXTRACT     RUNNING     E_M         00:00:00      00:00:03    
    EXTRACT     ABENDED     P_M         00:00:00      02:50:50 

    目标端开启mgr、r_m进程

    start mgr
    start r_m

    使用info all命令查看启动状态,成功如下

突然间遇到问题:投递进程会异常中断,解决后再补充剩下的

0