이미 설치한 2노드에 2개노드 더 추가하기 - 1

- 모 기업의 DATABASE 시스템이 4NODE의 최고사양의 시스템이란다.

- 또한 엑셈에서 나온 책을 보면 노드별로 이름을 주고 보통 3개노드에서 데이터가 전달되는 내용이

나온다.

 

형태가 2노드일때랑 물론 큰상관은 없으나 그래도 노드별로 나눠져서 데이터가 싱크마춰지는 모습을

각 노드별로 전달되는 데이터를 볼수있어서 시도하기로 했다.

다이나믹 리마스터링 되는 모습등을 볼 기회를 만들어서 직접 경험을 해보자.

 

4노드.......  내컴퓨터 죽는거 아니냐 ;;;;;

 

기존 node1 설치할때 ocfs2 설치하고 configure 와 ocfs mount 까지 완료후.

node1 에서

$ORACLE_CRS_HOME /oui/bin/addNode.sh 실행.

그러면 installer 실행되며

 

/etc/hosts에 추가로 등록한 rac3 , rac4 정보를 입력후 설치 ~~~

그러면 node3 ,node4 에 crs_1 경로에 클러스터웨어가 설치된다.

 

설치가 되고 아래화면이 나올때 root 권한으로 실행.

이게 난감하다. 보통 업무상으로 노드를 한번에 두개를 추가할일이 없기때문이가 보다.

나같은 경우는 node를 한번에 2개를 추가해야 하다보니.

 

에러가 하나 발생한다.

/u01/app/oracle/oraInventory/orainstRoot.sh 각 node(3,4) 실행 - 성공

/u01/app/oracle/product/10.2.0/crs_1/install/rootaddnode.sh - 실패가 떨어진다.

정확하게는 실패라기보단

-bash: line 1: rac4: command not found

이런 메세지가 떨어진다.

 

변수에 노드 1개의 값만 넣어지면 이상이 없으나 $NODES_LIST 라는 변수에 node 값이 rac3 rac4 가

같이 있는 바람에 저런 에러가 난다.그래서

rootaddnode.sh 의 파일 가장 아래이 명령어 라인.

$SU $CRS_ORACLE_OWNER -c "$CH/bin/cluutil -sourcefile $OCRCONFIG -destfile $CH/srvm/admin/ocr.loc -nodelist $NODES_LIST" 이걸 그냥 2번실행하게 해준다.

 

$SU $CRS_ORACLE_OWNER -c "$CH/bin/cluutil -sourcefile $OCRCONFIG -destfile $CH/srvm/admin/ocr.loc -nodelist rac3"

$SU $CRS_ORACLE_OWNER -c "$CH/bin/cluutil -sourcefile $OCRCONFIG -destfile $CH/srvm/admin/ocr.loc -nodelist rac4"

 

/u01/app/oracle/product/10.2.0/crs_1/install/rootaddnode.sh

다시 실행해주면 ~~~ 에러안나고 잘 된다.

 

node3 에서

/u01/app/oracle/product/10.2.0/crs_1/root.sh

[root@rac3 ~]# /u01/app/oracle/product/10.2.0/crs_1/root.sh
WARNING: directory '/u01/app/oracle/product/10.2.0' is not owned by root
WARNING: directory '/u01/app/oracle/product' is not owned by root
WARNING: directory '/u01/app/oracle' is not owned by root
WARNING: directory '/u01/app' is not owned by root
WARNING: directory '/u01' is not owned by root
Checking to see if Oracle CRS stack is already configured
/etc/oracle does not exist. Creating it now.
OCR LOCATIONS =  /ocfs/clusterware/ocr
OCR backup directory '/u01/app/oracle/product/10.2.0/crs_1/cdata/crs' does not exist. Creating now
Setting the permissions on OCR backup directory
Setting up NS directories
Oracle Cluster Registry configuration upgraded successfully
WARNING: directory '/u01/app/oracle/product/10.2.0' is not owned by root
WARNING: directory '/u01/app/oracle/product' is not owned by root
WARNING: directory '/u01/app/oracle' is not owned by root
WARNING: directory '/u01/app' is not owned by root
WARNING: directory '/u01' is not owned by root
clscfg: EXISTING configuration version 3 detected.
clscfg: version 3 is 10G Release 2.
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node <nodenumber>: <nodename> <private interconnect name> <hostname>
node 1: rac1 rac1priv rac1
node 2: rac2 rac2priv rac2
clscfg: Arguments check out successfully.

NO KEYS WERE WRITTEN. Supply -force parameter to override.
-force is destructive and will destroy any previous cluster
configuration.
Oracle Cluster Registry for cluster has already been initialized
Startup will be queued to init within 90 seconds.
Adding daemons to inittab
Expecting the CRS daemons to be up within 600 seconds.
CSS is active on these nodes.
        rac1
        rac2
        rac3
CSS is inactive on these nodes.
        rac4
Local node checking complete.
Run root.sh on remaining nodes to start CRS daemons.

 

node4 에서

/u01/app/oracle/product/10.2.0/crs_1/root.sh

[root@rac4 ~]# /u01/app/oracle/product/10.2.0/crs_1/root.sh
WARNING: directory '/u01/app/oracle/product/10.2.0' is not owned by root
WARNING: directory '/u01/app/oracle/product' is not owned by root
WARNING: directory '/u01/app/oracle' is not owned by root
WARNING: directory '/u01/app' is not owned by root
WARNING: directory '/u01' is not owned by root
Checking to see if Oracle CRS stack is already configured
/etc/oracle does not exist. Creating it now.
OCR LOCATIONS =  /ocfs/clusterware/ocr
OCR backup directory '/u01/app/oracle/product/10.2.0/crs_1/cdata/crs' does not exist. Creating now
Setting the permissions on OCR backup directory
Setting up NS directories
Oracle Cluster Registry configuration upgraded successfully
WARNING: directory '/u01/app/oracle/product/10.2.0' is not owned by root
WARNING: directory '/u01/app/oracle/product' is not owned by root
WARNING: directory '/u01/app/oracle' is not owned by root
WARNING: directory '/u01/app' is not owned by root
WARNING: directory '/u01' is not owned by root
clscfg: EXISTING configuration version 3 detected.
clscfg: version 3 is 10G Release 2.
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node <nodenumber>: <nodename> <private interconnect name> <hostname>
node 1: rac1 rac1priv rac1
node 2: rac2 rac2priv rac2
clscfg: Arguments check out successfully.

NO KEYS WERE WRITTEN. Supply -force parameter to override.
-force is destructive and will destroy any previous cluster
configuration.
Oracle Cluster Registry for cluster has already been initialized
Startup will be queued to init within 90 seconds.
Adding daemons to inittab
Expecting the CRS daemons to be up within 600 seconds.
CSS is active on these nodes.
        rac1
        rac2
        rac3
        rac4
CSS is active on all nodes.
Waiting for the Oracle CRSD and EVMD to start
Oracle CRS stack installed and running under init(1M)
Running vipca(silent) for configuring nodeapps
/u01/app/oracle/product/10.2.0/crs_1/jdk/jre//bin/java: error while loading shared libraries: libpthread.so.0: cannot open shared object file: No such file or directory

 

에러가 난다. - 이에러 역시나 vipca XX같은 에러 기존 노드 설치할때 수정했던 부분 다시 해준다.

이왕 할겸 node3,4 노드 둘다 해준다.

vipca => 추가 하기 : unset LD_ASSUME_KERNEL

 

/u01/app/oracle/product/10.2.0/crs_1/bin/vipca

이제 이상없이 vip 까지 잘 설정되고 마쳐진다.

 

 

오~~~~~ 설치 성공 ㅡ_ㅡ;;;;;;;; 이걸 어찌 다 관리한다냐;;;

[oracle@rac1 bin]$ crs_stat -t
Name           Type           Target    State     Host
------------------------------------------------------------
ora.devdb.db   application    ONLINE    ONLINE    rac1
ora....b1.inst application    ONLINE    ONLINE    rac1
ora....b2.inst application    ONLINE    ONLINE    rac2
ora....SM1.asm application    ONLINE    ONLINE    rac1
ora....C1.lsnr application    ONLINE    ONLINE    rac1
ora.rac1.gsd   application    ONLINE    ONLINE    rac1
ora.rac1.ons   application    ONLINE    ONLINE    rac1
ora.rac1.vip   application    ONLINE    ONLINE    rac1
ora....SM2.asm application    ONLINE    ONLINE    rac2
ora....C2.lsnr application    ONLINE    ONLINE    rac2
ora.rac2.gsd   application    ONLINE    ONLINE    rac2
ora.rac2.ons   application    ONLINE    ONLINE    rac2
ora.rac2.vip   application    ONLINE    ONLINE    rac2
ora.rac3.gsd   application    ONLINE    ONLINE    rac3
ora.rac3.ons   application    ONLINE    ONLINE    rac3
ora.rac3.vip   application    ONLINE    ONLINE    rac3
ora.rac4.gsd   application    ONLINE    ONLINE    rac4
ora.rac4.ons   application    ONLINE    ONLINE    rac4
ora.rac4.vip   application    ONLINE    ONLINE    rac4

여기까지가 CRS 설치 .