Debug Size:

4.2 - RSF-1 Configuration Using RSFADM

The command 'rsfadm' (/opt/HAC/RSF-1/bin/rsfadm) can be used to easily configure the cluster. It makes changes to the cluster using an HTTP API, storing the configuration parameters in a database before writing them to a config file. Because of this, the config file should not be edited manually, since that would cause it to hold different information to the database.
NOTE - Ensure that all ZFS pools are imported on one node. In our example, POOLA and POOLC are imported on romulus and POOLB is imported on remus. Also ensure the VIPs are not already in use elsewhere before proceeding with RSF-1 configuration
Before the cluster can be initialised, rsfmon should be running in bootstrap mode. This can be checked using the 'show' subcommand - it should show a cluster of one node, with the cluster name 'Ready_For_Cluster_Configuration':
root@romulus:~# rsfadm show Global information:
Nodes:
Cluster name : Ready_For_Cluster_Configuration Poll time : 2
Config CRC : 7f14
0 : romulus (romulus) available
RSF-1 release 3.11.0p9, built on 16-Sep-2015-16:13
Services:
(none)
Heartbeats: (none)
root@romulus:~#
To make sure RSF-1 can communicate with RSF-1 on the remote node, the discover- nodes subcommand can be used:
root@romulus:~# rsfadm discover-nodes Node 0: remus root@romulus:~#
Now, to create the cluster, the hostnames of the two nodes will need to be provided to the init subcommand:
root@romulus:~# rsfadm init romulus remus
Oct 22 15:23:07 romulus RSF-1[907]: [ID 702911 local0.alert] RSF-1 hot restart: services may be running.
root@romulus:~#
Now that the cluster is initialised, the 'show' subcommand should show a 2 node cluster with a single network heartbeat between the nodes:
root@romulus:~# rsfadm show Global information:
Nodes:
Cluster name : HA-Cluster Poll time : 1
Config CRC : 9e82
0 : remus (remus) available
RSF-1 release 3.11.0p9, built on 16-Sep-2015-16:13
1 : romulus (romulus) available
RSF-1 release 3.11.0p9, built on 16-Sep-2015-16:13
Services:
(none)
Heartbeats:
0: NET remus --> romulus VIA romulus: Up - last heartbeat #6 (updated
Thu Oct 22 15:26:44)
1: NET romulus --> remus VIA remus: Up - last heartbeat #5 (updated Thu
Oct 22 15:26:44)
2 heartbeats configured, 2 up, 0 down root@romulus:~#
A second network heartbeat can be added (optionally) to increase resiliency. In this example, romulus_priv and remus_priv will be used for the network addresses:

root@romulus:~# \
rsfadm create-hb -t net -d romulus:romulus_priv,remus:remus_priv Machine: romulus, Interface: romulus_priv
Machine: remus, Interface: remus_priv
Oct 22 15:29:12 romulus RSF-1[29192]: [ID 702911 local0.alert] RSF-1 hot restart: services may be running.
root@romulus:~#
root@romulus:~# rsfadm show heartbeats
0: NET
(updated
1: NET
remus --> romulus Thu Oct 22 15:29:25) remus --> romulus
VIA romulus_priv: Up - last heartbeat #12
VIA romulus: Up - last heartbeat #12 (updated VIA remus_priv: Up - last heartbeat #12
VIA remus: Up - last heartbeat #12 (updated
Thu Oct 22 15:29:24)
2: NET
(updated
3: NET
romulus --> remus Thu Oct 22 15:29:26) romulus --> remus
Thu Oct 22 15:29:26)
4 heartbeats configured, 4 up, 0 down root@romulus:~#
The cluster is now set up with two network heartbeat channels, so we can now add a service to control the zpool POOLA (currently imported on romulus). As in the example in the previous section, the service will use the floating IP address 'sales_staff-public' (defined in /etc/hosts on both nodes). Also, the primary server for this service will be set to romulus, and the vip will be plumbed into net1:
root@romulus:~# \
rsfadm create-svc POOLA -v sales_staff-public -i net1 -p romulus
Oct 22 16:53:58 romulus RSF-1[29880]: [ID 702911 local0.alert] RSF-1 hot restart: services may be running.
root@romulus:~#
root@romulus:~#
root@romulus:~#
root@romulus:~# rsfadm show
Global information:
Nodes:
Services:
0 :
Heartbeats: 0: NET
Cluster name : HA-Cluster Poll time : 1
Config CRC : 9fbb
0 :
1 :
remus (remus) available
RSF-1 release 3.11.0p9, built on 16-Sep-2015-16:13
romulus (romulus) available
RSF-1 release 3.11.0p9, built on 16-Sep-2015-16:13
POOLA, IP address sales_staff-public, "RSF-1 cluster service"
remus --> romulus (updated Thu Oct 22 16:57:22) 1: NET remus --> romulus
VIA romulus_priv: Up - last heartbeat #203
VIA romulus: Up - last heartbeat #203 (updated
4: NET romulus --> remus (updated Thu Oct 22 16:57:22) 5: NET romulus --> remus
VIA remus_priv: Up - last heartbeat #205
VIA remus: Up - last heartbeat #204 (updated
stopped automatic running automatic
unblocked on remus unblocked on romulus
Thu Oct 22 16:57:23)
2: DISC remus --> romulus VIA
id1,sd@n60018400000055659e5a0001/a,raw:512,id1,sd@n60018400000055659e5a0001/ a,raw:518: Up - last heartbeat #203 (updated Thu Oct 22 16:57:23)
3: DISC remus --> romulus VIA id1,sd@n60018400000055659e5b0002/a,raw:512,id1,sd@n60018400000055659e5b0002/ a,raw:518: Up - last heartbeat #203 (updated Thu Oct 22 16:57:23)
Thu Oct 22 16:57:22)
6: DISC romulus --> remus VIA
id1,sd@n60018400000055659e5a0001/a,raw:518,id1,sd@n60018400000055659e5a0001/ a,raw:512: Up - last heartbeat #204 (updated Thu Oct 22 16:57:22)
7: DISC romulus --> remus VIA id1,sd@n60018400000055659e5b0002/a,raw:518,id1,sd@n60018400000055659e5b0002/ a,raw:512: Up - last heartbeat #204 (updated Thu Oct 22 16:57:22)
8 heartbeats configured, 8 up, 0 down
root@romulus:~#
Next is to add the second service - POOLB, which will run on remus by default and use floating IP hostname 'support_staff-public', which will also be plumbed into net1:
root@romulus:~# zpool list NAME SIZE ALLOC FREE POOLA 2.95G 237K 2.95G POOLB 2.95G 184K 2.95G rpool 9.69G 6.02G 3.67G root@romulus:~# \
CAP DEDUP HEALTH ALTROOT 0% 1.00x ONLINE -
0% 1.00x ONLINE -
62% 1.00x ONLINE -
rsfadm -v create-svc POOLB -v support_staff-public -i net1 -p remus
Oct 22 17:13:09 romulus RSF-1[23839]: [ID 702911 local0.alert] RSF-1 hot restart: services may be running.
root@romulus:~# rsfadm show
Global information:
Nodes:
Services:
0 :
1 :
Heartbeats: 0: NET
Cluster name : HA-Cluster Poll time : 1
Config CRC : f200
0 :
1 :
remus (remus) available
RSF-1 release 3.11.0p9, built on 16-Sep-2015-16:13
romulus (romulus) available
RSF-1 release 3.11.0p9, built on 16-Sep-2015-16:13
POOLA, IP address sales_staff-public, "RSF-1 cluster service" stopped automatic unblocked on remus
running automatic unblocked on romulus
POOLB, IP address support_staff-public, "RSF-1 cluster service" starting automatic unblocked on remus
stopped automatic unblocked on romulus
remus --> romulus (updated Thu Oct 22 17:13:27) 1: NET remus --> romulus
VIA romulus_priv: Up - last heartbeat #18
VIA romulus: Up - last heartbeat #18 (updated
Thu Oct 22 17:13:28)
2: DISC remus --> romulus VIA
id1,sd@n60018400000055659e5a0001/a,raw:512,id1,sd@n60018400000055659e5a0001/ a,raw:518: Up - last heartbeat #18 (updated Thu Oct 22 17:13:28)
3: DISC remus --> romulus VIA id1,sd@n60018400000055659e5b0002/a,raw:512,id1,sd@n60018400000055659e5b0002/ a,raw:518: Up - last heartbeat #18 (updated Thu Oct 22 17:13:28)
4: DISC remus --> romulus VIA id1,sd@n60018400000055659e5b0004/a,raw:512,id1,sd@n60018400000055659e5b0004/ a,raw:518: Up - last heartbeat #18 (updated Thu Oct 22 17:13:28)
5: DISC remus --> romulus VIA id1,sd@n60018400000055659e5b0005/a,raw:512,id1,sd@n60018400000055659e5b0005/ a,raw:518: Up - last heartbeat #18 (updated Thu Oct 22 17:13:28)
6: NET romulus --> remus (updated Thu Oct 22 17:13:30) 7: NET romulus --> remus
VIA remus_priv: Up - last heartbeat #20
VIA remus: Up - last heartbeat #19 (updated
Thu Oct 22 17:13:30)
8: DISC romulus --> remus VIA
id1,sd@n60018400000055659e5a0001/a,raw:518,id1,sd@n60018400000055659e5a0001/ a,raw:512: Up - last heartbeat #19 (updated Thu Oct 22 17:13:30)
9: DISC romulus --> remus VIA id1,sd@n60018400000055659e5b0002/a,raw:518,id1,sd@n60018400000055659e5b0002/ a,raw:512: Up - last heartbeat #19 (updated Thu Oct 22 17:13:30)
10: DISC romulus --> remus VIA id1,sd@n60018400000055659e5b0004/a,raw:518,id1,sd@n60018400000055659e5b0004/ a,raw:512: Up - last heartbeat #19 (updated Thu Oct 22 17:13:30)
11: DISC romulus --> remus VIA id1,sd@n60018400000055659e5b0005/a,raw:518,id1,sd@n60018400000055659e5b0005/ a,raw:512: Up - last heartbeat #19 (updated Thu Oct 22 17:13:30)
12 heartbeats configured, 12 up, 0 down
root@romulus:~#
From the show command, you can see that when each service is added, rsfadm chooses two disks from the corresponding zpool to use for heartbeats, so now that we have two services, there are 4 disk heartbeats. Further rsfadm commands can be used to add or remove disk heartbeats if necessary.
Finally, the third pool - POOLC - needs to be added to the first service - POOLA - so that POOLA and POOLC fail over together:
root@romulus:~/rsf-1/rsfadm# rsfadm modify-svc POOLA create-pool POOLC Oct 22 17:44:23 romulus RSF-1[2090]: [ID 702911 local0.alert] RSF-1 hot restart: services may be running.
root@romulus:~/rsf-1/rsfadm# rsfadm show
Global information:
Cluster name : HA-Cluster
Nodes:
Services:
0 :
1 :
Heartbeats: 0: NET
Poll time : 1 Config CRC : 1c47
0 :
1 :
remus (remus) available
RSF-1 release 3.11.0p9, built on 16-Sep-2015-16:13
romulus (romulus) available
RSF-1 release 3.11.0p9, built on 16-Sep-2015-16:13
POOLA, IP address sales_staff-public, "RSF-1 cluster service" stopped automatic unblocked on remus
running automatic unblocked on romulus
POOLB, IP address support_staff-public, "RSF-1 cluster service"
remus --> romulus (updated Thu Oct 22 17:45:16) 1: NET remus --> romulus
VIA romulus_priv: Up - last heartbeat #53
VIA romulus: Up - last heartbeat #53 (updated
8: NET romulus --> remus (updated Thu Oct 22 17:45:16) 9: NET romulus --> remus
VIA remus_priv: Up - last heartbeat #53
VIA remus: Up - last heartbeat #53 (updated
running automatic stopped automatic
unblocked on remus unblocked on romulus
Thu Oct 22 17:45:17)
2: DISC remus --> romulus VIA
id1,sd@n60018400000055659e5b0007/a,raw:518,id1,sd@n60018400000055659e5b0007/ a,raw:512: Up - last heartbeat #53 (updated Thu Oct 22 17:45:17)
3: DISC remus --> romulus VIA id1,sd@n60018400000055659e5a0001/a,raw:512,id1,sd@n60018400000055659e5a0001/ a,raw:518: Up - last heartbeat #53 (updated Thu Oct 22 17:45:17)
4: DISC remus --> romulus VIA id1,sd@n60018400000055659e5b0002/a,raw:512,id1,sd@n60018400000055659e5b0002/ a,raw:518: Up - last heartbeat #53 (updated Thu Oct 22 17:45:17)
5: DISC remus --> romulus VIA id1,sd@n60018400000055659e5b0004/a,raw:512,id1,sd@n60018400000055659e5b0004/ a,raw:518: Up - last heartbeat #53 (updated Thu Oct 22 17:45:17)
6: DISC remus --> romulus VIA id1,sd@n60018400000055659e5b0005/a,raw:512,id1,sd@n60018400000055659e5b0005/ a,raw:518: Up - last heartbeat #53 (updated Thu Oct 22 17:45:17)
7: DISC remus --> romulus VIA id1,sd@n60018400000055659e5b0008/a,raw:518,id1,sd@n60018400000055659e5b0008/ a,raw:512: Up - last heartbeat #53 (updated Thu Oct 22 17:45:17)
Thu Oct 22 17:45:16)
10: DISC romulus --> remus VIA
id1,sd@n60018400000055659e5b0007/a,raw:512,id1,sd@n60018400000055659e5b0007/ a,raw:518: Up - last heartbeat #53 (updated Thu Oct 22 17:45:16)
11: DISC romulus --> remus VIA id1,sd@n60018400000055659e5a0001/a,raw:518,id1,sd@n60018400000055659e5a0001/ a,raw:512: Up - last heartbeat #53 (updated Thu Oct 22 17:45:16)
12: DISC romulus --> remus VIA id1,sd@n60018400000055659e5b0002/a,raw:518,id1,sd@n60018400000055659e5b0002/ a,raw:512: Up - last heartbeat #53 (updated Thu Oct 22 17:45:16)
13: DISC romulus --> remus VIA id1,sd@n60018400000055659e5b0004/a,raw:518,id1,sd@n60018400000055659e5b0004/ a,raw:512: Up - last heartbeat #53 (updated Thu Oct 22 17:45:16)
14: DISC romulus --> remus VIA id1,sd@n60018400000055659e5b0005/a,raw:518,id1,sd@n60018400000055659e5b0005/ a,raw:512: Up - last heartbeat #53 (updated Thu Oct 22 17:45:16)
15: DISC romulus --> remus VIA id1,sd@n60018400000055659e5b0008/a,raw:512,id1,sd@n60018400000055659e5b0008/ a,raw:518: Up - last heartbeat #53 (updated Thu Oct 22 17:45:16)
16 heartbeats configured, 16 up, 0 down
root@romulus:~/rsf-1/rsfadm#
Again from the show command, two more disk heartbeats have been added.

The full list of rsfadm commands can be found by running 'rsfadm –h':
root@romulus:~# rsfadm
rsfadm -h
CLI for administration of an RSF-1 cluster Usage:
rsfadm [OPTIONS] [command options] Options:
-v|--verbose
increase output debug level
-V|--version print version
Subcommands: discover-nodes
init
destroy
reset
show
create-hb
delete-hb
create-svc
delete-svc
modify-svc
create-vip delete-vip create-pool delete-pool create-hb delete-hb
control-svc help
root@romulus:~#
The 'help' subcommand can be used to get a summary of all commands, or more information about one command:
root@romulus:~# rsfadm help destroy
destroy
root@romulus:~#
Destroy cluster Usage:
rsfadm destroy [OPTIONS]
Options:
-f|--force
Force destroy the cluster
Prev Page
Next Page