IPA Replica Uninstall Error - Fixed

Issue

When removing a disconnected IPA replica server which had replication agreements with multiple IPA hosts, the meta-data for that replica is still contained on still running IPA servers. This does not allow you to remove the disconnected IPA server completely. There are special tasks you can use to remove this meta-data. For example:

Raw

Initial topology:  (simple triangle)
2 - 3
 \ /
  1

# on host1:
ipa-replica-manage disconnect host1 host2
ipa-replica-manage del host3  # not sure if this one is relevant here

# on host2:
ipa-server-install --uninstall -U

# on host1:
ipa-replica-prepare -p $ADMINPW --ip-address=$HOST2_IP $HOST2

# on host2:
sftp root@$HOST1:/var/lib/ipa/replica-info-$HOST2.gpg
ipa-replica-install -U --setup-dns --forwarder=$DNSFORWARD -w $ADMINPW -p $ADMINPW /dev/shm/replica-info-$HOST2.gpg
...
The host qe-blade-04.testrelm.com already exists on the master server. Depending on your configuration, you may perform the following:

Remove the replication agreement, if any:
    % ipa-replica-manage del qe-blade-04.testrelm.com
Remove the host entry:
    % ipa host-del qe-blade-04.testrelm.com

# on host1:
ipa-replica-manage del $HOST2
'$HOST1' has no replication agreement for '$HOST2'

ipa host-del $HOST2
ipa: ERROR: invalid 'hostname': An IPA master host cannot be deleted or disabled

Resolution

Its recommended to take a full backup first before deleting any entries from LDAP database.

Now delete the orphaned IPA master references from LDAP database.

Raw

ldapdelete -r -x -D "cn=Directory Manager" -W 'cn=hostname.domain.com,cn=masters,cn=ipa,cn=etc,dc=domain,dc=com'

My Case:

ldapdelete -r -x -D "cn=Directory Manager" -W 'cn=ipaserver2.infra.alphacloud.ae,cn=masters,cn=ipa,cn=etc,dc=infra,dc=alphacloud,dc=ae'

This bug is being worked upon by IPA engineering in the following BZ:
https://bugzilla.redhat.com/show_bug.cgi?id=826677

Make sure dse.ldif does not contain any replication agreement for deleted host.

Root Cause

In a replicated environment, when you disconnect a master, the meta-data for that master is still contained in the other servers.

vdsmd restart to stabilize RHEV host

I am observing RHEV environment closely and found out that after several fixes, it runs like a charm. But, There is a certain activity that I have to perform every month to keep it stable. That activity is below command:


MAKE SURE OF NO VM is running on that host before running below command:

service vdsmd restart; sleep 90; service vdsmd restart

This command has some constraints; like:

1) There should be no VMs on this Host. VMs should not be migrating on or from this host
2) VMs should not be in a "?" state (if it's ? state VMs will reboot)
3) Run this command only if host shows inconsistent behavior

Please use this information to make it bug free.

RHEV-Mamanger (RHEVM) Console for maintenance

For maintenance purposes, you can use VNC console:

Find out vnc port of the VM via either virsh command or via virt-viewer file (from rhevm itself) and use tightVNC to get the console and do maintenance on RHEV Manager.
------------
[virt-viewer]
type=vnc
host=10.192.72.11
port=5903
password=bSl5QqDaF7IV