Recovering vCloud Director from disabled DRS on a vCenter Cluster

When you disable DRS on a vCenter cluster all resource pools in the cluster are deleted.  When vCloud is managing a vCenter cluster it creates various resource pools to store its VMs in, and if the vCenter resource pools are deleted, vCloud will be unable to manage its VMs.  In this article I’m going to how to recover from disabling DRS on a vCloud controlled vCenter cluster.   The best way to recover from this issue is to restore the vCenter & vCloud databases to the same point in time.  The problem with this is that you lose any information since the backups and may be left with VMs on the hosts that vCloud and vCenter don’t know about.  I believe that VMware also has a tool that can fix this issue so I’d call them as well.  This post is more informational and to reference if you’re all of of options.  Since we will be editing the vCloud database in this post, please make sure you back it up before proceeding.

vCloud / vCenter Configuration

I have a single org vDC named ‘admin-payg’.

2014-04-29_12-16-12

Here is the view from vCenter.

2014-04-29_9-06-45

Inspecting the databases

Now I’m going to query the vCenter and vCloud databases to show the current data.

vCenter is vc55a

vCloud is vcd55b

Let’s look at vCenter first.

Display vCenter’s resource pools

use vc55a select * from VPX_entity where TYPE_ID = 4
2014-04-29_11-37-35

The id column contains the resource pool moref so the moref of the admin-payg resource pool is 302.  Keep this in mind.

Now we will look at vCloud’s database.

Display all org vDCs and their related vCenter resource pool

use vcd55b select namesub_rp_moref from org_prov_vdc
2014-04-29_11-46-05

We see that the vCloud org vDC ‘admin-payg’ is mapped to a vCenter resource pool with a moref of 302.

Display the vCloud vApps, their vCloud org vDC and the moref of the underlying resource pool

use vcd55b select vm_container.name , org_prov_vdc.name, org_prov_vdc.sub_rp_moref from vm_container, org_prov_vdc where vm_container.org_vdc_id = org_prov_vdc.id
2014-04-29_11-39-16

We can see that our Linux-Minimal vApp is in the admin-payg org vDC and that org vDC’s resource pool.

Display org vDC resource pools and their morefs from the vrp_rp table

use vcd55b select name, sub_rp_moref from vrp, vrp_rp where vrp_rp.vrp_id = vrp.id
2014-04-29_11-44-08

Disabling DRS on the cluster

2014-04-29_11-48-47

2014-04-29_11-48-59

Starting with vCenter 5.1, you can now save resource pool trees.  See Retaining resource pools when disabling VMware DRS clusters for more info.

When we disable DRS, save the resource pool snapshot.

2014-04-29_11-49-16

After DRS has been disabled you can see that there are no resource pools in the cluster.

2014-04-29_11-49-35

If we try to power the vApp on in vCloud, we get:

2014-04-29_11-50-13

Restoring the resource pool

Turn DRS back on and then perform the following:

2014-04-29_11-52-08

Select the resource pool tree snapshot and upload it.

Inspecting the databases after the resource pool tree has been restored

use vc55a select * from VPX_entity where TYPE_ID = 4

2014-04-29_11-53-16

Remember earlier that the ‘admin-payg’ resource pool had a moref of 302?  Now it’s 307 and vCloud will be confused by this.

Here are a few queries and their results.

use vc55a select * from VPX_entity where TYPE_ID = 4
use vcd55b select name, sub_rp_moref from org_prov_vdc
use vcd55b select name, sub_rp_moref from vrp, vrp_rp where vrp_rp.vrp_id = vrp.id
use vcd55b select vm_container.name, org_prov_vdc.name, org_prov_vdc.sub_rp_moref from vm_container, org_prov_vdc where vm_container.org_vdc_id = org_prov_vdc.id
use vcd55b select name, sub_rp_moref from org_prov_vdc

2014-04-29_11-55-59

 Getting vCloud back in sync with vCenter

From the queries above we can see that vCenter has our ‘admin-payg’ resource pool moref as 307 while vCloud still thinks it’s 302.  We need to update the following tables with the new moref of 307:

vrp_rp
org_prov_vdc

update vrp_rp set sub_rp_moref = ‘307’ where sub_rp_moref = ‘302’

update org_prov_vdc set sub_rp_moref = ‘307’ where sub_rp_moref = ‘302’

After these tables have been update vCloud will be able to manage the VMs in these resource pools.

 

 

Advertisements

4 Comments on “Recovering vCloud Director from disabled DRS on a vCenter Cluster”

  1. Elad says:

    Hi,

    Your page help me allot and I recovered all resource groups for my vCloud.
    BTW when running the 2 queries,

    update vrp_rp set sub_rp_moref = ‘307’ where sub_rp_moref = ‘302’
    update org_prov_vdc set sub_rp_moref = ‘307’ where sub_rp_moref = ‘302’

    it wont find/update anything, you will need to add for example ‘resgroup-307’ instead the ‘307’

    Thanks,

    Elad

  2. Elad says:

    Hi,

    I found how to fix the problem with the System vDC resource group (this resource group is in charge of all Edge GW routers)
    I used this link from VMware https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2073392

    To find the new System vCD ID number on the vCenter SQL database run the command – ‘use vc.database select * from VPX_entity where TYPE_ID = 4’ look for the System vDC ID number
    To find the Service_rp_moref old ID run the command on the vCloud SQL server – ‘use vcloud select * from vc_computehub’

    For example:

    The new ID for the System vDC that we got from the vCenter database is 51783
    The old ID that we have on the vCloud database is – 1311

    To fix it run the command on the vCloud database – update vc_computehub set service_rp_moref = ‘resgroup-51873’ where service_rp_moref = ‘resgroup-1311’

    Reboot the vCloud server and you done.

    Regards,

    Elad


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s