Installing Pivotal CF on vSphere

Posted: December 13, 2013 | Author: Chris Greene | Filed under: Uncategorized | 2 Comments

In this post I’ll show how to install Pivotal’s CF on vSphere.  For more information on Pivotal’s CF see http://www.gopivotal.com/paas

You can download the deployment appliance at https://network.gopivotal.com/

After you download the appliance (OVA), deploy it into your vSphere environment.  I deployed mine into my management cluster.  The deployment process is straight forward and similar to most OVA/OVF deployments where you select the IP information, admin password, etc.  There is an issue with the current appliance where if there is a DHCP server on your network, the appliance’s network interface will pick up a DHCP address even if you have provided a static IP during the deployment.  This can be resolved by rebooting the appliance.  It will not pick up a DHCP address again after the reboot.  After the appliance has been deployed, access it via a web browser and log in.

Initial Setup

Once you log in you should see

Select vSphere.  The first category selected will be vCenter credentials.

Enter your information, save and select vSphere configuration.

Network name is the portgroup where the Pivotal CF VMs will be placed.  The rest of the fields should be self-explanatory.  Save and select Network configuration.

The information provided here will be used by your Pivotal CF VMs.  Select NTP servers when you’re finished.

Lastly, select Resource sizes and Install.  You should the following screen.

In vCenter you will see a few VMs being created.  The powered on VM should be your Micro BOSH VM.

If everything goes well, you will see

Install Elastic Runtime

Go back to the Product Dashboard, select Add Product and Add on Elastic Runtime.

Select HA Proxy, fill in the HA Proxy IP and Generate Self-Signed RSA Certificate.

For the HAProxy IP, enter the third IP after your reserved IP range.  If you receive a message that the HAProxy IP is already in use, try the next IP.  I believe two IPs are used by Pivotal CF after your reserved IP range so the third one should work.  Select Save and Cloud Controller.

In my lab I created a sub-domain named cf.vmware.local.

Important 

You need to create a wildcard record that points to your HAProxy, Cloud Controller or load balancer IP.  Since I didn’t fill in any info for the Cloud Controller and am not using a load balancer, I created a wildcard record that pointed to my HAProxy IP as such: *.cf.vmware.local > 192.168.70.53

If you don’t have the ability to modify DNS records in your environment, look at using xip.io.

Save and select SAML login.

For my environment the SSO URL was https://vc5c.vmware.local:7444/ims/STSService

Select Web Consoles and Resource sizes and make any changes needed.  The only change I made on Resources sizes was to set the DEA’s RAM to 8192 because of resource restrictions in my lab.

Select Install Updates when you’re ready.  You may receive a warning that you don’t have the required amount of CPU cores.  Press ignore to proceed.

The install process should take less than an hour and you will see a lot of activity in your specified vCenter resource pool.  VMs will be repeatedly cloned, powered on and off, etc.  Eventually you should see

Download the cf client

Now we need to download the cf client.  First, we need to get the credentials to login to the Pivotal CF management portal.  Go to Elastic Runtime > Credentials and grab the UAA administrator user username/password.

Connect to the Pivotal CF management portal via a browser.  The address should be something like https://console.cf.vmware.local.  Enter the UAA user credentials.

Select Tools on the navigation menu.

Download the CF CLI for your OS.

I was also able to install the cf client on a Ubuntu machine by running “sudo gem install cf”

Connect to the Pivotal CF instance

Once you have the client installed, can connect to your Pivotal CF instance.

cf target https://api.cf.vmware.local

cf login admin (UAA password used to log into the web portal)

Select 1 to go back to a command prompt.

Create VMware organization

cf create-org vmware

Create user chris@vmware.local in the VMware oganization

cf create-user chris@vmware.local -o vmware

Setting permissions for the new user

Go back to the web management portal and select Users > Org Roles > Development Roles.   Initially it will look like

Select everything for the new user

Logout as admin and log back in as the newly created user.

Go back to the cf client.

cf logout

cf target api.cf.vmware.local -o vmware -s development

cf login chris@vmware.local

Select 1 for Development

Authenticating… OK
1: development
2: production
3: staging
Space> 1

Switching to space development… OK

Push sample app to Pivotal CF

git clone https://github.com/andypiper/sinatra-static-web.git

cd sinatra-static-web/

chris@dev:~/cf/sinatra-static-web$ cf push
Name> firstapp

Instances> 1

1: 128M
2: 256M
3: 512M
4: 1G
Memory Limit> 256M

Creating firstapp… OK

1: firstapp
2: none
Subdomain> firstapp

1: cf.vmware.local
2: none
Domain> cf.vmware.local

Creating route firstapp.cf.vmware.local… OK
Binding firstapp.cf.vmware.local to firstapp… OK

Create services for application?> n

Save configuration?> n

Uploading firstapp… OK
Preparing to start firstapp… OK
—–> Downloaded app package (216K)
—–> Using Ruby version: ruby-1.9.3
—–> Installing dependencies using Bundler version 1.3.2
Running: bundle install –without development:test –path vendor/bundle –binstubs vendor/bundle/bin –deployment
Fetching gem metadata from https://rubygems.org/……….
—–> Downloaded app package (216K)
—–> Using Ruby version: ruby-1.9.3
—–> Installing dependencies using Bundler version 1.3.2
Running: bundle install –without development:test –path vendor/bundle –binstubs vendor/bundle/bin –deployment
Fetching gem metadata from https://rubygems.org/……….
dler cache.
—–> WARNINGS:
You have not declared a Ruby version in your Gemfile.
To set your Ruby version add this line to your Gemfile:”
ruby ‘1.9.3’”
# See https://devcenter.heroku.com/articles/ruby-versions for more information.”
—–> Uploading droplet (24M)
Checking status of app ‘firstapp’…
1 of 1 instances running (1 running)
Push successful! App ‘firstapp’ available at firstapp.cf.vmware.local
chris@dev:~/cf/sinatra-static-web$

Access the new app

Open a web browser and go to http://firstapp.cf.vmware.local

 

---

Reporting vCloud Organization Network IP Allocations with vCenter Orchestrator

Posted: December 11, 2013 | Author: Chris Greene | Filed under: Uncategorized | Leave a comment

In our environment we have a PowerCLI script that generates various dashboards.  One dashboard reports on org vDC usage and we needed to list the used / total IPs for org networks.  This information is easy to get with external networks:

connect-ciserver cloud

$externalNetwork = Get-ExternalNetwork “external network”

$externalNetwork.UsedIpCount

$externalNetwork.TotalIpCount

Unfortunately, the objects returned by Get-OrgNetwork don’t have these properties, and I wasn’t able to find a way to get them via PowerCLI.  I also didn’t see how to get them using the vCloud API, but didn’t look too hard so it may be there.  Fortunately, vCO org vDC network objects have a method named getAllocatedAddresses().  So I could get the information from vCO, but how would I get the data into the report that’s ran on the reporting server?

Creating a scheduled task

The current PowerCLI scripts are ran as scheduled tasks on the reporting server.  I decided to add another scheduled task on the reporting server that called the vCO workflow (Org Network IP Allocation Report).  I call the workflow using curl, which isn’t included with Windows so the first step was to download and install it from http://curl.haxx.se/download.html.  The scheduled tasks properties are:

Program/script: C:\Program Files (x86)\curl\curl.exe

Add arguments: -i -k -H “accept:application/xml” -H “content-type:application/xml;charset=UTF-8” -u vcoadmin:password -X POST https://vco.example.com:8281/api/workflows/e2e9b56a-7867-44a6-9b9f-3b3e69fe1175/executions/ -d “<execution-context xmlns=’http://www.vmware.com/vco’><parameters></parameters></execution-context>&#8221;

The workflow id is e2e9b56a-7867-44a6-9b9f-3b3e69fe1175.   You can get the workflow id by pressing ctrl-c while the workflow is highlighted in the vCO client or using the method in https://kickingwaterbottles.wordpress.com/2013/06/05/scheduling-vco-workflows-with-curl-and-the-rest-api/

The quoted value after -d is the data that you’re sending with the POST request.  The workflow takes no input so there are no parameters.

Workflow schema

Workfow details

The main work is done in the “Build network list” scripted task.  The results of this scripted task are inserted via the built-in workflow “Run program in guest” to insert a series of cvs values into a file on the reporting server.  This file is then read via PowerCLI’s Import-Csv cmdlet and the values are used to build the dashboard.

The contents of “Build network list” are:

A couple of notes

• The information reported works for us, but may not be exactly what you’d expect.  The IP allocation is off by one because I actually wanted to report the available IPs in the IP pools.  When using a network that creates a vSE gateway, the gateway will consume an internal IP and count as an allocated IP, but I wasn’t interested in that so I removed it.
• This workflow should probably be broken up to be more modular, but like a lot of things once I got the workflow done it was time to move on to something else.  I’ll try to fix it up at a later time, which of course won’t happen 🙂
• The for loop in getAvailableIps contains two values (startAddress and endAddress) that calculate the total number IPs based off a range.  This was the solution I came up with and probably terrible but it works.
• Just looking at the code makes me want to go and change a lot of things but if I waited for everything to be perfect, I’d never make a post.  I hope what I have will help someone, and I hope PowerCLI gets UsedIpCount and TotalIpCount properties for org networks so this isn’t necessary.
• I’m not a programmer so please forgive the code.

I thought this was an example of how vCO can be used.  Often times you run into a road block while trying to implement something and vCO is great at bypassing those roadblocks due to all the functionality it contains.  Here is the code for the Build network list scripted task.

networks = []

for each(var org in vcdHost.getOrganizations()) {
    for each(var orgVdc in org.getVdcs()) {
        for each (var orgVdcNetwork in orgVdc.getOrgVdcNetworks()) {
            ipUsage = getAvailableIps(orgVdcNetwork)
            fenceMode = orgVdcNetwork.configuration.fenceMode
            parentNetwork = "n/a"
            if (orgVdcNetwork.configuration.parentNetwork) {
                parentNetwork = getParentNetworkName(orgVdcNetwork.configuration.parentNetwork)
            }
            networks.push(org.name + "," + orgVdcNetwork.name + "," + fenceMode + "," + parentNetwork + "," + ipUsage[0] + "," + ipUsage[1])
        }
    }
}

function getAvailableIps(orgVdcNetwork) {
    network = null
    totalIps = 0
    usedIps  = 0

    if (orgVdcNetwork.configuration.fenceMode == "bridged") {
        usedIps = orgVdcNetwork.getAllocatedAddresses().length
    }
    else {
        // Substract one for networks that require a vSE since getAllocatedAddresses counts the vSE.  
        // The vSE's IP doesn't count towards the IP pool.  
        usedIps = orgVdcNetwork.getAllocatedAddresses().length -1
    }
    
    for each(var ipScope in orgVdcNetwork.configuration.ipScopes.ipScope.enumerate()) {
        for each (var ipRange in ipScope.ipRanges.ipRange.enumerate()) {
            startOctets = ipRange.startAddress.split('.')
            endOctets   = ipRange.endAddress.split('.')
            startAddress =  (16777216 * startOctets[0])/1 + (65536 * startOctets[1])/1 + (256 * startOctets[2])/1 + startOctets[3]/1 
            endAddress   =  (16777216 * endOctets[0])/1   + (65536 * endOctets[1])/1   + (256 * endOctets[2])/1   + endOctets[3]/1 
            totalIps += (endAddress - startAddress) + 1
        }
      }
    return ([usedIps, totalIps])
}

function getParentNetworkName(parentNetworkReference) {
    parentNetwork = vcdHost.getEntityByReference(VclFinderType.EXTERNAL_NETWORK, parentNetworkReference)
    return (parentNetwork.name)
}

---

Unable to delete vApp after failed creation

Posted: December 6, 2013 | Author: Chris Greene | Filed under: Uncategorized | Leave a comment

I ran into an issue today where a vApp wasn’t created successfully because of an API call to add a network to the vApp was made while the vApp was in a busy state.  The vApp networking section was empty, but if you tried to add the network to the vApp manually, you would receive an error that the network already existed on the vApp.  If you looked at the vApp through the API or REST client, no networks were visible, but the vCloud database was updated with the network.

When deleting vApp, the vApp went into a pending state and the following error was displayed.

java.lang.NullPointerException

at com.vmware.vcloud.dl.net.vappnetwork.impl.VAppNetworkManagerImpl.findVcdNetwork(VAppNetworkManagerImpl.java:2006)

at com.vmware.vcloud.dl.net.vappnetwork.impl.VAppNetworkManagerImpl.deleteFromDb(VAppNetworkManagerImpl.java:2110)

at com.vmware.ssdc.backend.services.impl.VAppManagerImpl.deleteFromDb(VAppManagerImpl.java:1204)

at com.vmware.ssdc.backend.services.impl.VAppManagerImpl.deleteVAppFromDb(VAppManagerImpl.java:1115)

at com.vmware.ssdc.backend.services.impl.VAppManagerImpl.deleteVAppFromDb(VAppManagerImpl.java:1075)

I reproduced the issue in my lab so I could experiment.  Trying to delete the row in the database for the VM resulted in:

Now I knew what table contained the vApp network: vapp_logical_resource.

Delete the invalid database entries

I’m not the best with SQL so these queries are probably terrible but they work.

vApp name: direct

vApps Table: vm_container

Find the invalid entry in vapp_logical_resource

select vapp_logical_resource.id from vm_container inner join vapp_logical_resource on vm_container.sg_id = vapp_logical_resource.vapp_id where vm_container.name = ‘direct’

Delete the invalid entry from vapp_logical_resource

delete from vapp_logical_resource where id = (select vapp_logical_resource.id from vm_container inner join vapp_logical_resource on vm_container.sg_id = vapp_logical_resource.vapp_id where vm_container.name = ‘direct’)

Update vApp status

Now that the invalid database entry is gone, you should be able to delete the vApp through vCloud, but first you need to set the creation_status field for the vApp to RESOLVED to clear the pending status.

Check the current status

select name, creation_status from vm_container where name = ‘direct’

direct,DELETING_CONTENTS

Update the status

update vm_container set creation_status = ‘RESOLVED’ where name = ‘direct’

Now you can right-click the vApp and select delete.

Of course if you’re going to do this on a system you care about, make sure to back up the database, be careful and get confirmation from VMware support.

 

 

---