Provisoning vRealize Automation Catalog Items with Powershell and REST

I’ve dabbled with Powershell in the past but never considered it one of my go to languages since it was only available on Windows. This limited my usage of Powershell to primarily working with VMware’s PowerCLI, but now that Powershell has been ported to Linux and OSX I’ve decided to put more effort into learning it. At work I’m working with vRealize Automation (vRA) so I thought I’d try to provision some vRA catalog items via Powershell and REST. Compared to other languages I’ve used, Powershell makes working wtih RESTful APIs very easy. Let’s get started.

This post is designed so you can follow along line by line. You can find a functionalized version on Github.

You can follow along by opening up a Powershell window. Let’s set up some variables:

$vraServer = ‘vra72.vmware.local’
$catalogItem = ‘CentOS 7 Base’
$username = ‘cloudadmin’
$password = ‘VMware1!’
$tenant = ‘vsphere.local’

Getting a login token

The first thing we need to do is get a login token from vRA. You can find out more about this in the vRA API Documentation. You can see an example request in curl here. We can get a login token by making a request to the following URL:

$url = “https://$($vraServer)/identity/api/tokens”

As described in the above links, we must be a POST request and the body of the request must be our username, password and tenant in JSON format. Powershell makes this really easy. Let’s start by defining a hash table with the required values:

$properties = @{‘username’ = $username; ‘password’ = $password; ‘tenant’ = $tenant}

If we view the contents of our $properties variable, we will see the following:

2017-01-24_21-15-46.png

Now let’s convert this into a Powershell object:

$bodyObject = New-Object –TypeName PSObject –Property $properties

Viewing the contents of $bodyObject gives us:

2017-01-24_21-18-01.png

We still need to convert this to JSON as this is what the request for a token requires. This can easily be done like so:

$body = $bodyObject | ConvertTo-Json

The $body variable looks like this:

2017-01-24_21-19-47.png

If you’ve worked with JSON in the past this should look familiar.

Now we are going to request another hash table containing our headers:

$headers = @{“Content-Type” = “application/json”; “Accept” = “application/json”}

We are now ready to request our login token:

$request = Invoke-WebRequest $url -Method POST -Headers $headers -Body $body

Here you can see we used Poweshell’s Invoke-WebRequest commandlet. We specified the method as POST along with our headers and body. The $request variable looks like this:

2017-01-24_21-23-03.png

If you look at Content you’ll see that it contains our token along with other information such as the expiration date. The value of Content is a Powershell String, but we want it in JSON format so we can parse out the id. We can do this by running:

$content = $request.content | convertFrom-json

$content contains:

2017-01-24_22-17-29.png

To get the id, we just need to grab the id field:

$bearerToken = $content.id

The $bearerToken variable will look something like this:

2017-01-24_21-28-28.png

Requesting a catalog item

To request a catalog item we need to perform the following:

  1. Make a GET request on the entitled catalog items
  2. Make a GET request to get the view of our catalog item
  3. Get a the request template for our catalog item
  4. Make a POST request for our catalog item

I’m not going to go into as much detail as in the Getting a login token section as it’s the same pattern over and over.

In the following block of code we specify the URL that contains our entitled catalog items, define $headers again but this time we specify an Authorization header and use our bearer token, make our request and convert the returned content into JSON:

$url = “https://$($vraServer)/catalog-service/api/consumer/entitledCatalogItems/”
$headers = @{“Content-Type” = “application/json”; “Accept” = “application/json”; “Authorization” = “Bearer ${bearerToken}”}
$request = Invoke-WebRequest $url -Method GET -Headers $headers
$content = $request.Content | ConvertFrom-Json

If we look at the content field of $contentJson, we see we have various catalog items:

2017-01-24_22-18-34.png

How do we know what’s in each of these? Let’s grab the first one item and see what type of Poweshell object it is:

2017-01-24_22-19-01.png

So we can see that the BaseType is an object. This means we can use the Get-Member commandlet to get all the objects members or fields:

2017-01-24_22-19-51.png

The catalogItem field looks promising. Let’s check it out:

2017-01-24_22-20-16.png

OK, so we know we look at $contentJson.content we get a list of catalog items and we can look at the catalogItem field to get details about each one. To find which item we are looking for $catalogItem, we can do the following:

$consumerEntitledCatalogItem = $content.content | where { $_.catalogItem.name -eq $catalogItem }

Let’s verify that we have the correct catalog item:

2017-01-24_21-55-01.png

In the next block we will get the catalog item’s id since that’s what we will append to the catalog item view URL, make our request and convert the results to a Powershell object:

$consumerEntitledCatalogItemId = $consumerEntitledCatalogItem.catalogItem.id
$url = “https://$($vraServer)/catalog-service/api/consumer/entitledCatalogItemViews/$($consumerEntitledCatalogItemId)”
$request = Invoke-WebRequest $url -Method GET -Headers $headers
$content = $request.Content | ConvertFrom-Json

If we look at at the links field of $content, we will see:

2017-01-24_22-03-52.png

We need the href that ends in template and we can get it like this:

$requestTemplateURL = $content.links | ? { $_.rel -eq ‘GET: Request Template’ }

Let’s go ahead and get the POST URL while we’re at it:

$requestPOSTURL = $content.links | ? { $_.rel -eq ‘POST: Submit Request’ }

Now we can make a request to $requestTemplateURL. This will allow us to access a “template” of the request data. Once we have this we can either use the same data to make a request or modify somehow like changing the number of CPUs we want:

$request = Invoke-WebRequest $requestTemplateURL.href -Method GET -Headers $headers

2017-01-24_22-11-50.png

Now we have everything we need to make the catalog item request:

$request = Invoke-WebRequest $requestPOSTURL.href -Method POST -Headers $headers -body $request.content

If the request was successful, you should see a status code of 201 and the request should be In Progress in vRA:

2017-01-24_22-13-14.png

Here are all of the steps:

$vraServer = ‘vra72.vmware.local’
$catalogItem = ‘CentOS 7 Base’
$username = ‘cloudadmin’
$password = ‘VMware1!’
$tenant = ‘vsphere.local’

$url = “https://$($vraServer)/identity/api/tokens”

$properties = @{‘username’ = $username; ‘password’ = $password; ‘tenant’ = $tenant}

$bodyObject = New-Object –TypeName PSObject –Property $properties

$body = $bodyObject | ConvertTo-Json

$headers = @{“Content-Type” = “application/json”; “Accept” = “application/json”}

$request = Invoke-WebRequest $url -Method POST -Headers $headers -Body $body
$content = $request.content | convertFrom-json
$bearerToken = $content.id

$url = “https://$($vraServer)/catalog-service/api/consumer/entitledCatalogItems/”
$headers = @{“Content-Type” = “application/json”; “Accept” = “application/json”; “Authorization” = “Bearer ${bearerToken}”}
$request = Invoke-WebRequest $url -Method GET -Headers $headers
$content = $request.Content | ConvertFrom-Json
$consumerEntitledCatalogItem = $content.content | ? { $_.catalogItem.name -eq $catalogItem }
$consumerEntitledCatalogItemId = $consumerEntitledCatalogItem.catalogItem.id
$url = “https://$($vraServer)/catalog-service/api/consumer/entitledCatalogItemViews/$($consumerEntitledCatalogItemId)”
$request = Invoke-WebRequest $url -Method GET -Headers $headers
$content = $request.Content | ConvertFrom-Json
$requestTemplateURL = $content.links | ? { $_.rel -eq ‘GET: Request Template’ }
$requestPOSTURL = $content.links | ? { $_.rel -eq ‘POST: Submit Request’ }
$request = Invoke-WebRequest $requestTemplateURL.href -Method GET -Headers $headers
$request = Invoke-WebRequest $requestPOSTURL.href -Method POST -Headers $headers -body $request.content

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Advertisements

Getting Started with vSphere Integrated Containers using vRealize Automation

Overview

In this post how show how to set up vSphere Integrated Containers (VIC) and integrate them into vRealize Automation (vRA). The vSphere Integrated Containers documentation can be found here. I’ll only reference the Install Guide in this post.

vSphere Integrated Container resources:

The following software versions were used:

  • vSphere Integrated Containers 1.0 (or 0.8?)
  • vRealize Automatin 7.2
  • vCenter 6.0 U2 Build:3634794
  • ESXi: 6.0 U2 Build: 3620759

Read the rest of this entry »


PowerCLI script to monitor recent vSphere Tasks

I haven’t had much time to look at vSphere 6.5 yet, but appearently the C# client is no more. I used to always have the C# client up as a diagnostic tool so that I could watch events trigger in real time as opposed to having to refresh the vSphere web client. Normally I’d open the C# client in full screen and then arrange my browsers so that they would cover the C# client but leave room so I’d be able to see the recent tasks pane. This would allow me to work in another product like vRealize Automation and see events appear in vCenter immediately.

If I can’t use the C# client to do this going forward, I thought I’d create a little PowerCLI script to provide me with the same information. I’m not sure how well it will work out yet, but you can find it on GitHub.

The result look like this:

2016-11-29_20-03-58.png


Simple Powershell Script to Monitor vRealize Automation

Lately I’ve been learning vRealize Automation (vRA) and it has involved bringing the environment up and down frequently as well as breaking and fixing various services to see how the system would respond. I got tired of going into the VAMI (port 5480 of the vRA appliance) and selecting the Refresh button to get status on all of the various services so I created a little Powershell script that will display the status of the vRA services.

The simplest way to invoke the script is with:

get-vRAHealth vra71.vmware.local

Where vra71.vmware.local is my load balancer VIP for vRA. By default the script will continously refresh every 5 seconds.

You can disable the looping like so:

get-vRAHealth vra71.vmware.local -loop $false

And control the refresh interval:

get-vRAHealth vra71.vmware.local -refresh 10

Here is the output:

2016-11-07_22-10-21.png

The script can be found on GitHub and below:

function get-vRAHealth() {
  <#    .SYNOPSIS     Displays health status of vRA components   .DESCRIPTION      Displays health status of vRA components   .EXAMPLE     get-vRAHealth vra71.vmware.local   .EXAMPLE     get-vRAHealth https://vra71.vmware.local -loop $true   .EXAMPLE     get-vRAHealth https://vra71.vmware.local -loop $true $sleep 2   #&amp;amp;gt;

  param(
    [Parameter(Mandatory=$true,Position=0)]
    [string]$url,

    [Parameter(Mandatory=$false,Position=1)]
    [string]$loop=$true,

    [Parameter(Mandatory=$false,Position=2)]
    [Int32]$refresh=5
  ) 

  $uri = [System.Uri] $url

  if ($uri.Host -eq $null -and $uri.OriginalString) {
    $uri = [System.Uri] "https://$($uri.OriginalString)"
  }

  if ($uri.Scheme -eq 'http') {
    $uri = [System.Uri] "https://$($uri.Host)"
  }

  if ($uri.LocalPath -ne '/component-registry/services/status/current') {
    $uri = [System.Uri] "$($uri.AbsoluteUri)component-registry/services/status/current"
  }

  while ($true) {
    clear
    Write-Host "Checking $($uri.AbsoluteUri)"

    try {
      $content = Invoke-WebRequest $uri.AbsoluteUri

      if ($content.StatusCode -eq 200) {
        $json = $content.Content | ConvertFrom-Json
        $json.content | select serviceName, `
	                  @{N='Registered';E={ $_.serviceStatus.serviceInitializationStatus }}, `
	           	  @{N='Available';E={ if (!$_.notAvailable) {'True'} else {'False'}}}, `
	                       lastUpdated, `
		               statusEndPointUrl `
		      | ft -auto
        if ($loop -eq $false) { break }
      } else {
          Write-Host "Unable to access vRA Component Registry. Error: $content.StatusCode"
      }
    } catch {
       Write-Host "Unable to access vRA Component Registry. Error: $_.Exception.Message."
  }
  sleep $refresh
  }
}

HAProxy as a Load Balancer for vRealize Automation

In this post I’m going to show how to use HAProxy as a load balancer for vRealize Automation. I used Ubuntu 14.04 LTS for the OS.

Install HAProxy

 sudo apt-get install haproxy

Add sub interfaces to VM

/etc/network/interfaces

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet dhcp

auto eth0:0
iface eth0:0 inet static
address 192.168.3.7
netmask 255.255.255.0

auto eth0:1
iface eth0:1 inet static
address 192.168.3.10
netmask 255.255.255.0

auto eth0:2
iface eth0:2 inet static
address 192.168.3.27
netmask 255.255.255.0

HAProxy Config

A lot of this config are the defaults. I added the section so you can enable to the LB stats page. The bottom has my edits for vRA. There are three sections for the appliance, IaaS Web and IaaS Manager. I’m not an HAProxy expert so probably has some things that could be improved. I tried to add all the recommendations (persistence, load balancing policy, timeout, etc) as described in the vRA LB Guide.

/etc/haproxy/haproxy.cfg


global
 log /dev/log local0
 log /dev/log local1 notice
 chroot /var/lib/haproxy
 stats socket /run/haproxy/admin.sock mode 660 level admin
 stats timeout 30s
 user haproxy
 group haproxy
 daemon
 debug

maxconn 2048
ssl-server-verify none

# Default SSL material locations
 ca-base /etc/ssl/certs
 crt-base /etc/ssl/private
 tune.ssl.default-dh-param 2048

# Default ciphers to use on SSL-enabled listening sockets.
 # For more information, see ciphers(1SSL). This list is from:
 # https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/
 ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS
 ssl-default-bind-options no-sslv3

defaults
 log global
 mode http
 option forwardfor
 option forwardfor
 option httplog
 option dontlognull
 timeout connect 5000
 timeout client 50000
 timeout server 50000
 errorfile 400 /etc/haproxy/errors/400.http
 errorfile 403 /etc/haproxy/errors/403.http
 errorfile 408 /etc/haproxy/errors/408.http
 errorfile 500 /etc/haproxy/errors/500.http
 errorfile 502 /etc/haproxy/errors/502.http
 errorfile 503 /etc/haproxy/errors/503.http
 errorfile 504 /etc/haproxy/errors/504.http

listen stats
 bind 192.168.3.201:80
 mode http
 log global
 stats enable
 stats uri /stats
 stats realm Haproxy\ Statistics
 stats auth admin:VMware1!

# vRA 7.1 Distributed

# vRA VA

frontend vra71-va
 bind 192.168.3.7:443 ssl crt /etc/ssl/private/wildcard.pem
 mode http
 default_backend vra71-va-backend

backend vra71-va-backend
 mode http
 balance roundrobin
 stick on src table vra71-va-backend
 stick-table type ip size 200k expire 30m
 default-server inter 3s
 timeout check 10s
 option httpchk GET /vcac/services/api/health
 http-check expect status 204

 server vra71c 192.168.3.11:443 check ssl verify none
 server vra71d 192.168.3.12:443 check ssl verify none

# vRA IaaS Web

frontend vra71-iaas-web
 bind 192.168.3.10:443 ssl crt /etc/ssl/private/wildcard.pem
 mode http
 default_backend vra71-iaas-web-backend

backend vra71-iaas-web-backend
 mode http
 balance roundrobin
 stick on src table vra71-iaas-web-backend
 stick-table type ip size 200k expire 30m
 default-server inter 3s
 timeout check 10s
 option httpchk GET /wapi/api/status/web
 http-check expect string REGISTERED

 server vra71c-web 192.168.3.13:443 check ssl verify none
 server vra71d-web 192.168.3.14:443 check ssl verify none

# vRA IaaS Mgr

frontend vra71-iaas-mgr-https
 bind 192.168.3.27:443 ssl crt /etc/ssl/private/wildcard.pem
 mode http
 default_backend vra71-iaas-mgr-backend

backend vra71-iaas-mgr-backend
 mode http
 balance roundrobin
 stick on src table vra71-iaas-mgr-backend
 stick-table type ip size 200k expire 30m
 default-server inter 3s
 timeout check 10s
 option httpchk GET /VMPSProvision
 http-check expect rstring ProvisionService

 server vra71c-mgr 192.168.3.25:443 check ssl verify none
 server vra71d-mgr 192.168.3.26:443 check ssl verify none

Certificate

This is the wildcard cert I’m using for vRA. You just need to include the cert and private key in the proper order.

/etc/ssl/private/wildcard.pem 

-----BEGIN CERTIFICATE-----
MIIF7zCCA9egAwIBAgICEAkwDQYJKoZIhvcNAQELBQAwgYsxCzAJBgNVBAYTAlVT
MREwDwYDVQQIDAhDb2xvcmFkbzEMMAoGA1UECgwDTGFiMQwwCgYDVQQLDANMYWIx
...
w5HvHhi/K6f1qeeBr+xKxTEvz3gfQvxEgSxMmMRbffqGM4UbMHkDuJq4H4yrow48
XavIE+zwl1EiDvzEcz5ThbAWSL5fRu6SB0eeYldr4uEGJ/8=
-----END CERTIFICATE-----
-----BEGIN RSA PRIVATE KEY-----
MIIEpQIBAAKCAQEAwekJRhfC3NHja9waE5W0lxA3HebfThF9nMbUpoYUK+TvFKz7
Mkl9mUp/RS/YDYsVnQ3cUNx83bITDmc3EbIVYzF8rMv1BjQCM4ewrhhbQuBnivoI
...
7XcWYfeZuFz2GJ+3+Wt6EzEaV3DmoU0nuULRkoOSFi7FXCxsFLPVzzuZZgRWXFiN
q6p+3O9rYgelJ0P4a5mtPlWdJJZ2bAe9A0tB/px+xdFtuEuzyed0gbA=
-----END RSA PRIVATE KEY-----

Stats Page

Here is an example of what the HAProxy stats page looks like:

2016-10-10_21-09-59.png

 

 


NetApp ONTAP 9 Simulator Guide by Neil Anderson

I want to take a moment to mention the great work Neal Anderson did in his NetApp ONTAP 9 Simulator Guide. This incredibly detailed (117 pages) guide not only shows you how to set up a NetApp ONTAP 9 simulator for your lab, but you’ll also learn how install/configure:

Storage is critical to any infrastructure and while you’re learning it’s easy to quickly install a storage solution that’s not representative of what you’ll see in the real world. I try to make my lab environment match what I see in real production environments as much as possible. That means using segregated networks with VLANs, firewalls, load balancers, CA signed certificates, and with the help this guide you’ll be able to implement a real world storage solution.


Kubernetes 1.3 HA Walkthrough – NGINX

Table of Contents

You can find all of the config files on the GitHub page.

Overview

Let’s test out the environment by installing NGINX.

Create the NGINX deployment

We are going to create an NGINX deployment where we will have an NGINX replica on each of our Kubernetes worker nodes. A replica will be Kubernetes pod:

kubectl run nginx –image=nginx –port=80 –replicas=3

If successful, you will see: deployment “nginx” created

Let’s get a listing of each of the pods:

kubectl get pods -o wide

NAME                   READY STATUS  RESTARTS AGE IP          NODE
nginx-2032906785-44ndh 1/1   Running 2        16d 172.16.43.3 kube-worker2
nginx-2032906785-mvpft 1/1   Running 2        16d 172.16.19.2 kube-worker1
nginx-2032906785-tyfmu 1/1   Running 2        16d 172.16.23.2 kube-worker0

Here we can see that there is a pod on each of the worker nodes. Notice the IP address. Each of the pods (not docker container) will get an IP address from a network that was given out by etcd. Each worker node will be given a unique network range by etcd.

Create the NGNIX service

Now we need to create a Kubernetes Service to expose our NGINX deployment so we can reach it. I’m going to make the service of type NodePort, which will make the service accessible on any pod using the same port.

kubectl expose deployment nginx –port=80 –type=NodePort

View the service’s details

kubectl describe services nginx

Name:             nginx
Namespace:        default
Labels:           run=nginx
Selector:         run=nginx
Type:             NodePort
IP:               172.16.242.11
Port:              80/TCP
NodePort:          31153/TCP
Endpoints:        172.16.19.2:80,172.16.23.2:80,172.16.43.3:80
Session Affinity: None

Notice the NodePort and Endpoints. The Endpoints are our Kubernetes pods. If we launch a web browser and go to one of our worker nodes and use port 31153, we should see NGINX:

2016-10-09_16-47-19.png

Lastly, let’s check out one of the pod’s details:

kubectl describe pod nginx-2032906785-44ndh

Name: nginx-2032906785-44ndh
Namespace: default
Node: kube-worker2/192.168.3.184
Start Time: Thu, 22 Sep 2016 20:20:45 -0600
Labels: pod-template-hash=2032906785
 run=nginx
Status: Running
IP: 172.16.43.3
Controllers: ReplicaSet/nginx-2032906785
Containers:
 nginx:
 Container ID: docker://19ffedded8de834da2e072f012c5081655b7149172d2c00d31944c7fe2499766
 Image: nginx
 Image ID: docker://sha256:ba6bed934df2e644fdd34e9d324c80f3c615544ee9a93e4ce3cfddfcf84bdbc2
 Port: 80/TCP
 State: Running
 Started: Sat, 08 Oct 2016 14:59:03 -0600
 Last State: Terminated
 Reason: Completed
 Exit Code: 0
 Started: Sun, 02 Oct 2016 14:38:18 -0600
 Finished: Mon, 03 Oct 2016 08:27:44 -0600
 Ready: True
 Restart Count: 2
 Environment Variables: <none>
Conditions:
 Type Status
 Initialized True
 Ready True
 PodScheduled True
Volumes:
 default-token-jeqk5:
 Type: Secret (a volume populated by a Secret)
 SecretName: default-token-jeqk5
QoS Tier: BestEffort
No events.

I’m not going to go into all of the details here, but you can read the Kubernetes Documentation.