Azure Service Fabric is a distributed systems platform that makes it easy to package, deploy, and manage scalable and reliable microservices. It can be deployed on Azure, but also on-premise, thus making OpenStack a great choice for the latter.
We are going to use Juju to deploy our Azure Service Fabric cluster directly on OpenStack with Active Directory for authentication / authorization and HAProxy to do a reverse proxy for the GUI and API endpoints across the cluster.
Deployment steps
Before anything else, make sure the following prerequisites are available in your OpenStack environment:
- Make sure OpenStack images for both Windows Server 2012 R2 or 2016 and Ubuntu Xenial are available in Glance. Any OpenStack supported hypervisor works, including Hyper-V, KVM and ESXi. Bare Metal (Ironic) works as well.
- Create a Neutron private network connected with a router to a public network that has two unallocated floating IPs available for the deployment.
- Set up the Nova flavors to be used by Juju in order to boot the instances, see here for more details.
Once all the prerequisites are met, install the Juju binaries by executing the following commands on a Ubuntu Xenial machine that has access to the OpenStack cloud API:
|
1 2 3 |
sudo add-apt-repository ppa:juju/stable --y sudo apt-get update sudo apt-get install juju-2.0 -y |
After Juju is installed, you need to add your OpenStack cloud API endpoint details. First of all, create a yaml file with the following structure:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
clouds: openstack: type: openstack auth-types: [userpass] regions: RegionOne: endpoint: '<keystone_auth_url>' credentials: openstack: default-region: RegionOne default-credential: openstack_user openstack_user: auth-type: userpass username: '<keystone_user>' password: '<keystone_password>' tenant-name: '<keystone_tenant_name>' domain-name: default |
Now, execute the following commands to let Juju know about your cloud:
|
1 2 |
juju add-cloud openstack <path_to_yaml_file> juju add-credential openstack -f <path_to_yaml_file> |
Generate Juju simplestreams metadata:
|
1 2 3 4 5 6 7 8 |
KEYSTONE_ENDPOINT="<keystone_auth_url>" WIN2012R2_UUID="<win2012r2_glance_image_uuid>" XENIAL_UUID="<xenial_glance_image_uuid>" juju metadata generate-image -a amd64 -i $WIN2012R2_UUID -s win2012r2 -d $HOME \ -u $KEYSTONE_ENDPOINT -r RegionOne juju metadata generate-image -a amd64 -i $XENIAL_UUID -s xenial -d $HOME \ -u $KEYSTONE_ENDPOINT -r RegionOne |
Now, you are ready to create a Juju controller instance on OpenStack (this will also allocate and associate the first floating IP):
|
1 2 3 4 5 6 7 |
juju bootstrap openstack openstack \ --constraints "mem=2G" --metadata-source=$HOME \ --config agent-metadata-url=https://streams.canonical.com/juju/tools/ \ --config default-series=xenial \ --config network=<private_network_name> \ --config use-floating-ip=true juju model-config use-floating-ip=false |
We are now going to use Juju charms to deploy and configure a three node Azure Service Fabric cluster with Active Directory (AD) node-to-node and client-to-node security. On top of this we will add HAProxy to load-balance requests for the GUI and API endpoints of the Azure Service Fabric cluster.
In this scenario, we deploy charms for the following services: Azure Service Fabric, Active Directory and HAProxy.
The Azure Service Fabric charm needs the following resources in order to be deployed. Those need to be downloaded on your Juju client machine before starting the actual deployment:
- .NET framework version 4.5.1 (or higher);
- Service Fabric standalone zip package (section Download the Service Fabric standalone package).
Once you have your Juju resources ready, the following commands will do the magic and take care of the entire deployment:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
juju deploy cs:~cloudbaseit/azure-service-fabric \ --num-units 3 --series win2012r2 --constraints "mem=3G" \ --resource dotnet-installer="<dot_net_framework_installer_path>" \ --resource asf-zip-package="<service_fabric_zip_package_path>" juju config azure-service-fabric security-type=Windows \ change-hostname=True juju deploy cs:~cloudbaseit/active-directory \ --series win2012r2 --constraints "mem=3G" juju config active-directory administrator-password="Passw0rd" \ safe-mode-password="Passw0rd" \ domain-user="jujuadmin" \ domain-user-password="Passw0rd" \ domain-name="cloudbase.local" \ change-hostname=True juju deploy cs:haproxy --series xenial --constraints "mem=2G" juju expose haproxy juju add-relation azure-service-fabric active-directory juju add-relation azure-service-fabric haproxy |
You can monitor the deployment progress:
|
1 |
watch juju status |
This is how the deployment looks like in the Juju GUI:
After the deployment finishes, allocate and assign the second floating IP to the HAProxy instance by executing the following commands:
|
1 2 3 4 5 6 |
JUJU_MACHINE_ID=$(juju status | grep haproxy/0 | awk '{print $4}') FLOATING_IP=$(neutron floatingip-create '<public_network_name>' | \ grep floating_ip_address | awk '{print $4}') NOVA_MACHINE_ID=$(nova list | egrep "juju-.+-default-$JUJU_MACHINE_ID" | awk '{print $2}') nova floating-ip-associate $NOVA_MACHINE_ID $FLOATING_IP echo "Associated floating IP $FLOATING_IP to HAProxy instance" |
Given the HAProxy public address, you may then:
- Query the API at the following endpoint: <haproxy_floating_ip>:19000
- Access the Service Fabric dashboard at the the following URL: http://<haproxy_floating_ip>:19080
- Access any deployed application endpoint via Service Fabric reverse proxy at the following URL: http://<haproxy_floating_ip>:19081/<service_instance_name>/. More details about this here.
One of the following two Active Directory domain user credentials are required to gain access to either the GUI or the API endpoints of the Azure Service Fabric cluster. The AD users have been requested by the Service Fabric charm from Active Directory during the deployment process, one (asf-admin) having full cluster administrative privileges, the other (asf-user) being just a normal user with read-only access to the cluster.
Passwords for these domain users are randomly generated by the AD charm. You can find their passwords by running the get-ad-user-credentials Juju action:
|
1 2 3 |
UNIT="azure-service-fabric/0" ACTION_ID=$(juju run-action $UNIT get-ad-user-credentials | awk '{print $5}') juju show-action-output $ACTION_ID |
This is how the Service Fabric Explorer dashboard looks like after you login:
If at any point in time you decide to scale-up your cluster, you can add one or more nodes to it by simply typing the following command:
|
1 |
juju add-unit --num-units 2 azure-service-fabric |
Juju will spawn more instances on OpenStack, do the proper installation / configuration and join them to the existing Service Fabric cluster.
Deploy applications from Visual Studio
In order to be able to publish applications from Visual Studio directly on your Service Fabric cluster, you need to install the Service Fabric runtime, SDK and tools for Visual Studio.
Before you start publishing applications, you need to make sure that you can successfully create a connection to the Service Fabric cluster. Since the client-to-cluster communication uses WindowsCredential Active Directory authentication, you will need to make sure you have the domain credential stored before trying to publish an application from Visual Studio. We will use the Cmdkey tool to store the Service Fabric cluster administrator credential in PowerShell:
|
1 2 |
cmdkey.exe /add:<haproxy_floating_ip> ` /user:"<ad_domain>\asf-admin" /pass:"<asf_admin_password>" |
The <haproxy_floating_ip>, as well as both the <ad_domain> and the <asf_admin_password> are the ones obtained in the previous steps. To test if you can successfully connect to the cluster, you can execute the following PowerShell cmdlet:
|
1 2 |
Connect-ServiceFabricCluster -ConnectionEndpoint <haproxy_floating_ip>:19000 ` -WindowsCredential |
This will return True and give you some information about the currently established session to the cluster.
Once you have validated that you can connect to the Service Fabric cluster, you can start publishing applications. For a quick demonstration, start Visual Studio, go to File -> New -> Project and create a Service Fabric Application using the Stateful Service template:
After the application is created, right click on it from the solution explorer and click Publish:
Specify the correct connection endpoint, expand Advanced Connection Parameters and set WindowsCredential to True. You should see a green check-mark near the connection endpoint box indicating that a successful connection to the cluster was made:
Now you are ready to publish the application by hitting the Publish button. In a few seconds the application should have been successfully published, which you can see either in the Visual Studio console or by checking the Service Fabric dashboard:
Notes:
- If you get any errors when trying to publish the application from Visual Studio, make sure you target the .NET framework version you have previously installed on your cluster nodes.
- If any application deployed on the Service Fabric cluster is using TCP ports not present in the port range defined in the charm config, new Windows firewall rules need to be defined on the cluster nodes. This can easily be done via the Juju client for the entire cluster at once:
1juju run --application azure-service-fabric './hooks/open-windows-firewall-ports.ps1'
What’s next?
Stay tuned for learning how to move your Service Fabric applications between on-premise OpenStack and Azure public cloud!
