Tuesday, March 31, 2015

Top 10 Azure Glossary: Demystified

1. Affinity Group

“Affinity group” AKA Scale units helps co-locating related resources in close proximity to reduce network latency.  for e.g., when you launch a multi-tiered web application with front end tier, business logic tier and database server, you don’t want to place these resources in different parts of the datacenter instead you want to group it together for better network performance.  Azure highly recommends Affinity Group for grouping of related resources but doesn’t mandate. Azure Data centers consists of multiple Affinity groups and not all the affinity groups contains all the Services of Azure, for e.g. New High power VM Families, Internal Load  Balancers, Reserved IPs may not be available  in all the Affinity Groups.

2. Regional VNet

Regional VNet is the enhanced version of VNet. Until 2014, VNet was originally bound to Affinity Groups which is just a sub section of Azure Data Center. Affinity group has limited set of resources and it doesn’t contain all the services offered within a region. As of this writing, Azure has 17 regions spread globally and planning to power up many more datacenters. When you create a Regional Virtual Network, it can span the entire region and thus you can avail all the services available within the region and not limited to Affinity Groups.

3. Availability Set

Azure’s main promise is High Availability. To achieve HA for your applications, it is always recommended to run at least two instances of your solution to qualify for HA and 99.95 Azure SLA. Availability Set has two main concepts called Fault Domain & Upgrade Domain. As the name suggest Fault Domain is an individual or group of Container/Rack placed inside the Azure Datacenter that shares Power and Network Switches. 2 Virtual Machines placed under an Availability group, will be virtually deployed in two different Fault Domains so that problems occurred in one Fault domain will not affect another. Upgrade Domains is a categorization of resources to manage Host Operating Updates and patches. This helps us with avoiding both the VMs get updated or patched at the same time.

4. Resource Group

Resource Group helps you to group all the related services together for better resource management, tagging and billing. Not to be confused with Affinity Groups, which is keeping virtual resources close proximity. For e.g. If you manage 2 different projects 1. Internal SharePoint Portal, 2. Public facing corporate website built on PHP. Each of this solution have different set of resources and hence you may want to group them together. Key pointers about Resource Group at this moment are
  1. Default and Maximum Resource group that you can create within a subscription is 500.
  2. Resource Group should not be confused with Active Directory Group functionally, both are two different services.
  3. Linking of shared resources between groups is not fully functional yet
  4. Resource Group can span regions.

5. Endpoint

Be default VMs launched within Virtual Networks can communicate to each using their private address, but if you want to make VMs placed in different Networks irrespective of whether it is within azure/on premise/other cloud, you need public IPs otherwise called as Endpoints. When you create VMs, ports like Remote Desktop, Windows PowerShell Remoting, and Secure Shell (SSH) are automatically opened, but you can also open other ports like FTP, SMTP, DNS, HTTP, POP3, IMAP, LDAP, HTTPS, SMTPS, IMAPS, POP3S, MSSQL, and MySQL as it requires. Each endpoint in the VM has two ports i.e. Public Port & Private Port. Public port is used for incoming traffic from the internet and private port is for internal communication with other services within the virtual network.

6. Public Virtual IP Address/Dynamic IP Address

When you first create a Cloud Service in Azure you will be assigned with Virtual Public IP Address. This VIP will not be released until all the VMs placed inside the Cloud services is successfully Deleted or Stopped (De-allocated). Dynamic IP Address (DIP) are nothing Private IP address allocated by DHCP (Dynamic Host Control protocol), also note that it bounds to the VNet CIDR block defined by the user. Similar to the Public IP, DIPs are also not release until all the VMs placed inside the Cloud services is successfully Deleted or Stopped (De-allocated).

Reserved Virtual IP Address

Users can reserve IP addresses for their subscription. This helps them with predictable IP address that can associated with their Cloud Services and Virtual Machines. By default when you delete or stop (De-allocate) your instances the VIPs will be released to Azure IP address pool, but when you reserve IPs it will remain in your subscription until to remove Reserved IPs from your subscription.

7. Instance Level Public IP Address

Instance level IP address is associated directly to the Virtual Machine Instances rather than to the Cloud Services where you back all the Virtual Machines within. Currently you can only allocate one PIP to a VM instance and it is not currently supported Multi NIC VMs. Instance Level IP addresses can be used when you simply want to connect your VM with an IP instead of using Cloud Service endpoints opened individually for each ports like http://mytestvm.cloudapp.net:8080. Other benefits includes receiving traffic on any port instead of selective ports which is best suitable Passive FTP where the selection of the ports are completely dynamic in nature, similarly outbound traffic from VM can be routed via PIP. At this moment, requesting of Instance level IPs as well as allocation if IPs can only be done using Windows PowerShell and Rest APIs.

8. X-PLAT CLI

It’s a command line interface for Windows, Linux and IOS Platforms. You might be familiar with Windows PowerShell CLI, the favorite power shell scripting utility of IT pros used to automate and execute remote commands, but it’s meant only for windows. X-PLAT CLI built using JavaScript/Node.js is an alternate solution which brings the same power to non-Microsoft platforms. You can download Windows installer here and OS X installer here and find Linux installation instructions here.

9. Cloud Service

Out of all the Naming Conventions of Microsoft Azure, Cloud Service is the single most confusing and ubiquitous term. Cloud Service is a very broad term and used by everyone, everywhere basically for one reason, anything hosted out of premise is generally called as Cloud Service. Cloud Service in Azure is nothing but a DNS name e.g. http://<<contonso>>.cloudapp.net or http://<<contonso>>.azure websites.net which could be mapped with a custom domain. Creating cloud service is the first step of creating public interfaces like WebApp, Mobile services  or Azure VM.  

10. App Services

Azure App Service is the new term coined by Microsoft recently which consolidates Websites (Web Roles/Worker Roles), Web Jobs, Mobile Services, API Services together and offers it as a package. As of writing this article, it’s currently available only in the preview portal. There was lot of confusions within the Developer community as when to choose Web Roles, Website, Mobile Services etc because of close resemblance to each other. In fact you can create a mobile services using Worker Role or a Web role. Now let’s look at what these individual services can do

Web App

This is nothing but Azure Websites that helps developers to quickly build websites using variety of different programming languages and host and scale seamlessly using Azure PaaS offering.

Mobile App

Azure Mobile App service is purpose built for 3 Key Scenarios. 1. Enterprise SSO with AD, 2. Push Messaging and 3. Social Integration. Mobile service is completely platform agnostic and technology agnostic, means you can build mobile services for variety of different platforms like iOS, Android, Windows with both .Net or JavaScript as the back end.

Logic App

It’s a new breed of service targeted at developers and technical business users to orchestrate and create API workflows. APIs found everywhere, almost all the services exposes APIs. Logic Apps helps you to connect various APIs together in a secured and organized manner. Logic App provides out of box Social Media connectors for Twitter, Facebook, Yammer. Enterprise Connectors for SAP, Marketo, Salesforce and Azure Data Service connectors for  Sharepont, Mobile Services, Storage etc. If you don’t find connectors of your favorite services, you can build one by yourself using API App service.

API App

It’s an API hosting Service where you can build APIs using various programming languages including C#,Java, Python,Node.js,PHP and  host it with Azure Apps service. API App connects seamlessly with Azure Web App/Mobile and Logic App. The 2 major benefits of API app are 1. Simplification of Security using AD/SSO and OAuth and 2. Quick API deployments and automated versioning support.

Note: This post is originally posted at 8KMiles blog on March 30, 2104. 

Wednesday, March 18, 2015

Azure Virtual Network vs AWS Virtual Private Cloud

Statuary Warning: This content is totally neutral to any cloud provider. The blog content is strictly time bound and we request you to read the respective cloud providers documentation and refer the current status of the respective service updates portal of Azure & AWS.

Amazon has been a fore runner in the cloud computing arena and pioneered many industry revolutionizing services like EC2, VPC etc. AWS’s initial offering EC2-classic platform allowed customers to run ec2 instances on a flat global network shared by all the customers, also there were other attributes including shared tenancy, restrictions on Security Groups and lack of Network Access control lists concerned security minded customers. AWS then introduced EC2-VPC, an advanced platform which provisions logically isolated section of the AWS Cloud. AWS EC2-VPC supports Shared/Dedicated Tenancy, Improved Network Security Groups/Network Access Control etc., Enterprise Customers and SMB customers gained more confidence with the VPC architecture and started adopting AWS better than before.

In 2013, Azure turned its focus from being just a PaaS provider into a Full-fledged IaaS provider to avoid the competitive edge and market loss. In order to compete with the early starter AWS, Azure introduced many new services and importantly Virtual Networks, “a Logically Isolated network” the VPC version of Azure within its Datacenter. Azure’s Virtual Network resembles VPC in many aspects and in fact behaves similar in many cases but there are few differences as well.

In this blog, we’ll see those differences in detail and off course the similarities as well. It’s all about Networking, so let’s begin with

Subnet

Subnets are the building blocks of Private Networks. Subnets are a great way to divide the bigger network into many smaller networks and place the workload depends on the nature of the data that it deals with. AWS being an IaaS provider has matured tools like their management portal, Cloud Formation Templates, CLIs and programmable APIs to launch subnets. AWS also provides Wizards to automate the common VPC architectures such as
  • VPC with a Single Public Subnet
  • VPC with Public and Private Subnets
  • VPC with Public and Private Subnets and Hardware VPN Access
  • VPC with a Private Subnet Only and Hardware VPN Access
This helps users to greatly reduce the VPC setup time and simplifies the entire process. AWS makes creating complex Networks like a child play using the Wizard, anyone who wants to create and provision multi-tiered Web application or any workload in public-private subnet in minutes.
Azure Virtual Network also allows us to create subnets of any quantity using the Management portal, PowerShell, CLI. Unlike AWS, azure doesn’t currently have wizards to create the common architectures like the ones mentioned above.

 Security

Security is the primary driving force why Virtual network is preferred over public facing endpoints. AWS provides various virtual Security services to provide maximum security both at Virtual Instance level, subnet level and overall network Level.

Security Group

AWS “Security Groups” helps protecting instances by configuring inbound and outbound rules. Users can configure what ports to open to accept traffic from what source and similarly configure outbound ports from EC2 instances.

Azure’s naming convention is “Network Security Group” is currently available only for Regional Virtual Networks (Read what regional Network is) and not available for VNet that has Affinity Group Associated. You can have max 100 NSGs per subscription (hope this is the hard limit enforced, MSDN doesn’t explains it further).
AWS allows us to create 200 Security groups per VPC, for example if you have 5 VPCs you can create 200 * 5 = 1000 Security groups totally, but Security groups in both clouds cannot span regions.
Unlike AWS, Network Security Group of Azure can be associated to VM Instance, Subnets and hybrid i.e (Subnet and VM), this is a powerful multi-layer protects that a VM can get, click here to read more.  Azure currently doesn’t offer user interface to add/edit security groups, so users must use PowerShell and REST APIs to setup the same (Refer the below Powershell Workflow).

Network ACLS

Azure and AWS supports Network Access control list. ACLs allow users to selectively permit or deny traffic to your Networks.  Both the clouds states it as an enhancement or an optional security mechanism on top of security groups and other security mechanisms. ACLs in azure is currently limited to securing Endpoints (What is Endpo
As of writing this article, you can only create Network ACLs using Powershell and REST API commands.ints) and doesn’t offer the same flexibility and control as AWS provides. ACL in AWS allows us to set Access control at the subnet level, i.e. if you allow http traffic to a subnet, all the EC2 instances inside the subnet can receive HTTP traffic, however if you have configured not to allow HTTP traffic in certain EC2 those traffic will be filtered by Security Groups. Azure’s Network ACLs behaves almost similar except it works for an endpoint.
Note: Azure recommends either Network Access Control List or Security group, not both at the same time, because functionally they do the same. If you have configured Network ACL and wanted switch to Security Groups, first you must remove the Endpoint ACLs and configure Security Group.

Custom Routing Tables

Custom routing tables contains list of Routing Rules to determine how the traffic should flow inside the subnet.
In AWS, Each subnet must be associated with a route table, which controls the routing for the subnet. If you don't explicitly associate a subnet with a particular route table, the subnet uses the main route table of the VPC.
Windows Azure provides default routing across subnets within a single virtual network, but does not provide any type of network ACL capability with respect to internal IP addresses.  So in order to restrict access to machines within a single virtual network, those machines must leverage Windows Firewall with Advanced Security (Refer the diagram).
Microsoft must be cooking this feature in their kitchens. We can expect this delicious feature in Azure restaurant soon.
(Image source: Technet Blog)

Dedicated Instances

Amazon provides Dedicated EC2 instances that run in VPC on hardware that is dedicated to a single customer. Dedicated Instances are physically isolated at the host hardware level from other dedicated instances of other customer accounts. Although currently dedicated instances in VPC doesn’t work with many main stream services including EBS block storage but there are certain cases where dedicated instances are preferred by the customers.
Azure doesn’t offer dedicated instances at this moment, however customers have raised requests with Microsoft for such offering, it is expected that Microsoft will consider this request and bring in support for Dedicated Instance.

Virtual Network Interfaces

Virtual Network interface card (NIC) is a virtual appliance that can be plugged and unplugged with VMs. This provides full time connectivity with the Network and helps route certain networks to certain NICs.
AWS allows you to attach multiple Elastic Network interface cards to EC2, however AWS restricts this capability to certain EC2 Families and not all. As of writing this article, C3 /C4/CC2/CG1/CR1/HI1/HS1/I2/M2/R3 Large families are allowed to plug maximum of 8 Network interfaces and 30 private IP Addresses.
Azure also supports this feature, however just like AWS, azure also restricts to Multiple Virtual NIC for only certain large machines.  Azure lets you to create Multiple NICs on the following VM categories
  • Large (A3) and A6: 2
  • ExtraLarge (A4) and A7: 4
  • A9: 2
  • D3: 2
  • D4: 4
  • D13: 4
Azure has enabled this feature only on their IaaS offering and not in PaaS. There are some more limitations like only Public facing Virtual IP address is supported in the default NIC, adding or removing of IP is not allowed once the VM is created. Users cannot apply Network security or Forced Tunneling to the Non Default NIC. However, we can expect Microsoft’s Network team enabling and removing some of the current limitation in the upcoming months. Click here to read more.

DNS Service

DNS is a very crucial part of Networking and it’s very essential to avoid latency and unnecessary networking hopping. AWS Route53 provides a highly available and redundant DNS service that connects user requests to various services of AWS such as EC2, ELB, or S3 and it can also be used to route users to infrastructure outside of AWS.
Currently Azure doesn’t offer DNS services and requests users to add DNS redirects to CloudApp.Net url given to all the services of Azure cloud. However, there are loads of request from Azure customers to build DNS system to get out of the Redirection issue.

Connectivity

Inter connectivity lets different networks connect each other. Cloud providers provides 3 basic inter connectivity option

Direct Internet Connectivity

AWS allows users to associate Public IPs to EC2 instances there by allowing internet connectivity to those machines and similarly VMs in the private subnet gain internet access by routing through NAT instances in the public subnet.
Azure lets users to configure public endpoints aka Public IP addresses to VMs inside the subnet thereby VMS can be connected with other systems.

VPN over IPsec

VPN over IPsec is an IP based connection methodology to interconnect two different networks, irrespective of networks within cloud/ outside, cloud to on premise network etc., broadly there are two types of VPN routing protocols used 1. Static Routing protocol 2. Dynamic Routing protocol.
Azure and AWS provide support for Static and Dynamic Routing, however Azure at this moment doesn’t support Active Routing Support (BGP) but Azure has published a huge list of VPN device manufactures who support BGP routing.

Private Connectivity using Exchange Provider

Private connectivity option mainly focused towards enterprise customers who have bandwidth heavy workloads.  Private connection by ISPs can provide much better performance than Internet. Both AWS and Azure has partnered with major Telecom and ISVs to offer private connectivity between their clouds and customer’s on premise infrastructure. Azure supports most of their features through Express Route except certain features like Service Bus, CDN, RemoteApp, Push Notifications etc. (Click here to read more).  Similarly AWS supports All AWS services, including Amazon Elastic Compute Cloud (EC2), Amazon Virtual Private Cloud (VPC), Amazon Simple Storage Service (S3), and Amazon DynamoDB can be used with AWS Direct Connect. As far as the SLA is concerned, AWS doesn’t provide SLA for this service, but Azure on the other hand promises 99.9% SLA, otherwise the customer can claim service credits.

SDK & Tools

Azure & AWS provide programmable SDKs and APIs to deal with various services of networking options provided by these clouds. Developers can create a Virtual network using Azure’s PowerShell and CLI or the management dashboard, similarly AWS allows the users to configure VPC using CloudFormation templates, Rest APIs and CLIs.

Summary

The intention of this article is to highlight certain intricate differences and not an in-depth comparison guide. AWS being the pioneer in the IaaS space has lot of matured options and tools set to offer, but Azure on the other hand is currently building and maturing their IaaS offering. Azure being Conventional Software provider focused mainly on enabling their windows environment to suit and operate within IaaS offering, hence all the services newly launched and services in preview seems to be more Windows focused. Microsoft welcomes partners and vendors to build the Providers/Adaptors/Connectors/APIs for the Open Source programming languages like Python or Ruby n Rails etc. Azure from its inception focuses Enterprise customers and goes with Hybrid Story, AWS on the other end tasted their success with startups and SMB customers now trying to build Enterprise storyline to take AWS to the next level. 

Note: This post is originally posted at 8KMiles blog on March 16, 2104.