This is part 3 of the following blog series:

• My first FlexPod! (Part 1 – Introduction)
• My first FlexPod! (Part 2 – Hardware Overview)
• My first FlexPod! (Part 3 – Hardware Configuration)
• My first FlexPod! (Part 4 – Quality of Services)
• My first FlexPod! (Part 5 – Hyper-V Cluster)
• My first FlexPod! (Part 6 – System Center)

Hardware Configuration:

If you have read part 1 and part 2 you should now know a bit more about Cisco UCS and our hardware combination. In this part I will share you my hands-on experience with Cisco UCS. Note I am not going to explain how I configured the Nexus switches, Catalyst switches, ASA firewalls and NetApp storage devices. With this blog I want to focus on Cisco UCS in general.

Before we received the hardware I did a lot of research, read dozens of white papers and deployment guides. I also attended a two-day bootcamp at Cisco and followed an online training. There is also a Cisco UCS Platform Emulator available. The UCS Platform Emulator is a really good tool to help you get started. It allows you to build a virtual UCS environment and configure everything as you normally would within UCS Manager. Of course you can’t install an Operating System on a emulated server and use KVM. But other then that almost everything.

This already showed me that UCS has so many benifits and provides a solution for many customers. If you are new to UCS; don’t get yourself overwelmed by all these features and settings. Because once you get to know UCS you will love it!

Initial Setup:

After racking and connecting the FI’s (Fabric Interconnects) it was time to run the initual setup. Both FI’s operate in active/active mode, acting as a cluster. One FI is the primary, and the other FI is the subordinate. During failover or planned maintenance they will switch as primary. Each FI has its own Management IP Address, and there is one Cluster IP Address. The cluster IP is always assigned to the primary FI.

• Primary IP + Cluster IP
• Subordinate IP

The initual setup was very easy. With a console cable on one of the FI’s you enter a hostname, the primary IP and cluster IP (used for out-of-band management). You also enter things like a username and password. You then configure the subordinate IP on the other FI and it will automatically join the cluster because they are interconnected with a 2x 1GbE management plane. Within 3-5 minutes you have configured a UCS domain in a cluster.

UCS Manager:

Once the UCS domain is operational you can open a browser and connect to one of the management IP Adresseses you configured during the initial setup. You then have the option to launch UCS Manager or KVM Manager:

NOTE: You can also open a KVM console from UCS Manager. But KVM Manager acts as a shortcut which has proven to be usefull, because you don’t have to navigate through the UCS Manager GUI.

Of course for both you have to login with the username and password, which you have configured during initial setup. Once you logged in you enter the UCS Manager GUI:

As you can imagine, it is almost impossible to blog about the entire UCS Manager. So I will only write about a few topics. What you notice right-away is that UCS Manager makes a clear difference between ‘Equipment‘, ‘Servers‘, ‘LAN‘, ‘SAN‘ and ‘VMs‘, as you can see on the tabs in the upper-left corner. And you will also notice that UCS is policy driven for the most part.

There are different orders to configure UCS. If you know your way around, you probably configure it in the following order:

1. Configure IP Pools and Policies.
2. Configure VLANs and other network settings/resources.
3. Configure Port-Channels (e.g. with uplink Nexus switches).
4. Configure Unified Ports as Server Ports (e.g. for Blade Chassis or Rack Servers).
5. Create Service Profile Templates.
6. Create Service Profiles from Service Profile Templates.
7. Associate Service Profiles to servers (equipment).
8. …

With this blog I won’t go through that exact order, but just to give you an idea.

Configure Unified Ports:

When you open UCS Manager for the first time; although you might have already connected Blade Chassis or Rack Servers, you won’t see the hardware as equipment yet. You first have to configure the Unified Ports as Server Ports on the FI’s as the following screen-capture illustrates:

Once you configured one or more Server Ports and connect a Blade Chassis (or Rack Server); the entire Blade Chassis will be automatically detected by UCS Manager and shown under equipment. UCS Manager provides you with a lot of detailed information about every single piece of hardware. Such as the Blade Chassis, the I/O Modules, the Blade Servers and all their internal components. The information is shown with photos, graphical views, table views and much more. It is so much detailed, so far I haven’t seen it with iDRAC (DELL) or iLO (HP). Don’t get me wrong, nowadays I still work with iLO and iDRAC which both work very good. But UCS is just a different level.

But even though the equipment is detected, it is still unusable. You cannot even boot a server yet. You first have to connect some uplink network devices (e.g. Nexus switches) and assign a ‘Service Profile‘ to a server with all your ingredients.

Connect Resources:

One thing you need to do is creating VLANs, create Port-Channels to uplink switches and assign VLAN’s to those Port-Channels. It is very easy to configure on the UCS side. But of course you also have to configure vPCs (virtual Port-Channels) with a CLI on the Nexus switches. Those VLANs are then available as LAN resources within UCS that you can assign (pin) to vNICs or vHBAs.

Configure Pools and Policies:

To be able to create a Service Profile (or Service Profile Template) your first have to configure Pools and Policies.

• Pools such as ‘Server Pools‘, ‘UUID Pools‘, ‘MAC Pools‘ and ‘IP Pools‘.
• Policies such as ‘BIOS Policies‘, ‘Boot Policies‘, ‘Adapter Policies‘, ‘Maintenance Policies‘, ‘Local Disk Config Policies‘, ‘QoS Policies‘, ‘LAN Connectivity Policies‘, ‘VMQ Connection Policies‘, ‘Threshold Policies‘ and may more.

These are just a few examples. Some are required, some are optional.

Service Profiles and Service Profile Templates:

Once you have configured all required pools and polices you are ready to configure a Service Profile or Service Profile Template.

Service Profile:

During the Service Profile wizard you will use the pools and policies you created earlier. Once you configured a Service Profile you will understand why I like UCS so much. A Service Profile is full of options and features. For some parts, like networking, it is if you are creating a VM exactly the way you want. And the way you configure it makes sense. The following screen-captures give you some idea about what you can configure:

Of course these are not all settings, this is just the wizard. After the Service Profile is created you can navigate to the Service Profile and configure whatever you want.

vNICs/vHBAs:

One thing I really like about UCS is Converged Networking. With a Cisco VIC, which is a CNA (Converged Network Adapter), you can create vNIC’s and vHBA’s. You can add a vNIC with ease for every requirement. Let’s say you need a vNIC that needs to function as an iSCSI Adapters, or a vNIC for a Hyper-V vSwitch. With the right policies and settings UCS will configure the vNIC and switch port (on the FI’s) with optimal settings. So if you need eight different vNIC’s, why not create them all. You don’t have to share a vNIC for different purposes. And you don’t have to worry about NIC Teaming either. You can mix vHBA’s and vNIC’s together. And you can even apply different MTU sizes. As a bonus you can configure QoS on them, which of course is needed when you deal with Converged Networking.

The following screen-captures illustrates a few of the options you can configure on a vNIC:

Service Profile Templates:

The nice thing about a Service Profile Templates is that you can clone them. For example, you can create a Service Profile Template for a Hyper-V Server with all your settings, policies, pools, vNIC’s and vHBA’s. You then clone the template to one or more Service Profiles. Of course each Service Profile gets its own UUID, MAC Addresses and IP Addres(ses) from the pools you created earlier, which in a certain sense makes them unique. Once they are cloned, you can associate each Service Profile to equipment (servers). This will make sure the configuration on your Hyper-V Servers are consistent.

Imagine you have created a Service Profile Template, which is configured as a your standard for a Hyper-V Server. You then connect a Blade Chassis which is automatically detected. The Blade Servers show up as equipment. You can then create multiple Service Profiles from the Service Profile Template, and associate them to each server. When you boot the Blade Servers they start with exactly the same hardware configuration, BIOS settings, including the vNIC’s. Of course each of them with a unique UUID and MAC Addresses.

UCS Manager can even go a step further. You can create Server Pools. You can fill those pools manually, but you can also create policies such as a ‘Server Pool Policy’ and ‘Server Pool Policy Qualification’. This allows you to automatically add a server (equipment) based on certain criteria (e.g. server model, local disks, processor type, amount of cores and/or memory) to a Server Pool. New servers that are automatically assigned to a pool are then automatically associated with a Service Profile. This allows you to deploy servers rapidly. But you probably only going to use this when you need automation on a large scale.

Firmware Management:

UCS Manager has some nice features for updating firmware. I did a lot testing and I am impressed. Actually this is the best system I have seen so far. Because it works so good, it keeps the firmware consistent across the entire system. First of all, from time to time Cisco releases three different firmware/software packages:

• UCS Infrastructure Software Bundle
• UCS B-Series Software Bundle
• UCS C-Series Software Bundle

You can import those packages with UCS Manager. The UCS Infrastructure Software Bundle includes the UCS Manager software and all firmware for the main UCS components, such as the Fabric Interconnects and Fabric Extenders (I/O Modules). As the name suggests, the UCS B-Series Software Bundle and UCS C-Series Software Bundle includes all firmware for those components, such as the Blade Chassis, I/O Models, Fans, PSU, all server models and their internal components.

Update Infrastructure

Of course it is important that you read the release notes before you update software and firmware. I am not going to explain everything in detail. But it comes down to the following. You can update the infrastructure and your servers. When you update the infrastructure each FI and Fabric Extender (I/O Module) has to be rebooted one by one. Once you initiate the update UCS does this automatically for you. You do have to acknowledge before each reboot takes place. This allow you to check the status of the system before you continue.

Host Firmware Package Policy

Updating the firmware of Blade Servers and Rack Servers is absolutely easy! Each Service Profile is configured with a Host Firmware Package Policy. UCS Manager comes with a default policy, but you can also create your own. This policy allows you to configure which firmware packages should be applied to your server(s).

You can change the package to a new or previous version. When you reboot a server, the firmware is updated fully automatically. You don’t have to insert a media, and you don’t have to press a key. And with firmware this means every single component. Because you can select a previous version this means you can also do a roll-back. Because you use a policy this makes sure the firmware levels of your servers using that policy are kept consistent. And because it is done automatically it is also fast with minimum downtime.

A Host Firmware Package Policy can even go a step further. Let’s say you want to downgrade the firmware of a certain component (e.g. RAID Controller). With the same policy you have granular control of the firmware version for each component. This allows you to select another (previous) version. It is then applied to each server using that policy.

Firmware Automatic Synchronization Server Policy

Another great feature is Firmware Synchronization. Once configured, with this feature you can automatically update the entire firmware when a new Blade Chassis is added to UCS. So you rack a Blade Chassis, insert the Blade Servers and you power it on. Once you configure the FI’s with Server Ports, the Blade Chassis is automatically detected and the entire Blade Chassis is update automatically, including the Blade Servers.

Overall Experience:

My overall experience with the configuration of Cisco UCS is very good. The initial setup was easy, and you have the Fabric Interconnects running within minutes. Once you have created your first Service Profiles and that server boots up like you configured it, you get a smile on your face. May your wish come true. The way you can deploy a Hyper-V or VMware Server is amazing. You don’t have to worry about a minimum set of NICs. You just create as many vNICs as you want. The vNICs are configured with the optimal settings, QoS and it offers support for all kind of purposes. If you add uplink switches or another storage device. You can easely connect them and you don’t need a network administrator to configure all your switch ports for your vNICs.

Firmware Management couldn’t be easier. Our Hyper-V Servers are all consistent. They are configured with the default Host Firmware Package Policy. Once a new packages is released updating the firmware is easy. Of course you still have to be cautious when you update the infrastructure like the FI’s. Always read the release notes, make backups and make the right preperations.

I also like the fact that you only use a few 10GbE twinax cables per Blade Chassis, which keeps it simple. Adding twinax cables to a Blade Chassis to provide more bandwidth is easy. It does not require additional configuration other than configuring Server Ports. The fact that it is a Unified Fabric with FC, FCoE and Ethernet mixed together definitely offers scalability in terms of scale-up and scale-out. If something changes or is added to your environment, you can make it available to your UCS servers without too much effort. While at the same time giving the performance and flexibility you need.

I do have to admit; If you start with UCS for the first time it can be a bit overwhelming, because it has so many features and settings. It is not that surprisingly, because it requires you to have a certain knowledge about compute, network and storage. But once you get to know it, it is not that difficult. It is just another way of deploying compute you have to get used to. Before you begin with UCS I always recommend to use the Cisco UCS Platform Emulator, read white papers and deployment guides. And of course a workshop or online training is advised. Keep in mind, that Cisco has many reference designs and deployment guides available, you don’t have to invent the wheel.

Ok, that’s it for now. Click on the link below to continue with the next part.

NEXT >>> My first FlexPod! (Part 4 – Quality of Services)