When I first start working with an API, I aim for low-hanging fruit. REST APIs, by nature, should be very generic in how they’re interacted with; however, there’s usually small nuances to take into consideration. For example, I recently found out that the VMware Cloud on AWS API uses a csp-auth-token header for authentication and authorization.
While authorization and authentication to the VCF API was straightforward (SDDC Manager username and password), I struggled the first time with POSTing a new VMware license due the API requiring a specific format for productType.
In a recent post, I wrote about interacting with VCF using the API to add a new license key as a simple way to begin familiarizing myself with the API. As a huge proponent of PowerShell, I began looking for a module to talk to the API but came up empty handed. I began working on a module with vSphere admins in mind because I know the important role PowerShell plays in day-to-day operations. During a conversation with Jase McCarty, he told me about the PowerVCF project which does exactly that! The module was initially developed and is maintained by Brian O’Connell and has 50 cmdlets which covers ~70% of the API calls in VCF 3.9.0:
Get/New/Set/Remove Workload Domain
Get/New/Set/Remove vSphere Cluster
Get/New/Remove Network Pool
Get/New/Remove Network IP Pool
Get/New/Remove License Keys
Get/Set participation in CEIP
Get/Start Backup Configuration
Get/Request Log Bundle
Get/Set Microsoft Certificate Authority
Get/Request Certificate CSRs
Get/Set Depot Credentials
Get PSCs & vCenter servers managed by SDDC Manager
Get NSX-V Managers
Get NSX-V/T Clusters
Get vRealize Log Insight info
Get vRealize Lifecycle Manager & Environment info
Get vROPS info
PowerVCF is also mostly compatible with VxRAIL too with the exception of commissioning and decommissioning hosts, working with network pools, and creating and removing workload domains.
The biggest functionality missing right now are creating and deleting PKS/Horizon workload domains and creating/joining/tearing down federations. I’m currently working on the PKS workload domain functions and plan to submit a PR soon!
If you’ve recently deployed VCF and looking to orchestrate functionality, I highly recommend checking this module out! If you enjoy creating PowerShell cmdlets and looking to contribute to a project, you’ll find quite a few opportunities to help us work towards feature parity!
Today I took my first VMware certification exam in 7 years and happy to report that I successfully passed the Professional vSphere 6.7 Delta Exam 2019(2V0-21.19D) to become a VMware Certified Professional again!
The VMware hypervisor hasn’t had significant changes in 7 years since I took the VCP 5 exam and I never stopped working with vSphere so it didn’t require a tremendous amount of time to prepare. The topics I spent the most time on were reviewing vSphere HA/DRS updates, vCenter HA and PSCs, content libraries, SSO domains, and security enhancements. It’s especially helpful that VMware created the Delta Exam as it allows professionals to test and re-certify only on the information that is new or has changed since the previous exam and not require candidates to sit through the complete VCP exam.
Now that I’ve been at VMware for 6 months, I felt it was timely to get my VCP-DCV updated. It was also helpful that all VMware employees receive three free exam vouchers per year! In the bigger picture, I want to become more versed with the major components of VMware Cloud Foundation that I’m not as familiar with: NSX, and vRealize. In 2020 I will focus on completing the VCP-NV and VCP-CM certifications to get deeper knowledge of these technologies and the value they bring to organizations.
About the Exam
The exam’s foundation is focused on a candidate who already has a VCP-DCV 6.5 and has 6-12 months of experience installing, configuring, and managing vSphere. The exam consists of 40 single and multiple choice questions covering a wide variety of topics such as:
Architecture and Technologies
Products and Solutions
Planning and Designing
Installing, Configuring, and Setup
Performance-tuning, Optimization, and Upgrades
Troubleshooting and Repairing
Administrative and Operational Tasks
The full exam prep guide can be found on VMware’s Education site here.
Since it’s been such a long time since I held an active VMware certification, I want to help others who may be on a similar journey. In upcoming blog posts, I will comprehensively cover the biggest and most complex additions/changes to vSphere since vSphere 5.5. If you have topics you’d like to see covered, drop a comment!
A common question I
receive from customers is why they don’t see a VMware
Cloud Foundation license in the MyVMware portal. What appears instead is
licenses for each individual product that make up the VCF
edition you purchased. Which is typically:
I’ve been at VMware for 12 weeks now and continuing to work towards being a vSAN expert. One of my many challenges facing that goal is not only learning the current state of vSAN’s features and capabilities (the latest being 6.7U3) but also learning how vSAN operated in previous versions to articulate to my customers why feature X in this release is relevant to them.
VMware has released updates to vSAN 75 times since the initial release in 2014 and 12 updates in 2019 alone. So where is the best place to start for having a foundational understanding of modern vSAN functionality? VMware called version 6.6 their “Biggest Release Ever” back in 2017 and admittedly, while at Pure Storage, that’s the version that I started to recognize that vSAN had matured a lot so this version would be the basis for level setting my knowledge on what most customers’ experience with vSAN will be. However, of the handful of customers that I support in my Global Accounts role at VMware, most are running at least vSphere 6.5U3 so vSAN 6.6.1 will be the basis for my learning.
One of the confusions I’m adjusting to diving into vSAN is that vSphere and vSAN versions don’t match. One would reasonably expect a product built into another one to have matching versions but they rarely do. Interestingly, they have matched in the past! One of the most helpful documents I’ve used at VMware while ramping up is KB 2150753, Build numbers and versions of VMware vSAN. I’ve referenced this KB article many times to correlate vSphere and vSAN versions. At the end of the day, matching version numbers is a nice to have “feature” but not matching is the reality of two separate business units working on their own products with specific goals and milestones to reach different major and minor releases.
I’m going to highlight major performance and usability enhancements to vSAN in the past 4 release:
A typical minor dot-release for vSAN: a few new enhancements but nothing major. Although there were 12 updates to 6.6.1 since it’s initial release (Express Patches, Patches, and Updates), I couldn’t find any release notes. Fundamentally, these were the most important features in this release:
VUM Integration: VUM integration automates the process of ensuring that hardware installed in the cluster is on the VMware Compatibility Guide (or HCL). It also provided firmware updates for select hardware vendors such as Dell, Lenovo, Supermicro, and Fujitsu. A known issue in this release is that Custom ISOs are not supported in vSAN build recommendations and hosts built on custom ISOs will display as Non-Compliant.
Storage Device Serviceability (Blink Disk Lights): When a device fails, it’s extremely important to be able to find it in the server! This feature gives you the ability to select the disk in the UI and make the LED light blink. Great feature but in this release, it’s limited to HPE DL/ML series servers with Gen 9 controllers.
What Was New in vSAN 6.7 GA
A big usability enhancement in this release was the HTML 5 Client becoming the standard interface for vSphere! Other notable performance enhancements included:
This feature includes three main components: congestion control mechanisms, a dispatch/fairness scheduler, and a bandwidth regulator. In essence, under contention vSAN has the ability to throttle I/O caused by resync operations in favor of prioritizing VM I/O. Before this feature was added, VM I/O was in an every-man-for-himself battle that could cause performance. The adaptive nature of this feature means it’s always on and allows it to be an invisible vSAN operations that doesn’t need any user-defined capabilities. The Adaptive Resync Deep Dive on StorageHub goes into much greater detail.
New Health Checks in vSAN Health
vSAN Health is a cloud-connected, built in framework for providing proactive health checks for vSAN clusters. Participation in VMware’s Customer Experience Improvement Program (CEIP) is mandatory to realize this benefit. This capability was initially released in vSAN 6.6 and additional checks were added in 6.7 included:
Host maintenance mode verification
Host consistency settings for advanced settings
Improved vSAN and vMotion network connectivity checks
Improved vSAN Health Service installation check
Physical Disk Health checks combine multiple checks into a single health check
Improved HCL check
Firmware checks are now independent of driver checks
This release had 3 new features to improve performance and relaiability when using stretched clusters. Namely:
Intelligent site continuity: If there’s a partition in the cluster (link goes down, etc), vSAN will first validate which site provides maximum data availability before establishing a quorum with the witness. For example, if Site A (preferred) lost a node or a device during the partition and objects are in a degraded state but Site B (secondary) is healthy, vSAN will consider Site B active until Site A is healthy again.
Witness traffic separation: A separate vmkernel NIC can be dedicated for vSAN witness traffic when using stretched clusters. Previously it was required for the data network to communicate with the vSAN witness host and that VLAN to be stretched across the WAN as well. When deploying stretched clusters, separating witness traffic is recommended.
Efficient inter-site resync: A proxy host is established for components that need to be resynced across sites following a failure instead of copying the objects across the WAN to meet the storage policy requirements
More details on vSAN 6.7 GA updates can be found in the release notes.
What Was New in vSAN 6.7 Update 1
vSAN 6.7U1 seems it was the biggest update to vSAN since 6.6 and there’s a lot of great performance and usability enhancements in this release!
The following tasks are performed to speed up and ease the deployment process of vSphere clusters:
Setup HA, DRS, and vSAN
Select vSAN deployment type
Network configuration including vSphere Distributed Switching
Disk Group configuration
Enable Deduplication & Compression / Encryption
Remember how in 6.6.1 there was VUM integration? Well kinda…what was missing was the ability to utilize VUM to update vSAN clusters when using OEM-specific ISOs. That’s fixed in this release but still no ability to update vSAN through VUM with custom ISOs.
When entering a host into maintenance mode whether to perform updates or simply decommission it, vSAN will now perform a full simulation of the activity (assess the capacity/availability impact of host going into maintenance mode and ability for cluster to redistribute object components) and report back success or failure.
Additionally, the “object repair delay timer” setting (around since vSAN 5.5) is now in the GUI. This allows an administration to modify the amount of time to wait for vSAN to rebuild data when components are out of compliance with the storage policy due to a disk or node failure.
vSAN now has awareness of TRIM/UNMAP commands sent from the Guest OS and can reclaim previously allocated blocks as free space.
Mixed MTU Support for 2 Node and Stretched Clusters
Remember that Witness Traffic Separation (WTS) feature in 6.7 GA? It was nice that a different vmkernel port could be used to separate vSAN data traffic from witness traffic; however, it was still required that the MTU matched on all vmkernel interfaces. That changed in 6.7U1 and now it’s possible to have Jumbo Frames on the vSAN data vmkernel interfaces while using a standard MTU setting on the vmkernel interface for witness traffic!
Enhanced Health Checks & Support
Network performance health check ensures that sufficient performance can be achieved
Display and classify multiple, VCG-approved storage controller firmware versions such as not latest, latest, and not on HCL
Expanded diagnostics in vSAN Support Insight which give GSS tools to capture network diagnostic data and further reduce the need for collecting and transmitting logs
More details on vSAN 6.7 Update 1 features can be found in the release notes.
What’s New in vSAN 6.7 Update 3
Finally! We’ve made it to the current version of vSAN and you may have noticed that we skipped over Update 2. That’s because vSphere 6.7 Update 2 didn’t include any new features or enhancements to vSAN so it was skipped. I guess VMware tries to keep versions aligned after all?
Update 3 is another huge leap forward for vSAN with the biggest being the introduction of Cloud Native Storage. This isn’t specifically tied to just vSAN. Instead, it enables vSphere to provide persistent storage to Kubernetes and gives the vSphere administrator the ability to select the required storage (vSAN, VMFS, NFS) for the pod. There’s an excellent doc on Getting Started with VMware Cloud Native Storagehere which walks you through setting up a k8s cluster, deploying applications, and managing container volumes.
VUM integration gets another update: instead of showing only the latest version of vSAN, you can create new baselines to stay that allow you to stay at the current version and only show new patches and updates
New Monitoring and Dashboards
Capacity Monitoring Dashboard has been redesigned to provide better visibility into overall as well as granular utilization. New insights per site, per fault domain, and host/disk level
Resync: improved accuracy when displaying time remaining to complete a resync
Data migration pre-check: new dashboard that provides detailed information when performing data migration activities for maintenance mode tasks. Provides insight into object compliance, cluster capacity, and even predicts the health of the cluster before placing a host into maintenance mode
In the past, when vSAN was resyncing components, it would use a single thread to copy the data. This isn’t really a problem if the components are small as they’re likely to transfer quickly; however, what if we have many max-size components (255GB) due to large VMDKs? For example, a 5TB VMDK will span over 20x 255GB components. In vSAN 6.7U3, it will now leverage numerous parallel streams per component to make resyncs complete faster. Bandwidth for this process is managed by Adaptive Resync that was introduced in 6.7 GA.
Introducing Automatic Rebalance
In previous versions of vSAN, administrators could manually initiate a proactive rebalance after being alerted by a vSAN health check that disk(s) were imbalanced. Now automatic rebalancing can be configured to enable vSAN to handle these operations without user intervention. Information on how to enable automatic rebalancing can be found here. Be sure to adjust the vSAN health check to prevent unnecessary alerts!
New Tool: vsantop
vSphere administrators have been using esxtop for years and now there’s a similar tool, vsantop, to measure CPU usage for storage-related tasks to help with troubleshooting and support cases. This can be especially useful to provide quantifiable measurements to assist administrators understanding the impact of using data services like dedupe & compression or data at rest encryption.
There is still significant enhancements that improved I/O handling, resync and rebalancing performance, and degraded device handling since vSAN 6.6.1 that weren’t mentioned here. VMware has made significant investments in vSAN since it’s release in 2014 and serves as a solid foundation for on-premises and hybrid cloud storage.
This exercise was very productive to help me understand the progress that vSAN has seen over the last 2 years and has better prepared me to discuss upgrade paths and new features with customers.
VMware Cloud Foundation 3.8 was released in July 2019 and the biggest news in this release is the addition of public RESTful APIs for common tasks that are performed for workload domains and other day 2 operations. Managing Cloud Foundation in the SDDC manager is incredibly intuitive but customers have significant investment in existing IT and business systems such as vRA or ServiceNow.
In large scale cloud foundation deployments like I work with in Global Accounts, this will be a heavily used feature because customers now have the ability to utilize existing provisioning workflows in vRA or create new workflows that allow ops teams to orchestrate even higher levels of automation. Some common operational tasks that are available in version 1 of the API are:
Commission and decommission hosts
Create and delete workload domains
Manage network pools
Cloud Foundation 3.8 also adds capability for the SDDC manager to patch and upgrade all vRealize Suite components and NSX-T. In previous versions, SDDC manager could deploy vRealize Suite but initial config, patching, and upgrades were handled manually through each individual component. The Cloud Foundation engineering teams has been rapidly deploying enhancements and this version comes just 6 weeks since the last major release.
For further details such as release notes and planning and upgrade guides for Cloud Foundation 3.8, visit VMware Docs.
Let’s be honest — if you’re a VMUG member, you get quite a few emails from VMUG and probably delete them without looking or quickly scan it and then delete it. I tend to do the latter but the one I received this morning caught my attention and quickly turned to excitement and I wanted to do my part to promote what I’m expecting to be a very beneficial event.
Lately I’ve spent a lot of after hours time working on my own professional development and specifically focusing on leadership as I feel that my future roles in technology will require that skill. But it’s also an important skill in my role as father raising 3 children.
The upcoming VMUG virtual event’s keynote speaker will be VMware CEO, Pat Gelsinger, where he will share his “Five L’s of Leadership.” The event will also include 5 members of the VMUG community that will share their experience ranging from broad topics such as resume writing, networking, and public speaking as well as deeper topics to help you identify your brand and use it for your future success. I’m looking forward to hearing each of the following speakers:
A Public Speakers Guide to Public Speaking, Chris McCain, Director of Technical Certifications @ VMware
Soft Skills, Resume Building and Networking are Some of the Toughest Areas to Master, Paul Nadeau, Sr. SD-WAN Systems Engineer @ VMware
Tips and Habits to Advance Your IT Career, Ariel Sanchez, Sr. Technical Account Manager @ VMware
Growing From VI Admin to SRE, Michael Roy, Product Line Marketing Manager @ VMware
Achieving Happiness: Building Your Brand and Your Career, Amanda Blevins, Sr. Director & Chief Technologist @ VMware
Over my 15 year career in IT, all of these skills have been extremely important to plot a course, go on a journey, and execute on those goals. The two latest journeys I’m taking are public speaking and building my brand. I’ve been fortunate to have found the VMware community through social media nearly 10 years ago and found industry experts to follow and learn from but I’m making an concerted effort now to raise my voice and share my ideas.
I hope you’ll join me along the way. To join the VMUG virtual event on September 19 from 9 AM – 3 PM, register here: https://vmugvirtualseptevent.vfairs.com. Let your voice be heard too! Share what you learned at the event on social media and your plan to sharpen your skills.
After reflecting on my personal goals and the recent announcement that VMUG was joining the Dell Technologies User Community, I’ve decided to step down from the leadership role of the New Orleans VMUG effective immediately and focus my attention on building the Docker community in Louisiana. This hasn’t been a rash decision because of the “acquisition” of VMUG by DTUC but rather an affirming indicator that now is the right time to move on.
I’ve been part of VMUG leadership since 2010 when I started the Baton Rouge VMUG (and sequentially the New Orleans VMUG in 2011). My motivation to start the groups stemmed from my lack of knowledge about virtualization, the desire to learn what others are doing, and what new technologies were coming to the market. I didn’t want to just be a consumer though, I wanted to create an environment where I could learn and then help others who had the questions of their own and start a feedback loop. After 6 years, I can look back and say that I was successful in achieving those goals and now I’m looking for the next challenge.
Why Docker and why now?
I’ve been following Docker since 2014 and have watched the project mature and continue to gain in popularity. In 3 short years it has become the de facto standard for building cloud native applications and a robust ecosystem is forming around it much like with VMware a decade ago. The technology is still in it’s infancy and the population of IT professionals don’t know about it but the number of early adopters continues to rise and drive further development and innovation at a rapid pace. Now is the perfect time to get in at the ground floor and begin building a community.
As an open source project, community is at the heart of what has fueled Docker’s growth. Forty one percent of Docker’s contributors are individuals! That speaks volumes for the commitment that thousands of people around the world have made to improving a project. At it’s core, Docker, Inc. embraces community and local groups have exploded in popularity. As of May 2017, there are 288 groups around the world. I encourage you to take a look at the list of Docker meetup groups and get involved. There’s even an online meetup if you can’t attend one locally.
This is also an exciting opportunity for me to grow as an IT professional and network with a different group of professionals. I’ve been an amateur developer since 8th grade but knew it wasn’t a career path for me but as infrastructure has become more “cloud”-like, my interests have shifted to trying to learn more about applications and how they’re deployed. What platforms are used? What barriers exist to reliably getting stable code into production? What day-to-day challenges do developers face? I look forward to the opportunity to personally grow and being a part of the Docker community will allow that as it is currently geared more towards developers. However, the technology is becoming mainstream and the necessity to enable ops teams to successfully manage containers will be more important. It will undoubtedly lead to an even larger ecosystem and participation by former virtualization engineers as their organizations restructure their environments.
What’s your take on VMUG becoming part of the Dell Technology User Community (DTUC)?
First and foremost, my decision has nothing to do with disliking Dell EMC nor do I have a lot of commentary on what this means for the future of VMUG. My dissatisfaction lies with how the VMUG CEO, President, and Board handled this very important decision that aligns a previously independent community with a corporate run community. I strongly disagree with this path for VMUG and I don’t feel that Dell EMC brings value to a community that has been successfully grown and managed through grassroots efforts. VMUG HQ has greatly veered away from their original objective of supporting leaders to build a successful community for VMware customers. I know my fellow leaders will defend their ability to bring in sponsors and topics that are in the best interest of their local community regardless of the alignment with Dell EMC. I’m uncertain what the future holds for VMUG but I know that communities will stick together no matter what obstacles are presented.
Having spent the last 3.5 years as a VMUG leader of two different VMUGs and spent time talking to over a dozen other leaders, one issue persists in the VMUG community: lack of customer participation. VMUG recognized this and implemented the Feed4ward program to, “encourage every interested member to share their knowledge at a VMUG local group meeting or User Conference”. Knowledge sharing is what everyone’s there for but most of the time people are nervous about public speaking, don’t think they know enough to discuss topics with others, or they think what they do isn’t that different or interesting. That can all changes now!
With the release of vSphere 6 on March 12, everything is new to everyone. Not many people have downloaded it in their test/dev/lab environment and (hopefully) no one has deployed it in production yet! There are 11 vSphere ecosystem products that got updated and probably thousands of new features or enhancements to discuss. If you think just an “upgrading to ESXi 6” presentation will be boring, look at upgrading or starting to use one of the other supporting vSphere products such as vRealize Automation or Operations Manager. Maybe you’re a SMB and using or looking to use vSphere Data Protection or vSphere Replication. What was the upgrade or setup process like? How do you manage it? Did you ever have to recover from a backup or replica? Any gotchas? There’s plenty of opportunity now to get started giving back to your local VMUG community. If you want mentoring, look into the VMUG Feed4ward program!
Take this time to get out in front and start getting familiar with the new features and the associated documentation. Many organizations will look to upgrade once update 1 rolls around (I was in this crowd) which will probably be released in 6 months. Take the lead, become the expert, and be a staple in your local community.
The local VMUG leaders will probably already have a “What’s new in vSphere 6” slot carved out at the next meeting but if there’s a product feature or enhancement you like, love, or have always wanted to see, speak with them about adding a deep dive into that topic. It’s highly unlikely they’ll say no!