The Gurleyman

Published November 11, 2015 by Chris

Replicated Recovery With Rubrik

Rubrik makes instant recovery easy everywhere. As I wrote four months ago, it only takes a few clicks to bring a previous version of any protected VM into production. In 2.0, the great folks at Rubrik enhanced this capability with replication.

Replication is a word that means many things to many people and could quickly get abused in comparisons. In our previous data protection solution, replication of backups was limited to scheduled jobs and practically meant our off-site backups were anywhere from 3 hours (best case) to 48 hours (worst case) old, with no guarantees.

Rubrik takes a refreshingly different tactic. In its policy-based world, backups are driven by SLAs (gold, silver, bronze, etc), which are defined by frequency and retention of snapshots. Replication is married to these policies and is triggered upon the completion of VM backups.

For example, this morning one of our mission-critical SQL servers in our Gold Repl SLA domain started a backup job at 6:35am and completed that job one minute later at 6:36am. Gold Repl takes snapshots every 4 hours, keeps those hourlies for 3 days, and then keeps dailies for a month. As the “Repl” denotes, it also replicates and retains 3 days of those backups at another site. Oh, and as the cherry on top, it additionally archives the oldest backups to Amazon S3. Pretty comprehensive, eh?

Storage Technology Virtualization

disaster recovery replication Rubrik

Published October 29, 2015 by Chris

How To Fix Infrastructure Navigator Error: “An unknown VM access error has occurred.”

While deploying vRealize (formerly, vCenter) Infrastructure Navigator (VIN) yesterday, I ran into an access error that wasn’t at all pleasant.

Access failed. An unknown VM access error has occurred.

I had deployed the virtual appliance per the 5.8.4 documentation on pubs.vmware.com, and had specifically created a Virtual Machine Access role as defined. I set it in Global Permissions for all children and verified that it propagated to the VMs that reported this error (all of them).

Searching VMware KBs and Google for a resolution proved mostly fruitless. I finally came across Scott Norris’s post about resetting the VIN database, which gave me the nugget to resolve my issue. As I look at it now, I’m not quite sure why his pointed me to the answer, but it was the only one out there with exactly the same error–all others were about “discovery errors”. If what I provide below doesn’t solve your issue, check out Scott’s “reboot” option for a more comprehensive refresh.

So what was the problem/answer? DNS.

When I deployed the OVA and reached the field for comma-separated DNS servers, I listed mine–all four of them–like this: 192.168.1.11,192.168.1.12,10.1.1.11,10.1.1.12. I’m quickly learning that four is not a friendly quantity in OVAs or Linux things in general. In the vein of The Matrix, those who write into /etc/hosts seem to like Trinity, or three, as a max. Send it four resulted in none committing.

Fixing it came through these steps:

1. Open the console to the VIN virtual appliance

2. Hit Enter on “Login” to reach the CLI/login prompt

3. Login as “root”

4. Run “yast”

5. Arrow down to “Network Devices”, tab over to “Network Settings” (on the right), and hit Enter

Technology Virtualization

error VMware vRealize

Published October 13, 2015 by Chris

How To Upgrade to ESXi 6.0 And Overcome “Incompatible” Status

With the release of ESXi 6.0 Update 1a, which fixed the network connectivity issue that plagued all ESXi 6.0 releases until October 6, I have begun my own journey from 5.5 to 6.0. I’m taking a new approach for me, though, as I use Update Manager to perform an upgrade rather than the fresh installs I have always preferred.

Why? Because I learned at VMworld 2015 from the authorities (designers) that upgrading is actually VMware’s recommended path. You can read more from my notes on session INF5123.

What follows below assumes that you have already rebuilt or upgraded to vCenter 6.0 Update 1. In Update 1, the Web Client now supports Update Manager so that everything can be performed there. No more thick client! Now if we can just get rid of Flash…

Step 1: Import ESXi Image

From the home landing page of the vSphere Web Client, navigate here:

Update Manager
- Select an Update Manager server
  - Go to Manage
    - Then ESXi Images
      - Import ESXi Image…
        
        Browse to the ISO

esxi6_import

Technology Virtualization

Dell ESXi QLogic upgrade VMware VMworld

Published October 12, 2015 by Chris

What Does The Dell+EMC Merger Mean To The Industry?

This morning, Dell and EMC announced their impending merger as Dell and Silver Lake set out to acquire EMC and its holdings with cash and stock, while maintaining VMware as an independent, publicly-traded company. The event sets off incredible tidal waves financially and technologically and raises many questions.

To that end, the CEOs and other principals from Dell, EMC, VMware, and Silver Lake held conference calls with shareholders and media/analysts this morning. The following 9 questions from participants of the latter call–New York Times, Financial Times, Boston Globe, Wikibon, and others–cover most of the big questions on everyone’s minds. In keeping with Dell’s private holding (and EMC’s soon-to-be), “no comment” showed up a few times where we all hoped to find insight. Time will tell.

Security Storage Technology Virtualization

Dell EMC VMware

Published October 2, 2015 by Chris

Is The Data Efficiency Metric A Bunch Of Hot Air?

Since the advent of thin provisioning, the concept of “data efficiency” has been used to describe how storage arrays deliver on large capacity requests while only reserving what’s actually occupied by data. For me, 3PAR (pre-HP) pioneered this in their thin provisioning and “Zero Detect” feature combo, which they like to deem a sort of deduplication 1.0 (they deduped zeroes in stream).

With the wider implementation of deduplication and compression, the “data efficiency” term has stepped (or been pushed) into marketing spotlights around the industry. HP 3PAR still promotes it, EMC XtremIO positions it at the top of their array metrics, and Pure Storage has it at the top-left of their capacity bar.

Is it bad to calculate and show? No. It’s a real statistic after all. Does it have any power or intrinsic value, though? No.

Storage Technology

3PAR deduplication Pure Storage XtremIO

Published September 30, 2015 by Chris

Cultivating Community In Technology

Through the years, I have been blessed and cursed by technology communities. Some foster cooperative atmospheres, pour in valuable content, and stoke ingenuity. Others are deserts and catacombs of enigmatic data with often similarly convoluted support structures.

I recently put together some feedback points for a burgeoning community and wanted to share it with the wider realm. Hopefully other organizations can read and improve their user groups and the domains they provide for collaboration.

Special thanks and credit to SolarWinds and their awesome investment in the Thwack community for many years. You’re one of the benchmarks for the thoughts below.

Technology

community SolarWinds

Published September 14, 2015 by Chris

Enable Logging for Sourcefire Security Intelligence Feeds

Last week I had a situation where an external client was reporting a complete inability to access any of our internet-accessible resources. If they changed IPs, though, they were fine. Considering it was over the Labor Day weekend, that we hadn’t changed *anything*, and that only this one client was having the problem, it seemed clear that the cause wasn’t on my side. They were small with outsourced IT, though, so pointing the finger back didn’t help much.

The fly in the ointment even for pointing at a cause was also that I didn’t have any logs of their traffic getting dropped. Normally I can search for initiating IPs or zoom into a small time window and find connections. Not this time. That left me with my own mental loophole of doubt about whether I actually did have an issue on my end.

So I opened a case with Cisco TAC, gave them the troubleshoot file dumps and asked for analysis. They stumbled some because I wasn’t able to attempt a reproduction on demand, since I didn’t have access to the client’s network to try things. TAC tried disabling some completely unrelated Snort rules, which did nothing (as expected). Eventually, I added a temporary explicit trust rule on their traffic so that they could regain access while we dug deeper. Once allowed, the connections were visible in FireSight Management Center (Defense Center) > Analysis > Connections > Events.

A couple days later, TAC returned with the root cause. The Sourcefire Security Intelligence Feed for Malware was blocking the client’s IP. That would have been really great to know on day 1, so I could have asked the client’s IT to address it. Of course, when I did give that root cause back to their IT, they responded that they knew about being blacklisted two days before reporting access issues to our network. Did they mention that at all? Nope. Where’s the love, people?

Networking Security Technology

blacklist Cisco logging Sourcefire

Published September 3, 2015 by Chris

VMworld 2015: Managing vSphere 6.0 Deployments & Upgrades, Part 1 #INF4944

Dilpreet & Mohan did a great job laying out the install and upgrade paths to reach vCenter 6.0, whether in Appliance or Windows mode. As I mentioned yesterday, the VMware team is encouraging customers to choose the Appliance (VCSA) moving forward due to increased performance, decreased complexity, and overall conformity. You gain much and lose nothing…except:

vSphere Update Manager (VUM), which will be integrated into VCSA in a 2016 release (Yay!). Presumably this will be vSphere 6.1 as that would put vSphere 6.0 a year old and this landmark milestone is too big (in my opinion) for a mere “Update 2”.

Dilpreet was kind to explain that the update which brings VUM integration will pull the VUM configuration & data from existing VUM (Windows) servers and into the VCSA component. This is great to hear as I was unclear about this walking away from the “Part 2” session yesterday.

Please check out the notes below and remember to pay attention to the order of your upgrades. Reference the KBs mentioned so you’re on a supported model & path.

Technology Virtualization

vCenter VMware VMworld

Published September 2, 2015 by Chris

VMworld 2015: Virtual SAN – Technical Deep Dive & What’s New #STO5906

This session is/was true to its title and definitely dove deep into Virtual SAN (VSAN). Due to the extreme nature of details, requirements, parameters, etc, I decided to conclude the live notes about 40 minutes into the presentation. With much respect for court reporters and typists, finishing out the slide notes would have been of no more value that practicing my typing skills.

VSAN looks promising and maturing as a solution. While the concept of metro stretched clusters sounds very intriguing, I believe it is only practical in the right use cases. My own environment, for example, involves significant writing with extended operations, which would not be feasible to replicate live. Local performance would suffer greatly while database crunching generated large amounts of data requiring acknowledgement from the remote site before proceeding.

On the other hand, if your environment is web-scale or low-write intensity, then VSAN stretched clusters may offer great value to you. As always, it depends.

The closing consideration is sheer cost of a VSAN solution. The “HY-4” recommended starting point retails around $10-15K per node (read: $40-60K for the HY-4). That is hardware only, so vSphere and VSAN licensing costs pile on top of that.

The beta preview with dedupe and erasure coding for space efficiency may take VSAN to the next level and make even its premium cost more palatable. IMO: external storage is still the path until this possibility brings down the cost (assuming capacity, not compute, is the limitation).

Storage Technology Virtualization

VMware VMworld VSAN

Published September 2, 2015 by Chris

VMworld 2015: Managing vSphere 6.0 Deployments & Upgrades, Part 2 #INF5123

Excellent presentation by Brian and Salil! They did a great job laying out the upgrade paths, caveats, as well as legacy references for folks coming from ESXi 4.x and early 5.x.

The biggest takeaways were 1) the encouragement to choose the upgrade path for ESXi host upgrades and 2) the announcement of vCSA including VUM in the next version. VMware now fully recommends vCSA for deployments moving forward–with Windows VUM out the way, this is something I can get behind!

A VMware fling exists to upgrade from vCenter Server to vCenter Server Appliance (vCSA) in certain scenarios. Flings aren’t production/supported tools, but may be helpful in the right use case(s).

Highly recommend this session/videos and the notes below!

Technology Virtualization

VMware VMworld

The Gurleyman Posts