What’s Involved In Upgrading XtremIO to 3.0?

xtremio_logoThe procedure for upgrading EMC XtremIO storage arrays to their latest major code release (3.0) has caused no shortage of conversation among the enterprise storage community. Granted, a large portion of that derives from competitors and marketing material which are keen to take advantage of this hurdle in the XtremIO track.

For those unfamiliar, the hurdle is the disruptive and destructive nature of the 3.0 upgrade process. To move from 2.4 to 3.0, customers must move all data from the brick(s) to another storage platform. EMC promises to provide the loaner gear to swing said data for the upgrade, but that doesn’t alleviate the infrastructure and migration impact of such a task (especially if some things are physical and without niceties like Storage vMotion).

We’ve had our share of challenges getting to this point, as you can read from prior posts, but we’re finally here. Since others are following closely behind, I thought it would be helpful to document the steps necessary to complete this upgrade (where possible, I’ll include the actual upgrade-to-3.0 tech details, but those are mostly handled by EMC).

1. Create an EMC Support Request (SR) requesting upgrade

This step is sort of a misnomer, because the XtremIO 3.0 upgrade isn’t handled by the EMC Support team. Due to the nature of swinging data, loaner hardware, etc, EMC Sales and Professional Support actually handles the process as a “free upgrade”.

2. Contact your EMC Account Team with the SR# and any environment needs (loaner storage, time frame, etc)

In my experience, the EMC Support team tried to handle this part for me (I already had loaner hardware on hand and just needed the upgrade itself), but they were unable to get through to my team, so customer contact broke the stalemate (after 12 days of Support-to-Sales emails).

3. Schedule the upgrade and perform any prerequisites (information gathering, etc)

The upgrade team typically wants 10 days lead time to schedule, and your environment may need even more, depending on maintenance windows and migration efforts.

4. Migrate data off of the XtremIO bricks to be upgraded — this means everything!

If you’re a VMware (or Hyper-V) shop, storage vMotion should reduce this pain point, especially if EMC loans you an equivalent XtremIO loaner brick. Speaking from experience, svMotion between XtremIO arrays is lightning fast.

5. Upgrade Day: Download the upgrade code and XMS

This step and those that follow may depend on your engagement. Initially, EMC wanted to schedule on-site field personnel, but since we had already been through an upgrade on a test array, I requested a remote session. If it had happened on-site, I believe the field engineer would have completed this step and possibly others.

The EMC engineer assigned to the upgrade should email an FTP link with the 3.x (in my case, 3.0.1-11) upgrade code, as well as a new XMS OVA. Download these (800MB each, roughly) to the place where you’ll host the remote session.

6. Find a spare terabyte for the new XMS

This one caught me by surprise. The new XMS OVA includes a 850GB+ thin provisioned disk. While that is indeed “thin”, the sheer possibility of it growing to 850GB caused me to create an extra datastore on my other storage to allow for the XMS to grow.

7. Deploy the new XMS

Typical OVA/OVF deployment in vSphere Client. Pick a new IP address and update DNS if you are maintaining the same XMS hostname (we had issues with reusing the same IP, even after clearing arp, etc).

8. Configure the XMS

User: xinstall, Option 4 to configure the XMS

9. Upload the upgrade code to the XMS

File: upgrade-to-3.0.1-11.tar, Target location: /images

10. Upgrade the cluster

Option 6, use filename from above, specify number of bricks, watch it roll!

11. Recreate and start the cluster

Reference fresh deployment documentation.

create-cluster cluster-name=<name> expected-number-of-bricks=<1,2,4,6> sc-mgr-ip=<ip_address>

11a. Encryption

The SE enabled encryption on our array, but this wasn’t necessary in our case, so he then disabled it. Switching modes is a cluster-stopped change, so make sure this is in the desired state before proceeding and starting the cluster. “self” enabled encryption. “disabled”, well, disables it.

stop-cluster cluster-id=1
modify-clusters-configuration cluster-id=1 encryption-command=switch-mode encryption-mode=<self | disabled>

11b. Start the cluster

start-cluster cluster-id=1

12. Set NTP server(s) and timezone

modify-datetime ntp-servers=["<ntp_server_1>","<ntp_server_2>"] timezone=<local_tz>

Use the ‘show-timezones’ command to list them.

13. Set XMS hostname (FQDN)

modify-server-name server-name="<fqdn_of_XMS>"

14. EMC SE creates a business services case to move the cluster to the installed state

This is definitely not a customer activity as EMC has to login to their business services portal. The case they create changes to state of the cluster in the EMC Install Base (IB) database. Since ours has been in production for 9 months, I’m not quite sure what it moved “from”, but it’s installed now.

15. Reconfigure ESRS monitoring

modify-syr-notifier enable site-name="<site_name>" connection-type=esrsgw esrs-gw-host="ip_or_hostname"

The SE (who was definitely used to working with EMC PS/FE) was unclear on some of this, and we were only able to set one IP for ESRS. If someone else knows how to configure an ESRS cluster in XtremIO, please comment.

16. Edit ESRS Configuration Tool to set new XMS IP address

Since the IP address of the new XMS is different, ESRS needs to be changed accordingly. Edit this on your gateway server(s).

Complete.

The upgrade required approximately 4 hours of time to complete, but probably 25-50% of this was due to the remote engineer’s unfamiliarity with a few tasks/details as well as a dinner break on his end (understandable). I’ll be remapping and provisioning shortly, but it looks good at this point.

5 Comments

  1. Chris P said:

    ESRS cluster is not supported yet. Only a single gateway. Should change by 4.0

    December 25, 2014
    Reply
  2. Chris said:

    Thanks, Chris. Yeah, I saw that in the release notes after the upgrade was complete.

    December 26, 2014
    Reply
  3. Mike Mazzerelli said:

    This is terrible. Why should anyone want to stress this gruesome upgrade process… My advice, take a serious look at 3PAR. Its light-years ahead.

    February 25, 2015
    Reply
  4. Chris said:

    We have a 3PAR V400 (P10400) on the floor as well and enjoy our experience with it. It’s up for replacement and we’re taking a fresh look at everything, including new 3PAR, so we’ll see what wins out.

    February 27, 2015
    Reply

Leave a Reply