Step-by-step upgrade from DRBD 0.7 to DRBD 8

I’ve been asked by a number of people on how to do an upgrade from DRBD version 0.7 to DRBD 8. This upgrade does necessitate some minimal service down time, but it’s really not rocket science. And no, it does not force you to sync all of your data all over again.

Here’s my quick write-up.

An initial word of caution

Do not, I repeat do not attempt your upgrade unless you have at least read this blog entry to the finish.

Getting ready

First, you need to make sure that you have both your DRBD 8 userland binaries and kernel module ready to install. For our support customers, this means that you simply download two RPMs (or .debs) from our support web site. Make sure you have the right packages; you want those that match your system architecture and (for the kernel module) also your running kernel. Also, make sure you get DRBD 8.2.3 or later; there were some bugs in pre-8.2.3 versions that ran a slight chance of messing up 0.7 metadata conversions.

To be on the perfectly safe side, it may be wise to also have the RPMs of your latest 0.7 version at the ready. Just in case you wish to abort the upgrade in the middle and revert.

Second, and here is your down time: you need to shut down your Heartbeat services on both nodes, unmount any file systems that reside on DRBDs, and make all DRBD devices Secondary. This is necessary for the meta data conversion that follows. You want to be sure your DRBD devices are in sync, block by block.

The sequence of commands you issue is this:

  • /etc/init.d/heartbeat stop
  • mount (this should show no DRBDs being mounted anymore)
  • drbdadm secondary all (just in case)
  • cat /proc/drbd (this should show all your DRBDs as being Secondary)
  • drbdadm down all
  • rmmod drbd

Out with the old…

At this point, DRBD is completely disabled, you should not see a /proc/drbd file anymore.

You can now uninstall your old (DRBD 0.7) packages. The DRBD 8 userland binaries will conflict with the 0.7 packages, so you won’t be able to complete the next step without previously uninstalling those.

… in with the new.

Now issue rpm -ivh, dpkg -i, or whatever else is appropriate for your distribution, giving the names of the two DRBD packages you’re installing.

Your distribution may also provide DRBD 8 packages in an installation archive; in order to install from there, you will need to issue apt-get install package, yum install package, rug install package or similar. Just make sure that your distro archive indeed carries 8.2.3 or above.

After you have installed your new packages, for every configured resource, issue this command:

  • drbdadm create-md resourcename

At this point, drbdadm will detect that DRBD 0.7 metadata is already in place for the resource, and will ask you whether you want to upgrade that metadata to DRBD 8 format.

If, for any reason, drbdadm fails to detect the existing DRBD 0.7 metadata on that device, and asks whether you want to create a new set of metadata, do not continue. Say “no”. If you’re a support customer, open a support incident. If you are a community user, post a problem report to the mailing list. You have not changed anything about your DRBD configuration at this point. You may safely uninstall your DRBD 8 packages, reinstall the DRBD 0.7 ones, and restart DRBD and Heartbeat to maintain full service redundancy until your issue is resolved.

If DRBD’s 0.7 metadata has been detected correctly, say “yes” to convert to version 8 format. This conversion step must be completed on both nodes. From that point forward, your DRBD metadata is no longer compatible with DRBD 0.7. But that’s probably of no importance, since you’re upgrading your second node simultaneously, anyway.

Then, you can continue:

  • modprobe drbd
  • cat /proc/drbd

At this point, /proc/drbd will not yet show any configured devices, but you should already see the DRBD 8 version information. Now, after you have completed all of the above steps on both nodes, do:

  • drbdadm up all

And again:

  • cat /proc/drbd

Finishing up

Now, all of your devices should be listed as cs:Connected, st:Secondary/Secondary, and ds:UpToDate/UpToDate.

When everything is hunky dory, go ahead and fire up Heartbeat again. Your trusted cluster manager should now make DRBDs Primary as necessary, mount filesystems, and start your cluster resources.

If you know your stuff and are well prepared, the entire upgrade procedure should take no more than 10 minutes or so.

One more thing: if you have been using the group keyword in the syncer section(s) of your drbd.conf, be advised that it is no longer supported in version 8. It has been supplanted by the after keyword which enables you to sync resources after one another. More often than not, however, you’ll want to sync all your resources in parallel, so in most cases you won’t be needing the after option.

7 Responses to Step-by-step upgrade from DRBD 0.7 to DRBD 8

  1. Hexa says:

    “After you have installed your new packages, for every configured resource that uses internal meta data (meta-disk internal), issue this command:”
    This is only haf of the truth. Even external metadata disks need to have metadata converted to 0.8 format.

  2. Florian Haas says:

    You are correct. The reference to internal metadata was a leftover from an earlier version of this post, which detailed that you had to handle external metadata differently. As of DRBD 8.0.7, the steps for meta data conversion are identical regardless of whether you are using external or internal metadata.

    Fixed in text. Thanks.

  3. Brad says:

    Just wanted to comment here. Your post talks about getting drbd 8.2.3 or greater, and yet a comment on this article indicates that 8.0.7 or later will work fine for metadata conversion.

    My issue is that Debian stable will be using DRBD 8.0.x, since it is your stable release. That is, zero protocol and config changes. Anyhow, is the upgrade identical for 8.0.x as 8.2.x? If so, please clarify, as I am not sure from what I see above. I get the impression that 8.0.x is the same, but I am not sure.

  4. Christian says:

    Thanks a lot for this step-by-step documentation. It worked fine for my migration.

  5. There is a number of configuration parameters introduced in 8.x. But only the group keyword was explicitly mentioned in this blog entry. Does it mean that DRBD 8.x will operate exactly the same way as DRBD 0.7.x given drbd.conf left from the latter version?

    • Florian Haas says:

      Yes, what I described for the 8.0 branch also works for 8.2 (actually, 8.3 as that has now superseded 8.2).

    • Florian Haas says:

      Synchonization groups (the group keyword in DRBD 0.7) were supplanted by synchonization dependencies (the after keyword in DRBD 8). And the varous -cmd options in DRBD 0.7 were replaced by the handlers section. Those are all the non backward compatible config changes that I am aware of. So unless you are using one of those options, your drbd.conf should play nicely with DRBD 8.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: