Expand RAID6 with 3ware and Openfiler
Tuesday, February 15th, 2011 | Author: Ozzik

Had some problems last week with expanding the RAID6 I have on a storage with Openfiler. This is what was needed to be done:
I have a storage with 16 hard drives for storage+2 for the OS. When I built it almost 3 years ago the idea was to have 1 RAID6 with 14 drives and 1 RAID1 with 2 drives obviously. Don’t ask me why - it was stupid to begin with and thus was never really used (the RAID1 that is). But came the time when we needed that additional space of 2 drives and the decision was to cancel the RAID1 and add the spare drives to the main RAID6, expanding the existing space.

The RAID controller is 3ware 9650se and after reading the manual it seemed pretty simple: delete the RAID1, have the spare drives and do the RAID6 migration to include them. After the initialization(is this the right word?) I was supposed to have the 25-26TB of space instead of the existing 22.

And so it was. Well, almost. After everything was finished I saw the new empty space in the Openfiler’s web gui: 3775GB or something like that. But the thing was that when trying to create the new volume it would create a ridiculous 4MB thingy, although I entered the max size. After fiddling with this for a bit I realised I can’t risk the data (all the company’s docs and files) any further and it was decided to make a clean start. So I transferred them all to the backup storage and started playing.

By the way, one other thing that made me want to reinstall the OS was the fact that there was some problem with the last updates which were sometime in 2008. I know this because I already tried updating the backup storage before that (a month ago) and completely messed it up. I ended up installing fresh OS and updating it - it went smooth. And since I needed some of the updates badly it was in my interest anyway to do this.

If you care about what updates I needed so badly, it was the samba update. Apparently, win7 has some upgraded mechanisms of authentication, so if you try accessing the openfiler’s shares from win7 by name - all goes well, but if you do this by IP - you’ll end up entering the credentials over and over again - loop that never ends. There is a fix - you just need to uncheck a couple of boxes on the windows machine, but when you have more than one of those - it just makes sense to upgrade the Openfiler.
The solution for win7 is:
Go to: Control Panel -> Administrative Tools -> Local Security Policy
Select: Local Policies -> Security Options
There are two settings to change.
“Network security: LAN Manager authentication level” -> Send LM & NTLM responses
“Minimum session security for NTLM SSP” -> uncheck: Require 128-bit encryption
taken from here

Anyway, back to me. Formatted the OS drive, installed, updated and configured. And all that only to find out that nothing’s changed partition-wise. I still saw all the data partition: 22TB of previous partitions and 3.7TB of the new empty space. Trying to create a partition ends in 4MB. WTF?!

So it’s not the OS. Re-read the user guide from 3ware - not that many options to make mistakes. What then? The Openfiler’s forums never mentioned anything like this. So I decided that since I have the backup I should just delete the RAID and start the data partition from scratch. It was pretty sucky because I knew that I’ll have to restore the data afterwards - at least 6TB for now. That’s another week working on the backup. Oh, well…

So here we go, 3ware web gui, remove the unit, create new unit, initialize. Meanwhile access the Openfiler’s gui, see the 26TB space, create the 26TB partition. Wait, WHAT?! 22TB?!

This is where I was about to loose it. Mostly because it was so unexpected: I was supposed to be done twice by then.
Now, as you might know, if you want to have a more than 2TB partition on Linux you have to label it GPT. Otherwise it defaults to MSDOS label. I knew that because I learned it the first time I created this storage 3 years ago. You have to ssh into the machine and run

parted /dev/sdX
mklabel gpt

replace the X with the right letter, of course.
Now, here’s the thing: I didn’t do it this time and yet it did recognize the 20+TB. How’s that? Maybe the partition wasn’t really deleted although the RAID was removed and recreated.

fdisk -l

showed me that the partition was in fact GPT.
So I ran parted and saw this (syntax):

 mklabel LABEL-TYPE         create a new disklabel (partition table)

This is when it hit me - the partition table didn’t change despite the fact that RAID was destroyed. Or was it…?
Anyway, a simple new

mklabel gpt

on this drive solved all of my problems. Of course, by then all of the data was destroyed, but I was ready for it and hopefully by the end of this week our office will be back on the main storage.

Hope it’ll help someone not to loose it when dealing with this.


VN:F [1.9.6_1107]
Rating: 10.0/10 (1 vote cast)
VN:F [1.9.6_1107]
Rating: 0 (from 0 votes)
Expand RAID6 with 3ware and Openfiler, 10.0 out of 10 based on 1 rating