Hey,
so before I tell you all about backing up the openfiler via rsync, I thought I should tell you about how we dealt with the servers themselves, the storages that is.
Just so you understand - I prefer white boxes over brands. Storage-wise anyway.
Why? I’ll explain.
I had to deal with purchasing the storages this past couple of months, so I talked to a lot of dealers.
Man, did they piss me off! You see, they don’t offer to sell you hardware anymore, they offer a solution. Yeah, this new fancy word in the industry. We offer a solution. It’s like they think they invent something so complicated that only they have it, and only they can manage and support it, and eventually you have to pay them like a zillion dollars for something you should call “a solution” and be proud of it.

I mean, it’s not the first time I purchased something from a dealer, believe me - I have a lot of experience in that area, but it seems to me that it’s somehow different when you buy an IBM x3550 server for about $3k from when you buy the same server that is 2U instead of 1U, with additional 10 sata hard drives, and suddenly the price goes up to $10k! WTF?! Almost a thousand bucks a disk? But wait, that’s not all. They don’t just offer it as a NAS, no! They offer it as a DAS. Yeah, a Direct Attached Storage. That means you actually have to connect it to another server just to manage it and share the storage. I’ve had one of these at home at one point - it’s a nightmare. Ok, maybe there are some big organizations that might find that useful - I don’t deny that, but it doesn’t mean that if you want a 10TB storage - this is the only way to go!

And then there’s the price of the hard drives. First of all, they want you to believe that the disks they’re putting inside are specially made and tested and that’s why they cost $500 for 1TB! Man, I do believe they test those drives well and everything, but it seems to me a bit unreasonable , since I can purchase one in a local store for $80. Now, of course you can tell me that I’m talking crazy, since you can’t really put an $80 hard drive in production… But I’ll tell you this. While their drives might be very reliable, you still can never get a 100% guarantee they won’t fail. And only RAID 6 and higher will save you when they fail 2 at a time (and I’ve seen it happening on the very production machine with the very original hard drives).

By the way, it doesn’t matter to them that there’s a 1.5TB drive on the market, which will actually give you another 50% - they only sell 1TB - that’s what they have - take it or leave it. And if you want to later replace the old 1TB drives for 1.5TB or whatever will come next - you have to do it through them. Because if you replace it with your own drive - you loose warranty for the whole server.
And their prices, as indicated above, just can’t get any more attractive.

I guess you already know where I’m getting at: there’s nothing better than getting two identical servers and having them backing up one another. Active-active, active-passive, whatever works for you. Of course, you should still have a RAID on each, but you’ll be much safer this way.

So basically, I think it’s much chipper, safer and easier buying two relatively simple machines and building a cluster, than being a sucker and purchasing a “solution”, and if they’re good sellers - they’ll probably have you pay them few thousands a year for “support”. And I think we all know how support works: oh, no sir - you’re not paying for these services, or this is a very serious technical issue - we’ll have to consult our bla-bla department. Just cut the crap, you don’t wanna be a sucker - do it by yourself.

Anyway, as you can see - I’m really pissed of, but this is just the tip of the iceberg, you’ll really see me going when I start talking about open source.

So what did we buy? Well, for a production environment we wanted something powerful and fast with at least 14-15TB of space, so we went for a bit pricey, but solid Teramaxx server. You might not heard of it, but it’s a nice small company that makes these custom made boxes with a very nice configuration. So we ended up with 2 quad core CPUs, 16GB of RAM, 3 Gb NICs, 2 PSUs, 2 3ware controllers and 16 1TB hard drives for the storage, plus 2 500GB drives for the OS. The price did come up as high as $10k, but we got twice we would if we got a “solution”.
1 of the RAID controllers handles the 2 500GB disks with the OS. The other one takes care of the actual storage.

Once it was in the air, we had to back it up with another machine. We decided that it should not be something this fancy, as its main purpose is data backup and only if the situation is really bad, we will make the switch for a day or two.
So for the second machine we went with a simple HP Proliant DL380 G5, which we got for under a thousand bucks from ebay (used, but in great condition). Then we worked on it a bit. If you’re not familiar with this box, it has a built in RAID controller that can’t control more than 4 drives in the default configuration. If you want to expand it to another 4 - you have to buy a cable, a “special” sata cable that in the cheapest store costs $125!! And than you have to purchase a RAID controller to handle it. And if you buy a native HP one(forgot the model) - it will only transfer at sata speed (150MB/sec). Not 3Gbit/sec, as sata II. WTF?!

I got really mad, so I threw that backplane out of the box, bought a used 3ware 9650SE-8LP RAID controller for $250 on ebay (which came with 2 sata cables - 4 connectors each) and put it in 8 1.5TB drives. Each disk cost me $200. Additionally, I connected 1 500GB ($70) drive to the onboard sata for the OS.

I also had to buy the hard drive cages - $22 a piece. And the only thing left was the power connectors. I went to my favorite electronics lab, and the guy replaced them for me (cut and solder) for $50 - cut off the original connector that goes to the backplane and replace it with 2×2 power connectors from the burned down PSU. Then multiply it by 2 via sata power connectors and you’re done. Lots of work? No. Lots of savings? Yes. Let your mind get creative? Definitely yes! To sum it up:
$3200 for an unwarranted HP server with 1 Core2duo CPU, 1GB of RAM, 1 PSU, 3ware RAID 6 controller and 8 1.5TB+1 500GB hard drives. Now that’s what I call cost effective!

Anyway, next time - rsync, as promised.
C ya.

Ozzik.