What is Unraid and how to build an Unraid media server

[Manni]

I think I found my case
https://www.scan.co.uk/products/logic-c ... -s-minisas

[Pauven]

It's so similar to mine, I can't help but like it.

It's not clear which SAS connector it is using, I only saw a reference to MiniSAS. You would need to know exactly which connector (there's more than one) before order cables, with the other end the correct connector for whatever controller card(s) you get. But it looks like you have time to figure all that out.

[Manni]

Glad I get your blessings, and thanks again for steering me in the right direction

A clever user left the essential info in a comment:

"Your hot swap drives will plug into the backplane circuit board. You will need 6 x Molex connectors to provide power. Molex pins are rated at 11 amps, so you must have at least two Molex connectors and splitters, but three is going to provide a margin of safety.

You will also need 6 SAS leads. If it's the same as mine (a nearly identical model) you will need 8643 plugs on the end that connects to the backplane."

As I plan to buy this controller to go with it: https://www.scan.co.uk/products/24-port ... as-pcie-30 I'll have to get 6x 8643 on each end, as it looks like the 9305 uses the same connectors.

[Pauven]

Re the disks when you replace the controller, the issue is that they might have a different disk ID

No, the disk ID is part of the disk, has nothing to do with the controller.

this can happen when you update the f/w of your controller, and the controller reports IDs differently.

Where are you getting this info from? The disk ID is the manufacturer's ID combined with a serial number of the disk. Controller shouldn't ever change that.

There are many users that mix and match different brands of controllers plus their MB ports, and they move drives around as they see fit. I've never seen any reports of disk ID's changing. I won't pretend to know how Lime Tech solved this, but as far as I know their solution is solid. I've personally moved disks between two different Unraid servers of completely different builds, and the ID's were constant.

If for some reason the array config did get messed up, the easiest solution would be to remove parity, define the array with your existing drives that already are populated with data, tell Unraid to trust the config, add parity back in and let it rebuild the parity. I've actually used this trick before when I had a major drive blowout, but was able to recover the data outside the array. Since the drives are not striped, your data is whole on each individual drive, so you just tell Unraid to adopt it, so to speak, and the only thing you lost was your parity safety net.

In that case, you have to recreate the array, and provided you get the disks in the correct order, all should still be there.

In the olden days, with single parity, drive order didn't even matter. The parity data is just a checksum of all the bits at the same location on all disks, just basic math, doesn't matter which order you sum the bits. I'm admittedly confused as to how Lime Tech figured out how to do dual parity, and I'm not sure if order matters. In my mind it matters, but somehow I think it still doesn't matter - but I really don't know.

You have to remember, everything you learned about NAS and RAID from other solutions is out the window with Unraid, hence the name. Don't make assumptions based upon any other products. Now, for any of the above, including the ID, if you've seen something mentioned regarding Unraid, I'd like to know, especially if my understanding is flawed.

For now, MC has moved gracefully to disk 3 and has started to fill it up. It's still moving data at a nice and steady 110MB/s, so it's definitely more performant than the Windows client, as the speed would fluctuate between 112MB/s and 80MB/s, and even down to 60MB/s if I paused the copy process.

Thanks again for all your help, hopefully the rest of the process will take place while I do other things now, I'll update you when I get the new controller/cables in a week or so, unless I find the time to explore the VM rabbit hole, in which case you might hear from me (assuming it's still ok to bother you with this, don't hesitate to let me know if it's not).

Awesome! No bother at all, I enjoy this stuff.

[Manni]

Where are you getting this info from? The disk ID is the manufacturer's ID combined with a serial number of the disk. Controller shouldn't ever change that.

Here is what can happen when you change the f/w and the disks are not reported the same way by the controller: https://forums.unraid.net/topic/33187-s ... ent-322305

So I assume this could also happen when you change the controller, but the solution (new config) seems simple enough. I've tried that already during my tests to familiarise myself with non-essential data in the array.

Remember, I don't have any parity disks yet to speed up the migration (I did some performance tests initially, which confirmed the 50% drop in perf, so I recreated the array without them before starting my migration). I'll only add them when all the data is there and the new controller is happy with it. So that's one less thing to worry about.

[Pauven]

You will also need 6 SAS leads. If it's the same as mine (a nearly identical model) you will need 8643 plugs on the end that connects to the backplane.

That's a scary comment - what if it's not the same? I'd double-check with the seller/manufacturer.

As I plan to buy this controller to go with it: https://www.scan.co.uk/products/24-port ... as-pcie-30 I'll have to get 6x 8643 on each end, as it looks like the 9305 uses the same connectors.

Just be careful that users report it as supported in Unraid. I didn't recognize the card, so I googled and found some Unraid users reporting success with an LSI 9305, and also mention of an Avago 9305, but not Broadcom. Most likely these are all the same card under different names, but do your homework. Many a user has bought a card that doesn't work with Unraid - most work but not all, learn from other's mistakes.

By the way, that's a steep price for an all-in-one card solution. I speak from experience, as I essentially did the same thing. At the time, I was trying to make a low energy consumption server, and I chose a mini ITX motherboard with only a single full-size slot, so I had to get a 24-port card.

Assuming you have (or will get) a motherboard with at least 3 slots, you could (and probably should) get multiple 8port cards, or maybe 1x 8 and 1x 16. This stuff will be hidden away inside a case, you'll rarely see it (I haven't looked at mine in years, probably dusty as hell). As I mentioned earlier, my 24-port card is really just 3 separate cards (controller) merged into one behind a PCIe hub, and linux sees and reports it as 3 cards and 1 hub, not 1 24-port card. You should only chase a 24-port card if you are somehow slot constrained. You can get 3 cards and a new motherboard for less than that one 24-port card.

Also, don't follow the siren's call promising better performance for IOPS or anything else - these are just basic SATA ports, nothing much special about them. As long as you don't introduce any bottlenecks like we discussed, pretty much all of these cards perform exactly the same when used as HBA's. And they're functionally the same as your motherboards SATA ports too. In fact, if you had a motherboard with 4 or 8 SATA ports on it, then you could use those with a 4-to-1 SAS breakout cable (in reverse) to connect to the backplane, works the same.

[Manni]

With my currently available motherboards, I only have one PCI-E 8x available as the other PCI-E slot is taken by the GPU. I already have MB, CPU, memory, so it's worth spending a bit on the STA controller. I don't have enough SATA 6GB on the MBs, only 4 in two different controllers on the 3770K MB (currently used along with 8 ports from the Adaptec) and I don't think I have any SATA 6GB on the 2500K MB, so that's not even an option. I plan to use whatever ports are on the MB fo the SSD cache disk(s), that's all.

I don't listen to any sirens (remember I sent you the video that explains that controllers with more bandwidth are not needed with HDDs), but that's the only 24-port card that's recommended on the Unraid compatibility list, though admittedly not the Broadcom variant, but my intention was to check that before purchasing.

Re the cables, I'll wait until I have the case and the controller here, that's the only way to be sure which cables it needs. But it's likely to be 8643 both ends. We'll see.

EDIT: found a LSI branded, cheaper refurbished one but I generally prefer to pay a bit more to get new and with warranty. Also remember, I claim the VAT back as it's a company expense, so the prices are not as bad as they look.

[Pauven]

Where are you getting this info from? The disk ID is the manufacturer's ID combined with a serial number of the disk. Controller shouldn't ever change that.

Here is what can happen when you change the f/w and the disks are not reported the same way by the controller: https://forums.unraid.net/topic/33187-s ... ent-322305

So I assume this could also happen when you change the controller, but the solution (new config) seems simple enough. I've tried that already during my tests to familiarise myself with non-essential data in the array.

Thanks for that link. Strike experienced this problem because his original card wasn't working right at all, and was misreporting the ID. Notice how the ID looked completely different before and after the FW upgrade - I've never seen anything look like the before example he provided. It was borked. After the FW fixed his controller, it reported correctly, and that value should be the same regardless of controller reporting it.

So I think you just stumbled onto a special case of a user having a malfunctioning controller card. This is nothing to worry about, unless you notice any drive ID's looking odd like that too. Typically, drive ID's should make sense, with a manufacturer ID, a model number, and a unique serial number. Often you will have several drives bought at the same time that have serial numbers just a few apart.

Here's mine, and you can google HGST_HUH728080ALE604 and find the exact model drive I have. Same with WDC_WD30EFRX and ST8000NM0055 (one of the fabled problematic 8 TB Seagate Ironwolf or in this case Seagate Exos drives).

image.png (238.36 KiB) Viewed 5852 times

Remember, I don't have any parity disks yet to speed up the migration (I did some performance tests initially, which confirmed the 50% drop in perf, so I recreated the array without them before starting my migration). I'll only add them when all the data is there and the new controller is happy with it. So that's one less thing to worry about.

Yeah, so then it's next to impossible for you to bork this up! That's the beauty of Unraid, you can blow away and redefine your array a dozen times, and as long as you tell Unraid to trust the drives (and not zero them out), it will just work. Just like if you plugged them into a Windows machine. There is literally no striping or parity data on these drives, so you could pull one out and plug it into an external USB enclosure and take it on the road if you wanted, just so long as you had an OS that could read whatever disk format you chose (assuming you chose XFS, but I never asked).

I've lost RAID data when the physical controller died. I've lost RAID data when the Windows definition decided to quit. I know RAID is supposed to be redundant, but in my experience it is quite fragile, and worth spending the big $ for a solid controller. But even when disaster has stuck (and with over a dozen years under my belt, there's been times) I've never, ever lost a single byte with Unraid. The most important thing is if and when disaster strikes, stop! Go to the forums (not here, Unraid) and get user input on how to proceed. Chances are your data isn't actually lost and can be fully recovered, as long as you don't freak out and make the wrong move that wipes out your data permanently. I've had dual drive failures back when I was running single parity, and I still didn't lose data!

[Manni]

I know that the disk info was borked, but I'm not entirely convinced that two different controllers will not format the info slightly differently, even if it's valid info (mine is). I'm not worried, as I said the solution (new config) is simple enough. I'm just ready in case it happens.

Re the advantages of UnRAID, you're preaching a convert I've done some research before going that way and the fact that the data isn't striped over all the disks is a big reason why I decided to move to UnRAID (and move away from conventional RAID boxes), so I fully understand how it works. Also given that my limitation is my gigabit network (I have no intention to move to 10Gb any time soon), I don't lose any performance vs RAID 6, which was a concern I had. I never got more than 115MB/s on my gigabit network, so UnRAID doesn't do badly (assuming the disks are fast enough individually obviously). The fact that I can have only one drive on when accessing the data is great (I organised it that way). Makes a lot more sense than having 24 drives spin up (as on my Synology!).

And in 15 years of RAID use (I started with a Thecus N5200!) I've never lost any data, but I do backups to external drives before any big migration/expansion. If I had not done that, I would have lost data a few times.

[Jamie]

Hi Guys,

I just got caught up with today's discussion. Looks like your running with MC. Thanks to Paul for mentioning it. Sorry I didn't mention it in my post about Krusader. I used the traffic cop scenario before I learned about MC and mentioned it to someone, who flew in and out of this thread. He needed migration help for unraid . The traffic cop scenario ran from 100 mb/s to 85 mb/s for me too but I stayed with it because the learning curve was small and I had plenty of time to migrate in batches. I also had concerns about the power company turning off the power during the migration. I live in the so called woods so electrical power is not very dependable.

My hardware knowledge is light years behind you both so it's great to read your posts and learn from them. I too was a developer until 10 years ago so I do understand software speak including most things dos, windows, .net, java...... The kids now a days probably can run light years ahead me though. I try to keep up with technology but my need to know isn't as strong as it was when I was working.

Well, I hope that Manni has resolved all the issues thanks to Paul. I can't wait to read tomorrows episode of As Unraid Turns. For those who don't know this reference, I am referring to a afternoon soap opera "As the world turns".