Decorum for the Forum:
Privacy Policy: Click Here to Review (updated September 30, 2020)
- Be nice. If you want to be mean, try Reddit.
- No Piracy. If you want to be a thief, there are dark places on the internet dedicated to that.
- No Cracking. Discussions on AnyDVD, DeUHD, DVDFab, UHDKeys and similar tools are not permitted here.
- No Spamming. If you want to make a buck, work smarter... somewhere else.
- No Adult Content. Half the internet is dedicated to adult content. This half isn't.
Privacy Policy: Click Here to Review (updated September 30, 2020)
What is Unraid and how to build an Unraid media server
Re: What is Unraid and how to build an Unraid media server
Orders placed. I went with the Dell H310 version of the 9211-8i, and I bought from The Art of Server since they do a few upgrades and the product is tested for all 8 lanes. Got the reverse breakout cables from Amazon. All said, my total bill with shipping and taxes came to about $203, so not too bad.
I guess my server's out of commission for another week until it arrives. I can't believe I just spent an entire day trouble-shooting, researching, and shopping. Certainly not what I had planned for my day off.
I guess my server's out of commission for another week until it arrives. I can't believe I just spent an entire day trouble-shooting, researching, and shopping. Certainly not what I had planned for my day off.
President, Chameleon Consulting LLC
Author, Chameleon MediaCenter
Author, Chameleon MediaCenter
Re: What is Unraid and how to build an Unraid media server
Sounds good. I know how it feels, but I'm sure you'll be happy you've done it.
You were right about the 9305-16i. I don't know why, I thought that was the one I had purchased, but I checked and I purchased a cheaper PCI 2.0 model. That's why I swapped the 16-port and the 8-port controllers to the PCI 2.0 mobo (that had 2 PCI 2.0 8X slot) and I upgraded the other motherboard so that the 24-port could get its full 7.8GB/s bandwidth in its PCI 3.0 8x slot, hence 330MB/s per disk. That way on the "slow" server B I'm limited to 250-500 MB/s per disk, and on the "fast" server A I'm limited to 330MB/s per disk (if all bays used).
You were right about the 9305-16i. I don't know why, I thought that was the one I had purchased, but I checked and I purchased a cheaper PCI 2.0 model. That's why I swapped the 16-port and the 8-port controllers to the PCI 2.0 mobo (that had 2 PCI 2.0 8X slot) and I upgraded the other motherboard so that the 24-port could get its full 7.8GB/s bandwidth in its PCI 3.0 8x slot, hence 330MB/s per disk. That way on the "slow" server B I'm limited to 250-500 MB/s per disk, and on the "fast" server A I'm limited to 330MB/s per disk (if all bays used).
Re: What is Unraid and how to build an Unraid media server
You're smart to be thinking ahead about these things. All your numbers have me going back over my numbers, and I've come to an unexpected realization.
When I first built this server, I was using a mini-ITX board that only had a single PCIe 2.0 x16 slot (yep, 1 slot, that's all). The Highpoint 2760A controller I chose also had a PCIe 2.0 x16 bus connection. So, throughput per drive is around 320 MB/s. But I came close to buying a cheaper 2.0 x8 card, and almost bought an ITX board that only had a 2.0 x8 slot (cheapskate MB makers, it was a lot of work to find a x16 capable MB...). At the time, the math was solid, even on 2.0 x8 I could get around 160 MB/s if using all 24 ports, or 182 MB/s if only using 22. The 3TB WD Red drives only maxed out at 154 MB/s, so that would have been more than sufficient.
But I never anticipated going to 8TB 7200 RPM drives, which are over 50% faster, peaking around 235 MB/s. It would have been bottlenecked on that PCI 2.0 x8 connection, but only after removing the last of the 3TB drives. I never thought I'd need faster, but I bought it anyway just in case.
Now that I'm ripping out the 2760A, I'm realizing that technically I've never needed the extra speed. I came close, in a few more years the last of my 3TB drives will be gone. But PCIe 2.0 x 8 would have been sufficient for the past 9 years I've had this card, since on Unraid those 3TB drives have always set the slower pace.
These new 2.0 x8 cards will technically increase my bandwidth up to 500 MB/s per drive, way more than I'll ever need, since each card only supports 8 drives.
When I first built this server, I was using a mini-ITX board that only had a single PCIe 2.0 x16 slot (yep, 1 slot, that's all). The Highpoint 2760A controller I chose also had a PCIe 2.0 x16 bus connection. So, throughput per drive is around 320 MB/s. But I came close to buying a cheaper 2.0 x8 card, and almost bought an ITX board that only had a 2.0 x8 slot (cheapskate MB makers, it was a lot of work to find a x16 capable MB...). At the time, the math was solid, even on 2.0 x8 I could get around 160 MB/s if using all 24 ports, or 182 MB/s if only using 22. The 3TB WD Red drives only maxed out at 154 MB/s, so that would have been more than sufficient.
But I never anticipated going to 8TB 7200 RPM drives, which are over 50% faster, peaking around 235 MB/s. It would have been bottlenecked on that PCI 2.0 x8 connection, but only after removing the last of the 3TB drives. I never thought I'd need faster, but I bought it anyway just in case.
Now that I'm ripping out the 2760A, I'm realizing that technically I've never needed the extra speed. I came close, in a few more years the last of my 3TB drives will be gone. But PCIe 2.0 x 8 would have been sufficient for the past 9 years I've had this card, since on Unraid those 3TB drives have always set the slower pace.
These new 2.0 x8 cards will technically increase my bandwidth up to 500 MB/s per drive, way more than I'll ever need, since each card only supports 8 drives.
President, Chameleon Consulting LLC
Author, Chameleon MediaCenter
Author, Chameleon MediaCenter
Re: What is Unraid and how to build an Unraid media server
If I had been really smart I'd have listened to your advice and would have bought two cards instead of a single 24-port for server A. I only realised after the fact that it meant I had to have PCI 3.0 8x bandwidth to not suffer a bottleneck to 166MB/s (ouch!), hence the mobo upgrade... Anyway, I had to upgrade the mobo of server B for other reasons, so I don't regret it as the old server A mobo ended up in server B.
My WD Red Pro drives on server B peak at 265MB/s each, so I use the 8-port card for the first 8 bays and the 16-port for the next 16, that way the limitation of the 6-port (250MB/s) doesn't kick in as I never use all the drives at the same time on that one. The max I would use is 20 data drives (plus two parity), so no more than 14 disks on the 16-port, which raises the bandwidth limitation to 285MB/s per disk. If I was using the 16-port for the first 16 drives (as logic would dictate), then they would be limited to 250MB/s. Not that it makes a huge difference given that after the first 500GB, it becomes 250MB/s and then less...
My WD Red Pro drives on server B peak at 265MB/s each, so I use the 8-port card for the first 8 bays and the 16-port for the next 16, that way the limitation of the 6-port (250MB/s) doesn't kick in as I never use all the drives at the same time on that one. The max I would use is 20 data drives (plus two parity), so no more than 14 disks on the 16-port, which raises the bandwidth limitation to 285MB/s per disk. If I was using the 16-port for the first 16 drives (as logic would dictate), then they would be limited to 250MB/s. Not that it makes a huge difference given that after the first 500GB, it becomes 250MB/s and then less...
Re: What is Unraid and how to build an Unraid media server
Over the past year and a half, I've come to distrust the USPS shipping estimates. Often it would take an extra week or two beyond the estimate.
Fingers crossed that they've dealt with the issues, as I might actually get my new Dell H310's today:
Not too bad for shipping coast to coast. Amazon already delivered the two reverse breakout cables on Monday, so I might be able to get the server back up and running today if the USPS delivers (haha, delivers...).
I'm still a bit anxious about doing the heart transplant on my production-class server. Maybe I can get it up and running again as-is and do a quick differential backup, since it's been a few months since the last one. Better safe than sorry.
Fingers crossed that they've dealt with the issues, as I might actually get my new Dell H310's today:
Not too bad for shipping coast to coast. Amazon already delivered the two reverse breakout cables on Monday, so I might be able to get the server back up and running today if the USPS delivers (haha, delivers...).
I'm still a bit anxious about doing the heart transplant on my production-class server. Maybe I can get it up and running again as-is and do a quick differential backup, since it's been a few months since the last one. Better safe than sorry.
President, Chameleon Consulting LLC
Author, Chameleon MediaCenter
Author, Chameleon MediaCenter
Re: What is Unraid and how to build an Unraid media server
I'll keep my fingers crossed for you. ![Smile :)](./images/smilies/icon_e_smile.gif)
To reassure you, I've done a heart transplant three times now, and all went without a glitch (apart from the USB device connected to the VM, see my post above, but as I don't think you're changing the mobo and only the HBA adapters, you should be fine).
I've finished my huge migration/restore (165TB of data in total in 2 servers!). It took less time than estimated thanks to Unraid great flexibility and the ability to restore to different disks in the same server from different sources simultaneously (I was most of the time transferring at 500-600MB/s instead of the 100-200MB/s estimated). I followed your suggestion of short names (A4K and B2K) for the servers, and I kept the share names as short as possible, without any spaces and all in lowercase.
Server A has now 15 disks (12 data WD Red 6TB, 2 parity WD PRo 6TB, 1TB SSD cache) for a total of 72TB in the array. I have 7 WD Red 6TB spare disks for that server, so I have room to grow. This is for all my 4K UHD Bluray disks. 3770K, 16GB Ram, 1 VM for MyMovies/CMC.
Server B has 21 disks (19 data WD Red Pro 6TB, 2 parity also WD Pro 6TB, 1TB SSD cache) for a total of 114TB in the array. I have 3 6TB WD REd Pro spares. This one contains all my 2K Blurays (2D and 3D). Also 3770K, 16GB Ram, no VM.
Parity building/check takes around 14 hours on both servers.
Both servers support WOL and are back online in 1-2 minutes max, which is a great improvement over the QNAP (8-10 minutes) and even the Synology (4-6 Minutes). Server A works fine, B has some issues waking up sometimes, I'm trying to debug that. I'm also trying to finetune the network activity to keep them alive, especially when doing low bandwidth things such as CM metadata generation, CCC full export etc, especially as most of the written data goes to cache and cache activity is excluded. Maybe I should include it and ignore ethernet activity. Anyway, that's the only part I'm still working on.
As all the data is now protected with dual parity on my two Unraid servers, I'm going to install my backup disks into the Synology 2411 (12-bay with 4TB Hitachi), 1211 (12 bay-extension with 3TB Hitachi) and TS859 (8-bay with 2TB Hitachi) for backup purposes only. The Thecus N5200 (5-bay with 1.5TB WD) had already been repurposed for backup duties a long time ago, due to its ridiculously small capacity (5TB). I only keep it because it's RAID5 protected. That's around 90-100TB of usable backup space in total, enough to backup all my 4K and 3D content.
That's a bit of a belt and braces strategy given that I also have the original disks, but quite often when I go back to the original physical disk after 5+ years I realise it's not readable anymore. Plus the time to rip all these is a lot, so it's more to save the time it would take re-ripping than anything else. 4K UHD bluray disks seem to be a lot more fragile than bluray, even if I only use them once (to rip them after purchase), then they are put back in their case and never touched again.
![Smile :)](./images/smilies/icon_e_smile.gif)
To reassure you, I've done a heart transplant three times now, and all went without a glitch (apart from the USB device connected to the VM, see my post above, but as I don't think you're changing the mobo and only the HBA adapters, you should be fine).
I've finished my huge migration/restore (165TB of data in total in 2 servers!). It took less time than estimated thanks to Unraid great flexibility and the ability to restore to different disks in the same server from different sources simultaneously (I was most of the time transferring at 500-600MB/s instead of the 100-200MB/s estimated). I followed your suggestion of short names (A4K and B2K) for the servers, and I kept the share names as short as possible, without any spaces and all in lowercase.
Server A has now 15 disks (12 data WD Red 6TB, 2 parity WD PRo 6TB, 1TB SSD cache) for a total of 72TB in the array. I have 7 WD Red 6TB spare disks for that server, so I have room to grow. This is for all my 4K UHD Bluray disks. 3770K, 16GB Ram, 1 VM for MyMovies/CMC.
Server B has 21 disks (19 data WD Red Pro 6TB, 2 parity also WD Pro 6TB, 1TB SSD cache) for a total of 114TB in the array. I have 3 6TB WD REd Pro spares. This one contains all my 2K Blurays (2D and 3D). Also 3770K, 16GB Ram, no VM.
Parity building/check takes around 14 hours on both servers.
Both servers support WOL and are back online in 1-2 minutes max, which is a great improvement over the QNAP (8-10 minutes) and even the Synology (4-6 Minutes). Server A works fine, B has some issues waking up sometimes, I'm trying to debug that. I'm also trying to finetune the network activity to keep them alive, especially when doing low bandwidth things such as CM metadata generation, CCC full export etc, especially as most of the written data goes to cache and cache activity is excluded. Maybe I should include it and ignore ethernet activity. Anyway, that's the only part I'm still working on.
As all the data is now protected with dual parity on my two Unraid servers, I'm going to install my backup disks into the Synology 2411 (12-bay with 4TB Hitachi), 1211 (12 bay-extension with 3TB Hitachi) and TS859 (8-bay with 2TB Hitachi) for backup purposes only. The Thecus N5200 (5-bay with 1.5TB WD) had already been repurposed for backup duties a long time ago, due to its ridiculously small capacity (5TB). I only keep it because it's RAID5 protected. That's around 90-100TB of usable backup space in total, enough to backup all my 4K and 3D content.
That's a bit of a belt and braces strategy given that I also have the original disks, but quite often when I go back to the original physical disk after 5+ years I realise it's not readable anymore. Plus the time to rip all these is a lot, so it's more to save the time it would take re-ripping than anything else. 4K UHD bluray disks seem to be a lot more fragile than bluray, even if I only use them once (to rip them after purchase), then they are put back in their case and never touched again.
Re: What is Unraid and how to build an Unraid media server
That's an amazing accomplishment, congrats!
It's also taken me over a decade to get to this point, and I remember more than a few discs that were a battle to rip, and some battles I lost. I would never want to do that again. Like you, I consider my original discs a fragile backup, my Unraid server is my main media source, and I have an additional backup to external USB HDD's that I keep offline 99% of the time. If I can't recover with this setup, then I'll probably just exit the game. The only thing I'm missing is an off-site backup. I came very close to having that setup recently, but ultimately that arrangement fell through. I do have a nice fire and water resistant storage box, though, so that makes for a bit safer on-site storage.
Speaking of backups, mine has been running for a little over 5 hours, and I'm beginning to get concerned that I will run out of space. I have 5x 16TB drives in a 80TB pool, and before the backup I had 5.69 TB free, which was less than I realized. So far it's only completed the 4K share, and has been working on the Blu-ray share for the past 2.5 hours. There's still the DVD and TV shares to go. Free space has dropped to 3.18 TB. Going from memory, I don't think I've done so much in the DVD's and TV shares, so I'm thinking I'll be okay, but I didn't expect as much as 2.5 TB of new data to backup, and I'm blowing through that so I really don't know how much is left to copy.
This got me looking to buy a new external drive to expand my backup pool. I was disappointed that the 16TB drives I bought 2 years ago are the same price now, I figured they'd be cheaper. I'd prefer to buy a 20TB drive, but those are ridiculously expensive in external form factor. I can find an 18TB drive at a comparable price as compared to the 16TB, so that's what I'd likely get if I had to buy now. I'm hoping I still have 2-3TB of space left over after this backup completes, and I can put off buying another drive for a while longer. Maybe things will be better in the fall.
I did get the Dell H310 controller cards in the mail, so if this backup ever completes I'm ready to do the transplant.
1000% yes. I've had brand new discs that seem to deteriorate the very first time I use them, almost like a Mission Impossible secret message. And I perceive that this is occurring with more frequency with my 4K discs. My Spears & Munsil UHD 4K disc worked pretty great in the beginning, as I was using it for benchmarking my projector. But then I noticed that some of the demo videos struggled to play. As I kept trying to play them, they progressively got worse. Eventually the disc wasn't even recognized anymore by my HTPC, it literally wouldn't recognize the disc was even in the drive. I loaded it up in my office PC, and struggled to find a drive that could mount and read the disc. Eventually I was able to rip it to backup, from which it plays beautifully.Manni wrote: ↑Wed Apr 06, 2022 4:33 pm That's a bit of a belt and braces strategy given that I also have the original disks, but quite often when I go back to the original physical disk after 5+ years I realise it's not readable anymore. Plus the time to rip all these is a lot, so it's more to save the time it would take re-ripping than anything else. 4K UHD bluray disks seem to be a lot more fragile than bluray, even if I only use them once (to rip them after purchase), then they are put back in their case and never touched again.
It's also taken me over a decade to get to this point, and I remember more than a few discs that were a battle to rip, and some battles I lost. I would never want to do that again. Like you, I consider my original discs a fragile backup, my Unraid server is my main media source, and I have an additional backup to external USB HDD's that I keep offline 99% of the time. If I can't recover with this setup, then I'll probably just exit the game. The only thing I'm missing is an off-site backup. I came very close to having that setup recently, but ultimately that arrangement fell through. I do have a nice fire and water resistant storage box, though, so that makes for a bit safer on-site storage.
Speaking of backups, mine has been running for a little over 5 hours, and I'm beginning to get concerned that I will run out of space. I have 5x 16TB drives in a 80TB pool, and before the backup I had 5.69 TB free, which was less than I realized. So far it's only completed the 4K share, and has been working on the Blu-ray share for the past 2.5 hours. There's still the DVD and TV shares to go. Free space has dropped to 3.18 TB. Going from memory, I don't think I've done so much in the DVD's and TV shares, so I'm thinking I'll be okay, but I didn't expect as much as 2.5 TB of new data to backup, and I'm blowing through that so I really don't know how much is left to copy.
This got me looking to buy a new external drive to expand my backup pool. I was disappointed that the 16TB drives I bought 2 years ago are the same price now, I figured they'd be cheaper. I'd prefer to buy a 20TB drive, but those are ridiculously expensive in external form factor. I can find an 18TB drive at a comparable price as compared to the 16TB, so that's what I'd likely get if I had to buy now. I'm hoping I still have 2-3TB of space left over after this backup completes, and I can put off buying another drive for a while longer. Maybe things will be better in the fall.
I did get the Dell H310 controller cards in the mail, so if this backup ever completes I'm ready to do the transplant.
President, Chameleon Consulting LLC
Author, Chameleon MediaCenter
Author, Chameleon MediaCenter
Re: What is Unraid and how to build an Unraid media server
Keep us posted! ![Smile :)](./images/smilies/icon_e_smile.gif)
![Smile :)](./images/smilies/icon_e_smile.gif)
Re: What is Unraid and how to build an Unraid media server
My array backup completed with a scant 1.3 TB left in my backup pool. Sheesh, talk about cutting it close. I don't think I can do any more backups until I add a 6th drive to the pool. Luckily it completed without running out of space. Lesson learned - double-check the combined size of your shares in Unraid before doing the backup. I just added up the 4 shares, and Unraid reports they are using 78.6 TB, which matches up with the free space remaining after the backup. Since I only do backups every 3-6 months, hopefully drive prices will drop and sizes will climb - fingers crossed.
Heart transplant completed, parity check running, yay.
Initial perception of parity check speed is that it's unchanged, yay. Should complete in another 16.5 hours, then I'll know for sure.
New idle power consumption low below 100 watts, yay:
My UPS only reports in 1% increments, so actual idle is somewhere between 89 and 100 watts. Oh, and since that's what the UPS sees, that includes any other devices I have plugged in on the battery backup side, which is both the monitor and USB backup drives all in standby. I'll have to test if it goes down further if I unplug all those.
The transplant was not without some minor hiccups. The Dell H310 cards lift one end out of the PCIe slot if I tighten the bracket screw at the other end. Never had that happen before. I know the seller replaces the brackets with mesh ones for better airflow, perhaps these brackets are out of spec. I've loosened the screws 1-2 turns, and the cards sit flat, but aren't really secure. These cards don't have the tab that lets them work with the GPU retention mechanism. I'm a little worried these may vibrate loose - especially the screws which could come loose and fall onto the MB and cause a short.
One of the cards is in the slot next to my graphics card, restricting airflow on the H310's heatsink. Not crazy about this. I can move the GPU one slot over, but I think that means I lose an M2 slot due to lack of lanes on this MB. I'm not currently using that M2 slot, but always planned to add a 2nd cache drive for a mirrored RAID cache setup.
In the BIOS, I made sure to enable SATA Hot-plug support, it was disabled by default. Lucky catch on my part, because I definitely use hot-plugging. No idea what happens if you hot-plug on those SATA ports without hot-plug support enabled.
All of the Linux assigned device letters have changed. I know this doesn't matter a lick, but I had worked hard with my previous setup to have them increment from sdb->sdv in line with the Unraid slot numbers. It's my OCD kicking in, now the order is changed and reversed and random. I don't think this is correctable either - I guess the controller card is just wired differently. The best I could do is sequential in reverse order.
Since these H310's are in IT mode, I expected boot up to be quicker. It's not, still the long slow process of loading the card's ROM.
All-in-all, not too bad. I'll reach out to The Art of Server where I bought the H310's and ask about those brackets.
Heart transplant completed, parity check running, yay.
Initial perception of parity check speed is that it's unchanged, yay. Should complete in another 16.5 hours, then I'll know for sure.
New idle power consumption low below 100 watts, yay:
My UPS only reports in 1% increments, so actual idle is somewhere between 89 and 100 watts. Oh, and since that's what the UPS sees, that includes any other devices I have plugged in on the battery backup side, which is both the monitor and USB backup drives all in standby. I'll have to test if it goes down further if I unplug all those.
The transplant was not without some minor hiccups. The Dell H310 cards lift one end out of the PCIe slot if I tighten the bracket screw at the other end. Never had that happen before. I know the seller replaces the brackets with mesh ones for better airflow, perhaps these brackets are out of spec. I've loosened the screws 1-2 turns, and the cards sit flat, but aren't really secure. These cards don't have the tab that lets them work with the GPU retention mechanism. I'm a little worried these may vibrate loose - especially the screws which could come loose and fall onto the MB and cause a short.
One of the cards is in the slot next to my graphics card, restricting airflow on the H310's heatsink. Not crazy about this. I can move the GPU one slot over, but I think that means I lose an M2 slot due to lack of lanes on this MB. I'm not currently using that M2 slot, but always planned to add a 2nd cache drive for a mirrored RAID cache setup.
In the BIOS, I made sure to enable SATA Hot-plug support, it was disabled by default. Lucky catch on my part, because I definitely use hot-plugging. No idea what happens if you hot-plug on those SATA ports without hot-plug support enabled.
All of the Linux assigned device letters have changed. I know this doesn't matter a lick, but I had worked hard with my previous setup to have them increment from sdb->sdv in line with the Unraid slot numbers. It's my OCD kicking in, now the order is changed and reversed and random. I don't think this is correctable either - I guess the controller card is just wired differently. The best I could do is sequential in reverse order.
Since these H310's are in IT mode, I expected boot up to be quicker. It's not, still the long slow process of loading the card's ROM.
All-in-all, not too bad. I'll reach out to The Art of Server where I bought the H310's and ask about those brackets.
President, Chameleon Consulting LLC
Author, Chameleon MediaCenter
Author, Chameleon MediaCenter
Re: What is Unraid and how to build an Unraid media server
Great news, glad it went smoothly for you.
I had the opposite experience with my backup. The Synology died on me with a blue light when I tried to reformat the 12x4TB Hitachi. Spent most of the evening and this morning trying to debug this, I’ve given up for now. It only seems to work when there is no disk. As you as you add a disk, format the array and reboot, it hangs forever. It tried everything I could think of, no way around that. I’ll never know if decommissioning it caused this, or if I dodged a bullet and managed to migrate everything to Unraid before that happened. If the latter, obviously chuffed with my preemptive strike….
I had the opposite experience with my backup. The Synology died on me with a blue light when I tried to reformat the 12x4TB Hitachi. Spent most of the evening and this morning trying to debug this, I’ve given up for now. It only seems to work when there is no disk. As you as you add a disk, format the array and reboot, it hangs forever. It tried everything I could think of, no way around that. I’ll never know if decommissioning it caused this, or if I dodged a bullet and managed to migrate everything to Unraid before that happened. If the latter, obviously chuffed with my preemptive strike….