What is Unraid and how to build an Unraid media server

[Manni]

Thanks a lot, I have most of these plug-ins installed already (as I read the whole thread) but there are a few I’ll give a try.

Thanks also for the disk settings, I’ll look into it and into the scheduler as well.

[Jamie]

Hi Manni,

When I was migrating from my Drobos to unraid, I tried to use Krusader but I noticed that it populated all the folder and sub folder names before actually migrating the files. Unraid stopped and said that my drive shares ran out of disc space because the file sharing on unraid was set to fill up and also the folder and subfolder names already were created on the unraid discs so unraid would not move to the next drive. I had to use windows file explorer as a traffic cop between my Drobos and unraid to migrate my data because it populated the folder and sub folder names as it went along. During my research I found other users with the same issue. I tried dolphin too but had the same issue with it as well.

Have you experienced this problem in your testing and were you able to work around it?

[Manni]

Hi Jamie,

I haven’t experienced this issue because I read the whole thread before posting, so I learnt from your experience I tried to use Krusader in “batches” but I noticed that its performance was atrocious, so I decided to use a Windows client instead.

I don’t understand why my data transfer windows performance (60MB/s) is now half of my direct windows to unraid performance (112MB/s). I thought it was the laptop or the fact that the first disk was getting full, but I now have the same performance drop when I copy from my QNAP to the Unraid server, irrespective of where the data is being written on the disk.

I was surprised when I did my first performance tests to have no performance drops when going through a Windows client, but maybe I did the test wrong. The only thing I changed in the meantime was swapping the Adaptec board from Raid to HBA. Maybe I’ll swap it back, but it looks more like a network performance bottleneck than a disk performance bottleneck.

Anyway, as I thanked Paul for his detailed explanations, thank you for starting the thread and for sharing your experience. I’m a lot more familiar with building my own rigs (I’ve done this for more than 30 years and I used to work in IT, so that part was easy for me), but unraid was completely new to me so I learnt a lot from your discoveries as you went along and that saved me a lot of time.

EDIT: my overnight copy froze at 25% (while copying to disk 2). The speed had slowed down to a trickle and the Adaptec board was sounding its error, even though I switch it to HBA and disabled the error in its bios. I wasn't able to stop the array, or to reboot/shutdown, so I had to power off. This is worrying, it's the second time this happens. I blamed the first one on the Adaptec card being set to RAID, or to Krusader, but if I get the same issue in HBA mode maybe it's not compatible. Any recommendation for 16+ ports SATA boards?

EDIT2: It looks like the Adaptec board doesn't save the settings and goes back to its defaults. When I checked them after the forced power off, the mode was back to RAID. I tested and the performance isn't better in RAID, so I'll put it back to HBA. I also noticed that often the drives controlled by the Adaptec kept spinning even when not in use, while the drives controlled by the motherboard were spinned down as expected. Maybe using different controllers is causing issues. I'm expecting more SAS to SATA cables (currently the Adaptec controls only 8 disks as I only managed to get 2 sets of cables (4-port each). I'll do more tests when I control all the disks with the Adaptec board.

EDIT3: Apparently the read check is the monthly read check. For some reason it was set to daily (that's the only task in the scheduler). I disabled it, so hopefully it won't start tonight. By the way I'm using the latest 6.9.2 UnraidOS, so there might be differences with Paul's install.

EDIT4: I'm thinking about sending the Adaptec back for an LSI SAS 9305-24i, which seems to be working out of the box. More expensive, but more ports so if it works I'll be able to use it in the upcoming 24-bay server.

[Pauven]

Hi Manni,

When I was migrating from my Drobos to unraid, I tried to use Krusader but I noticed that it populated all the folder and sub folder names before actually migrating the files. Unraid stopped and said that my drive shares ran out of disc space because the file sharing on unraid was set to fill up and also the folder and subfolder names already were created on the unraid discs so unraid would not move to the next drive. I had to use windows file explorer as a traffic cop between my Drobos and unraid to migrate my data because it populated the folder and sub folder names as it went along. During my research I found other users with the same issue. I tried dolphin too but had the same issue with it as well.

Have you experienced this problem in your testing and were you able to work around it?

Jamie, I thought we had resolved this by using Midnight Commander. From page 14 of this thread:

You might be able to use midnight Commander.

Midnight Commander comes with Unraid, it is a Linux tool. From the command line you enter 'mc' to start it. You would need to use Unassigned Devices to mount your FlexRAID share as a local disk to allow Midnight Commander to reach it.

Going from memory, I think Midnight Commander works like Krusader/Dolphin, but I'm not sure it's been a long while.

Well... I've never used MC... And until a moment ago... I never heard of it.

I doesn't look to difficult... so I started copying my BluRay folder on my first FlexRAID drive to the my BluRay share...

We may have a winner... So far it appears to be working to my purpose... The first folder (movie) in the BluRay folder is 12 Monkeys... And that is the only folder (plus it's subfolders) that is being copied to the BluRay share. So it appears it creates directories & copies one-by-one.

I'll play with it more tomorrow... I work nights and have to get ready for work... As long as I can figure out how to select more than one folder to be copied... This just might work.

Fingers crossed.
John

[Pauven]

I don’t understand why my data transfer windows performance (60MB/s) is now half of my direct windows to unraid performance (112MB/s). I thought it was the laptop or the fact that the first disk was getting full, but I now have the same performance drop when I copy from my QNAP to the Unraid server, irrespective of where the data is being written on the disk.

The max real-world sustained throughput on a gigabit connection is 116 MB/s (123 MB/s if using Jumbo frames). While gigabit ethernet is full-duplex and is technically capable of sustained max throughput bidirectionally, most consumer grade hardware and software are under-provisioned, and can only maintain max throughput in a single direction, read or write, not bidirectionally.

This means when you use your Windows PC as a traffic cop you'll realize only about 50% throughput, since the PC has to receive the data and send the data, and is managing the read and write requests in addition to just passing data. That 60 MB/s throughput is to be expected and about the best possible in this scenario.

The solution is to get rid of the man (err... PC) in the middle.

In Unraid, use Unassigned Devices to mount your QNAP shares. Then use Midnight Commander (which is the only tool we've identified that copies the folders in the correct sequence) to perform the copies.

The above is slightly oversimplified, and I'm setting you up for potential failure, so here's some more instructions to make sure you achieve success.

First, install Unassigned Devices and Unassigned Devices Plus. Then on the MAIN screen, click ADD REMOTE SMB/NFS SHARE, and create a share to your QNAP server. Take note of the resulting share's path.

Midnight Commander (MC) is a command line (console) application. Do NOT be tempted to simply open a console window and run it directly, as your copies will be taking days/weeks, and if you loose your console window connection then you lose control.

You will need to use Screen, which handily allows you to create multiple virtual console windows (so you can run different tasks in each), but more importantly allows you to re-attach to any session should you become disconnected (or want to connect from another PC). Screen is part of the Nerd Pack tool suite. Install Nerd Pack, then go to Settings > Nerd Pack and install Screen.

Next, MC does support using a mouse from its DOS era GUI, but only if you use the right terminal software. From Windows, use Putty. Open a Putty session, connect to your Unraid server, then type screen to start a Screen session, then type mc to launch Midnight Commander. Next, test your mouse. You should be able to click around the MC window as if it was a real Windows app.

Next, navigate on the left side to your target Unraid folder/share (I'm assuming you know how to do this already), and on the right navigate to your QNAP share. Select the folder you want to copy on the right side, then click the Copy button. That's it.

Note: when you click Copy, what is copied is what is currently highlighted. So always highlight your source folder on the right as the last step. If you click a folder on the left last, then you will be copying left to right, instead of right to left.

If your Putty session gets closed, you can reattach by doing the following. Run Putty, connect to your Unraid server, and type screen -ls to list your previous sessions. If there is only a single session, you can type screen -x to reattach to it. Otherwise if you had opened more than one screen session, then you have to specify the ID, i.e. screen -x 29878.pts-6.myuserid.

Screen is an awesome tool, and way too powerful for me to detail how to fully use it here. So definitely google for a linux screen tutorial to learn how to create and manage sessions.

If you follow all of the directions above, then you should be able to easily copy at near max throughput. I agree the Krusader and Dolphin are the wrong tool for this job, don't let that detract from you trying MC.

[Manni]

Hi Paul,

Thanks for the detailed reply, especially regarding using MC, which I'll definitely try.

I am 100% aware of the possible performance limitations when using the "man in the middle", I even refered to it in my post. But I was convinced I had tested 112MB/s transfer speed using my Ryzen, and I was correct

For some reason, I can now copy at full gigabit speed again from the Qnap to the Unraid server using my windows Ryzen:

Full speed.JPG (62.25 KiB) Viewed 1479 times

This is what I got in my initial tests. But for some reasons, it doesn't hold and sometimes hobbles the speed to 60MB/s.

Even when it does, the connection is gigabit, but seems to be limited to half duplex. No idea what triggers this.

Edit: As you can see, the performance also fluctuates between 80MB/s and 112MB/s, for no obvious reason (nothing else running on either server):

Speed fluctuation.JPG (60.53 KiB) Viewed 1468 times

Now the plot thickens following the edits to my last post:

I've tried to resume the copy from the last batch that hung, and it hang on exactly the same file, in the same folder. So I tried to copy the folder to my local SSD on the Ryzen to rule out an issue in the source data, and it copied right away. So for now I've skipped this folder, and as you can see in the screenshot above, the copy is going happily at full speed. Once it's done, I'll try to copy the folder that caused the issue again.

I've lost my confidence in the Adaptec card and I've requested a return and refund. I'm going to buy an LSI, most likely a 9201-16i HBA if I can find one. Which LSI adapter are you using? How many ports? You've probably listed it somewhere but I think they are 8 port ones, and I want 16 ports minimal on a single slot (I need to have a GPU in this build).

I've found this one: https://www.amazon.co.uk/gp/product/B00 ... G8OB&psc=1 so I need to find if it's an HBA model or if I need to change the f/w [edit: seems to be HBA]. I also want to make sure as these boards are quite old that they will handle large HDDs, up to 20TB. Otherwise I need to buy something else.

Edit: Found a better case: https://www.servercase.co.uk/shop/serve ... ujEALw_wcB

[Manni]

I found an explanation (or rather a trigger) for the performance drop mentioned above.

If I pause a transfer in Windows, when I resume the transfer drops from 112 MB/s to 60MB/s.

If I cancel and start again, the speed is still limited to 60 MB/S.

I have to stop the array to get the 112 MB/s back, which is why it was resolved by the reboot earlier today.

No idea what would cause this drop in performance, but at least I know what can trigger it and resolve it.

[Pauven]

I'm not sure I would recommend this, but here's the controller I'm using: HighPoint 2760A, which is a 24-port SAS/SATA controller. It matches up with my case, so it greatly simplified cabling (just 6 cables for 24 drives). It came in 3 flavors, 8-port, 16-port and the 24-port I got. My 24-port model actually has 3 8-port controllers on it, and a PCI plex/hub thingy, so the OS sees it as 3 separate cards connected via a hub. The secret here is that each of these controllers is basically the same as the AOC-SASLP-MV8.

The reason I don't recommend this path (besides $$$ cost and high power consumption), is that there is a known issue with certain Seagate drives on LSI controllers, and these affect the MV8 which means my controller is affected as well. The issue is that certain 8TB/10TB Ironwold ST8000 drives will fail out of your array if you're on Unraid 6.9.x. That's why I'm still running 6.8.3, as I have two affected drives. It was really scary to have 2 drive fake fail at the same time, I thought I was going to lose data. There are some workarounds, but I haven't tried them yet.

One thing you need to consider for controller cards is if you're going the SAS route (requires SAS bays, probably a whole SAS compliant server case) or just individual SATA cables (sounds like what you're already doing). I think Jamie ended up taking the SATA route. You can easily mix and match SATA ports on your motherboard with extra SATA ports on a controller card, minimizing how many cards you need. If you get a SAS controller (like the LSI 16-port model you linked) you will need to use breakout cables (1 SAS to 4 SATA) unless you have a bays/backplanes with SAS connectors.

I just double-checked the controller card I recommended to Jamie, and it is the popular LSI 9201-8i. So Jamie, heads up if you have any 8TB Seagate Ironwolf type drives. These cards are cheap (if you can still find them), I think around $60-$70 each, but I haven't look in a long time. Since they are LSI, you may want to avoid if you have any Seagate 8TB or 10TB drives.

The LSI00244 that you linked looks like a newer version of my card, just in 16-port flavor. Again, LSI (hard to avoid, it seems) so good chance it will have the dreaded Seagate 8TB drive issue. That price is high for a HBA adapter, but still way cheaper than my 24-port card (I think I paid over $700 for it). If you have the spare slots, multiple 8-port cards like the LSI 9201-8i make more sense cost wise. There's also 16-port versions like the LSI 9201-16i. (HAHA, I just read that the LSI00244 is the 9201-16i!!!, so I've been talking about the same card all along - the price you found seems high).

The main thing you want to be cognizant of (and I haven't done the math for you) is that you don't overtax the available PCIe bandwidth. You need to double-check how many lanes and what PCIe generation a card uses (and your motherboard provides), and divide that by the number of SATA ports. For example, if you have 16 large & fast SATA II drives, they might peak out around 200 MB/s, so you would need about 3.2 TB/s of PCIe connectivity. PCIe 2.0 x4 only provides about 2 GB/s, so you would have a bottleneck. So a PCIe 2.0 x4 card is suitable for up to 8 SATA, but not 16. You need either PCIe 2.0 x8, or PCIe 3.0 x4, to have no bottlenecks with a 16-port card.

I'm sure you know this, but I'll state it anyway just in case. You need to verify the PCIe of the card and the individual motherboard slot. Some cards and/or slots are physically x16 but only wired for x8 or x4. On some motherboards, a x16 slot reverts to x4 if you enable certain other components that consume some of those lanes.

That LSI00244 / 9201-16i is a PCI-e 2.0 x8, so it has sufficient bandwidth if you put it in the right MB slot. I know I kept stating that the LSI cards have a known issue with 8TB Seagate drives, yet I've provided no alternatives. I guess I'm out of the loop, haven't looked at hardware much in years. There's some subsections on the Unraid forum where this stuff is discussed a lot, so for latest recommendations I'd go there: https://forums.unraid.net/forum/33-storage-devices-and-controllers/

There's also some general wisdom to avoid SATA port multipliers (massive bottlenecks) and also avoid any Marvel controllers. Somehow my card shows up as a Marvel controller (Fix Common Problems always warns me) yet I also have the Seagate 8TB issue (maybe it's not just limited to LSI cards, what do I know...).

If you're not confused by now, then you haven't been paying attention. I know I've certainly succeeded in confusing myself...

I've also found this case that looks great for up to 18-20 drives, so I could have 16 drives on the SATA board and a few SSDs using the MB SATA ports. https://www.amazon.co.uk/gp/product/B08 ... UTF8&psc=1

That's a pretty case, but I don't think I would recommend it. Swapping drives is likely to be a pain. I think you need a solution using hot-swap bays. You can build your own, but I don't think this case can accommodate those bays.

It's definitely your choice. I know availability and price are factors, and that Fractal is available and cheap. But I personally feel the tradeoff is too great. This goes back to the case discussions that Jamie and I had earlier in this thread. My perspective hasn't changed. Most new cases these days do not have lots of external 5 1/4 drive bays, which is what you need for installing those hot-swap 3.5" bays. So you either need to be buying an older case design, or buying something purpose built for lots of external 3.5" drives.

Unfortunately, it seems like some of the manufacturers have left this space over the past few years, so your path will be harder.

[Pauven]

I found an explanation (or rather a trigger) for the performance drop mentioned above.

If I pause a transfer in Windows, when I resume the transfer drops from 112 MB/s to 60MB/s.

If I cancel and start again, the speed is still limited to 60 MB/S.

I have to stop the array to get the 112 MB/s back, which is why it was resolved by the reboot earlier today.

No idea what would cause this drop in performance, but at least I know what can trigger it and resolve it.

Are you using a cache drive? It might be getting in the mix and influencing the results.

It might also be that your transfers start off fast, but ultimately always settle down to 60 MB/s, even after a reboot.

[Manni]

Nope, no cache and as you can see, we're talking about very long transfers (1TB and more). Also the speed goes back up to 112MB/s which wouldn't make sense if it was the cache or the unraid disk performance going down as the drive fills up.

Anyway, I'm setting up MC but right now I've lost everything as when I tried to define a static IP I lost connection... I'll let you know how I get on with MC once I've resolved this.