What is Unraid and how to build an Unraid media server

[Pauven]

You can use MC with a keyboard if you need to. Arrow keys and Tab and stuff if I recall correctly. Easier with a mouse, but not a dealbreaker. Same requirement though, make sure you are highlighting what you want to copy before you start the copy process, and it copies to the other side.

[Manni]

Did a test on a single folder, speed is a bit below the Windows client (when it works at full speed), 105 MB/s vs 112 MB/s, but it's a lot more convenient if I can do it in one go as it can just run 24/7 until it's done.

MC Speed.JPG (189.31 KiB) Viewed 1409 times

I'll start the copy of the whole share and I'll let you know how it goes.

EDIT: yes, as I said, no mouse isn't a dealbreaker, I'm used to the DOS interface, I used to use MC on DOS

[Manni]

Sorry, I don't see how I can select (highlight) multiple folders in the source window before starting the copy. I've already copied some titles, so I have to start the copy in the middle of the list. Not sure why the mouse isn't working. I looked at the help, can't see anything about that.

EDIT: it came back to me, it's INSert to select a file/folder.

I've started a copy of a bunch of titles, and the speed is 110MB/s so close to the max.

I'll keep you posted.

[Pauven]

I used to use MC on DOS

What?! I didn't realize it existed on DOS. I never even heard of it until a few years ago on Unraid. Figured it was a Linux thing. I'm a C=64 baby, and DOS was my playground for years after that. I still use batch files even today for some enterprise work. Command lines are awesome. A lot of CMC wouldn't work without command lines...

105 MB/s vs 112 MB/s

Well it should be 105 MB/s consistent, instead of fluctuation from 120 down to 60, so maybe it still wins the race.

Sorry, I don't see how I can select (highlight) multiple folders in the source window before starting the copy. I've already copied some titles, so I have to start the copy in the middle of the list. Not sure why the mouse isn't working. I looked at the help, can't see anything about that.

I don't think you can multi-select. You would have to select the parent. Sux that you're halfway down the list, I don't have a good answer for you, unless you move data around on your source system first, then copy the entire parent folder. If you name the parent folder for your target parent folder, it should work pretty well. I.e. copying /DVDs to /DVDs.

[Pauven]

EDIT: it came back to me, it's INSert to select a file/folder.

Sweet, now I'm learning too!

[Manni]

Performance seems stable at 110 MB/s, so all is good for now.

This is what I was using in DOS, Norton Commander. I guess these Linux tools took their inspiration from there...

https://en.wikipedia.org/wiki/Norton_Commander. I think I might have used the Windows version when Windows was a bit less productive to use, in the early days.

I went ZX-81 > TRS-80 > Apple II > IBM PC.

Apart from a few Mac laptops, I'm a PC guy.

I used to develop software in Assembler 6502 (for the Apple II) and 8080 (for PC), so batch files are super evolved

Then I moved to Forth (which was great, I even programmed a Forth interpreter for my Apple II, burnt into an EEPROM). I developed a few apps in dbase/foxbase, then I stopped developing, more than 30 years ago, until I came back to less hardcore development with Wordpress a few years ago to design and maintain our website(s). Apart from a few lines of PHP code here and there when I'm stuck, I wouldn't call myself a developer as I mostly get plugins to work together, but it helps to have that (outdated) background when I work with other developers...

Anyway, that's my geeky credentials.

EDIT: found this case, 16-bay but 4U, with two 5.25 bay: https://www.servercase.co.uk/shop/serve ... s-sc-4316/

[Pauven]

I remember Norton Commander. Yeah, the resemblance is uncanny.

Those are definitely some geeky creds. You fit right in around here, good company. I think most CMC users are pretty geeky - I think you have to be to want to rip discs and catalog them in My Movies.

My dad used to work on programming mainframes back in the day - you know, the kind of mainframe that fills a room and you can walk inside of it, and programs were stored on punch cards (don't drop it!!!). He loved it and definitely passed his love for all things computing onto me. He had me programming and doing calculus by the time I was in 3rd grade. Seemed normal to me, but in hindsight I see that has really given me some advantages in the IT world - I live, breath and speak technology whereas many of my cohorts simply learnt it in college.

While I strive to make CMC easy enough to use for non-techies, I love having techies around to chat with. My jobs would be easier if everyone was a techie... but then again, I'd probably be out of work.

I never have gotten into anything like assembler. I bought a book once, and the only thing I understood was the line at the bottom of each page that read "page no.".

While I certainly do a lot of development for career and CMC, I consider myself more of an unexpected developer and not a super strong coder. I feel like I have to work twice as hard at some of this stuff, because I've never been formally taught. Oddly, right now I'm working with a crew of "offshore .NET developers" (if you know what I mean), and each day we have a 1-hour meeting to review the development progress and they get to ask me questions and I provide all the direction and guidance - the odd part is that I feel like the strongest developer of the bunch, and I don't do much of anything in .NET myself. It seems like to me that these guys (and gals) should be running circles around me, but each day I'm explaining basic stuff like SQL query joins. It seems crazy to me that in this group of coders, somehow, I'm the most techy.

EDIT: found this case, 16-bay but 4U, with two 5.25 bay: https://www.servercase.co.uk/shop/serve ... s-sc-4316/

Oh, that's nice for a 16-bay! Good price too. It looks a little tight for wiring behind the drive bays, but at least the fans pull out easy enough. Full size fans should run quieter and cool better (but still loud, of course).

I like the SFF-8087 connectors too - you'll only need 4 cables, plus power.

For the two 5.25 bays, you could put optical in there, and that might come in handy for ripping. I keep my server in the basement, and since I only run a single My Movies instance inside the VM on that server, I have an USB Blu-ray drive connected via a USB over IP solution, and it get's mounted virtually inside my VM. This is handy for when I need to read disc ID's and chapters and such, otherwise I just rip from any PC straight to Unraid and just let My Movies import via folder monitoring.

While you could put a couple more 3.5" in those bays, another idea would be a 4x 2.5 hotswap bay (I think that's a thing). That would get you up to 24 drives, with 8 2.5". Not sure if it would be worth it, but nice to have that option.

[Manni]

The 4x2.5 hot swap is a good idea and it does exist (https://www.scan.co.uk/products/icy-doc ... or-525-bay) but that means buying 2.5 disks, and I'm not really into that. I have a few of them lying around from various laptops, but I wouldn't invest in 2.5" disks specifically for NAS use.

Of course what I might end up buying is this: https://www.servercase.co.uk/shop/serve ... e-sc-4336/

It's big though, but I guess I could start with one 16-port controller and add another one later down the line, but I would have to replace the MB by then as I only have two PCI-E slots in my salvaged mobos and both need a GPU. Even if it costs twice as much as a 16-bay, I would save if I don't have to build another complete server. My main worry is to have only 2 parity drives for 20+ data disks, but I guess as it's not RAID, even f you lose more than two disks in a catastrophic event, you don't lose eveything.

In fact I could time this expansion for when I replace my Ryzen monster mobo in a couple of years, and move it to the NAS (which would be completely overdoing it, but at least I'd have a lot of resources for VMs/docker).

Anyway, the MC copy is chugging along, it looks like it's going to handle the handover to disk 3 gracefully (it only copies one folder at a time, so doesn't behave like Krusader). Once this is confirmed, I'll shove it in the cupboard and I'll launch the copy of the rest of the share tonight. Hopefully it will be working on its own over the holiday to complete the copy, and by the time I get the LSI adapter it will be done. I just hope the drives will be identified the same way by the Adaptec and the LSI, otherwise I'll have to re-create the array, hopefully without losing anything as long as I put the drives in the right order.

[Pauven]

The 4x2.5 hot swap is a good idea and it does exist (https://www.scan.co.uk/products/icy-doc ... or-525-bay) but that means buying 2.5 disks, and I'm not really into that. I have a few of them lying around from various laptops, but I wouldn't invest in 2.5" disks specifically for NAS use.

Well, if the day Unraid truly supports SSD, it could come in handy for those. You can also have SSD's outside your array in Unassigned Devices. I'm with you, I don't like 2.5" spinners, but solid state is nice.

Of course what I might end up buying is this: https://www.servercase.co.uk/shop/serve ... e-sc-4336/

I'm not as crazy about that one. 36 drives is overkill for Unraid, they still have some artificial limits, though supposedly a feature coming in a future update is support for multiple arrays (which would also mean more than 2-parity). I also don't like the 2U MB area, or the 60mm screamer fans.

If I needed that many storage bays, I'd probably probably be looking at something from 45 Drives, as I like their novel HD storage solution. Pricey, but I think you can get Unraid tailored HBA versions at a lower price (but I could be wrong).

but I guess as it's not RAID, even f you lose more than two disks in a catastrophic event, you don't lose eveything

The hardest part is figuring out what you lost so you can replace/restore it... It can be handy to save a directory listing per physical disk so that you have a way of knowing what's where when it goes bye-bye.

In fact I could time this expansion for when I replace my Ryzen monster mobo in a couple of years, and move it to the NAS (which would be completely overdoing it, but at least I'd have a lot of resources for VMs/docker).

That's my plan exactly. When I built my current workstation, I planned from day one to convert it to Unraid duty. But the 1800X has done so well, and my current TR2950X is still good enough, I've been running this a year longer than I anticipated. The only real downside to my 1800X is I'm limited to 64GB. I would love double the memory for more VM's.

Anyway, the MC copy is chugging along, it looks like it's going to handle the handover to disk 3 gracefully (it only copies one folder at a time, so doesn't behave like Krusader). Once this is confirmed, I'll shove it in the cupboard and I'll launch the copy of the rest of the share tonight. Hopefully it will be working on its own over the holiday to complete the copy, and by the time I get the LSI adapter it will be done. I just hope the drives will be identified the same way by the Adaptec and the LSI, otherwise I'll have to re-create the array, hopefully without losing anything as long as I put the drives in the right order.

That's awesome, especially if you can just let it run while you enjoy the holidays.

From my understanding, Unraid controls the array based upon drive ID's, so even if you move the drives around or to different controllers, the ID should stay the same. The drive letters will change, but you'll learn that matters little. You should still take some screenshots of your array config before swapping anything, might make the process easier if it doesn't go smoothly. Otherwise, I've never swapped a controller before, so I don't know if there's any gotcha's I'm unaware of. It makes me a little nervous that you're copying now and doing a major hardware change later, but worst case I guess you can always just copy again so not a biggie.

[Manni]

Yes, I use a windows utility to print the content of my backup disks before I store them and I definitely plan to print the content of each unraid disk once they are full if there is a way to do so from the unraid OS.

Well spotted re the 2U mainboard area for the 36-bay case, I didn't clock that (or the 60mm fans), it would be a nightmare indeed. Looks like the 4U 16-bay with the two 5.25" bays is the best solution for now, unless I find a 24-bay one, but they don't seem to have any except some at extortionate price that I'll never pay for a NAS enclosure.

Re the disks when you replace the controller, the issue is that they might have a different disk ID, which means that unraid won't even recognise the array. this can happen when you update the f/w of your controller, and the controller reports IDs differently. In that case, you have to recreate the array, and provided you get the disks in the correct order, all should still be there.

I know there is a risk I might have to restart from scratch, but I'm seeing this as a way to stress-test the array, so it's not a big deal if I have to start again when I change the controller, I'll still have the QNAP disks (and I plan to keep them for a while, until I'm 100% sure I can rely on the Unraid server.

For now, MC has moved gracefully to disk 3 and has started to fill it up. It's still moving data at a nice and steady 110MB/s, so it's definitely more performant than the Windows client, as the speed would fluctuate between 112MB/s and 80MB/s, and even down to 60MB/s if I paused the copy process.

Thanks again for all your help, hopefully the rest of the process will take place while I do other things now, I'll update you when I get the new controller/cables in a week or so, unless I find the time to explore the VM rabbit hole, in which case you might hear from me (assuming it's still ok to bother you with this, don't hesitate to let me know if it's not).