Decorum for the Forum:
  • Be nice. If you want to be mean, try Reddit.
  • No Piracy. If you want to be a thief, there are dark places on the internet dedicated to that.
  • No Cracking. Discussions on AnyDVD, DeUHD, DVDFab, UHDKeys and similar tools are not permitted here.
  • No Spamming. If you want to make a buck, work smarter... somewhere else.
  • No Adult Content. Half the internet is dedicated to adult content. This half isn't.

Privacy Policy: Click Here to Review (updated September 30, 2020)

What is Unraid and how to build an Unraid media server

Show off your HTPC builds, NAS Servers, and any other hardware. Great place to ask for hardware help too.
User avatar
Pauven
Posts: 2777
Joined: Tue Dec 26, 2017 10:28 pm
Location: Atlanta, GA, USA
Contact:

Re: What is Unraid and how to build an Unraid media server

Post by Pauven » Sat Jan 01, 2022 4:38 pm

Happy New Year!

I've been taking some personal time over the holidays. Technically still doing that, but decided to drop in for a minute and see what's been going on.

Manni wrote: Thu Dec 30, 2021 4:54 pm The Unraid bug with Seagate Ironwolf drives is only with Unraid V6.9.x and only with LSI controllers. Also not all Seagate drives are affected, though those that are include 8TB models. Are yours Ironwolf, and are you using an LSI controller?
The issue affected my setup, but I'm running 8TB Seagate Exos (kissin' cousin to Iron Wolfs) and a Marvel based controller. I think the problem is both bigger and smaller than reported: bigger in that more drives and controllers are affected than you see in that thread, and smaller in that the overall impact of this issue seems to be a very small population of Unraid users.

In general, I would say to tread cautiously if you have any 8TB or 10TB Seagate drives, regardless of what's printed on the label or what controller you have them connected. My guess is that this is probably a drive firmware incompatibility with a Linux feature change, and that it has been mis-interpreted by the user base, and as such it could rear its head with any size drive assuming it has the firmware issue. I think this is why the problem is less widely reported than you would expect, since not every 8/10 TB drive was affected.

If you upgrade to v6.9.x, the problems should appear within a few days, quicker if you do a parity check. Essentially, you won't be able to make it through a parity check (or rebuild). False errors are reported, and Unraid takes the drives offline. The good news is that your data is not altered, so nothing is lost, but it is a pain to recover from this scenario. Essentially, I rolled back to 6.8, then told Unraid to trust my array was good and just adopt it as-is, and I was back to healthy. But it was a nerve-racking experience.

Jamie wrote: Fri Dec 31, 2021 7:26 pm I have been reading about the 6.9.x unraid seagate issue and it looks like unraid may not be fixed in newer versions. Looks like it will have to be fixed in an LSI driver update or the seagate fw.
That's my take on this too. I don't want to be stuck on 6.8, there's some great stuff coming in 6.10 and later. Whatever is causing the problem seems to be part of the newer Linux kernels, and doesn't really seem to be on Lime Tech's radar. There's always the possibility it gets resolved, but I think this is the new normal so I plan to try the fix myself. I've just been too busy to take on yet another project at the moment.

Manni wrote: Thu Dec 30, 2021 6:42 pm However, don't upgrade to 6.9.x without making the suggested changes to the drives f/w (they all have to be tweaked individually).
I'm conflicted on this. On the one hand, this seems like sage advice. But on the other, it is not common practice to tweak these settings, so in general I would say avoid this unless necessary.

The downside is that you have to upgrade to 6.9.x (or newer) just to see if the problem affects you, and then you get the fun of dealing with it. If you can upgrade to 6.9.x and complete a parity check, you're probably in the clear.

There's really not a win scenario here. You can either make questionable changes to your drives' configuration without knowing if they are needed, or you can upgrade and see if they are needed by waiting for the drives to get kicked offline. Manni's probably right, but I think you have to make your own judgement call about which path you prefer.

Just remember, having Unraid kick a drive offline does not mean you lost data, only that Unraid lost confidence in a drive. If it happens, don't panic, and definitely don't take any actions that could cause data to be wiped out. If you're unsure, just stop and ask for directions.

Also keep in mind that since we are applying the fix at the drive level, then any new/replacement drives need to be dealt with going forward, as you won't have the luxury of fixing them before upgrading to 6.9 or newer since you'll already be on it.

I think I've got 6 of these drives now. Two confirmed with the issue, two installed after reverting to 6.8.3, and two more pre-cleared but otherwise unused, installing soon. I had begun buying them because they were a great value - sometimes it's worth buying a better drive.

Manni wrote: Fri Dec 31, 2021 10:30 pm Interestingly enough, I’m currently running a couple 6TB I’m not sure about through a pre-clear, and weirdly one of them seems to hang the unraid, despite the fact that both went through the WD Diag write zero process without any problem (except that it took a bit longer than usual).
That's also one of the benefits of doing a pre-clear on your actual server, vs. doing one on a separate machine. It could be a bad drive, bad power, bad cabling, bad controller, or even some incompatible combination of parts. On my first server, I identified a couple drive bays that were flaky, and eventually learned to never put any drives in them. Never figured out the root problem, but the end result was randomly consistent.

Manni wrote: Fri Dec 31, 2021 10:30 pm Not very reassuring to have the whole server hang like this,
If you haven't already, you should take a few extra steps to help trap this issue. By default, logs are written to RAM, and are lost on reboot. You can configure them to write to your thumb drive. You can also stay logged into the console and monitor traffic on it. Typically, if there is a crash there are errors output on the console, and sometimes these errors don't make it into the log.

Manni wrote: Sat Dec 25, 2021 8:41 am Paul, when you have a moment, please could you sum up how you would suggest I set-up MyMovies, CCC and CMC, assuming MyMovies server in the Unraid VM and a single instance of CCC? I'd like to do it properly... It's not urgent of course.
I'm not sure if I know the best way, but I can share the steps I took.

I created a Win10 Pro VM that autostarts with Unraid. I installed My Movies and restored a DB copy from my old My Movies server. My Movies still accesses my Unraid shares as if it was on a different PC, no fancy local mappings, treat it like it really is a separate PC in another room.

Once that is running, then I just installed CCC & CME - for these I just copied those two sub folders from the zip file to C:\. This does mean I have to manually upgrade both of these, since I don't run CMC on this VM, but that's super easy to do. I didn't do anything special in configuring CCC or CME, other than maybe using localhost as the My Movies server name.

The only other thing I did special was using a USB over IP adaptor that allows me to have an external USB based UHD Blu-ray drive in my office that is connected to the My Movies VM over the network (since my server is far away from my office). If I didn't do this, then I would need to run a My Movies client on my office desktop in order to read disc titles/chapters. I know this sounds like overkill, but it works amazingly well. I can rip discs using this drive over my network at pretty much the same speed as ripping locally.

I'm not sure if I covered what you were after - everything I did is pretty much basic standard fare, nothing special.
President, Chameleon Consulting LLC
Author, Chameleon MediaCenter

Manni
Posts: 593
Joined: Wed May 22, 2019 5:27 am

Re: What is Unraid and how to build an Unraid media server

Post by Manni » Sat Jan 01, 2022 5:48 pm

Hi Paul,

Happy New Year!

I've done the same as you for MyMovies in the VM (except I simply did a sync from the MM server instead of restoring a backing, should be the same). Yes it access the shares on the network, even the "local" ones, so I reached the same conclusion.

Re CCC+CMC, that won't really work for me. First I will definitely not use the unraid server to rip disks. They will be in a hidden cupboard, with no easy access, so I'll keep ripping from my Ryzen HTPC.

Also CMC will be running from the HTPC too.

So I guess my question is do I just install one version of CCC on the Win10 VM on the unraid server, and then point to that one for each client running CMC? Or do you also need to install a copy of CCC per client, pointing to the Win10 VM install of MyMovies? That's where I'm unsure.

I also plan to run a MyMovies client on quite a few PCs, but I'm not sure MyMovies can revert an install from server+client to client only. Last time I tried it was still a server+client install, for some reason. Anyway I'll ask Brian's advice if that's the case.

User avatar
Pauven
Posts: 2777
Joined: Tue Dec 26, 2017 10:28 pm
Location: Atlanta, GA, USA
Contact:

Re: What is Unraid and how to build an Unraid media server

Post by Pauven » Sun Jan 02, 2022 8:39 pm

Yes, just the one copy of CCC running on the Win10 VM, that's all I have. Ditto with CME, which I use a few times a year to export my metadata as a backup in case My Movies ever stops working.

I think you said this VM would be on all the time (at least while you're home), so you would then point each CMC PC at the Win10 VM for CCC access.

CCC is the server to the CMC client. Where it gets confusing is that CCC is also the client to the My Movies server. So CCC is both a server and a client.

In my house, I only run a single My Movies install, and a single CCC install, both in the Win10 VM. For all collection management, I simply remote desktop into the Win10 VM and do it directly on the My Movies server, runs just as well as if I had a My Movies client on my local PC. That's also why I use my USB over IP solution to connect a Blu-ray drive in my office to the Win10 VM which is 2 floors away.
President, Chameleon Consulting LLC
Author, Chameleon MediaCenter

Manni
Posts: 593
Joined: Wed May 22, 2019 5:27 am

Re: What is Unraid and how to build an Unraid media server

Post by Manni » Mon Jan 03, 2022 3:29 am

Thanks, makes sense.

Do you have a link to your USB over IP solution?

Also have you found a way to get your VM to log into an account automatically on boot? Even with Autostart, I have to log in manually into an account with VMC before the VM can be accessed as a bridge for the Oppo. I used to do this with my HTPC but I can’t remember how, unless the account has no password which isn’t great for security purposes.

Last question, do you know how to move a VM? I’ve created it in the array for testing, but I’d like to move it the cache SSD as soon as I’ve enabled it, along with a few of the shares that I know will remain small.

Thanks!

User avatar
Pauven
Posts: 2777
Joined: Tue Dec 26, 2017 10:28 pm
Location: Atlanta, GA, USA
Contact:

Re: What is Unraid and how to build an Unraid media server

Post by Pauven » Mon Jan 03, 2022 4:48 pm

Manni wrote: Mon Jan 03, 2022 3:29 am Do you have a link to your USB over IP solution?
viewtopic.php?f=16&t=347

Manni wrote: Mon Jan 03, 2022 3:29 am Also have you found a way to get your VM to log into an account automatically on boot?
Why? I have a password on mine, and it does not login automatically. Does not cause any issues for My Movies or for CCC. My Movies runs as a background service. And CCC can be configured to run under an account, so it behaves like a background service that starts on boot, even though it really isn't a background service.

Do you have something else that requires the auto-logon? I do recall seeing a solution for this, probably related to My Movies since it requires a Windows account to have a password and users were complaining.
Manni wrote: Mon Jan 03, 2022 3:29 am Last question, do you know how to move a VM? I’ve created it in the array for testing, but I’d like to move it the cache SSD as soon as I’ve enabled it, along with a few of the shares that I know will remain small.
A VM is just some files in a directory. Should be easy to move. Easiest way is actually to change where your "domains" share is stored (set "Use cache" to Prefer):

image.png
image.png (82.51 KiB) Viewed 1521 times

That does assume you actually put your VM in your domains share (that's the default), and that in total all of your VM's are small enough to move the entire domains share to the cache drive. If you did something different, you would need to manually move the files, then update the VM to point at the new file location.

I'd also recommend making a backup of your domains/VM's into your array, for safe keeping.
President, Chameleon Consulting LLC
Author, Chameleon MediaCenter

Manni
Posts: 593
Joined: Wed May 22, 2019 5:27 am

Re: What is Unraid and how to build an Unraid media server

Post by Manni » Mon Jan 03, 2022 6:29 pm

Thanks for the USB over IP link, I replied there.

Re the VM, it looks straightforward enough, I'll try what you suggest. I know a VM is just a few files (I've been using Parallels on the Mac and VM Ware on the PC for years) but I didn't know if there was anything special with the unraid ones.

Re the password, yes I think it's because MyMovies requires a password for some features, though I remember that with my old HTPC I had managed to create a MyMovies account without a password that was able to login automatically and start CMC right away (it was in the startup folder), which was convenient. I just don't remember how I did it though...

Jamie
Posts: 942
Joined: Wed Dec 27, 2017 11:26 pm

Re: What is Unraid and how to build an Unraid media server

Post by Jamie » Fri Jan 07, 2022 9:27 pm

Hi Paul,

I have been having issues with the disk array autostarting on bootup. I have had this issue from the beginning, but now that I am beginning to set up a VM, I have decided that I need to fix it so that the array starts on bootup.

The array does startup manually with no issues.

I do have autostart disk setting set to "yes" and when I reboot it gets reset to "No" automatically on bootup. I have researched the issue but I am unable to fix it. I have made sure that I click apply before clicking done when I change the disk settings. I have gone into the disk settings again to make sure the setting that I selected "Yes" in this case sticks before rebooting.

There was a thread on the unraid forum that suggested that I should set the setting to "No" and reboot. Then set it to "Yes" and reboot again. I noticed on the second reboot that it booted very quickly, when it took nearly 7 minutes to boot before. I checked the system log and now it is very short and clean. Before it had an error on bootup but not any more. It still doesn't start the array unless I start it manually.

While I was typing this, I decided make sure the setting was "yes" and rebooted one more. The unraid boots within 2 minutes now but the setting was "No" again and the array did not start. Here's my log.

Jan 7 21:18:12 Tower kernel: md5: running, size: 5860522532 blocks
Jan 7 21:18:12 Tower kernel: md6: running, size: 5860522532 blocks
Jan 7 21:18:12 Tower kernel: md7: running, size: 3907018532 blocks
Jan 7 21:18:12 Tower kernel: md8: running, size: 3907018532 blocks
Jan 7 21:18:12 Tower kernel: md9: running, size: 3907018532 blocks
Jan 7 21:18:12 Tower kernel: md10: running, size: 3907018532 blocks
Jan 7 21:18:12 Tower kernel: md11: running, size: 3907018532 blocks
Jan 7 21:18:12 Tower kernel: md12: running, size: 3907018532 blocks
Jan 7 21:18:12 Tower kernel: md13: running, size: 3907018532 blocks
Jan 7 21:18:12 Tower kernel: md14: running, size: 3907018532 blocks
Jan 7 21:18:12 Tower kernel: md15: running, size: 3907018532 blocks
Jan 7 21:18:12 Tower kernel: md16: running, size: 3907018532 blocks
Jan 7 21:18:12 Tower kernel: md17: running, size: 3907018532 blocks
Jan 7 21:18:12 Tower emhttpd: shcmd (23): udevadm settle
Jan 7 21:18:12 Tower emhttpd: error: put_disk_settings, 3546: Is a directory (21): fopen: /boot/config/disk.cfg
Jan 7 21:18:13 Tower emhttpd: Autostart disabled
Jan 7 21:18:13 Tower kernel: mdcmd (55): stop
Jan 7 21:18:13 Tower kernel: md1: stopping
Jan 7 21:18:13 Tower kernel: md2: stopping
Jan 7 21:18:13 Tower kernel: md3: stopping
Jan 7 21:18:13 Tower kernel: md4: stopping
Jan 7 21:18:13 Tower kernel: md5: stopping
Jan 7 21:18:13 Tower kernel: md6: stopping
Jan 7 21:18:13 Tower kernel: md7: stopping
Jan 7 21:18:13 Tower kernel: md8: stopping
Jan 7 21:18:13 Tower kernel: md9: stopping
Jan 7 21:18:13 Tower kernel: md10: stopping
Jan 7 21:18:13 Tower kernel: md11: stopping
Jan 7 21:18:13 Tower kernel: md12: stopping
Jan 7 21:18:13 Tower kernel: md13: stopping
Jan 7 21:18:13 Tower kernel: md14: stopping
Jan 7 21:18:13 Tower kernel: md15: stopping
Jan 7 21:18:13 Tower kernel: md16: stopping
Jan 7 21:18:13 Tower kernel: md17: stopping
Jan 7 21:18:13 Tower emhttpd: shcmd (43): /etc/rc.d/rc.php-fpm start
Jan 7 21:18:13 Tower root: Starting php-fpm done
Jan 7 21:18:13 Tower emhttpd: shcmd (44): /etc/rc.d/rc.nginx start

Any ideas?

Just an fyi. I shut down the array before I changed the setting and saved, then rebooted. Same issue and the log looks the same. Still now boots in 2 minutes rather than 7 minutes

Jamie

Manni
Posts: 593
Joined: Wed May 22, 2019 5:27 am

Re: What is Unraid and how to build an Unraid media server

Post by Manni » Sat Jan 08, 2022 4:44 am

Not Paul but I have the same issue :)

Waiting to see if it happens with the new controller to investigate.

I’ve sent the Adaptec back and I’m waiting for two LSI (a 16-port and a 24-port) but I won’t receive them until week after next.

User avatar
Pauven
Posts: 2777
Joined: Tue Dec 26, 2017 10:28 pm
Location: Atlanta, GA, USA
Contact:

Re: What is Unraid and how to build an Unraid media server

Post by Pauven » Sat Jan 08, 2022 6:38 pm

Even though I have my VM's and Dockers autostart with the array, I do not have my array autostart. I've had it set to manual start for many years now.

My thinking is this - there is never a time that my server starts up without my intervention. If power fails and UPS is drained, it stays off after power is restored. Since I'm always involved, I am able to review all drives' basic array health before starting the array.

So my setup may have the same issue for all I know, but since I don't use this feature I'm in the dark.
Jamie wrote: Fri Jan 07, 2022 9:27 pm Jan 7 21:18:12 Tower emhttpd: error: put_disk_settings, 3546: Is a directory (21): fopen: /boot/config/disk.cfg
I did a quick google, and this error message seems to be saying that /boot/config/disk.cfg is a directory, which cannot be opened because it is not a file. I have no idea how or why this would be.

My suggestion is to post a support ticket asking for help on the Unraid forums.
President, Chameleon Consulting LLC
Author, Chameleon MediaCenter

Jamie
Posts: 942
Joined: Wed Dec 27, 2017 11:26 pm

Re: What is Unraid and how to build an Unraid media server

Post by Jamie » Sat Jan 08, 2022 7:55 pm

Thanks for your answer Paul. I will keep using Manual for now, but I will create a ticket as you suggested

Manni you can reply to this post as well if you desire. I honor all suggestions.

Do you use autostart for your VM? Does the VM autostart on array start? Do you use autostart for the CCC and mymovies machine?

Edit: I read in previous messages that you do an autostart on the VM. Does it autostart on array startup?

I am trying to create a vm right now but I am having problems with VNC starting for the install on the create button click. I have a few questions that your answers may help me move forward.

1. Does unraid have a default VNC or do I have to install one? Since the videos recommended VNC, I thought it would be the default. My system just goes back to the vms tab instead of going into a VNC window as the video shows.

2. The videos say you should only make your vdisk 70 GB at the most. I want to have a 2 tb of space for my VM. I need room for my surveillance as well as other software. What is your recommendation? how much space do you recommend for the vdisk? Is the vdisk like the c: drive or is it like a dedicated windows OS partition?

Edit: On other VMs I've done, I have had to allocate the VM for the space that I would use for the VM.

3. The videos and documentation recommend you use an IDE bus for the OS Install CDRom Bus and the VirtIO Driver CDRom Bus. I was wondering whether it should be sata instead?

4. Also where do you get the windows 10 pro iso download? I used the windows 10 creation tool to create the iso. Is this the right thing to do?

5. Can I restart the vm from the vm tab now and will the install continue, or should I remove the VM and start over with the vm creation? What is the best way to remove the VM? Just the vm or the disks too? I think in my case i should do both.

6. Can I create the VM remotely or should I go to my machine and use the monitor there at least for the install? If I could remotely create a windows VM with my laptop it would help me a lot. I have to stand at my unraid machine and I have a bad back.

7. Also, My monitor is not the best? It is a VESA monitor, so should I be using QXL for the vnc video driver? I am using integrated graphics too. I think this is why my vm won't start right now because I am using a VESA monitor and QXL. I could probably swap it out for a LCD monitor and try that monitor instead. I think I will swap them out, But I would like to know your viewpoint on when to use QXL, or Cirrus?
Last edited by Jamie on Sun Jan 09, 2022 1:45 pm, edited 4 times in total.

Post Reply