Decorum for the Forum:
  • Be nice. If you want to be mean, try Reddit.
  • No Piracy. If you want to be a thief, there are dark places on the internet dedicated to that.
  • No Cracking. Discussions on AnyDVD, DeUHD, DVDFab, UHDKeys and similar tools are not permitted here.
  • No Spamming. If you want to make a buck, work smarter... somewhere else.
  • No Adult Content. Half the internet is dedicated to adult content. This half isn't.

Privacy Policy: Click Here to Review (updated September 30, 2020)

What is Unraid and how to build an Unraid media server

Show off your HTPC builds, NAS Servers, and any other hardware. Great place to ask for hardware help too.
Manni
Posts: 593
Joined: Wed May 22, 2019 5:27 am

Re: What is Unraid and how to build an Unraid media server

Post by Manni » Sat Mar 26, 2022 5:39 pm

Jamie wrote: Sat Mar 26, 2022 5:24 pm Hi Paul,

What do you recommend for a preclear app that is compatible with 6.9.2? I just upgraded and I believe that the old preclear app is no longer compatible. Everything is running fine by the way.
Not Paul, but I recommended earlier Binhex-PreClear, as it works with unassigned disks as a plugin, and also as a docker if you prefer something more involved. It works fine with 6.9.2.

User avatar
Pauven
Posts: 2777
Joined: Tue Dec 26, 2017 10:28 pm
Location: Atlanta, GA, USA
Contact:

Re: What is Unraid and how to build an Unraid media server

Post by Pauven » Sun Mar 27, 2022 6:09 pm

Manni wrote: Sat Mar 26, 2022 12:44 pm Not sure if there is a way to get just power from molex or SATA and use the PWM header on the mobo to control the speed. I thought this was what the Noctua would do, but it doesn't seem to be the case.
My understanding of PWM fans is that there are 4 wires: Power, Ground, Sensor, and PWM Control. You should be able to run the Power/Ground to a SATA Molex for plenty of power, and Sensor and PWM control to the motherboard for speed control.

My case came that way, with Molex headers for power, and then a PWM sensor/control header to plug into the motherboard.

Jamie wrote: Sat Mar 26, 2022 5:24 pm What do you recommend for a preclear app that is compatible with 6.9.2?
No idea. I don't know what changed in 6.9.2 to make the existing options incompatible, and I don't know what options remain. Everything I have is already precleared, and I won't be buying any more drives for a while yet.
President, Chameleon Consulting LLC
Author, Chameleon MediaCenter

Jamie
Posts: 942
Joined: Wed Dec 27, 2017 11:26 pm

Re: What is Unraid and how to build an Unraid media server

Post by Jamie » Sun Mar 27, 2022 7:32 pm

Manni wrote: Sat Mar 26, 2022 5:39 pm
Jamie wrote: Sat Mar 26, 2022 5:24 pm Hi Paul,

What do you recommend for a preclear app that is compatible with 6.9.2? I just upgraded and I believe that the old preclear app is no longer compatible. Everything is running fine by the way.
Not Paul, but I recommended earlier Binhex-PreClear, as it works with unassigned disks as a plugin, and also as a docker if you prefer something more involved. It works fine with 6.9.2.
Thanks Manni,

Binhex Preclear seems to do everything The old preclear did. I noticed that It hasn't been updated since July 2021. Unassigned.devices.preclear is vague about exactly what it does but was last updated March 2022. I guess I will use binhex until they say it is unsupported.

Manni
Posts: 593
Joined: Wed May 22, 2019 5:27 am

Re: What is Unraid and how to build an Unraid media server

Post by Manni » Sun Mar 27, 2022 8:28 pm

Jamie wrote: Sun Mar 27, 2022 7:32 pm
Manni wrote: Sat Mar 26, 2022 5:39 pm
Jamie wrote: Sat Mar 26, 2022 5:24 pm Hi Paul,

What do you recommend for a preclear app that is compatible with 6.9.2? I just upgraded and I believe that the old preclear app is no longer compatible. Everything is running fine by the way.
Not Paul, but I recommended earlier Binhex-PreClear, as it works with unassigned disks as a plugin, and also as a docker if you prefer something more involved. It works fine with 6.9.2.
Thanks Manni,

Binhex Preclear seems to do everything The old preclear did. I noticed that It hasn't been updated since July 2021. Unassigned.devices.preclear is vague about exactly what it does but was last updated March 2022. I guess I will use binhex until they say it is unsupported.
Sorry for the confusion, binhex preclear is the docker, that’s the one I installed first, but then I found the unassigned devices preclear, which is much easier to use as it’s a plugin that adds a “pre clear” icon to the unassigned device, similar to the way the previous plugin/script worked. That’s the one I use now, as I find it more convenient and it works fine, in a very similar way to the old plugin/script. In fact I’ve uninstalled the docker.

Jamie
Posts: 942
Joined: Wed Dec 27, 2017 11:26 pm

Re: What is Unraid and how to build an Unraid media server

Post by Jamie » Sun Mar 27, 2022 9:01 pm

Thanks Manni. I will try unassigned.devices.preclear.

User avatar
Pauven
Posts: 2777
Joined: Tue Dec 26, 2017 10:28 pm
Location: Atlanta, GA, USA
Contact:

Re: What is Unraid and how to build an Unraid media server

Post by Pauven » Mon Mar 28, 2022 5:34 pm

Do you know if the U.D.preclear use Joe L's script?

That's what I liked about the previous preclear plugin, it allowed me to optionally use Joe L's script which I personally felt was superior.
President, Chameleon Consulting LLC
Author, Chameleon MediaCenter

Manni
Posts: 593
Joined: Wed May 22, 2019 5:27 am

Re: What is Unraid and how to build an Unraid media server

Post by Manni » Mon Mar 28, 2022 5:59 pm

It doesn't, but it uses an "advanced" script that works fine and gives you the option of skipping pre/post read and specify how many cycles.

The problem with Joe's script is that it wasn't maintained, which ended up causing lots of issues.

Frankly, I haven't looked back.

A few tips learnt during my mega reorganisation, in no specific order:

1) I started using the plugin "open Files", which is really useful to see if any file might be preventing driver or system standby. It complements well "Dynamix active streams". Sometimes the plugin "Dynamix Cache Directories" can prevent drive/system standby and I'm in the process of debugging/optimising. The 'Find' process of cache directories is often hogging the CPU and causing lots of activity, as it keeps reading the directories in order to keep them in the cache. I'm thinking of excluding the user share, so the system files directories are always cached, but not the media/data files. I think it's not optimised when user data becomes too much, though maybe changing some of the parameters might help.

2) I didn't use the "unbalance" plugin to reorganise my data because I quickly realised that as long as the source and destination drives were different, I could start concurrent instances of Screen/MC and move things around ten times faster as I could initiate 4 or 5 simultaneous transfers (array disk to array disk, NAS to array disk, external disk to array disk). Instead of 170-250 MB/s, I could get up to 1TB/s of cumulative transfer rate. I just had to think carefully about which share went well and allocate the correct disks. I did most of the moves disk to disk to control previsely the read/write pairs, and also a few share to share (when I needed MC to jump to the next disk when full). Obviously never mix disk/shares.

3) I had to replace the mobo on server A (to get PCI-E 3.0 and max the bandwidth of my 9305-24i) and after doing that, the VM wouldn't start. It's because it couldn't find a USB device that wasn't present on the new MB (or maybe the UPS cable plugged in a different port). I managed to edit the xml file using CA config editor, identify the missing USB device and mark it as optional, but if you don't want to experience this I recommend using the Libvirt Hotplug USB plugin, which once installed allows you to attach/detach USB devices before moving the VM, replacing the Mobo or unplugging a USB device if it's not coming back.

4) I swapped the 9305-24i and the 16/8 port controllers between the two servers, that didn't make any difference (as expected). All drives were detected fine. This is just to reassure others. All the controllers were LSI or compatible (Avago or Broadcom), which probably helped.

5) The plugin "parity check tuning" allows to fine-tune a parity check or rebuild over a few days, without having to wait for 6.10.0. I haven't tried it yet as I've suspended all my parity checks, but I plan to.

I'm almost finished moving things around and filling up server A (the last share transfer has started and should be done in a couple of days as it's 15TB). When that's done, I'll take one drive out (unneeded space) by starting a new config, and I'll re-enable parity. I'll then free up some drives from the backup and will finish server B, which should take a couple more weeks as I'll restore from the Synology DS2411, the Thecus N5200 and many individual HDDs. So far, so good!

Also I got rid of the fan controller plugin. The Noctua doesn't seem to pick-up on the PWM data from the mobo and also I decided that I wanted the fans to be always on, so that the controllers don't fry. I think they are designed to get a fair bit of airflow, and detecting HDDs temp (or MB temp) doesn't mean that the controllers don't need more ventilation, as they don't have a sensor. I set the speed to a fixed 2300-2500rpm, instead of the ear-breaking 3700rpm default, and it's fine like this.

User avatar
Pauven
Posts: 2777
Joined: Tue Dec 26, 2017 10:28 pm
Location: Atlanta, GA, USA
Contact:

Re: What is Unraid and how to build an Unraid media server

Post by Pauven » Mon Mar 28, 2022 8:10 pm

Manni wrote: Mon Mar 28, 2022 5:59 pm The problem with Joe's script is that it wasn't maintained, which ended up causing lots of issues.
I hadn't heard about any issues. Seems like such a basic tool (stress test drives by writing/reading data, then write a pre-clear signature), that unless Unraid changed the Cleared signature then maintenance would not be required. I do know that Joe L. had both refused to write a modern plugin for his script, and refused to let others build upon his work, which really sucked. While I know that 6.9.2 broke compatibility with the pre-clear plugin, I have no idea what broke, and if Joe's script still works - I used to run it manually without a GUI years ago.

Though since his script is pretty simple in concept, it seems plausible that the new U.D.preclear replicates the same process. I do recall that the original plugin (by gjardin?) had new logic that would make a pre-clear much quicker, which many users didn't like (myself included) because we wanted a stress test, not just the quickest possible pre-clear signature. Joe L's slower (by deisgn) process was more stressfull on a drive, to better weed out the weak ones. Hopefully U.D.preclear follows Joe's methodology, or allows for this. I honestly don't remember now what was different in jgardin's plugin, but I think it was more fundamentally optimized than skipping a pre-read and post-read. I think Joe's script had some randomness in the writes, or even read/writes, to introduce more stress during the pre-clear, moving the head more, which is why it took longer.

Manni wrote: Mon Mar 28, 2022 5:59 pm A few tips learnt during my mega reorganisation, in no specific order:
Awesome tips, thanks for sharing. I already do 1 & 2. Never used Unbalance myself. I've yet to do a MB swap with a VM, so #3 is in my future, I'll keep this in mind. #4 is what I expected, but more data points are always welcome. Unraid is smart to use drive ID's, and not controllers/ports/drive assignments. For #5, I've personally had a problem with pausing and resuming a parity check on 6.8.x, so the last thing I want to do is let a 3rd party plugin get involved. I'll wait for 6.10, and even then I'll be wary.


Glad your migrations are going well and ahead of schedule, that's awesome. Isn't the flexibility of Unraid awesome?!
President, Chameleon Consulting LLC
Author, Chameleon MediaCenter

User avatar
Pauven
Posts: 2777
Joined: Tue Dec 26, 2017 10:28 pm
Location: Atlanta, GA, USA
Contact:

Re: What is Unraid and how to build an Unraid media server

Post by Pauven » Fri Apr 01, 2022 10:52 am

Well, looks like I was a bit too ambitious in my early proclamations of success.

image.png
image.png (61.65 KiB) Viewed 6551 times

I guess Unraid decided to make an April's Fool of me. It's been running almost 9 hours, and hasn't moved at all in the past couple of hours. It obviously got stuck right at the beginning.

This is just when I first experienced these issues on 6.9.x last year. Back then, the entire GUI would hang in addition to the parity check. This time, the GUI is still mostly responsive, but doesn't show current data. It indicates that drives are all spun up, but drive temps are cool. CPU and MB temps seems frozen, along with fan speeds. The Dashboard doesn't show any info at all. Perhaps only the GUI is not working, not sure if the parity check is still running or not. Need to do some research...
President, Chameleon Consulting LLC
Author, Chameleon MediaCenter

Manni
Posts: 593
Joined: Wed May 22, 2019 5:27 am

Re: What is Unraid and how to build an Unraid media server

Post by Manni » Fri Apr 01, 2022 12:56 pm

Sorry to hear about that. I had something similar last night, but not during parity sync (that works fine, always), during pre-clear of six disks. I've uninstalled the fine-tune pre-clear plugin, and I'm going to try again. That used to work fine, so not sure what happened, but I had no access at all to the box (GUI, putty) yet the disks being resynced were being written to according to the LEDs. I always ask for an update every 25% of each step (highly recommended) so I let it run for a while, but I didn't get any update so I hard powered off, powered back on, did a parity resync (no error) and now I'm about to try again, but I'll stop the array first so that if there is any crash I don't have to resync parity again.

It's not 6.9.x though, I've been on it from the beginning and never had such an issue, except when I forgot to add unassigned devices to the S3 sleep plugin to prevent it from going to sleep during a preclear, when there was no activity on the array.

Post Reply