CMC Forums

Great to hear that so far, so good!

Thanks, Paul for all the info. My 2 Parity drives are ironwolf 8 tb drives. I have other seagate drives but they are not ironwolf. I'm going to try this during the weekend

Jamie

Got a few tips to share. I use Dynamix Auto Fan Control, and when I was troubleshooting my recent issues on 6.8.3, I uninstalled it thinking it was the problem (it wasn't).

Now that I'm getting my server back to normal, I reinstalled it, and took some time to reconfigure it. Years ago I had methodically discovered that the min PWM value was 135 for my server chassis fans, which gave me a min fan speed around 1100-1200. Not sure why, but it behaved differently today and I found I could go down to 125 and get a min fan speed of 1000 RPM, nice.

Now, if I go lower the fans don't go slower. Sure, I could set it to 0 and the fans would still spin at 1000 RPM, but that actually causes a secondary issue. The plugin will gracefully ramp your fan speed from your minimum PWM to 100%, and this ramp is most accurate if you correctly set your min PWM to the highest value right before the fans start to ramp up. So correctly tuned, the plugin ramps my fan speeds from 125 (about 42%) to 255 (100%).

If I set my min PWM to 0, then the ramping from 0 to 125 would do absolutely nothing, so I actually lose the lower half of the ramp steps, and I end up with bigger increments all the way through. So tip 1 is simply to properly set your min PWM.

Note the Detect option doesn't always work, so I recommend trying different values manually. Each time you hit Apply, give it a few minutes to update, and monitor your average fan speed for movement. Then repeat with a new value. With a bit of effort, you can find the absolute minimum PWM for your fan. That's tip #2, test manually and don't trust the Detect option.

Tip #3 is that the min fan speed might actually be too low for your preferences. In this case, simply increase the min PWM to a level that gives you the default cooling you think you need. This is doubly important if your min PWM actually shuts off the fans, as that might choke the airflow through your system. So sometimes you need more airflow to keep everything happy.

And finally is my new discovery that lead me here to post these tips. After configuring the fans, and altering the fan speeds of a few fans I didn't intend to touch, I decided to reboot. After Unraid came back up, my case fans stayed at their default speeds. I waited for over 20 minutes, and they never idled down. I also have the interval set to 1 minute, and looking in the log I couldn't find any log entries for the autofan except for where it got installed on boot. So I went to the Fan Auto Control config page, made a value tweak to enable the Apply button, and hit Apply. The secret here is that 'Apply' kills and restarts the autofan process. But when I did this, nothing was killed (indicating it wasn't running) and autofan was started. Within a minute I could tell the autofan process was running, as my fan speeds immediately dropped down.

So tip #4 is that the Dynamix Fan Auto Control may not start automatically on 6.9.2, so after a fresh boot, you may need to kick it off. It looks like you can do it manually with this command (which I grabbed from the log):

Jamie wrote: Wed Mar 23, 2022 6:56 pm Thanks, Paul for all the info. My 2 Parity drives are ironwolf 8 tb drives. I have other seagate drives but they are not ironwolf. I'm going to try this during the weekend

Jamie

I wouldn't pay too much attention to the label, as none of mine are IronWolf. Instead, pay attention to the model #'s. For example, mine are models ST8000NM0055 and ST8000VN0022, both of which have other users reporting issues on that Unraid support thread. Also, so far, I've only heard of issues with 8TB and 10TB drives, but not sure if that's just circumstantial.

My 2 parity drives are ST8000VN0022s. I guess I will not buy any more of these drives even though they were well priced

I'm happy to report my Unraid server is now 100% functional again and running better than ever.

My parity check complete in record time. Nice.

I'm seeing lower idle power consumption than before (most likely due to the returned case fans spinning 20% slower), with 115w idle power consumption. And in this case, Idle means: running Unraid, running a security camera recording Docker, spinning a HDD 24/7 to record security footage on it, running a Docker to provide DLNA service for recording TV on my network, and running a Win10 VM for My Movies with My Movies running. So Idle is a relative term, and it's basically like having 4 devices in one, each one consuming a miserly 30w. Opinions vary, but in my mind this is pretty sweet for a 23 drive, 88TB server.

Best yet, I was just updating my collection in My Movies, and for some reason it's faster than I remember. Especially when I viewed my Academy Awards Movie Collection, which typically takes 5-10 minutes to open, it opened in like a minute. Nothing changed as far as I know. Same hardware, same VM I shut down a month ago. Same My Movies version. But noticeably faster. Icing on the cake!

So it seems that Unassigned Devices was my problem all along, and one of those constant updates (near daily, WTF) somehow broke something really bad on 6.8.3. A simple upgrade to 6.9.2 solved all my woes, which was made possible by disabling EPC on the Seagate drives.

This experience has changed my perspective on Unraid versions. For years I've fallen into the habit of sticking on a working version when the currently available Unraid upgrades sounded a bit too risky for my liking. I'd simply wait months or years until finally the kinks were worked out before taking the leap. I was doing this yet again on 6.8.3 due to the Seagate issue. What's changed is that now a plugin I was depending upon, Unassigned Devices, through a constant barrage of updates managed to essentially crash my entire server.

I was under the impressions that developers set a minimum Unraid version and that if you weren't on a compatible version, the plugin wouldn't update. While this is true, it requires the developer to set this limit, and even more importantly to realize there is a limit that needs to be set. I was a bit too comfortable with letting plugins update.

From now on, if I decide to stay put on an Unraid version, I need to also stay put on the plugins. All too often these plugins are updated to be compatible with the latest Unraid version (sometimes still in beta), and I just learned the hard way how disastrous that can turn out. Hopefully this was a once in a lifetime issue, but the Unraid landscape has changed in the 15 years I've been playing with it, and I need to adapt to the new topography.

Great to hear all went well!

Thanks for providing your power use data.

I've made some changes my side as I wasn't happy with the power use I had on my main server. To remind you, here is what it was:

Manni wrote: Mon Jan 24, 2022 8:03 am i3770K at 3.5Ghz (I could use it at stock speed if it makes a significant difference)
16GB RAM
GPU: nVidia 7600GS (unfortunately I need it to boot, there is no iGPU on the MB)
All system files and VM on 1TB Samsung EVO840 SSD (cache)
12x4TB Hitachi HDDs in the array itself

Off/S3 Sleep (with WOL): 10-15W
During boot/power up: 200-210W
Idle after boot (all drives active, no activity except maybe the VM starting): 180W
Read check (all disks active): 215-225W (might be a bit more during a write, but they're unlikely to be all writing at the same time).
All drives asleep (except cache due to VM idle activity, CPU 7-10% load, with spikes up to 25% load): 98-115W

I've replaced the motherboard because I wanted to be able to use my 9305-24i at the max speed, and my old MB was only PCI 2.0. Also, it didn't have an iGPU, which meant I had to use a d-GPU, which cost some power.

So I got a P8Z77-V-LX with 1080p HDMi 1.4 iGPU and a single PCI 3.0 slot, which allows me to get the full 7.8GB/s bandwidth (hence 333MB/s if all 24 disks are used). I like that because if I get larger drives at some point, they will likely be able to do a bit more than 166MB/s, which was the limit with PCI 2.0 (all drives used). Some of my drives (WD Red Pro) do around 250MB/s, and larger drivers would likely go up to 280MB/s.

By the way I highly recommend the Speed Disk docker, which can benchmark the speed of individual disks and of the controller, and also detects if the current link is optimal or not. For example, the link with PCI 2.0 was 4GB/s, now I get the full 7.8GB/s:
Anyway, I now have a single LSI adapter in my main server, and here is the power use I get after these changes:

i3770K at 3.5Ghz (I've enabled EPU in the Asus BIOS)
16GB RAM
GPU: MB iGPU
All system files and VM on 1TB Samsung EVO840 SSD (cache)
12x6TB WD 6TB

Off/S3 Sleep (with WOL): 24W I've tried everything to get this down, impossible, but I'll live with it.
During boot/power up: 100-120W
Idle after boot (all drives active, no activity except maybe the VM starting): 115W
All drives asleep (except cache due to VM idle activity, CPU 7-10% load, with spikes up to 25% load): 75W
Read check (all disks active): 135-140W.

So a very significant saving (around 40%), except on standby/power off (+60%).

I haven't measured my other server as I'm in the process of replacing the 2500K by a 3770K (to save 14W and get more power so I can use a VM if needed as a backup) and swapping MBs (the one I was using had the power/disk LEDs on when powered off and doesn't have any fans detected in Unraid, and has a wrong temp for the CPU, so I'll use the one from the main server that I've just replaced as it works fine). The power off / standby use is much lower, around 7.5W, so that's good as it will be mostly in standby.

That 75w idle is enviable, very nice.

But that 25w S3 is baffling. If I had to guess, my first thought would be that the 9305-24i isn't properly sleeping. I think some motherboards make it configurable how PCIe devices are controlled in sleep modes, and maybe this MB is keeping it powered up. Or maybe there's another device on that MB that isn't properly sleeping. Sometimes these boards have 3rd party modules built-in that consume power.

Regardless, great numbers! They will climb as you add more drives, as even when in standby each drive trickles power. 10 more drives might add another 5-20 watts to your idle, but your S3 sleep should stay constant.

What size power supply are you using? One thing to remember is that peak power supply load comes when all drives spin up at the same time. For a brief moment, my server hits well over 400w during drive spin-up. I always wanted to use one of those highly efficient nano-PSU's, but they can't handle drive spin-up with this many drives.

Yes, I should try to take the LSI adapter out and see if I get the same number, it is weird indeed.

You could probably use your nano-PSU if you define spin-up groups in your disk settings, that way all the drives won't spin up at the same time:
Easy to try with your current PSU and measure your peak load.

I have an almost new Corsair SF750 in my main server (I had bought it for my E-GPU because it was silent, but I never really used it), and currently I have an old Antec 550 in the second server, but I'm about to replace it with a more recent and more efficient Seasonic 700W that I have lying around. All of them have enough power to handle the load with 24 disks, but the Antec 550 is borderline at full load as it's really old and probably no more than 70% efficient, while the Seasonic is an 80 Plus. [EDIT: actually I checked and my old Antec PowerTrio 550 has "up to 85%" efficiency, so I might just leave it in and see if it handles the load as the number of drives grow up].

Forgot to say I also have all the disks on two different rails from the PSU, so that the load is split (3 molex connectors / 12 disks per cable).

Manni wrote: Fri Mar 25, 2022 4:24 pm You could probably use your nano-PSU if you define spin-up groups in your disk settings, that way all the drives won't spin up at the same time:

From my understanding, that's not what spin-up groups do. Spin-up groups ensure that when 1 drive in a group spins up, all drives in that group spin up. Some have a need for this, but I never found a use for it.

What is needed is a staggered spin-up option, which doesn't exist. When I power on my server, the controller does a staggered spin-up. But when Unraid is running and all drives are asleep, starting a parity check immediately spins up all drives at the same time. Worst case is when the CPU is already heavily loaded from VM activity and a parity check starts.

6-8 years ago I engaged in a discussion on this on the Unraid forums, around the time spin-up groups were introduced. We asked for staggered, but never got it. I could go with a significantly smaller and more efficient PSU if Unraid had staggered spin-up. Seems like such a simple thing to program too.