Former overclocker playing around with mining

[Ericsson]

Hi guys.
Long time no see .. eh!?

I joined overclockers.com when I was discovering the PC and I had great time here. Overclocked my AMD durons, folded with team 32 a bit and so forth, that was long time ago! Then life took me elsewhere! Now I am having minor health issues (nothing life threatening) that leaves me with some time on my hands and in the past days I have been playing with "currency" mining zcash. Currently running 2x 1080 cards and one 1060. Each in old boxes I had lying around. I have a lot of catching up to do, both regarding hardware and software. The last windows operating system I used was win 2000 and I barely touched the later XP. I guess that leaves me as Linux guy since I have only used Linux since then. However I am no expert on Linux - far from it actually.

I want you to meet my first "mining rig" - skvetta. (means splashy)
I found here abandoned in trash container - half full of snow. She was frozen to the bottom of the container so I had to pry her loose with a steel pipe to get her out. I put here in the shower to melt off the snow and ice and she left a big pool of water (hence the name). Removed memory and cards and used compressed air to dry her out and blow away decade of dust and lint. In all her glory she is sporting Intel core 2 duo 6400, 2 gigs of ram and mini atx board with single pci-e slot. After quick test she indicated she was ready to race so I slapped into here one brand new 1080 card I managed to cry out from a local retailer for close to $1K!

She is currently running Linux Mint desktop version, with the l NV 384 drivers and cuda. Humming away "creating" something called "Zcash" hashing @ 500 sol/s per second using ewfb miner software. https://bitcointalk.org/index.php?topic=1707546.0

She is a beauty isn't she!

I find it amusing calling this "mining rig" since I actually sometimes work on different kind mining rigs as a mechanic. This is a bit different!
More later..

 

[Ericsson]

Output from nvidia driver with the miner working - as you can see there is no overclocking going on.

0) Power is limited to 160W, "out of the box" the card was running at 195W - reducing the power seems to have little effect on hash rate.
1) GPU Utilization is 100%
2) Increasing GPU clock increases hash rate - but this card freezes up right away just by adding 300 to the clock. Perhaps I could add more power and speed up the fans but for something I plan to live with and run for months on end it's hardly worth it for only few percent increase in hash rate.
3) Memory Utilization is 86% and memory usage is very low. I can increase memory clocks quite a bit but there seems to be little advantage in hash rate doing so. ( still experimenting)

xxxx@skvetta ~ $ nvidia-smi -q

==============NVSMI LOG==============

Timestamp                           : Fri Feb 16 04:42:05 2018
Driver Version                      : 384.111

Attached GPUs                       : 1
GPU 00000000:04:00.0
    Product Name                    : GeForce GTX 1080
    Product Brand                   : GeForce
    Display Mode                    : Disabled
    Display Active                  : Disabled
    Persistence Mode                : Enabled
    Accounting Mode                 : Disabled
    Accounting Mode Buffer Size     : 1920
    Driver Model
        Current                     : N/A
        Pending                     : N/A
    Serial Number                   : N/A
    GPU UUID                        : GPU-c49b9dc0-8691-0da4-03f7-69cd002d1e6b
    Minor Number                    : 0
    VBIOS Version                   : 86.04.17.00.E3
    MultiGPU Board                  : No
    Board ID                        : 0x400
    GPU Part Number                 : N/A
    Inforom Version
        Image Version               : G001.0000.01.03
        OEM Object                  : 1.1
        ECC Object                  : N/A
        Power Management Object     : N/A
    GPU Operation Mode
        Current                     : N/A
        Pending                     : N/A
    GPU Virtualization Mode
        Virtualization mode         : None
    PCI
        Bus                         : 0x04
        Device                      : 0x00
        Domain                      : 0x0000
        Device Id                   : 0x1B8010DE
        Bus Id                      : 00000000:04:00.0
        Sub System Id               : 0x85AA1043
        GPU Link Info
            PCIe Generation
                Max                 : 1
                Current             : 1
            Link Width
                Max                 : 16x
                Current             : 16x
        Bridge Chip
            Type                    : N/A
            Firmware                : N/A
        Replays since reset         : 0
        Tx Throughput               : 6000 KB/s
        Rx Throughput               : 24000 KB/s
    Fan Speed                       : 45 %
    Performance State               : P2
    Clocks Throttle Reasons
        Idle                        : Not Active
        Applications Clocks Setting : Not Active
        SW Power Cap                : Active
        HW Slowdown                 : Not Active
        Sync Boost                  : Not Active
        SW Thermal Slowdown         : Not Active
    FB Memory Usage
        Total                       : 8112 MiB
        Used                        : 632 MiB
        Free                        : 7480 MiB
    BAR1 Memory Usage
        Total                       : 256 MiB
        Used                        : 4 MiB
        Free                        : 252 MiB
    Compute Mode                    : Default
    Utilization
        Gpu                         : 100 %
        Memory                      : 86 %
        Encoder                     : 0 %
        Decoder                     : 0 %
    Encoder Stats
        Active Sessions             : 0
        Average FPS                 : 0
        Average Latency             : 0
    Ecc Mode
        Current                     : N/A
        Pending                     : N/A
    ECC Errors
        Volatile
            Single Bit            
                Device Memory       : N/A
                Register File       : N/A
                L1 Cache            : N/A
                L2 Cache            : N/A
                Texture Memory      : N/A
                Texture Shared      : N/A
                CBU                 : N/A
                Total               : N/A
            Double Bit            
                Device Memory       : N/A
                Register File       : N/A
                L1 Cache            : N/A
                L2 Cache            : N/A
                Texture Memory      : N/A
                Texture Shared      : N/A
                CBU                 : N/A
                Total               : N/A
        Aggregate
            Single Bit            
                Device Memory       : N/A
                Register File       : N/A
                L1 Cache            : N/A
                L2 Cache            : N/A
                Texture Memory      : N/A
                Texture Shared      : N/A
                CBU                 : N/A
                Total               : N/A
            Double Bit            
                Device Memory       : N/A
                Register File       : N/A
                L1 Cache            : N/A
                L2 Cache            : N/A
                Texture Memory      : N/A
                Texture Shared      : N/A
                CBU                 : N/A
                Total               : N/A
    Retired Pages
        Single Bit ECC              : N/A
        Double Bit ECC              : N/A
        Pending                     : N/A
    Temperature
        GPU Current Temp            : 64 C
        GPU Shutdown Temp           : 99 C
        GPU Slowdown Temp           : 96 C
        GPU Max Operating Temp      : N/A
        Memory Current Temp         : N/A
        Memory Max Operating Temp   : N/A
    Power Readings
        Power Management            : Supported
        Power Draw                  : 158.01 W
        Power Limit                 : 162.00 W
        Default Power Limit         : 198.00 W
        Enforced Power Limit        : 162.00 W
        Min Power Limit             : 90.00 W
        Max Power Limit             : 238.00 W
    Clocks
        Graphics                    : 1797 MHz
        SM                          : 1797 MHz
        Memory                      : 4513 MHz
        Video                       : 1607 MHz
    Applications Clocks
        Graphics                    : N/A
        Memory                      : N/A
    Default Applications Clocks
        Graphics                    : N/A
        Memory                      : N/A
    Max Clocks
        Graphics                    : 1974 MHz
        SM                          : 1974 MHz
        Memory                      : 5005 MHz
        Video                       : 1708 MHz
    Max Customer Boost Clocks
        Graphics                    : N/A
    Clock Policy
        Auto Boost                  : N/A
        Auto Boost Default          : N/A
    Processes
        Process ID                  : 8356
            Type                    : C
            Name                    : ./miner
            Used GPU Memory         : 621 MiB

 

[ZL1]

hehe, welcome back !!!

love the story, I too prefer not letting old hard go to waste and Im using a case from 1997 for one of my miners but I need to throw a word or two of caution
- please please tell me you are not using that old PSU for that beauty of a GPU, risk is immense, beside it being previously flooded, generic PSUs aren't great at keeping voltage straight (high ripple on the rails) and are a very big risk to hungry GPUs
- also, using that mobo may also risk your very expensive GPU

recommend getting a good quality PSU, like seasonic or corsair or superflower
recommend getting a good 2-6x gpu mobo, i can give you options on this too

running such a setup would also raise your efficiency since you will have one box running all 3 cards, less wasted watts for extras and better efficiency as well since I would recommend you get a gold/platinum PSU

you are in EU right ? me too, so I can point some online stores that are good and ship EU-wide
estimated cost for components would be 600eu, but think of the advantages and safety first at the end of the day

 

[Ericsson]

Thanks for the warm welcome ZL1
I had the feeling this place would be good for honest and fun exchange of views regarding our hobby and I feel welcome already!

Your words of caution are much appreciated, I just didn't want to clutter the story with details - but yes I did replace the power supply with one I had on stock. Still it's an old bugger but has some heft to it and 700w rating - and has never seen snow face to capacitors. As to the motherboard - I am going to take my chances.. for now.

I am in Iceland where everything if expensive but the water and electricity.

 

[ZL1]

Thanks for the warm welcome ZL1
I had the feeling this place would be good for honest and fun exchange of views regarding our hobby and I feel welcome already!

Your words of caution are much appreciated, I just didn't want to clutter the story with details - but yes I did replace the power supply with one I had on stock. Still it's an old bugger but has some heft to it and 700w rating - and has never seen snow face to capacitors. As to the motherboard - I am going to take my chances.. for now.

I am in Iceland where everything if expensive but the water and electricity.

Figured it was Iceland well its even better that way, no VAT: https://www.computeruniverse.net/en/ change country to Iceland to see no VAT prices
you get taxed on import right ? whats the limit ? you can buy in batches if the limit is low, shipping isn't expensive and they ship air, I buy from them all the time

I can give you a discount referral code as well, takes a 5euro off, but why not - FWBUDW9

you need the following:
https://www.computeruniverse.net/en/products/90683861/intel-celeron-g3930.asp
https://www.computeruniverse.net/en/products/90679642/msi-z270-a-pro.asp
https://www.computeruniverse.net/en/products/90671838/wd-green-wds120g1g0b-m-2-2280.asp
https://www.computeruniverse.net/en/products/90660881/crucial-4gb-ddr4-ct4g4dfs824a.asp
https://www.computeruniverse.net/products/90692512/seasonic-prime-80-plus-platinum.asp

if you're wondering how come I came up with the list so quick .. its simple, i copied you my previous order )

and if you wanna go with 3-6x GPUs you will need these

this is just something to consider, but if you get passionate about this mining stuff, you will eventually wanna go with a multi-gpu rig, trust me

 

[Ericsson]

Many thanks ZL1.
I have to pay local 24% vat on everything over ~ 10 EUR
And I am not in serious buying mode - yet.

Current plan is to utilize the old gear I have laying around. So I am only shelling out cash for the video cards. Then wait and see how things progress. I am running 2x 1080 and 2x1060 cards in four old boxes.

Right now I am checking mild overclocking some of the cards trying to figure out if it's worth it in hash rates. The miner software reports sols/sec and then I see average numbers from the pool. I guess it takes long time averages to see any real difference from the pool though. And honestly I feel the 1060 card I am currently playing with doest best at standard clock speed. That doesn't add up... So more testing I guess.

Also doing comparison between pools, nanopool vs flypool. So far they seem equal to me but again, only longer time intervals will tell the whole story.

Here is my little horse race on flypool: https://zcash.flypool.org/miners/t1Qq3xX9JLbqfjWmTGbXZYLx18uSnDnxV5o

Let me know if posting the link is a bad idea.

 

[Ericsson]

First casualty of my mining exercise - NorthQ 500w psu in 7 year old intel i5 box. Running 1060 card @110w. I was running the zcash wallet after few days off (full node if I understand correctly) and there was considerable CPU activity combined with the 110w to the card. I saw a flash and the 10A breaker flipped. Cant plug the PSU in -it just trips the breaker.

Come to think of it - this psu is probably 15 years old.
I am not going to cheap out on new psu if I decide to stick with mining.

So fer serious mining going forward, best buy PSU brands?
High efficiency , silent and capable of years of high amps?

 

[ZL1]

ouch, is the card ok ?? thats why I warned about using old/generic PSUs

you need a psu with good voltage regulation as I mentioned earlier, take seasonic, corsair, superflower, like this for ex https://www.computeruniverse.net/en/products/90697052/super-flower-leadex-ii-80-plus-gold.asp

you get charged 24% for importing, but the germans take 20 off at sale outside the EU, so you are paying only 4% more than germans would, its a good deal id say at the end of the day



the pool reports hashrate based on number of shares, so thats really the effective hashrate over there

 

[Ericsson]

Indeed you did warn me. What I forgot is how time flies when you get older. Only after the PSU failed did it occur to me how old it was. The power ratings were up to the task, but long years of use -not so much. I have been checking your list on "computeruniverse" and it looks like you have done your homework. This PSU is great bang for the buck. I import lots of stuff and I will buy from these guys, good prices and fair shipping cost.

As for the dramatic exit of my NorthQ PSU, there was no hardware damage. However I had to go into Bios of "skvetta" pictured above and save it again to get her running. She didn't like the shock when the NorthQ in "sleipnir" left to the afterlife. They were running on the same power strip. "sleipnir's" bios settings got scrambled up and I had boot problems but nothing serious. This is something I remember from my old overclocking days - bios is sensitive and can cause all kinds of errors if power shocks occur or if a box freezes up because of to high clock.

I did current measurements on my 3 currently mining boxes this afternoon to figure out how much power they actually use and to estimate PSU efficiency. I have one new 735w "Thunder V2" psu rated 80 plus (or whatever). It seems to be doing considerably better than the older units I am running. If the idea is to run these cards for years better PSU's will pay for them self's in the long run. Not to mention they might skip the fireworks - something totally worth it. If I am not to lazy I will swap the PSU's and repeat the power measurements to see the actual difference between the PSU's. The question I am dealing with is if it's worh it to shell out extra for "titanium" energy rating vs Platinum vs Gold in relation to energy cost. This boils down to how long will they be running. A year, 3 or 5?

 

[ZL1]

Indeed you did warn me. What I forgot is how time flies when you get older. Only after the PSU failed did it occur to me how old it was. The power ratings were up to the task, but long years of use -not so much. I have been checking your list on "computeruniverse" and it looks like you have done your homework. This PSU is great bang for the buck. I import lots of stuff and I will buy from these guys, good prices and fair shipping cost.

As for the dramatic exit of my NorthQ PSU, there was no hardware damage. However I had to go into Bios of "skvetta" pictured above and save it again to get her running. She didn't like the shock when the NorthQ in "sleipnir" left to the afterlife. They were running on the same power strip. "sleipnir's" bios settings got scrambled up and I had boot problems but nothing serious. This is something I remember from my old overclocking days - bios is sensitive and can cause all kinds of errors if power shocks occur or if a box freezes up because of to high clock.

I did current measurements on my 3 currently mining boxes this afternoon to figure out how much power they actually use and to estimate PSU efficiency. I have one new 735w "Thunder V2" psu rated 80 plus (or whatever). It seems to be doing considerably better than the older units I am running. If the idea is to run these cards for years better PSU's will pay for them self's in the long run. Not to mention they might skip the fireworks - something totally worth it. If I am not to lazy I will swap the PSU's and repeat the power measurements to see the actual difference between the PSU's. The question I am dealing with is if it's worh it to shell out extra for "titanium" energy rating vs Platinum vs Gold in relation to energy cost. This boils down to how long will they be running. A year, 3 or 5?

are you on 220v or 120 ? if you got 220v, then the leadexII will actually be your best choice, they are underrated to get gold in both 220 and 120 and are actually more efficient than gold level in 220v
platinum is also nice, titanium imo is overpriced

 

[87dtna]

My 1070 runs best with 80% power and +200 on the core clock and +900 memory. The more you lower power the more it lowers the core clock so you have to OC big to get it back. It only pulls 140w from the wall and gets 450 hash/s at this clocks. The memory doesn't help much, but it is a little like 10 hash/s and doesn't seem to affect power usage at all. I can get as high as 485 hash/s with 112% power and +75 core clock but is uses like 190w and that 50w wasn't worth 35 hash/s.

On the PSU, any 80+ gold will be good. You won't even notice a realistic difference for platinum so it's definitely not worth the extra money they cost....unless you can find one for the same price as a gold of course. The best brands IMO are seasonic, Corsair, Enermax, and Antec. There's a few others that are good too but those are the brands I usually recommend for price/performance value. I would recommend an 850w PSU, you can run ~3 decent GPU's on that and be in the optimal efficiency range. If you only plan on 2 GPU's per box, a 750w would do.

I don't know if Newegg ships to iceland but sometimes they get some crazy good refurbished deals like this corsair 850w gold PSU for $70-

https://www.newegg.com/Product/Product.aspx?Item=N82E16817139152

 

[Ericsson]

Thanks for the inf guys. Much appreciated.

I got 750W Seasonic gold locally and swapped psu's around so both my 1080 cards are running on new and capable supplies. I saw 7% current drop from one of the box after getting better supply.

I am playing around with power settings and some overclocking and following 24 hour averages hash rate reported by Nanopool. Unfortunaetly this takes quite a while to give results, and the psu blowout didn't help in this regard.

I hope I can post some interesting results in next few days. Currently it seems to me the reported hash rate from the EWFB miner can be suspect. For example at default power settings my 1080 cards are using 185w and EWFB reports 530 sols. The pools rate them at 480-500. Lower the power down to 160w and EWFB goes down to around 500 but the pools report higher hash rate on 24 hour averages. This is why I am focusing on the reported hash rate the pools and not only on what the miner is reporting.

I like Nanopool because it has nice webpage to check the average hash rate for each worker over different time intervals. However I am seeing clues that Flypool would make me more money - based on time intervals between payouts. My workers made 0.01 ZEC (zcash) every 12 hours on Flypool, On Nanopool I still can't reach that. Takes longer but - more testing will hopefully clear this up.

 

[Ericsson]

Another thing I am struggling with is changing clock settings over SSH using nvidia-settings on ubuntu or suse servers. It seems X hast to be running for this to work. Therefore I am just using desktop versions of OpenSuSuse and Mint. Not seeing any slowdowns because of this even if the graphics overhead is using some memory from the cards.

HEAT:
First I put all 4 boxes in the same room. No problem - it had a good size window I could open. Well, this small room got quite warm and there was considerable FAN humming noise going on. Open the window is all good -except we have had plethora of winter storms in past weeks and up up 20-30 m/sec blowing from different directions. So I had to constantly monitor the situation - if there was a snowstorm in the room or if some of our horizontal rain was finding it's way in. This got old very fast....

So Instead I spread the boxes around the apartment and just lowered the thermostat on the radiators. One is humming in the living room, one in the corridor and two in my spare room. I can barely hear them because they get enough cold air for the cards (open cases). And now they are just heating the entire apartment instead of creating Sahara in single room. And when the storms hit I can keep the windows closed.

 

[ZL1]

Another thing I am struggling with is changing clock settings over SSH using nvidia-settings on ubuntu or suse servers. It seems X hast to be running for this to work. Therefore I am just using desktop versions of OpenSuSuse and Mint. Not seeing any slowdowns because of this even if the graphics overhead is using some memory from the cards.

HEAT:
First I put all 4 boxes in the same room. No problem - it had a good size window I could open. Well, this small room got quite warm and there was considerable FAN humming noise going on. Open the window is all good -except we have had plethora of winter storms in past weeks and up up 20-30 m/sec blowing from different directions. So I had to constantly monitor the situation - if there was a snowstorm in the room or if some of our horizontal rain was finding it's way in. This got old very fast....

So Instead I spread the boxes around the apartment and just lowered the thermostat on the radiators. One is humming in the living room, one in the corridor and two in my spare room. I can barely hear them because they get enough cold air for the cards (open cases). And now they are just heating the entire apartment instead of creating Sahara in single room. And when the storms hit I can keep the windows closed.

well though I love linux for servers, for desktops and such its usually a pain so you know .. try windows maybe ? works very well with rdp on the latest win, older ones would kill video on rdp

strange that you get such heat .. I have 6 cards in a row and they are doing aok on heat, underpowered the cards right ? no point in running full power, 60-80% should be good

 

[87dtna]

I think thats his problem, they are all 100% because there's no software he can use to change the clocks/settings. He should be getting a lot more than 500 sol/s on a 1080 when my 1070 can get 475.

 

[ZL1]

I think thats his problem, they are all 100% because there's no software he can use to change the clocks/settings. He should be getting a lot more than 500 sol/s on a 1080 when my 1070 can get 475.

this
go windows, it will consume less and heat up less too

 

[Ericsson]

this
go windows, it will consume less and heat up less too

I am controlling both power and clocks on the cards with linux. I am just seeing the wattage directly, not power ratios like you guys. The 1080 come with "default" power setting (according to Nvidia 390 driver for linux) at 200w. I am running them currently at 160w. Thats, 80% of the default power limit, and 67% of the "max power limit"


Here are the numbers from "skvetta" -output from the driver just now -it's using 160W and doing probably 530 sols right now.

Power Readings
        Power Management            : Supported
        Power Draw                  : 159.66 W
        Power Limit                 : 160.00 W
        Default Power Limit         : 198.00 W
        Enforced Power Limit        : 160.00 W
        Min Power Limit             : 90.00 W
        Max Power Limit             : 238.00 W

Put 4 computers pulling 750w total in a small room and it just gets hot.
No mystery there.

I wouldn't mind trying windows 10 on one miner or two. But am not going to buy a copy of windows 10 just to test it out - since it's mostly worthless for me for everything else. And I haven't borrowed a copy of windows for over 15 years - can't say I missed it. I wouldn't know where to start.

 

[ZL1]

I am controlling both power and clocks on the cards with linux. I am just seeing the wattage directly, not power ratios like you guys. The 1080 come with "default" power setting (according to Nvidia 390 driver for linux) at 200w. I am running them currently at 160w. Thats, 80% of the default power limit, and 67% of the "max power limit"


Here are the numbers from "skvetta" -output from the driver just now -it's using 160W:

Power Readings
        Power Management            : Supported
        Power Draw                  : 159.66 W
        Power Limit                 : 160.00 W
        Default Power Limit         : 198.00 W
        Enforced Power Limit        : 160.00 W
        Min Power Limit             : 90.00 W
        Max Power Limit             : 238.00 W

Put 4 computers pulling 750w total in a small room and it just gets hot.
No mystery there.

nice! didnt know you can do that with lin .. i may just try myself, btw did you try to just forward X via ssh and run the gui on your workstation? it should work without having it running on the server

well thats why i recommend multi-gpu less heat and less power since you're running one cpu, one psu, one motherboard, and you can spread 4 cards well enough apart to keep em cool

 

[Ericsson]

controlling power and modes, reading temps and clocks is very easy over ssh. it's just the "nvidia-smi" tool that does this.

$nvidia-smi -pl 100 - sets the power level to 100w for example.

However the clocks and fan speed is controlled via "nvidia-settings" that is graphical tool in nature. On linux desktop it fires up a window that looks very similar to the windows version. Running this bugger over ssh requires some tricks on headless servers. I have managed with OpenSuse but not yet with ubuntu. I will figure it out one day.

Agreed on heat and power use - each of my old boxes is pulling 60-70w of power just idling. Many cards per box is obviously the way to go for serious miners. (the risers are in the mail on it's way here).

 

[ZL1]

controlling power and modes, reading temps and clocks is very easy over ssh. it's just the "nvidia-smi" tool that does this.

$nvidia-smi -pl 100 - sets the power level to 100w for example.

However the clocks and fan speed is controlled via "nvidia-settings" that is graphical tool in nature. On linux desktop it fires up a window that looks very similar to the windows version. Running this bugger over ssh requires some tricks on headless servers. I have managed with OpenSuse but not yet with ubuntu. I will figure it out one day.

Agreed on heat and power use - each of my old boxes is pulling 60-70w of power just idling. Many cards per box is obviously the way to go for serious miners. (the risers are in the mail on it's way here).

well forward X, it will open headless on your desktop
you need X11Forwarding yes set in the sshd_config on the server side, then just ssh -X user@host