GPU Help Please!

harlam357 · Feb 11, 2007

Don't know what I've done here guys... the log is below. I've got a Sapphire X1900GT (which suxs for Folding... ~11mins a frame) on CATS 6.11. But as you can see... I had it running fine on stock clocks. I just brought this rig out of a 24 hr. prime and running two cpu clients (1 big packet, 1 reg. grommy) just fine.

I've removed/cleaned/reloaded all nVidia chipset drivers & ATI display drivers. Plus installed the DX9c Dec. 2006 Update (that's what stopped me before). It is installed as a service with the option to allow the service to interact w/the desktop. But I know that's not it since it's EUE'ing just as easily if I try to run in console mode.

I've deleted the current WU and core and downloaded new... still, no help.

Found a thread here -> http://forum.folding-community.org/ftopic17841-0-asc-0.html

From what I'm reading there, I did everything minus remove the client-service and delete the client completely. I don't see how the client got borked!? But I'll try that tomorrow.

O wise and smart type knowledgeable people who are my teammates... any other suggestions!? :shrug:

Code:

[20:38:16] - Preparing to get new work unit...
[20:38:16] + Attempting to get work packet
[20:38:16] - Connecting to assignment server
[20:38:17] - Successful: assigned to (171.64.65.20).
[20:38:17] + News From Folding@Home: GPU folding beta
[20:38:17] Loaded queue successfully.
[20:38:18] + Closed connections
[20:38:18] 
[20:38:18] + Processing work unit
[20:38:18] Core required: FahCore_10.exe
[20:38:18] Core found.
[20:38:18] Working on Unit 02 [February 8 20:38:18]
[20:38:18] + Working ...
[20:38:18] 
[20:38:18] *------------------------------*
[20:38:18] Folding@Home GPU Core - Beta
[20:38:18] Version 0.10 (Mon Oct 30 12:32:17 PST 2006)
[20:38:18] 
[20:38:18] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.10.3077 for 80x86
[20:38:18] Build host: CYGWIN_NT-5.1 vishal-gpu 1.5.19(0.150/4/2) 2006-01-20 13:28 i686 Cygwin
[20:38:18] Preparing to commence simulation
[20:38:18] - Looking at optimizations...
[20:38:18] - Created dyn
[20:38:18] - Files status OK
[20:38:18] - Expanded 86919 -> 443709 (decompressed 510.4 percent)
[20:38:18] 
[20:38:18] Project: 2736 (Run 3, Clone 558, Gen 3)
[20:38:18] 
[20:38:18] Assembly optimizations on if available.
[20:38:18] Entering M.D.
[20:38:25] Working on 1258 GPUlambda
[20:38:25] Starting GUI Server
[20:49:28] Completed 1
[21:00:28] Completed 2
[21:11:27] Completed 3
[21:22:26] Completed 4
[21:33:26] Completed 5
[21:44:25] Completed 6
[21:55:25] Completed 7
[22:06:24] Completed 8
[22:17:23] Completed 9
[22:28:23] Completed 10
[22:39:22] Completed 11
[22:50:22] Completed 12
[23:01:21] Completed 13
[23:12:21] Completed 14
[23:23:20] Completed 15
[23:34:20] Completed 16
[23:45:19] Completed 17
[23:56:18] Completed 18
[00:07:18] Completed 19
[00:18:17] Completed 20
[00:29:17] Completed 21
[00:40:16] Completed 22
[00:51:16] Completed 23
[00:53:42] Service stop request received.

Folding@Home Client Shutdown.


--- Opening Log file [February 12 02:57:43] 


# Windows Console GPU Edition #################################################
###############################################################################

                       Folding@Home Client Version 5.91beta3

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: C:\Program Files\FAH_GPU
Service: C:\Program Files\FAH_GPU\FAH5.91beta3-console.exe
Arguments: -svcstart -verbosity 9 

Launched as a service.
Entered C:\Program Files\FAH_GPU to do work.

[02:57:43] - Ask before connecting: No
[02:57:43] - User name: harlam357 (Team 32)
[02:57:43] - User ID: 6EF067D40D534737
[02:57:43] - Machine ID: 3
[02:57:43] 
[02:57:43] Work directory not found. Creating...
[02:57:43] Could not open work queue, generating new queue...
[02:57:43] - Preparing to get new work unit...
[02:57:43] + Attempting to get work packet
[02:57:43] - Will indicate memory of 2046 MB
[02:57:43] - Connecting to assignment server
[02:57:43] Connecting to http://assign-GPU.stanford.edu:8080/
[02:57:43] - Autosending finished units...
[02:57:43] Trying to send all finished work units
[02:57:43] + No unsent completed units remaining.
[02:57:43] - Autosend completed
[02:57:43] Posted data.
[02:57:43] Initial: 40AB; - Successful: assigned to (171.64.65.20).
[02:57:43] + News From Folding@Home: GPU folding beta
[02:57:43] Loaded queue successfully.
[02:57:43] Connecting to http://171.64.65.20:8080/
[02:57:44] Posted data.
[02:57:44] Initial: 0000; - Receiving payload (expected size: 87362)
[02:57:44] Conversation time very short, giving reduced weight in bandwidth avg
[02:57:44] - Downloaded at ~170 kB/s
[02:57:44] - Averaged speed for that direction ~170 kB/s
[02:57:44] + Received work.
[02:57:44] + Closed connections
[02:57:44] 
[02:57:44] + Processing work unit
[02:57:44] Core required: FahCore_10.exe
[02:57:44] Core found.
[02:57:44] Working on Unit 01 [February 12 02:57:44]
[02:57:44] + Working ...
[02:57:44] - Calling 'FahCore_10.exe -dir work/ -suffix 01 -checkpoint 15 -service -verbose -lifeline 1480 -version 591'

[02:57:45] 
[02:57:45] *------------------------------*
[02:57:45] Folding@Home GPU Core - Beta
[02:57:45] Version 0.10 (Mon Oct 30 12:32:17 PST 2006)
[02:57:45] 
[02:57:45] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.10.3077 for 80x86
[02:57:45] Build host: CYGWIN_NT-5.1 vishal-gpu 1.5.19(0.150/4/2) 2006-01-20 13:28 i686 Cygwin
[02:57:45] Preparing to commence simulation
[02:57:45] - Looking at optimizations...
[02:57:45] - Created dyn
[02:57:45] - Files status OK
[02:57:45] - Expanded 86850 -> 443709 (decompressed 510.8 percent)
[02:57:45] 
[02:57:45] Project: 2736 (Run 9, Clone 182, Gen 3)
[02:57:45] 
[02:57:45] Assembly optimizations on if available.
[02:57:45] Entering M.D.
[02:57:51] Working on 1258 GPUlambda
[02:57:52] Starting GUI Server
[02:57:58] mdrun_gpu returned 114
[02:57:58] Going to send back what have done.
[02:57:58] logfile size: 7125
[02:57:58] - Writing 7663 bytes of core data to disk...
[02:57:58] Done: 7151 -> 2696 (compressed to 37.7 percent)
[02:57:58]   ... Done.
[02:57:58] 
[02:57:58] Folding@home Core Shutdown: EARLY_UNIT_END
[02:58:01] CoreStatus = 72 (114)
[02:58:01] Sending work to server


[02:58:01] + Attempting to send results
[02:58:01] - Reading file work/wuresults_01.dat from core
[02:58:01]   (Read 3208 bytes from disk)
[02:58:01] Connecting to http://171.64.65.20:8080/
[02:58:01] Posted data.
[02:58:01] Initial: 0000; Conversation time very short, giving reduced weight in bandwidth avg
[02:58:01] - Uploaded at ~8 kB/s
[02:58:01] - Averaged speed for that direction ~8 kB/s
[02:58:01] + Results successfully sent
[02:58:01] Thank you for your contribution to Folding@Home.
[02:58:05] Trying to send all finished work units
[02:58:05] + No unsent completed units remaining.
[02:58:05] - Preparing to get new work unit...
[02:58:05] + Attempting to get work packet
[02:58:05] - Will indicate memory of 2046 MB
[02:58:05] - Connecting to assignment server
[02:58:05] Connecting to http://assign-GPU.stanford.edu:8080/
[02:58:06] Posted data.
[02:58:06] Initial: 40AB; - Successful: assigned to (171.64.65.20).
[02:58:06] + News From Folding@Home: GPU folding beta
[02:58:06] Loaded queue successfully.
[02:58:06] Connecting to http://171.64.65.20:8080/
[02:58:06] Posted data.
[02:58:06] Initial: 0000; - Receiving payload (expected size: 87365)
[02:58:07] - Downloaded at ~85 kB/s
[02:58:07] - Averaged speed for that direction ~127 kB/s
[02:58:07] + Received work.
[02:58:07] Trying to send all finished work units
[02:58:07] + No unsent completed units remaining.
[02:58:07] + Closed connections
[02:58:12] 
[02:58:12] + Processing work unit
[02:58:12] Core required: FahCore_10.exe
[02:58:12] Core found.
[02:58:12] Working on Unit 02 [February 12 02:58:12]
[02:58:12] + Working ...
[02:58:12] - Calling 'FahCore_10.exe -dir work/ -suffix 02 -checkpoint 15 -service -verbose -lifeline 1480 -version 591'

[02:58:12] 
[02:58:12] *------------------------------*
[02:58:12] Folding@Home GPU Core - Beta
[02:58:12] Version 0.10 (Mon Oct 30 12:32:17 PST 2006)
[02:58:12] 
[02:58:12] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.10.3077 for 80x86
[02:58:12] Build host: CYGWIN_NT-5.1 vishal-gpu 1.5.19(0.150/4/2) 2006-01-20 13:28 i686 Cygwin
[02:58:12] Preparing to commence simulation
[02:58:12] - Looking at optimizations...
[02:58:12] - Created dyn
[02:58:12] - Files status OK
[02:58:12] - Expanded 86853 -> 443709 (decompressed 510.8 percent)
[02:58:12] 
[02:58:12] Project: 2736 (Run 0, Clone 144, Gen 4)
[02:58:12] 
[02:58:12] Assembly optimizations on if available.
[02:58:12] Entering M.D.
[02:58:19] Working on 1258 GPUlambda
[02:58:19] Starting GUI Server
[02:58:24] mdrun_gpu returned 114
[02:58:24] Going to send back what have done.
[02:58:24] logfile size: 7107
[02:58:24] - Writing 7645 bytes of core data to disk...
[02:58:24] Done: 7133 -> 2701 (compressed to 37.8 percent)
[02:58:24]   ... Done.
[02:58:24] 
[02:58:24] Folding@home Core Shutdown: EARLY_UNIT_END
[02:58:26] CoreStatus = 72 (114)
[02:58:26] Sending work to server

WarriorII · Feb 11, 2007

I think Pete would be the guy to help you out here. He has a couple of GPU units online. Check back tomorrow, he's in Canada. (East Cost)

Good luck on getting this up & running.

Have a Cheesey Poof in the mean time.

pscout · Feb 12, 2007

is this a dedicated rig? ... not sure why you want to run gpu as a service?

Not sure if the service is related to your eue issue, but i don't run any of mine that way.

Other than clean install of drivers, the directx update fom the faq link, and confidence that the card's oc is stable via 3dmark tests i can't think of anything else to check. except are you oc'ing the pci-e? I have played with it up to 115 on one of my mobos but 105-110 seems stable on the others that i do oc the interface.

There's my brain dump ... my guess is its related to the service ... but mostly becuz it violates the kiss principle.

/edit ... don't have a gt so i don't know if there are specific issues with it. My gpu's are all on 6.5 or 6.11. 6.5 on the ones i haven't messed with since the beginning of beta but most are 6.11 now since i have 1 1950 pro that need them.
Did a dust bunny get sucked into the cooler ... 3dmark should fail? 1 last thing, i have never had problems with atitool in setting the clocks, but have heard of others reporting problems when they adjust fan speed with it. i pretty much run my fans at fixed speed since gpu load is constant and they are all caseless.

davekusa · Feb 12, 2007

I've been running into the same exact issue. Cleaned/Deleted/ReInstalled twice.Pulled heatsink off and reapplied AS5.

Right now GPU voltage at 1.35 (was 1.4). GPU temp was 62 now 58
and I had to underclock GPU memory. Stock 600 now 570.
Seems to have started same time the new Beta core came out.

Keep us up to date

pscout · Feb 12, 2007

I haven't noticed any misbehaviour on any of mine lately. But i also try to keep my oc's a bit conservative so i don't have to watch them too much.
I did have a dust bunny problem with one in early dec, and then a mobo going bad.

All but one of mine are running on vmware rigs, and since their perf is all degraded, I guess their load and temps have too. ... so i am avoiding the extra load the newer wu's might be creating?
Also, cuz of the issue with the smp 2605's, 1-2 gpu instances have been stopped for times over the last few days.

I run all mine at 1.375V except the 1950 pro which i can't change.

leelegend · Feb 12, 2007

i didnt think the GPU client worked as a service?
CLI with tray-it or GUI only

lee

ChasR · Feb 12, 2007

I had similar troubles on a X1950 XTX when I used ATI tool to build a profile and force 3D clocks without remembering to up the voltages to the 3D volts. That may not relate to your issue except perhaps the qpu is being undervolted or the ram timings aren't being relaxed automatically as speed is increased. Artifacts should be present particularly if the ram is the problem.

PaciFIST69 · Feb 12, 2007

I've had some troubles with my GPU also over the last couple of weeks. ATItool was stable for hours but if I let it fold overnight (I only run the GPU at night) it would fail within the first hour. After tweaking various things with no success, I tried disabling my screensaver(really basic, WinXP logo) last night. Lo and behold, it's still folding this morning. It's too early to tell if that has fixed the problem completely, but I have a good feeling about it.

I know this is a long shot for your problem H, but disable screensaver if you haven't already

PS: I remember someone writing "screensavers are the scourge of folding" b/c they had an OpenGL SS that was eating all their points

dfonda · Feb 12, 2007

I remember someone writing "screensavers are the scourge of folding" b/c they had an OpenGL SS that was eating all their points

Yup. useless might as well put the thing to sleep to save juice and monitor life, bothers the heck out of defrag as well.

LandShark · Feb 12, 2007

have you tried to run/install it in console instead of service and see if it still EUE?? as far as I can tell, the current GPU beta is pretty stable and I haven't mess with any of mine for awhile.

harlam357 · Feb 12, 2007

pscout said:
is this a dedicated rig? ... not sure why you want to run gpu as a service?

Not sure if the service is related to your eue issue, but i don't run any of mine that way.

Other than clean install of drivers, the directx update fom the faq link, and confidence that the card's oc is stable via 3dmark tests i can't think of anything else to check.

There's my brain dump ... my guess is its related to the service ... but mostly becuz it violates the kiss principle.

Yes and No... dedicated when I'm not doing something else on it, which is pretty rare... but it is cased and I have a couple HDs in it for storage, etc. So I do hit that box over the network from time to time.

I run everything as a service... just the way I like it. And to test my Folding Service Control program that I've been reluctant to release (I think I'm gonna upgrade it to .NET first- but that's another discussion).

Nope... not the service. I can go to the command line and fire up the client as a console and I get the same thing. FYI- I had it running fine as a service before.

To make it run as a service (first install as service), then go to services.msc and select properties on the service -> LogOn -> "check" -> Allow this service to interact w/the Desktop.

This is a Brand New x1900gt. I've only had it about a month... and temps are way in check by ATITool... I don't believe I have a dust bunny issue here.

I'm going to delete the service, download a-new, and try to fire up in console first. Like I said, I've already been through a driver removal, clean (DriverCleaner Pro), and reinstall from the chipset up to vid. And ran the DX9c Dec. 2006 update as well (that's what stopped me the first time I had trouble... so it shouldn't be the DX update). :shrug:

silent bob · Feb 12, 2007

try setting your screen saver to Blank so its just a black screen . I run mine in Console and blank screensaver , never had any problems yet *knock on wood*
The only problem I do have is when i want to play a game , it always cuts off and I have to restart it again after playing

harlam357 · Feb 12, 2007

No screen savers either... since they can rob cycles from regular cpu clients as well, it's always one of the first thing I turn off on a fresh install.

harlam357 · Feb 13, 2007

Ok, had it running last night for a spell... but then the same junk again.

Removed the service, removed the FAH GPU folder all together, then made a New folder and re-downloaded the client from Stanford. Config -> fire in console -> Fold (~2 hours). Said ok, seems like that did it. Fired off one of my cpu clients (service of course). Immediately got an EUE on the GPU client.

When I was running the GPU initially I had no other FAH running... so it appears I'm having problems running the GPU with other FAH clients. That being said, I previously had this rig running VMWare & GPU on the previous set of dimms.

I just dropped Brand New memory in this box... but I did a 24hr prime blend with no issues (it is highly clocked Infineon CE-5 - 270MHz 3-3-2-7). However, I'm well aware that FAH can have higher impacts on memory than even prime can. So I guess I need to start looking at the memory settings some more. Again, two cpu clients seem fine... it was still running both those this morning, but either combined with the GPU is not stable. :shrug:

PaciFIST69 · Feb 13, 2007

Seems unlikely that it's a memory problem if 2 cpu clients and dual prime are stable. Guess it would pretty easy to check by just putting the memory on a divider and seeing if you can fold.

Another long shot here - try having one cpu client already running and then start the GPU. I've got the same setup as you here with 1 cpu client as a service and gpu client as console and I always start gpu with 1 cpu already running.

PS/offtopic: nice ram, where did you get it and the stuff in your sig?

harlam357 · Feb 13, 2007

I am inclined to agree Fist... but I can't think of any other reason why I can't get it working.... or maybe this x1900gt is a flake. :shrug:

I'll do as you suggest and put the memory on a higher divider and see if that cures it. Good call! :thup:

Tks!

The stuff I just dropped in that rig is Patriot 2Gig 3500LLK (2-3-2-5 2.6v). Again, doing 270MHz @ 3-3-2-7 2.74v... from reviews I've read this stuff is CE-5. The stuff in my sig is CE-6. With the recent shortage of good DDR around these sticks are hard to find. The OCZs were bought new from the Egg ~6 months ago and the Patriots are RMA replacement for my previous PC4800 2x512mb TCCD kit that wouldn't clock anymore. Heck, I'm getting more from the 2x1gb sticks than I was anymore with the 2x512mb sticks. :beer:

ChasR · Feb 13, 2007

When you reinstalled last time you didn't forget to change the machine ID from the default of 1 did you? That could explain an instant EUE on cpu client startup.

harlam357 · Feb 13, 2007

Would explain the instant conflict but alas, no... I'm not a client rookie.

I like to fancy myself as a semi-client expert. Second only to the ChasR man himself.

I just came back from the house only to find that rig rebooted... again!

Same symptom I was having before I switched RAM. /me thinks I need to look at something else... picky @SS DFI boards. :bang head

harlam357 · Feb 14, 2007

To follow up on this... tried Fist's suggestion and put the RAM on a higher divider. Now only running them @ 245MHz. GPU Folding & UP client on a big packet. My guess is that the RAM wasn't 3D stable at those speeds. Although 2D/Prime stable. My CE-6 is 3D stable up to 270MHz, but it appears these CE-5s are not so stable. Will continue working with them @ 270MHz using 3DMark, etc to see how they behave in that scenario.

I say not 3D stable b/c the gpu client is in effect stressing the system RAM similarly as a game or 3D bench mark would. I'll continue to investigate further, but I'm going to run like this overnight and see how it goes.

Thanks for the help all! :thup:

And my bad... doesn't appear to be a totally GPU related error.

PaciFIST69 · Feb 15, 2007

That's interesting H. I didn't know GPU FAH was so memory intensive. If you get a chance, could you put the memory back to where it was and run 1 instance of prime and atitool 3d view together and see if it's stable? Just curious

GPU Help Please!

Senior Fold-a-holic

Moderator #666

Senior Newbie Cannon-Fodder - R. I. P. good budd

Member

Senior Newbie Cannon-Fodder - R. I. P. good budd

Member

Senior Member

Member

Senior Golfer

Super Shark Moderator

Senior Fold-a-holic

Team 32 Folding Ghetto Hanging PC Senior

Senior Fold-a-holic

Senior Fold-a-holic

Member

Senior Fold-a-holic

Senior Member

Senior Fold-a-holic

Senior Fold-a-holic

Member

Similar threads