Core A7 WU's failing. "returned: FAILED_2 (1 = 0x1)"

Hey All!

I've been a bit out of it lately as I've been busy with Halloween and office renovations. When I checked on my CPU folders, I discovered a new core (A7) that I've done some reading up on here. YAY!

However, many of my CPU's folders are failing with the following error: WU00:FS00:FahCore returned: FAILED_2 (1 = 0x1)
Does anyone know why this happening and how I can fix it? Half of my CPU folders are down with this issue.

System:
Linux: CentOS 6.8
Kernel: 2.6.32-642.6.2.el6.x86_64
FAHClients: 7.4.4.1 and 7.4.15 Beta
Client-Type = advanced
Additional Flags: forceasm
Slots: 24 cores and 48 cores.

Hardware: AMD Opteron 6176 SE x 4 for a total of 48Cores.

Any help or pointing in the right direction to help me solve this would be much appreciated.

Thanks!

-Kirk

Wow! I did a Google search and the results I get are from 2013 so.... no help there. I really never learned more about Linux other than installing so.... no help there either.

Sorry.

Try this thread from FF:-

https://foldingforum.org/viewtopic.php?f=96&t=29273

I have not had this on my 4P's as I'm using 6276's on both. I think it does matter the CPU.

OS: Ubuntu 14.04.02 LTS
Folding client: 7.4.4 & 7.4.15

Nathan_P said:

Try this thread from FF:-

https://foldingforum.org/viewtopic.php?f=96&t=29273

Click to expand...

Thanks for the tip! Unfortunately, the issue persists. I'm not getting the same error message as him.

The following is the entirety before the error occurs:


In fact, looking through my other CPU folders I see that the only WU's that are successfully running are the A4 cores that are downloaded. All A7's are giving me this error. Perhaps it is CPU dependent? Anyone else running 6100 series successfully?

-Kirk

Ramonetb said:

many of my CPU's folders are failing with the following error: WU00:FS00:FahCore returned: FAILED_2 (1 = 0x1)

Click to expand...

I have had that error and I can't remember at this moment how I fixed it.

When my brain fog clears I hope I can be of more help.

Here we go: link

Basically I had to update to Ubuntu 14.04 and use the 7.4.15 client or the a7's kept dying on me. After that I did have a7's running on 4x 6128s.

It looks like 14.04 is the issue, i'm running 14.04 and using client 7.4.4, a7 works just fine.

Nathan_P said:

It looks like 14.04 is the issue, i'm running 14.04 and using client 7.4.4, a7 works just fine.

Click to expand...

I did notice that as 14.04 was completing its install but before rebooting, client 7.4.4 picked up an a7.

I think the new client is suppose to distribute work to the cores more effectively, but I have seen no evidence of that yet.

Sorry for the delays again. I had a couple Halloween parties to attend this weekend and my brother, ProteinPete, and I were finishing up my costume. I'll post some pics when I get a chance.

As for Folding A7:

Hmmm.

I'm not running Ubuntu, but CentOS 6.8 (RedHat essentially). I've tried the beta client 7.4.15 and it didn't resolve the issue. I hope this isn't a kernel issue.
Do you happen to know what Kernel Ubuntu 14.04 is running on? I'll try to quickly find what version on the webs....

-Kirk

Ramonetb said:

Sorry for the delays again. I had a couple Halloween parties to attend this weekend and my brother, ProteinPete, and I were finishing up my costume. I'll post some pics when I get a chance.

As for Folding A7:

Hmmm.

I'm not running Ubuntu, but CentOS 6.8 (RedHat essentially). I've tried the beta client 7.4.15 and it didn't resolve the issue. I hope this isn't a kernel issue.
Do you happen to know what Kernel Ubuntu 14.04 is running on? I'll try to quickly find what version on the webs....

-Kirk

Click to expand...

I know with Ubuntu 14.04 there was a problem with not having python2 installed on fresh installs. Perhaps CentOS has that problem as well. I will try to find the exact command for U14.04.


The kernal is Linux tiger1 3.13.0-98-generic .



The thing missing from a fresh install of U14.04 was python-gnome2, to install it:

sudo apt-get install python-gnome2

At least for me that fixed U14.04 so that it would run properly.

I'm not running CentOS, but as for Ubuntu 14.04.02 LTS I had no problems installing 7.4.4 or 7.4.15. I did this through the GUI and not the command line, I manually downloaded the install files from Satnford then double clicked on them.
So try installing from the GUI and see if that helps.

WhitehawkEQ said:

I'm not running CentOS, but as for Ubuntu 14.04.02 LTS I had no problems installing 7.4.4 or 7.4.15. I did this through the GUI and not the command line, I manually downloaded the install files from Satnford then double clicked on them.
So try installing from the GUI and see if that helps.

Click to expand...

I think you did an upgrade install of U14.04. A fresh install is different because the old packages are not still present in some cases.

orion456 said:

I think you did an upgrade install of U14.04. A fresh install is different because the old packages are not still present in some cases.

Click to expand...

Hey hey hey don't go using that bad word "upgrade" near me jk I never upgrade a OS to a higher lvl, when I installed 14.04.02 LTS, it was from scratch. I always wipe out what is on the drive before I install new (Win or Linux).
Now I have had problems with older F@H clients in Linux but not with 7.4.4 & 7.4.15 for Linux.

WhitehawkEQ said:

Hey hey hey don't go using that bad word "upgrade" near me jk I never upgrade a OS to a higher lvl, when I installed 14.04.02 LTS, it was from scratch. I always wipe out what is on the drive before I install new (Win or Linux).
Now I have had problems with older F@H clients in Linux but not with 7.4.4 & 7.4.15 for Linux.

Click to expand...

If you did a new install, I wonder why my new install didn't work until I installed python2 again?

I am now getting the error on an i7 running Ubuntu 10. It has shut down folding because of too many failures on a7.

I will try installing 14.04 on that machine and see if it works again.

-------- update ---------------

I changed to U16.04, added the V7.4.15 client and the i7 970 started folding a7 cores despite the fact it doesn't have AVX.

Looks like older Linux has problems with the new core a7.

Well, I ran into something a little odd. When trying to install the Beta 4.7.15 FAHControl, I'm told that my version of python is out of date. This is on a similar track to what you ran into Orion456, with having to update python-gnome2.

For more specifics, CentOS 6.8 comes with python 2.6 installed. 4.7.15 wants 2.7. I can't simply replace 2.6 with 2.7 as the revision change breaks MANY things in RedHat on this release.

Just the same, I compiled and alt installed 2.7, added some symlinks, and tried to get FAHControl going. Had to force install, since it didn't see the dependencies listed... but they are there. Nevertheless, Control refuses to run indicating "fah" is an unknown module. Still poking at this.


All this background simply to say that, I know others have gotten the Core_A7 to run on their 7.4.4 installs... but is it possible for the core to actually function that it requires python 2.7? Would that really be necessary (I know nothing about how the cores work)? I'm about ready to set up a test machine running CentOS7. But that changes so many variables (like kernel revisions, etc) that I think the test is lost. Although, it would prove (again) that the core works with 6100 processors.

I don't suppose anyone who's running the A7's would be able to check their installed python version? Especially if you're running on a 6100 series. WOO!

Keepin' kicking the box...

-Kirk

python -V returns 2.7.12. My hosts all running Ubuntu 16.04 with recent updates, upgraded from 14.04 or earlier versions. Client v7.4.15. Core A7 has run OK on i7-2600K with AVX and i7-860 with SSE2. Also has run OK on 6700HQ under Win10.

HayesK said:

python -V returns 2.7.12. My hosts all running Ubuntu 16.04 with recent updates, upgraded from 14.04 or earlier versions. Client v7.4.15. Core A7 has run OK on i7-2600K with AVX and i7-860 with SSE2. Also has run OK on 6700HQ under Win10.

Click to expand...

Thanks, sir!
Good info to have.

I've also started a thread on the Stanford folding forums about the issue.

-Kirk

Ramonetb said:

Thanks, sir!
Good info to have.

I've also started a thread on the Stanford folding forums about the issue.

-Kirk

Click to expand...

I thought the problem for me was that some dependency in Python2 is no longer needed in subsequent versions and so it wasn't installed in U14.04. That missing dependency prevented FAH from installing properly. Reinstalling Python2 gives the dependency back. So it wasn't python itself but some software connected to it. I also understood that FAH doesn't correctly list all its dependencies properly and so the missing dependency isn't flagged to be downloaded. <<--- my weak understanding of Linux prevents me from making total sense of these statements, assuming they have any relation to the real problem.

I've been mucking about with this on the Stanford Folding forums and it seems the main issue with A7 cores is the version of GLIBC. When trying to run the core manually, the following output is produced:


As you can see, it also needs a libstdc++ that supports GLIBCXX 3.4.15.

And from my post on the Stanford Forum:

In an attempt to get this running under CentOS 6, I've done the following:

Keep in mind that we are missing versions of glibc, I've gone and downloaded versions 2.14, 2.15, and 2.17, compiled them, and installed them their respective directories in /opt. (Although, only the latest is needed).
For the curious, a handy quick step guide:


I've then added their library locations to my immediate (non system wide) library path thusly:


This has resolved the majority of the issues. However, the remaining one, libstdc++.so.6 requiring GLIBCXX_3.4.15 is not so easy to deal with. Downloading the library file itself is all well enough as it also depends on the now installed glibc versions I just mentioned. But actually loading the library so far is proving to be problematic.


or whatever flavor you prefer of libstdc++ that carries support of GLIBCXX_3.4.15. This results in a segfault.

If one were to be crazy and change the symbolic link of /usr/lib64/libstdc++.so.6 to point directly towards the newer libstdc++.so.6.0.19, you would cripple the system. Linux IS glibc afterall (as the saying goes) and these varying versions get tricky.

I was hoping to find a way to do this without having to compile gcc from scratch and add a whole other build along side, but so far I haven't hit on the right combination.

Ultimately, Core_A7 is built demanding libraries that are newer than CentOS 6 or Ubuntu 10 supplies. I'm curious again to know whether this is strictly necessary and whether or not a version of the core can be released that supports older libraries. CentOS 6.8 isn't THAT old as Enterprise class systems go....

But I've just read that the A7's are gone for the moment. The core was released for testing it seems and there will be future projects with it? This is good! Hopefully they will build in support for older GLIBC's if it's possible. I'll keep poking around at it to try and find a combo that works on "older" OS's.

-Kirk

if you are not able to upgrade the host os, perhaps create a virtual machine with the required OS. Back when the gpu client was windows only, we used to run GPU client on windows host and the CPU client in linux vm.