Optimize my Opteron dual core rigs

[moz_21]

I'm looking to get more PPD out of these 2 rigs. Currently running SMP under Ubuntu 9.10 64bit. They are both Opteron dual core rigs, one at 2.75GHz and one at 2GHz. Both have 2GB of ram.

FahMon shows the 2.75GHz at approximately 1500PPD and the 2GHz at approximately 1200 PPD.

I installed FAH with fah_install

I changed checkpoints to 30 minutes, set bigpackets and set the -advmethods flags under advanced options when asked. However, I don't see the -advmethods flag in my client.cfg and my FAHlog.txt only shows the -smp option.

My client.cfg for both:

[settings]
username=moz_21
team=32
passkey=
asknet=no
bigpackets=big
machineid=1
local=1

[http]
active=no
host=localhost
port=8080

[core]
checkpoint=30

[clienttype]
type=3

The beginning of my FAHlog.txt showing the -smp option:

--- Opening Log file [December 28 04:21:32] 


# SMP Client ##################################################################
###############################################################################

                       Folding@Home Client Version 6.02

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: /opt/foldingathome/1
Executable: /opt/foldingathome/1/fah6
Arguments: -smp 

[04:21:32] - Ask before connecting: No
[04:21:32] - User name: moz_21 (Team 32)
[04:21:32] - User ID: 13A722767B5E3E49
[04:21:32] - Machine ID: 1
[04:21:32] 
[04:21:32] Work directory not found. Creating...

etc................

So what can/should be changed to improve performance?

 

[ChasR]

THis is where Ambient Fiction is supposed to pipe in and tell you to add "export MPICH_NO_LOCAL=1", without the quotes, to your start up script. The easiest way to do it is to create a shortcut on the desktop with gedit with the following script:

export MPICH_NO_LOCAL=1
cd ~/folding
./fah6

THis assumes you have the fah6 executable in a folder named folding and that you have added -smp -verbosity 9 to the extra parameters of advanced config. Edit: Looking at your log, you did neither, so you'll have to adjust the path and put the flags in after ./fah6 if you don't want to rerun config.

All SMP WUs are big so that setting has no effect when -smp is specified. You may get fewer a1 core WUs with -advmethods, which is denoted by type=3 in the client.cfg file.

 

[moz_21]

THis is where Ambient Fiction is supposed to pipe in and tell you to add "export MPICH_NO_LOCAL=1", without the quotes, to your start up script. The easiest way to do it is to create a shortcut on the desktop with gedit with the following script:

export MPICH_NO_LOCAL=1
cd ~/folding
./fah6

THis assumes you have the fah6 executable in a folder named folding and that you have added -smp -verbosity 9 to the extra parameters of advanced config. Edit: Looking at your log, you did neither, so you'll have to adjust the path and put the flags in after ./fah6 if you don't want to rerun config.

All SMP WUs are big so that setting has no effect when -smp is specified. You may get fewer a1 core WUs with -advmethods, which is denoted by type=3 in the client.cfg file.

I edited /etc/defaults/foldinathome to include the -verbosity 9 and that's showing up in the FAHlog.txt now. I'm not sure how to add the export command to my startup script. It also appears I cannot run export as root, which is how I have FAH installed. Take a look at it:

#!/bin/sh
#
# Original version by Jonathan Kotta <jpkottaATgmailDOTcom>

# init script for the Linux Folding@Home client

PATH=/bin:/usr/bin:/sbin
. /lib/lsb/init-functions

# vvv these variables are modified by the install script vvv
fah_dir=/opt/foldingathome
executable=fah6
# ^^^ these variables are modified by the install script ^^^

config_dir=$fah_dir/config
# typically, this is /var/run, but users don't have access to that
pid_dir=/var/tmp

if [ x"$UID" = x0 -o x"$USER" = xroot -o x"$USER" = x ] ; then
    user=foldingathome
else
    user=$USER
fi

# there must always be at least one processor
test -x $fah_dir/1/$executable || exit 1

CLIENT_OPTS=''
if [ -f /etc/default/foldingathome ] ; then
    # system-wide defaults
    # per-client defaults can be set as well
    . /etc/default/foldingathome
fi

################################################################################
start()
{
    # this is a work around for a bug in start-stop-daemon
    # Debian Bug #222524
    if [ x"$USER" = x"$user" ] ; then
        user_flag=''
    else
        user_flag="--chuid $user"
    fi
    
    num=0
    for client_dir in $fah_dir/[1-9] ; do
        num=`expr $num + 1`
        cd $client_dir
        log_begin_msg "Starting Folding@Home client $num"

        # per-client options
        # the client?.options file is just like the defaults file
        options_file="$config_dir/client${num}.options"
        if [ -f $options_file ] ; then
            . $options_file
        fi

        start-stop-daemon --start --background --quiet \
            --chdir $client_dir \
            --nicelevel 19 \
            $user_flag \
            --exec $client_dir/$executable \
            --make-pidfile --pidfile $pid_dir/$executable-$num.pid \
            -- $CLIENT_OPTS

        err=$?
        if [ x"$err" = x0 ] ; then
            log_end_msg $err
        else
            log_end_msg $err
        fi
    done
}

################################################################################
stop()
{
    num=0
    for client_dir in $fah_dir/[1-9] ; do
        num=`expr $num + 1`
        log_begin_msg "Stopping Folding@Home client $num"
        
        start-stop-daemon --stop --quiet \
            --retry 10 \
            --user $user \
            --exec $client_dir/$executable \
            --pidfile $pid_dir/$executable-$num.pid 
        
        err=$?
        if [ x"$err" = x0 ] ; then
            rm -f $pid_dir/$executable-$num.pid
            log_end_msg $err
        else
            log_end_msg $err
        fi
    done
}

################################################################################
send()
{
    num=0
    for client_dir in $fah_dir/[1-9] ; do
        num=`expr $num + 1`
        log_begin_msg "Sending Results for Folding@Home client $num"
        
        su \
            --shell /bin/sh \
            --command "cd $client_dir/ ; ./$executable -send all" \
            -- $user

        err=$?
        if [ x"$err" = x0 ] ; then
            log_end_msg $err
        else
            log_end_msg $err
        fi
    done
}

################################################################################
status()
{
    for client_dir in $fah_dir/[1-9] ; do
        echo "Logfile in $client_dir:"
        tail $client_dir/FAHlog.txt
        echo
    done

    if is_running ; then
        echo "Client is running."
    else
        echo "Client is not running."
    fi
}

################################################################################
is_running()
{
    num=0
    running=1 # return status is like in C, zero is successful
    for client_dir in $fah_dir/[1-9] ; do
        num=`expr $num + 1`
        if [ -e $pid_dir/$executable-$num.pid ] ; then
            running=0
        fi
        if ps -ef | grep -vE "grep|ps" | grep "$executable" >/dev/null ; then
            running=0
        fi
    done
    return $running
}

################################################################################
case "$1" in
    start)  
        start 
        ;;
    stop)
        stop
        ;;
    restart|force-reload|reload)
        # These are technically supposed to do different things.
        # In particular, reload means reload the configuration without stopping
        # the program, which is not possible in this case.
        stop || exit 1
	# F@H has trouble starting immediately after stopping
	for i in $(seq -w 60 -1 1) ; do 
	    echo -n "Restarting in $i seconds."
	    sleep 1
	    echo -ne \\r
	done
	start
        ;;
    send)
        # manually send all results to the server
        # leave client in the state (running or not) as it was found
        if is_running ; then
            stop
            while is_running ; do
                sleep 1
            done
            echo
            send
            echo
            start
        else
            send
        fi
        ;;
    status)
        status
        ;;
    *)
        echo "Usage: $0 {start|stop|restart|reload|force-reload|send|status}" >&2
        exit 1
        ;;
esac

exit 0

 

[ChasR]

You could enter the export command just about anywhere, I'd put it in the line below PATH=

THis is the first time I've seen anyone follow the Ubuntu guide.

 

[moz_21]

You could enter the export command just about anywhere, I'd put it in the line below PATH=

THis is the first time I've seen anyone follow the Ubuntu guide.

I tried it right below the path and while the script executes, how do I know that it's done the export?

Does nobody use the Ubuntu guide? I find it easy, just download fah_install, unpack, execute the install script, answer a couple questions and BAM, it runs on startup.

 

[ChasR]

cat /proc/cmdline should showall the variables loaded. Hopefully MPICH_NO_LOCAL=1 appears in the list.

 

[moz_21]

cat /proc/cmdline should showall the variables loaded. Hopefully MPICH_NO_LOCAL=1 appears in the list.

user@phenom2:~$ cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-2.6.31-16-generic root=UUID=3741c794-7396-41e7-88ed-4b1c54555c40 ro quiet splash

Whereas a plain old export will show "declare -x MPICH_NO_LOCAL=1" but only if I run "export MPICH_NO_LOCAL=1" before calling "export".

 

[ChasR]

Ahh on notfred's distro, the cmdline lists all variables. It doesn't do that in Ubuntu. Use printenv instead.

 

[moz_21]

Ahh on notfred's distro, the cmdline lists all variables. It doesn't do that in Ubuntu. Use printenv instead.

Ok I did sudo -s -H then executed export MPICH_NO_LOCAL=1 and get:

root@mythbuntu:i~# printenv
MPICH_NO_LOCAL=1
SHELL=/bin/bash
TERM=xterm
USER=root

However if I log out then sudo -s -H and run printenv again it disappears.

I guess what I need is a way to load the export and process at the same time.