[mccoyn]

There is some discussion about developing a stress test program in another thread. That thread is mostly centered around adapting a program which calculates pi to be a stress test. The purpose of this thread is to discuss another approach, in which we write a stress test program from the gound up.

Prime95 and Super_pi_mod are both programs designed to solve a complex mathimatical program. They happen to have two qualities that make them good as a stress test. 1) They push the processor, putting it under more stress than most other programs. 2) They validate their results so that if the processor makes an error it will likly be detected. My problem, though, is that they arn't designed for these things. I suspect that a concentrated effort to satisfy these two goals will do a better job than just finding the program that happens to satisfy them best.

I'd like to get ideas about developing this program to make it the best possible. I'd also like to try and raise volunteers to work on it. This is a big project and the more people working on it the better. There are plenty of different coding jobs as well as non-coding jobs to do.

Here are some brainstormed ideas I've got (some of them may have been brought up by others.)

- The basic idea is to write a loop that can be rerun many times stressing the CPU. In the course of this loop there should be checks to determine if any errors occured.

- Errors should be reported. It would be nice if it could keep running and report an error count. I think thats a dangerous approach since you are overstressing the CPU and if it switches to an OS thread and errors you could get corruption.

- A boot disk, that allows it to run without loading an OS or any hard disks would protect the system while running stress tests.

- Lots of different loops should be developed. I'd like to cover every opcode in some loop, so that that operation can be tested under stress. Also there is the possibility of switching between different types of opcodes. A bitwise and operation would put the control circuitry under stress. A floating point divide operation would put the ALU under stress. Switching between these, might keep the CPU under more stress than doing just one.

- With lots of loops we need an ability to test each one and decide which stresses the CPU most. The only way I can think of measuring stress is with the CPU temp. So, we need a way to check the CPU temp and calculate a stress value. Once we decide on a most stressful loop we can use that primarily and switch in other loops to test different operations under stress.

- With so many loop possibilities and so many different processors it might be better if we could write a program that can write the loops then run them. This can't be an interpretive thing, we have to write a program that writes machine code directly.

- I would like to do this in Qt. This would give us the ability to create an MS Windows version and a Linux version for a boot disk. I think C++ is a good language because we can embed assembly directly in it (I'm not sure how well this crosses platforms.)

This is, of course, a big project. Several tasks need to be done.

- Research to find white papers for processors that describe opcodes and their implementations as well as when NOOPS are inserted. People familiar with the technical details for processors (or willing to get familiar) would be good for this.
- Assembly coders.
- GUI coders and designers.
- Coders for maintaining the port between Linux and MSWIN.
- Documentation.
- Testing on many processors (or just buy me a bunch!)
- Linux distribution and ISO builder.

What I would like to hear is further ideas for features and people who would be willing to help and in what capacity.

[Sucka]

Sounds like a good plan you've got brewing. I wish i was versed in coding, but alas i know nothing about it.

If i may add one thought..the thing i hate most about strees testing programs is the fact that there is essentially no way to tell what is causing the errors. I wish there was a test that at the very least could blend both CPU and memory together and determine which piece of hardware is causing the failure. Sure there is Memtest for testing memory, but that too can be influenced by CPU. I also don't like the idea of it being bootable. In theory that's a good idea, but as with Memtest, you can be fully stable there, and that doesn't always translate into stability within windows. I am probably asking/expecting to much out of a stress test program, but those are the things that annoy me.

In any case, good luck with your project. Hope it works out well for the community. With all the Prime freaks we have around here this should go over very well

[mccoyn]

I'd like to create two versions. A linux version that can be put on a boot disk and a windows version that will run under the OS. For the windows version, I want it to stop when it encounters an error instead of continue and count the number of errors. This is to protect the OS. I've noticed that sometimes Prime95 causes a BSoD. I suspect this is because the OS switches to a system thread and an error occurs there. This is potentially damaging and I'd like to avoid it.

I had been thinking of focusing solely on the processor and not working on memory at all. That could be a later addition. The problem, with a non-boot memory test is that the OS is managing and remapping memory, which prevents programs from getting the direct access needed to test it properly. Even a program running off a Linux boot disk has this problem (though we could modify the kernal.) Anyways, my knowledge in this area is less and we would need someone else to do most of that work. If we wait for me to learn about it, it will only be after the CPU stress portion is complete.

I had thought about testing to see which operations break down and if they are repeatable. This is a lower level and probally wouldn't be helpful to anyone trying to overclock.

Thanks for the ideas, keep them coming.

[macklin01]

I'm quite interested! Maybe we could find some good fluid mech problems to test with. Level set problems can be very CPU-intensive. Also, a looping suite of apps might be good. Image processing could be good because you could always do a pixel-by-pixel comparison with a known good result. Fast Fourier transforms could be good for testing, too. As would prime factorizations, etc. They don't even have to be efficient operations: just CPU-intensive. Matrix-vector products could be good, and you could check the results by subtracting off known results and checking that error is near machine zero. (This is a problem you'll encounter when checking for errors on floating point operations on any sort of complicated problem.) Also, matrix inversion problems would be good and intensive.

As for mmx/sse/assembly code, I have no idea. Would it be terrible to use compiler optimizations instead? If you're trying to test on a complicated thing

As for memory, I agree that it should be a separate problem. In fact, memtest86 pretty much does what you would like it to.

[deathstar13]

ill volenteer any help needed in testing. i cant code or anything near that.
one idea i have if you want the bootable version is maby incorperate it into the ultimate boot disk? just an idea.

[mccoyn]

Yeah, I kind of thought we would include other tools useful for stability testing and overclocking on the boot disk. I wonder if we could put a boot loader like Grub on a CD and run memtest86 from it. I think, however, that a boot disk is a long way off.

macklin01, I kind of wanted to stay away from pre-existing math problems and focus 100% on writing code that stresses the CPU and verifies itself. First of the code would have to be very well optimized. I think if we allow ourselves to very the actual instructions and what actually gets calculated we can get more optimized than just changing the order a set calculation is done. More flexibility to allow for a better solution is the idea.

[diehrd]

Home grown stress test ? When I read that, my first thought was get married ! ! LMAO

[macklin01]

Interesting!

So, are you looking more do lots of adds / subtracs / moves / etc? The nice thing about a math problem is that it will have a known answer against which to check for errors. But I suppose that we could still have a saved solution anyways for any set of operations. You could certainly imagine having different "savepoints" where the result is known at various stages and could be computed at those steps. If the test is small, it wouldn't be hard at all to have these "savepoints" all in memory to easily check for errors.

It would be good to have some tests that fit entirely in L1 (if possible), since those will run the fastest and put a lot more stress on the CPU; spending less time waiting to get info from the L2 cache, it can spend more time executing. Likewise, to stress test the L2 cache, it would be good to have problems that fit entirely in L2 cache.

In a way, it might be nice to limit to a bootable floppy, rather than run in Windows. Then the test won't need to compete with the OS threads as much, and thereby spend a greater percentage of the time stressed. Also, it might be good to have both single-threaded and double-threaded apps. I'd imagine 2-threaded would better test P4's and dual-core chips, but worse test others, since some execution time would be diverted to thread switching, etc. (Then again, I'm not by any means experienced on threading issues.)

This is always been something fun I've wanted to do, so it's very interesting to talk about it! -- Paul

@diehrd: I believe a "home grown stress test" is referred to as a child. ;-) A teenager is a special case.

[mccoyn]

Small tests that fit in the L1 is definantly the idea. Of course, caches do generate heat so involving the caches is at least worth exploring. I was thinking of using small sets of operations that output the same numbers they input. Like a * b / a = b. Then using random numbers or long series of operations that can be calculated quickly like doing a*b / a, except do a sumation in place of the multiplication. Really thats the tricky part. Write some code that will maximumly stress the CPU while still providing checkpoints to verify operation.

If we do things right (lots of non-dependant instructions of different types) the P4 should get just as much stress with one thread as with two. Multi-threading is worth implementing if just for the dual core systems. Also we should try it on the P4 to be sure.

"Home grown stress test." I don't have a wife or kids, so that explains why I have time I guess. I did have trouble coming up with a descriptive name for this thread. I will probally have the same problem coming up with a name for the project. Any suggestions for a name would be appreciated.

[macklin01]

Very cool!

I like that idea. One thing, though: the output and input are only guaranteed equal in some cases. Like (a*b)/a should be fine, since it could be an integer operation, and a divides a*b. But (b/a)*a, which would have to be a float, may or may not equal b in floating point arithmetic. So, exact matching may be a problem for floating point ops. If you were to decide to finish with a ceil and convert to int, then you should be able to check errors. (I suppose that checkpoints should help avert these problems, but the skeptic in me questions whether or not all CPU architectures would round the same way, giving rise to possible false positive errors.) We definitely can't just restrict to integer ops, since (to the best of my very limited knowledge) they are computed on different parts of the CPU, and so a true stress test needs to stress both types of operations. (Right?)

I really like your ideas. It makes a great deal of sense and is easy to test. (And easier operations would be a lot easier to code in assembly and/or convert for SSE/SSE2/SSE3/MMX.) I wonder if it might be worthwhile to just focus on newer processors, say just those with SSE2 and above, and whatever AMD's version is. (3dNOW!?)

"Home grown stress test": ah, don't worry: everyone was just kidding around there. Maybe a shorter name would be good, but you are the project lead/creator/owner, so you definitely get to do fun things like naming! Are you planning on submitting something to sourceforge later on?

-- Paul

[DaWiper]

I know only know one person(local) that know assembler and he told me that the fastest way in assembly language to do simple multiplication/division is bitshift left/right. Bitshift is a native assembler instruction in almost any cpu.
I would think a stressfull test would be to do something like:
starting value: 00001111(=15)
bitshiftleft one step
value would then be :00011110(=30)
bitshift one more time
value would then be :00111100(=60)
etc.
Repeat until the number is big enough.
Then do bitshift right the same amount of times.
The idea is that if everything is calculated right the starting value should be equal to the ending value.

Another thing is large floating point values. I tried with doublefloat. values(64-bti=18 digits) in blitz max. Either it wasn't enough or the loop was to slow to push the cpu over the cliff. Not sure, but as both prime and superpi operates with much much larger float. values you would probably need very large ones.

[RJARRRPCGP]

Originally Posted by mccoyn

I've noticed that sometimes Prime95 causes a BSoD.

It's not majorly surprising to see that behavior occur when the processor is real-bad unstable.