Thanks for the link bythegods. Unfortunately, I have already tried everything there. Here is the current situation:
Drivers tried:
ATI 3.10.......no problems
Omega 3.10 softmod.......bsod when 3d apps start
Omega 3.9 softmod.......bsod when 3d apps start
Omega 3.8 softmod.......bsod when 3d apps start
Omega 3.7 softmod.......bsod when 3d apps start
Fast writes disabled and enabled for all drivers, no change.
VPU recover enabled bsod with softmod drivers.
VPU disabled no bsod, but definately a problem. I get something that looks like the same small piece of video written several hundred times on the screen and the error message mentioned above on reboot.
Used Cat uninstaller then hunted down all references to ATI and deleted them.
Went into the registry and removed all ATI references. (got a little click happy there and wiped out all my basic video drivers too, now I can't install any drivers for the card, windows disabled it, so now I have to hunt down my XP disk and repair it.
I also tried the patched ati2mtag file coltice refered to above. Same results.
I turned down the overclock to 11*200, used a different molex connector, and upped agp voltage to 1.8, and 2 volts.
I also upped Vcore to 1.825.
I tried with and without hyper and heir and even lowered the agp to 4x. I also tried AGP aperture at 64, 128, and 256.
I also redownloaded and reinstalled directx9.0b and ran dxdiag. A funny thing there: dxdiag finds no errors, direct draw works perfectly, but during the direct3d tests, the spinning cube looks like it's spinning at mach one and appears wobbly. This obviously leads me to believe that it must have something to do with direct3d.
Keep in mind, the card works great when not modded, even in 3d apps, but this problem has me intrigued.
I am running out of ideas here, and I am also starting to think that it's the card. I have never heard of a card soft modding and looking great in 2d but then having this problem in 3d. I am giong to work on it the rest of the day, and if I can't resolve it, I'll go back to the standard 3.10 drivers and be happy. After all, I got exactly what I paid for