No, you can't use FFTs smaller than 8k or larger then 4096k (though you can choose any values in between). You can tell Prime to use as much of your system memory as you want. But don't give it the full amount of physical memory. If you have 512M use ~430-450M, if you have 1024M, use ~900M, etc.