does that means that the only way is to manually change those physical jumpers?
Depends on the motherboard. The later ones of that vintage allowed you the choice of doing some things manually with the physical jumpers or doing all adjustments through the bios. I think once you got into the boards using DDR2 memory the latter arrangement prevailed.
What you need to do is to get a handle on the things you need to change in order to overclock (ask that question first) and once you are settled on that determine if you can manipulate all those things in bios or if some of them can only be done with jumpers. At the very least you would want to be able to:
1. Change the FSB (aka, CPU Frequency, HT Reference)
2. Memory speed (you would need to be able to lower it say, from 800 mhz to 666 to create some headroom for overclocking.) Memory speed dividers may look like "x4 or x5.33" instead of being expressed in frequencies.
3. CPU voltage.
It would also be helpful if you had the option to lower the NB frequency (or CPUNB frequency, not the chipset NB) in order to create some overclocking headroom for that component.
If you want, take a digital pic of the relevant sections of the bios and attache them with a post so we can help you identify and sort out some of these things. Bios terminology varies tremendously from one board manufacturer to another so that can be a challenge.
To attach a pic, click on Go Advanced at the bottom of any new post window and then click on the little paperclip icon at the top of the advanced post window where the tools are.