Yes the cache on the P4's is higher, but that's not the most important gain that you get from upgrading to a P4.
When you have a 2.4C, you start out with a 200 FSB (800 effective) which is quite a bit faster than the Celeron's if I remember right.... and if you oc the 2.4C to 3.4 gigs ( 3.2-3.4 is a reasonable oc on the 2.4 with good cooling), your FSB jumps to 1150 Mhz... which is significantly higher than the Celeron's FSB.
To properly oc the 2.4's you have to have several things though.
1) Good cooling... i.e a Thermalright or equivalent HS ( SLK-948 or SP-94) with a medium to high performance fan ( Panaflo H1B, Vantec Tornado, Delta HHE or EHE).
2) Some relatively fast ram... DDR 433 or 466 ( or faster if you can afford it). You can change the ratios if you have slower ram, but your performance is better if you can remain at a 1:1 FSB/RAM speed ration for as long as possible. This isn't that important for most things b/c the higher FSB you gain from overclocking usually outweighs the loss that you get from using ratios. If you use memory intensive programs though, you'll probably notice the loss quite a bit more.