Two things, first it's not stupid and second no you should not.
That capacitor is not just for radio noise, it protects your whole electrical system from flyback voltage from your ignition coil.
The old canister type electrolytic capacitors in most cases have failed by the time ten or twenty years have gone by and you'll notice that usually that's "ok" in the same sense that usually it's OK to ride a motorcycle without gloves and a helmet, right up until it isn't.
Ford part number 7L1Z-18801-A replaces the canister type capacitor with a solid state unit with a lower ESR and longer lifetime. It's not free but it's cheaper than going nuts trying to track down computer problems. Different connector but you can depin the connector using standard depinning tools and crimp or solder your old connector on.
