I get that thought, but it would be like not funding a road in your town because you don't drive down it.
Schools and education improve everyone. Under educated kids turn into adults. Adults you will have to work with and along side, be it flipping your burgers to building your fences.
Then when you have kids, its not like the money you start to pay instantly makes the schools better. Poorly funded schools don't get updated, don't get new books, or even air-conditioning.
I know this because here in Oregon, we passed a proposition in the mid 90's that limited funding (via tax collection). It made it so our schools are severely underfunded. I was in high school then, it did not do anything to me. But now that I have kids, the school they are in doesn't have AC and the heat works so poorly they all have to wear their winter clothes in the class room. The school was built in the 60s is and has been at capacity for years but no money to improve it.
The teachers are wonderful and do what they can.
We are ranked 40-45th out of all 50 states. Its embarrassing.