Currently those outside the public-school system pay the tax and do not receive the government benefit of a taxpayer funded education. The important point is that those outside public schools SHOULD EITHER be out of the tax base, or receive the benefit of a government funded education at the same dollar level as those in the public-school system in the form of a voucher, or better-yet, an education spending account.
Many will complain about fixed costs of a school and worry that enough students and the money that comes with each child will be there to properly fund that school. That complaint is always there, regardless of school choice. If a town losses population, thus also student population, their education funding is reduced to match.