Critter-proofing a garden like that would be BRUTAL. The only successful "deer-proof" fence plan I've ever encountered requires that the top of the fence be 8 feet above ground, and that the fence itself be tilted outward from its base at about 30 degrees short of vertical. That seems to be the only fence deer won't try to jump, because sure they can make the 8 foot jump up, but they can't tell if they can clear the tilted interior.
For rabbits and ground squirrels and skunks and the like, you'll need to dig out about 2 feet of soil and make a box inside that hole. Said box needs to be heavy-duty plastic-coated wire and has to cover the entire bottom and all 4 sides, plus sticking up at least a foot above ground at the base of the fence. Then you fill it back up with dirt to actually grow the garden. Leave off the low fencing or use uncoated wire and rabbits, ground squirrels, and similarly-sized garden destroyers can and will chew right through. The mesh should be under 1 inch to keep moles and rats out, because anywhere a rat can stick its head, its whole body follows. As for mice, your best bet is don't kill any snakes in your yard--many of the venomous AND non-venomous species in the US live on rats & mice. Heck, corn snakes are rumored to have gotten their name hanging around in corncribs and silos to eat the mice and rats that invariably swarm such places.
If I ever win the lottery, tho, I'm having a massive glassed-over "courtyard" garden and I'll be referring back to this plan!
