You'll be interested in Where Mathematics Comes From, by George Lakoff and Rafael E. Núñez. Lakoff & Núñez argue that all of our mathematical concepts are based on some kind of conceptual metaphor, where we take the rules of a domain that we understand intuitively and apply them into a new, mathematical context. For example, the concept of "motion along a path" gives us the concept of a number line, which in turn provides a natural way to think about negative numbers:
Moreover, this metaphor provides a natural extension to negative numbers - let the origin be somewhere on a pathway extending indefinitely in both directions. The negative numbers will be the point-locations on the other side of zero from the positive numbers along the same path. This extension was explicitly made by Rafael Bombelli in the second half of the sixteenth century. In Bombelli's extension of the point-location metaphor for numbers, positive numbers, zero, and negative numbers are all point-locations on a line. This made it commonplace for European mathematicians to think and speak of the concept of a number lying between two other numbers - as in zero lies between minus one and one. Conceptualizing all (real) numbers metaphorically as point-locations on the same line was crucial to providing a uniform understanding of number. These days, it is hard to imagine that there was ever a time when such a metaphor was not commonly accepted by mathematicians! (Lakoff & Núñez, p. 73)
Lakoff & Núñez explicitly argue that all of human mathematics is created by the application of human-specific cognitive machinery and culture, and that the mathematics of an alien species might look very different.
The following references might or might not be useful for purposes of metamathematics, but mentioning them anyway since I happen to have them at hand from a course on number cognition that I took some years back: Piazza et al. (2007) and Brannon (2006) on how numbers are represented in the brain, Carey (2004) on how children come to learn the concept of integers, Frank et al. (2008) on numbers as a cognitive technology, and Gallistel et al. (2006) for an argument that real numbers rather than integers are what the brain represents.
Personally I also think that an extraordinarily interesting paper is Dehaene & Cohen (2007), which argues that the existence of specialized neuronal circuitry for purposes such as arithmetic and reading is possible because our culture has managed to "hijack" pre-existing circuits which originally evolved for other purposes:
The neuronal recycling hypothesis consists of the following postulates:
Human brain organization is subject to strong anatomical and connectional constraints inherited from evolution. Organized neural maps are present early on in infancy and bias subsequent learning.
Cultural acquisitions (e.g., reading) must find their “neuronal niche,” a set of circuits that are sufficiently close to the required function and sufficiently plastic as to reorient a significant fraction of their neural resources to this novel use.
As cortical territories dedicated to evolutionarily older functions are invaded by novel cultural objects, their prior organization is never entirely erased. Thus, prior neural constraints exert a powerful influence on cultural acquisition and adult organization.
If you accepted these postulates, then arguably the neural constraints of the older circuits might considerably bias the way that human mathematics is carried out.
Brannon, E.M. (2006) The representation of numerical magnitude. Curr Opin Neurobiol. Apr 2006; 16(2): 222–229. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1626588/
Carey, S. (2004) Bootstrapping & the origin of concepts. Daedalus, Winter 2004. http://hum.uchicago.edu/ck0/kennedy/classes/s09/experimentalsemantics/carey04.pdf
Dehaene, S. & Cohen, L. (2007) Cultural Recycling of Cortical Maps. Neuron, Volume 56, Issue 2, Pages 384–398. http://www.sciencedirect.com/science/article/pii/S0896627307007593
Frank, M.C. & Everett, D.L. & Fedorenko, E. & Gibson, E. (2008) Number as a cognitive technology: Evidence from Pirahã language and cognition. Cognition 108, 819-824. http://www.letras.ufrj.br/poslinguistica/recursion/papers/18-frank-et-al.pdf
Gallistel, C.R. & Gelman, R. & Cordes, S. (2006) The Cultural and Evolutionary History of the Real Numbers. Culture and evolution. Cambridge, MA: MIT Press. http://hum.uchicago.edu/ck0/kennedy/classes/s09/experimentalsemantics/gallistel-etal05.pdf
Lakoff, G. & Núñez, R.E. (2001) Where Mathematics Come From: How The Embodied Mind Brings Mathematics Into Being. Basic Books.
Piazza, M. & Pinel, P. & Bihan, D.L. & Dehaene, S. (2007) A Magnitude Code Common
to Numerosities and Number Symbols in Human Intraparietal Cortex. Neuron 53, 293–305. http://www.sciencedirect.com/science/article/pii/S0896627306009895