After asking the question What is the most complex neural network... I realized I don't really have a good metric of "complexity" in a general sense. The simplest measure would likely be count of neurons or number of synapses, but that fails to take into account the structure of the network.
A couple measures of complexity are discussed in the paper Complexity of Predictive Neural Networks but they are very specific to a single task. One is the amount of work needed to learn a certain thing, and the other is how many neurons are needed to approximate a certain function.
Rough, animal based measures are often employees for the sake of grabbing headlines; such as the incorrect claims that The Blue Brain Project had emulated a neural network "as complex as" a cat's brain. C. elegans is a common and seemingly attainable level of complexity for an artificial neural network.
Animal based measures are relateable to the layman but seem questionable, especially when comparing a neural network to that of an animal who's neural network has not been completely mapped (as C. elegans has).
What is a meaningful measure by which artificial neural networks can be measured? How are such networks currently compared? Can any such metric appropriately measure complexity of such a system?