This is only one possible pathway. There are many potential ones. Further the fact that this is possible does not mean that it is the case always.
Stress activates the hypothalamic-pituitary-adrenal (HPA) axis. For example, a meta-analysis by Dickerson et al. (2004) demonstrated that an acute laboratory based stressor reliably increased cortisol levels, a hormone downstream of the HPA. That stress results in activation of the HPA is also well-demonstrated in animal models, where both physical (e.g., restraint or shock) and social stress (e.g., Engelman, 2004).
When stress occurs, various systems (e.g., amygdala implicated in threat detection) can lead to activation in the hypothalamus, where the paraventricular nucleus (of the hypothalamus) secretes corticotropin-releasing hormone (CRH). The CRH acts on the pituitary gland resulting in a release of adrenocorticotropic hormone (ACTH). ACTH is released into the blood, which allows it to reach the adrenal glands, specifically the adrenal cortex. This stimulates the adrenal cortex to produce and release glucocorticoids (GCs) into the blood (in humans primarily cortisol, in animals corticosterone). GCs in turn have broad ranging influences on the body. For example, they seem to play role in immune regulation and function (although whether they are anti-inflammatory or pro-inflammatory seems to depend on quantity and time (there was a lively debate about this at a Psychoneuroimmunology conference. For utterly inadequate notes on some of the presentation see here).
GCs also act in a negative feedback loop to shut off the HPA axis. Finally, relevant for cognitive functioning, specifically memory, GCs can regulate the hippocampus. This is believed to be adaptive in the short term, but when the hippocampus is chronically exposed to elevated levels of GCs, it can endanger cell function (for a review of GCs and the hippocampus, see Joels, 2001).
For example, using a longitudinal design, Huang and colleagues (2009) demonstrated that in people with Alzheimer's Disease, basal plasma cortisol (cortisol levels in the blood, other major ways to measure cortisol are via saliva or urine) was predictive of a decline in cognitive function as well as atrophy in the hippocampus (specifically, the temporal horn).
Somewhere I have a slightly longer reference list on stress and aging, with an emphasis on stress effects in cognitive and physical function over time. I do not have an answer for your final point, what can be done to minimize the effects. The only clear suggestion would be to minimize stress. I know there are some studies showing protective effects of physical activity for depression, physical function (that one's a no-brainer), and cognitive function. One can speculate that if chronically elevated GCs released from the adrenal cortex are the cause, that administering an ACTH antagonist or the like may help, but to date, I know of no studies trying to block the effects of stress (in rodent models of cancer, stress effects on tumor progression and metastasis have been effectively blocked using beta blockers, but there the primary stress pathway was believed to be sympathetic nervous system activation rather than HPA).