I tagged dreams here too because I think that's a prominent characteristic of REM sleep that might help you judge this. If I'm wrong, please remove the tag...but I don't think people remember dreams as often when they wake during non-REM sleep. That might be one cue, though I'm not certain of this.
Another idea off the top of my head would be recording yourself. Since REM = "rapid eye movement", you could probably tell whether you were in that stage before awakening if you trained a digital video recorder on your face (e.g., over your head, attached to a bedpost or headboard, aimed down at your face), and happen to sleep facing up sometimes. Rapid eye movements are noticeable through eyelids (kinda looks like a cat moving around underneath bedsheets—the raised hemispherical shape shifts around a lot), so given a decently high-resolution video recording of yourself, you could probably check this upon waking, or whenever afterward. If you don't see much eye movement, you might guess that you're not in REM sleep, though I'm not 100% sure you could make this judgment without error by yourself.
A relatively foolproof third option would be to sign up for a sleep study! I believe they routinely monitor brain waves that are more definitional, physiological markers of sleep stages. That might cost you, though.