A friend of mine freaked out after taking LSD for a while. She is fine now (it happened twenty years ago), but she was really scared - especially as there can be flashbacks, so you can not just opt out of not experiencing trips anymore.
Flashbacks are very rare, possibly even mythological. (A psyconaut friend of mine has reported only one 'flashback'-- it was a fraction of a second on the bus when she thought her hair had turned into a snake.)
