I grow them. It's not very difficult if you live in a hot climate.
Edit - Here's what I have going this year. We had a late start to summer so these things are barely getting going in terms of producing fruit, but there are a couple of "ghost chilies" as you call them in there if you look closely. I'm growing Bih Jolokias, Trinidad Habaneros, Douglahs, Trinidad Scorpions, and Moruga Reds.