Peppers are POKEY!! Hot peppers even more so. I'd give them 3 weeks, at least.The cayenne pepper seeds I got from the seed library didn’t sprout after two weeks, at a guess they were just too old.
I've had some sprout after 4 weeks. Probably the soil was too cool; they're picky. And pokey.