This question is rather hard to answer. I guess the best thing to say to this question is a staunch “maybe”. It all depends on how well you are taking care of the conditions inside your mouth. Healthy teeth are, paradoxically, not just a result of keeping your teeth clean, although that is the most basic part, without there are no clean teeth. But your teeth can become infected diseased and cavity ridden if you do not clean the periodontal tissues in your mouth, or the apparatus in your mouth, like orthodontics, and yes, retainers are orthodontics.
That being said there is no specific material or chemical reaction that makes tooth decay more likely in people who wear retainers as opposed to people who do not, so technically, no, wearing a retainer does not make you more likely to experience tooth decay. What it can do is trap more food debris that feed bacteria that cause tooth decay. So the golden rule of any orthodontics is brush more often and spend more time flossing and taking care of your teeth, and you should have no problems.