My father pointed out to me that the Targum Yonason Ben Uziel writes that tzitzis and tefillin are considered bigdei ish and are part of the issur of "Kli Gever". Most of us are aware of the claim that Rashi's daughter's wore tefillin. However, even if this is not true we do find in teh gemara that Michal bas Shaul wore tefillin and the chachamim did not protest. Furthermore, the Rambam paskens (Tzitzis 3:9) that women are allowed to wear tzitzis if they deisre. It would seem that the gemara and the Rambam argue on the Targum Yonason Ben Uziel.
Perhaps the machlokes is based on how to view tefillin. In general the issur of lo silbash kli gever is limited to when one wears the clothes as a malbush or tachshit. For example, if a man would wear a woman's raincoat to protect himself from the rain then there is no issur-he is wearing it for protection and not as a malbush/clothing. Similarly, a man can wear his wife's ring in order to keep it safe. (l'moshel-his hands are full and he has no pockets so the only safe place is on his finger). Again he is not wearing it as jewelry so it is mutar.
The question might be how do we view tefillin? Is it considered a tachshit or an article of clothing or is it something that we wear only to perform the mitzvah and it doesn't have the status of clothing or jewelry. Furthermore, even if one argues that it is a tachshit, maybe that is only for someone who wears it all day. But if you put it on solely to perform the mitzvah, maybe it is not considered wearing a tachshit.
In preparing my shiur this week (yes the chaburah zman has officially started up again), a came across a Maharshag in Chelek 1 Siman 36. He is discussing whether there is an issur hotza'ah on Shabbos if you wear something that is assur to wear-for example tefillin. He writes that tefillin in of themself are not considered a malbush or tachshit. We don't find that non Jews wear them. However, during the week it would be a tachshit for Jews who wear them for the mitzvah. However, this is only when one wears them for the mitzvah. If one does not wear them for the mitzvah-like on Shabbos then it would not be a tachshit but would be a masa-a burden and you would be chayav for hotza'ah on Shabbos. He then adds that the gemara says that if you find tefillin on Shabbos in the street you should wear them one pair at a time. However, he quotes the Magan Avraham (301:54) who says this only applies for a man. A woman would not be allowed to wear the tefillin since for her it would not be a malbush since she doesn't wear them during the week.
I guess what comes out of the Maharshag is precisely the opposite of what I wanted to say. According to the Maharshag, wearing tefillin for the mitzvah is a tachshit. The question is would that apply to the issur of kli gever as well?