Catholics also like the infancy gospel of James.
But the Church Fathers universally recognize it as apocryphal and not one list of canonical texts includes it (while 1 and 2 Clement, the Apostolic Constitutions, the Didache, and the Shepherd of Hermas are in fact included in some canons).
It's a respected document because it testifies to traditions that always were part of the Church of Jerusalem to begin with, concerning the life of Mary. But the actual text itself only appeared late, and so the obvious lack of authenticity was sufficient to exclude it from the list of texts that should be read by the faithful.
It's not like you'll go to Hell for reading it but everything in it that's true is already part of our liturgical tradition, so it's not like you're missing out on anything.