I can't speak for everyone, but for me, waifus are a coping mechanism for not having a real, human woman as a partner. Thus for so many years, they're a gateway for masturbatory fantasies. Usually starting with just a simple as imagining me hugging her and then cuddling her in bed. My mind then turns to lewd acts with her, and finally, it ends with me masturbating by humping my body pillow until I reach orgasm.
Years ago I fell away from the faith and am just now getting my act together. Waifuism has led me down a road of habitual masturbation. Alongside pornography, the two led me to masturbate constantly. We're talking 5+ times a day. This is something I still struggle with, but it's nowhere near as bad now.
But another point I'd like to add is that fantasizing about waifu(s) gets in the way of glorifying and praying to God. In that time I'm imagining my waifu, I could be praying to God. I find this temptation striking at me often at night. I'll be laying in bed, praying to God, but then a thought of my waifu enters my mind, and I start thinking of her instead of God. It takes away my time from God, and I can't help but think that that first thought about her comes from the demonic, trying to pull me away from God. Often I catch myself doing this, apologize to God and get back to praying, but other times, I indulge in the thought and just think of her.
It probably falls within the realm of venial sin, but I'm just a layman, so I can't say for sure. Consult your priest for council. Perhaps they'll have better insight as to why it may be wrong and what nature of sin it takes.
For me though, I know it leads me to sin, so I treat it as a temptation.