Static stretching is a waste of time but it is possible for yoga to be very, very different from mere stretches. I learned what yoga is derived from in a single night. At it's core, it is the release of physical tension. Stretching can mean fighting tension instead of releasing it. Yoga is a kind of body-oriented meditation where you learn to feel the state of the body instead of ignoring it. You won't understand the point of it unless you can fit it in a larger framework, and it's often used as a starting point for spiritual endeavors. That could indeed be a waste of time—especially if you don't know what "spiritual" implies. If it becomes religious, yes, there is certainly a huge waste of time to be had there.
If it's done right, you'll feel energy flow through you more efficiently from better blood flow and aches and chronic tensions can disappear. It can help your posture as well. Because of this it might be rewarding to try one class to see if they teach the kind of yoga that will help your health and well-being as long as you understand that it is possible that they don't teach that at all. They can't force you to keep coming and the worse that will happen is you get to be in the company of cute women for an hour.