It's a platform, like skype, tumblr, facebook.
I think it's a very interesting and unique platform.
I came into it knowing that it was used for ERP and by griefers. Since, I've found some interesting builds. Virtual experiences. I quite like that aspect of SL. I also like, in a way, how it's becoming like a deprecated cyber-wasteland. LL are trying to milk it for money. People are incompetent and still use prims to build shit. Land prices are sky high. There's no cohesive theme in areas, like LL tried to create. The place is littered with empty corporate and university sims.
Sure, there are a lot of losers on here. Then again, there are people out there who simply, out of circumstance, do not have a lot of social interaction, or have found that they need more than they set themselves up for. Most of us are in the 20-30 age group. That is currently where most people are finishing educations, and trying to get jobs. "Young adults" in a sense, people coming into adulthood. If people want to paint that age group as losers, that's their choice. Income isn't everything.
I think those attitudes you describe are due to a high entry threshold. For starters, people will have trouble learning what all the features in a viewer does, let alone how things work in SL itself. It's inherently unintuitive, and the tutorial islands only give you info on how to move and talk.