Whoever originally tied them to the tracks is ultimately responsible.
But, fun fact: everyone who let themselves end up tied to tracks fucked up. It's not a random act of god. They failed to secure their person against mad Snidely Whiplashes with sufficient enthusiasm, and are now paying the price. This breaks the symmetry between the 'kill' and 'let die' versions. The fat man did nothing wrong.
Whether you pull the lever or don't, Snidely Whiplash is guilty of murder, and you're not. Who you happen to save is purely down to personal taste. If they don't like it, they shouldn't have let themselves get tied to train tracks. Maybe their funeral attendees will learn better for next time.
However, as a private citizen, you will literally never see a trolley problem. Snidely Whiplash doesn't exist, nobody gets tied to train tracks, and even if they did you don't hang around manually-operable track levers. Not even metaphorically. The trolley problem is a problem for states.
E.g, do you execute murderers? If you don't, they may go and murder again. If you do, you might execute an innocent man. By the displacement theory you can lock them up until they're 35, but that's expensive. If you spent that on, say, blood drive advertisements instead, you might well save more lives than you lose in executed innocents. However, all of these options save different people. The distributions are dissimilar. Necessarily the state is saying they care more about one group than another.
And that's why getting the trolley problem right is important, but also why getting it right is irrelevant. States have never given the slightest shit about philosophy, so even if you work it out correctly and all agree, they're still going to do horrible corrupt state things instead.