The trolley car problem presents the moral issue of weighing one life versus many. In this problem, a driver of a trolley car with failed brakes has the option to either let the trolley continue on its path and kill five workers on the tracks ahead, or divert the trolley to a side track where only one worker is present, killing him to save the five. This is a moral dilemma where many people might use consequential reasoning to say taking one life is better than taking five.
Upon altering the scenario, where the person is taken from the role of the driver and placed as an onlooker above a bridge, the person is given the choice to push a fat man off the bridge to block the trolley, albeit killing him in the process. In this scenario, many people would consider killing an innocent man to save the five workers murder, although the stakes are the same: one life versus five. Here, people are judging the intrinsic quality of the act itself instead of mere numbers.
I think the answer could be in the amount of responsibility you have in each case. In the first case, you are the driver, and whether you divert the trolley or not, someone will die, and you will get blamed. It would be better to get less blame for killing one person than killing five. This introduces a pragmatic element to the decision-making process, considering not just the immediate consequences but also the social and legal aftermath.
In the second case, you are not an active participant in the situation, but an onlooker. You can avert your gaze and leave, shunning the responsibility of killing the fat man to save 5 workers. But is there a moral difference between actively causing death and allowing deaths to occur through inaction?
On a larger scale, like in a society, inaction can cause harmful and inhumane systems to prevail.