Never forget the value of your money, how lucky you are to have it & what it means to others 

Lets take Africa: from a United Nations statistic more than half of African countries have an average annual per capita income of US$600. This is the same in many parts of Asia and Latin America. In the developing world their is little 'middle-class' so average annual per capita income stats mean very little. More than likely your shoestring budget of $20/day is more than many earn in a week; your week's budget could be up to a year's worth of secondary education and it goes on. Even somewhere as semi-developed as Thailand, someone with a decent (high-school) education - which few can obtain (often held back by cost rather than capability) - would earn ~US$150-200/month (say as hotel receptionist), much, much less for a more menial job if they can find one.

The deal is, we all get into haggling, want to keep the price of our trip down and hate foreigner pricing, but a fruit vendor's profits for a day could be as low as few bucks and he needs to get the bus home! Be realistic, read, talk to locals and develop your own understanding.