Toxic chemicals are dangerous stuff -- hence the label toxic. That's why you can't just toss hazardous waste in a trash can or dump nuclear materials in a river. If you get uranium in your drinking water, you're bound to feel sick or find yourself with several extra limbs. (The latter is pure speculation, I admit.)
Bad workplaces aren't all that different. You might not grow a third arm from a toxic job, but you will slowly realize that your mental and even physical healths are suffering from undesirable work conditions.
Sure, you could quit, but how many of us are eager to leave our jobs in this economy? Until you find a better opportunity, you need to work with what you've got, and what you've got is a toxic workplace.