The invisibles

I am always wondering if i have the right to take photos of people in a vulnerable situation, like homeless people in the street etc. I can`t stop cencuring myself even if i know that these kind of photos may hit the sensible chords of some other people or maybe they can produce a positive reaction in the right direction. I feel like an indiscreet intruder violating the invisible borders of their difficult life.