Measure your worth based on yourself, not on others.
Everyone sees the world based on his or her own biases and circumstances. When someone pours negative thoughts on another person, it's a greater reflection on their own deficiencies than the person they're attacking.
Basing your self-esteem on another person's opinions is never a good idea. Instead, take your value from what you know to be the truth about yourself from the inside out, not the other way around.