How do I keep him away from these people?

I am not one to be judgmental over who a person chooses to be friends with, but when that person is close to me, and they're choosing to hang out with scum, I tend to become protective. It's only natural, right? So, my boyfriend has begun to hang around these complete morons who supposedly are into drugs. Well, I didn't believe it at first until today when I saw two of them stumble out of the bathroom, wiping at their noses, bloodshot eyes, just not acting normal. Now, I am totally convinced that they are into drugs and am worried they are going to try and drag him into it too. I do not want him falling into this, and I am becoming really worried. What should I do?