Caffeine alters the brain, yes, but in the same way as cocaine and heroin ? No. Although Perhaps you just mean that like any drug it alters the brain.
There are so many 'drugs' in ordinary food I could write about it all day. So saying 'no' to drugs means you would literally starve probably. For instance, bananas and milk contain tryptophan. This is an amino which affects the brain (sedative, even mildly psychedelic at a sufficient dose). It is the 'root' of all tryptamine psychedelics (like magic mushrooms). If you drink diet coke, it has no caffeine, but it contains phenylalinine. Also a naturally occurring amino, and a stimulant. It is the 'root' of all phenethylamine drugs (ecstasy, methamphetamine ...).
Herbs, spices - all contain mind altering compounds (whether you call them drugs, compounds, alkaloids, chemicals, medicine - all depends on the 'spin' you want to put on it).
So you can't NOT take drugs.
Some are way stronger than others in their mental and physical effects, so it comes down to education and discrimination.