For one who do not practice or practice only a little, what you say would work well. But when the connection to God become more important than anything in the world only spiritual practice is the way to go.
Why would you think that one cancels out the other?
The point here concerns interactions with the opposite sex and to learn how to be respectful and treating eachother equally while also maintaining self-control.
What you are saying is merely avoiding the problem instead of solving it.
It's like someone who really doesn't want to steal money, refuses to go and work at a bank or as a cashier as a "solution" for that problem.
That's not a solution at all. Instead, learn how to be around money without putting some in your pocket.
That's a much better way to go about it.
That way, you are mentally prepared for the day you will find yourself in the presence of money that isn't yours.
Same goes with women. Especially if you live in the west. You ARE going to end up in social situations where you will have to deal with / converse with / work with women - regardless of how they are dressed or act.
Don't you think it's a better idea to be mentally prepared for that moment instead of trying to go out of your way to avoid being in such situations?
The psychological fight against temptation is an inner struggle. A struggle that is not won by simply removing the temptation. That's simply avoiding the fight altogether, which will only result in you being unprepared for the day you can't run away.