We have this many places in Europe. The police are not even allowed to wear guns in Norway (and frankly do not need them) unless there is some special intelligence or something making a reason for it. That does not absolve the need for state controlled monopoly on violence. It only means that is should be limited and wielded with the utmost care.