It may also surprise you to learn that strawberries, raspberries, blackberries and boysenberries are not berries. Their seeds are outside the pericarp.
Many plants produce berries and a majority of these berries are either poisonous to humans or simply inedible. Berry bearing plants are called bacciferous. Most berries, indeed most fruits, are a bright color that contrasts with the plant's green leaves. Researchers surmise that this is so that animals can find them more easily, eat them and disburse the seeds.
Botany does not always win out over common usage. Just look at the common names of flowers; how many daisies aren't really daisies? For the average person, a berry remains any small, juicy fruit.