There's one plant that I question having in my garden every year, and that's Monarda or bee balm. I once grew three different types, but am now down to just one: 'Marshall's Delight'.
Its bee-attracting blooms are what save it ever year, because it has some problems. The first is that it spreads quite vigorously, but that doesn't bother me -- I deal with vigorous plants all the time. The second problem is that it is guaranteed to have powdery mildew every year.