Pulling out cool season annuals that are still somewhat colorful is never easy, even if they are already deteriorating. There is always the desire to stretch their season as late as possible until they succumb to warmer weather. Only a few can perform all year, or get cut back to hide below other taller plants until the weather gets cool enough in autumn for them to regenerate and bloom again.
Regardless of all the resistance, removing the annuals of a previous season relinquishes space for annuals that are appropriate to the next season, whether cool to warm season, or warm to cool season. Even if new annuals are initially wimpy relative to the older annuals that were removed, they should proliferate and bloom better than lingering unseasonable annuals would if not removed.
Timing is somewhat important. There is no point in removing cool season annuals too early if the weather is still too cool for warm season annuals. However, there is no point in planting too late either. Delayed planting only delays growth and bloom. Some warm season annuals, especially those grown from seed sown late in winter, prefer to get established while the weather is still cool.
French marigold is probably the most traditional warm season annual for bright yellow, orange and bronze. Lobelia contrasts excellently with rich blue and purple, and can also be purplish rose or white. Petunia can do even more with brighter and more variable colors. Cosmos provides pastel pinks and white on taller plants. Cockscomb colors rival those of marigold, and can also be red.
Pincushion flower, annual statice and zinnia are popularly enjoyed as bedding plants, and also work well individually, behind lower bedding plants, or in planters of mixed annuals or perennials. Verbena and moss rose cascade nicely from such mixed planters. Sadly, brightly colored and formerly popular busy Lizzie (impatiens) are either rare or unavailable because of a mold disease.
Nasturtium and alyssum are warm season annuals that are often grown through winter as well. Where they are allowed to naturalize and bloom throughout the year, deteriorating old plants might need to be groomed out as they get replaced by self sown plants. New nasturtium should be sown as seed, instead of planted as seedlings from cell packs. Alyssum grows well by either means.