Rock gardens, as they are known now, evolved from more traditional alpine gardens that were designed to display specimens of small plants found in the mountains of Europe. These plants were mostly compact and slow growing to be adapted to the harsh climates and rocky soils of higher elevations, but do not compete well with more aggressive plants. They consequently need their own space, and are quite happy surrounded by rock or gravel to insulate the surface of the soil and control weeds. Slight slopes, as well as the abundance of rock, enhance drainage to replicate mountainous conditions.

Compact plants from all over the world now find homes in rock gardens. The compact nature of rock garden makes them practical for small spaces, particularly where slopes make other gardening difficult. Sculptural, colorful or lichen covered rocks, as well as garden sculpture, can be as ornamental as the plants that surround them. Larger stones can substitute for low retaining walls.

Plants for rock gardens do not need to be from mountainous areas, but should be compact. They must also be appropriate to the particular microclimate. For example, small succulents and cacti want to be as exposed in a rock garden as they would be in a real landscape. Lily-of-the-valley shrub conversely prefers sheltered cooler spots.

Trees for rock gardens are generally very small. Pendulous (weeping) and dwarf Japanese maples are more desirable than those that eventually grow into mid sized trees. Dwarf forms of spruce, eastern hemlock, cedar, fir and false cypress all work well. There are also several dwarf pines to choose from, including dwarf cultivars of Scots, Japanese red, tanyosho, mugho, white and pinon pines.

Dwarf forms of cotoneaster, hebe, holly, rockrose, germander and Japanese barberry have become as popular in rock gardens as the more traditional small heathers and heaths. There are several compact junipers as well. Some rock gardens plants grow into small shrubbery. Others sprawl about as mats or dense ground cover.  

Thrift (sea pink), campanula, snow-in-summer, fleabane, coral bells, plantain lily, evergreen candytuft, carpet bugle (Ajuga spp.), and small herbs like thyme and oregano are some of the more popular perennials for rock gardens. Mat forming penstemon, columbine, cranesbill, saxifrage, rockcress and dwarf types of dianthus and primrose are more difficult to obtain, but worth trying. There are of course many small succulents like aeonium, echeveria, stonecrop (sedum) and ice-plant.

Crocus, grape hyacinth, snowdrop and wild freesia are the only common bulbs that are well suited to rock gardens. Uncommon dwarf cyclamen (not Cyclamen persicum), small types of narcissus and diminutive iris that can sometimes be found by mail order catalogues, online, or occasionally in nurseries that stock unusual plants are actually more proportionate to rock gardens than they are in more typical types of gardens.