Gray is not effortless. A long coat can look awkward even when every piece underneath belongs to the same color family, because neutrality does not erase differences in undertone or shape. Matching solves surprisingly little. If your coat leans blue while your sweater leans yellow, their shared gray label will not necessarily make them look comfortable together.
Undertone deserves your attention. Hold the coat beside cream and a crisp white in daylight, where a brownish or bluish cast may become easier to spot than under shop lighting. Treat these as clues. A warmer gray often sits comfortably with oatmeal, chocolate, and muted olive, while a cooler gray can work with navy, white, or blue denim without requiring a monochrome outfit. Contrast is allowed here. You can mix warm and cool colors deliberately, but repeating one of those tones elsewhere helps the difference look chosen rather than accidental.
Volume needs some editing. When a coat has dropped shoulders and a generous body, adding a thick sweater and wide trousers can hide the outline you intended, especially with the coat buttoned. Give your layers room. Try a finer knit beneath an oversized coat, or keep a fuller sweater for a cut that accommodates it without pulling across the upper arms. Check the side view. Before you buy another gray scarf to fix the outfit, would opening the coat, changing the sweater, or showing a little ankle do more for the proportions?