Blinding sunlight on white fabric does not betray a simple truth: white can keep you cooler than black. That judgment rests on how light and heat actually move, not on how bright your shirt looks against the page of your book.
White cloth wins first at the surface. It scatters. Hard. Most incoming solar radiation is reflected or bounced around within the fibers instead of being converted into heat at the outer layer, while dark dyes absorb a broad spectrum and turn that energy straight into thermal radiation right against your skin. You still get enough visible light to read because your eyes are absurdly sensitive; a tiny slice of the spectrum, even after reflection losses, is plenty for legible contrast on a page.
The bigger advantage hides in the air between fabric and skin. A white shirt that reflects and diffuses energy keeps the temperature of that air gap lower, which reduces conductive and radiative heat flux into your body and gives sweat evaporation more headroom. Dark cloth, by heating up, drives infrared emission inward as well as outward, effectively raising the mean radiant temperature your body experiences. You absorb similar daylight for vision, but vastly different radiant heat loads, and your nerve endings care about the latter, not the brightness of your novel.
There is also a directional trick. White fabric redirects a chunk of sunlight back out into the environment instead of locking it into the textile matrix, which limits the total heat capacity of the garment during exposure. Dark cloth stores more energy, stays warmer even when a cloud passes, and slows cooling by convection. Your book only needs a modest luminance on its surface; your body, trapped under the wrong spectrum, pays the thermal bill.