Stick it in your bra?
Or if you’re going braless, as I know you do sometimes @Tim, I’d clip it inside my cycling shorts waistband (so the pump sits against my skin rather than on the outside of my shorts, that way it moves less). I’d normally do that round the back so it doesn’t get in the way when you bend over.