You're not holding anyone up. You're traffic, going
somewhere, the same as everyone else on the road. Nobody stuck behind a bin lorry thinks
the bin lorry should be riding in the gutter.
The Highway Code's hierarchy of road users (Rule H1) is blunt about this: the more damage
you can do, the more responsibility you carry. The car behind you can kill you. You can,
at worst, scratch some paint. It's on them to pass you properly, not on you to get out of
the way.
And the delay you're so worried about causing is what, ten seconds? They'll lose more than
that at the next lights while you filter past them, and somehow you're the one who feels bad
about it.
Nobody ever got a thank-you for riding in the door zone. Give it up.