On The Safe Side

Avoiding danger; with a margin for error.

Examples:

  • Just to be on the safe side, let's order another hundred chairs.

Origin: First recorded in 1811