Count Against

Be disadvantageous to; be reckoned as a negative factor for someone.

Examples:

• His earnings this year will count against his Social Security benefits.

Origin: Early 1900s