Go South

Deteriorate or decline.

Examples:

• The stock market is headed south again.

Origin: first half of 1900s; generally thought to allude to compasses where south is down