Well, unfortunately.. most high end gaming cards are going to be very long, similar to how long 9800GTX's are. So your options for a high end card are limited in those regards, and in a case that small.. I think it would be a bad environment for the card (E.G not a lot of space for airflow, really cramped, really hot).
You'd be better off keeping the card, and dumping 50-80$ in a nice mid tower for cheap like these;
http://www.newegg.com/Product/Produc...82E16811119077
http://www.newegg.com/Product/Produc...82E16811119152
Pretty cheap with good airflow, the second one has much better airflow. Naturally full towers provide more room and better airflow, but also cost more, and are very huge.
Just my 2 cents on it though, hope you get something figured out.
PS: There wasn't a way you could have re-routed the cables to work around the huge cooler? I mean if the physical connectors on the motherboard were interfering, like sticking out into the cooler so the cables couldn't even bypass then I understand, but cable management could have been a fix.
EDIT: And I'm assuming you have a pre-built, and might not know how to change parts from one case to another, but since you know how to change graphics cards I figured this isn't a long shot off, if you do know how - I can recommend doing that, if you don't, well.. it isn't too late to learn.