yeah, when he raised you on the river, did you really think your straight was good??
It'd be a tough fold, but the raise was bad. You make him sound like just the kind of player who will stick around and make horrible calls drawing to the flush. Unfortunately for you, this is one of the times he hit.
Although he could argue implied odds, since he got most of your stack once the flush hit....
