That's ~85% more damage with skillX, +30% CC  with that skill etc. If X happens to be very useful after modification, than it pays off to roll affix few time, if you got only one of them. Also, items with 2 good affixes - say xHP/hit and xSup/hit and something stupid (+damage while surrounded for ranged comes to mind), rerolling few times might give +CC% or something actually useful.

I'd be careful with money, but few rerolls won't make expense, you'll be even when you sell it, and Fate - well, it's only 3 now. Used to be 5, more worth. 

Look at stats and most of them can be sold immediately, I just put few examples where it's worth a try to modify...

Don´t ever bother recycling it. You may get good items for it, but if you selling it instead, you are guaranteed to get Fate Points for it too. And Fate Points is something you DO need.