You should be able to just pull straight out, there is no clip to disengage. Its possible to unclip the wire harness/connector from the lower dash, think this would happen before any damage would occur.
Mine has always been tough to remove as well. In the end I just end up pulling on the plug slowly and firmly. The plastic sure does bend, but so far has not broken. I don't make a habit out of unplugging my scan gauge though for this very reason.