Problem with rear door handle

I was swapping in my chrome LTZ handles on my car earlier today, the first 3 went in no problems. The fourth just wouldn't go in. Then the **** bolt came out. After trying for almost 2 hours to get it back in working order, I just gave up. I managed to get the bolt back in the hole, behind the black piece, several times but it just won't work properly anymore.

Any ideas on what to do?

Pic: http://i.imgur.com/1JocdYD.jpg