What about inside jokes in the best man speech? 

These will often get you into trouble in the best man speech because many in the audience won’t understand the joke.  They often get bored as you deliver the best man speech because they don’t relate to the story that is leading to the punch line.