My specialist subject is...

Well, I have read very many stories here that discuss American football. It seems to be taken as read that the world understands the references. I have written rugby into my tale, a sport that has a far, far wider following than gridiron, but I realise that there are many folk who know absolutely nothing of it.
I have also written one of my other passions into my stuff, namely music. I get really into the swing and the flow and....
How do people write a "specialist" episode without spending half a chapter explaining "pastime for dummies"?