Is my starter going bad?

Sounds like the bendix is not returning quickly enough. I would check the starter disengagment time and the teeth on the bendix and ring gear. Most likely it is the starter itself, IMHO.

Good luck.