The engine should NOT be running when you pull the SPOUT plug out or put it in. I'm not exactly sure of what it will do but, I've been warned against doing it that way.
Could your spout be bad? You said you pulled it and plugged it in while running. I'm just guessing here, but I was told not to do that too. Don't know why. But maybe disconnecting the spout while running could damage something.