Not necessarily. The solenoid on the starter can stick and hold without power
However, given that a recent change was adding the new smaller relay, odds are very likely that that is the issue.
Here's another relay I recommend. The famous Bosch 75A relay. I know, it's overkill at $35 each, but it's one of those things where that $35 will pay for itself the first time one of the cheaper $5 relays leaves you stranded somewhere. I've had one of these running my E-fan for over 15 years now. I'll likely use one when I do the same factory solenoid delete as well.
https://www.delcity.net/store/Bosch-Tyco-Relays/p_823743.h_823744.r_IF3003?mkwid=YnN3zMyF&crid=11472933143&msclkid=5aba46024ddd1f24ed741ad6198b2c7f&utm_source=bing&utm_medium=cpc&utm_campaign=Shopping - PLA's(BSC)&utm_term=1100315444322&utm_content=Ad group #1