Thanks thgoebel, I'll check it out. I have my doubts though because when this happens and I manage to catch it and turn it off on the switch (HA) then the light flickers on/off rapidly once. Example:
1. Light turns off (without relay)
2. It stays off by up to 5 seconds
3. I catch it before this time and turn it off via HA (even though it's not lit)
4. I hear the relay click
5. The light flickers on/off