Every day you lose 0.01% of your elo rating if you have a rating of > 1516, that is the only condition.

Also there seems to be a bug right now with elo gain in duel not updating every round, and only after a rejoin?
This is a known bug, it does update but just not visually in-game.