Make sure the color background is only cleared once

The cancel and finished callbacks are both run if we cancel the surface animation without restart and we only want to call clear once for everytime we call setbackground otherwise the background color counter would be out of sync and we might not remve the color background when expected

Bug: 202141435
Change-Id: I0027d581b60c3acf0e02e66026d49c41435882ce