Only consider to keep activity config if invisible requested

Otherwise unlike activity always has a valid initial config, wallpaper
may get an empty config (frame) if it is initializing when invisible.

Bug: 251371010
Test: atest WindowManagerServiceTests#testRelayoutExitingWindow

Change-Id: I41c1b350d56657b5e93456f8e57b788696747d92