Fixed an issue where children dissappeared the wrong way

Because we were comparing relative to absolute positions,
we need to calculate the next view the right way.

Test: add children in group, remove the last / middle one
Fixes: 35193074
Change-Id: I415a978b84d528ded402b99b872769b98aa49ec0