Move clear all to recycler view (2/2)

This finishes moving the clear all button to the recycler view.
Primarily, this CL deals with changes to calls that depended on recycler
children being task item views and starting at the 0th index.

Bug: 114136250
Test: Build, manual test
Change-Id: Icecf257409207de351345997205def11e1048ab0