Dont mark adjacent display group if in non-desktop

We currently mark the displays requesting for a new group and are in non
desktop mode. This would cause issues for setups like Auto where there
can potentially be multiple external displays comprised of multiple
groups

Bug: 432178456
Test: Manual
Flag: EXEMPT trivial change
Change-Id: Iccc90bdcee8c522149f2b4c852a4fdebd06405ad