drm: don't unnecessarily enable the polling work

... by properly checking connector->polled. This doesn't matter too
much because the polling work itself gets this slightly more right and
doesn't set repoll if there's nothing to do. But we can do better.

v2: Chris Wilson noticed that I broke polling, since repoll will never
ever be set true. Fix this up, and simplify the logic a bit while at
it.

Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Dave Airlie <airlied@redhat.com>