Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit a8fb311a authored by Ronald Braunstein's avatar Ronald Braunstein
Browse files

Batch migration of frameworks/base TEST_MAPPING to test_module_config modules 

As part of go/test-module-config we are moving test options from
TEST_MAPPING -> Android.bp files.

In previous Cls, we created the new `test_module_config` rules in
Android.bp

This is updating the TEST_MAPPING file to use those rules.
It is also removing "FlakyTest and IgnoreTest" exclude annoations as
they are now added in gcl files per run rather than ad-hoc per module.

I have a script that looks at the generated tradefed config file for the
new options added in Android.bp files, then it looks at TEST_MAPPING
files and find the places to update where the options match for the
test.

I am also doing abtd runs of each TEST_MAPPING file before and after my
change and ensuring the number of tests run is the same (or at least as
many). There are cases where tradefed would comping include-filters
across TEST_MAPPING entries for the same module, but now they will be
purposefully split up, causing some tests to be run under two different
modules.

Flag: TEST_ONLY
Test: Ran adbt on each TEST_MAPPING and compared before and after
results.  Verified we were still running all the tests we were before.
i.e. after the adbt run, I would download the test artfiact for the
tradefed detailed evenvt and compare test counts.
You can see CtsAppTestCases became CtsAppTestCases_cts_requesttileserviceaddtest, etc.
I'm not including results for all 100 TEST_MAPPING files, but I did
verify with scripts and eyes.
Minor differences (like 2011 vs 2009) on a test that didn't change are
ignored, but in general there were more tests run, not fewer.

	% diff <(grep started frameworks_base_services_core_java_com_android_server_statusbar_TEST_MAPPING/BASE.details) <(grep started frameworks_base_services_core_java_com_android_server_statusbar_TEST_MAPPING/NEW.details) | grep run
	<     [run x86_64 CtsAppTestCases (testCount: 6,attempt: 0) started]
	>     [run x86_64 CtsAppTestCases_cts_requesttileserviceaddtest (testCount: 6,attempt: 0) started]
	<     [run x86_64 CtsLocationFineTestCases (testCount: 96,attempt: 0) started]
	>     [run x86_64 CtsLocationFineTestCases_android_server_location (testCount: 96,attempt: 0) started]
	<     [run x86_64 FrameworksNetTests (testCount: 2009,attempt: 0) started]
	>     [run x86_64 FrameworksNetTests (testCount: 2011,attempt: 0) started]

Test-Mapping-Slo-Bypass-Bug: b/335015078
Change-Id: If6e3cd0624ac1c16f1cd088566d967769f47199c
parent f824197c

TEST_MAPPING

+18 −156
Original line number Diff line number Diff line 
{

  "presubmit-large": [

    {

      "name": "FrameworksServicesTests",

      "options": [

        {

          "include-annotation": "android.platform.test.annotations.Presubmit"

        },

        {

          "exclude-annotation": "androidx.test.filters.FlakyTest"

        },

        {

          "exclude-annotation": "org.junit.Ignore"

        }

      ]

      "name": "FrameworksServicesTests_Presubmit"

    }

  ],

  "presubmit-pm": [

    {

      "name": "PackageManagerServiceServerTests",

      "options": [

        {

          "include-annotation": "android.platform.test.annotations.Presubmit"

        },

        {

          "exclude-annotation": "androidx.test.filters.FlakyTest"

        },

        {

          "exclude-annotation": "org.junit.Ignore"

        }

      ]

      "name": "PackageManagerServiceServerTests_Presubmit"

    }

  ],

  "presubmit": [

    {

      "name": "ManagedProvisioningTests",

      "options": [

        {

          "exclude-annotation": "androidx.test.filters.FlakyTest"

        }

      ]

      "name": "ManagedProvisioningTests"

    },

    {

      "file_patterns": [
@@ -46,86 +19,28 @@ 
        "SystemServer\\.java",

        "services/tests/apexsystemservices/.*"

      ],

      "name": "ApexSystemServicesTestCases",

      "options": [

        {

          "exclude-annotation": "androidx.test.filters.FlakyTest"

        },

        {

          "exclude-annotation": "org.junit.Ignore"

        }

      ]

    },

    {

      "name": "FrameworksUiServicesTests",

      "options": [

        {

          "exclude-annotation": "androidx.test.filters.FlakyTest"

        }

      ]

    },

    {

      "name": "FrameworksInputMethodSystemServerTests",

      "options": [

        {"include-filter": "com.android.server.inputmethod"},

        {"exclude-annotation": "androidx.test.filters.FlakyTest"},

        {"exclude-annotation": "org.junit.Ignore"}

      ]

    },

    {

      "name": "ExtServicesUnitTests-tplus",

      "options": [

        {

          "exclude-annotation": "androidx.test.filters.FlakyTest"

        }

      ]

    },

    {

      "name": "ExtServicesUnitTests-sminus",

      "options": [

        {

          "exclude-annotation": "androidx.test.filters.FlakyTest"

        }

      ]

    },

    {

      "name": "FrameworksCoreTests",

      "options": [

        {

          "include-annotation": "android.platform.test.annotations.Presubmit"

      "name": "ApexSystemServicesTestCases"

    },

    {

          "exclude-annotation": "androidx.test.filters.FlakyTest"

      "name": "FrameworksUiServicesTests"

    },

    {

          "exclude-annotation": "org.junit.Ignore"

        }

      ]

      "name": "FrameworksInputMethodSystemServerTests_server_inputmethod"

    },

    {

      "name": "FrameworkPermissionTests",

      "options": [

        {

          "include-annotation": "android.platform.test.annotations.Presubmit"

      "name": "ExtServicesUnitTests-tplus"

    },

    {

          "exclude-annotation": "androidx.test.filters.FlakyTest"

      "name": "ExtServicesUnitTests-sminus"

    },

    {

          "exclude-annotation": "org.junit.Ignore"

        }

      ]

      "name": "FrameworksCoreTests_Presubmit"

    },

    {

      "name": "FrameworksInProcessTests",

      "options": [

        {

          "exclude-annotation": "androidx.test.filters.FlakyTest"

      "name": "FrameworkPermissionTests_Presubmit"

    },

    {

          "exclude-annotation": "org.junit.Ignore"

        }

      ]

      "name": "FrameworksInProcessTests"

    },

    {

      "name": "vts_treble_vintf_framework_test"
@@ -159,78 +74,25 @@ 
   // infra during the hardening phase.

   // TODO: this tag to be removed once the above is no longer an issue.

   {

     "name": "FrameworksUiServicesTests",

     "options": [

       {

         "exclude-annotation": "androidx.test.filters.FlakyTest"

       }

     ]

   },

   {

     "name": "ExtServicesUnitTests-tplus",

     "options": [

       {

         "exclude-annotation": "androidx.test.filters.FlakyTest"

       }

     ]

   },

   {

     "name": "ExtServicesUnitTests-sminus",

     "options": [

       {

         "exclude-annotation": "androidx.test.filters.FlakyTest"

       }

     ]

   },

   {

     "name": "TestablesTests",

     "options": [

       {

         "exclude-annotation": "androidx.test.filters.FlakyTest"

       }

     ]

     "name": "FrameworksUiServicesTests"

   },

   {

     "name": "FrameworksCoreTests",

     "options": [

       {

         "include-annotation": "android.platform.test.annotations.Presubmit"

     "name": "ExtServicesUnitTests-tplus"

   },

   {

         "exclude-annotation": "androidx.test.filters.FlakyTest"

     "name": "ExtServicesUnitTests-sminus"

   },

   {

         "exclude-annotation": "org.junit.Ignore"

       }

     ]

     "name": "TestablesTests"

   },

   {

     "name": "FrameworksServicesTests",

     "options": [

       {

         "include-annotation": "android.platform.test.annotations.Presubmit"

     "name": "FrameworksCoreTests_Presubmit"

   },

   {

         "exclude-annotation": "androidx.test.filters.FlakyTest"

     "name": "FrameworksServicesTests_presubmit"

   },

   {

         "exclude-annotation": "org.junit.Ignore"

       }

     ]

   },

   {

     "name": "PackageManagerServiceServerTests",

     "options": [

       {

         "include-annotation": "android.platform.test.annotations.Presubmit"

       },

       {

         "exclude-annotation": "androidx.test.filters.FlakyTest"

       },

       {

         "exclude-annotation": "org.junit.Ignore"

       }

     ]

     "name": "PackageManagerServiceServerTests_Presubmit"

   }

 ]

}

apex/jobscheduler/service/java/com/android/server/TEST_MAPPING

+1 −4
Original line number Diff line number Diff line
@@ -11,10 +11,7 @@ 
  ],

  "postsubmit": [

    {

      "name": "FrameworksMockingServicesTests",

      "options": [

        {"include-filter": "com.android.server"}

      ]

      "name": "FrameworksMockingServicesTests_android_server"

    }

  ]

}

apex/jobscheduler/service/java/com/android/server/deviceidle/TEST_MAPPING

+1 −4
Original line number Diff line number Diff line
@@ -6,10 +6,7 @@ 
  ],

  "postsubmit": [

    {

      "name": "FrameworksMockingServicesTests",

      "options": [

        {"include-filter": "com.android.server"}

      ]

      "name": "FrameworksMockingServicesTests_android_server"

    }

  ]

}

apex/jobscheduler/service/java/com/android/server/job/TEST_MAPPING

+4 −18
Original line number Diff line number Diff line 
{

    "presubmit": [

        {

            "name": "CtsJobSchedulerTestCases",

            "options": [

                {"exclude-annotation": "androidx.test.filters.FlakyTest"},

                {"exclude-annotation": "androidx.test.filters.LargeTest"}

            ]

            "name": "CtsJobSchedulerTestCases_com_android_server_job"

        },

        {

            "name": "FrameworksMockingServicesTests_com_android_server_job_Presubmit"
@@ -19,26 +15,16 @@ 
            "name": "CtsJobSchedulerTestCases"

        },

        {

            "name": "FrameworksMockingServicesTests",

            "options": [

                {"include-filter": "com.android.server.job"}

            ]

            "name": "FrameworksMockingServicesTests_com_android_server_job"

        },

        {

            "name": "FrameworksServicesTests_com_android_server_job"

        },

        {

            "name": "CtsHostsideNetworkPolicyTests",

            "options": [

                {"include-filter": "com.android.cts.netpolicy.HostsideRestrictBackgroundNetworkTests#testMeteredNetworkAccess_expeditedJob"},

                {"include-filter": "com.android.cts.netpolicy.HostsideRestrictBackgroundNetworkTests#testNonMeteredNetworkAccess_expeditedJob"}

            ]

            "name": "CtsHostsideNetworkPolicyTests_com_android_server_job"

        },

        {

            "name": "CtsStatsdAtomHostTestCases",

            "options": [

                {"include-filter": "android.cts.statsdatom.jobscheduler"}

            ]

            "name": "CtsStatsdAtomHostTestCases_statsdatom_jobscheduler"

        }

    ]

}

apex/jobscheduler/service/java/com/android/server/usage/TEST_MAPPING

+1 −7
Original line number Diff line number Diff line 
{

  "presubmit": [

    {

      "name": "CtsUsageStatsTestCases",

      "options": [

        {"include-filter": "android.app.usage.cts.UsageStatsTest"},

        {"exclude-annotation": "androidx.test.filters.FlakyTest"},

        {"exclude-annotation": "androidx.test.filters.MediumTest"},

        {"exclude-annotation": "androidx.test.filters.LargeTest"}

      ]

      "name": "CtsUsageStatsTestCases_cts_usagestatstest"

    },

    {

      "name": "CtsBRSTestCases"