Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 13cbc53a authored by Michael Butler's avatar Michael Butler
Browse files

Early test termination for unsupported models 

Some service drivers won't be able to support all operations. In those
cases, try to prepare the model anyway. If it works, proceed with the
test; if it fails, terminate the test early.

For example, the hvx nn service does not support svdf, so it should
attempt to prepare the model, but the test should not fail if the hvx
driver reports an error.

Bug: 63905942
Test: mm, expected behavior when running on hvx service
Change-Id: I51aa1911145f0d17590906c21314d2bdc24cffea
parent e51f3478

neuralnetworks/1.0/vts/functional/GeneratedTestHarness.cpp

+11 −11
Original line number Diff line number Diff line
@@ -100,6 +100,17 @@ void Execute(const sp<IDevice>& device, std::function<Model(void)> create_model, 
        EXPECT_TRUE(prepareReturnStatus == ErrorStatus::NONE ||

                    prepareReturnStatus == ErrorStatus::GENERAL_FAILURE);

    }



    // early termination if vendor service cannot fully prepare model

    if (!fullySupportsModel && prepareReturnStatus == ErrorStatus::GENERAL_FAILURE) {

        ASSERT_EQ(nullptr, preparedModel.get());

        LOG(INFO) << "NN VTS: Early termination of test because vendor service cannot "

                     "prepare model that it does not support.";

        std::cout << "[          ]   Early termination of test because vendor service cannot "

                     "prepare model that it does not support."

                  << std::endl;

        return;

    }

    ASSERT_NE(nullptr, preparedModel.get());



    int example_no = 1;
@@ -192,17 +203,6 @@ void Execute(const sp<IDevice>& device, std::function<Model(void)> create_model, 
        // retrieve execution status

        executionCallback->wait();

        ErrorStatus executionReturnStatus = executionCallback->getStatus();

        if (!fullySupportsModel &&

            static_cast<ErrorStatus>(executionReturnStatus) == ErrorStatus::GENERAL_FAILURE) {

            LOG(INFO) << "Ignoring execution results from model that is not supported by the "

                         "vendor service driver";

            std::cout << "[          ]   Ignoring execution results from model that is not "

                         "supported by the vendor service driver"

                      << std::endl;

            continue;

        }

        LOG(INFO) << "CONTINUING TO CHECK VALUE";

        std::cout << "[          ]   CONTINUING TO CHECK VALUE" << std::endl;

        EXPECT_EQ(ErrorStatus::NONE, executionReturnStatus);



        // validate results