Disabling test cases for functionality configured out

Hi. Do we have a mechanism to disable python unit tests for functionality that has been configured OFF?

For example with USE_VM_PROFILER configured OFF the test test_runtime_vm_profiler.py still runs results in a failure.

I don’t see an obvious mechanism in place to disable such tests based on the USE_ flags… ?

That’s a good question. Assuming that you’re referring to the Jenkins CI tests: the Jenkinsfile sets the build parameters (which flags are turned on or off) and the sequence of unit tests that are run for each build:

Therefore, we should make sure that test_runtime_vm_profiler.py is only run when the USE_VM_PROFILER flag is enabled. If that’s not the case, I suggest that we create a new .sh test script for builds that enable the profiler.

We tried to encourage the writing test case though to be agnostic of availability of a certain features. Based on runtime checks(e.g. if cuda is not available skip cuda test).

In the case of VM profiler, because we see there is less dependencies needed to enable this feature, the testcase was written in a way that is not feature agnostic.

@tqchen

Do we have helpers or tests to detect hardware run time features ?

We are going to have tests in the testsuite which expect particular instructions in cpus that are not necessarily tvm build time detectable. However the tests need to be able to run in the presence and get skipped in the absence of actual hardware.

regards
Ramana

We have functions fo detecting cuda features, but as far as i know perhaps we lack cpu feature detection.