Testing for 0 <= X <= 100

It is common for programs to expect numerical variables to accept values matching a certain range. A good example is a percent variable with allowed values from 0 to 100 inclusive.

Test for such code will often validate correct operation with a value inside the range and expect an error for a value outside the range. As a bonus the test may be expecting errors when testing with values on both sides of the range!

Reproducer

$ pip install https://github.com/sixty-north/cosmic-ray/zipball/master

$ cosmic-ray run --test-runner nose --baseline=10 example.json percent.py -- test1.py
$ cosmic-ray report example.json

job ID 6:Outcome.SURVIVED:percent
command: cosmic-ray worker percent replace_Lt_with_LtE 0 nose -- -v test1.py
--- mutation diff ---
--- a/example_12/percent.py
+++ b/example_12/percent.py
@@ -1,7 +1,7 @@


 def validate_percent(percent):
-    if ((percent < 0) or (percent > 100)):
+    if ((percent <= 0) or (percent > 100)):
         raise Exception('Percent must be between 0 and 100')
     return percent


job ID 15:Outcome.SURVIVED:percent
command: cosmic-ray worker percent replace_Gt_with_GtE 0 nose -- -v test1.py
--- mutation diff ---
--- a/example_12/percent.py
+++ b/example_12/percent.py
@@ -1,7 +1,7 @@


 def validate_percent(percent):
-    if ((percent < 0) or (percent > 100)):
+    if ((percent < 0) or (percent >= 100)):
         raise Exception('Percent must be between 0 and 100')
     return percent


job ID 16:Outcome.SURVIVED:percent
command: cosmic-ray worker percent number_replacer 0 nose -- -v test1.py
--- mutation diff ---
--- a/example_12/percent.py
+++ b/example_12/percent.py
@@ -1,7 +1,7 @@


 def validate_percent(percent):
-    if ((percent < 0) or (percent > 100)):
+    if ((percent < 1) or (percent > 100)):
         raise Exception('Percent must be between 0 and 100')
     return percent


job ID 17:Outcome.SURVIVED:percent
command: cosmic-ray worker percent number_replacer 1 nose -- -v test1.py
--- mutation diff ---
--- a/example_12/percent.py
+++ b/example_12/percent.py
@@ -1,7 +1,7 @@


 def validate_percent(percent):
-    if ((percent < 0) or (percent > 100)):
+    if ((percent < 0) or (percent > 101)):
         raise Exception('Percent must be between 0 and 100')
     return percent


total jobs: 20
complete: 20 (100.00%)
survival rate: 20.00%

To fully test this code you have to test with both border values and with the next possible values, which are outside of the range. Testing with a value that falls within the range, but isn’t a border one doesn’t affect mutation testing.

$ cosmic-ray run --test-runner nose --baseline=10 example.json percent.py -- -v test2.py
$ cosmic-ray report example.json

total jobs: 20
complete: 20 (100.00%)
survival rate: 0.00%

Note

This is similar to Testing for X != 1

Source code

percent.py
def validate_percent(percent):
    if percent < 0 or percent > 100:
        raise Exception("Percent must be between 0 and 100")

    return percent
test1.py
import percent
import unittest

class TestPercent(unittest.TestCase):
    def test_value_in_range(self):
        result = percent.validate_percent(50)
        self.assertEqual(result, 50)

    def test_value_not_in_range(self):
        with self.assertRaises(Exception):
            percent.validate_percent(500)

        with self.assertRaises(Exception):
            percent.validate_percent(-10)

if __name__ == "__main__":
    unittest.main()
test2.py
import percent
import unittest

class TestPercent(unittest.TestCase):
    def test_lower_boundary(self):
        result = percent.validate_percent(0)
        self.assertEqual(result, 0)

    def test_upper_boundary(self):
        result = percent.validate_percent(100)
        self.assertEqual(result, 100)

    def test_below_lower_boundary(self):
        with self.assertRaises(Exception):
            percent.validate_percent(-1)

    def test_above_upper_boundary(self):
        with self.assertRaises(Exception):
            percent.validate_percent(101)

if __name__ == "__main__":
    unittest.main()