How to set a pylint score threshold?

Question

I'd like to extract my pylint rating and set a threshold. Let me explain, for example, if the score is under 5, I want exit 1; And if my code is rated higher than 5, I want exit 0 and continue my Jenkins procedure.

Answer 1

Since pylint 2.5.0 there is a new argument called --fail-under that resolves this question without needing external tools or scripts.

In this example, pylint will exit with error when score is under 8:

pylint --fail-under=8 python_code.py

Answer 2

Here's a way to access the pylint API it in Python. The following code should be saved to a file and executed with first argument to the script to be module/file to lint:

import sys
from pylint import lint

THRESHOLD = 5

if len(sys.argv) < 2:
    raise ArgumentError("Module to evaluate needs to be the first argument")

run = lint.Run([sys.argv[1]], do_exit=False)
score = run.linter.stats['global_note']

if score < THRESHOLD:
    sys.exit(1)

Answer 3

Install

> pip install pylint-fail-under

And you can check the threshold value as below

pylint-fail-under --fail_under=6.0 test_pylint_code.py (or path)

If the score is below 6.0 it returns a message

ERROR: score 5.3999999999999995 is less than fail-under value 6.0

Else it returns exit code 0.

Link to official documentation is https://pypi.org/project/pylint-fail-under/

Answer 4

The fail-under option also didn't work for me. So, I explored the pylint module code and came across the logic which checks the score against the fail-under value, and then performs sys.exit with different digits:

• 0 if the score is above the threshold, and
• nonzero if below threshold.

In both cases, it exits without raising an error or exception; hence, we no error is output to the console.

Modify the pylint/lint/run.py file as required to force code failure if the score is below some threshold:

if score_value >= linter.config.fail_under:
    sys.exit(0)
else:
    # We need to make sure we return a failing exit code in this case.
    # So we use self.linter.msg_status if that is non-zero, otherwise we just return 1.
    raise RuntimeError(f"ERROR: score {score_value} is less than fail-under value {linter.config.fail_under}")
    #sys.exit(8)