Overview of Response Sets

In order for compliance projects to be successful, managers must know how to select the most adequate response set for each interview that will be sent. There are various types of sets that specify the possible response options for the Compliance questions in interviews. The selection of the most adequate one for each interview depends on the purpose of the project (an audit with binary responses, an audit that allows requirements to be partially met, evaluation of the maturity of processes, etc.) and other variables, such as the type of authoritative document with which compliance will be measured.

The selection of the response set for a compliance interview must be done carefully during a project, since the set is directly related to the generation of compliance metrics for the Compliance questions. To understand these metrics, it’s essential to first understand what response sets are, how they are created, and how they are used in compliance projects.

When preparing a compliance project, a response set must be selected for each compliance interview that will be sent. Sets for the various types of compliance analyses can be created in the Knowledge module (see figure below).

When creating or editing a response set, we can see that it has properties (title, language, description, author, etc.) and various response options. For example, the Maturity response set shown in the figure below has six response options. Note each response option has specific values associated for the Compliance Index and the Compliance Level. Option 2 (Insufficient) is associated with a Compliance Index equal to 20% and a Compliance Level of “Not Compliant”.

Response sets allow a value for the Compliance Index and the Compliance Level to be attributed to each response option available for a Compliance question, which measures the organization’s compliance with one or more requirements that have previously been associated with the question. For example, for the requirement “All employees should wear their identification badges”, the following type of compliance question can be created:

“Concerning use of your identification badge on the organization’s premises, mark the option below that best describes your habits.”

Suppose the response set with four options shown in the table below is used for the interview that contains the above question:

This means that interviewees will only have these four response options available when answering this question in the interview, and that for each response option selected, the system will automatically associate a Compliance Index (100% for the “I always wear it” option, for example) and a Compliance Level (“Compliant” for a Compliance Index of 100%).

The Compliance Index represents how much the organization respects or is in compliance with a certain requirement. The closer this value is to 100%, the greater the organization will be in compliance with the requirement. For example, once the answers provided by the various interviewees for a compliance interview are consolidated (considering the response set and its ranges), a Compliance Index for a certain requirement is shown to be 90%. This is a high percentage, but will it be sufficient for the requirement to be considered fulfilled by a certain organization?

This depends on the organization (its internal environment and context), on the nature and origin of the requirement (if required or optional), on the authoritative document it is associated with (internal policy, external regulation, law, etc.), and even the type of project underway (compliance audit, evaluation of the maturity of processes, etc.). In the system, the Compliance Level indicator allows a certain requirement to be defined as “Compliant”, “Not Compliant”, “Partially Compliant”, or “Not Applicable”, depending on the value of the Compliance Index. For example, a Compliance Index of 80% may be considered “Compliant” for a certain type of requirement (evaluation of the maturity of a process), but this same index of 80% may be considered “Not Compliant” in an audit where partial compliance with requirements is unacceptable.

As an analogy, the Compliance Index may be thought of as a grade on a test. If a student scores 70% on the test, the student may not know if it is enough to pass the class even though it is a high score. In this analogy, the Compliance Level indicates if the student passed the class or not with this score. For a certain university, it may be that a student be approved (Compliance Level) with a score of 60% (Compliance Index), but perhaps not in another class with the same grade in another university. For this reason, two indices must be used in the Compliance module to understand the results obtained. In the response set used in the example above, the interpretation of the value of 70% for the Compliance Index is that it is sufficient (Compliance Level = Compliant), or that it is enough to pass the class.

Note that a Compliance Level of “Not Applicable” for a certain response option does not generate compliance metrics for the requirements associated with the Compliance question. In the example above of whether or not employees wore their badges, if employees had no badges, the adequate response option would be “We do not have badges”, associated with a Compliance Level of “Not Applicable”.