Evidence Scoring System

In simple terms, instead of looking at evidence and scoring Yes or No for adequacy, an inspector has to look at 6 defined aspects and score them from 1 to 4 to justify their decision.

The scoring adds some nuance, but it is still the Inspector's judgement, the same as before.

4 = Evidence shows an exceptional standard
3 = Evidence shows a good standard
2 = Evidence shows some shortfalls
1 = Evidence shows significant shortfalls

Don't waste time trying to understand the scoring system

• The scoring system and aggregation of points is a toolkit designed to assist inspectors.
• It reflects a professional opinion, it is subjective and dependent entirely on the inspector.
• Understanding the methodology used is unnecessary and adds no benefit.

Improved scores are only achieved through robust processes and evidence from your governance system and ensuring that this aligns with the Quality Statements.

The best time to tackle scores and ratings is at an actual inspection or assessment by engaging with the inspector on how they arrived at the rating.

 What you need to know

1. Each Quality Statement receives a score based on the evidence available
2. These are aggregated to arrive at a score for the Domain Question
3. The scores are aggregated again to arrive at an Overall score

What you don’t need to know:-

There are aggregation rules with sub-rules on how everything is added up and weighted. You cannot predict this accurately, and the inspector has leeway to change this depending on their findings and judgement.

The formulas are not fixed:-

Aggregation: “If we identify concerns during an assessment, we will use our professional judgement to decide whether to depart from applying our ratings principles”

Weighting: “We can adjust the following principles for combinations where it is not appropriate to treat ratings equally”

Old ratings are redundant

The old ratings are not like-for-like, you cannot compare them to new ratings as they are measured completely differently

What this means is that a Provider rated as Good might still be Good, but they may well fall short on the new standards, as the focus has changed more towards external feedback from the old site inspections and internal evidence.

In very simple terms, this is all you need to know:-

1. Each Quality Statement has 6 types of Evidence
2. The inspector scores each of the evidence  types
3. Adding up these scores gives a total score for each Quality Statement
4. Adding up Quality Statement scores now give you a total score for the Key Question/Domain

(By adding up we mean a method of calculating and aggregating is used to arrive at an "average")