Performance Quotients:
A Performance Measurement Tool for MTF Awards Committee

Dave Clingan 1/12/08

DEFINITION OF PQ

Performance Quotient (PQ) is a formula for measuring the strength of a track or field mark as compared with the American Record and the top three ranked marks in the same event and age group. As an indicator of performance strength, it is designed to be used as a tool to assist the MTF Awards Committee in evaluating and comparing the accomplishments of athletes in various events and age groups.

There is no measuring tool or system that can definitively determine the better of two marks in different competition events. However, the PQ can provide some insight as to which athlete was more dominant within his/her age group and event during the year.

The MTF rankings database provides the source of marks used in PQ calculation.

CALCULATION OF PQ

The formula for PQ is as follows:

P = Performance
R = American Record

A = Top Ranked Mark in the event
B = Second Ranked Mark in the event

C = Third Ranked Mark in the event
V = Average of R, A, & B

For Distances (field events), PQ =

97 x [P – (A – C)x.1]

---------------------------

V

For Times (running events), PQ =

97 x V

--------------------------
P – (C – A)x.1

CORRECTIVE MECHANISMS

(1) The formula contains a correction for events in which the American Record is "Bemanesque", meaning so superior that any seasonal mark measured against it will always be disadvantaged. Therefore, the American Record is averaged with the top two seasonal marks (V = average of R, A and B), to adjust for hyper-superlative records.

(2) The formula contains a correction for a bias that has been referred to as the "compression effect". It is the nature of some events that marks are clustered very closely together, while others are more spread out. In compressed events, the top ranked mark typically has greater mathematical proximity to the record. For example, the top ranked M40 100m time of 10.50 is 93% of the WR (9.79); whereas the top ranked M40 discus throw of 52.53m is only 71% of the WR (74.08). Common sense tells us that the difference is due more to the compression effect than to a significant quality discrepancy between the time and the throw.

In the PQ formula, the compression correction is expressed as (A – C) x.1 for distances and (C – A) x.1 for times. This expression provides for an adjustment that reflects range of separation between the first ranked mark and the third ranked mark in the event. The greater the range, the greater the correction.

(3) Events in which there were only one or two athletes were not amenable to the PQ formula. This occurred primarily in older age divisions (85 and up). In order to include outstanding performances in such cases, a simplified version of the formula was used:

V = average of the American Record (R) and the Performance (P)

For Distances: PQ = 100 x P/ V
For Times: PQ = 100 x V/P

VALIDATION

PQ scores were calculated for performances derived from the 2007 outdoor rankings database. This database consists of over 5,400 masters athletes. Not included for purposes of this study are: (a) indoor marks, this study are 30-34, (b) athletes age 30-34, and (c) combined events scores) Athletes who's PQ scores placed them in the top 40 within their respective age bracket (M35-59, M60+, W35-59, W60+) were extracted from the data base and closely examined. Extraction of the top 40 athletes in each bracket is accomplished by selecting only those athletes who scored above a certain PQ threshold (approximately 96). This data is contained here.

MICRO VALIDATION

Micro validation is a process of looking at the selected PQ scores and assessing whether the data makes sense. 13 the top 17 M35-59 PQ scores exceeded previous American Records, which is to be expected. The highest score, achieved by M35 Weight Thrower Aaron Linerud at 106.83 was remarkable in that it not only broke the AR by a substantial margin (topping the old mark of 12.15m by almost 4m) but also in that it surpassed the second and third ranked marks by margins of about one and two meters respectively. Not all top PQ scores involve record-breaking performances. For example, Peter Magill did not break his own American Record in the M45 3000m, but he achieved the 9^th highest PQ at 100.34 with a time of 8:48.80 due to its superiority over the second and third fastest times, which were nearly one minute slower.

It is expected that an athlete who is consistently good at several events would achieve similar PQ scores in each of those events. Here is an example:

NOLAN SHAHEED M55
800m 2:05.43 = 99.38
1500m 4:22.89 = 98.59
5000m 16:40.14 = 98.53
MILE 4:42.7 = 98.24

In my experience with these particular running events, I believe the Shaheed's times are very comparable and therefore the similarity in PQ scores is consistent with my expectations.
Obtaining more micro validation feedback from other athletes with experience in other competition categories would be extremely beneficial. For example, I would like to hear from throwers, jumpers and sprinters if the PQ scores in those events are consistent with their expectations based on their experience.

MACRO VALIDATION

Macro validation involves statistically analyzing the PQ scores in various categories to look for undesired trends or biases in the system. The PQ formula should not inherently favor one event over another, or one age group over another. In other words, all athletes should have an equal shot at achieving an elite level PQ score regardless of which event they compete in or which age group they are in. The charts below show how many athletes achieved PQ scores ranked in the top 40 highest scores, based within various types of events and within different age groups. Additional study would be helpful in interpreting this data, but the chart does not reveal any glaring biases or inequities which would not likely be explained by variances in the number of active athletes per event or age group and other factors. Athletes of all events and all age groups are well represented.

Distribution of TOP 40 PQ Scores Among Events

	Distance	Sprints	Throws	Jumps	Hurdles
M35-59	10	12	9	6	3
M60+	8	10	10	8	5
W35-59	19	9	5	5	6
W60+	11	13	15	6	4

Distribution of TOP 40 PQ Scores Among 5 Year Age Groups

M35-59	W35-59	M60+	W60+
M35 = 4	W35 = 5	M60 = 6	W60 = 8
M40 = 9	W40 = 12	M65 = 8	W65 = 7
M45 = 12	W45 = 13	M70 = 5	W70 = 8
M50 = 10	W50 = 5	M75 = 4	W75 = 6
M55 = 8	W55 = 8	M80 = 6	W80 = 6
		M85 = 4	W85 = 3
		M90 = 3	W90 = 3
		M95 = 4

RECOMMENDATIONS

I recommend that PQ scores be calculated by the Masters T&F rankings system for the 2007 Indoor and Outdoor seasons. It will be possible to produce a summary page of top PQ performances for the entire 2007 year as a tool for helping to identify the most exceptional marks of that year.

PQ scores can be used by the awards committee for the purpose of evaluating performances as they see fit. However, it is not recommended that PQ scores be used as the sole criteria in evaluating the relative merit of different performances. It is only intended as a tool to assist in making such an evaluations.

Finally, use of PQ scores in the process of evaluating 2007 awards nominees will undoubtably produce valuable feedback which will help improve the system.