How to Grade Coaches

Reader Thomas H checks in with this interesting idea for rating a coaching staff. Kick it around as open thread today: how do grade coaching staffs?

I live in Cleveland, Ohio.  5 weeks into the 2011 NFL season and Browns fans were jumping all over the head coach – Pat Shurmur.  Is he a good coach?  Is he a bad coach?  Is he just mediocre?  Who knows?  Who can say?

I wondered, at 2-2, is Pat Shurmur a better head coach than Jim Caldwell at 0-5?  By record, Pat Shurmur is better.  But, is he really a better head coach than Jim Caldwell?

I spoke with many people about the issue of grading a head coach and his coaching staff.  It was recommended that I grade on the following:  red-zone offensive efficiency; red-zone defensive efficiency; player potential versus potential realized; reviewing every individual coaching position by responsibility and effectiveness; etc…  I was thinking more of grading a coaching staff based on the following criteria:  team record; team record adjusted to account for improvement over last year; offensive, defensive, and special teams effectiveness (Total DVOA); strength of schedule; team unity and cohesiveness (the coach’s ability to control his players and staff, as well as his ability to avoid in-game and game-losing gaffes).

I considered including 2 more elements to the grading criteria:  the loss of an MVP-caliber player; and additional points gained for making the playoffs (seed # 1 & 2 = best; seed # 3 & 4 = next best; seed # 5 & 6 = next next best; else zero).  I resisted adding the first because the Patriots lost Tom Brady for the season in game 1 and still won 11 games that season.  The loss of an MVP-caliber player meant absolutely nothing in that instance. Why should the coach get extra bonus points for losing a great player when the other players picked it up, kept going, and had a successful year? Regarding the second point – scoring based on playoff seeding – a large part of that is outside the power and control of the coaching staff.  Some of the seeding is luck, some depends on which conference and/or division the team is in.  Look at the 2010 Seahawks.  Do a 7-9 record and a wildcard birth really warrant giving the coaching staff extra points?  In any other division in the league, 7-9 would have been mediocre and would not have warranted a playoff spot.  Since the coaching staff can’t control for luck and conference / division placement, I decided not to grade on it. As a result, I excluded both of these elements from my grading process.

So, here is how my proposed grading process works:

Element

Description

Weight

Maximum
Coaching Score

1

Team Record

1

1.000  (100.0%)

2

Adjusted Team Record

1

1.000  (100.0%)

3

Strength of Schedule

1

1.000  (100.0%)

4

Team Power Rank (Total DVOA)

1

1.000  (100.0%)

5

Team Unity and Cohesiveness

1

1.000  (100.0%)

5.000  (100.0%)

Each Element is weighted equally.  The team record is included twice: once as Element 1; and once as a part of a calculation in Element 2.  Since the overall win-loss record holds more weight at the end of the season than any other statistic, I included it twice.  Also, in some franchises, the head coach gets fired due to their win-loss record.  It’s the first thing anyone looks at, so I am treating it as if it holds more weight than any other Element.  Elements 3, 4, and 5 are weighed one time each.  Each Element could have a maximum of 1.000, or 100.0%.  The maximum total score for a coaching staff could be 5.000, or 100.0% (weighted average).

Once a numerical score is assigned to each Element, a “Summary” grade is assigned to the Element.  That breakdown is as follows:

Wins

Games

Grade

Summary

0

÷

16

=

0.000

=

0.0%

Disastrous

1

÷

16

=

0.063

=

6.3%

Awful

2

÷

16

=

0.125

=

12.5%

Awful

3

÷

16

=

0.188

=

18.8%

Awful

4

÷

16

=

0.250

=

25.0%

Bad

5

÷

16

=

0.313

=

31.3%

Bad

6

÷

16

=

0.375

=

37.5%

Mediocre

7

÷

16

=

0.438

=

43.8%

Mediocre

8

÷

16

=

0.500

=

50.0%

Mediocre

9

÷

16

=

0.563

=

56.3%

Good

10

÷

16

=

0.625

=

62.5%

Good

11

÷

16

=

0.688

=

68.8%

Very Good

12

÷

16

=

0.750

=

75.0%

Excellent

13

÷

16

=

0.813

=

81.3%

Excellent

14

÷

16

=

0.875

=

87.5%

Exceptional

15

÷

16

=

0.938

=

93.8%

Exceptional

16

÷

16

=

1.000

=

100.0%

Perfect

The same grading scale is used to assign a “Summary” grade to the weighted average of the Coaching Score.

I selected 5 teams in 2011 to analyze and review:  Browns, Colts, Bills, Lions, and Packers.  The overall Coaching Scores of the 5 selected teams were as follows:

Team

Record W-L
(Winning % Rate)

Coaching
Score

Coaching
Summary

Colts

0-5  (0.0%)

30.9%

Bad

Browns

2-2  (50.0%)

41.8%

Mediocre

Lions

5-0  (100.0%)

84.0%

Excellent

Bills

4-1  (80.0%)

85.6%

Excellent

Packers

5-0  (100.0%)

88.4%

Exceptional

Please notice that the Lions have a worse Coaching Score than the Bills even though the Lions have a better record.  The Bills have a much better Total DVOA and a stronger Strength of Schedule, so that mathematically accounts for more than 1 extra victory. Also, look at the Bills Coaching Score.  It’s higher than their Winning % Rate

I follow the Colts as much as I can, but am closest in proximity to Cleveland. I get to hear all of the bad stuff:  motorcycle crashes on bye weeks; contract issues and complaints; coaching mistakes; etc…  Although the Browns have a better record than the Colts, there is significantly less team unity, cohesiveness, and control.  Let’s see how that affects their overall grade:

The Browns record is 2-2.  The Browns have an Element 1 score of 50.0%.  The Browns have had 3 major gaffes in 5 weeks, so they lose 60.0% of their possible points in Element 5 – leaving them with a 40.0% Element 5 score.  When you weigh all five Elements, the Browns Coaching Score is 41.8%.  The Colts, on the other hand, have a very good Element 5 score (100.0%) but have a dreadful record (0-5 = 0.0%).  The Colts overall Coaching Score is 30.9%.  If we were to adjust the Colts record to 2-2 (to match the Browns 2011 record), the adjusted Coaching Score for the Colts would be 53.3%.  That’s 11.5 absolute % points better than the Browns current Coaching Score…  all on account of the coach’s inability to control his team and avoid major distractions and gaffes. It’s hard to concentrate on football operations when you have to focus on fixing or addressing the major mistakes.

For perspective on Jim Caldwell and the Colts coaching staff, let’s look back at 4 select seasons:  1992; 1999; 2006; and 2009.  1992 marked the season where Ted Marchibroda helped turn things around… with a +8 win differential over the 1991 season.  1999 saw Jim Mora at the helm… and a +10 win year-to-year turn around.  2006 was the Tony Dungy-led Super Bowl-winning season.  2009 was Jim Caldwell’s first year as a head coach – also a Super Bowl season…  Their season breakdowns are as follows:

Year

Coach

Record W-L
(Winning %)

Year-to-Year
Record Δ (# of Wins)

Coaching
Score

Coaching
Summary

1992

Marchibroda

9-7  (56.3%)

+8

53.9%

Mediocre

1999

Mora

13-3  (81.3%)

+10

73.0%

Very Good

2006

Dungy

12-4  (75.0%)

-2

76.1%

Excellent

2009

Caldwell

14-2  (87.5%)

+2

80.2%

Excellent

Of the 4 seasons analyzed above, the best Coaching Score was in 2009 – an overall score of 80.2%.  The coach:  Jim Caldwell.  Who is a bad coach now?  Surely not Jim Caldwell’s coaching staff in 2009!  Also, consider the fact that very few pieces of that coaching staff have changed in the last 2 years, so he likely hasn’t magically become a bad coach over night.

If you look at the Coaching Score and compare it to the Winning % Rate, you will notice that one coaching staff had a higher Coaching Score than Winning % Rate.  It was the 2006 coaching staff, which was led by Tony Dungy.  It appears that the Coaching Score was better than the team’s record would indicate (75.0% win rate), meaning that the coaching staff was likely in a great position to pull all the right strings and inspire all the right players once the team got to the playoffs.  It worked.  The Colts came home as NFL champions.  One team in 2011 has a better Coaching Score than Winning % Rate – the Bills.  It will be very interesting to see how the coaching staff and team performs if they make the playoffs.  Perhaps they will have similar results as the 2006 Colts.

Looking back to 1992, my memory recalls the 1992 season as being fantastic.  Chief Brodi – as my buddy always called him – really turned things around.  Yet, when you look at his Coaching Score, you realize that 53.9% is rather mediocre.  So much for my fond memories of 1992!  If it weren’t for the 3 negative scores in Element 5 for the 2011 Browns, the Browns coaching staff would be just as good (53.8%) as the 1992 Ted Marchibroda-led Colts coaching staff.  Hurts, doesn’t it!  See if that makes the folks in Cleveland feel any better!

Let’s do one last thing…  Let’s take the Coaching Scores for the 5 teams I selected to analyze in 2011 and multiply them by the number of games in an NFL season.  The results are as follows:

Team

Coaching
Score

# of
Games

Projected
# of Wins

Colts

30.9%

x

16

=

4.944

Browns

41.8%

x

16

=

6.688

Lions

84.0%

x

16

=

13.440

Bills

85.6%

x

16

=

13.696

Packers

88.4%

x

16

=

14.144

 

IF the Coaching Score could predict the 16-game record, then the Colts could win 4 games this season.  The Browns could win 6 games.  The Lions and Bills could win 13 games.  The Packers could win 14 games… maybe more…  Only time will tell.

When you review all of this, you can argue that the players make the coaching staff look good (the Peyton Manning argument).  Perhaps the coaches get too much credit and the players not enough credit. But, I also can argue that coaches can implement systems and call plays that can make good players look bad.  Consider Peyton Hillis in the current Browns offensive scheme.  He is being mis-used.  His potential isn’t being maximized. I would put that on the coaching staff…  So, even though you may say that the players control the outcome of the games and, therefore, the team record, I suggest that the player’s role is to execute the coaching staff’s game plan on a weekly basis.  The game plans and ability to adapt and change during the game is the responsibility of the coaching staff.  It’s a symbiotic relationship; one needs the other.  The players need the coaching staff’s game plans, knowledge, leadership, and adaptability, and the coaches need the players to execute.  It is impossible to separate the two. By looking at the Total DVOA, I see the Total DVOA rank as a reflection of the execution by the players.  And, to me, that is directly tied back to the coaching staff – offense, defense, and special teams.  So, in my opinion, the Total DVOA is as much a coaching grade as it is a player’s grade.

Quantcast