As the College Football Playoff rankings are now unveiled on ESPN every week – and for the final time on Selection Sunday the first weekend in December – fans naturally have opinions about the selections.
The process behind determining which team is ranked where is more extensive than you might think. It’s a series of detailed meetings with intricate data, debate comparing teams’ qualifications and following committee protocol – one that loosely includes crispy bacon. Yes, bacon.
The Fiesta Bowl took part in a recent Mock Selection exercise, along with representatives from other Bowl games, to experience exactly how the rankings come to be on weekly basis. We had the same U-shaped conference table, chairs, laptop computers and projection screens that the committee uses and in the identical meeting space.
There was the hat rack outside the room where the committee members symbolically leave their personal allegiances behind, an ironing board to signify the committee is ‘ironing out the details’ and a hot plate that is well-stocked with bacon that the committee routinely picks at during recusals and breaks.
We also received a two-inch thick binder that outlined the protocol in determining the best four teams. Some of the factors in determining the rankings include conference championships, strength of schedule, head-to-head comparisons, outcomes of common opponents and other relevant factors that may have affected team performance or would do so in the postseason. The binder also had an information sheet for every school in the FBS.
We took on the 2013-14 season data for our exercise, which was to determine the season-ending rankings which lead to the playoff teams and bowl match-ups.
The process consists of seven rounds, each involving a listing step and a ranking step. In the first three rounds, there are six teams that make up the round, ultimately voting for three spots. The final four rounds have the same process, with eight teams considered for four places.
We took the first listing step and entered in six teams in no particular order. When the results were tabulated, we had a three-way tie for spots 4-6, which had never happened before.
So our first charge was to determine which two among Ohio State, Stanford and Baylor would make the top six, along with Alabama, Auburn, Florida State and Michigan State. We looked over the results of all three teams, debated the strength of schedule and who was conference champion. There were differing opinions and, after 20 minutes, we called to vote. Ohio State and Stanford passed on to the next round, with Baylor left out.
Now that we had six schools in the pool, we had to rank them in order to determine the top three teams. Our discussions quickly revealed that undefeated Florida State was No. 1 followed by Auburn. We then talked about ranking the remaining four teams in order.
The result: Florida State at No. 1, Auburn at No. 2 and Michigan State at No. 3.
With Alabama, Ohio State and Stanford as carryovers, we added six more teams in the listing step for Round 2. Baylor, Missouri and South Carolina joined into the group.
It became pretty clear that Alabama would be rated fourth and once again there would be back-and-forth among Baylor, Ohio State and Stanford to define which teams would be included in the top six.
We looked at how Stanford had two losses, including one to a 5-7 Utah team, and a significantly stronger strength of schedule. Baylor had scored a lot of points and only one loss, at ranked Oklahoma State. Ohio State was undefeated in the regular season and only lost to No. 3 Michigan State in the Big Ten Championship Game. So it was a ‘good’ loss. But Baylor and Stanford were conference champions.
The Utah loss stood out so we decided to look deeper into that game. It was a night game at Utah, a difficult place to play, and Stanford had six trips into the Utah red zone but couldn’t capitalize, including a goal-line stand in the final minute. The final score was 27-21.
Then we studied Utah’s results from that year, since the Utes were 5-7. We uncovered that Utah had close losses to quality opponents, had the ninth-toughest schedule and was better than its 5-7 record. Going through those extra steps made us conclude that a loss to a 5-7 Utes team wasn’t as bad as it seemed at first.
Posted on screens all around the room were the game-by-game results for all three teams, side-by-side. While Ohio State had only one loss and to the third-ranked team in the country, what stood out were the wins each team had.
That was a big point we made and a mantra carried throughout the exercise… a focus on what the wins were instead of who the losses were against.
Baylor had wins over Oklahoma and Texas, Stanford owned five wins over teams with nine-plus wins and six wins over teams that would likely be ranked in the final Top 25. Ohio State only beat one team with nine wins or more.
After 36 minutes of deliberation, the order for 4-6 was finalized, including having a re-vote. Alabama finished fourth to advance into the College Football Playoff Semifinals, followed by Stanford at No. 5 and Baylor at No. 6. The Buckeyes were out.
If you recall the first listing step, Ohio State made the six-team group and Baylor was left out. So the committee’s mantra to closely review all the available datapoints worked in evaluating the teams.
By the time the first six teams were seeded, 1:19 had elapsed.
This process continued until the top 25 teams were ranked. At any time during the process, there is the ability to call for a re-vote, as we did with slots 4-6.
After four hours, we agreed to our Top 25 and plugged in the teams to current-day conference affiliations and bowl match-ups. While the actual 2014 Fiesta Bowl showcased Baylor facing UCF, in this exercise Baylor was slotted into the Sugar Bowl as Big 12 champion.
In our mock, UCF still came to Arizona, but took on Oklahoma State.
Some other notes from the day:
- TV monitors display comparisons of up to four teams at one time
- When asked how our experience compared with how the actual committee weighs the teams, we spent more time trying to build consensus, while the committee compares teams and then votes to find out where the committee stands on rankings.
- We had a lot of ties in our voting, with ties in each of the four rounds
- We had a four-way tie in the fourth round listing step, between Arizona State, Duke, Louisville and UCLA. There has never been a four-way tie in the CFP Selection Committee history
- We had a tie in the final vote of the day, to determine No. 23 and No. 24. The teams? Texas and Texas A&M, which drew a hearty laugh in the room. Texas ended up winning the head-to-head vote
- Louisville was ranked No. 17 in the Fiesta Bowl’s initial Top 30 and No. 18 in the final BCS Rankings, but didn’t make it into the final Top 25 once the exercise was complete
- Oregon finished the 2013 BCS rankings at No. 10 but placed No. 15 in our mock exercise
- Notre Dame was not ranked in the final 2013 BCS rankings but ranked No. 19 in our exercise
- This was one of six mock selection exercises. Others comprised conference representatives, athletic directors, media, student journalists and ESPN announcers