Demystifying the Bar Grading Process

It’s ironic. You spend years in law school, a fortune in tuition, and months in bar review to prepare for the most difficult bar exam in the country. You study harder than you’ve ever studied in your life. Then your bar review instructor informs you that the bar exam graders get paid about $3 to spend an average of 90 seconds to read and grade an essay – that precious product of all your hard work that could make or break your future career plans.

I’ll admit I was glad to know up front about the brevity in which my answers would be reviewed by some anonymous grader, flying through hundreds of blue books at the speed of light. It forced me to write more succinctly and answer the call of the question.

At the same time, I was a little offended at the notion. I mean, there had to be more to it than just some guy off the street speed-reading blue books and assigning a score based on a 90-second impression of an answer it took me an hour to write. I wanted to know that my efforts in producing quality essay answers were being reviewed by people making a quality effort to grade my work. There had to be more to the story.

Well, there is more. I checked into the State Bar’s grading system, and learned about teams of graders who undergo a fairly extensive grading calibration process for each essay question before the actual grading begins. Here’s the skinny on what they do:

There’s a separate team of graders for each essay and PT.

There are eight teams of graders – one team for each essay question and PT. Each team consists of 11 experienced graders and up to 3 apprentice graders, as well as assigned supervisors from the Committee of Bar Examiners (“Committee”). These teams are selected from a “grading pool” maintained by the Committee which is comprised of about 150 attorneys, the majority of whom have been grading bar exams for at least five years. So, each grader on an exam only reviews answers for one question. What this boils down to is you will have 8 different graders – one for each answer – on the written portion of the exam.

Each team undergoes 3 Grading Calibration Sessions.

The first session takes place 2-3 weeks after the exam, followed by two more sessions in the next few weeks. The sessions usually take place on Saturdays and the graders are paid a fee for participating in each session – they are not volunteers. Prior to the First Calibration Session, each grading team receives its assigned exam question and a set of sample answer books to evaluate independently. The samples are stratified by law school, repeater status, etc. so the graders see a cross section of the application population who took the exam.

First Calibration Session

Graders discuss any discrepancies found in their assigned question, and the patterns or problems in the answer books reviewed prior to the meeting. They then determine what weights to give to the issues raised by the question. After this discussion, a set of 15 answer books is distributed to the team. Starting with the first answer, the graders individually read and assign a grade to it, and then the group discusses and debates all assigned grades for that answer before arriving at a consensus grade. This process is repeated for each answer in the set. After reaching a consensus on grades for all 15 answers, the grading team is then given a new set of 20-25 answer books, which they grade independently without any further discussion, and submit grades for review at the Second Calibration Session.

Second Calibration Session

At this session, the supervising member of the grading team distributes grading guidelines that he or she drafted based upon the discussion at the first session. The team also receives statistical information on their independent grading from the First Calibration Session, reread any answers where significant disagreement was seen, and resolve the differences through further discussion. An additional 10 answer books are then read, discussed, and assigned a consensus grade. At this point, each grading team has now read, discussed, and consensus-graded 45-50 answers from their assigned question. The teams are then given their first grading assignments. Based on the agreed-upon standards, graders assign raw scores in 5-point increments on a scale of 40 to 100, and in case you wondered, the quality of handwriting and accuracy of spelling and grammar is not considered in assigning a grade.

Third Calibration Session

At this last meeting, grading standards are re-reviewed and the graders discuss their experiences and findings from their first grading assignments. They then read and grade 10 additional answer books, followed by discussion and grade calibration to ensure they are still grading to the same standards. After this session, the graders complete the remainder of the grading assignments.

Okay, so it’s not so ironic. There is a significant effort made to evaluate my significant effort. There is a system in place. It may not be a perfect system, since humans are involved and let’s face it – we ain’t perfect. But humans must be involved, as this is more than choosing answer A, B, C or D. Personally, I can’t think of a better way to grade the masses on such a subjective and voluminous compilation as the written portion of the California Bar Exam.