(UPDATE: So I wrote this post a while ago. Since then, this happened. I’m leaving this up for posterity, but I can say correct decisions are starting to be made on the topic. Hurray! – RB)
I was on Facebook last night when a pastor friend of mine posted this post from Rev. Timothy Cargal’s blog on the Office of Preparation for Ministry website. In it, he talks about the results of this years Bible
Trivia Content Exam.
And then goes on to defend this years BCE and the changes that they have made.
When I called it the Bible Trivia Exam, I wasn’t just speaking of my own opinion. Everyone I went to seminary with – everyone – had roughly the same opinion of the test. It doesn’t test your fitness for ministry or your professional abilities – that’s the purpose of the other four exams – and it doesn’t test your useful knowledge of scripture, but rather if you picked up the trivia of scripture that they decided to test you on this year. For example, this year1 they were tasked with matching salutations to epistles. Mind you, not once in my ministry would have I ever benefited from knowing off the top of my head which multiverse Pauline salutation goes with which letter, but somehow a junior in seminary needs to know that to show familiarity with scripture.
Before I really get into why I’m frustrated, I need to write out what the purpose of this exam is. When you are a junior2 (or just finished junior year), you take a test to demonstrate your familiarity with scripture. The point – supposedly – is to demonstrate that you are familiar enough with scripture to successfully complete seminary and also point out areas of weakness that you have within scripture.3
At least, I think that’s why you can fail it. Try as I might, I can’t rationalize another reason why this is a necessary exam to pass and not just an evaluation of current state of knowledge. And before someone says that it’s proof of biblical knowledge for a professional reason, the descriptions of the 5 exams specifically say this isn’t that and the senior exams are specifically professional exams in contrast to this exam. So it must be proof of knowledge to continue into the ordination process or seminary studies.
What change recently was two things. First, they stopped releasing the new questions to the exam. For 30 years students could access the exams – paper form and then, in recent years, online form to take as practice tests. The test has historically used 83% old questions and 13% new questions but importantly, those questions could pull from the entire history of the test.
And the old tests were a good study tool. The OPM is convinced that students were just memorizing answers to pass the test, but every student preparing said that they learned things about scripture using the old tests. It actually was quite helpful in giving us a wider primer is scripture.
I still remember who Jael is to this day because of it, and I read that story specifically because of studying for the exam.
In 2009, the OPM decided that they would stop releasing the old exams entirely. No releasing of questions missed, no allowing CPM’s to view the exams, nothing. So every year since 2009, the exam has been completely hidden except to test takers actively taking the test. All you get is your score and how well you did on each section of scripture.
Every year they have come up with new questions and phased out old questions. And as it has come to pass, this is the year when the exam is 100% previously unreleased questions. They still recycled questions, but every question was created post-2009. And this year, because multiple choice wasn’t quite…adequate?…they added the matching section.
Now, there is nothing particularly egregious about those changes without consequences. Yes, it takes away the only useful resource short of stringently studying the entirety of scripture in the months before taking the exam – all while being in seminary and having an internship and trying to also remain emotionally healthy and not burning out – but if people continued to pass at a similar rate and everyone agreed the changes were good, then it’s just students complaining because they want it to be easier.
That’s not a valid complaint, no matter how much I would have wanted it to be.
Why I even took the time to write this post – my BCE was in 2011 and this has no relevance to me directly – is the results of this year’s test.
71.7% of people taking the exam failed this round. Compared to the Fall 2009 failure rate of 18.3%.
Rev. Cargal chose to write that as a percent passed (28.3%) but I write it as the majority, not the minority, to not hide what actually happened: over 70% of inquirers taking the exam were told that they do not have the biblical knowledge to continue in the ordination process or their seminary studies (I’m still not sure which).
Which is insane. I know that it can’t possibly be that over the last 6 years students have gotten progressively less qualified for seminary that the failure rate jumped 291.8%, and I know that people who passed previously were mostly all ready and qualified to continue in the ordination process or seminary studies. This isn’t an indication that it was too easy before with the old exams, no matter how much the OPM wants the stubbornly claim that it is.
This is a sign that the test is flawed, and I think that everyone knows it. Some people are good at an exam like this and have this variety of recall. They are the ones you want on your team for Trivial Pursuit and you don’t want to go against in Trivia Crack. But ministry is not a game, and your level of Biblical knowledge and familiarity is not based upon recall of minutia.
But what I think is most frustrating, given all of this, is the absolute defense of the changes. Not once in the note – that was originally an e-mail to CPM’s and as such, not immediately public – did Rev. Cargal mention that they would evaluate the testing procedures, take a hard look at if students truly were unprepared for study, or even acknowledge the frustration that would come from this. It is all a description of the changes and a strong implication – intentional or not – that the 70% failure rate should be expected.
And on top of it, when CPM members commented that the CPM’s would benefit from seeing the questions and what their inquirers got wrong, the only answer that was given was security concerns. He implied, hopefully unintentionally, that test takers – who are studying to be pastors, mind you – are going to look for ways to cheat, and CPM’s – pastors and ruling elders tasked with preparing people for professional ministry – can’t be trusted to follow the instructions of the OPM and not give students the test questions beforehand. There is only a security risk if someone is posing that risk. The level of mistrust that the security comment shows is astounding.
And what is most astounding and confusing, to me, given everything else, is that this junior exam that is not supposed to weed people out but give guidance in their preparation is being made intentionally harder while the professional exams are being made considerably easier. Not that long ago they were 3 hour exams of 3 questions, 1 open book and 2 closed book, and proctored. And hand written not that long ago. When I took them, they were 3 hours, 3 questions, completely open book, and unproctored, which was perfect for me but also required less preparation. Now they are 9 hours, 3 questions, unproctored, and considerably easier because there is no effective time limit now.4
It makes no sense. I know I probably shouldn’t write about this and blast an entire office of the General Assembly publicly, but the changes are bad and I’m going to call them bad. The Presbyteries Cooperative Committee on Exams seems to be at least in connection with a segment of the church and actually taking comments and criticisms to heart with the senior exams (and responding to the criticisms they are not acting on) but whomever is responsible for the BCE seems to be stubbornly holding some ideal up without considering the response and a belief that they are right despite widespread disagreement.
I’m going to point out that my pastor friend is not a young friend, nor an old pastor. She is second career, wore her Master’s robe the same day that I did, and is not rebelling against the system. And she disagrees with the direction it is going, too.
I just wish, above all else, that the response to criticism was actually a response, and not just, “(It is an) exam security measure.” Because if I was one of the inquirers that failed this round, you can bet I would be furious (and it sounds like at least one CPM is pretty agitated, too).
Comments? Thoughts? A solution? Because, frankly, I’m at a loss, and it doesn’t even affect me.
1This was according to committee member on a CPM somewhere within our denomination after talking to inquirers who failed the exam. I can’t guarantee it because the OPM refuses to give any information about the actual content of the exam so take it with the reliability you can reasonably attribute to it. Given the content of the previous exams, though, I wouldn’t doubt it…
2Your junior year of seminary is your first year. Then you have middler year(s) and senior year.
3Mind you, my CPM and I have never discussed which areas I was weakest in with that test, only that I passed. And I don’t fault them for it in the least; I feel they probably view it similarly, even if they would never admit it.
4To answer a potential question, any candidate could work with their committee to make adjustments to the testing to accommodate learning disabilities that would not interfere with fitness for ministry.