A Tale of Two Sittings (TBE Review)

By Donald Williams posted 07-02-2018 14:05


“It was the best of times, it was the worst of times…” Boy, does this describe the Exam 5 TBE experience, or what? Okay, maybe calling the make-up test “the best of times” is an exaggeration – after all, it’s not lying on a beach somewhere with a cold drink in my hand…or even just a solid 8 hours of sleep (I have three young kids). But the difference between the initial and make-up sittings was like night and day.

You’ve probably heard the horror stories by now, which I alluded to in my last blog on the first TBE sitting on May 4th. My remote proctor check-in didn’t seem to have any major issues, though it took about a half hour in total. However, just a few minutes in, the proctor paused my exam without warning. Before a half hour was up, I was completely locked out of my exam and unsure what was happening – if I was still connected to the exam, if the proctor could get my messages, if my timer was still ticking away, or if I was even allowed to call someone for help. For about 45 minutes, I didn’t receive any communication from my proctor, and was told by PSI customer support to just wait it out. Well, my exam eventually started again without notice, but before too long I got kicked out again in a different way and was forced to go to the PSI website and re-install the secure browser. This happened once more before the exam was finished. Luckily, the proctor was able to add on additional time for all of the delays. Was it the right amount of time? Who knows? The proctor didn’t seem to be able to see the clock after resetting it, because I was interrupted multiple times before the timer ran out, either telling me my exam was over too soon, or just asking to see the clock. When it was all said and done (about 8 hours later), though, I had attempted every question, and despite all my ordeals was better off than many others.

Fast forward to the make-up exam on May 31st, and the scene was immediately much more familiar. A proctor was there in person, checking IDs and whiteboards. While a little less formal than normal (e.g. there was no reading rules from a script), just having a body in the room that I could communicate with in the event of some question or technological failure was reassuring. I started my exam 15 minutes after the others in the room because that was the closest time slot available when I signed up. The exam was taken through Google Chrome, which was an improvement over the original testing environment because it expanded the Excel workspace on the screen. The feel of the make-up exam was very similar to the first exam with two major exceptions: the first and most notable was that I went the entire length of the exam without losing connection or being interrupted a single time; the second was that the auto-save action was slightly more distracting than the first sitting, even to the point of interrupting keystrokes when it hadn’t previously done so. Some of the Excel functionality still wasn’t there in the second exam but, having not been in the first exam, this wasn’t any worse. The proctors and I were out of the room by 5 hours and 10 minutes after the start of my exam time – right on schedule. I would imagine this process was probably even easier on the proctors without having all of those exam packets to deal with afterwards.

For me, there are three main takeaways from the experience of sitting for both exams:

  • More testing, please! Obviously, the CAS tested these systems before committing to this format; however, whatever testing was done still left PSI with the impression that their systems could handle the load, and that was clearly not the case. Not only is it good to know that the system can handle the candidate pool traffic, but it would be appreciated if we could know the anticipated performance differences between machines. I realize we can’t test every single computer model out there, but if there are patterns, say, between brands, it would be helpful to everyone, especially those shopping for new hardware, to know what works best.
  • Early communication is key. Many people were frustrated with how late changes were made to the tech requirements. I thought the CAS did a great job in communicating after the first exam – the make-up announcement was out before I even submitted my exam. During the week after, it seemed like we were getting almost daily updates. This is the type of urgency we would like to see going forward.
  • Troubleshooting – what should we do if something goes wrong? What will the proctors do if something goes wrong? The CAS should be able to draft some “what-if” guidelines to address the most common scenarios and communicate those to the candidates both ahead of time and on exam day before the exam starts.

With the make-up exam having gone so much more smoothly than the original sitting, and even being put together so quickly, I am curious to see what the TBE situation for the Fall sitting will be. Regardless of the outcome, be assured that your Candidate Liaison Committee representatives have all emphasized the need for early decisions and prompt communication. Personally, I wouldn’t mind seeing something similar to the make-up session, with hopefully the added functionality of the function key shortcuts in Excel. But my opinion is somewhat colored by the less-than-desirable experience of the first exam – pretty much anything would be better than that. I realize that many CAS members were overly gracious with their time, volunteering more than usual to accommodate our situation; however, I’m confident that, should the CAS decide to go with TBE and the live proctor route, we can work together for a more efficient distribution system in that respect. What would you like to see happen in the Fall? Leave a comment below and make your opinion known.

“It is a far, far better thing that I do, than I have ever done; it is a far, far better rest that I go to than I have ever known.” According to Dickens, I think this means I passed my exam with a 10, and I’ve definitely got the resting part down. While I may not find out my score(s), I look forward to finding out the final results soon. To wrap up my experience on this first TBE sitting, I would like to extend my thanks to all of the CAS members who reached out to me and the other candidates and empathized with our situation, as well as those who volunteered so much time on such short notice to proctor and grade the extra set of exams. This was a great example of our community working together to solve a problem and achieve a common goal, and with that fellowship I am confident that we will be able to produce a successful sitting in the Fall…whatever it may look like.