As Senior Director, Exam Development and Performance Analysis for ICMA® (Institute of Certified Management Accountants), I’m fortunate not only to be part of a great team that develops a first-class exam, but I get to see the results of the team’s efforts. At the heart of the CMA® (Certified Management Accountant) exam are multiple-choice and essay questions. We have a team of more than 40 CMAs who put extraordinary effort into writing questions for the CMA exam. The efforts of this talented team of writers results in the development of more than 1,000 questions annually.
Question writers are assigned to write questions only in their areas of expertise, ensuring questions of the highest quality. CMA question writers come from all walks of life—academics in accounting and finance as well as accounting and financial professionals working in a variety of roles and industries. They also come from all parts of the globe, including Canada, China, Germany, India, New Zealand, Saudi Arabia, UAE, and the United States. This diversity plays an important part in making the CMA exam robust, relevant, and global in nature. The one thing CMA question writers have in common is their dedication and commitment to the CMA program.
THE PROCESS
The journey from concept development to exam question is long and rigorous. When ICMA receives questions from the writers, we assign each question a unique identifier. Questions are subject to an initial staff review during which they are checked for accuracy, edited for clarity, and categorized in one of three cognitive levels. The cognitive levels, or Bloom’s Taxonomy, were created in 1956 to promote higher levels of thinking. Cognitive level A is definitional and tests knowledge and comprehension; cognitive level B is analytical and tests application and analysis; and cognitive level C challenges the candidate to interpret a data set or situation and make a recommendation, and it tests synthesis and evaluation. It’s important to note that cognitive levels don’t represent a question’s level of difficulty but the thought process that’s needed to answer a question. The cognitive levels enable the CMA exam to test candidates at the highest level of thinking.
THE DOCUMENTS
Each question must then be linked to the Content Specification Outline (www.imanet.org/cso), an important document that details the content that is tested in each section of the CMA exam, and to the Learning Outcome Statements (www.imanet.org/los), which spell out what a candidate is expected to know in each content area. It’s at this point that a question can move to the final review and approval by one of ICMA’s exam review groups. These review groups consist of members of the ICMA Board of Regents and the Exam Review Committee. They represent a talented group of subject-matter experts with a great breadth and depth of knowledge.
The exam review groups conduct an exhaustive review of every question and make the final determination of the question’s suitably for the CMA exam. This second review is done to ensure that every question on the exam relates to the CMA body of knowledge and is something a CMA needs to know. Only at this point is a question ready to become part of the CMA exam.
The Content Specification Outline that represents the CMA body of knowledge is a carefully crafted document. It’s created through an exhaustive process that begins with a job analysis. Best practice is to conduct the job analysis every five to seven years to identify the current knowledge, skills, and abilities needed to be a CMA. The process begins with a job analysis survey developed by a team of subject-matter experts.
The job analysis survey for the current CMA exam was sent to accounting and financial professionals who were asked to rank the importance of a variety of knowledge and skills in 11 domains to the work they do in their current position. The survey was distributed worldwide, reflective of the global nature of the CMA program, and key demographic data was collected to ensure the survey was reaching the appropriate audience. An analysis of the data collected from the job analysis survey resulted in the changes made to the content of the CMA exam effective January 2015.
The CMA exam has a large and robust bank of questions that’s continually enlarged and refreshed. Approved questions are periodically added to the exam question bank and tested as unscored questions. These questions remain unscored until sufficient psychometric data is collected to decide whether to score the question or delete it from the exam. This ensures that a candidate’s performance on the exam is based only on questions that perform well.
THE REVIEW
Before the exam window opens, several members of the ICMA team review the exam carefully. This is an important quality-control check to make sure everything is correct and accurate. Only then is the exam released, with testing at more than 500 Prometric testing centers in more than 70 countries worldwide.
The computer-based exam is delivered using linear on-the-fly testing (LOFT). LOFT delivers each candidate a unique and random set of multiple-choice questions. Every exam delivered using LOFT is similar in the level of difficulty. LOFT helps to ensure that the CMA exam is valid, reliable, and fair. A valid exam is one that accurately measures what the test is trying to confirm. In this case, we are confirming an individual’s ability to be a CMA.
A reliable exam is one that performs consistently, where an individual who tests a second time without increasing his or her knowledge will achieve the same score. A fair exam is one that’s appropriate for all qualified candidates regardless of age, race, gender, or nationality. This is particularly important for a global exam like the CMA, which has more than 28,000 candidates spread across the Americas, Asia/Pacific, Europe, and the Middle East/Africa.
The psychometric data confirms that the CMA exam is valid, reliable, and fair!
THE RESULTS
As a result of the hard work and effort of many to develop a rigorous, valid, reliable, and fair exam, CMAs can be confident that they have the skill set needed to be successful in their careers. Additionally, employers can be confident that when they hire a CMA, they are adding an effective, capable team member to their accounting and finance departments. If you want to join this group of nearly 4,000 highly qualified individuals who attain CMA certification annually, please visit IMA’s website at www.imanet.org to learn more about the CMA exam and the CMA program.
September 2016
