How Certifications Are Born
Page 2 of
3 1 2 3
Even though Novell hadn’t planned on certification becoming its own profit center, it actually returned more than their original investment. Meanwhile Novell’s resellers sold more NetWare, and made more money. And Novell’s ultimate customers installed more NetWare and were more efficient in the office place. Clark says, “It was an absolute win-win-win scenario.”
Once an organization has decided to create a particular certification program, the next decision is whether to develop it themselves or hire a professional certification building company to facilitate the process. In fact, most companies do outsource certification program development because the kind of people who author certification programs rarely exist in most IT companies.
The Players & Resources
The talent pool needed to create a valid and reliable certification program includes subject matter gurus familiar with the nuts and bolts of the product as well as one or more individuals with expertise in psychometrics, a field that cross-breeds psychology with statistics. Psychometricians are concerned with the design and analysis of research and the measurement of human characteristics, which is a critical part of certification test development.
A slew of additional human and financial resources are needed. According to Novell Certification Program Manager Nancy Seamons, the list includes course developers, exam developers, operation costs to store and deliver student kits, and increased marketing costs. Outsourced expenses include outsourced development costs, exam costs, vendor exam delivery costs, and then fostering acceptance by outside training partners to deliver the courses.
None of these things come cheaply, and program development usually involves input from multiple areas within the organization. “Buy-in is the most critical thing because management has to approve of the whole process,” says Yeung. “You need resources available to help in development of the test – subject matter experts from all over the company, your training department, your tech support department if it’s a product-related certification,” she adds.
The Process
SAS Institute develops their IT certifications in-house. Linda Althouse, Ph.D., a psychometrician and manager of the SAS Certification Program, says that her company follows standard test development techniques, which means “starting with a job task to let us know how are people using our software, and what are they doing with our applications development package.”
Based on that, they develop a test that they send out for a survey to see if a specific task statement is important for a particular job. That gives SAS a gauge as to how much emphasis the exam should have in that particular area.
SAS then takes that data and develops a test, bringing in item writers who write the items. SAS reviews the items in a technical review and they go through a code check to make sure the code works. This step can take up to several months. SAS follows that with a beta test to find out what items work and what items don’t work, and then puts together the actual production exam.
After that, SAS monitors the test to make sure that their item statistics still perform well. They keep an eye out for things such as 100% of exam takers suddenly getting an item right because someone published the item on a mass bulletin board.
