Where does "Certs Wanted/Needed" data come from?

At around this time of year, it's not unusual to see stories that report on the most popular, the highest-paying, or the most in-demand IT certs out there. I love reading stories like that, not just because I like to keep up with what's going on, but also because I always want to know where the data that drives such reports comes from. The source of such data is often a survey of some kind, for which it helps to know the following:

   ?  Who's been surveyed, or from what population were the reported results drawn?

   ?  How many surveys were completed, or what size was the actual sample from which reported results came?

   ?  What are the demographics for the surveyed population? Pick your metrics of interest here: age, race, education, years in the industry, reported pay �

   ?  What is the margin of error? (Often, you won't even see a number like this published, but that's mostly a sign that the survey was smaller and less formal, not necessarily a sign of suspect information.)

When I craft my own articles that rank leading or most popular certifications, I try to draw on as many sources of information as I can to put such lists together.

Those sources include as many relevant surveys as I can find (I especially like the ones from Certification Magazine; the various "World" publications — InfoWorld, ComputerWorld, NetworkWorld, and so forth; CompTIA and other industry associations; and niche publications that focus on security, storage, data centers; and on and on ... ), the usual "Best of ... ," "Most popular for employers," "Most popular for IT Pros," and similar reporting in the trade press and online, and anything else I can find that attempts to rate or rank these things.

This helps me put a slate of candidates together, after which I have to conduct a number of "reality checks" before I can figure out what's worth mentioning and what's best left alone.

When I get a list of certs together, though, the next step is to visit a whole slew of job boards — including Dice.com, Indeed.com, SimplyHired.com, and others — and to put tables of numbers together based on mentions of the certs by name or by acronym to see who's actually including them in job postings at the moment.

You'd be surprised how informative a collection of this kind of data can be, especially in the aggregate (adding all the mentions together across a half-dozen such sites or more seems to smooth out the data and level out certain sites that prefer some credentials to others with sites that go vice-versa).

The next step is to look at the program sponsors for the certs that remain in consideration. I try to get a sense of who's behind the cert (or cert program) under consideration, especially to see if the cert is legit and valid, or if it's a thinly disguised attempt to cash in on training dollars from interested parties who may not really understand how to separate solid, reputable sponsors from less desirable specimens of sponsorship.

In this context, I look for program longevity, transparency regarding costs, value, membership and number of certs earned, and what kinds of metrics and safeguards the sponsor discloses to ensure that its certifications are relevant to today's and tomorrow's marketplace needs for IT skills, knowledge, best practices, compliance, and so forth and so on.

With what I hope is a vetted list of candidates, what I do next is to check in with my network, to see what my friends and colleagues (those who I can contact in timely fashion) think of my list, and what they might be able to suggest that I either overlooked or didn't even know about.

This sometimes leads to a bit of iterative cycling, especially when new things pop up that have to be rated and ranked again, as described in the preceding paragraphs. This step is important, because I often learn that some certs look good on paper (or online) but may not actually cut the mustard when it comes to delivering the right set of skills and knowledge needed in the workplace.

Finally, I'll check in on some active cert-related forums and interest groups online, to see what they think of my prospective candidates, or if they're tuned into something that still hasn't managed to get onto my radar. I don't usually get too many surprises at this stage of the process, but sometimes they do pop up.

When they do, they're usually either of the "Why on earth did you include this?" or "How could you not include that?" variety. This, too, can lead to another cycle through the preceding steps, but that's all to the good as far as I'm concerned.

The more you check your data, and the more eyeballs you put on it, and the more feedback you solicit and incorporate, the better your results will be. Hopefully, this helps readers understand why reading survey results or Top 10 Lists is one thing, and deciding what certs you'd like to pursue yourself with your own time, money and energy is entirely another.