If you aren't familiar with the term, "Moneyball," it originates with the Oakland A's general manager Billy Beane's (and his mentor Sandy Alderson). The Moneyball method helped Oakland build a playoff baseball team on a smaller budget by relying on statistical analysis to acquire new players. When it came to baseball talent, Oakland knew it could never financially go head to head with the biggest teams in major league baseball, so the team started using an unheard of method in baseball, the Sabermetric principles.
This research and analytics based approach helped the A's identify undervalued athletes in the competitive MLB talent pool (as well decide which high school and college players to draft). Using this method the A's were able to reach the playoffs three consecutive years. Catalyst IT is applying similar principles to the IT hiring process to help companies build a better team of IT professionals.
[ Get sage advice on IT careers and management from Bob Lewis in InfoWorld's Advice Line blog and newsletter. ]
In 2001 Michael Rosenbaum, president and founder of Catalyst IT services began his journey with the goal of recruiting IT talent based on metrics rather than traditional hiring methods which include resumes, phone interviews and in-person interviews.
According to Rosenbaum, traditional hiring methods are biased and rely too heavily on the interviewer's perspective. "Resumes and interviews have never been a great way to figure out whether or not someone is going to be good in a job or role," says Rosenbaum. You wouldn't want to trust your company's bottom line to a hunch, so why would you hire people on your team that way?
Although Catalyst IT keeps the actual signals and data its uses confidential, Rosenbaum shares some insight into what he's learned from 10 years of using a Moneyball-like model for hiring IT workers and building Agile development teams.
Hiring practices meet big data
"This method enables us to look at performance metrics instead of simply credentials, which haven't really been able to predict whether someone will deliver," says Rosenbaum. The idea behind this Moneyball approach is to marry big data with the hiring process.
Using massive amounts of data across all the persons in their organization, Catalyst says it can predict with some certainty who will be the high performers on any given project. They use data-points such as how many functions/projects can a team complete in two-week window, how many can an individual complete in a two-week window, defect and rework rates, QA metrics, social networking data points, how they interact when applying online and more. "We are typically looking at a couple thousand data points," says Rosenbaum.
Rosenbaum says he plans to hire 150 people at Catalyst over the next year and estimates the company will look at more than 10,000 applicants before choosing ones that they feel are not only great employees but also that they fit into the culture of the company they will be working with.
The process is almost completely automated, says Rosenbaum and starts with an online application that asks for some basic information. Using algorithms they choose candidates for the next portion, which is a much more detailed online application that takes a few hours to complete. Applicants are being monitored and judged via resume data, keystrokes, time on page, public domain data and other proprietary signals.
Algorithms will then take that data and generate a probability score of whether that person will be a high performer over time. Catalyst's HR people get an immediate percentage number. If applicants reach a certain threshold then they will be interviewed and a majority wind up being brought onboard. In the first four months, new employees work with a team manager and perform tasks that the company has performed for their clients many times in the past, so they get to know how things are done there and also to build a baseline of metrics.
Build a winning IT team
There's another sports adage that applies here: Great players win games, but great teams win championships. Finding people who are not only great developers but also fit into your company culture is challenging in a competitive IT job market.
"How a developer handles stress and uncertainty is incredibly important, because the software development world is often one of big problems and short deadlines. One of the overarching qualities that we value most when hiring new talent, particularly as a software development firm, is the ability for a programmer to solve problems under duress," says Rosenbaum. Ten years of using this method has taught Rosenbaum a few things about building teams with the right people. Here are three key points, according to Rosenbaum.
1) Look beyond the resume
When you're looking for new talent, do your best to look beyond the resume. As an industry, we're so accustomed to using the resume as a barometer of potential employee success. This approach can seriously mislead and deter companies from finding the best talent. For example, in 10 years of screening more than 10,000 potential employees, I have found zero statistically relevant correlation between whether or not a candidate has a degree (undergrad or masters) and whether or not the candidate will succeed as a software developer.
Whether it's giving a potential employee a sample project to work on, or even conducting interviews without the resume at all, try to remove the potential bias and subjective analysis that can come from staring at a candidate through the dim lens of a piece of paper.
2) Big data is not impersonal
Big data is largely used to achieve increased financial results for companies across the industry. But we've found that the power of big data is far more compelling in areas of business that are classically subjective.
Big data can absolutely be personal --look at websites like Match.com, where thousands of people are connected to each other based on variables and insights they may not have discovered on their own. Similarly, when we use prescreening and analysis to bring a new employee on-board, we're working hard to make sure he or she is joining an environment in which they are geared for success. As they continue along their role at Catalyst, big data and objective measurement help us make sure they're continuing to thrive.
3) Culture matters more than experience
I've learned that I'd rather have a group of crack professionals who work well together as a team, than try to string together a bunch of lone rangers and pray they get along. In other words, I'd rather coach the New England Patriots than the Dallas Cowboys.
Rich Hein is a senior writer for CIO.com. He covers IT careers. Follow everything from CIO.com on Twitter @CIOonline, on Facebook, and on Google +.
This story, "How to use a 'Moneyball' approach to building a better IT team" was originally published by CIO.