Software Development Ranking
The boom in software development in the United States has created an intense demand for software developers that is far outpacing the supply. While it can be difficult to assess the level of skill and qualification of software developers by any definitive criteria, one qualitative 2015 survey listed eight metrics for developer performance:
speed, personality, lines of code, test coverage, code quality, technical debt, number of bugs, and closing tickets.
However, based on other provided sources, several factors stood out more consistently as potential metrics for evaluation. These include, but are not limited to, the consolidation of various school rankings, a school's research influence, accreditation, competition results, and the level of growth in the IT industry based on location. It is important to note that certain attributes, such as intelligence, problem-solving ability, and depth of knowledge can be difficult to assess, especially in groups, due to their somewhat amorphous nature. They may also be subject to personal interpretation and are thus better evaluated on an individual basis. Below, I will elaborate on the five aforementioned metrics and more details on the 2015 survey.
1. CONSOLIDATED SCHOOL RANKINGS
While school rankings vary based on the algorithm used to calculate the score, sources indicate that there are several schools which consistently score in the top 5 positions for Computer Science programs. By consolidating these different ranking systems and finding those programs which appear across the board, it is possible to come to a more reliable conclusion regarding the potential quality of the school. According to College Choice, World University Rankings, Niche, and Computer Science Rankings, MIT was consistently ranked among the top five schools, while other schools in these same lists varied in their position based on the particular criteria of the ranking system. It is therefore probable that MIT offers a higher quality of education that scores well in any ranking system, and as a result, may produce superior software developers and computer scientists.
2. SCHOOL'S RESEARCH INFLUENCE
Especially at the graduate level, the influence of individual professors' work has a profound effect on the kinds of students the school attracts. According to Science Magazine, a new AI tool known as the Semantic Scholar was developed to help measure the influence of scholarly work. If implemented successfully and deemed trustworthy, this tool could be used by hiring committees. Once completed, the system will measure a work's influence by predicting the work's future impact. The AI attempts to evaluate the "velocity" and "acceleration" of a particular scholarly work by measuring how often it gets cited, providing hiring committees with a good indicator of how influential a particular individual may become in the field. When this practice is used to enlist tenure-track professors, the resulting effect may be that the hiring school attracts some of the most ambitious prospective students. It can be assumed that schools producing influential research will attract higher quality students, creating competition for acceptance. Under the guidance of highly influential scholars, it can also be assumed that high-quality professionals will also be produced by these programs.
Institutions such as ABET, a non-profit, non-governmental organization, specialize in college and university program accreditation in the fields of applied and "natural science, computing, engineering, and engineering technology" at both the undergraduate and graduate level. ABET accreditation is used by students and prospective employers alike to evaluate the quality of different university programs and their effectiveness in preparing students to enter the workforce.
Having been in practice for over 80 years, ABET maintains its credibility through the enforcement of strict criteria developed by professionals from member societies. As a non-profit organization, all of ABET's peer-review processes are voluntary, and thus respected for their critical value to academic programs. Boasting over 2200 experts within the aforementioned disciplines, academia, and government, ABET's seal of approval can help evaluate which schools and programs produce quality computer scientists and software developers.
4. COMPETITION RESULTS
Competitions such as Hackathons, the Amazon Alexa Prize, and Google Code Jam may attract the most passionate and talented future computer scientists and software developers. Because of the nature of these competitions, it is unlikely that those without talent, skill, and passion will place well. While each of these contests has different criteria and may test different skills, it may be possible to deduce an individual's promise in the industry based on their performance. Code Jam, for example, challenges contestants to solve algorithmic puzzles, a task which may reflect the capacity for problem-solving skills, depth of knowledge, and intelligence. However, these are somewhat subjective categories and may be understood differently by different people. Conversely, the Amazon Alexa Prize challenges teams of students from colleges and universities to build a functional AI that can converse with users about various topics and events. The performance of these teams may help employers infer which university and college programs are more effective than others in training computer scientists and software developers.
5. RATE OF IT GROWTH BASED ON LOCATION
According to ACT, the demand for software developers exceeds the available supply. While Silicon Valley is the traditional home of large tech companies like Apple and Google, the United States is experiencing massive growth in unexpected places: Kansas, Indiana, Idaho, and Louisiana. According to Business Wire, employment in Kansa sand Indiana had grown more than 30% in 2016. Recode cited similar statistics, with employment in Kansas up 38% from 2014-2016, and Indiana growing by 32%. Where the IT industry is experiencing growth and creating demand for jobs in software development may influence where the most competent professionals migrate. Although this relationship is likely far more complex and not a 1:1 equation, IT industry growth may be correlated to successful business ventures, which in turn may be correlated to the quality of professionals in the industry.
According to a survey of 300 developers, the speed of the developer, and the developer's personality were considered the most important performance metrics. Percentages are provided below.
The Speed of developer — 15.3%
Developer Personality — 12.6%
Lines of Code — 10.8%
Test Coverage — 10.8%
Code Quality — 6.3%
Technical Debt — 7.2%
Number of Bugs — 4.5%
Closing Tickets — 3.6%
Other — 28.8%
It is difficult to establish definitive metrics that can rank/qualify software development skills. However, several factors stood out as viable methods of assessment. By consolidating credible school ranking systems, paying attention to a school's research influence, consulting accrediting organizations, examining the results of competitions, and noting areas of IT industry growth, it may be possible to assess relative skill level. However, measuring attributes such as depth of knowledge, intelligence, and problem-solving ability may be challenging when applied to a group of software developers. Such measurements may be better suited for individual candidates and assessed on a subjective basis.