Over the last few decades, women and minorities in the US have made pretty remarkable strides in professional careers. But progress has been uneven, especially in the realm of information technology, where diversity in the workplace has just never caught on.

Indeed, women and minority representation seems to be slipping in the computer biz, at least at the epicenter of IT, Silicon Valley. A 2010 article in Mercury News, reported that Hispanics, blacks, and women make up a smaller share of the region’s computing workforce today than they did several years ago. From the article:

[O]f the combined work force of 10 of the valley’s largest companies — including Hewlett-Packard, Intel, Cisco Systems, eBay and AMD — shows that while the collective work force of those 10 companies grew by 16 percent between 1999 and 2005, an already small population of black workers dropped by 16 percent, while the number of Hispanic workers declined by 11 percent. By 2005, only about 2,200 of the 30,000 Silicon Valley-based workers at those 10 companies were black or Hispanic. The share of women at those 10 companies declined to 33 percent in 2005, from 37 percent in 1999.

That trend was even reflected in the less tech-centric roles of upper management. From 2000 to 2005, the number blacks and Hispanics in top management declined 20 percent, while the number of women in these positions declined 2 percent.

There are plenty of people in the community concerned about this disparity, none more so than Richard Tapia, mathematician and professor in the Department of Computational and Applied Mathematics at Rice University. While not teaching and writing math research papers, he has been crusading for underrepresented minorities and women in the computing field.

His signature Tapia Celebration of Diversity in Computing conference brings together speakers across the ethnic/gender spectrum to talk about the work. The latest Tapia get-together, held last week in San Francisco, drew more than 500 people — a record attendance. The model of the conference is to spotlight women and underrepresented minorities successful in the computer arena in order to inspire the next generation of potential techies.

A sampling of speakers at the event last week:

• Blaise Agüera y Arcas, Architect of Bing Mobile and Places at Microsoft, spoke about how computer vision is working its way into consumer devices.
• Deborah Estrin, Professor of Computer Science and Electrical Engineering at UCLA, discussed the use of mobile phones for Participatory Sensing and its social implications.
• Google VP of Engineering Alan Eustace talked about how the industry is organizing the world’s data.
• HP scientist, Patty Lopez explained how imaging and color algorithms are being incorporated into scanners, cameras, and other products.
• John Kubiatowicz, Professor of EECS at the University of California at Berkeley, gave his take on how operating systems should be rearchitected for the manycore era.
• Ayanna Howard, Georgia Tech professor and founder of the Human-Automation Systems Lab, talked about how autonomous robots can be used to help gather data on climate change.
• Former IBM chair Irving Wladawsky-Berger gave a wide-ranging presentation on the changing nature of research and innovation in the 21st Century.

About three-fourths of the Tapia attendees this year were college students, some undoubtedly still trying to decide what to major in and what careers to pursue. Ethnicity-wise, 35 percent of the conference-goers were black and 30 percent Hispanic. Just over half of all the attendees were women.

I spoke with Dave Patterson, professor in Computer Science at UC Berkeley, who chaired the conference this year, to get some of his thoughts on why diversity seems so elusive to this community. Patterson, a computer guru who was a principle driver behind the early RISC (Reduced Instruction Set Computing) architectures, and a pioneer in parallel computing, has admired Richard Tapia for a long time. “When he called and asked me to help I couldn’t refuse,” he told me.

Like many in the community, Patterson finds it difficult to reconcile the great success of the computer industry with its appearance as a white man’s club. “For those of us in the field, we’re amazed at what’s happened with hardware and software,” he explains. “The impact of information technology on society is something we’re proud of. But it’s a little embarrassing about the lack of women and minorities in the field.”

The reasons for this lack of diversity are multi-faceted. The Mercury News piece pointed to educational disparities and the lack of role models in the industry. It also suggested that the influx of Silicon Valley foreign-born workers on H-1B visas are displacing home-grown minorities disproportionally. Ironically, the importation of Asians and other international workers into the tech arena could be making the industry less diverse overall.

For women, Patterson thinks computing has been “tarred with the nerd image,” which tends to drive females away from the profession. Prior to the establishment of that cultural stereotyping, he says there was a higher fraction of women researchers in the field. With recent increases in computer science enrollment and the widespread use social networking, he believes that trend could reverse itself.

The question arises, though, why should we even care? The computer industry seems to be humming along nicely without reflecting the ethnic and gender demographics of the US. Is the interest in diversity anything more than liberal guilt?

According to Patterson, diversity is not about political correctness. Part of this, he says, is about making the most of the talent at hand. On the software end, Patterson says only a fraction of the population — maybe 5 or 10 percent — can master algorithmic thinking. So just by the force of numbers, the greater the cross-section of the population you’re able to attract, the more tech talent you’re going to be able to tap.

Another aspect to this, he says, is how computing science gets funded by the government. If information technology R&D is perceived as pork barrel spending for white men’s hobbies, it’s less likely to attract broad-based political support. Having a diverse set of constituencies in the workforce will ensure it gets needed attention from the government.

But perhaps at the most basic level is the interplay between technology and its inventors. A core driver of science, and especially engineering, says Patterson, is creativity, which can loosely be defined as putting together information in new ways for some useful purpose. In many cases, people’s life experience drives how they connect that information, and individuals with dissimilar backgrounds tend to bring those unique views into the creative process.

The US entertainment industry is probably the best example of how an extremely diverse workforce turned a relatively small business into a $100-plus billion industry over the last half century. (Revenue from the motion picture industry alone is now worth over $50 billion.) It would be hard to imagine Hollywood and the US music business if it resembled the white male-dominated demographics of today’s computing industry.

Even though the historical track record of diversity in IT has been disappointing, there’s at least some reason to hope. Fifty years ago doctors were almost exclusively men; today more than half of the medical students are women. High achievers in minority populations have made inroads into law, medicine, and more recently, top government positions.

In computing, even the relatively uniform workforce we have today has managed to deliver search engines, smartphones, and generally ubiquitous access to computing. But Patterson thinks this is just the beginning. He says in the future when people look back at what we have currently — computers crashing, frustratingly-slow internet speeds, and 8-lb laptops — these are going to be seen as the bad old days.

“We’ve just scratched the surface of what computing is going to do,” he says. “I really believe that the next few decades are going to be amazing.”