“Science is really ripe for disruption. A lot of the practices are still very much rooted in their analog beginnings.” That is how Kaitlin Thaney, Director of the Mozilla Science Lab—a new open science initiative focused on innovation, best practice, and skills training for research—began her plenary talk at XSEDE13 in San Diego last month.

Thaney believes that the web has fundamentally transformed how we interrogate, how we interact with content, how we discover information, our work environments, and our agility. “We at Mozilla are focusing on how we can help researchers use the power of the open web to change science’s future,” she said.

The Mozilla Science Lab is funded by the Alfred P. Sloan Foundation, a leading funder in digital scholarship and information technology with projects focused on data sharing, new forms of metrics for scholarly research, and other technologies on the library side. Thaney sees this as a larger play to see how all of them can work together toward a broader means of coordinating these various efforts to provide infrastructure and also leadership in the open science space while pushing this practice into the mainstream. 

Thaney makes the case that our current systems are still designed to create friction despite their original intentions. “Modern discoveries are still locked up in various silos and not as interlinked as they could be,” she said. “Even looking at the way we reward scientific contribution, it is still very much reliant on a paper-based system. We are starting to see what the behavioral effects will be from this sort of reward system, and what we are seeing is more error creep into the way research is done.”

Citing Elizabeth Iorns, Co-founder of Science Exchange, Thaney said that “up to 70 percent of research from academic labs cannot be reproduced, representing an enormous waste of money and effort.” She pointed to this statistic as reflective of a larger problem that the research community should address by developing a better way to reflect and reward research contributions and practices. “It is really keeping us from that whole notion of building on additional knowledge, or prior knowledge, or standing on the shoulders of giants, and really looking at how we can start to minimize not only the amount of error but also the amount of duplication,” said Thaney.

Moving Forward

Luckily we have the Internet, which Thaney views as one of the most important advances since the printing press; however, inventing and putting into practice are two very different things. “Even though the web was created by scientists, I think we can all probably largely agree that the web has not yet transformed science in the same way that it has transformed the way we do business or the way we look at education,” said Thaney.

Step one is gateways. It is no coincidence that there are commonalities between her project at Mozilla and XSEDE’s Science Gateways, the theme of XSEDE13. Both look at preparing web-based resources so science and engineering researchers can have better access to the data, the information, the software, and the code through online community spaces. Both hope making these tools more accessible will increase the power behind the computation. 

Thaney said that in her previous work at the technology company Digital Science they were looking at application-based technologies that would help bring researchers a little bit closer to a world of reproducibility and better practice via the web. They found that the most successful work was with tools that were mapped into a ubiquitous layer, where users didn’t even realize they were interacting with a web-based tool anymore.

Closing the Gap

Starting with the physical aspects of research—the lab mice, the DNA, the cell lines—there are ways to attach digital imprints. “Whether it’s a tracking number or some other means of attaching identity to it, the community can start to integrate that into the systems to make it a little bit easier for the next researcher who comes along to be able to reproduce and start from that without having to reverse engineer,” explained Thaney.

Research isn’t cheap, she reminded the conference attendees. “For example, getting one Huntington’s protein, that can cost you $400, only to get it to your lab and realize there is a massive efficacy issue around using various proteins and antibodies. And now you need to start from scratch again.”

“The Mozilla Science Lab is looking at what we could possibly learn from the open source community, both in terms of culture and technology,” said Thaney. That community believes that sharing risk and also reward by mixing internal and also external involvement and development will often times get you much further toward your goal. “Looking at the broader idea of having innovation not just be putting your own ideas into a funnel and then pushing it out to the greater world, but having it be much more of a porous process,” Thaney said.

Thaney’s and Mozilla’s mission also includes increasing digital literacy. Relying on ad-hoc, self-taught practices to close the gap between what you are expected to know and what you have the resources to learn doesn’t scale, according to Thaney. “The amount of times researchers have come to me and said ‘I wanted to do this post-doc but my PI told me I need to know Python along with a range of things I have never experienced before, and so I just bought a book.’ That is not best practice.”

One way the Mozilla Science Lab is increasing digital literacy is by working with a program called Software Carpentry. The program teaches basic computational competency and digital skills to researchers to help provide the springboard needed for 21^st century science. 

Measuring Up and Fighting Fears

A traditional metric for evaluating the impact of scholarly research has been the number of times an article is cited; however, with the growing use of alternative metrics, or “altmetrics,” citation is no longer the only way to measure success. How many times an article has been bookmarked, tweeted, shared, blogged about, or cited in Wikipedia can now matter just as much. “The data there is actually quite staggering in some context. Where a paper it is associated with might be cited ten times, the data might be reused 1,500. That is really powerful to be able to go to your funder or be able to go to your research administrator and say this is a useful bit of my research,” explained Thaney.

Thaney understands the fear that goes along with the practice of open science:  “Relinquishing information and opening it up for anyone to make use of it can be slightly terrifying, especially when there is a lot of time that goes into collecting that information, but using the power of the network, we can really have the opportunity to open it up to as many eyes as possible to see the broader use and impact of what that research may be.”

XSEDE13 participant Anthony Frachioni, who will enter into the physics PhD program at Binghamton University, New York, in the fall, said that while he is excited at the prospect of an open science and research world, at the end of the day he still lives in constant fear he will get scooped. “What do you do about credit and courtesy?” he asked Thaney.

“A lot of it comes down to being human. The technology there has shown we have the ability to start to making this information available, but when it comes to the reward mechanisms, we need to start providing some sort of assurance, because otherwise, we can only go so far,” she responded. “I can say that researchers getting scooped simply because they made some information available early happens a lot less than you might think, but I understand when it comes to job security, we need something far more concrete.”

Thaney said that concern is why Mozilla, through a separate program, is looking into ways to provide badges, or some other sort of accolade within the researcher’s university or institution, to show proof of intellectual property in a particular area. The hope is that having something to show for sharing information will provide comfort for the researcher while cautioning others to think twice about scooping it. “It is not an easy problem to solve, but we are working on it,” concluded Thaney.    

Even without concrete solutions, Frachioni remains enthusiastic about the possibilities of open science. “This is all completely unprecedented, and that isn’t going to give us any reassurance at the end of the day,” he said, “but I think that it’s our responsibility to enter into this knowing that and knowing that we should do it anyway.” 

The annual XSEDE conference, organized by the National Science Foundation’s Extreme Science and Engineering Discovery Environment with the support of corporate and non-profit sponsors, brings together the extended community of individuals interested in advancing research cyberinfrastructure and integrated digital services for the benefit of science and society. XSEDE13 was held July 22-25 in San Diego; XSEDE14 will be held July 13-18 in Atlanta. For more information, visit https://conferences.xsede.org/xsede14.