ISCB Member Feedback Sought on Revised Software Sharing Policy Statement

International Society for Computational Biology Revised Software Sharing Policy Statement

Draft approved by the ISCB board of Directors on February 14, 2008

Open for comment from the ISCB membership and bioinformatics community

Comment period closes April 15, 2008

I. Introduction
Bioinformatics software availability is extremely important to the field of bioinformatics. The International Society for Computational Biology (ISCB) is committed to the advancement of the understanding of living systems through computation. In support of that mission, we believe that research results should be shared with the scientific community so that they can be reproduced and built upon. Scientific research may include the development of software and algorithms. Therefore, ISCB is disseminating this statement to make recommendations on software availability policies for funders of bioinformatics research, for scientific journals that publish bioinformatics research, for bioinformatics researchers, and for their employers.

This statement has been revised from the original 2002 statement, incorporating feedback from the ISCB membership.

II. Recommendations

1. Publishers, granting organizations, employers and researchers have a responsibility to uphold the core principle of sharing methods and results. If a researcher's software is necessary to understand, reproduce and build on scientific results, then the software should be made available. This principle is imperative for peer-reviewed scientific publications, recommended policy for granting agencies, and encouraged practice wherever individuals and organizations are committed to advancing science. ISCB supports the recommendations of the National Academies of Sciences report, "Sharing Publication-Related Data and Materials: Responsibilities of Authorship in the Life Sciences."
2. Grantors and publishers should require statements of software availability in grant proposals and research reports. These statements should clearly describe how to obtain the software, and terms of use. The statements should be specific about cost, source code availability, redistribution rights (including for derived works), user support, and any discrimination among user types.
3. No single licensing or distribution model is appropriate for all research projects, and therefore should not be mandated by either publishers or grantors.

III. Implementation when software sharing is warranted

1. In most cases, it is preferable to make source code available. We recommend executable versions of the software should be made available for research use to individuals at academic institutions.
2. Open source licenses are one effective way to share software. For more information, see the definition of open source, and example licenses, at www.opensource.org.

For more information, see the previous posting which includes information about the original 2002 policy statement, member discussion, and useful links.
We invite the computational biology community provide comments on this blog, or to send email to policy@iscb.org.

[Notes added 3/14/2008:
(1) An old version of the second sentence in section III-1 was erroneously included when this was first posted on 3/13/2008; it has been fixed now.
(2) The Board is releasing this proposed language for discussion by the ISCB membership; it will not become final until after the 1-month open discussion period, whereupon the Board may revise it further in response to ISCB member feedback.]

ISCB Members and US Visas

The International Society for Computational Biology (ISCB) is concerned about the ability of non-US scientists to participate in scientific meetings and research in the US. We believe this exchange of ideas benefits the scientific enterprise as well as the countries and individuals involved.

We have heard reports of ISCB members who are not non-US citizens having trouble coming to the US to work or study, and also having trouble leaving to visit family or attend scientific meetings (because it is hard to get back into the US). While the situation has improved over the last few years, there remain problems.

We have an opportunity, through our relationship with FASEB, to contribute to testimony in front of the House Science Committee in February about scientists' problems with visas. Your feedback must be in our hands by 25th January, 2008, in order for it to be useful in the Committee hearing. However, we will use any information we receive after that date for further advocacy, so please do not hesitate to send it in.

Can you help us by telling us of problems you or your colleagues have experienced with entry into the US? Please send email to policy@iscb.org (preferred, so that we can gather additional information from you if necessary), or post your experiences here (especially if you'd like to remain anonymous), or both.

We are particularly interested in answers to the following questions:

1) Are you experiencing delays getting visas or outright rejections of your applications?
2) Are you seeing this problem from particular countries?
3) Specifically, what problems are people experiencing (i.e., difficulty getting consular appointments; delays in application processing; denial of visas; problems with US-VISIT system)?
4) For each problem, is it due to visa applicants not following the existing guidelines and restrictions (such as not applying far enough ahead of time, failing to schedule a consular interview, providing incomplete applications, country-specific single entry reciprocity agreements), or is the problem a failure of the US immigration system to follow its own policies?
5) What change, if any, do you feel we ought to advocate?

ISCB urges its members who have had problems to contact the National Academies International Visitors Office (http://www7.nationalacademies.org/visas/index.html), and fill out the questionnaire. This office has been very helpful at interceding behind the scenes on behalf of scientists with the Dept. of State and Homeland Security. And the data they generate is very useful for making the case for improved visa processing.

Thank you,
The ISCB Public Affairs Committee

Data and Software Sharing Policy

This message is a request for feedback from the computational biology community.

In 2002, ISCB developed a policy statement on bioinformatics software availability, which defined 5 levels of software availability and made the following recommendations:

1. Given the variety of meanings of "open source", that people define what they mean when they use the term.

2. That government funding agencies encourage grant proposals to specify the availability of software using at least the ISCB-defined levels.

3. That government funding agencies not mandate that all software created with grant money be available via an open-source license.

4. That government funding agencies require that all software created with grant money be available at a minimum in binary form, and free to non-commercial users.

This policy was developed without sufficient input from ISCB members, and the Public Affairs committee is revisiting this topic. We will distribute relevant educational materials (see links below) and opinion pieces, hold a meeting at ISMB/ECCB 2007, gather input on this blog and via email, and otherwise gather feedback from the community. We hope to develop a revised policy statement, or guidelines, that will be useful to the community as well as to government funding agencies and scientific journals.

Our questions to you (please answer in the comments section or direction to public-affairs@iscb.org):

1. Is there a problem?

• Is there a need to define software availability clearly?
  
• Should we expand the scope from government funding agencies to publications? Or beyond? Should we expand the scope to include data sharing?
• What should government agencies and journals require in terms of software availability? Should ISCB make a recommendation?
• Should authors and grant-writers be required to clearly define the availability of their software?
  
• Is there a problem currently with published articles, in that it is difficult to reproduce the results due to lack of access to data or software? Have you had personal experience with this?
  
• Does it make sense to allow researchers at companies to be charged a fee for software but require that it be provided to academics at no charge?
  
• If you have terabytes of data, how does that affect your ability to share it?
• Are there privacy concerns with sharing of human genomic data?
  
• What is really needed to allow results to be verified and built upon?

2. What should be done to address the problem?

• Do you agree or disagree with the 2002 ISCB policy statement, and why?
• When you publish a paper or develop software for a grant, how do you make your software and data available?
• What should ISCB do in addition to, or instead of, releasing a policy statement? (Has the previous policy statement had any effect?)
• What would YOU be willing to do to help ISCB address this issue?

Please respond in the comments section of this post, or via email to policy@iscb.org, or by attending the meeting on this topic at ISMB/ECCB 2007.

Reference material:

• NAS report on publication-related data and materials sharing (can read online for free)
• Science news article on the NAS Cech report
• PLoS Computational Biology data and software sharing policy
  
• Communication From The Commission To The European Parliament, The Council And The European Economic And Social Committee on scientific information in the digital age: access, dissemination and preservation (Brussels, 14.2.2007, COM(2007) 56 final)
• Statement of Policy on Research Tools and the Policy on Sharing Publication-Related Materials, Data and Software of the HHMI (Howard Hughes Medical Institute) consistent with the guidelines by the NIH (National Institutes of Health).
• NCI/CaBIG open source software licensing guidelines
• NIH statement on sharing research data
• NHGRI Data release and access principles and policy
• NHGRI Rapid data release policy
• Update on NHGRI Rapid data release policy for large-scale community resource projects
• ENCODE project data release project
• NIAID Microbial sequencing data release policy
• NHLBI data release policy
• NIH intramural sequencing center
• NSF policy on dissemination and sharing of research results
• Malaria genome project data release policy
• Wellcome Trust data sharing report
• Lee Hood letter in SCIENCE on IP
• Nature's data and materials availability policy
• Nature's copyright statement
• Science's data and materials availability policy
• PNAS' data and materials availability policy
• Bioinformatics' data and materials availability policy

Technology to allow reviewers to anonymously view URLs

Reinhard Schneider and colleagues have proposed the following approach to protecting grant reviewers' anonymity. Comments from the ISCB community would be welcome; please post your comments by Friday, May 12, 2006!

Draft proposal: 0.1 (RS, April 10^th, 2006)

How ISCB can help to allow the use of URL’s related to bioinformatics grant proposals

Background: See the proposed ISCB policy statement on URLs in grant proposals.

Solution: ISCB could function as the “anonymizer” for the reviewers.

Possible Technical implementation:

1. Use of an “anonymizer” service on the web
2. Implementation of a proxy server under the supervision of ISCB
3. Implementation of a terminal server under the supervision of the ISCB

1) Using “anonymizer” services

The use of an anonymizer service is quite simple and can be done without much user intervention. The basic principle of these services is the use of one single static address (proxy or a net of proxies), which is shared by many users. Examples for these services are:

http://anon.inf.tu-dresden.de/index_en.html

http://www.the-cloak.com/anonymous-surfing-home.html

http://www.shadowsurf.com/

http://surfshield.net/

http://proxify.com/

http://www.guardster.com/?Services-Free_Web_Proxy

http://anonymouse.org/anonwww.html

2) Proxy-server setup by the ISCB

As an alternative to the above mentioned anonymizer services, one can use a proxy server that would be owned by the ISCB. Proxy servers are similar to the anonymizer, i.e. web pages are retrieved by the proxy server rather than by the person actually browsing the Web. By running such a proxy-server the ISCB could guarantee that no information is collected or otherwise abused. But there are important differences: proxy servers don't help with cookies, hostile applets or code (see below: possible problems).

3) Setup of a terminal server on an ISCB server

This solution would require the setup of a server running Microsoft “Windows Server 2003 Terminal Services”. The reviewer would login into this terminal server and would start a web-browser from this machine. By doing this all the traffic to the destination web-server will originate from the ISCB terminal server and as such protect the anonymity of the reviewer 100%.

Client software is freely available for Windows systems, Apple computer and Linux systems.

The funding for such a solution would need to cover the basic hardware for the server, the license for the operating system and the terminal server licenses (per user) as well as the maintenance costs for running this service.

Additional advantage: The URL’s can be password protected and can contain even unpublished data. The user would then provide the site login information in the grant proposal. By using the terminal server solution the reviewers could login anonymously and the proposal submitter would only know that her/his site is being reviewed.

Possible problems with solution 1 and 2:

There are ways of tracking down a web-surfer behind an anonymous proxy server. The following technologies can be used to break the anonymity of a user: Cookies, JavaScript, VBScript, Java, ActiveX and plug-ins. There are methods to increase the security of the user's own IP-number, but either they restrict the functionality (cookies, JavaScript…), are not 100% secure or require substantial knowledge and configuration of network setups (see http://www.freeproxy.ru/en/free_proxy/faq/anti_proxy.htm).

Due to these restrictions, option number three (terminal server) seems to be the most appropriate solution to secure the anonymity of reviewers and simplicity of use. The costs should be in a reasonable range and could possibly be covered by a small grant for running this service.

Possible workflow of the implementation

• User registers at the ISCB web-site (Name, e-mail, title of proposal)
• Confirmation mail is sent to the user with a link for activating the account (with or without administrator intervention?)
• ISCB server sends username and password for the terminal server to the user
• User puts username and password into her/his grant application
• Reviewer logs into ISCB terminal server

Validity of account: 6 months (maximum?; automatic deletion after max days?)

Should ISCB support the inclusion of URLs in grant proposals?

ISCB requests member feedback on releasing the following policy statement. Please post your feedback by Friday, May 12, 2006. To post feedback, click on "comments" at the bottom of this post. You can leave a comment as "Anonymous" without needing to set up an account.

The Statement

1. ISCB recommends that bioinformatics funding agencies encourage reviewers to follow those URLs in bioinformatics grant proposals that provide information important to the grant review. Funding agencies should not forbid reviewers from following URLs in grant proposals.

2. ISCB recommends that bioinformatics funding agencies establish proxy Web servers for use by reviewers to facilitate anonymous access to applicant web sites.

Background

This statement is motivated in part by the current grants policy of the US National Institutes of Health, which states "URLs may not be used to provide information necessary to the review because reviewers are under no obligation to view the Internet sites. Moreover, reviewers are cautioned that they should not directly access an Internet site as it could compromise their anonymity." (See URL http://grants2.nih.gov/grants/funding/phs398/phs398.pdf) That is, examination of web logs by an applicant might reveal the identity of a reviewer.

In addition, several NIH review panels now take the more restrictive position of forbidding reviewers to follow URLs.

Submission of software via a CD-ROM in conjunction with a grant would be one way to avoid using the Internet to demonstrate software or database capabilities to reviewers, however, the NIH also does not allow CD-ROMs to be submitted with a grant proposal.

Funding agencies are also concerned that URLs could be used to circumvent grant page restrictions if an applicant places additional information on their web site.

Another concern behind this policy is that there is no permanent record of the contents of any URL, and that if a grant PI later challenges a review by saying, for example, "This review is not competent because the reviewer's objection is clearly addressed by this information on my web site" there is no way for NIH to later validate what was on the PI's web site at an earlier time.

Rationale

There are several related issues here. One issue pertains to third-party Web sites ("Third-party URLs") such as articles in online journals. Another relates to Web sites maintained by the applicant PI or their associates ("PI URLs");

1. Third-party URLs. There is simply no question that reviewers should be allowed to access third-party sites. Grant applications already contain extensive references to third-party information, namely scientific publications. As the model of scientific publishing evolves, some publications are available only through the web, and others are most efficiently accessed via the web. It is a waste of the reviewers’ time to forbid them from accessing such sites. With third-party sites there are no issues of anonymity nor of circumventing page limits.

2. PI URLs. The deliverables of many bioinformatics projects are databases or software packages that are resident on the web, and can most efficiently be accessed by reviewers through the web. Direct review of database and software packages via the web is both extremely informative, and extremely time efficient for the reviewer. To prevent the reviewer from interacting directly with a database or software package prevents the reviewer from having first-hand knowledge of the database or software that is extremely valuable, and introduces a serious risk that their knowledge is inadequate to perform an informed review. In addition, for large projects, part of their funding typically covers a service component that is also important for reviewers to assess. However, reviewers should be cautioned against basing a negative judgment on a single problematic session that could be caused by network outages that are beyond the control of the project. And small projects in particular cannot be expected to provide perfect 24x7 service.

It should be left to the discretion of the reviewer which URLs they consider important to the grant review. Although bioinformatics database and software-related applications are likely candidate projects where consideration of Web information will be important, the reviewer will be the best judge of when to follow a URL.

The anonymity concern of the NIH can be solved if reviewers use proxy servers, which shield the identity of the person accessing a web site. We recommend that the NIH fund the creation of proxy servers for use by grant reviewers; ISCB would be willing to host and operate such proxy servers.

NIH's page limitation concern can be solved if reviewers are not REQUIRED to read anything accessible through URLs.

The NIH concern of the lack of a permanent record for URLs is valid, but is outweighed by the other factors. Note also that reviewers are already influenced by many other subjective factors that are not part of the grant application.