Technology to allow reviewers to anonymously view URLs

Reinhard Schneider and colleagues have proposed the following approach to protecting grant reviewers' anonymity. Comments from the ISCB community would be welcome; please post your comments by Friday, May 12, 2006!

Draft proposal: 0.1 (RS, April 10^th, 2006)

How ISCB can help to allow the use of URL’s related to bioinformatics grant proposals

Background: See the proposed ISCB policy statement on URLs in grant proposals.

Solution: ISCB could function as the “anonymizer” for the reviewers.

Possible Technical implementation:

1. Use of an “anonymizer” service on the web
2. Implementation of a proxy server under the supervision of ISCB
3. Implementation of a terminal server under the supervision of the ISCB

1) Using “anonymizer” services

The use of an anonymizer service is quite simple and can be done without much user intervention. The basic principle of these services is the use of one single static address (proxy or a net of proxies), which is shared by many users. Examples for these services are:

http://anon.inf.tu-dresden.de/index_en.html

http://www.the-cloak.com/anonymous-surfing-home.html

http://www.shadowsurf.com/

http://surfshield.net/

http://proxify.com/

http://www.guardster.com/?Services-Free_Web_Proxy

http://anonymouse.org/anonwww.html

2) Proxy-server setup by the ISCB

As an alternative to the above mentioned anonymizer services, one can use a proxy server that would be owned by the ISCB. Proxy servers are similar to the anonymizer, i.e. web pages are retrieved by the proxy server rather than by the person actually browsing the Web. By running such a proxy-server the ISCB could guarantee that no information is collected or otherwise abused. But there are important differences: proxy servers don't help with cookies, hostile applets or code (see below: possible problems).

3) Setup of a terminal server on an ISCB server

This solution would require the setup of a server running Microsoft “Windows Server 2003 Terminal Services”. The reviewer would login into this terminal server and would start a web-browser from this machine. By doing this all the traffic to the destination web-server will originate from the ISCB terminal server and as such protect the anonymity of the reviewer 100%.

Client software is freely available for Windows systems, Apple computer and Linux systems.

The funding for such a solution would need to cover the basic hardware for the server, the license for the operating system and the terminal server licenses (per user) as well as the maintenance costs for running this service.

Additional advantage: The URL’s can be password protected and can contain even unpublished data. The user would then provide the site login information in the grant proposal. By using the terminal server solution the reviewers could login anonymously and the proposal submitter would only know that her/his site is being reviewed.

Possible problems with solution 1 and 2:

There are ways of tracking down a web-surfer behind an anonymous proxy server. The following technologies can be used to break the anonymity of a user: Cookies, JavaScript, VBScript, Java, ActiveX and plug-ins. There are methods to increase the security of the user's own IP-number, but either they restrict the functionality (cookies, JavaScript…), are not 100% secure or require substantial knowledge and configuration of network setups (see http://www.freeproxy.ru/en/free_proxy/faq/anti_proxy.htm).

Due to these restrictions, option number three (terminal server) seems to be the most appropriate solution to secure the anonymity of reviewers and simplicity of use. The costs should be in a reasonable range and could possibly be covered by a small grant for running this service.

Possible workflow of the implementation

• User registers at the ISCB web-site (Name, e-mail, title of proposal)
• Confirmation mail is sent to the user with a link for activating the account (with or without administrator intervention?)
• ISCB server sends username and password for the terminal server to the user
• User puts username and password into her/his grant application
• Reviewer logs into ISCB terminal server

Validity of account: 6 months (maximum?; automatic deletion after max days?)

Should ISCB support the inclusion of URLs in grant proposals?

ISCB requests member feedback on releasing the following policy statement. Please post your feedback by Friday, May 12, 2006. To post feedback, click on "comments" at the bottom of this post. You can leave a comment as "Anonymous" without needing to set up an account.

The Statement

1. ISCB recommends that bioinformatics funding agencies encourage reviewers to follow those URLs in bioinformatics grant proposals that provide information important to the grant review. Funding agencies should not forbid reviewers from following URLs in grant proposals.

2. ISCB recommends that bioinformatics funding agencies establish proxy Web servers for use by reviewers to facilitate anonymous access to applicant web sites.

Background

This statement is motivated in part by the current grants policy of the US National Institutes of Health, which states "URLs may not be used to provide information necessary to the review because reviewers are under no obligation to view the Internet sites. Moreover, reviewers are cautioned that they should not directly access an Internet site as it could compromise their anonymity." (See URL http://grants2.nih.gov/grants/funding/phs398/phs398.pdf) That is, examination of web logs by an applicant might reveal the identity of a reviewer.

In addition, several NIH review panels now take the more restrictive position of forbidding reviewers to follow URLs.

Submission of software via a CD-ROM in conjunction with a grant would be one way to avoid using the Internet to demonstrate software or database capabilities to reviewers, however, the NIH also does not allow CD-ROMs to be submitted with a grant proposal.

Funding agencies are also concerned that URLs could be used to circumvent grant page restrictions if an applicant places additional information on their web site.

Another concern behind this policy is that there is no permanent record of the contents of any URL, and that if a grant PI later challenges a review by saying, for example, "This review is not competent because the reviewer's objection is clearly addressed by this information on my web site" there is no way for NIH to later validate what was on the PI's web site at an earlier time.

Rationale

There are several related issues here. One issue pertains to third-party Web sites ("Third-party URLs") such as articles in online journals. Another relates to Web sites maintained by the applicant PI or their associates ("PI URLs");

1. Third-party URLs. There is simply no question that reviewers should be allowed to access third-party sites. Grant applications already contain extensive references to third-party information, namely scientific publications. As the model of scientific publishing evolves, some publications are available only through the web, and others are most efficiently accessed via the web. It is a waste of the reviewers’ time to forbid them from accessing such sites. With third-party sites there are no issues of anonymity nor of circumventing page limits.

2. PI URLs. The deliverables of many bioinformatics projects are databases or software packages that are resident on the web, and can most efficiently be accessed by reviewers through the web. Direct review of database and software packages via the web is both extremely informative, and extremely time efficient for the reviewer. To prevent the reviewer from interacting directly with a database or software package prevents the reviewer from having first-hand knowledge of the database or software that is extremely valuable, and introduces a serious risk that their knowledge is inadequate to perform an informed review. In addition, for large projects, part of their funding typically covers a service component that is also important for reviewers to assess. However, reviewers should be cautioned against basing a negative judgment on a single problematic session that could be caused by network outages that are beyond the control of the project. And small projects in particular cannot be expected to provide perfect 24x7 service.

It should be left to the discretion of the reviewer which URLs they consider important to the grant review. Although bioinformatics database and software-related applications are likely candidate projects where consideration of Web information will be important, the reviewer will be the best judge of when to follow a URL.

The anonymity concern of the NIH can be solved if reviewers use proxy servers, which shield the identity of the person accessing a web site. We recommend that the NIH fund the creation of proxy servers for use by grant reviewers; ISCB would be willing to host and operate such proxy servers.

NIH's page limitation concern can be solved if reviewers are not REQUIRED to read anything accessible through URLs.

The NIH concern of the lack of a permanent record for URLs is valid, but is outweighed by the other factors. Note also that reviewers are already influenced by many other subjective factors that are not part of the grant application.