D -- WEB-BASED EXAMINATION TOOL

Notice Date

February 27, 2001

Contracting Office

U.S. Postal Service, Headquarters Purchasing, Room 4541, 475 L'Enfant Plaza, SW., Washington, DC 20260-6230

ZIP Code

20260-6230

Response Due

March 2, 2001

Point of Contact

jschank@email.usps.gov NO PHONE CALLS

E-Mail Address

contact Jim Schank, Purchasing Specialist (jschank@email.usps.gov)

Description

2/27/2001 Answers to Questions on CBD Posting 50D715;Web-based examination tool Question 1: After reviewing the solicitation for services to provide a "Web-Based Tool to Conduct Postal Examinations", we were unclear as to whether this was for the purchase of a tool that might have already been selected or for services to design and develop such a tool. Secondly if such a tool does exist, then is the solicitation for purposes of having this tool customized only. If the tool has been selected, can you identify the manufacturer and product name. Answer 1: We are looking to purchase commercial off-the-shelf software that would require a minimum of customization. We are not interested in developing a custom tool. No tool has been selected. Question 2: Under mandatory requirements, question 11 states: The application must be able to lock out unapproved applications. Could you please provide more detail about this question? What types of applications would you be required to lock out? Answer 2: The tool must not allow a test taker to screen print, print questions or answers, save, or copy any part of what he/she sees. Therefore, the web application must lock out any other application that might be on the desktop. Question 3: Describes the desired USPS testing environment. Are the PCs that are used to take the exam in a monitored exam room (i.e., controlled environment)-OR-can the various exams be downloaded from the USPS website and taken from a PC at any location? We ask this question to clarify the mandatory requirement: The administration of this exam should be controlled from a central site, or from a local exam administrator's workstation, with the ability to change as the need arises and would have to integrate into an Oracle database. Answer 3: Postal exams are monitored in a controlled environment; however, a Web tool that we are seeking must enable its exams to be downloaded by authorized examination administrators at remote locations who are connected to the postal intranet. Question 4: Please describe the tester authorization process. You state that the application must lock out unapproved applications. a) Who are the test takers? Answer 4: They are applicants for entrance (new hires), in-service (current employees), and in craft (certain groups of employees within a craft/union designation) examination requests. Question 5: Are the test takers seeking employment with he USPS? Answer 5: Yes. Question 6: Are potential test-takers current USPS employees who want to advance their position within the USPS? Answer 6: Yes. Question 7: How are the test takers approved? Answer 7: Applicants are approved by USPS certified examination administrators. Question 8: Do they complete an application on-line? Answer 8: Some applicants, yes. Some also use an IVR to register for examinations. All applicants are approved by the examination administrator who is overseeing the process. Question 9: Will approval criteria be available so that this application can automate the approval process and immediately know which exams the applicant is eligible to take? Answer 9: Yes, eventually. However, we are not looking for this functionality right away. Question 10: If a potential applicant is not approved, can s/he try to get approval by resubmitting another application? Answer 10: No Question 11: What are the USPS's key concerns for the web-based examination tool (in order of importance)? Is this security, reliability, flexibility? Please clarify. Answer 11: In order of importance they are: 1) reliability -- when we use this tool, we want to be reasonably assured it will work; 2) security -- we must meet requirements to protect sensitive data and protect the integrity of the testing process; and 3) flexibility -- although we are looking for the silver bullet, we recognize that one tool may not fit all our requirements. We are looking for the tool(s) that will meet the most of our requirements. Question 12: Please describe the current Postal infrastructure. What is the OS of the server machine? Answer 12: The application server and any supporting software must conform with postal infrastructure requirements as defined in USPS Infrastructure Tool Kit on http://itk.usps.gov/ in Publication 147, Web Standards Guide. We currently use NT running Oracle Application Server (OAS); however, we will consider other alternatives. We will provide access to an Oracle database to store exams, responses, and results. Question 13: How are these tests arranged within the web-based test exam application? Are we correct to assume that there are a series of tests that can be administered only to pre-approved applicants of various categories of jobs? Answer 13: Yes. Question 14: Please describe how these tests are arranged by the USPS administrator. Answer 14: Examinations are arranged by a field HR unit when it is determined that there will be a need to hire within a specific amount of time. People apply to take specific exams; if they pass, they are placed on an office register by job category according to score. Question 15: Does the following requirement mean that this application must provide the capability to generate different tests from a large list of valid questions/answers? USPS Requirement: The application must be able to import questions and answers from a text document, allow the modification of existing examinations and ensure the ability to add future examinations. Answer 15: Yes Question 16: I have a question regarding the accessibility standard component. The link provided takes me to accessibility standards for buildings, water fountains, etc., but I could not seem to locate software standards. Is this a general requirement or are there specific software standards for disabilities that I am missing? Answer 16: In 1998 Congress amended the Rehabilitation Act and strengthened provisions covering access to information in the Federal sector. As amended, section 508 of the Rehabilitation Act requires access to the Federal government's electronic and information technology. It applies to all Federal agencies when they develop, procure, maintain, or use such technology. Check the web site for the Access Board, who is responsible for developing access standards -- it is www.access-board.gov or section508nprm@access-board.gov. Question 17: Given that you expect providers to demonstrate their products on or about the week of March 12, are we to assume that you are only considering existing products, or will you be considering custom application development products? Answer 17: We will only consider existing products that would require a minimum of customization. Question 18: If an existing product satisfies some (but not all) of the mandatory requirements, can that product be considered with the understanding that the product will be modified during the period of performance? Answer 18: Yes. Question 19: Can you provide any additional information on the level of effort desired? For example, do you desire support to accomplish the following tasks: A.) System installation. Answer: Yes; B.) Migration of existing testing data. Answer: No; C.) Training for operations of the system. Answer: Yes D.) Testing administration; Answer: No Question 20: What is the nature of the questions that the software will present to the test taker? Are they all multiple choice, or are there other types of questions (e.g. open ended text, matching, re-ordering). Answer 20: Some tests will be structured with multiple choice answers only. Some tests will require open-ended, free-form text responses, as well as multiple choice answer sections. Some tests will also include matching and/or re-ordering, as well as multiple choice answer sections and/or multiple choice and matching for responses. Question 21: The document makes reference to the hardware to be used by the test takers. Are there any requirements regarding the application server that will host the system? For example, if our solution requires an NT Server with Internet Information Server and a Java Servlet engine, is that acceptable? Will the USPS be able to provide access to a database management system such as Oracle that can be used to store tests, responses, and results? Answer 21: The application server and any supporting software must conform with postal infrastructure requirements as defined in USPS Infrastructure Tool Kit on http://itk.usps.gov/ in Publication 147, Web Standards Guide. We currently use NT running Oracle Application Server (OAS); however, we will consider other alternatives. We will provide access to an Oracle database to store exams, responses, and results. Question 22: Where can we get additional information about the Postal Examination Processing System (PEPS) that this web based system will feed? What types of interfaces are required? Answer 22: The PEPS system is flexible with regard to feeding information into it. We can use a string of data (delimited) or we can append to an Oracle table via the application. Question 23: Mandatory requirement 8 says "The application must allow an applicant to go back to his/her place in the examination with the remaining time left following Internet failure or hardware problems." When the test taker is in a remote location, the determination of an Internet failure or hardware problem is problematic, because no system can distinguish between a true failure, and one intentionally introduced by the applicant (e.g. by terminating an internet connection, or powering off a machine). Further, general delays caused by Internet congestion can have an impact on the actual time required to take the test. These are issues inherent with on-line testing, and can only be solved with the introduction of controlled testing environments, where an administrator can monitor the test taker and the testing environment. How does the USPS view this tradeoff between control and flexibility of the testing venue? Answer 23: Postal examinations are monitored by a certified postal examiner and will be given on the postal intranet. If possible, the online exam should be conducted on the client. At the conclusion of the exam, the information should be uploaded via the intranet. If the intranet is busy or down, there should be a method to upload the completed exam when it can occur. Question 24: The application must allow the administration of up to ten different types of tests and a minimum of 100 of the same tests at the same time. Does this mean a candidate can take a maximum of 10 different examinations during one (1) administration or one hundred (100) candidates taking the same or different examinations? Answer 24: 100 candidates concurrently. Question 25: "The application must store a record of the data entered by the test taker on the application server. This record must identify the applicant, the test given, the response of the applicant, the date and time of the test, and the amount of time the test took. This record must be in ASCII compliant format." The requested data is typically stored in a candidate history data base for reconstruction of the actual examination, along with the candidates responses, taken by the candidate. Is the ASCII file to contain trace/debug type data? Answer 25: No. The data will eventually be loaded into a centrally located Oracle database. Question 26: "The USPS will purchase the tool(s) outright rather than pay by the seat." Please clarify purchase tool(s). Answer 26: The web test tool that we are asking for bids on. Question 27: "Test administrator has the ability to select only certain questions in an examination to create individualized tests." Is it the intent of the administrator to select certain questions for each candidate? Please clarify, (questions are typically selected from a data base to construct individualized tests. Answer 27: Some of the USPS exams have several "Parts". Each "Part" may revolve around a particular skill or ability. We would like to be able to pick a specific "Part" that the candidate needs to take. Question 28: "Ability to bookmark items for the test taker to revisit." Does this mean to allow the candidate the ability to skip questions and return later if time remains on the examination? Answer 28: Yes Question 29: "The supplier shall list any certification held by the provider about this application." Please clarify Certification. Answer 29: Any certification. Question 30: Should I submit pricing data. Answer 30: No, not during the pre-qualification stage.

Record

Loren Data Corp. 20010301/DSOL009.HTM (W-058 SN50E7U0)