DOT, FED HWY ADMN, Contracts and Procurement, 400 7th ST, SW, Rm 4410, Washington, D.C. 20590

A -- SAFETY MANAGEMENT SYSTEM: EVALUATIONS SOL DTFH61-95-R-00118 DUE 070695 POC Contact, Lise S. Lyles, HCP30-A, (202) 366-4229 BACKGROUND AND OBJECTIVES: This Broad Agency Announcement (BAA) identifies the Federal Highway Administration's (FHWA) intention to sponsor research in the area of evaluation of the Intermodal Surface Transportation Efficiency Act (ISTEA) mandated Safety Management Systems (SMS) and their outputs. The broad objective of this research is to develop methods for conducting Safety Management System Evaluations. The ISTEA required the development of six management systems, including the Safety Management System (SMS). An Interim Final Rule (IFR) for the management systems was issued on December 1, 1993. This IFR required that ''each State develop, establish, and implement, on a continuing basis an SMS for all public roads, except for federally owned public roads.'' The IFR also required each State to develop a work plan by October 1, 1994 ''...that identifies major activities and responsibilities and includes a schedule that demonstrates full operation and use of the SMS by October 1, 1996.'' Included in the IFR is the requirement that one of the SMS components is to be the ''evaluation of the effectiveness of activities that relate to highway safety performance to guide future highway decisions.'' In addition, individual States have raised the question of how they might determine if the SMS is meeting its stated goal of ''reducing the number and severity of crashes by ensuring that all opportunities to improve highway safety are identified, considered, implemented as appropriate and evaluated in all phases of highway planning, design, construction, maintenance, and operation and by providing information for selecting and implementing effective highway safety strategies and projects.'' The issue of evaluating the SMS was discussed in detail as part of a conference on ''Safety Management Systems -- Methodology'' sponsored by the Transportation Research Board in August, 1994. The proceedings from that conference established terminology and a framework for the discussion of SMS evaluations. In discussing the evaluation of the SMS itself, two types of evaluations were identified: product evaluations and process evaluations. The product of the SMS was defined as ''the reduction of accident frequency and severity,'' while the activities by which the product is obtained constitute the process. In concept, the better the process, the better the product will be. Two levels of evaluation were also defined: macro evaluations and micro evaluations. ''Macro'' evaluations were defined as those that examine the SMS as a whole, while ''micro'' evaluations look at specific programs or actions. Therefore, to evaluate the effectiveness of the SMS at the ''macro'' or system level, one can examine how well the SMS process has been implemented (a macro- process evaluation) and/or how effective the SMS has been in terms of its overall effect on the product; reductions in the total number and severity of accidents (a macro-product evaluation). Participants at the conference had varied opinions as to how the macro-process evaluation might take place. In addition, there was considerable debate as to the feasibility of conducting a macro- product evaluation and the appropriate methodology to use in such an evaluation. The participants did agree that additional research on these questions was needed. In addressing the specific evaluation requirement in the IFR, the conference participants recognized the need to evaluate programs and activities to determine their effectiveness in improving highway safety. The participants also recognized the difficulty and inefficiency of trying to conduct these ''micro'' product evaluations on a State by State basis. The proceedings included a specific recommendation that these evaluations would be best conducted if States could cooperate in their evaluation activities and utilize expert guidance in both the preparatory and analysis stages. The objective of this BAA is to promote creativity and innovation in the development of alternative approaches for conducting the evaluations described above. The scope of this announcement is intentionally broad to invite innovation and to provide the FHWA with the flexibility to sponsor research which best addresses its mission and goals. In keeping with FHWA's desire for innovation and flexibility, the specific approach in the selected proposals will vary and will be negotiated. B. REQUIREMENTS. Develop a methodologically-sound approach for conducting one or both of the following SMS-related evaluations: 1. SMS ''Macro'' evaluation -- How can the overall effectiveness of the SMS be evaluated? Should this evaluation consist of a process evaluation, a product evaluation, or both? What method(s) are appropriate for conducting these evaluations? How can the evaluations be used by the States to determine if the SMS is meeting their needs and can be expected to produce the desired results? How can the evaluations identify those parts of the SMS that are working effectively and those that need to be modified? 2. SMS ''Micro'' evaluation -- What method(s) can be used to conduct evaluations of specific SMS activities (countermeasures, programs, policies)? Is a pooled-data approach involving multiple States feasible? What are the benefits to this approach from a State perspective? What controls would be necessary to insure that statistically-valid results are obtained? What are the advantages and disadvantages of pooling the data on a regional basis versus a national basis? What is the appropriate federal role in these evaluations? How should the information from these evaluations be maintained and disseminated for future use? The Contractor's research effort shall include, but not be limited to: (a) a thorough review of the relevant literature (from the United States and internationally); (b) a detailed presentation of the proposed methodology; (c) a discussion of the theoretical aspects of the proposed methodology (including statistical techniques to be used, sample size required, level of confidence, etc.); and (d) a discussion of the practicality of the proposed methodology (impacts on current practice, resources required, etc.). Offeror proposals should identify which of the two different evaluation levels are being addressed and should provide a detailed overview of the technical approach that will be undertaken. C. DELIVERABLES. The specific deliverables will depend upon the scope and objectives of the negotiated contract. At a minimum, a final report shall be submitted that presents the proposed evaluation methodology and discusses both the theoretical and practical implications of this methodology. Additional deliverables such as interim reports could be required and will be identified in the negotiated statement of work. D. CONTRACT PERIOD. Negotiable, but not to exceed 12 months. E. CONTRACT TYPE. A cost reimbursement type contract is anticipated. F. SUBMISSION OF PROPOSALS. There will be no formal RFP or other solicitation for this equirement. Proposals responding to this BAA must be valid for 200 days from the date specified for receipt of proposals, and may be funded at any time during that validity period. Submit eight (8) copies of your proposal by 4:00 p.m. EST on July 6, 1995, to the Department of Transportation, Federal Highway Administration, 400 Seventh Street, S.W., Room 4410, HCP30-A, Washington, D.C. 20590, Attention: Lise S. Lyles. Please show BAA number and the closing date on the forwarding envelope. Late proposals will be handled in accordance with FAR 15.412. G. INSTRUCTIONS FOR PREPARATION OF PROPOSALS. Any resulting contracts will contain appropriate federal agency clauses for the type of contract awarded. Prior to the contract award, the Contractor shall be required to submit ''Certifications and Representations.'' Data and reporting requirements will be negotiated with the firm(s) selected for contract(s). In responding to this announcement, please prepare your proposal in two parts, as follows: 1. TECHNICAL - Submit a work plan that reflects a review of pertinent research in the proposed area and that describes in detail how the offeror will proceed if awarded a contract. Detail the tasks envisioned for the work, the approach, the methodology, the data collection techniques, any equipment requirements and deployment, other investigation tools to be employed, and the amount of time necessary to perform each task. Resumes of key personnel shall be included as attachments. A separate Statement of Work, detailing the technical tasks to be accomplished and suitable for contractual incorporation, shall be included with the proposal. Note: The technical proposal shall not exceed 50 single-sided, 8.5 X 11 inch pages. Text shall be in 12 pitch or larger type, double-spaced. The page limit includes all information, i.e., charts, illustrations, resumes, attachments, etc. 2. COST - The cost or price breakdown shall be submitted utilizing Standard Form 1411 (Contract Pricing Proposal Sheet) as a cover sheet, together with supporting detailed, complete cost data, including any subcontractor data, in accordance with FAR 15.804-6. Include a person-hour breakdown by task. H. EVALUATION OF PROPOSALS AND AWARD EVALUATION CRITERIA Proposals received will be evaluated in accordance with the evaluation criteria specified herein through a scientific review process. The basis for selecting the proposals for acceptance will be based on a combination of technical merit and cost. Technical factors are more important than cost factors, but since the goal of this effort is to have multiple awards within the available funds, cost will be considered in the evaluation. The technical evaluation will be based on the following criteria: 1. Demonstrated understanding of the issues involved in the evaluation of Safety Management Systems. 2. The technical merit of the proposed research approach, including its completeness, quality, implementation potential, and innovation. 3. Qualifications, capabilities and experience of the staff, with particular emphasis on the principal investigator. I. GOVERNMENT RIGHTS - The Government anticipates that approximately $225,000 will be dedicated to this program in FY- 1995, and anticipates making multiple awards. It is anticipated that two awards will be made for each evaluation level (macro and micro) and that no single proposal will be funded for more than $110,000. The Government reserves the right to incorporate ideas from successful offerors into future contracts. This Broad Agency Announcement does not commit the Government to pay for response preparation costs. (0139)

Loren Data Corp. http://www.ld.com (SYN# 0001 19950522\A-0001.SOL)