Loren Data's SAM Daily™

FBO DAILY ISSUE OF JUNE 09, 2010 FBO #3119
MODIFICATION

A -- Design and Conduct of an Evaluation of TRIO Implementation and Outcomes - Solicitation 11

Notice Date

6/7/2010
 

Notice Type

Modification/Amendment
 

NAICS

541720 — Research and Development in the Social Sciences and Humanities
 

Contracting Office

Department of Education, Contracts & Acquisitions Management, Contracts (All ED Components), 550 12th Street, SW, 7th Floor, Washington, District of Columbia, 20202
 

ZIP Code

20202
 

Solicitation Number

ED-IES-10-R-0015
 

Archive Date

7/13/2010
 

Point of Contact

Natasha Boyce, Phone: 202-245-6128, Vidya Vish, Phone: 2022456167
 

E-Mail Address

Natasha.Boyce@ed.gov, vidya.vish@ed.gov
(Natasha.Boyce@ed.gov, vidya.vish@ed.gov)
 

Small Business Set-Aside

Total Small Business
 

Description

See Revised Attachment C - Award Fee Evaluation Plan Department of Education's Answers to Offeror's Technical and Business Questions Design and Conduct of a Study of TRIO Implementation and Outcomes ED-IES-10-R-0015 Note from the Department of Education: The proposals are due on Monday, June 28, 2010 by noon (12:00 PM) (EST). If the proposals are delivered before 9:30 AM, please contact Brian Johnson at 202-245-6206 or Natasha Boyce at 202-245-6128. If the proposals are delivered after 9:30 AM, you could reach Brian Johnson, Natasha Boyce, or William Spitzgo at 202-245-6497. Each vendor is required to submit total of three volumes: (1) Volume I - Technical proposal, Volume II- Business proposal and Volume III- Past performance reports. Each volume should have one original and four copies. Past Performance: Please submit minimum of three past performance reports for grants/contracts that were awarded in last three years. In addition you may want to submit subcontractors past performance reports. However, subcontractors past performance reports are optional. Q1. In Previous Evaluations of Upward Bound and other TRIO Programs, Page 6: In conjunction with the 2007 changes to the targeting of the program, ED launched a second experimental evaluation of the Upward Bound program, which was to focus on the program's impact on higher risk children. ED terminated the evaluation in February 2008 in response to a Congressional ban on spending for the evaluation. Hence, evaluation of participants who entered the program between October 1992 and March 1994 remains the most recent national evaluation of Upward Bound. Will ED release to the Contractor copies of any protocols and/or preliminary findings related to this terminated study? If so, can such protocols and/or preliminary findings be used as sources for protocol development for the current work outlined in this RFP? Answer to Q1: The contract for the second experimental evaluation was terminated in February 2008 in the midst of the recruitment phase. The contractor had completed the OMB package for the design of the evaluation, which included a form for the collection of baseline student data prior to randomization. The OMB package is attached to this document. No outcomes data was collected. Q2. Key Research Questions for the Study, Q.10, Page 13: To what extent is there meaningful variation in implementation strategies within Upward Bound grantees and between Upward Bound grantees and other pre-college programs? Does a meaningful or representative list of "other" pre-college programs exist? Will ED assist the contractor in identifying types of "other" pre-college programs? Can ED assist in access to these "other" programs? Answer to Q2: ED does not have knowledge of a representative list of other pre-college programs. The American Youth Policy Publication, Success at Every Step: How 23 Programs Support Youth on the Path to College and Beyond provides examples of such programs, but is not a representative list. (See http://www.aypf.org/publications/SuccessAtEveryStep.htm.) If the cooperation of non-TRIO pre-college programs is needed for the design and feasibility phase or the optional evaluation, the contractor will be responsible for gaining access to non-TRIO programs and attaining their cooperation with the study. ED will provide a letter of support for the study that may be useful in the recruitment efforts. Please note the technical proposal review criteria state that each proposal will be rated on the technical qualifications of the proposed staff, including documentation of substantive knowledge of relevant education issues, including policy and practice of programs (especially Upward Bound and other TRIO programs) designed to promote college attendance and completion of students from disadvantaged backgrounds, and their documented experience and skill in conducting rigorous research on those programs, including the collection and analysis of detailed data on implementation strategies. Q3. Will ED contribute names to the list of TWG panelists? Answer to Q3: The contractor is responsible for submission of the names of the technical working group members. The potential TWG members are subject to the review and approval of the COR. (See Task 2, p. 17 of the Performance Work Statement). ED expects the contractor to have enough substantive and technical expertise to be able to recommend the appropriate panel members. 4. Please clarify the role of TWG. Under subtask 3.1 the proposal states "the COR and the TWG shall review the draft data collection instruments within two weeks" (p. 10). Yet, under Task 6 the proposal states, "The revised feasibility report will provide the basis for discussion during the scheduled TWG meeting." Does the contractor meet with TWG at one time only? Or is the contractor expected to maintain communication with the group throughout the 21 months? If ongoing communication is expected, please explain the role of TWG during the Feasibility Phase. Answer to Q4: The TWG will meet only once during the feasibility phase. However, the TWG members' consultant agreements with the prime contractor should state that they will be available for review and consultation on key evaluation products, such as data collection instruments, outside of the meetings. 5. Under subtask 3.1 the contractor is expected to pilot test the UB director interview protocol, and under subtask 3.2 it is noted that the contractor can not collect data without OMB approval. Please clarify the expectations of the contractor to pilot test instruments. Can a contractor pilot test any of the protocols or procedures on 9 or fewer programs or individuals before OMB approval? If so, which entity would make the selection of this pilot group of 9, the COR or the Contractor? Answer to Q5: The pilot test should be conducted on fewer than 9 UB directors so that it is not subject to OMB approval. The contractor will provide a recommended list of UB directors and projects with whom to conduct the survey, and the list will be subject to COR approval. 6. Please explain more about the purpose of the contractor having its own IRB approval (subtask 3.2). While a contractor may not require IRB approval, many colleges and educational institutions do require IRB approval for any studies conducted with their students and/or staff. Would ED assist the Contractor in working with colleges and other educational institutions to acquire their IRB approval in a timely manner? Answer to Q6: This procurement will include activities involving human subjects that are covered (i.e., not exempt) under the protection of human subjects regulations. Hence, the contractor will be required to provide documentation of review and certification of the project from the contractor's cognizant IRB. (For more information about ED and federal regulations regarding research involving human subjects, see: http://www2.ed.gov/policy/fund/reg/clibrary/humansub.html). Contracts that are conducted for IES are not required to obtain individual institutions (e.g. school districts/grantees) IRB approval. However, Upward Bound host institutions (including colleges, universities and community organizations), state education agencies, and school districts involved in the full evaluation may require the contractor to submit documents for approval in their own IRBs or research committees. Where applicable, the contractor is responsible for obtaining approval from relevant IRBs or research committees of the sites that will participate in the study. 7. Will ED facilitate communication with critical stakeholders to obtain access to the extant data sources (Task 4), i.e., APR data from Upward Bound grantees, federal Financial Aid records, National Student Clearinghouse data, and Impact Evaluation of Upward Bound files? Answer to Q7: The COR will provide the following data to the contractor: APR data from Upward Bound grantees and the Impact Evaluation of Upward Bound restricted use data file. The COR will help the contractor access the federal student financial aid records. The contractor will need to purchase data from the National Student Clearinghouse, which is a nonprofit organization. (See http://www.studentclearinghouse.org). Please note that the contractor will need to comply with ED's contractor security screening requirements before it can have access to ED data with personally identifiable information, including the Upward Bound Annual Performance Report data, the data from the Impact Evaluation of Upward Bound, and the federal student financial aid records. Please note the technical proposal review criteria state that each proposal will be rated on the technical qualifications of the proposed staff, including documentation of experience and skill in the use of data from the National Student Clearinghouse Data and federal Student Financial Aid Records. 8. Will ED facilitate communication with states about their longitudinal student data records system? Answer to Q8: In the design and feasibility phase, the contractor will be responsible for communication with states and relevant districts about their longitudinal student records systems. ED will provide a letter of support for the study that may be useful in gaining access to state education agencies. In the optional evaluation phase, the contractor will be responsible for contacting states to recruit them to participate in the study and for obtaining access to state longitudinal data. ED will provide a letter of support for the study that may be useful in the contractor's recruitment efforts. Please note the technical proposal review criteria state that each proposal will be rated on the technical qualifications of the proposed staff, including their documented experience and skill in gathering and analyzing state or school district longitudinal school records data and their documented experience and skill in gaining cooperation of grantees, states, and high schools for an evaluation effort. 9. Under Task 4, the contractor is expected to develop an analysis plan using "the restricted use files from ED's Impact Evaluation of Upward Bound" to answer several research questions. It is not clear whether the data to answer the questions are available in the Impact Evaluation of Upward Bound. Is ED looking for an analysis plan with the existing data set to answer the questions? If so, should we assume the Impact Evaluation of Upward Bound files contain the necessary data to answer the questions? Or, does ED want the contractor to identify additional data or data sets needed to answer the research questions? Answer to Q9: The COR will provide the contractor individual student-level data from ED's Impact Evaluation of Upward Bound in the form of a restricted use file. Attachment B to the RFP, the 5th follow-up survey from the Impact Evaluation of Upward Bound, provides information on the types of variables included in the data files. The PWS states that during the design and feasibility phase of the study, the contractor shall design and conduct data analyses using the data from the Impact Evaluation of Upward Bound. Note that instructions to Offerors states "Offerors are encouraged to suggest improvements to the approaches in the PWS, along with a justification of the merits of these alternative approaches." Hence, if the offeror believes the analysis plan would be improved by the inclusion of additional data sets, the offeror may want to suggest and justify the improvements in its proposal. 10. One Task 4 deliverable is an "Analysis Plan for Restricted Use Data from Upward Bound Impact Evaluation." However, under Task 6, it is expected that the feasibility report will include a chapter that uses data from the Upward Bound Impact Evaluation to examine the research questions listed under Task 4. Is the contractor expected to develop a plan (Task 4) and carry out the analyses for the feasibility report? Answer to Q10: In the design and feasibility phase of the contract, the contractor will develop an analysis plan for the restricted use of the Upward Bound Impact Study, and carry out the analyses. Task 6 of the PWS includes the following language on the feasibility and design report (p. 21): "The report will also include a chapter that uses data from the Impact Evaluation of Upward Bound to examine the extent to which Upward Bound participants and non-participants are served by SSS and McNair as college students and an analysis of the extent to which student outcome (e.g., postsecondary completion rates) varied by project characteristics such as type of host institution (two-year versus four-year college versus community organization) and the selectiveness of the host institution's undergraduate admissions process." 11. Under subtask 5.2, will ED work with the contractor to identify grantees for site visits? Will ED facilitate communication with grantees to help the contractor arrange the site visits? Answer to Q11: Upon contract award, ED will provide the contractor an electronic list of Upward Bound grantees and the contact information of their project directors. The contractor is responsible for development of a list 20 recommended grantees for the site visits, which is subject to COR approval. Once COR approves the list, ED will send a letter of support for the study that will remind grantees that cooperation with the evaluation is required as a condition of the grant. The contractor will be responsible for contacting and developing cooperative relationships with each of the grantees in order to schedule and conduct the site visits. Please note the technical proposal review criteria state that each proposal will be rated on the technical qualifications of the proposed staff, including the documented experience and skill in gaining cooperation of grantees, states, and high schools for an evaluation effort. 12. Task 17- Collection of Student Surveys and Parental Consent Forms (Cost Plus Award Fee Task), page 31: In fall 2012, prior to the administration of the student survey, the contractor shall obtain active parental consent for the collection of the survey. Is parental consent required for all students or only those under the age of 18? Answer to Q12: Upon further reflection, ED has determined that under federal law, passive consent for the collection of the student survey (for students under the age of 18) will be sufficient. The contractor will only need to obtain active parental consent when it is a grantee or school district requirement. Parental consent is not required for students over the age of 18. 13. Our understanding is that the small business prime is required to do 51% of the work. If the small business contracts with an individual consultant for 5% of the work, does that 5% count towards the 51% of the work? Answer to Q13: ED would prefer the prime to do 51% of the work, but it is permissible for the prime to subcontract with a consultant for 5% of the work. 14. It is possible to have co-PIs where one PI is from the small business and one PI is from a contracted research company? Answer to Q14: Yes. 15. Are tables included in the 75 page maximum? Do tables need to be double spaced? Answer to Q15: Yes, tables should be included in the 75-page maximum. Tables do not need to be double spaced. 16. Does IES want the technical and budget approach for the Phase 2? Answer to Q16: Yes. While many aspects of the evaluation approach will not be determined until the feasibility phase is partially or fully completed, for the purposes of preparing a response to this Statement of Work, the instructions to offerors states that offerors should develop their business and technical proposals based on several assumptions about the features and timelines of the quasi-experimental evaluation of implementation strategies in Upward Bound. See p. 61 of the document labeled "ED-IES-10-0015" for further details. 17. What are the expectations for the length (i.e., number of days) of the 20 site visits to be conducted during Phase 1? Are there particular sites which would be targeted in the 20 visits beyond targeting a diverse set of sites? Answer to Q17: Offerors should propose the sites to be targeted and the duration of site visits in a manner that they think would best fulfill the technical requirements of the RFP. 18. On page 66, there is a request for 4 recent contracts but on page 68 the request is for three. Should the application include three or four? Answer to Q18: Please submit minimum of three past performance reports for grants/contracts that were awarded in last three years. In addition you may want to submit subcontractors past performance reports. However, subcontractors past performance reports are optional. 19. Must all tables and figures be paginated within the technical proposal page limitations? Answer to Q19: Yes, all tables and figures may be paginated within the technical proposal page limitations. 20. May tables and figures use reduced font sizes? Answer to Q20: Yes, Figures and tables may be excluded from the 12-point font requirement, but should use no smaller than an 8-point font. 21. In reading the RFP for the TRIO evaluation on page 61 there is a reference to sampling 2,750 secondary students. However, given that a major goal of Upward Bound is to increase the number of students completing secondary education, continuing to post-secondary education and completing a college education, it seems that it would be crucial to include Upward Bound alumni as well. Would it be acceptable to include alumni of Upward Bound for the past two to three years in addition to the secondary students? Perhaps this question is addressed elsewhere in the materials provided on the fbo.gov site, but I could not find any reference to sampling post-secondary students. Answer to Q21: On p. 11, the RFP explicitly asks that the contractor's design and feasibility report address possible data collection strategies for "Upward Bound alumni", that is, earlier cohorts of participants. "Given that UB is intended to increase college enrollment and completion, the contractor shall explore evaluation designs in which students from earlier cohorts are linked to more recent data on college enrollment and completion. The challenge of using data from earlier cohorts of participants is that it would need to be linked to implementation data from prior years. OPE has performance report data on college enrollment of select cohorts of UB participants but has not collected information on implementation strategies needed to link the mix of services to outcomes. The contractor shall discuss data collection strategies for implementation data in a design that focused on earlier cohorts." 22. The instructions state that the estimated level of effort is approximately 3.5 person years for the base contract and 7 person years for the optional phase, assuming 1 person year = 2,080 hrs. Is this correct: 10.5 person years for the entire contract? Answer to Q22: Yes. 23. On p. 1 of the Award Fee Evaluation Plan (AFEP), it states that the technical performance of the Contractor will be evaluated on four key deliverables and that the award fee will be capped at $80,000. In the "Award Fee Earnings by Key Deliverable" table on p.6 of the same document, only two deliverables are listed and the maximum fee available appears to be $75,000. Please clarify. Answer to Q23: The AFEP has been changed to state that the award fee will be capped at $75,000. 24. On p. 3 of the Award Fee Evaluation Plan (AFEP), one of the key deliverables listed to evaluate the contractor's performance is "Small Business Participation - to be evaluated at the end of the fourth year of the contract." Since this procurement is a small business set aside, we are confused by what is meant by this deliverable. Please clarify. Answer to Q24: The AFEP has been changed to eliminate the reference to small business participation. 25. On p. 3 of the Award Fee Evaluation Plan (AFEP), it states that the COR will use "participant evaluations provided by the contractor to evaluate the contractor's performance." Could you please provide further detail about these evaluations as we are unable to find any other reference to them in the PWS or AFEP. Answer to Q25: The AFEP has been changed to eliminate the reference to participant evaluations. 26. On page 11, Section B.4, reference is made to the number of months for review for the award fee. The number of months in missing. Answer to Q26: Award fee evaluation will be conducted at the end of the contract therefore the number of months was left blank. 27. Should past performance materials submitted per sections L and M be bound separately or included as part of the Technical Proposal. Page 66 indicates materials should be submitted "as a separately bound part of its proposal" while p. 68 indicates Past Performance shall be included in the Technical Proposal. If included in the Technical Proposal, does it count against the 75-page limit? Answer to Q27: Each vendor is required to submit total of three volumes: (1) Volume I - Technical proposal, Volume II- Business proposal and Volume III- Past performance reports. Each volume should have one original and four copies. Please submit minimum of three past performance reports for grants/contracts that were awarded in last three years. The past performance materials will not count against the 75-page limit for the technical proposals. The past performance materials will not count against the 75-page limit for the technical proposals. 28. The cost proposal instructions (pages 64 and 65 of the RFP) indicate that offerors and their subcontractors should provide detailed information to support the cost proposal and should not use fully loaded labor rates for the purposes of costing. We are assuming, and want to confirm, that this requirement is applicable to the cost reimbursable tasks only, as such level of cost detail is inconsistent with FAR regulations in 15.403 regarding competitive fixed price elements of this proposed contract. Answer to Q28: This requirement is applicable to the fixed price tasks and cost reimbursement tasks. The Department of Education requests to evaluate the build-up of costs for the fixed price tasks as well. 29. Page 65 also indicates that offerors "may propose 3% base fee and 3% Award fee for cost reimbursement tasks." We respectfully submit that the proposed 6% total fee (assuming 100% of the award fee is achieved) is below the level of typical CPAF contracts. In particular, it is lower than the total potential fee offered in IES research contracts over the past 8 years. May offerors propose different base and award fee percentages consistent with this practice, with the understanding that the Government will ultimately award to the offeror it considers as providing the best overall value to the Government? Answer to Q29: The Department of Education intends to award a 3% base fee for the Cost Reimbursement tasks based on the competitive nature of the procurement. Offerors may propose above or below the base fee anticipated by the Department. Award fees will be fixed to the amounts in the Award Fee Plan (Total $75,000). 30. On page 65 the RFP appears to suggest that a fee of 3% should be applied to fixed price tasks. This is inconsistent with the FAR, which only speaks to profit limitations on cost type contracts. It is also inconsistent with well established industry practices regarding reasonable profit rates for fixed price contracts. Because all of the financial and performance risk is assumed by the contractor under a fixed price arrangement, profit rates are typically higher than for cost reimbursable contracts. Could the Department please clarify this statement? Answer to Q30: The Department of Education wishes to clarify that only the Award fee amounts are fixed in the Award Fee Plan. All other fees shall be negotiated. 31. For consistency in budgeting the evaluation phase, how many Upward Bound target schools should be assumed to be in the sample (target and comparison) for each of the 45 grantees? Answer to Q31: Please assume that 10 Upward Bound target schools will be in the sample for each of the 45 grantees.
 

Web Link

FBO.gov Permalink
(https://www.fbo.gov/spg/ED/OCFO/CPO/ED-IES-10-R-0015/listing.html)
 

Place of Performance

Address: U.S Department of Education, OCFO/Contracts & Acquisitions Management, 550 12th Street, SW, 7th Floor, Washington, District of Columbia 20202, Washington,, District of Columbia, 20202, United States

Zip Code: 20202
 

Record

SN02169939-W 20100609/100607234617-8cc165ae35595eaa2220a3409f36534f (fbodaily.com)
 

Source

FedBizOpps Link to This Notice
(may not be valid after Archive Date)

---

| FSG Index  |  This Issue's Index  |  Today's FBO Daily Index Page |