When evaluating proposals received in response to competitive procurements, the government may utilize a variety of techniques to ensure it obtains best value depending on the specific circumstances of the acquisition. In such procurements involving any one or a combination of source selection approaches, government evaluators often assign strengths and weaknesses to various aspects of the proposals to justify the government’s tradeoff decisions. A strength is an aspect of an offeror’s proposal that improves the probability of successful performance. Meanwhile, a weakness could be a defect in the proposal that increases the risk of unsuccessful performance. Notably, Federal Acquisition Regulation (FAR) § 15.306(d)(3) requires agencies to discuss significant weaknesses with offerors in the competitive ranges. However, not all strengths and weaknesses are created equal, and evaluators may not derive adjectival ratings from a mechanical count of the strengths and weaknesses alone. In other words, the government must qualitatively assess proposals based on the stated criteria in the solicitation when assigning adjectival ratings in competitive procurements. Additionally, agencies must adequately document their rationale so that a bid protest adjudicative forum such as the Government Accountability Office (GAO) has sufficient basis to review the underlying evaluation decision. Should the government assign adjectival ratings on a mechanical counting of strengths and weaknesses alone or fail to adequately document its evaluation rationale, the resulting award may be protested.
In B-422616, a decision issued on August 28, 2024, the GAO sustained such a bid protest challenging the National Nuclear Security Administration’s (NNSA) mechanical counting of strengths and weaknesses in a best value evaluation. The NNSA issued the small business set-aside request for quotation (RFQ) for Agile software development support services via the General Services Administration’s (GSA) schedule for information technology (IT) professional services, pursuant to the Federal Acquisition Regulation (FAR) subpart 8.4. The RFQ provided that responsive quotations would be evaluated based on three stated criteria: (1) technical understanding, (2) relevant experience, and (3) cost/price – listed here in the order of importance. The non-cost/price factors were to be evaluated using the adjectival rating provided in the RFQ. The government’s technical evaluation panel (TEP) supported its assignment of adjectival rating with strengths and weaknesses, while making other narrative findings regarding the responsive quotations. While the RFQ did not provide a definition of “strength,” the evaluation report stated that a strength could be assigned for an aspect of the quotation that was considered to meaningfully increase the government’s expectation of successful performance.
The awardee's quotation was assigned the adjectival rating of "Good" for both non-cost/price factors, with two strengths and one weakness. The NNSA TEP concluded that the awardee's quotation merited the adjectival rating of "Good" because its strengths outweighed the weaknesses. The awardee's proposed price was $ 42,910,442. Meanwhile, the protestor received an "Acceptable" rating for the technical understanding factor and an "Outstanding" rating for the relevant experience factor. The protestor was assigned a total of three strengths and three weaknesses, all in the technical understanding factor. The protestor proposed price was $ 48,041,023. The TEP noted significant strengths in the incumbent protestor’s quotation, including that it demonstrated a clear understanding of the intent and purpose of work defined in the PWS. The protestor was also assigned a strength under the staffing plan element for presenting a labor mix consistent with the scope of the PWS and proposing personnel that demonstrated a very good understanding of the overall level of effort. Meanwhile, the protestor’s weaknesses were relatively minor. For instance, the protestor was assigned a weakness for generally failing to follow the format of the RFQ and PWS in describing the technical approach for performing the work. Similarly, the remaining two weaknesses were assigned for the protestor’s failure to confirm its attendance in bi-weekly stand-up meetings and its failure to address that it would facilitate system demonstrations as needed.
Notably, in assigning the protestor's quotation an "Acceptable" rating for the technical understanding factor, the TEP noted that it assigned the rating based on its finding that the protestor's strengths and weaknesses were generally offsetting. To support the adjectival rating, the TEP merely regurgitated the RFQ's description of what constituted an Acceptable rating. Ultimately, based on the comparison of the assigned adjectival ratings, the NNSA source selection authority (SSA) determined that the awardee's quotation was superior to the protestor's quotation in the most important non-cost/price factor and that the protestor's quotation offered no technical benefits over the awardee's that would justify paying the approximately $5 million price premium.
In the protest that followed, the protestor alleged that the NNSA TEP’s assigned ratings for its technical understanding factor was unreasonable because the agency simply counted the number of strengths and weaknesses to arrive at the rating. According to the protestor, its strengths were substantive, and its weaknesses were relatively minor. Therefore, the protestor argued that the government’s evaluation failed to substantively determine and document whether the strengths and weaknesses did, in fact, offset one another. The protestor also contended that it deserved a higher rating in the technical understanding factor because the RFQ defined the rating of “Good” as containing strengths that outweighed any weaknesses. The agency essentially provided no explanation – whether contemporaneous or in post-protest documentation – to support the reasonableness of the disputed “Acceptable” rating. Thus, the protestor argued that even though it was assigned the same number of strengths and weaknesses, its quotation should have received a higher adjectival rating of “Good” in the technical understanding factor because its strengths qualitatively outweighed its weaknesses.
In agreeing with the protestor, the GAO noted that the evaluation record failed to adequately explain the assigned adjectival ratings. For instance, the NNSA TEP did not explain how or why the protestor’s strengths and weaknesses offset one another. Additionally, while it was evident that qualitative aspects of the protestor’s quotation were considered and documented as strengths and weaknesses, the evaluation record failed to explain how the NNSA TEP weighed those strengths and weaknesses to ultimately arrive at the assigned rating. Furthermore, the NNSA’s evaluation of the awardee’s quotation was similarly defective. The awardee’s evaluation only documented that its two strengths outweighed its one weakness, meriting it a “Good” rating. Therefore, the GAO determined that absent further explanation demonstrating how the strengths and weaknesses were weighed and assessed, there was no indication that the NNSA TEP did anything more than simply subtract the number of assessed weaknesses from the number of assessed strengths while regurgitating the RFQ description of the overall adjectival rating as support. The GAO concluded that such a mechanical counting of strengths and weaknesses to assign adjectival ratings left the GAO nothing to review to ensure that the agency’s evaluation was consistent with the solicitation terms and any applicable procurement statutes and regulations.
When assigning adjectival ratings, the government must do more than just mechanically count the strengths and weaknesses. In competitive procurements, evaluators must qualitatively assess the underlying merits of the offerors’ proposals. The proper assignment of adjectival ratings using strengths and weaknesses necessarily involves assessing their weights relative to each other. In making evaluation decisions along the best value continuum, evaluators must also adequately document supporting rationale behind their evaluation decisions. When evaluators fail to do so, they run the risk that protest adjudicative forums would not have a sufficiently detailed record to review their evaluation decisions. When available, disappointed offerors in competitive procurements should request a debrief or a brief explanation following the award to better understand the adequacy of the government’s supporting rationale for their evaluation decisions. If the evaluation results indicate that the adjectival ratings are based on a mechanical count of the strengths and weaknesses, disappointed offerors in line for award may challenge the award decision as unreasonable.
This Bid Protests Insight provides a general summary of the applicable law in the practice area and does not constitute legal advice. Contractors wishing to learn more are encouraged to consult the TILLIT LAW PLLC Client Portal or Contact Us to determine how the law would apply in a specific situation.
