Best-worst scaling identified adequate statistical methods and literature search as the most important items of AMSTAR2 (A measurement tool to assess systematic reviews).

Abstract Objective To assess the relative importance of AMSTAR2 items. Study Design and Setting A best-worst scaling object case was conducted among a sample of experts in the field of systematic reviews (SR) and meta-analyses (MA). Respondents were asked in a series of 15 choice tasks to choose the most and the least important item from a set of 4 items from the master list, which included the 16 AMSTAR2 items. Hierarchical Bayes analysis was used to generate the relative importance score (RIS) for each item. Results The most important items highlighted by our 242 experts to conduct overview of reviews and critically assess SR/MA were the appropriateness of statistical analyses and adequacy of the literature search, followed by items regarding the assessment of risk of bias, the research protocol and the assessment of heterogeneity (RIS>6.5). Items related to funding sources and to the assessment of study selection and data extraction in duplicate were rated as least important. Conclusion Although all AMSTAR2 items can be considered as important, our results highlighted the importance of keeping the two items (the appropriateness of statistical analyses and the adequacy of the literature search) among the critical items proposed by AMSTAR2to critically appraise SR/MA.