Processed meat intake and chronic disease morbidity and mortality: An overview of systematic reviews and meta-analyses

Summary

Despite the nutritional value of meat, a large volume of reviews and meta-analyses suggests that processed meat intake is associated with an increased risk of chronic diseases. However, assessments of the quality of these published reviews internal validity are generally lacking. We systematically reviewed and assessed the quality alongside summarizing the results of previously published systematic reviews and meta-analyses that examined the association between processed meat intake and cancers, type II diabetes (T2D), and cardiovascular diseases (CVD). Reviews and meta-analyses published until May 2018 were identified through a systematic literature search in the databases MEDLINE and EMBASE, and reference lists of included reviews. The quality of the systematic reviews and meta-analyses was assessed using A Measurement Tool to Assess Systematic Reviews (AMSTAR). All eligible reviews had to comply with two quality requirements: providing sufficient information on quality assessment of the primary studies and a comprehensive search. The results were summarized for T2D, CVD, and each of the different cancer types. The certainty in the estimates of the individual outcomes was rated using the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) method. In total, 22 systematic reviews were eligible and thus included in this review. More than 100 reviews were excluded because quality assessment of the primary studies had not been performed. The AMSTAR score of the included reviews ranged from 5 to 8 indicating moderate quality. Overall, the quality assessments of primary studies of the reviews are generally lacking; the scientific quality of the systematic reviews reporting positive associations between processed meat intake and risk of various cancers, T2D and CVD is moderate, and the results from case-control studies suggest more often a positive association than the results from cohort studies. The overall certainty in the evidence was very low across all individual outcomes, due to serious risk of bias and imprecision.

Keywords:

Meat – Systematic reviews – Database searching – Research quality assessment – cardiovascular diseases – Test statistics – Case-control studies

Introduction

Chronic diseases such as cancers, type II diabetes (T2D) and cardiovascular diseases (CVD) are a substantial burden to society globally. According to World Health Organization (WHO), 36 million deaths per year, equivalent to 63% of all global deaths, can be attributed to these diseases and almost 40% of these deaths occur before the age of 70 years [1]. Modifiable risk factors for the development of chronic diseases include smoking, alcohol intake, physical activity, and body weight. In addition, dietary behaviors play a major role in the development of many chronic diseases [2]. Specifically, prudent dietary patterns, such as the Mediterranean diet, have been shown to be associated with reduced risk of several chronic diseases and mortality, as well as being beneficial for some of the risk factors for chronic diseases, such as hypertension [3–6]. Conversely, the typical “Western” diet has been associated with a higher risk of chronic diseases [7], although, at least for CVD, the evidence has been conflicting [8].

One of the main components of the “Western” diet is meat and meat products, which make a substantial contribution to the daily dietary intake of total energy, protein and fat, as well as important vitamins and minerals. A high consumption of processed meat products in relation to the risk of multiple chronic diseases has been studied extensively in reviews and meta-analyses, which have led to recommendations to moderate the consumption of preserved meat, such as sausages, salami, bacon and ham, for disease risk reduction worldwide [9–12]. However, the lack of quality assessment of the reviews providing the evidence base for these recommendations needs to be acknowledged [13–15]. Reviews that are not systematic, i.e. narrative reviews suffers from flaws, such as lack of reproducibility, lack of transparent methods, and a large degree of subjectivity that may be misleading in the conclusions made. Consequently, some authors have suggested placing constraints on the inclusion criteria of the reviews in relation to search strategy and quality assessment [16, 17], to ensure that the included reviews are ‘systematic’ and to guarantee a minimum level of methodological thoroughness.

The objective of this study was to provide a critical assessment of the available systematic reviews that examined the association between processed meat intake and the most common chronic diseases, i.e. different types of cancers (incidence and mortality), T2D (incidence), and CVD (incidence and mortality). As part of the assessment of the internal validity of the systematic reviews, we further aimed to explore potential reasons for heterogeneous results in meta-analyses by considering variations in factors such as study design and quality.

Methods

This systematic review was performed in accordance with the recommendations of the Cochrane Collaboration and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [18]. The study protocol was pre-specified and registered in advance of the literature search in the International Prospective Register of Systematic Reviews (PROSPERO) (CRD42017055272). No changes to the methods were made after commencement of this protocol, except for a post hoc quality evaluation of one review. This evaluation was performed using A Measurement Tool to Assess Systematic Reviews (AMSTAR).

One researcher (IC) conducted the literature search using a pre-specified search strategy (S1 Table), and two out of four researchers independently (MNH, IC, KMR, JFR) screened titles and abstracts of the selected articles, thoroughly assessed the full text reports according to the eligibility criteria, and performed data extraction of each of the studies eligible for inclusion. The review authors were not blinded to the journal titles, study authors/institutions or year of publication. The quality assessments using A Measurement Tool to Assess Systematic Reviews (AMSTAR) were completed independently by two out of four researchers (MNH, IC, KMR, JFR). Disagreements were resolved through discussions or by consulting a third reviewer (BLH).

Search strategy

The literature search was performed using the databases MEDLINE via PubMed (from 1966) and EMBASE via Ovid (from 1974) on the 8^th of May 2018. Reference lists of the included systematic reviews were screened to capture relevant systematic reviews that were not found during the initial search. The following key words were used: “meat”, “review”, “meta-analysis”. The search was limited to title/abstract, the filter “humans” was used, and only English literature was considered. Moreover, restrictions were made for the following key words to exclude systematic reviews examining meat that has been contaminated with i.e. campylobacter, or other kind of zoonosis or pathogens: “zoonotic”, “pathogen”, “bacteriocins”, “microbial”, “antimicrobial”, “campylobacter”, “contamination”, “contaminated”, “food allergy”. The search strategy is presented in S1 Table.

Study selection

The selected systematic reviews, including systematic reviews of systematic reviews, and meta-analyses were imported to the reference management software, EndNote X7.4, and duplicates were removed. We further only considered reviews that examined a healthy adult population (≥18 years) at baseline, and thus reviews restricted to specific patient populations were excluded. The primary outcomes were incidence and/or mortality of any cancers, T2D, and any CVD; thus reviews concerning other disease outcomes where excluded. There were no specific restrictions regarding the definition of processed meat. If reviews mentioned “processed meat” and/or refereed to processing methods (i.e. salted meat) and/or listed processed meat products (bacon, ham, sausages, luncheon meats, etc.), they were considered eligible for inclusion. Reviews that presented combined results on processed meat with other types of meat, e.g. fresh red meat (unprocessed beef, pork, lamb, etc.), were excluded.

Full-text versions were obtained and examined for any review that appeared to meet the inclusion criteria based on the title/abstract, or where a definite decision could not be made based on the title/abstract alone.

Quality assessment and data extraction

Full-texts of the selected systematic reviews and meta-analyses were appraised using AMSTAR criteria [19,20]. AMSTAR is an 11-item tool to assess the general methodological quality of systematic reviews [19,20]. This tool has been internally and externally validated and has been found to have good reliability [21]. Based on the tool, a score was calculated were each AMSTAR item met give 1 point and the maximum score is 11. A score from 0–4 indicates low quality, 5–8 moderate quality, and 9–11 high quality, which is the most frequently used categorization method [22].

We excluded systematic reviews and meta-analyses that did not assess or document the scientific quality of the included primary studies (did not meet AMASTAR item number 7) as well as those reviews that did not use at least two electronic sources in their search strategy (did not meet AMSTAR item number 3) [19].

Subsequently, descriptive information of the remaining included systematic reviews was extracted using a predefined structured form developed for this review, separately for the three main outcomes and their risk factors. The information was related to study design, study population, exposure characteristics, number of included studies, authors’ conclusions, and funding. As part of the assessment of the internal validity of the systematic reviews, we further aimed to explore potential reasons for heterogeneous results in meta-analyses by considering variations in factors such as study design and quality. The results were summarized for each cancer type, T2D, and CVD.

Post hoc evaluations of the certainty in the estimates of the individual outcomes of interest were rated using Grading of Recommendations, Assessment, Development and Evaluations (GRADE) method [23]. Four possible ratings of the quality were available: high, moderate, low, and very low. Downgrading was done, by investigating the following five domains: 1) risk of bias; 2) inconsistency; 3) indirectness; 4) imprecision and 5) publication bias.

Results

Study selection

After removing duplicates, the literature search resulted in 1,161 records (flowchart presented in Fig 1). Of these, 1 extra duplicate was identified and 894 papers were excluded during the title and abstract screening due to not being relevant for the present review, which resulted in 266 papers extracted for full-text assessment.

During the full-text assessment, we excluded reviews that: were not systematic reviews (k = 84); were conference abstracts (k = 20); did not present results for processed meat alone, i.e. separate the results of processed meat from total meat intake or fresh red meat intake (k = 26); did not include cancer, T2D or CVD as outcomes (k = 4), included the wrong population (individuals with T2D) (k = 1), did not perform a quality assessment of primary studies (AMSTAR Q7, k = 107) and did not perform a comprehensive search (AMSTAR Q3, k = 1), thus resulting in a total of 22 eligible systematic reviews [13,15,24–43]. A list of the reviews excluded during the full-text assessment, which includes the reason for exclusion for each review, can be found in S2 Table.

To alleviate any concerns regarding inaccurate conclusions that may be drawn by not including all existing reviews, we also performed a post hoc quality assessment of one additional systematic review that did not perform a comprehensive search but did assess quality.

Description of included systematic reviews

Out of the 22 eligible systematic reviews, 20 included meta-analyses and 2 were systematic reviews of systematic reviews. The studies were all published between 2010 and 2017. The characteristics of all the studies, sorted by the outcome, are summarized in Tables 1–5, and the results of the meta-analyses are presented in Tables 6 and 7. An overview of all results from the subgroup analyses are presented in S5 Table.

Quality assessment of included systematic reviews

None of the 22 included systematic reviews received the maximum AMSTAR score (Table 1–6; detailed quality scores provided in S3 Table). Based on the AMSTAR criteria they were all classified to be of moderate quality, with a mean value of 6.36 and a range from 5 to 8. The items within the AMSTAR scoring systems that were most infrequently fulfilled across the included reviews were the AMSTAR item number 1,4, 8 and 11. The AMSTAR item number 1 concerned whether an “a priori” design was provided (i.e. referred to development of a protocol, ethics approval, or a predetermined/a priori published research objectives), and only one out of the 22 reviews fulfilled that [35]. The AMSTAR item number 4 referred whether the status of publication (grey literature / unpublished literature) was used as an inclusion criterion. To receive a “yes” in this item, the authors should state that they searched for reports regardless of their publication type, in other words that they searched for “grey literature” or “unpublished literature”. There were only 3 out of 22 systematic reviews that fulfilled this item [15,30,34]. The AMSTAR item number 8 referred to whether the scientific quality grading was used appropriately in formulating conclusions, i.e. describing the quality of the evidence [35], and only 5 out of 22 systematic reviews fulfilled that [30,33,34,39,40]. All reviews assessed publication bias, when applicable, and overall there were no indication of such bias, regardless of test, except in three reviews (on hepatocellular carcinoma [32], gastric cancer [41] and cancer mortality [33] that all favored significant results (Tables 6 and 7). The AMSTAR item number 11 considered whether the potential conflict of interest was addressed. To receive a “yes” for this item, conflict of interest needs to be addressed both for each of the included studies, and for the systematic review itself. There were only two out of 22 systematic reviews that addressed conflict of interest appropriately according to AMSTAR [30,34]. A narrative AMSTAR evaluation per outcome has provided in S6 Table.

In accordance to the GRADE approach, results from observational studies (case-control and cohort studies) are by default considered to be of low quality, yet the quality may be upgraded to moderate if there are no issues with confounding, there is a large effect, and/or there is a consistent dose-response relationship. Since this was not the case, as residual confounding is always suspected to be present in observational studies, and the effect and the dose-response pattern were modest or not significant, the certainty of the effect estimate was downgraded to very low, due to serious risk of bias and/or serious imprecision. There were issues with inconsistency (heterogeneity) in more than half of the included meta-analyses (reported in Table 4), but there were no issues regarding indirectness or publication bias (Tables 6 and 7).

Qualitative synthesis of included systematic reviews on cancers

Most of the included systematic reviews concerned different types of cancer (19 out of 22 systematic reviews).

Esophageal cancer

Both reviews indicated an overall association between processed meat intake and risk of esophageal cancer with a summary relative risk (RR) estimate for the highest versus the lowest categories of processed meat intake of 1.32 (95% CI: 1.08, 1.62) and 1.33 (95% CI: 1.04, 1.69) in the meta-analyses by Choi et al. [28] and Zhu et al. [42], respectively (Table 6)). Choi et al. based their meta-analysis on 15 case-control studies and 3 cohort studies [28], whereas Zhu et al. included 12 case-control studies and 3 cohort studies [42]. Both meta-analyses showed a large degree of heterogeneity of approximately 60% (Table 6). Nine of the case-control studies, and the 3 cohort studies were overlapping in the meta-analyses of the systematic reviews (S4 Table). In Choi et all [28], they reported meta-analyses subdivided by study design showing that the direct associations between a high processed meat intake and risk of esophageal cancer remained for the case-control studies (RR: 1.36, 95% CI: 1.07, 1.74) but not for the cohort studies (RR: 1.25, 95% CI: 0.83, 1.86).

Head and neck cancer (nasopharyngeal carcinoma)

The results of this review [31] were based on 13 case-control studies, and suggested an increased risk of nasopharyngeal carcinoma among individuals with low intake of processed meat (<30 g/week) compared to those never eating processed meat (RR: 1.46; 95% CI: 1.31, 1.64; Table 6). The meta-analysis also showed heterogeneity across the primary studies (I² = 61%, p = 0.004; Table 6).

Pancreatic cancer

The review included 6 case-control studies and 14 cohort studies [27] and reported that processed meat consumption (highest versus lowest category) was positively associated with pancreatic risk in case-control studies (RR: 1.62; 95% CI: 1.17, 2.26) [27], however no association was observed in cohort studies (RR: 1.09; 95% CI: 0.96, 1.23) [27] (Table 6). The meta-analysis showed substantial heterogeneity among both the case-cohort studies (I² = 58%) and the cohort studies (I² = 51%; Table 6).

Liver cancer (hepatocellular carcinoma)

Processed meat intake was not associated with liver cancer in this meta-analysis (RR 1.01; 95% CI: 0.79, 1.28; Table 6), based on 2 case-control studies and 3 cohort studies. There was no significant heterogeneity across the studies (Table 6).

Gastric cancer

Three of the studies [29,30,41] concluded that there was an association between processed meat intake and gastric cancer risk; while one study (Zhao et al.) concluded the same in case-control studies, but found no association in cohort studies [26] (Tables 1 and 6). The meta-analysis by Fang et al. included only cohort studies (k = 12) and found a RR of 1.15 (95% CI: 1.03, 1.29; neither I² nor a p-value for the test of homogeneity was not reported) [29], whereas the meta-analysis by Zhu et al. included 17 case-control studies and 9 cohort studies (S4 Table), and found a RR 1.45 (1.26, 1.65), with high degree of heterogeneity across the studies (I² = 61.0%) [41]. In addition, Zhao et al. included in their meta-analysis 25 case-control studies and 8 cohort studies and found a RR of 1.76 (95% CI: 1.51, 2.05) with heterogeneity across the studies (I² = 59%) and RR of 1.23 (95% CI: 0.98, 1.55) with moderate heterogeneity (I² = 43%), respectively. Among the cohort studies there were 8 studies overlapping between the Fang et al. and the Zhu et al. reviews and 5 studies overlapping between Fang et al., Zhu et al. and Zhao et al.; whereas there were 6 case-control studies overlapping between Zhu et al. and Zhao et al. reviews (S4 Table). In the subgroup analysis, Zhu et al. found similar associations in both the case-control and the cohort studies (S5 Table) [41].

Brain cancer (glioma)

Based on 13 case-control studies and 4 cohort studies, the meta-analysis in the systematic review by Saneii and colleagues found no association between processed meat intake and glioma risk (RR: 1.14; 95% CI: 0.98, 1.33) [35]. There was some degree of heterogeneity across studies (I² = 50.6%). The systematic review of systematic reviews by Quach and colleagues [34] included one study [44], which found an increased risk of adult glioma in relation to a high cured meat intake.

Ovarian cancer

On the basis of 5 cohort studies, the meta-analysis did not suggest an association between processed meat consumption and ovarian cancer (Tables 1 and 6)–either by low vs. high intake or in the dose response analyses (S5 Table). There was no significant heterogeneity across the primary studies (Table 6).

Non-Hodgkin lymphoma

The two systematic reviews reached different conclusions: the meta-analysis by Solimini et al. found no association [36], while the meta-analysis by Yang et al. suggested a direct association with risk of Non-Hodgkin lymphoma (RR: 1.17; 95% CI: 1.07, 1.29 for high vs. low processed meat intake) [40]. The meta-analyses from the two reviews included the same 3 cohort studies but the case-control studies differed; out of a total of 14 case-control studies, Solimini et al. included 8 case-control studies and Yang et al. included 13 case-control studies; only 7 case-control studies overlapped between the two reviews. None of the meta-analyses showed significant heterogeneity across the primary studies (Table 6). Both meta-analyses found associations in the case-control studies, but not in the cohort studies (S5 Table).

Lung cancer

In relation to processed meat consumption and lung cancer risk, the included systematic review suggested no association [39]. There was a high degree of heterogeneity across the studies in the meta-analysis (I² = 79.5%).

Oral cavity and orophanx cancer

The meta-analysis indicated an association (RR: 1.91; 95% CI: 1.19, 3.06) [38]. Moreover, there was a high degree of heterogeneity across the studies in the meta-analysis (I² = 85.9%; Table 6).

Renal cell carcinoma

The meta-analysis of this systematic review was based on 15 case-control studies and 4 cohort studies, and found an overall RR of 1.13 (95% CI: 1.03, 1.24) with moderate heterogeneity (I² = 45.6%; Table 6) [24].

Colorectal cancer

The authors concluded that there was a positive association between processed meat consumption and risk of CRC, which was based on a meta-analysis of 11 case-control studies that found a RR of 1.36 (95% CI: 1.09, 1.69) with considerable heterogeneity (I² = 76%) and on a meta-analysis of 12 cohort studies that found a RR of 1.15 (95% CI: 1.07, 1.24) with low heterogeneity (I² = 27%; Table 6).

Cancer mortality

The results from O´Sulivan et al. were based on 3 cohort studies [33], and Wang et al. supplemented their systematic review with additional 2 cohort studies [13]; both reviews showed an association between processed meat intake and cancer mortality (O´Sulivan et al.: RR: 1.13, 95% CI: 1.09, 1.17; Wang et al.: RR: 1.08, 95% CI: 1.06, 1.11). Moreover, neither of the meta-analyses showed significant heterogeneity in the meta-analyses (Table 6).

Qualitative synthesis of included systematic reviews on diabetes

The result of the meta-analysis suggested an association (RR: 1.19; 95% CI: 1.11, 1.27), with significant heterogeneity (p<0.001; Table 7). The meta-analysis was conducted based on 7 cohort studies.

Qualitative synthesis of included systematic reviews on coronary heart disease

According to the results of the meta-analyses, increased processed meat intake was associated with risk of CHD (RR: 1.42; 95% CI: 1.07, 1.89 (heterogeneity: p = 0.04)) [15]. The results were based on 1 case-control study and 4 cohort studies (Table 3).

Qualitative synthesis of included systematic reviews on stroke

In regard to stroke, Kim et al. reported an association (RR of 1.17 (95% CI: 1.08, 1.25) with low heterogeneity (I² = 0.0%) [43], however Micha et al. did not (RR of 1.14 (95% CI: 0.94, 1.39; heterogeneity not analyzed) [15] (Table 4). The results were based on 2 cohort studies for Micha et al. [15] and 5 cohort studies for Kim et al. [43], with none of the studies overlapping (S4 Table).

Results from post hoc quality assessment analyses

Its meta-analysis [45] was based on 3 cohort studies, with only one overlapping with the study of Micha et al. [15] and found that processed meat consumption increased the risk of stroke (RR: 1.17 1.09, 1.27)) [45].

Qualitative synthesis of included systematic reviews on CVD mortality

The results on CVD mortality (O´Sulivan et al. [33] showed an association: RR: 1.17; 95% CI: 1.02, 1.33); Wang et al. [13]: RR: 1.15; 95% CI: 1.07, 1.24). WhileThe total number of cohort studies included in the meta-analysis of O´Sulivan et al. [33] and Wang et al. [13] was 7, but only three studies overlapped (S4 Table), and the meta-analyses showed a large degree of heterogeneity (Table 7).

Discussion

In this overview, we assessed the quality alongside summarizing the results of published systematic reviews and meta-analyses that examined associations between processed meat consumption and the risk of multiple chronic diseases. We assessed the methodological quality of published systematic reviews and meta-analyses using the AMSTAR tool, and found a suboptimal quality of most previous reviews. Associations were more often found when reviews were based on results from case-control than when based on cohort studies, suggesting that the better the study design, the lower the probability of an association.

According to GRADE, the quality of evidence was very low for the individual outcomes. The rating was based on observational study design, in combination with serious risk of bias, and/or serious imprecision.

In the study selection process, we excluded 107 reviews that did not assess and document the scientific quality of the included studies (AMSTAR item number 7). In line with other authors [16,17], we did this to ensure that the included reviews were ‘systematic’. The AMSTAR score of these reviews were already short of two points, but to accommodate any concerns regarding inaccurate conclusions that may be drawn by not including all published reviews, we performed a post hoc quality assessment of the additional systematic review that was “a priori” not included because only one database was searched, but did assess quality of the primary studies (AMSTAR item number 3). As anticipated, the AMSTAR score for this single review was moderate. Whether searching one database captures most of the existing literature is debatable. Coverage by the most commonly used databases has earlier been shown to be high (>90%). However, recall estimates (defined as the percentage of relevant records retrieved divided by the total number of included studies in the individual systematic reviews), even for the best performing databases (EMBASE/MEDLINE), have been shown to be insufficient in retrieving references for systematic reviews (< 50%), when the databases were used alone [46]. In the following, we discuss in more detail the main results from the included reviews by disease outcomes, considering the designs of the primary studies as a quality indicator.

Cancers

The results of the included systematic reviews and meta-analyses suggested that overall a high intake of processed meat was related to an increased risk of esophageal cancer, nasopharyngeal carcinoma, gastric cancer, oral cavity, renal cell carcinoma, CRC, and cancer mortality. Except for gastric cancer and total cancer mortality, the evidence for an association between processed meat intake and the cancers, seemed to a large degree to be driven by results from case-control studies rather than cohort studies. The same was true for results on pancreatic cancer. As pointed out in several earlier studies, interpretation of results based on case-control studies only should be done cautiously, because of their well-known methodological limitations.

Cancer in the liver, brain (glioma), ovaries, lung and PCC did not seem to be associated with intake of processed meat. For the most part, these results were consistent across studies with different designs. Regarding Non-Hodgkin lymphoma, there were discrepancies in the results from the two included meta-analyses, which may be explained by the fact that the primary studies differed between the two meta-analyses. Also, the increased risk of Non-Hodgkin lymphoma with high processed meat intake seemed to be driven by the results from the case-control studies. The two meta-analyses that examined processed meat intake and risk of Non-Hodgkin lymphoma included the same cohort studies. Considering shortcomings of case-control studies, the conclusion based on the meta-analysis by Solimini et al. [36] suggesting no association between processed meat intake and Hodgkin lymphoma may be most valid.

Diabetes and cardiovascular disease

An association was observed between a high intake of processed meat and risk of T2D, along with CVD incidence and mortality. For processed meat intake and risk of stroke, one study did not find an association [15], while the other did [43]. Most of the primary studies on processed meat intake and risk of diabetes and cardiovascular diseases that were in the meta-analyses were based on cohort studies, albeit limited to few (diabetes: k = 7; CHD: k = 5; stroke: k = 2 and k = 5; CVD mortality: k = 4 and k = 6).

Strengths and limitations

A strength of the present overview lays in the systematic quality assessment of multiple reviews examining the influence of processed meat on three main common chronic diseases. However, there are inherited limitations when conducting overviews, umbrella reviews or ‘reviews of reviews’. The conclusion on the outcomes here relies on the methodological quality of primary study selection, specific eligibility criteria, and adequacy of the reporting in systematic reviews. Thus, even well-conducted systematic reviews may have relied on an evidence base that was poor or limited from the primary studies included. Moreover, relevant important results from primary studies may be lacking in this present synthesis, due to the stringent study selection criteria of the included systematic reviews, or because important primary studies may have been published after the search date in the systematic reviews [47].

Following the guidelines of the AMSTAR quality assessment tool in the present study, we registered our protocol prior to the search; further, the study selection and quality assessment were performed independently by two researchers. We performed a comprehensive literature search, however we only selected published systematic reviews. Since published reviews are systemically different from unpublished reviews, issues regarding publication bias should also be kept in mind, especially because of the inability to capture these quantitatively for systematic reviews.

Limitations of the quality assessment tool AMSTAR

A number of limitations of the AMSTAR tool have previously been emphasized [20,48]. First, the AMSTAR version we used did not specifically address systematic reviews of non-randomized studies. Such a tool (AMSTAR 2, which was recently released) was being developed at the time this work was conducted. Secondly, the AMSTAR relies on what information is available and reported in the systematic review, which makes evaluation of especially older systematic reviews prone to a low score, i.e. the methodological quality may be underestimated. We did not contact the review authors in attempt to avoid false-negative results. Other tools, such as ROBIS, may be considered more applicable for this purpose [49], although many of the signaling questions between AMSTAR, AMASTAR 2 and ROBIS overlap. The reasons for selecting AMSTAR were that AMSTAR is valid, reliable, easy to use, with high inter-rater agreement and a widely used instrument [21], also for research on diet and health [50–52]. Indeed, previous studies show that inter-rater agreement in AMSTAR is higher compared to ROBIS (AMSTAR > 80% versus ROBIS ≈ 60%) [53]. Even though a formal comparison of results between the two tools of risk of bias may be of interest for future research; systematic reviews in other research fields have already shown good consistencies between AMSTAR and ROBIS, i.e. reviews that showed low quality on AMSTAR also tended to demonstrate high risk of bias on ROBIS [53,54].

Conclusions

Many previous reviews report adverse associations between a high processed meat intake and risk of various cancers, T2D and CVD, but most were of moderate methodological quality, where evidence for associations were more often found when reviews were based on results from case-control than when based on cohort studies, suggesting that the better the study design, the lower the probability of an association. Moreover, the overall certainty in the evidence was very low across all individual outcomes, due to serious risk of bias and imprecision.

A systematic quality assessment of each of the primary studies in a review should be performed in future systematic reviews prior to formulating a concrete conclusion of the evidence.

Supporting information

S1 Table [docx]
Search strategy in MEDLINE via PubMed and EMBASE via Ovid.

S2 Table [docx]
Excluded studies after full-text assessment.

S3 Table [docx]
Quality assessment using AMSTAR.

S4 Table [docx]
Study matrix.

S5 Table [docx]
Subgroup analyses relevant to this SR.

S6 Table [docx]
A narrative AMSTAR evaluation per outcome.

S7 Table [doc]
PRISMA 2009 checklist.