|
Ecolabelling of printed matter - part II 5 Critical reviewA critical review by the external expert Kim Christiansen, 2.-0 LCA Consultants (kc@lca-net.com) has been conducted. The review report is shown below with the authors’ comments in italics. The review has focused on the correctness and completeness of the summary and conclusions versus the body text of the report as well as the overall quality of the methods and data applied, especially the use of average and marginal data. The critical review statement is structured according to the requirements of the ISO 14040-series; although detailed comments are, placed under section 5.1.4. 5.1 Result of the critical reviewOverall a well-presented study structured according to the phases of an LCA study. The assumptions and underlying calculations are presented in reasonable detail, and the limitations well discussed. This gives the impression that the results are robust, or at least, as robust as possible given the limitations at hand. The LCA study is intended to be used for criteria and methodology development within ecolabelling of printed matter (Preface, page 5) i.e. the results will be disclosed to the public. According to ISO 14040 therefore a panel of interested parties shall conduct the critical review. The status of the report for the present critical review by an external expert is therefore assessed to be for internal use only. It is recommended to involve the steering group formally in the critical review process in the finalization of the study report. The members of the steering committee representing interested parties have had the drafted final report for review. Furthermore, comments, inputs etc. from the steering committee have currently been included in the study (see Preface). The report does not make reference to the ISO 14040-series. Therefore, formally, the requirements of the ISO 14040-series do not apply. Nevertheless, the study and the report are clearly structured according to the framework and guidelines of the ISO standards, and the EDIP methodology applied in the study (page 19) is also announced as fulfilling the requirements of the ISO 14040-series. Similarly, at better introduction to attributional and consequential LCA could be included, as both average and marginal data are used in the study (which will impact the conclusions on e.g. scenario 2). A reference to the ISO 14040-series has been added (see Introduction). A better introduction to attributional (“average”) and consequential (“marginal”) LCA is not included. The use of average data in the reference scenario for the processes (technology, consumptions and emissions) at the model printing company is deliberately chosen because we want to reflect an average situation. For most of these processes or factors the printing company has at least to some degree a choice, i.e. these factors are steerable. For consumption of electricity there is no choice and the use of marginal data is therefore more relevant. Besides we want to avoid the site-specific variations in potential impact from electricity production in this LCA study. 5.1.1 MethodologyFocusing on the manufacturing (production) of the final printed matter can be justified, but it is not per se a requirement for ecolabelling; actually, environmental impacts from e.g. paper production within the impact categories of toxicity and eco-toxicity could be of higher importance than found, if a similar detailed approach was applied. A similar detailed approach for upstream processes has not been possible to use because of lack of data and the resource scoping of the study. However, this issue is discussed extensively in the report and the conclusion based on existing knowledge is that inclusion of upstream emissions to a higher degree probably would not change the overall result of this study, see for example Section 3.2 (chemicals in general) and Section 4.1.3.1 (AOX). Furthermore, besides AOX, the other group of potentially most significant contributing emissions from paper production, i.e. emissions of metals to water, is actually included in the life cycle impact assessment:
A weak point is that the framing of the problem could have been stronger, particularly with defining a meaningful reference scenario, and in structuring the subsequent scenario inventories and the sensitivity analysis. Both the use of scenarios and the "sensitivity analysis" included in the study are approaches to test the robustness of the scoping of the study; the procedure outlined in ISO 14043 could be used to make this more clear. Choice of scenarios should be based on both what is thought relevant based on earlier studies and on iteration with the inventory analysis and the impact assessment; it is not very clear if this is the case in the present study. The choice of the scenarios is actually based on what is thought to be relevant based on earlier studies (e.g. paper, scenario 2 and scenario 3) and on what came up in the first, second and third iteration (e.g. biocides, scenario 6 and ink consumption, scenario 4). The choice of the scenarios based on the iterations was discussed within the steering committee. The purpose of the scenarios in this study is to indicate the importance of a certain parameter (e.g. ink consumption) for the impact profile of printed matter when this parameter is varied within a range representing what can be observed within the printing industry today (e.g. 1.8 kg ink/fu – 26.5 kg ink/fu). This is very important information when dealing with ecolabelling criteria based on a life cycle perspective. The main purpose of the sensitivity analysis in this study is to analyse the robustness of the reference scenario (main scenario – based on average values for consumptions and emissions) by for example using alternative normalisation references/weighting factors and alternative allocation principles for paper. However the variation in parameters (e.g. type of cleaning agent used) is also covered in the sensitivity analysis. So some of the results from the sensitivity analysis can both be used for the robustness evaluation of the reference scenario and when finding out which weight to put on a certain parameter when developing ecolabelling criteria on printed matter. Furthermore, the choices made in defining the reference scenario are somewhat inconsistent, notably that the marginal technology is used for electricity and paper production, but not in the printing technology chosen, and that transport is only partly included. The use of an average printing process should be clearly placed with respect to best/worst performance and the marginal technology. Intending to apply the results for ecolabelling but excluding new technologies within e.g. the printing company is not justified as market shares are increasing (page 33). The consequences of using average energy data for electricity production is shown in scenario 1. Transport is only partly and implicitly included but the importance of transport is assessed on the basis of several former studies, see Section 4.1.4.1. As pointed out above, the use of average data in the reference scenario for the processes at the model printing company is deliberately chosen because we want to reflect an average situation. The results of this study are to be used as a basis for ecolabelling in Scandinavia and Europe using average technology as a starting point. The effect of using BAT (Best Available Technology) is at least to some degree reflected in the scenarios run by varying parameters within observed ranges (e.g. paper spillage, scenario 3) but focusing explicitly on BAT technology is not within the scope of this study although the framework used here could easily be used in such a study. The choice of impact assessment categories (not "criteria", page 19), time scope and use of marginal versus average data (not "approaches") is not very well justified (explain why and how). "Waste water treatment" encompasses a major variation in technology and performance; using geographic arguments for including or not including this process is not justifiable. The scope definition of choosing a 1 year life time for the product examples depicted is not justified (page 30 and 32). In fact, the reference scenario is not 1 scenario (modeled product system) but actually several e.g. with or without WWT, with or without avoided energy consumption and avoided emissions from incineration and recycling of paper etc. The choice of assessment criteria is justified, see Section 1.1.2.2. The choice of marginal versus average data is commented above. The inclusion of generic wastewater treatment (WWT) with a biological step is justified in Section 2.4.5. However including the variation in technology and performance of WWT is not within the scope of this study. The choice of one year as product life time is assessed to best represent the average of the different printed matter types by sheet fed offset (Bøg, 2003) but is also chosen for the sake of normalization of the LCA results, see Section 1.1.2.1.The reference scenario is actually one well defined scenario (see Section 2.4) for example it is without WWT, and avoided energy consumption and avoided emissions from incineration and recycling of paper is allocated to paper. However, results are shown in different variations to inform the reader, and whenever variations are included it is pointed out, for example by marking paper with gross instead of net (reference scenario) indicating that avoided energy consumption and avoided emissions from incineration and recycling of paper are not allocated to paper in the actual situation. The choice of energy scenario is shown to be very important, so the choice of gas for marginal technology may need to be more robustly defended. Also, scenario 7, "no waste water emitted" is not meaningful; if no waste water is discharged, there will be sludge from the internal WWTP, but the sludge disposal process is not included. The choice of natural gas for marginal technology is documented in Frees et al. (2004). As pointed out several times in the report (e.g. Section 4.1.4.2. and Section 4.2) the treatment processes of chemical waste have not been possible to include in this study due to data lack and resource scoping. Furthermore, it is pointed out (see Section 4.2) that the inclusion of treatment of chemical waste will increase the liability of this LCA study. However for waste water it is preliminarily assessed that treating the waste water as chemical waste under controlled conditions will give rise to a significantly lower potential impact than emitting the waste water directly to the water recipient. The normalisation factors used in the study increase the importance of toxicity impacts relative to the other impacts considered (and they are also given a higher weight than the other impacts when calculating the overall load). Thus there is a danger that the major conclusion of the study (that the printing process is as important, or more important, than the paper input) may be somewhat due to the methods chosen (or at least that the methods and conclusions are self reinforcing). In exploring the sensitivity of the system to normalisation and weighting, it is recommended to use different weighting (value) methods and to show the consequences for the results of these value choices (e.g. Ecoindicator99). The effect of using other normalisation references and weighting factors for the EDIP method is tested by introducing updated factors and factors covering Europe, see Section 4.1.1. The effect of introducing European factors instead of Danish ones does not significantly change the overall result of the study. Furthermore, a scenario where the chemical related impact categories (ecotoxicity and human toxicity) are excluded is compared to previous studies on offset printed matter based on both EDIP, Ecoindicator95, CML and CPM, in which chemical emissions are only included to a limited degree or not at all. The result of this comparison shows that if the chemical related impact categories are excluded from the reference scenario, the importance of paper is increased to the same level as shown in the previous studies (about 70%), see Section 4.1.3.1. Due to resource scoping of the study, inclusion of other impact assessment methods has not been possible. Besides, using Ecoindicator99 in this chemical dominated LCA study would most probably be insufficient because only a limited number of characterisation factors are available for ecotoxicity (40 - 50) and sufficient data for calculating new ones does not exist in most cases. Furthermore, the way to do the calculations is not fully transparently described and the underlying models are not available to the LCA practitioner. The interpretation phase of an LCA is used to make the conclusions, and the sensitivity etc. of the conclusions is tested. The overall uncertainty and sensitivity of e.g. the inventory analysis and the impact assessment should be conducted as part of these phases. However, the rational for the choice of data sources and the use of data quality indicators is not included in the study; and this should be justified. The data quality indicators are not described separately in this study but the quality of the data which is used is indicated in the presentation of the data, for example the composition of the raw materials presented in Section 2 can be either based on newly updated data sheets from producers, personal communication with producers or secondary sources like literature published 10 years ago. The sensitivity analyses based on the different scenario variations to the basic scenario could be structured in a more systematic way. In particular, it would be valuable to know what the data ranges presented in Table 12 represent, e.g. best technology, worst technology, and where the marginal technology lies in the range. This is especially important for those variables found to change the results to such a very significant extent (e.g. ink consumption, alternative biocide agent). The data in Table 12 is expected to represent ranges covering from worst technology (worst technique/bad house keeping) to best technology (state of the art) but this is not specified in the main part of the documentation. The marginal technology is not known in all cases and defining it has not been within the scope of this study. 5.1.2 Data"Readily available data" from different sources are used and as stated with different functional units (page 19). This is a serious flaw in the data quality. The ranges depicted on the original data, could be interpreted as making the data non-applicable in the development of ecolabelling criteria. Different functional units are not used – all data are normalized to the functional unit: 1 ton sheet fed offset printed matter. The ranges within the data are reflecting the actual situation in the printing industry from about 1990 to 2002. Using the average for the reference scenario and the ranges in the alternative scenarios/sensitivity analysis actually makes these data very valuable for developing ecolabelling criteria on printed matter. Data in e.g. Seedorf et al. (1993) and Boethling (1984) are out of the time scope of the study. This is not discussed or justified. Seedorf et al. (1993) is not used alone but only for confirmation of data from 2000, 2001 and 2002 from producers of film developer and film fixer, see Section 2.1.2 and 2.1.3. Boethling (1984) is used for documentation of biodegradation of benzalkonium chloride in wastewater treatment plants with a biological step, as this reference is the most relevant for this type of substance. Data sources and actual primary data of the inventory data (categories) are listed in the appendices but the "raw" spreadsheet format, the mixed use of English and Danish terms and the limited introduction to each annex makes it difficult to benefit from the inclusion of the data. Annex B has been restructured and Danish terms in Annex C has been translated to English. The inventory data should be presented disaggregated according to the major processes and life cycle stages in the report – including the variation of the data and other data quality indicators - and not just presented aggregated in Annex C. Table 17 does not show, which stages are most important as referenced in the summary (page 22). In order to restrict the size of the report it has been chosen only to present the aggregated results for the LCA in Annex C. The relevant disaggregated results are described mainly in the text of the report. The most important stages related to Table 17 are extensively described in the text in Section 3. Table 12 includes the ranges but these are not propagated into the inventory of the processes. Also, the range of data depicted in Annex B is not used as uncertainty indications (ranges) in Annex C. The most important ranges are used in the different scenarios and sensitivity analysis. The ranges depicted in Table 12 (last column from the left) are based on the data in Annex B. The precision of the data is over-reported e.g. the percentage of ink disposed of as chemical waste – 19.6% if a very exact figure based on a range of 2.4-45.9% and the calculation is not included (page 34). What is the range (lowest reported value and highest reported value or a normal-distribution or...)? This comment is relevant for many other data included e.g. scenario 3: Spillage of exactly 32.1% versus 3.3% (page 35). The reporting of data precision in the report has been revised in some cases. The data used for calculating the average and the range shown in Table 12 (last column from the left) and for example the ink disposed as chemical waste is shown in Annex B. Also, the mere size of figures 2, 3, 4, 5, and 6 makes them difficult to read; a table with the data should be included showing the impact contributions of the specific processes or life cycle stages included in the study. This comment is also relevant for many other figures of the report, which are not easy or impossible to read; tables should be included e.g. in annexes with the specific data of the impact assessment results. The exact data on which the shown figures are based are to a very high degree given in the text of the report. Figure 10 seems to be the most correct figure for presenting the aggregated and weighted results on a process level. The conclusion of the summary should be based on this. This is not done because the starting point for the development of ecolabelling criteria is the processes occurring at the printing company, and only paper and energy use at the printing company are shown separately because special focus is already paid to these. However a differentiation of processes into upstream and downstream activities is discussed and shown graphically (e.g. Figure 10) in the report. The use of electricity and (district) heating in the printing process presented at page 66 should be in a table with the variations included, which will improve readability. The data are already given in the text. The presentation of the data sources (or inventory data sources) in Annex A should include not only the reference but also the representativeness etc. – the data quality indicators – of each source. Some sources are identified by author(s) other by name of consulting firm and other by title of publication; this inconsistency should be remedied. Also, a few of the referenced primary data sources lack the year of the publication e.g. 1.2.4 Ammonia, or the primary data source is not referenced e.g. hot melt glue. Sometimes abbreviations are used which are not likely to be known by the reader e.g. COWI-NBE. Annex A has been revised where reasonably possible within the resource scoping of this study. Data for recycling of paper can be found in "Returpapirstatistikken" – reading a graph in a publication for the general public from the Danish EPA is not convincing. The calculation of the recovery rate should be included in the report, not in an annex listing data sources. This is also the case for the energy recovery assumption – and who is assuming? The primary data source for the recycling of paper has been included. A very small detail, but: "dk-Teknik" should be dk-TEKNIK ENERGY & ENVIRONMENT (now FORCE TECHNOLOGY...); MGR is Morten Grinderslev. Has been erased 5.1.3 InterpretationThe most significant finding of the study (i.e. the large contribution by the printing process to the overall environmental load when toxicity is taken into effect) might be an artefact of having a more detailed inventory list for printing than the other processes in the life cycle (e.g. paper) - a more detailed inventory with more substances contributing to the overall environmental load will show an increased load relative to processes with a less complete inventory. Also, paper is still the most important contributor to the weighted overall environmental impact – although there is no discussion on the significance of the difference between 48% for paper (production!) and 41% for printing (process). This changes if the recycling of paper and heat recovery is taken into consideration (31% and 41%, respectively), which would a more correct result to report, but the most correct would be with the upstream ink-production as a separate process – as in figure 10 – i.e. this figure should be presented first and then the others as different types of aggregation of this. These issues are already commented above. However, the fact that paper gross has an importance of 48% is shown in Figure 8, and alternative allocations for paper production are discussed in Section 4.1.2. The credits of recycling, i.e. avoiding production of virgin paper and of heat recovery, are assigned to the "paper" processes which for the comparison of paper production and recycling with printing is justified, although assignment of the credit of recycling paper to the user of the printed matter delivering it to recycling would be more methodologically correct. Assigning the credit for recycling of paper to the user would be meaningless in this context. Adding to this are the inconsistencies in the inventory data, e.g. COD, VOC do not have equivalency factors in the method used, and so are not included in the impact assessment, which adds to the uncertainty of the result having only 33% and 26% of the substances emitted to air and 25% and 37% of the substances discharged to water characterized to toxicity and ecotoxicity, respectively (page 21); the adjustment of these percentages by excluding substances not contributing is valuable and could be included as a more correct expression of the coverage, but it is still only around 50%. (Both calculations are based on quantities, I assume?). VOC actually has characterisation factors (equivalency factors) but only for photochemical ozone formation. COD is not contributing to any of the impact categories considered in the study but may contain substances which would be ecotoxic. Inclusion would require a specification of the substance content underlying the COD. As described in Section 3.2 the referred 33%, 26%, 25% and 37% are all based on numbers of substances - not the emitted quantity. The referred around 50% (actually 48% for human toxicity and 53% for ecotoxicity) is based on quantity as also described in Section 3.2. Coverage of 48% - 64% (based on quantities and excluding non contributing substances) for human toxicity and ecotoxicity, as is the case in this study, is probably at the level off what is achievable with the knowledge of today. Also, consequences of not having data on e.g. siccatives and softeners should be discussed – not just referenced as likely significant contributors, and the rough estimate on the importance of AOX emissions from paper production should be made available in the report – especially in view of the importance of acute and persistent ecotoxicity (page 22 and 24). The consequences of not having data on siccatives, softeners etc. and the importance of AOX emissions are actually discussed, see Section 3.3 and Section 4.1.3.1. Presentation of the impact assessment results by bar diagrams should be supplemented by tables with the figures and the variation (uncertainty) of the results included. The results depicted in e.g. figure 3 (page 60) adds little value to the interpretation – it's not easy to read, and no indications of the significance of the differences are included. Also the legend should indicate that the paper production, recycling and incineration are excluded, not just paper. It has been chosen as form of presentation not to include tables with the figures shown in the graphs but to describe and discuss the results in the text. That the paper net value is used in Figure 3 is described in the figure text. In the summary, references to the scenarios as well as the wording of the scenarios should be consequent (e.g. that waste paper is paper spillage in production, that WWTP is scenario 5, and that substituting the biocide is scenario 6). Also, the scenarios should be better presented in the summary (as this is already lengthy). Is corrected except for the degree to with the scenarios are presented in the Summary and conclusions which is maintained because it is assessed to be in balance with the degree to which other issues are represented. The lack of data on waste and the ability to differentiate between the impacts of different waste types is a drawback of the study results in itself, as stated for land filled paper waste and treatment of chemical waste (page 27), but furthermore the waste treatment processes should be included in an LCA study; this should be added as a comment in the summary (page 26 or 27). The sentence that “the waste treatment processes should be included in an LCA study” is included. The issue of the degree to with waste is included in the study and the consequences thereof are discussed and described in several sections of the report, e.g. Section 1.1.2.2, Section 1.1.2.7 and Section 4.1.4.2. The conclusion on the applicability of the data for offset printed matter in general seems to be robust based on the comparison with Drivsholm et al. (page 72f) within the scope of the study, but as indicated above, this scope can be questioned. See comments on scoping above. 5.1.4 ReportThe structuring of the report might be improved by using the outline of the ISO 14040-series more consequently. Also the hierarchy of the headings does not seem logical in size and form, but this might be due to reporting requirements of the Danish EPA: The conclusion of the summary should be in an abstract; the interpretation phase of the LCA results in the conclusion as such – or it is the conclusion. Having conclusion in both the summary and the interpretation is somewhat confusing and at least redundant, although present text is not exactly copied. The conclusion is included in the “Summary and conclusions” because it is a stand-alone section. The language is sometimes un-precise e.g. "generic" could be interpreted as covering all types of sheet fed offset printed matter and similarly, the term "fictitious" sheet fed printing company could be interpreted as not being based on real-world information and data and focusing only on the manufacturing of the printed matter. It is recommended to change the wording, including the title, e.g. "Life cycle assessment of model off-set sheet printed matter". Fictitious has been substituted by model Paper gross and paper net should be explained in the summary as well, e.g. by footnotes. Also, the different printing processes and printed matter products could be better presented; this would be especially valuable for the comparison with the study by Drivsholm et al. Paper gross and paper net is actually explained in the “Summary and conclusion”, see Section “Goal and scope definition”. References should be included in table and figure text (e.g. table 3 and 4). This would make the figure text two large in some cases and is not done. The references are described in the text refereeing to the figures. Legends are missing in Figure 1 (e.g. what is EU TGD?). Figure 1 is revised A few references are missing in the reference list (Eurostat, 2002; Tillmann et al., 1991; and those given as references within references e.g. von Däniken A and Chudacoff M, 1995; Franke et al., 1995; Boustead, 1993; Christensen, 1991). Use of indirect references omits the option of a preliminary judgment on the data quality by knowing the primary source. Use of references is revised. The "anonymous" references should be clarified (personal communication, plant data, in-house report). Further clarification not possible The headings of the appendices should be included in the list of contents (page 4). Is done Annex B includes data representing the processes of the model-manufacturing site, not the "inventory process" – the annex is not depicting an overview of the process of doing the inventory! Annex B is also not sufficiently documented. The two parts of the annex should be separated i.e. the model printing site and the data of the 11 real-world printing sites. Also the columns of the latter could include the age of the data and other data quality requirements – and be organised according to the introductory text of the annex! The few headings in Danish should be translated. Annex B has been revised where reasonably possible within the resource scoping of this study. In Annex C the use of kg and g is inconsistent. Also some names and units are missing for some of the entries (data categories) in the resources list. The use of names in Danish of e.g. the different waste types makes it difficult for a non-Danish reader to find the relevant data. Annex C has been revised where reasonably possible within the resource scoping of this study.
|