pyam
an open-source Python package
for IAM scenario analysis¶
Daniel Huppmann, IIASA, huppmann@iiasa.ac.at¶
The presentation is available at
iiasa.github.io/ene-present.github.io/cdlinks-delhi2018¶
The pyam package is available at github.com/IAMconsortium/pyam¶
The package was developed by Matthew Gidden and Daniel Huppmann.
It is released under an APACHE 2.0 Open-Source license.
The presentation is based on a talk by Matthew Gidden given at IAMC 2017, Recife, Brazil and the tutorial notebooks of the pyam package.
This presentation is licensed under
a Creative Commons Attribution 4.0 International License.
Diagnostics, analysis and visualization tools
for Integrated Assessment timeseries data¶
First steps with the pyam package¶
The pyam package provides a range of diagnostic tools and functions
for analyzing and working with IAMC-style timeseries data.
The package can be used with data that follows the data template convention of the Integrated Assessment Modeling Consortium (IAMC). An illustrative example is shown below; see data.ene.iiasa.ac.at/database for more information.
| model | scenario | region | variable | unit | 2005 | 2010 | 2015 |
|---|---|---|---|---|---|---|---|
| MESSAGE V.4 | AMPERE3-Base | World | Primary Energy | EJ/y | 454.5 | 479.6 | ... |
| ... | ... | ... | ... | ... | ... | ... | ... |
Source of tutorial data¶
The timeseries data used in this tutorial is a partial snapshot of the scenario database compiled for the IPCC's Fifth Assessment Report (AR5):
Krey V., O. Masera, G. Blanford, T. Bruckner, R. Cooke, K. Fisher-Vanden, H. Haberl, E. Hertwich, E. Kriegler, D. Mueller, S. Paltsev, L. Price, S. Schlömer, D. Ürge-Vorsatz, D. van Vuuren, and T. Zwickel, 2014: Annex II: Metrics & Methodology.
In: Climate Change 2014: Mitigation of Climate Change. Contribution of Working Group III to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change [Edenhofer, O., R. Pichs-Madruga, Y. Sokona, E. Farahani, S. Kadner, K. Seyboth, A. Adler, I. Baum, S. Brunner, P. Eickemeier, B. Kriemann, J. Savolainen, S. Schlömer, C. von Stechow, T. Zwickel and J.C. Minx (eds.)]. Cambridge University Press, Cambridge, United Kingdom and New York, NY, USA. Link
The complete AR5 scenario database is publicly available at tntcat.iiasa.ac.at/AR5DB/.
Scientific references for selected tutorial data¶
The data snapshot used for this tutorial consists of selected data from two model intercomparison projects:
- Energy Modeling Forum Round 27 (EMF27), see the Special Issue in Climatic Change 3-4, 2014.
- EU FP7 project AMPERE,
see the following scientific publications:
- Riahi, K., et al. (2015). "Locked into Copenhagen pledges — Implications of short-term emission targets
for the cost and feasibility of long-term climate goals."
Technological Forecasting and Social Change 90(Part A): 8-23.
DOI: 10.1016/j.techfore.2013.09.016 - Kriegler, E., et al. (2015). "Making or breaking climate targets: The AMPERE study on
staged accession scenarios for climate policy."
Technological Forecasting and Social Change 90(Part A): 24-44.
DOI: 10.1016/j.techfore.2013.09.021
- Riahi, K., et al. (2015). "Locked into Copenhagen pledges — Implications of short-term emission targets
for the cost and feasibility of long-term climate goals."
Technological Forecasting and Social Change 90(Part A): 8-23.
The data used in this tutorial is ONLY a partial snapshot
of the IPCC AR5 scenario database!
This tutorial is only intended for an illustration of the pyam package.
Import package and load data from the AR5 tutorial csv snapshot file¶
First, we import the pyam package and load the timeseries data snapshot from the file tutorial_AR5_data.csv in the pyam/tutorial folder.
%matplotlib inline
import pyam
data = '../../pyam/tutorial/tutorial_AR5_data.csv'
df = pyam.IamDataFrame(data=data)
What's in our dataset?¶
As a first step, we use a number of functions to find out what is included in the snapshot.
df.models()
df.scenarios()
df.regions()
df.variables(include_units=True)
A first look at the data¶
We use the temperature outcome as the first variable of interest in our data snapshot.
v = 'Temperature|Global Mean|MAGICC6|MED'
df.filter({'region': 'World', 'variable': v}).line_plot(legend=False)
Categorization of scenarios¶
We use the temperature outcome as a first criteria for categorization of scenarios.
The function categorize() assigns all scenarios fulfilling a number of criteria to a specific category.
The function metadata() applies a categorization to all scenarios.
df.metadata(meta='uncategorized', name='temperature')
df.categorize(
'temperature', 'Below 1.6C',
criteria={v: {'up': 1.6, 'year': 2100}},
color='cornflowerblue'
)
df.categorize(
'temperature', 'Below 2.0C',
criteria={v: {'up': 2.0, 'lo': 1.6, 'year': 2100}},
color='forestgreen'
)
df.categorize(
'temperature', 'Below 2.5C',
criteria={v: {'up': 2.5, 'lo': 2.0, 'year': 2100}},
color='gold'
)
df.categorize(
'temperature', 'Below 3.5C',
criteria={v: {'up': 3.5, 'lo': 2.5, 'year': 2100}},
color='firebrick'
)
df.categorize(
'temperature', 'Above 3.5C',
criteria={v: {'lo': 3.5, 'year': 2100}},
color='magenta'
)
Checking for uncategorized scenarios¶
df.filter({'temperature': 'uncategorized'})[['model', 'scenario']]\
.drop_duplicates()
The pyam package includes the function require_variable() to check a-priori whether a variable exists. The option exclude: True marks these scenarios as "exclude" in the metadata, so that they can be easily removed from further analysis.
df.require_variable(variable=v, exclude=True)
Plotting the temperature outcome again using the categorization¶
We repeat the plot, this time excluding the uncategorized scenarios and using the 'temperature' metadata column to assign colors. The colors of the individual categories were defined in the function categorize() above.
df.filter({'variable': v, 'exclude': False})\
.line_plot(color='temperature')
Using the categorization to analyse other variables¶
We now plot the timeseries data of the 'Primary Energy' variable, using the color-coding of the Temperature categorization to analyse the correlation between energy consumption and warming.
df.filter({'variable': 'Primary Energy', 'exclude': False})\
.line_plot(color='temperature')
Filtering scenarios by Primary Energy in the base year¶
To get clearer understanding of the relationship between Primary Energy and Warming, we focus on only those scenarios that have similar levels of Primary Energy in the base year (2010).
We first use the function validate() to check that certain values are within a given range,
df.validate(criteria={'Primary Energy': {'lo': 400, 'year': 2010}}).head()
Assigning valid scenarios to a new category¶
We assign those scenarios that have a Primary Energy level above 400 EJ/y in 2010 to a new category, and then re-display the previous figure including only these scenarios.
df.metadata(meta='uncategorized', name='PE')
df.categorize(name='PE', value='high',
criteria={'Primary Energy': {'lo': 400, 'year': 2010}})
df.filter(
{'variable': 'Primary Energy', 'exclude': False, 'PE': 'high'})\
.line_plot(color='temperature')
Highlighting particular models and scenarios¶
Next, we want to check how one particular model behaves within an ensemble of scenarios.
df.metadata(meta='uncategorized', name='model_family')
pyam.categorize(
df, filters={'model': 'MESSAGE*'}, name='model_family', value='MESSAGE',
criteria={'Primary Energy': {'lo': 400, 'year': 2010}},
marker='o')
from pyam.plotting import run_control
rc = run_control()
rc.update({'marker': {'model_family': {'uncategorized': None}}})
df.filter(
{'variable': 'Primary Energy', 'exclude': False, 'PE': 'high'})\
.line_plot(color='temperature', marker='model_family')
And just for the fun of it, let's add scenario linestyles, too...¶
df.filter(
{'variable': 'Primary Energy', 'exclude': False, 'PE': 'high'})\
.line_plot(color='temperature', marker='model_family',
linestyle='scenario', legend=True)
Further analysis using metadata¶
Rather than plotting the development over time, it is often useful to extract and visualize key indicators. In this example, we determine the year of peak warming and plot this indicator against the cumulative CO2 emissions from 2010 until that year.
def peak_warming(x, peak_year=False):
peak = x[x == x.max()]
if peak_year:
return peak.index[0]
else:
return float(max(peak))
mean_temperature = df.filter(filters={'variable': v}).timeseries()
df.metadata(
mean_temperature.apply(peak_warming, raw=False, axis=1),
'median warming at peak')
df.metadata(
mean_temperature.apply(peak_warming, peak_year=True, raw=False, axis=1),
'year of peak warming')
co2 = df.filter({'region': 'World', 'variable': 'Emissions|CO2'})\
.timeseries() / 1000
df.metadata(
co2.apply(lambda x:
pyam.cumulative(x, first_year=2010,
last_year=df.meta.loc[x.name[0:2],
'year of peak warming']),
raw=False, axis=1),
'cumulative CO2 emissions (2010 to peak warming)')
df.filter({'exclude': False}).\
scatter(x='cumulative CO2 emissions (2010 to peak warming)',
y='median warming at peak')
df.filter({'exclude': False}).\
scatter(x='cumulative CO2 emissions (2010 to peak warming)',
y='median warming at peak',
color='temperature')
df.filter({'exclude': False}).\
scatter(x='cumulative CO2 emissions (2010 to peak warming)',
y='median warming at peak',
color='temperature', marker='model_family')
We had previously defined the marker for the scenarios not categorized by model family as None, so no marker is shown in the previous scatterplot.
We can easily reset that marker as illustrated below.
rc.update({'marker': {'model_family': {'uncategorized': '*'}}})
df.filter({'exclude': False}).\
scatter(x='cumulative CO2 emissions (2010 to peak warming)',
y='median warming at peak',
color='temperature', marker='model_family')
Looking at regional disaggregation for a particular scenario¶
We use the 'EMF27-550-LimBio' scenario from the MESSAGE model to more closely look at the regional break
df_ = df.filter({'model': 'MESSAGE*', 'scenario': 'EMF27-550-LimBio',
'variable': 'Emissions|CO2'})
df_.filter({'region': 'World'}, keep=False).bar_plot(bars='region')
df_.filter({'region': 'World'}, keep=False).\
filter({'year': 2010}).pie_plot(category='region')
df_.filter({'region': 'World'}, keep=False).\
filter({'year': 2050}).pie_plot(category='region')
df_.filter({'region': 'World'}, keep=False)\
.filter({'year': 2100}).pie_plot(category='region')
And finally - who doesn't like maps?¶
This feature is work in progress. The following figure is based on the unit test and the CEDS Harmonization work by Matt Gidden.
import matplotlib.pyplot as plt
import cartopy
fig, ax = plt.subplots(
subplot_kw={'projection': cartopy.crs.PlateCarree()}, figsize=(10, 7))
df.map_regions('iso').region_plot(ax=ax, cbar=False)
