IODE and larray
IODE offers many ways to interact with larray Array objects.
Lets start with necessary imports and loading the sample data:
[1]:
import numpy as np
import larray as la
from iode import (SAMPLE_DATA_DIR, comments, equations, identities, lists, scalars,
tables, variables, Sample, NA)
[2]:
# ---- load equations, identities, scalars and variables ----
# Note: test binary and ASCII 'fun' files are located in the 'SAMPLE_DATA_DIR'
# directory of the 'iode' package
comments.load(f"{SAMPLE_DATA_DIR}/fun.cmt")
equations.load(f"{SAMPLE_DATA_DIR}/fun.eqs")
identities.load(f"{SAMPLE_DATA_DIR}/fun.idt")
lists.load(f"{SAMPLE_DATA_DIR}/fun.lst")
scalars.load(f"{SAMPLE_DATA_DIR}/fun.scl")
tables.load(f"{SAMPLE_DATA_DIR}/fun.tbl")
variables.load(f"{SAMPLE_DATA_DIR}/fun.var")
# ---- print the number of objects present in the above workspaces ----
len(comments), len(equations), len(identities), len(lists), len(scalars), len(tables), len(variables)
Loading C:\soft\miniconda3\envs\py312\Lib\site-packages\iode\tests\data/fun.cmt
317 objects loaded
Loading C:\soft\miniconda3\envs\py312\Lib\site-packages\iode\tests\data/fun.eqs
274 objects loaded
Loading C:\soft\miniconda3\envs\py312\Lib\site-packages\iode\tests\data/fun.idt
48 objects loaded
Loading C:\soft\miniconda3\envs\py312\Lib\site-packages\iode\tests\data/fun.lst
17 objects loaded
Loading C:\soft\miniconda3\envs\py312\Lib\site-packages\iode\tests\data/fun.scl
161 objects loaded
Loading C:\soft\miniconda3\envs\py312\Lib\site-packages\iode\tests\data/fun.tbl
46 objects loaded
Loading C:\soft\miniconda3\envs\py312\Lib\site-packages\iode\tests\data/fun.var
394 objects loaded
[2]:
(317, 274, 48, 17, 161, 46, 394)
Add one variable:
[3]:
data = list(range(variables.nb_periods))
data[0] = NA
data[-1] = np.nan
time_axis = la.Axis(name="time", labels=variables.periods_as_str)
array = la.Array(data, axes=time_axis)
variables["A4"] = array
variables["A4"]
[3]:
Workspace: Variables
nb variables: 1
filename: c:\soft\miniconda3\envs\py312\Lib\site-packages\iode\tests\data\fun.var
description: Modèle fun - Simulation 1
sample: 1960Y1:2015Y1
mode: LEVEL
name 1960Y1 1961Y1 1962Y1 1963Y1 1964Y1 1965Y1 ... 2009Y1 2010Y1 2011Y1 2012Y1 2013Y1 2014Y1 2015Y1
A4 na 1.00 2.00 3.00 4.00 5.00 ... 49.00 50.00 51.00 52.00 53.00 54.00 na
Update one variable.
Update all values of a variable:
[4]:
data = list(range(variables.nb_periods))
data[0] = NA
data[-1] = np.nan
time_axis = la.Axis(name="time", labels=variables.periods_as_str)
array = la.Array(data, axes=time_axis)
variables["AOUC"] = array
variables["AOUC"]
[4]:
Workspace: Variables
nb variables: 1
filename: c:\soft\miniconda3\envs\py312\Lib\site-packages\iode\tests\data\fun.var
description: Modèle fun - Simulation 1
sample: 1960Y1:2015Y1
mode: LEVEL
name 1960Y1 1961Y1 1962Y1 1963Y1 1964Y1 1965Y1 ... 2009Y1 2010Y1 2011Y1 2012Y1 2013Y1 2014Y1 2015Y1
AOUC na 1.00 2.00 3.00 4.00 5.00 ... 49.00 50.00 51.00 52.00 53.00 54.00 na
Set the values for range of (contiguous) periods:
[5]:
# variable[t:t+x] = la.Array(var_names, t:t+x)
data = [1.0, NA, 3.0, np.nan, 5.0]
periods = ["1991Y1", "1992Y1", "1993Y1", "1994Y1", "1995Y1"]
time_axis = la.Axis(name="time", labels=periods)
array = la.Array(data, axes=time_axis)
variables["AOUC", "1991Y1:1995Y1"] = array
variables["AOUC", "1991Y1:1995Y1"]
[5]:
Workspace: Variables
nb variables: 1
filename: c:\soft\miniconda3\envs\py312\Lib\site-packages\iode\tests\data\fun.var
description: Modèle fun - Simulation 1
sample: 1991Y1:1995Y1
mode: LEVEL
name 1991Y1 1992Y1 1993Y1 1994Y1 1995Y1
AOUC 1.00 na 3.00 na 5.00
Update multiple variables at once
[6]:
# using a larray Array
data = [[28.89, 31.90, 36.66, 42.13, 9.92],
[np.nan, -39.96, -42.88, -16.33, -41.16],
[1.023, np.nan, 1.046, np.nan, 1.064]]
periods = ["1991Y1", "1992Y1", "1993Y1", "1994Y1", "1995Y1"]
names_axis = la.Axis(name="names", labels=["ACAF", "ACAG", "AOUC"])
time_axis = la.Axis(name="time", labels=periods)
array = la.Array(data, axes=[names_axis, time_axis])
array
[6]:
names\time 1991Y1 1992Y1 1993Y1 1994Y1 1995Y1
ACAF 28.89 31.9 36.66 42.13 9.92
ACAG nan -39.96 -42.88 -16.33 -41.16
AOUC 1.023 nan 1.046 nan 1.064
[7]:
variables["ACAF, ACAG, AOUC", "1991Y1:1995Y1"] = array
variables["ACAF, ACAG, AOUC", "1991Y1:1995Y1"]
[7]:
Workspace: Variables
nb variables: 3
filename: c:\soft\miniconda3\envs\py312\Lib\site-packages\iode\tests\data\fun.var
description: Modèle fun - Simulation 1
sample: 1991Y1:1995Y1
mode: LEVEL
name 1991Y1 1992Y1 1993Y1 1994Y1 1995Y1
ACAF 28.89 31.90 36.66 42.13 9.92
ACAG na -39.96 -42.88 -16.33 -41.16
AOUC 1.02 na 1.05 na 1.06
Arithmetic Operations On Variables With larray Array objects
IODE variables can be used in arithmetic operations with larray Array objects.
Let’s first reload the variables database to start from a clean state. Then we will select a subset of variables for the examples below:
[8]:
# reload variables to start from a clean state
variables.load(f"{SAMPLE_DATA_DIR}/fun.var")
# select a subset of variables for the examples below
vars_subset = variables["A*", "1991Y1:1995Y1"]
vars_subset
Loading C:\soft\miniconda3\envs\py312\Lib\site-packages\iode\tests\data/fun.var
394 objects loaded
[8]:
Workspace: Variables
nb variables: 5
filename: c:\soft\miniconda3\envs\py312\Lib\site-packages\iode\tests\data\fun.var
description: Modèle fun - Simulation 1
sample: 1991Y1:1995Y1
mode: LEVEL
name 1991Y1 1992Y1 1993Y1 1994Y1 1995Y1
ACAF 26.24 30.16 34.66 8.16 -13.13
ACAG -30.93 -40.29 -43.16 -16.03 -41.85
AOUC 1.02 1.03 1.03 1.05 1.05
AOUC_ 0.96 0.97 0.98 0.99 1.00
AQC 1.06 1.11 1.15 1.16 1.16
[9]:
data = [2.0, 3.0, 4.0, 5.0, 6.0]
time_axis = la.Axis(name="time", labels=vars_subset.periods_as_str)
array = la.Array(data, axes=time_axis)
array
[9]:
time 1991Y1 1992Y1 1993Y1 1994Y1 1995Y1
2.0 3.0 4.0 5.0 6.0
[10]:
# subtract the values of an larray Array object
# from the values of a single variable
updated_ACAF = vars_subset["ACAF"] - array
updated_ACAF
[10]:
Workspace: Variables
nb variables: 1
filename: c:\soft\miniconda3\envs\py312\Lib\site-packages\iode\tests\data\fun.var
description: Modèle fun - Simulation 1
sample: 1991Y1:1995Y1
mode: LEVEL
name 1991Y1 1992Y1 1993Y1 1994Y1 1995Y1
ACAF 24.24 27.16 30.66 3.16 -19.13
[11]:
data = [[2.0], [3.0], [4.0], [5.0], [6.0]]
names_axis = la.Axis(name="names", labels=vars_subset.names)
time_axis = la.Axis(name="time", labels=["1995Y1"])
array = la.Array(data, axes=[names_axis, time_axis])
array
[11]:
names\time 1995Y1
ACAF 2.0
ACAG 3.0
AOUC 4.0
AOUC_ 5.0
AQC 6.0
[12]:
# subtract the values of an larray Array object from the values
# of a subset corresponding to a single period
vars_subset_1995Y1 = vars_subset[:, "1995Y1"] - array
vars_subset_1995Y1
[12]:
Workspace: Variables
nb variables: 5
filename: c:\soft\miniconda3\envs\py312\Lib\site-packages\iode\tests\data\fun.var
description: Modèle fun - Simulation 1
sample: 1995Y1:1995Y1
mode: LEVEL
name 1995Y1
ACAF -15.13
ACAG -44.85
AOUC -2.95
AOUC_ -4.00
AQC -4.84
[13]:
# define the larray Array object to be used in the example below
data = np.array([[1.0, 2.0, 3.0, 4.0, 5.0],
[6.0, 7.0, 8.0, 9.0, 10.0],
[11.0, 12.0, 13.0, 14.0, 15.0],
[16.0, 17.0, 18.0, 19.0, 20.0],
[21.0, 22.0, 23.0, 24.0, 25.0]])
names_axis = la.Axis(name="names", labels=vars_subset.names)
time_axis = la.Axis(name="time", labels=vars_subset.periods_as_str)
array = la.Array(data, axes=[names_axis, time_axis])
array
[13]:
names\time 1991Y1 1992Y1 1993Y1 1994Y1 1995Y1
ACAF 1.0 2.0 3.0 4.0 5.0
ACAG 6.0 7.0 8.0 9.0 10.0
AOUC 11.0 12.0 13.0 14.0 15.0
AOUC_ 16.0 17.0 18.0 19.0 20.0
AQC 21.0 22.0 23.0 24.0 25.0
[14]:
# subtract the values of na larray Array object
# from the values of a subset of variables
new_vars_subset = vars_subset - array
new_vars_subset
[14]:
Workspace: Variables
nb variables: 5
filename: c:\soft\miniconda3\envs\py312\Lib\site-packages\iode\tests\data\fun.var
description: Modèle fun - Simulation 1
sample: 1991Y1:1995Y1
mode: LEVEL
name 1991Y1 1992Y1 1993Y1 1994Y1 1995Y1
ACAF 25.24 28.16 31.66 4.16 -18.13
ACAG -36.93 -47.29 -51.16 -25.03 -51.85
AOUC -9.98 -10.97 -11.97 -12.95 -13.95
AOUC_ -15.04 -16.03 -17.02 -18.01 -19.00
AQC -19.94 -20.89 -21.85 -22.84 -23.84
Import/Export the Variables workspace from/to LArray Array
To import / export the content of the variables workspaces from/to a LArray Array object, use the from_array() and to_array() methods:
[15]:
# ---- to LArray Array ----
# See Variables.to_array
arr_vars = variables.to_array()
print(f"Variables as LArray Array:\n{arr_vars.info}")
# ---- from LArray array ----
# See Variables.from_array
variables.from_array(arr_vars)
Variables as LArray Array:
394 x 56
names [394]: 'ACAF' 'ACAG' 'AOUC' ... 'ZKFO' 'ZX' 'ZZF_'
time [56]: '1960Y1' '1961Y1' '1962Y1' ... '2013Y1' '2014Y1' '2015Y1'
dtype: float64
memory used: 172.38 Kb
It possible to export a subset of the IODE databases:
[16]:
# export a subset of the IODE Variables database as a LArray Array
vars_subset = variables["A*;*_", "2000Y1:2010Y1"]
vars_subset
[16]:
Workspace: Variables
nb variables: 33
filename: c:\soft\miniconda3\envs\py312\Lib\site-packages\iode\tests\data\fun.var
description: Modèle fun - Simulation 1
sample: 2000Y1:2010Y1
mode: LEVEL
name 2000Y1 2001Y1 2002Y1 2003Y1 2004Y1 2005Y1 2006Y1 2007Y1 2008Y1 2009Y1 2010Y1
ACAF 10.05 2.87 -0.93 -6.09 -14.58 -26.54 -28.99 -33.38 -38.41 -37.46 -37.83
ACAG -41.53 18.94 19.98 21.02 22.07 23.11 24.13 25.16 26.19 27.23 28.25
AOUC 1.12 1.14 1.16 1.17 1.17 1.18 1.20 1.22 1.26 1.29 1.31
AOUC_ 1.10 1.14 1.15 1.16 1.15 1.16 1.19 1.20 1.21 1.23 1.25
AQC 1.34 1.38 1.41 1.42 1.40 1.40 1.40 1.41 1.43 1.45 1.46
... ... ... ... ... ... ... ... ... ... ... ...
WCF_ 3716.45 3863.90 3999.57 4147.95 4242.58 4320.97 4463.72 4665.91 4916.65 5042.74 5170.60
WIND_ 1000.14 1035.22 1070.93 1102.91 1115.33 1124.07 1144.27 1178.12 1231.49 1268.86 1301.03
WNF_ 2334.76 2427.49 2512.87 2606.28 2665.82 2715.11 2804.93 2932.23 3089.99 3169.32 3249.75
YDH_ 7276.61 7635.91 7958.39 8331.07 8653.40 8950.54 9299.54 9685.61 10228.84 10630.74 10995.83
ZZF_ 0.69 0.69 0.69 0.69 0.69 0.69 0.69 0.69 0.69 0.69 0.69
[17]:
array = vars_subset.to_array()
array.shape
[17]:
(33, 11)
[18]:
array
[18]:
names\time 2000Y1 ... 2010Y1
ACAF 10.046610792200543 ... -37.82742883229439
ACAG -41.53478656734795 ... 28.253928978210485
AOUC 1.1162376230972206 ... 1.3071099004906368
AOUC_ 1.10195719812178 ... 1.2501643331956398
AQC 1.3386028553645442 ... 1.4628683697450802
GAP_ 4.510095736743436 ... 3.376524129115177
GOSH_ 1957.0895201725107 ... 2834.234071582971
PAF_ 1.1974908430779156 ... 1.343502315992764
PC_ 1.2347266809771256 ... 1.4948195858546014
PFI_ 1.073515977227475 ... 1.2922777989010075
PROIHP_ 1.1054128 ... 0.95035726
QAFF_ 5748.518880218095 ... 6973.598752540967
QAF_ 4785.221048752499 ... 5936.55814807025
QAI_ 963.2971444232843 ... 1036.991862780098
QAT_ 7184.569798738995 ... 8726.21701183414
QBBPPOT_ 7655.481802392836 ... 9605.212225913185
QC_ 4855.41327831261 ... 5889.766110465185
QQMAB_ 5180.218215045446 ... 6936.181730632514
QS_ -35.212024420768714 ... -35.212024420768714
TFPFHP_ 0.993773299183911 ... 1.0977439739214365
VAFF_ 6883.859391184664 ... 9369.040711615937
VAF_ 5730.319881574676 ... 7975.839742305225
VAI_ 1153.535694715543 ... 1393.1903508914916
VAT_ 8880.730664031325 ... 12525.5814075134
VC_ 5995.108321903193 ... 8804.137738026035
VS_ 16.860422365860263 ... 10.939949669199798
WBF_ 2693.7277691578774 ... 3747.687848793519
WBU_ 5074.19063621057 ... 7283.594651285259
WCF_ 3716.4475089520292 ... 5170.600010384268
WIND_ 1000.1445769794319 ... 1301.025126372868
WNF_ 2334.7636275081923 ... 3249.7517024908175
YDH_ 7276.607740221424 ... 10995.831392939246
ZZF_ 0.68840039 ... 0.68840039
For the variables workspace, the to_array() method have additional arguments. For instance, the periods_as_type argument allows to export the periods as a specific type. The periods_as_type argument can be one of the following: str, int or float. The default value is str:
[19]:
array = vars_subset.to_array(time_axis_name="years", periods_as_type=int)
array
[19]:
names\years 2000 ... 2010
ACAF 10.046610792200543 ... -37.82742883229439
ACAG -41.53478656734795 ... 28.253928978210485
AOUC 1.1162376230972206 ... 1.3071099004906368
AOUC_ 1.10195719812178 ... 1.2501643331956398
AQC 1.3386028553645442 ... 1.4628683697450802
GAP_ 4.510095736743436 ... 3.376524129115177
GOSH_ 1957.0895201725107 ... 2834.234071582971
PAF_ 1.1974908430779156 ... 1.343502315992764
PC_ 1.2347266809771256 ... 1.4948195858546014
PFI_ 1.073515977227475 ... 1.2922777989010075
PROIHP_ 1.1054128 ... 0.95035726
QAFF_ 5748.518880218095 ... 6973.598752540967
QAF_ 4785.221048752499 ... 5936.55814807025
QAI_ 963.2971444232843 ... 1036.991862780098
QAT_ 7184.569798738995 ... 8726.21701183414
QBBPPOT_ 7655.481802392836 ... 9605.212225913185
QC_ 4855.41327831261 ... 5889.766110465185
QQMAB_ 5180.218215045446 ... 6936.181730632514
QS_ -35.212024420768714 ... -35.212024420768714
TFPFHP_ 0.993773299183911 ... 1.0977439739214365
VAFF_ 6883.859391184664 ... 9369.040711615937
VAF_ 5730.319881574676 ... 7975.839742305225
VAI_ 1153.535694715543 ... 1393.1903508914916
VAT_ 8880.730664031325 ... 12525.5814075134
VC_ 5995.108321903193 ... 8804.137738026035
VS_ 16.860422365860263 ... 10.939949669199798
WBF_ 2693.7277691578774 ... 3747.687848793519
WBU_ 5074.19063621057 ... 7283.594651285259
WCF_ 3716.4475089520292 ... 5170.600010384268
WIND_ 1000.1445769794319 ... 1301.025126372868
WNF_ 2334.7636275081923 ... 3249.7517024908175
YDH_ 7276.607740221424 ... 10995.831392939246
ZZF_ 0.68840039 ... 0.68840039
It is also possible to import a subset of the IODE databases:
[22]:
# existing variables
names = variables.get_names("A*")
# new variables
names += ["AX"]
periods = ["1990Y1", "1991Y1", "1992Y1", "1993Y1", "1994Y1", "1995Y1"]
data = [[0.0, 1.0, 2.0, 3.0, 4.0, 5.0],
[6.0, 7.0, 8.0, 9.0, 10.0, 11.0],
[12.0, 13.0, 14.0, 15.0, 16.0, 17.0],
[18.0, 19.0, 20.0, 21.0, 22.0, 23.0],
[24.0, 25.0, 26.0, 27.0, 28.0, 29.0],
[30.0, 31.0, 32.0, 33.0, 34.0, 35.0]]
names_axis = la.Axis(name="names", labels=names)
time_axis = la.Axis(name="time", labels=periods)
array = la.Array(data, axes=[names_axis, time_axis])
array
[22]:
names\time 1990Y1 1991Y1 1992Y1 1993Y1 1994Y1 1995Y1
ACAF 0.0 1.0 2.0 3.0 4.0 5.0
ACAG 6.0 7.0 8.0 9.0 10.0 11.0
AOUC 12.0 13.0 14.0 15.0 16.0 17.0
AOUC_ 18.0 19.0 20.0 21.0 22.0 23.0
AQC 24.0 25.0 26.0 27.0 28.0 29.0
AX 30.0 31.0 32.0 33.0 34.0 35.0
[ ]:
# note that the new variable 'AX' has been added with NA values
# for the periods present in the Variables sample but not in the Array
variables.from_array(array)
variables["A*;B*", "1988Y1:1997Y1"]
Workspace: Variables
nb variables: 10
filename: c:\soft\miniconda3\envs\py312\Lib\site-packages\iode\tests\data\fun.var
description: Modèle fun - Simulation 1
sample: 1988Y1:1997Y1
mode: LEVEL
name 1988Y1 1989Y1 1990Y1 1991Y1 1992Y1 1993Y1 1994Y1 1995Y1 1996Y1 1997Y1
ACAF 25.35 17.17 0.00 1.00 2.00 3.00 4.00 5.00 32.17 39.94
ACAG -37.24 -25.99 6.00 7.00 8.00 9.00 10.00 11.00 -40.24 -32.93
AOUC 0.95 0.98 12.00 13.00 14.00 15.00 16.00 17.00 1.05 1.08
AOUC_ 0.88 0.91 18.00 19.00 20.00 21.00 22.00 23.00 1.00 1.03
AQC 0.93 0.94 24.00 25.00 26.00 27.00 28.00 29.00 1.16 1.20
AX na na 30.00 31.00 32.00 33.00 34.00 35.00 na na
BENEF 281.77 345.16 284.18 197.60 194.04 197.24 231.75 286.67 340.56 348.32
BQY -20.18 -11.33 -34.10 -1.26 -13.75 52.16 66.63 91.09 104.68 113.52
BRUGP 44.67 45.09 0.00 49.77 52.58 51.70 52.06 52.20 52.70 52.85
BVY -18.70 -10.99 -34.10 -1.30 -14.70 58.10 75.90 105.50 123.20 135.62