Registering a dataset that will exist in the future¶

Here we use a freva plugin run that has been applied¶

In [17]:
import freva
import xarray as xr
hist_id = 3085 # We can get this ID using the freva.history command
_ = freva.register_future_from_history_id(hist_id)

Let's search for the data¶

In [18]:
list(freva.databrowser(variable="tx90petccdi"))
Out[18]:
['future:///scratch/b/b380001/futures/6def5135a687932d27f419a3e993b5bd68aa03425ff0378cfb7745c0aef497a5/cmip5/output1/mpi-m/mpi-esm-lr/historical/yr/atmos/1day/r1i1p1/tx90pETCCDI/tx90pETCCDI_1day_mpi-esm-lr_historical_r1i1p1_199007020000-199207011200']

The data doesn't exist yet, but can be created on demand:¶

In [19]:
dset = xr.open_mfdataset(
    freva.databrowser(variable="tx90petccdi", 
                      execute_future=True
    )
)
dset


Out[19]:
<xarray.Dataset>
Dimensions:      (time: 3, lon: 192, lat: 96, bnds: 2)
Coordinates:
  * time         (time) datetime64[ns] 1990-07-02 1991-07-02 1992-07-01T12:00:00
  * lon          (lon) float64 0.0 1.875 3.75 5.625 ... 352.5 354.4 356.2 358.1
  * lat          (lat) float64 -88.57 -86.72 -84.86 -83.0 ... 84.86 86.72 88.57
Dimensions without coordinates: bnds
Data variables:
    time_bnds    (time, bnds) float64 dask.array<chunksize=(3, 2), meta=np.ndarray>
    tx90pETCCDI  (time, lat, lon) float32 dask.array<chunksize=(3, 96, 192), meta=np.ndarray>
Attributes: (12/13)
    CDI:                      Climate Data Interface version 2.0.5 (https://m...
    Conventions:              CF-1.4
    institution:              Max Planck Institute for Meteorology
    ETCCDI_institution:       UNSW Australia & FUB Berlin
    ETCCDI_institution_id:    UNSW-CCRC,FUB-IfM
    ETCCDI_software:          climdex.pcic
    ...                       ...
    contact:                  k204230
    frequency:                yr
    creation_date:            2023-09-28T11:10:27Z
    title:                    ETCCDI indices computed on 0
    history:                  Thu Sep 28 13:11:53 2023: cdo -s setlevel,0 cac...
    CDO:                      Climate Data Operators version 2.0.5 (https://m...
xarray.Dataset
    • time: 3
    • lon: 192
    • lat: 96
    • bnds: 2
    • time
      (time)
      datetime64[ns]
      1990-07-02 ... 1992-07-01T12:00:00
      standard_name :
      time
      long_name :
      time
      axis :
      T
      array(['1990-07-02T00:00:00.000000000', '1991-07-02T00:00:00.000000000',
             '1992-07-01T12:00:00.000000000'], dtype='datetime64[ns]')
    • lon
      (lon)
      float64
      0.0 1.875 3.75 ... 356.2 358.1
      standard_name :
      longitude
      long_name :
      lon
      units :
      degrees_east
      axis :
      X
      array([  0.   ,   1.875,   3.75 ,   5.625,   7.5  ,   9.375,  11.25 ,  13.125,
              15.   ,  16.875,  18.75 ,  20.625,  22.5  ,  24.375,  26.25 ,  28.125,
              30.   ,  31.875,  33.75 ,  35.625,  37.5  ,  39.375,  41.25 ,  43.125,
              45.   ,  46.875,  48.75 ,  50.625,  52.5  ,  54.375,  56.25 ,  58.125,
              60.   ,  61.875,  63.75 ,  65.625,  67.5  ,  69.375,  71.25 ,  73.125,
              75.   ,  76.875,  78.75 ,  80.625,  82.5  ,  84.375,  86.25 ,  88.125,
              90.   ,  91.875,  93.75 ,  95.625,  97.5  ,  99.375, 101.25 , 103.125,
             105.   , 106.875, 108.75 , 110.625, 112.5  , 114.375, 116.25 , 118.125,
             120.   , 121.875, 123.75 , 125.625, 127.5  , 129.375, 131.25 , 133.125,
             135.   , 136.875, 138.75 , 140.625, 142.5  , 144.375, 146.25 , 148.125,
             150.   , 151.875, 153.75 , 155.625, 157.5  , 159.375, 161.25 , 163.125,
             165.   , 166.875, 168.75 , 170.625, 172.5  , 174.375, 176.25 , 178.125,
             180.   , 181.875, 183.75 , 185.625, 187.5  , 189.375, 191.25 , 193.125,
             195.   , 196.875, 198.75 , 200.625, 202.5  , 204.375, 206.25 , 208.125,
             210.   , 211.875, 213.75 , 215.625, 217.5  , 219.375, 221.25 , 223.125,
             225.   , 226.875, 228.75 , 230.625, 232.5  , 234.375, 236.25 , 238.125,
             240.   , 241.875, 243.75 , 245.625, 247.5  , 249.375, 251.25 , 253.125,
             255.   , 256.875, 258.75 , 260.625, 262.5  , 264.375, 266.25 , 268.125,
             270.   , 271.875, 273.75 , 275.625, 277.5  , 279.375, 281.25 , 283.125,
             285.   , 286.875, 288.75 , 290.625, 292.5  , 294.375, 296.25 , 298.125,
             300.   , 301.875, 303.75 , 305.625, 307.5  , 309.375, 311.25 , 313.125,
             315.   , 316.875, 318.75 , 320.625, 322.5  , 324.375, 326.25 , 328.125,
             330.   , 331.875, 333.75 , 335.625, 337.5  , 339.375, 341.25 , 343.125,
             345.   , 346.875, 348.75 , 350.625, 352.5  , 354.375, 356.25 , 358.125])
    • lat
      (lat)
      float64
      -88.57 -86.72 ... 86.72 88.57
      standard_name :
      latitude
      long_name :
      lat
      units :
      degrees_north
      axis :
      Y
      array([-88.572166, -86.722534, -84.861969, -82.99894 , -81.134979, -79.270561,
             -77.405891, -75.541061, -73.676132, -71.811134, -69.946083, -68.080994,
             -66.215874, -64.350731, -62.485569, -60.620396, -58.755211, -56.890015,
             -55.024807, -53.159595, -51.294376, -49.429153, -47.563927, -45.698692,
             -43.833458, -41.96822 , -40.102978, -38.237736, -36.37249 , -34.507244,
             -32.641994, -30.776745, -28.911493, -27.04624 , -25.180986, -23.315731,
             -21.450476, -19.585218, -17.719961, -15.854704, -13.989446, -12.124187,
             -10.258928,  -8.393669,  -6.528409,  -4.66315 ,  -2.79789 ,  -0.93263 ,
               0.93263 ,   2.79789 ,   4.66315 ,   6.528409,   8.393669,  10.258928,
              12.124187,  13.989446,  15.854704,  17.719961,  19.585218,  21.450476,
              23.315731,  25.180986,  27.04624 ,  28.911493,  30.776745,  32.641994,
              34.507244,  36.37249 ,  38.237736,  40.102978,  41.96822 ,  43.833458,
              45.698692,  47.563927,  49.429153,  51.294376,  53.159595,  55.024807,
              56.890011,  58.755211,  60.620396,  62.485569,  64.350731,  66.215874,
              68.080994,  69.946083,  71.811134,  73.676132,  75.541061,  77.405891,
              79.270561,  81.134979,  82.99894 ,  84.861969,  86.722534,  88.572166])
    • time_bnds
      (time, bnds)
      float64
      dask.array<chunksize=(3, 2), meta=np.ndarray>
      Array Chunk
      Bytes 48 B 48 B
      Shape (3, 2) (3, 2)
      Dask graph 1 chunks in 2 graph layers
      Data type float64 numpy.ndarray
      2 3
    • tx90pETCCDI
      (time, lat, lon)
      float32
      dask.array<chunksize=(3, 96, 192), meta=np.ndarray>
      long_name :
      Percentage of Days when Daily Maximum Temperature is Above the 90th Percentile
      units :
      %
      CDI_grid_type :
      gaussian
      CDI_grid_num_LPE :
      48
      history :
      Created by climdex.pcic 1.1.11 on Thu Sep 28 13:10:27 2023
      base_period :
      1991-1991
      Array Chunk
      Bytes 216.00 kiB 216.00 kiB
      Shape (3, 96, 192) (3, 96, 192)
      Dask graph 1 chunks in 2 graph layers
      Data type float32 numpy.ndarray
      192 96 3
    • time
      PandasIndex
      PandasIndex(DatetimeIndex(['1990-07-02 00:00:00', '1991-07-02 00:00:00',
                     '1992-07-01 12:00:00'],
                    dtype='datetime64[ns]', name='time', freq=None))
    • lon
      PandasIndex
      PandasIndex(Index([    0.0,   1.875,    3.75,   5.625,     7.5,   9.375,   11.25,  13.125,
                15.0,  16.875,
             ...
              341.25, 343.125,   345.0, 346.875,  348.75, 350.625,   352.5, 354.375,
              356.25, 358.125],
            dtype='float64', name='lon', length=192))
    • lat
      PandasIndex
      PandasIndex(Index([  -88.5721664428711,   -86.7225341796875,  -84.86196899414062,
              -82.99893951416016,  -81.13497924804688,  -79.27056121826172,
              -77.40589141845703,  -75.54106140136719,  -73.67613220214844,
               -71.8111343383789,  -69.94608306884766,  -68.08099365234375,
              -66.21587371826172,   -64.3507308959961,  -62.48556900024414,
              -60.62039566040039, -58.755210876464844,   -56.8900146484375,
              -55.02480697631836,  -53.15959548950195, -51.294376373291016,
              -49.42915344238281, -47.563926696777344, -45.698692321777344,
             -43.833457946777344,  -41.96821975708008,  -40.10297775268555,
             -38.237735748291016,  -36.37248992919922,  -34.50724411010742,
              -32.64199447631836, -30.776744842529297,   -28.9114933013916,
             -27.046239852905273, -25.180986404418945, -23.315731048583984,
             -21.450475692749023,  -19.58521842956543, -17.719961166381836,
             -15.854703903198242, -13.989445686340332, -12.124187469482422,
             -10.258928298950195,  -8.393669128417969,  -6.528409481048584,
              -4.663149833679199, -2.7978897094726562, -0.9326300024986267,
              0.9326298832893372,  2.7978899478912354,   4.663149833679199,
               6.528409481048584,   8.393669128417969,  10.258928298950195,
              12.124187469482422,  13.989445686340332,  15.854703903198242,
              17.719961166381836,   19.58521842956543,  21.450475692749023,
              23.315731048583984,  25.180986404418945,  27.046239852905273,
                28.9114933013916,  30.776744842529297,   32.64199447631836,
               34.50724411010742,   36.37248992919922,  38.237735748291016,
               40.10297775268555,   41.96821975708008,  43.833457946777344,
              45.698692321777344,  47.563926696777344,   49.42915344238281,
              51.294376373291016,   53.15959548950195,   55.02480697631836,
              56.890010833740234,  58.755210876464844,   60.62039566040039,
               62.48556900024414,    64.3507308959961,   66.21587371826172,
               68.08099365234375,   69.94608306884766,    71.8111343383789,
               73.67613220214844,   75.54106140136719,   77.40589141845703,
               79.27056121826172,   81.13497924804688,   82.99893951416016,
               84.86196899414062,    86.7225341796875,    88.5721664428711],
            dtype='float64', name='lat'))
  • CDI :
    Climate Data Interface version 2.0.5 (https://mpimet.mpg.de/cdi)
    Conventions :
    CF-1.4
    institution :
    Max Planck Institute for Meteorology
    ETCCDI_institution :
    UNSW Australia & FUB Berlin
    ETCCDI_institution_id :
    UNSW-CCRC,FUB-IfM
    ETCCDI_software :
    climdex.pcic
    ETCCDI_software_version :
    1.1.11
    contact :
    k204230
    frequency :
    yr
    creation_date :
    2023-09-28T11:10:27Z
    title :
    ETCCDI indices computed on 0
    history :
    Thu Sep 28 13:11:53 2023: cdo -s setlevel,0 cachedir/3113/tx90pETCCDI_yr_MPI-ESM-LR_historical_r1i1p1_1990-1992.nc outdir/3113/cmip5/output1/mpi-m/mpi-esm-lr/historical/yr/atmos/tx90petccdi/r1i1p1//tx90petccdi_yr_mpi-esm-lr_historical_r1i1p1_1990-1992.nc
    CDO :
    Climate Data Operators version 2.0.5 (https://mpimet.mpg.de/cdo)

The data has been loaded, we can work with it (for example plot it)¶

In [20]:
dset.sum(dim="time")["tx90pETCCDI"].plot()
Out[20]:
<matplotlib.collections.QuadMesh at 0x7fffb7fa21d0>

What happens if the data get's lost?¶

Let's delete the data:

In [21]:
!rm -fr /scratch/b/b380001/futures/6def5135a687932d27f419a3e993b5bd68aa03425ff0378cfb7745c0aef497a5

The data is still in the databrowser:

In [22]:
list(freva.databrowser(variable="tx90petccdi"))
Out[22]:
['/scratch/b/b380001/futures/6def5135a687932d27f419a3e993b5bd68aa03425ff0378cfb7745c0aef497a5/cmip5/output1/mpi-m/mpi-esm-lr/historical/yr/atmos/yr/r1i1p1/v20230928/tx90pETCCDI/tx90pETCCDI_yr_mpi-esm-lr_historical_r1i1p1_199007020000-199207011200.nc']

Because of that the data can be re-created:

In [23]:
dset = xr.open_mfdataset(
    freva.databrowser(variable="tx90petccdi",
                      execute_future=True
    )
)
dset


Out[23]:
<xarray.Dataset>
Dimensions:      (time: 3, lon: 192, lat: 96, bnds: 2)
Coordinates:
  * time         (time) datetime64[ns] 1990-07-02 1991-07-02 1992-07-01T12:00:00
  * lon          (lon) float64 0.0 1.875 3.75 5.625 ... 352.5 354.4 356.2 358.1
  * lat          (lat) float64 -88.57 -86.72 -84.86 -83.0 ... 84.86 86.72 88.57
Dimensions without coordinates: bnds
Data variables:
    time_bnds    (time, bnds) float64 dask.array<chunksize=(3, 2), meta=np.ndarray>
    tx90pETCCDI  (time, lat, lon) float32 dask.array<chunksize=(3, 96, 192), meta=np.ndarray>
Attributes: (12/13)
    CDI:                      Climate Data Interface version 2.0.5 (https://m...
    Conventions:              CF-1.4
    institution:              Max Planck Institute for Meteorology
    ETCCDI_institution:       UNSW Australia & FUB Berlin
    ETCCDI_institution_id:    UNSW-CCRC,FUB-IfM
    ETCCDI_software:          climdex.pcic
    ...                       ...
    contact:                  k204230
    frequency:                yr
    creation_date:            2023-09-28T11:13:07Z
    title:                    ETCCDI indices computed on 0
    history:                  Thu Sep 28 13:14:21 2023: cdo -s setlevel,0 cac...
    CDO:                      Climate Data Operators version 2.0.5 (https://m...
xarray.Dataset
    • time: 3
    • lon: 192
    • lat: 96
    • bnds: 2
    • time
      (time)
      datetime64[ns]
      1990-07-02 ... 1992-07-01T12:00:00
      standard_name :
      time
      long_name :
      time
      axis :
      T
      array(['1990-07-02T00:00:00.000000000', '1991-07-02T00:00:00.000000000',
             '1992-07-01T12:00:00.000000000'], dtype='datetime64[ns]')
    • lon
      (lon)
      float64
      0.0 1.875 3.75 ... 356.2 358.1
      standard_name :
      longitude
      long_name :
      lon
      units :
      degrees_east
      axis :
      X
      array([  0.   ,   1.875,   3.75 ,   5.625,   7.5  ,   9.375,  11.25 ,  13.125,
              15.   ,  16.875,  18.75 ,  20.625,  22.5  ,  24.375,  26.25 ,  28.125,
              30.   ,  31.875,  33.75 ,  35.625,  37.5  ,  39.375,  41.25 ,  43.125,
              45.   ,  46.875,  48.75 ,  50.625,  52.5  ,  54.375,  56.25 ,  58.125,
              60.   ,  61.875,  63.75 ,  65.625,  67.5  ,  69.375,  71.25 ,  73.125,
              75.   ,  76.875,  78.75 ,  80.625,  82.5  ,  84.375,  86.25 ,  88.125,
              90.   ,  91.875,  93.75 ,  95.625,  97.5  ,  99.375, 101.25 , 103.125,
             105.   , 106.875, 108.75 , 110.625, 112.5  , 114.375, 116.25 , 118.125,
             120.   , 121.875, 123.75 , 125.625, 127.5  , 129.375, 131.25 , 133.125,
             135.   , 136.875, 138.75 , 140.625, 142.5  , 144.375, 146.25 , 148.125,
             150.   , 151.875, 153.75 , 155.625, 157.5  , 159.375, 161.25 , 163.125,
             165.   , 166.875, 168.75 , 170.625, 172.5  , 174.375, 176.25 , 178.125,
             180.   , 181.875, 183.75 , 185.625, 187.5  , 189.375, 191.25 , 193.125,
             195.   , 196.875, 198.75 , 200.625, 202.5  , 204.375, 206.25 , 208.125,
             210.   , 211.875, 213.75 , 215.625, 217.5  , 219.375, 221.25 , 223.125,
             225.   , 226.875, 228.75 , 230.625, 232.5  , 234.375, 236.25 , 238.125,
             240.   , 241.875, 243.75 , 245.625, 247.5  , 249.375, 251.25 , 253.125,
             255.   , 256.875, 258.75 , 260.625, 262.5  , 264.375, 266.25 , 268.125,
             270.   , 271.875, 273.75 , 275.625, 277.5  , 279.375, 281.25 , 283.125,
             285.   , 286.875, 288.75 , 290.625, 292.5  , 294.375, 296.25 , 298.125,
             300.   , 301.875, 303.75 , 305.625, 307.5  , 309.375, 311.25 , 313.125,
             315.   , 316.875, 318.75 , 320.625, 322.5  , 324.375, 326.25 , 328.125,
             330.   , 331.875, 333.75 , 335.625, 337.5  , 339.375, 341.25 , 343.125,
             345.   , 346.875, 348.75 , 350.625, 352.5  , 354.375, 356.25 , 358.125])
    • lat
      (lat)
      float64
      -88.57 -86.72 ... 86.72 88.57
      standard_name :
      latitude
      long_name :
      lat
      units :
      degrees_north
      axis :
      Y
      array([-88.572166, -86.722534, -84.861969, -82.99894 , -81.134979, -79.270561,
             -77.405891, -75.541061, -73.676132, -71.811134, -69.946083, -68.080994,
             -66.215874, -64.350731, -62.485569, -60.620396, -58.755211, -56.890015,
             -55.024807, -53.159595, -51.294376, -49.429153, -47.563927, -45.698692,
             -43.833458, -41.96822 , -40.102978, -38.237736, -36.37249 , -34.507244,
             -32.641994, -30.776745, -28.911493, -27.04624 , -25.180986, -23.315731,
             -21.450476, -19.585218, -17.719961, -15.854704, -13.989446, -12.124187,
             -10.258928,  -8.393669,  -6.528409,  -4.66315 ,  -2.79789 ,  -0.93263 ,
               0.93263 ,   2.79789 ,   4.66315 ,   6.528409,   8.393669,  10.258928,
              12.124187,  13.989446,  15.854704,  17.719961,  19.585218,  21.450476,
              23.315731,  25.180986,  27.04624 ,  28.911493,  30.776745,  32.641994,
              34.507244,  36.37249 ,  38.237736,  40.102978,  41.96822 ,  43.833458,
              45.698692,  47.563927,  49.429153,  51.294376,  53.159595,  55.024807,
              56.890011,  58.755211,  60.620396,  62.485569,  64.350731,  66.215874,
              68.080994,  69.946083,  71.811134,  73.676132,  75.541061,  77.405891,
              79.270561,  81.134979,  82.99894 ,  84.861969,  86.722534,  88.572166])
    • time_bnds
      (time, bnds)
      float64
      dask.array<chunksize=(3, 2), meta=np.ndarray>
      Array Chunk
      Bytes 48 B 48 B
      Shape (3, 2) (3, 2)
      Dask graph 1 chunks in 2 graph layers
      Data type float64 numpy.ndarray
      2 3
    • tx90pETCCDI
      (time, lat, lon)
      float32
      dask.array<chunksize=(3, 96, 192), meta=np.ndarray>
      long_name :
      Percentage of Days when Daily Maximum Temperature is Above the 90th Percentile
      units :
      %
      CDI_grid_type :
      gaussian
      CDI_grid_num_LPE :
      48
      history :
      Created by climdex.pcic 1.1.11 on Thu Sep 28 13:13:07 2023
      base_period :
      1991-1991
      Array Chunk
      Bytes 216.00 kiB 216.00 kiB
      Shape (3, 96, 192) (3, 96, 192)
      Dask graph 1 chunks in 2 graph layers
      Data type float32 numpy.ndarray
      192 96 3
    • time
      PandasIndex
      PandasIndex(DatetimeIndex(['1990-07-02 00:00:00', '1991-07-02 00:00:00',
                     '1992-07-01 12:00:00'],
                    dtype='datetime64[ns]', name='time', freq=None))
    • lon
      PandasIndex
      PandasIndex(Index([    0.0,   1.875,    3.75,   5.625,     7.5,   9.375,   11.25,  13.125,
                15.0,  16.875,
             ...
              341.25, 343.125,   345.0, 346.875,  348.75, 350.625,   352.5, 354.375,
              356.25, 358.125],
            dtype='float64', name='lon', length=192))
    • lat
      PandasIndex
      PandasIndex(Index([  -88.5721664428711,   -86.7225341796875,  -84.86196899414062,
              -82.99893951416016,  -81.13497924804688,  -79.27056121826172,
              -77.40589141845703,  -75.54106140136719,  -73.67613220214844,
               -71.8111343383789,  -69.94608306884766,  -68.08099365234375,
              -66.21587371826172,   -64.3507308959961,  -62.48556900024414,
              -60.62039566040039, -58.755210876464844,   -56.8900146484375,
              -55.02480697631836,  -53.15959548950195, -51.294376373291016,
              -49.42915344238281, -47.563926696777344, -45.698692321777344,
             -43.833457946777344,  -41.96821975708008,  -40.10297775268555,
             -38.237735748291016,  -36.37248992919922,  -34.50724411010742,
              -32.64199447631836, -30.776744842529297,   -28.9114933013916,
             -27.046239852905273, -25.180986404418945, -23.315731048583984,
             -21.450475692749023,  -19.58521842956543, -17.719961166381836,
             -15.854703903198242, -13.989445686340332, -12.124187469482422,
             -10.258928298950195,  -8.393669128417969,  -6.528409481048584,
              -4.663149833679199, -2.7978897094726562, -0.9326300024986267,
              0.9326298832893372,  2.7978899478912354,   4.663149833679199,
               6.528409481048584,   8.393669128417969,  10.258928298950195,
              12.124187469482422,  13.989445686340332,  15.854703903198242,
              17.719961166381836,   19.58521842956543,  21.450475692749023,
              23.315731048583984,  25.180986404418945,  27.046239852905273,
                28.9114933013916,  30.776744842529297,   32.64199447631836,
               34.50724411010742,   36.37248992919922,  38.237735748291016,
               40.10297775268555,   41.96821975708008,  43.833457946777344,
              45.698692321777344,  47.563926696777344,   49.42915344238281,
              51.294376373291016,   53.15959548950195,   55.02480697631836,
              56.890010833740234,  58.755210876464844,   60.62039566040039,
               62.48556900024414,    64.3507308959961,   66.21587371826172,
               68.08099365234375,   69.94608306884766,    71.8111343383789,
               73.67613220214844,   75.54106140136719,   77.40589141845703,
               79.27056121826172,   81.13497924804688,   82.99893951416016,
               84.86196899414062,    86.7225341796875,    88.5721664428711],
            dtype='float64', name='lat'))
  • CDI :
    Climate Data Interface version 2.0.5 (https://mpimet.mpg.de/cdi)
    Conventions :
    CF-1.4
    institution :
    Max Planck Institute for Meteorology
    ETCCDI_institution :
    UNSW Australia & FUB Berlin
    ETCCDI_institution_id :
    UNSW-CCRC,FUB-IfM
    ETCCDI_software :
    climdex.pcic
    ETCCDI_software_version :
    1.1.11
    contact :
    k204230
    frequency :
    yr
    creation_date :
    2023-09-28T11:13:07Z
    title :
    ETCCDI indices computed on 0
    history :
    Thu Sep 28 13:14:21 2023: cdo -s setlevel,0 cachedir/3114/tx90pETCCDI_yr_MPI-ESM-LR_historical_r1i1p1_1990-1992.nc outdir/3114/cmip5/output1/mpi-m/mpi-esm-lr/historical/yr/atmos/tx90petccdi/r1i1p1//tx90petccdi_yr_mpi-esm-lr_historical_r1i1p1_1990-1992.nc
    CDO :
    Climate Data Operators version 2.0.5 (https://mpimet.mpg.de/cdo)

How can we check if the data is physically present?¶

Sometimes it might be useful to check if we can use the data straight away or the data has to be re-created. The databrowser doesn't get informed about the deletion of data automoatically. For example if we delete the data again:

In [24]:
!rm -fr /scratch/b/`b380001/futures/6def5135a687932d27f419a3e993b5bd68aa03425ff0378cfb7745c0aef497a5

the databrowser still shows the location on disk although the data doesn't exist anymore:

In [25]:
list(freva.databrowser(variable="tx90petccdi"))
Out[25]:
['/scratch/b/b380001/futures/6def5135a687932d27f419a3e993b5bd68aa03425ff0378cfb7745c0aef497a5/cmip5/output1/mpi-m/mpi-esm-lr/historical/yr/atmos/yr/r1i1p1/v20230928/tx90pETCCDI/tx90pETCCDI_yr_mpi-esm-lr_historical_r1i1p1_199007020000-199207011200.nc']

We can use the check_future method to check for the existence for futures. Every dataset that doesn't exsit anymore will be deleted from the databrowser and replaced by the special future:// url, indicating that this dataset doesn't exist but can be recreated. We can use a key=value pair search facet like for the databrowser method to sub select only certain datasets:

In [27]:
freva.check_futures(variable="tx90petccdi")

Let's search for the data again:

In [28]:
list(freva.databrowser(variable="tx90petccdi"))
Out[28]:
['future:///scratch/b/b380001/futures/6def5135a687932d27f419a3e993b5bd68aa03425ff0378cfb7745c0aef497a5/cmip5/output1/mpi-m/mpi-esm-lr/historical/yr/atmos/1day/r1i1p1/tx90pETCCDI/tx90pETCCDI_1day_mpi-esm-lr_historical_r1i1p1_199007020000-199207011200']