I'm new to NCO and willing to remap a wrfout files in order to create an interpolated (ratio = 3) wrfinput file (1 timestep, 2D/3D variables). I looked up the forum and the User Guide but was not able to find answers to my questions.
How to create a mapfile from a wrfout.nc as input (coarse) and a wrfinput.nc (fine) ?
Then, if I understand correctly the User Guide, I shoud type something like : remap -i wrfout.nc -m map_fl -o wrfinput_remapped.nc ; is it correct ?
Do NCO/remap handles staggered grid (as WRF has XLAT, XLON for T° and XLAT_U, XLON_U for U/V...) ? Because CDO can't with WRF files.
I read these threads 1,2,3 but didn't find any help.
I would really appreciate your help as I'm struggling finding my way out in the User Guide/on the internet.
Thanks'
Last edit: Paul MAUGER 2023-04-28
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thank you. I tried the Prototypical Regridding III: Infer Grid-file from Data-file, please see hereunder :
pmauger@datarmor2:/home3/datawork/pmauger/CONFIGS/TEST_WRFINPUT_FROM_CYR>ltotal75M-rwxrwxrwx1pmaugerdroos7.8MApr2801:48wrfinput_d01_2005_01-rwxrwxrwx1pmaugerdroos7.0MApr2801:48wrfinput_d02_2005_01-rwxrwxrwx1pmaugerdroos60MApr2805:42wrfinput_origin.nc-rwxr-xr-x1pmaugerdroos399KApr2822:03geo_em.d01.nc-rwxr-xr-x1pmaugerdroos405KApr2822:03geo_em.d02.ncpmauger@datarmor2:/home3/datawork/pmauger/CONFIGS/TEST_WRFINPUT_FROM_CYR>ncremap-swrfinput_origin.nc-dwrfinput_d01_2005_01-mmap_d01.ncncks:ERRORnco_grd_nfr()reportsanidentifiedgridvariable(XLATwithrank3and/orXLONGwithrank3)hasrankgreaterthantwo---gridvariablescurrentlymusthaverank1or2.HINT:Ifgridvariablesdonotvaryintime,thentemporallyaveragethem(with,e.g.,ncwa-atimein.ncout.nc)priortoinferringgridncremap:ERRORFailedtoinferdestinationgrid.Debugthis:ncks-O--no_tmp_fl--hdr_pad=10000--gaaremap_script=ncremap--gaaremap_hostname=datarmor2.ice.ifremer.fr--gaaremap_version=4.9.2--rgrinfer--rgrhnt_dst="/tmp/ncremap_tmp_hnt_dst.txt.pid11951"--rgrscrip="/tmp/ncremap_tmp_grd_dst.nc.pid11951""wrfinput_d01_2005_01""/tmp/ncremap_tmp_out.nc.pid11951"pmauger@datarmor2:/home3/datawork/pmauger/CONFIGS/TEST_WRFINPUT_FROM_CYR>ncks-O--no_tmp_fl--hdr_pad=10000--gaaremap_script=ncremap--gaaremap_hostname=datarmor2.ice.ifremer.fr--gaaremap_version=4.9.2--rgrinfer--rgrhnt_dst="/tmp/ncremap_tmp_hnt_dst.txt.pid11951"--rgrscrip="/tmp/ncremap_tmp_grd_dst.nc.pid11951""wrfinput_d01_2005_01""/tmp/ncremap_tmp_out.nc.pid11951"ncks:ERRORnco_grd_nfr()reportsanidentifiedgridvariable(XLATwithrank3and/orXLONGwithrank3)hasrankgreaterthantwo---gridvariablescurrentlymusthaverank1or2.HINT:Ifgridvariablesdonotvaryintime,thentemporallyaveragethem(with,e.g.,ncwa-atimein.ncout.nc)priortoinferringgridpmauger@datarmor2:/home3/datawork/pmauger/CONFIGS/TEST_WRFINPUT_FROM_CYR>ncwa-aTimewrfinput_d01_2005_01wrfinput_d01_2005_01.timeavgpmauger@datarmor2:/home3/datawork/pmauger/CONFIGS/TEST_WRFINPUT_FROM_CYR>ncwa-aTimewrfinput_origin.ncwrfinput_origin.timeavgpmauger@datarmor2:/home3/datawork/pmauger/CONFIGS/TEST_WRFINPUT_FROM_CYR>ltotal131M-rwxrwxrwx1pmaugerdroos7.8MApr2801:48wrfinput_d01_2005_01-rwxrwxrwx1pmaugerdroos7.0MApr2801:48wrfinput_d02_2005_01-rwxrwxrwx1pmaugerdroos60MApr2805:42wrfinput_origin.nc-rwxr-xr-x1pmaugerdroos399KApr2822:03geo_em.d01.nc-rwxr-xr-x1pmaugerdroos405KApr2822:03geo_em.d02.nc-rw-r--r--1pmaugerdroos7.7MApr2822:07wrfinput_d01_2005_01.timeavg-rw-r--r--1pmaugerdroos60MApr2822:08wrfinput_origin.timeavgpmauger@datarmor2:/home3/datawork/pmauger/CONFIGS/TEST_WRFINPUT_FROM_CYR>ncremap-swrfinput_origin.timeavg-dwrfinput_d01_2005_01.timeavg-mmap_d01.ncGrid(src):wrfinput_origin.timeavgGrid(dst):/tmp/ncremap_tmp_grd_dst.nc.pid12721nco_err_exit():ERRORShortNCO-generatedmessage(usuallynameoffunctionthattriggerederror):nco_inq_dimid()nco_err_exit():ERRORErrorcodeis-46.TranslationintoEnglishwithnc_strerror(-46)is"NetCDF: Invalid dimension ID or name"ncremap:ERRORFailedtogeneratemap-file.Debugthis:ncks-O--dmm_in_mk--thr_nbr=2--no_tmp_fl--hdr_pad=10000--gaaremap_script=ncremap--gaaremap_hostname=datarmor2.ice.ifremer.fr--gaaremap_version=4.9.2--grd_src="wrfinput_origin.timeavg"--grd_dst="/tmp/ncremap_tmp_grd_dst.nc.pid12721"--map="map_d01.nc""/tmp/ncremap_tmp_dmm.nc.pid12721""/tmp/ncremap_tmp_out.nc.pid12721">/dev/nullpmauger@datarmor2:/home3/datawork/pmauger/CONFIGS/TEST_WRFINPUT_FROM_CYR>ncks-O--dmm_in_mk--thr_nbr=2--no_tmp_fl--hdr_pad=10000--gaaremap_script=ncremap--gaaremap_hostname=datarmor2.ice.ifremer.fr--gaaremap_version=4.9.2--grd_src="wrfinput_origin.timeavg"--grd_dst="/tmp/ncremap_tmp_grd_dst.nc.pid12721"--map="map_d01.nc""/tmp/ncremap_tmp_dmm.nc.pid12721""/tmp/ncremap_tmp_out.nc.pid12721">/dev/nullnco_err_exit():ERRORShortNCO-generatedmessage(usuallynameoffunctionthattriggerederror):nco_inq_dimid()nco_err_exit():ERRORErrorcodeis-46.TranslationintoEnglishwithnc_strerror(-46)is"NetCDF: Invalid dimension ID or name"
I looked the page up for trace of "WRF" or Weather Reasearch & Forecast" but didn't find any. I attached 3 geogrid files from WPS (WRF), "Cyr" being the original grid I'm looking to interpolate from. I also tried with geo_em.Cyr.nc and geo_em.d01.nc without any success ; even Time averaged files.
Have you any idea how I can get ncremap to work with my files ?
May I ask what is the problem with WRF files ? I'm constantly facing failure with usual weather/climate toolkits (like CDO). Is this all about CF Convention ?
Thank your for reading the docs. You're making progress...
This command is incorrect because is supposed to specify the source grid-file and instead you give it the source data file to be regridded:
hello, I just wondering if you are using this wrfinput files to drive wrf again. I would like to use such wrfout files as the initial and boundary conditions. If so, may i know how to process the wrfbdy file? thanks !!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
This appears to work for me. First I time-average the grid variables. Then I supply a 1x1 destination grid for the regridder. The regridder infers the source grid from the input file,
regrids it into foo.nc and saves the resulting mapfile as map.nc.
Hello,
Thanks for your answer. However, I'm looking to regrid wrfinput_origin.nc to the dimensions of wrfinput_d01_2005-01 without having any preprocessed grid file. Typing the exaxt command you advised unfortunately led to an invalid out.nc.
Would you mind trying with both source and destination grid files as I did ? Did I get the command wrong ? I would like to understand what went wrong and, moreover, I really need this remapped file for my initial conditions to fit my boundary conditions.
Thanks for your help which is much appreciated.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
My bad, sorry. I don't understand what happened (and still happens) when I transfer from the super-computer to my own computer : 2 files out of 3 seem to be corrupted in the while.
Would you mind test with the new wrfinput_d01_2005-01.origin file I just uploaded (same link to the drive as above) ?
Ncview and ncdump ran well with this one on my PC. It's got the same grid as the other wrfinput which is corrupted, so this is OK to use as destination grid file.
Thanks
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
ncwa -a Time wrfinput_d01_2005_01.origin wrfinput_d01_2005_01.origin_t
ncwa -a Time wrfinput_origin.nc wrfinput_origin_t.nc
ncremap -d wrfinput_d01_2005_01.origin_t -m map.nc wrfinput_origin_t.nc ~/foo.nc
I lack the resources to investigate in any greater detail. However, I notice your NCO is > 3 years old. I suggest you upgrade in case the issues you encountered have been mitigated by features or bugfixes since 2021. Good luck, CZ
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Update for complete wrfinput/wrfout processing (compliant with WRF3.6.1 : variables list has been updated since)
I had some trouble with my previous script version. Some variables on the W staggered grid that I has separated from the unstaggered grid woudn't be accepted by NCO (PH, PHB, SH2O, W).
I corrected the problem by suppressing the division of wrfinput_ and wrfout_ into T,U,V axes instead of T,U,V,W. Then only SH2O wouldn't be accepted by NCO (edit : layers number problem : 3 vs 4). This is why, hereunder, I tested each variable one by one the end of the code. I have to optimize this part.
# DEFINE INPUTS# USUALLY $1 = wrfinput_d0*# USUALLY $2 = wrfout_*# SELECT FIRST TIMESTEP FROM WRFOUTncks-O-dTime,0$2wrfout# WRFOUT (1 timestep) ACCORDING TO T,U,V,Wncks-vTimes,MU,MUB,P,P00,PB,PH,PHB,PSFC,P_STRAT,Q2,QCLOUD,QGRAUP,QICE,QRAIN,QSNOW,QVAPOR,SST,T,T00,T2,TH2,TSK,U10,V10,Wwrfoutwrfout_Tncks-vTimes,Uwrfoutwrfout_Uncks-vTimes,Vwrfoutwrfout_V# ACCORDING TO T,U,V,Wcp$1wrfinncks-vTimes,ALBBCK,CANWAT,CF1,CF2,CF3,CFN,CFN1,CLAT,COSALPHA,DN,DNW,DTBC,DTS,DTSEPS,E,F,FCX,FNDALBSI,FNDICEDEPTH,FNDSNOWH,FNDSNOWSI,FNDSOILW,FNM,FNP,FRC_URB2D,GCX,HGT,ISLTYP,IVGTYP,LAI,LAKE_DEPTH,LAKE_DEPTH_FLAG,LAKEFLAG,LAKEMASK,LANDMASK,LU_INDEX,MAPFAC_M,MAPFAC_MX,MAPFAC_MY,P_HYD,P_TOP,QV_BASE,RDN,RDNW,RDX,RDY,RESM,SAVE_TOPO_FROM_REAL,SEAICE,SHDMAX,SHDMIN,SINALPHA,SNOALB,SNOW,SNOWC,SNOWH,SR,STEP_NUMBER,T_BASE,T_INIT,TISO,TLP,TLP_STRAT,TMN,TOPOSLPX,TOPOSLPY,U_BASE,U_FRAME,UOCE,VAR,VAR_SSO,V_BASE,VEGFRA,V_FRAME,VOCE,XLAND,XLAT,XLONG,Z_BASE,ZETATOP,ZNU,CPLMASK,DZS,LANDUSEF,SMCREL,SMOIS,SOILCBOT,SOILCTOP,TSLB,ZNW,ZSwrfinwrfin_Tncks-vTimes,MAPFAC_U,MAPFAC_UX,MAPFAC_UY,XLAT_U,XLONG_Uwrfinwrfin_Uncks-vTimes,MAPFAC_V,MAPFAC_VX,MAPFAC_VY,MF_VX_INV,XLAT_V,XLONG_Vwrfinwrfin_V# GRID DESCRIPTIONcdogriddeswrfin_T>grid_T.txtcdogriddeswrfin_U>grid_U.txtcdogriddeswrfin_V>grid_V.txt# REMAP (BILINEAR)cdoremapbil,grid_T.txtwrfout_Twrfout_T_remapcdoremapbil,grid_U.txtwrfout_Uwrfout_U_remapcdoremapbil,grid_V.txtwrfout_Vwrfout_V_remap# CONCATENATE ALL SUB-FILEScpwrfout_T_remapwrfout_remapncks-Awrfout_U_remapwrfout_remapncks-Awrfout_V_remapwrfout_remap# REPLACE TIME VARIABLE TYPE AND TIME COORDINATE BY WRFINncrename-dTimes,Timewrfout_remapncks-O-s'Times=char(Times)'wrfout_remapwrfout_remapncks-O-x-vTimeswrfout_remapwrfout_remapncks-A-vTimeswrfinwrfout_remap# CHANGE XLAT* & XLONG* ATTRIBUTESncks-C-x-vXLONG,XLAT,XLONG_U,XLAT_U,XLONG_V,XLAT_Vwrfout_remapwrfout_remap_xlonglat_okncks-A-vXLONG,XLAT,XLONG_U,XLAT_U,XLONG_V,XLAT_V,HGTwrfinwrfout_remap_xlonglat_ok# APPEND ALL REMAPPED VARIABLES TO WRFINPUT (to be optimized)#cp wrfin control_0 # FOR TESTSncks-A-C-vMUwrfout_remap_xlonglat_okwrfinncks-A-C-vMUBwrfout_remap_xlonglat_okwrfinncks-A-C-vPwrfout_remap_xlonglat_okwrfinncks-A-C-vP00wrfout_remap_xlonglat_okwrfinncks-A-C-vPBwrfout_remap_xlonglat_okwrfinncks-A-C-vPSFCwrfout_remap_xlonglat_okwrfinncks-A-C-vP_STRATwrfout_remap_xlonglat_okwrfin#cp wrfin control_P # IDEMncks-A-C-vQ2wrfout_remap_xlonglat_okwrfinncks-A-C-vQCLOUDwrfout_remap_xlonglat_okwrfinncks-A-C-vQGRAUPwrfout_remap_xlonglat_okwrfinncks-A-C-vQICEwrfout_remap_xlonglat_okwrfinncks-A-C-vQRAINwrfout_remap_xlonglat_okwrfinncks-A-C-vQSNOWwrfout_remap_xlonglat_okwrfinncks-A-C-vQVAPORwrfout_remap_xlonglat_okwrfinncks-A-C-vSSTwrfout_remap_xlonglat_okwrfinncks-A-C-vTwrfout_remap_xlonglat_okwrfin#cp wrfin control_SST_T # IDEMncks-A-C-vT00wrfout_remap_xlonglat_okwrfinncks-A-C-vT2wrfout_remap_xlonglat_okwrfinncks-A-C-vTH2wrfout_remap_xlonglat_okwrfinncks-A-C-vTSKwrfout_remap_xlonglat_okwrfinncks-A-C-vU10wrfout_remap_xlonglat_okwrfinncks-A-C-vV10wrfout_remap_xlonglat_okwrfin#cp wrfin control_UV10 # IDEMncks-A-C-vUwrfout_remap_xlonglat_okwrfinncks-A-C-vVwrfout_remap_xlonglat_okwrfin#cp wrfin control_UV # IDEMncks-A-C-vPHwrfout_remap_xlonglat_okwrfinncks-A-C-vPHBwrfout_remap_xlonglat_okwrfinncks-A-C-vWwrfout_remap_xlonglat_okwrfin# RENAME PROPERLY ACCORDING TO INPUT FILEname="$(basename -- $1)"cpwrfin$name# CLEANrm-rfgrid*wrfout_*wrfin_**pid*wrfinwrfinput# TIME COUNTER ENDSend=$(date+%s)echo"Elapsed Time: $(($end-$start)) seconds"
Last edit: Paul MAUGER 2023-06-09
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hello to the community,
I'm new to NCO and willing to remap a wrfout files in order to create an interpolated (ratio = 3) wrfinput file (1 timestep, 2D/3D variables). I looked up the forum and the User Guide but was not able to find answers to my questions.
I read these threads 1,2,3 but didn't find any help.
I would really appreciate your help as I'm struggling finding my way out in the User Guide/on the internet.
Thanks'
Last edit: Paul MAUGER 2023-04-28
You might want to read the first sections of this https://acme-climate.atlassian.net/wiki/spaces/DOC/pages/754286611/Regridding+E3SM+Data+with+ncremap
I suggest you try to construct the appropriate command for your use case based on those instructions.
Hello,
Thank you. I tried the Prototypical Regridding III: Infer Grid-file from Data-file, please see hereunder :
I looked the page up for trace of "WRF" or Weather Reasearch & Forecast" but didn't find any. I attached 3 geogrid files from WPS (WRF), "Cyr" being the original grid I'm looking to interpolate from. I also tried with geo_em.Cyr.nc and geo_em.d01.nc without any success ; even Time averaged files.
Have you any idea how I can get ncremap to work with my files ?
May I ask what is the problem with WRF files ? I'm constantly facing failure with usual weather/climate toolkits (like CDO). Is this all about CF Convention ?
Last edit: Paul MAUGER 2023-04-29
Thank your for reading the docs. You're making progress...
This command is incorrect because is supposed to specify the source grid-file and instead you give it the source data file to be regridded:
Try:
hello, I just wondering if you are using this wrfinput files to drive wrf again. I would like to use such wrfout files as the initial and boundary conditions. If so, may i know how to process the wrfbdy file? thanks !!
Hello,
Thanks for your encouragement. I obviously misunderstood what I read. Nevertheless, I tried your command and I got no error... BUT : the out.nc file is corrupted ; see images attached.
Please see the original files I'm working with : https://drive.google.com/drive/folders/19dBokRXJxAENWBuKY9ZO0RTT65cDUcpc?usp=share_link
Last edit: Paul MAUGER 2023-04-29
This appears to work for me. First I time-average the grid variables. Then I supply a 1x1 destination grid for the regridder. The regridder infers the source grid from the input file,
regrids it into foo.nc and saves the resulting mapfile as map.nc.
Hello,
Thanks for your answer. However, I'm looking to regrid wrfinput_origin.nc to the dimensions of wrfinput_d01_2005-01 without having any preprocessed grid file. Typing the exaxt command you advised unfortunately led to an invalid out.nc.
Would you mind trying with both source and destination grid files as I did ? Did I get the command wrong ? I would like to understand what went wrong and, moreover, I really need this remapped file for my initial conditions to fit my boundary conditions.
Thanks for your help which is much appreciated.
This input file appears to be corrupt:
My bad, sorry. I don't understand what happened (and still happens) when I transfer from the super-computer to my own computer : 2 files out of 3 seem to be corrupted in the while.
The 3 files though are looking good on the super-computer, I tested each of them before and after sending them to my laptop.
Would you mind test with the new wrfinput_d01_2005-01.origin file I just uploaded (same link to the drive as above) ?
Ncview and ncdump ran well with this one on my PC. It's got the same grid as the other wrfinput which is corrupted, so this is OK to use as destination grid file.
Thanks
I really don't understand why it doesn't work for me. I keep getting corrupted files. Also I miss some variables, as U and V.
I checked each file at each step (after ncks -d Time,0 ... and ncwa -a Time ...).
Please see screenshots attached below.
If this can be of any help :
Last edit: Paul MAUGER 2023-05-10
I lack the resources to investigate in any greater detail. However, I notice your NCO is > 3 years old. I suggest you upgrade in case the issues you encountered have been mitigated by features or bugfixes since 2021. Good luck, CZ
Hello,
I've got some help and succeeded at regridding WRF's NetCDF. Please find below the procedure to be used :
1) Split coarse and fine files to T, U and V grids :
T grid :
U grid :
V grid :
2) Remap each of the new coarse grids to fine grids :
3) Concatenate T, U and V remapped fine grids together :
4) Add times :
Last edit: Paul MAUGER 2023-05-16
Thanks for posting your solution!
Update for complete wrfinput/wrfout processing (compliant with WRF3.6.1 : variables list has been updated since)
I had some trouble with my previous script version. Some variables on the W staggered grid that I has separated from the unstaggered grid woudn't be accepted by NCO (PH, PHB, SH2O, W).
I corrected the problem by suppressing the division of wrfinput_ and wrfout_ into T,U,V axes instead of T,U,V,W. Then only SH2O wouldn't be accepted by NCO (edit : layers number problem : 3 vs 4). This is why, hereunder, I tested each variable one by one the end of the code. I have to optimize this part.
Last edit: Paul MAUGER 2023-06-09
Again, thanks for posting your detailed solution.