Menu ▾ ▴

ncap2 and variable names containing @

Help
mark
2020-02-20
2023-08-03

  • mark

    mark - 2020-02-20

    Hello,
    How to handle this kind of variables?
    Thanks,
    Mark

    int surface_type@MetaData(nlocs) ;

    ncap2: ERROR att_plain(): Unable to locate attribute surface_type@MetaData in input or output files.

     

    • Charlie Zender

      Charlie Zender - 2020-02-20

      Protect (precede) the @ with a backslash,\@ so that ncap2 interprets the @ as a literal character and not a syntax character.

       

  • mark

    mark - 2020-02-20

    But that results as below
    ncap2 -s 'water_sfc=(surface_type\@MetaData == 0)' in.nc out.nc
    Command-line script line 1, column 25: unexpected character '\'

     

    • Charlie Zender

      Charlie Zender - 2020-02-20

      try

      ncap2 -s 'water_sfc=("surface_type@MetaData" == 0)' in.nc out.nc

       

  • mark

    mark - 2020-02-20

    %ncap2 -s 'water_sfc=("surface_type@MetaData" == 0)' in.nc out.nc
    ncap2: ERROR ncap_var_var_op(): Cannot make attribute:~zz@string and variable:~int conform. So cannot perform arithmetic operation.

     

    • Charlie Zender

      Charlie Zender - 2020-02-20

      Sorry, I'm out of ideas. Maybe Henry knows and will chime in.

       

  • henry Butowsky

    henry Butowsky - 2020-02-21

    Hi Mark,
    There is no way in ncap2 to reference a variable with '@' in its name in ncap2 as it uses '@' to denote an attribute e.g time@units.
    The best course of action is to rename it to something else , do you ncap2 stuff then rename it back.

    ....Henry

     

    • Charlie Zender

      Charlie Zender - 2020-02-21

      That seems definitive. And unfortunate for the user. If it is definitive, please add this restriction to the ncap2 documentation. However, first answer this: Is is possible to modify the ANTLR grammar so that an '@' preceded by a backslash '\@' results in a literal '@' and thus a valid variable name string? At-signs not preceded by a backslash would continue to be treated as the lexical syntax indicating the start of an attribute name.

       

  • mark

    mark - 2020-02-21

    ncrename works fine with @ - seems only ncap2 issue

     

  • mark

    mark - 2020-02-21

    I should add preceded by "\" i.e. needs "\@"

     

  • henry Butowsky

    henry Butowsky - 2020-02-24

    HI All,
    looking at the NetCDF code - I have extracted the following definitions

    Below is from NC_name_check()
    And from ncgen3

    Looking at the ncap2 Lexer if we keep regular VAR_ID and ATT_ID as they are but modify quoted VAR_ID / ATT_ID.

    Charlie since 'th\@' is a valid name - I suggest the following.
    Since '/' is an illegal char we could denote a hex code by quoting it with /../ So

    'surface_type@MetaData'
    becomes
    ' surface_type/ 0x100/MetaData'
    '
    And if we wish to rver to the units of the variable it becomes:
    ' surface_type/ 0x100/MetaData@units'

    Also users can use this method for any other UTF8 chars

    ....Henry

    So the conclusion is that an ncname is "kind of UTF8"

    NC_name_check()

    /*

 * Verify that a name string is valid syntax.  The allowed name
 * syntax (in RE form) is:
 *
 * ([a-zA-Z0-9_]|{UTF8})([^\x00-\x1F\x7F/]|{UTF8})*
 *
 * where UTF8 represents a multibyte UTF-8 encoding.  Also, no
 * trailing spaces are permitted in names.  This definition
 * must be consistent with the one in ncgen.l.  We do not allow '/'
 * because HDF5 does not permit slashes in names as slash is used as a
 * group separator.  If UTF-8 is supported, then a multi-byte UTF-8
 * character can occur anywhere within an identifier.  We later
 * normalize UTF-8 strings to NFC to facilitate matching and queries.
 */

/* Problems:

1. Ideally, we assume the input is true ut8.
   Unfortunately, we may actually get iso-latin-8859-1.
   This means that there will be ambiguity about the characters
   in the range 128-255 because they will look like n-byte unicode
   when they are 1-byte 8859 characters. Because of our encoding,
   8859 characters above 128 will be handles as n-byte utf8 and so
   will probably not lex correctly.
   Solution: assume utf8 and note in the documentation that
   ISO8859 is specifically unsupported.
2. The netcdf function NC_check_name in string.c must be modified to
   conform to the use of UTF8.
3. We actually have three tests for UTF8 of increasing correctness
   (in the sense that the least correct will allow some sequences that
    are technically illegal UTF8).
   The tests are derived from the table at
        http://www.w3.org/2005/03/23-lex-U
   We include lexical definitions for all three, but use the second version.
*/
     

    • Charlie Zender

      Charlie Zender - 2020-02-24

      Thanks, Henry. That sounds good to me. Please proceed and post back here once you have verified that it works in the current snapshot of master so I can try it.

       

  • henry Butowsky

    henry Butowsky - 2020-02-24

    HI CHarlie,
    Don t know what i was thinking !!!! --
    We may want to use '/' to reference group variables in ncap2
    So I shall have to think soemthing els e up

    ...Henry

     

    • Charlie Zender

      Charlie Zender - 2020-02-24

      I suggest that double at-signs be converted to a single, literal, at-sign: @@ = @. TeXInfo uses this and it works well.

       

  • henry Butowsky

    henry Butowsky - 2020-02-24

    Ye its it fixes the problem at hand but what about a way of inputting a charater code ?
    lemme look at the UTF codes - maybe theres another one that netcfd doesnt like .

    ...Henry

     

  • Kasra K

    Kasra K - 2023-08-02

    There is alo a problem with variable that have a "period - dot" in their names, such as "RDRS_v2.1_P_P0_SFC".

    The following command fails:

    ncap2 -s 'RDRS_v2.1_P_P0_SFC *= 3' temp.nc out.nc
Command-line script:1:8: unexpected token: .1

    Sorry for posting on this old topic.

     

    • Charlie Zender

      Charlie Zender - 2023-08-03

      If you've tried the above suggestions and they do not work then I'm afraid you'll need to rename the variable (with ncrename), execute the script, and rename it back. This is why
      CF prohibits most special characters (except underscore) in variable names.

       

Log in to post a comment.

MongoDB Logo MongoDB