jtc Files

Holding latest builds (the latest build: October 5, 2020)

Changes up till now:

• issues [#16], [#17], [#18]: no functional impact, code safety improvements
• compiling issues [#19], [#20]
• issue [#21]: fixed an occasional uncaught exception might be thrown in peculiar walks (UT'd)
• issue [#22]: fixed a nasty performance regression noticeable on big JSONs for lexemes supporting interpolation: <..>R, <..>L, <..>D, <..>j (UT'd)
• fix for the generated auto-tokens issue introduced in the prior build (UT'd)
• fixed issues [#27], [#28] (affecting Linux only)
• fixed issues [#29], [#32], improved per-walk template behavior [#31]
• fixed/improved template-argument behavior in options -u/-i/-c: the behavior should match the behavior of -T option (string-interpolation of iterables might have produced different results)
• fixed a crash potentially might be occurring when a JSON root undergoes interpolation
• fixed an issue (#33), where a non-initial Regex lexeme might not be getting engaged (that's the regression from v1.76)
• added template auto-token $wuid which refers to deterministic walk's unique id for each walk given by user (handy for making JSON elements collections per each walk)
• introduced flow-control for walk loops using <>f .. ><f pairs: this is a use-case to resolve recursive lookup chains
• improvements:
  • improved namespace passing between option chain-sets and for -p/ -s options
  • improved trailing backslash parsing in all lexemes
  • enabled walks over a templated argument in -i / -u / -c options (as well enabled namespaces passing to the template)
  • reinstated namespace passing between interleaved walks (it's a regression - the functionality was lost after re-designing namespaces in v1.76)
  • improved behavior for lexeme <..>I when initializing namespace
  • enhanced performance for tokens {{}}, {{..}} (when tokens used as standalone then no interpolation is needed and JSON can be retrieved directly from walks)
  • added a use-case for label interpolations when string-interpolating iterables (handy for generating headers from labels/indices for CSV output), e.g.: <<<'{"tbl":["a","b","c"]}' jtc -w'[tbl]<:>k' -qqT'"{}"' will generate 0, 1, 2 output (instead of a b c) - that is predicated by last walked <:>k directive (lexeme spelling in this case is limited to :).
  • improved label ordering in JSON objects: now numerical labels (those made of digits only) are ordered numerically, while all other labels ordered literally

items to accomplish before the next release:

• [x] introduced $@ auto-REGEX namespace - it holds all the RE matches (entire matches, or group matches) in a JSON array. That way it's easy to split strings, e.g.: <<<'"abc, def, ghi"' jtc -w'<[^, ]+>R' -rT'{{$@}}' produces output: [ "abc", "def", "ghi" ]
• [ ] redesign and enhance internal template-interpolate logic: currently all interpolations are done via JSON serialization / deserialization, which is a slow way - rework Json class to allow parsing templates and rewrite interpolation so that it's done via binary construction (serdes way will remain only for string interpolations).
• [ ] introduce couple variants of the <..>v directive:
  • <var:<JSN/TMP>>v1 allow saving a JSON spelled literally, or out of a template right into a namespace (currently any lexeme value in <..>v directive is either a JSON or promoted to a JSON string)
  • <var:[{{$PATH}}, <JSON/TMP>]>v2 - the JSON in this form allows reconstructing a JSON in a namespace (i.e., incrementally build up a JSON in the namespace)
• [ ] implement streamed parsing of JSON (i.e. in the format similar produced by this walk: jtc -rw'<>e:' -T'[{{$PATH}}, {{}}]' - this would allow processing a virtually endless JSONs w/o any memory pressure. (parsing of such streamed JSON will be done in a concurrent thread)