Wednesday, December 29, 2021

[SOLVED] PCRE regex doesn't seem to work when used from shell with grep

Issue

I'm trying to output a node using a capture group for a decompiled DTS file using a PCRE regex passed into grep. I'm interested in the key-samkey {(...)}; node only.

Any ideas as to what I might be doing wrong or can you point at any alternate methods to extract the node with its contents? I can't use bash's =~ operator, because there is a requirement that we only use sh.

I tried patterns:

/(key-samkey {[.]*.+?[.]*};)/s

(key-samkey {[\s\S]*.+?(?=};))

The exact command I'm using is:

cat {input file} | grep -Po "{pattern}"

Both of these patterns seem to work correctly on online regex testing websites with PCRE syntax, but fail when executed from the shell.

The file which I'm running pattern matching on is structured like this: 

/dts-v1/;

/ {

    signature {

        key-samkey {
            required = "conf";
            algo = "sha256,rsa4096";
            rsa,r-squared = <xxxxxxxx>;
            rsa,modulus = <xxxxxxxx>;
            rsa,exponent = <0xxx 0xxxxxx>;
            rsa,n0-inverse = <0xxxxxxxxx>;
            rsa,num-bits = <0xxxxx>;
            key-name-hint = "samkey";
        };
    };
};

Solution

You're almost there. A regular expression you can use is

(?s)(key-samkey \{.+?\};).

  • (?s): The dot . matches everything (DOTALL)
  • \{ and \}: You have to escape these, because they have special meaning in a regex.
  • .+?: matches everything it can non-greedy, meaning, in this case, everything up to the first };

Then use the -z switch of grep, this replaces newlines in the input with null-bytes, so that grep sees the input as one big line.

Example: I stored your example in the file test.file:

> grep -Pzo '(?s)(key-samkey \{.+?\};)' test.file

key-samkey {
            required = "conf";
            algo = "sha256,rsa4096";
            rsa,r-squared = <xxxxxxxx>;
            rsa,modulus = <xxxxxxxx>;
            rsa,exponent = <0xxx 0xxxxxx>;
            rsa,n0-inverse = <0xxxxxxxxx>;
            rsa,num-bits = <0xxxxx>;
            key-name-hint = "samkey";
        };


Answered By - Tom Regner
at