Issue
I have a file that was created and I need to replace the last "," with "" so it will be valid JSON. The problem is that I can't figure out how to do it with sed
or even with grep
/piping to something else. I am really stumped here. Any help would be appreciated.
test.json
[
{MANY OTHER RECORDS, MAKING FILE 3.5Gig (making sed fail because of memory, so newlines were added)},
{"ID":"57705e4a-158c-4d4e-9e07-94892acd98aa","USERNAME":"jmael","LOGINTIMESTAMP":"2021-11-30"},
{"ID":"b8b67609-50ed-4cdc-bbb4-622c7e6a8cd2","USERNAME":"henrydo","LOGINTIMESTAMP":"2021-12-15"},
{"ID":"a44973d0-0ec1-4252-b9e6-2fd7566c6f7d","USERNAME":"null","LOGINTIMESTAMP":"2021-10-31"},
]
Of course, using grep
with -P
matches what I need to replace
grep -Pzo '"},\n]' test.json
Solution
An efficient solution would be to use perl
to read the last n
bytes of the file, then determine the position of the superfluous comma in those bytes (for ex. with a regex) and then replace this comma with a space character:
perl -e '
$n = 16; # how many bytes to read
open $fh, "+<", $ARGV[0]; # open file in read & write mode
seek $fh, -$n, 2; # go to the end minus some bytes
$n = read $fh, $str, $n; # load the end of the file
if ( $str =~ /,\s*]\s*$/s ) { # get position of comma
seek $fh, -($n - $-[0]), 1; # go to position of comma
print $fh " "; # replace comma with space char
}
close $fh; # close file
' log.json
The strong point of this solution is that it only reads a few bytes of the file for doing the replacement => that keeps the memory consumption to almost 0 and avoids reading through the whole file.
Answered By - Fravadona Answer Checked By - Mary Flores (WPSolving Volunteer)