Tuesday, October 25, 2022

[SOLVED] How to remove unwanted characters in a file (using a shell script)

Issue

I have a file which looks like this (file.txt)

AYOnanl3knsgv2StRr44  CRITICAL","component  MMP-FileService  [email protected]  CODE_SMELL
AYOnanl3knsgv2StRr45  CRITICAL","component  MMP-FileService  [email protected]  CODE_SMELL
AYOnanl3knsgv2StRr46  CRITICAL","component  MMP-FileService  [email protected]  CODE_SMELL
AYOnanl3knsgv2StRr47  CRITICAL","component  MMP-FileService  [email protected]     CODE_SMELL
AYOnanmeknsgv2StRr48  MAJOR",               MMP-FileService  [email protected]  CODE_SMELL
AYOnanm-knsgv2StRr4-  BLOCKER",             MMP-FileService  [email protected]      CODE_SMELL
AYOnanm6knsgv2StRr49  MAJOR",               MMP-FileService  [email protected]      CODE_SMELL
AYOnannKknsgv2StRr4_  BLOCKER",             MMP-FileService  [email protected]   BUG
AYODwmuBknsgv2StRqkr  MINOR",               MMP-FileService  [email protected]   CODE_SMELL
AYODwmuBknsgv2StRqkt  MINOR",               MMP-FileService  [email protected]   CODE_SMELL
AYODwmuBknsgv2StRqks  MINOR",               MMP-component    [email protected]   CODE_SMELL
AYODwmuBknsgv2StRqku  MINOR",               MMP-component    [email protected]   CODE_SMELL
AYODsI7Fknsgv2StRqac  MAJOR",               MMP-component    [email protected]   CODE_SMELL
AYODsI7Nknsgv2StRqad  MAJOR",               MMP-component    [email protected]   CODE_SMELL
AYODsI-Qknsgv2StRqai  MAJOR",               MMP-component    [email protected]   CODE_SMELL

I have to remove unwanted characters in 2nd column ","component and ",

then expected output

AYOnanl3knsgv2StRr44  CRITICAL  MMP-FileService  [email protected]  CODE_SMELL
AYOnanl3knsgv2StRr45  CRITICAL  MMP-FileService  [email protected]  CODE_SMELL
AYOnanl3knsgv2StRr46  CRITICAL  MMP-FileService  [email protected]  CODE_SMELL
AYOnanl3knsgv2StRr47  CRITICAL  MMP-FileService  [email protected]    CODE_SMELL
AYOnanmeknsgv2StRr48  MAJOR     MMP-FileService  [email protected]  CODE_SMELL
AYOnanm-knsgv2StRr4-  BLOCKER   MMP-FileService  [email protected]    CODE_SMELL
AYOnanm6knsgv2StRr49  MAJOR     MMP-FileService  [email protected]    CODE_SMELL
AYOnannKknsgv2StRr4_  BLOCKER   MMP-FileService  [email protected]    BUG
AYODwmuBknsgv2StRqkr  MINOR     MMP-FileService  [email protected]   CODE_SMELL
AYODwmuBknsgv2StRqkt  MINOR     MMP-FileService  [email protected]   CODE_SMELL
AYODwmuBknsgv2StRqks  MINOR     MMP-component  [email protected]   CODE_SMELL
AYODwmuBknsgv2StRqku  MINOR     MMP-component  [email protected]   CODE_SMELL
AYODsI7Fknsgv2StRqac  MAJOR     MMP-component  [email protected]   CODE_SMELL
AYODsI7Nknsgv2StRqad  MAJOR     MMP-component  [email protected]   CODE_SMELL
AYODsI-Qknsgv2StRqai  MAJOR     MMP-component  [email protected]   CODE_SMELL

This is what I tried

cat file.txt | tr -d '",' | sed 's/component//'

then output I got

YOnanl3knsgv2StRr44  CRITICAL  MMP-FileService  [email protected]  CODE_SMELL
AYOnanl3knsgv2StRr45  CRITICAL  MMP-FileService  [email protected]  CODE_SMELL
AYOnanl3knsgv2StRr46  CRITICAL  MMP-FileService  [email protected]  CODE_SMELL
AYOnanl3knsgv2StRr47  CRITICAL  MMP-FileService  [email protected]     CODE_SMELL
AYOnanmeknsgv2StRr48  MAJOR               MMP-FileService  [email protected]  CODE_SMELL
AYOnanm-knsgv2StRr4-  BLOCKER             MMP-FileService  [email protected]      CODE_SMELL
AYOnanm6knsgv2StRr49  MAJOR               MMP-FileService  [email protected]      CODE_SMELL
AYOnannKknsgv2StRr4_  BLOCKER             MMP-FileService  [email protected]   BUG
AYODwmuBknsgv2StRqkr  MINOR               MMP-FileService  [email protected]   CODE_SMELL
AYODwmuBknsgv2StRqkt  MINOR               MMP-FileService  [email protected]   CODE_SMELL
AYODwmuBknsgv2StRqks  MINOR               MMP-    [email protected]   CODE_SMELL
AYODwmuBknsgv2StRqku  MINOR               MMP-    [email protected]   CODE_SMELL
AYODsI7Fknsgv2StRqac  MAJOR               MMP-    [email protected]   CODE_SMELL
AYODsI7Nknsgv2StRqad  MAJOR               MMP-    [email protected]   CODE_SMELL
AYODsI-Qknsgv2StRqai  MAJOR               MMP-    [email protected]   CODE_SMELL

my executed shell command is applying to the other columns as well (in this case it has applied to 3rd column too) that is the problem I am having. Is there any way to apply command only for 2nd column?

Can someone help me to figure out this? Thanks in advance!

Note: I am not allowed to use jq or other scripting languages as JavaScript, Python etc.


Solution

Consider this approach, if you are sure the characters you wish to remove appear in the second column only (borrowed and adapted from here https://unix.stackexchange.com/questions/492500/awk-replace-one-character-only-in-a-certain-column)

cat file.txt | awk '{{gsub("\",(\"component)?","", $2)}} 1'
  • gsub("\",\"component?","", $2) for each input line, replace all the ",("component)? in 2nd field with blank - this is a regular expression saying find ", then optionally the part in brackets: "component. ? is the operator for optional
  • 1 is an awk idiom to print contents of $0 (which contains the input record)


Answered By - mjsqu
Answer Checked By - Candace Johnson (WPSolving Volunteer)
at