Wednesday, October 5, 2022

[SOLVED] Delete lines with same fields/strings

Issue

I rewrite my previous question because it was unclear.

I have test1.txt formatted in this way (this example has 3 lines)

Link;alfa (zz);some text;
Link;alfa (zz);other text;other text2;
Link;jack;

In another filetext test2.txt I have text formatted in this way without delimiters ; but only as simple string (this example has 2 lines)

tommy emmanuel
alfa (zz)

In test2.txt I have never Link word and I can I have ( ) characters but I have never ; delimiter character

I want to get result.txt written in this way

Link;jack;

Logic behind : In test2.txt I have alfa (zz). This string / field is the same in test1.txt - I have the same string alfa (zz) on first and second line between; delimiter. Condition: if this field match happens then that lines should be deleted and for this reason I written that I expect only 3rd line

Link;jack;

I test this code

sed 's/.*Link;//;s/;.*//' test2.txt | grep -Fvf- test1.txt

and this

awk -F \; '
FNR == NR {cull[$0]=""}
FNR != NR {
    for (str in cull) {
        if ($2 == str) {
            next
        }
    }
    print
}' test2.txt test1.txt > culled.txt

Problem is that it rewrite me same lines and don't delete lines with same fields

Update question:

According to anubhava answer and this example the presence of this kind of strings don't remove lines

If I have inside test2.txt

Dark Tranquillity - A Moonclad Reflection [ep] (1992) Melodic Death Metal
Dark Tranquillity - A Closer End [best of_compilation] (2008) Melodic Death Metal

then I can't match and remove lines in text1.txt if I have these lines

Link;Dark Tranquillity - A Moonclad Reflection [ep] (1992) Melodic Death Metal;Dark Tranquillity - A moonclad reflection [7'' Ep 1992_Slaughter Rec.].rar;https://disk.yandex.com/public?hash=JA7Gu2CysxSf2HhAKaBxmU%2By27B6dPd6uRwPFu%2B9x0s%3D;https://metalarea.org/forum/index.php?showtopic=5037

Link;Dark Tranquillity - A Closer End [best of_compilation] (2008) Melodic Death Metal;Dark Tranquillity - A Closer End [2008].rar;https://disk.yandex.com/public?hash=RCZbOrqci8lX%2Fa%2BPzhB6vchlr5rXyc%2B2NHiJNCu%2BQYM%3D;https://metalarea.org/forum/index.php?showtopic=48557

Solution

You may use this awk solution:

awk -F';' 'FNR == NR {
   gsub(/^[[:space:]]+|[[:space:]]+$/, "")
   cull[$0]
   next
}
!($2 in cull)' test2.txt test1.txt > culled.txt

cat culled.txt

Link;Dark Tranquillity - Enter Suicidal Angels [ep] (1996) Melodic Death Metal;Dark Tranquillity 1996 - Enter Suicidal Angels (EP).rar;https://disk.yandex.com/public?hash=fBvwBTBJ8%2Fx1mWXvl7usrAMe06esHZFDmHJWF8E2T6LK7Wvfu9Q5Qja9cb5JAU%2Fzq%2FJ6bpmRyOJonT3VoXnDag%3D%3D;https://metalarea.org/forum/index.php?showtopic=5041
Link;Sea Of Tranquillity - Darkened [demo] (1993) Death_Thrash Metal;eaofT3Dd.7z;https://disk.yandex.com/public?hash=DzCNqfEv2pydYzB0YWVvetV2Jx8QDCwktop3y8PIC%2BD5W%2Fnt8ikX81%2F7cf49g8dNq%2FJ6bpmRyOJonT3VoXnDag%3D%3D;https://metalarea.org/forum/index.php?showtopic=153504
Link;Dark Tranquillity - The Absolute [single] (2017) Melodic Death Metal (D);Dark Tranquillity - The Absolute (2017) Single MCD (+SATANIST666+).rar;https://cloud.mail.ru/public/ckFc/A5sQ6pqhb;
Link;Dark Tranquillity - Trail Of Life Decayed [ep] (1992) Melodic Death Metal;1991 - Trail Of Life Decayed.7z;https://www.mediafire.com/file/5pi74bqvujea9rg/1991_-_Trail_Of_Life_Decayed.7z/file;https://metalarea.org/forum/index.php?showtopic=5115
Link;Dark Tranquillity - Phantom Days [single] (2020) Melodic Death Metal (D);Dark Tranquillity - Phantom Days (2020) by Andrew.rar;https://www.mediafire.com/file/ho25i02j3ybgmty/Dark_Tranquillity_-_Phantom_Days_%25282020%2529_by_Andrew.rar/file;https://metalarea.org/forum/index.php?showtopic=341548
Link;Dark Tranquillity - Of Chaos And Eternal Night [ep] (1995) Melodic Death Metal;Dark Tranquillity - Of Chaos And Eternal Night (EP) [1995].rar;https://disk.yandex.com/public?hash=Ax%2B2Gfqzr9%2FdS87cgRcUhGBCoQzKZfz5ZDUa2U%2Fbsn4%3D;https://metalarea.org/forum/index.php?showtopic=5040
Link;Dark Tranquillity - Of Chaos And Eternal Night [ep] (1995) Melodic Death Metal;Dark Tranquillity 1995 - Of Chaos And Eternal Night (EP).rar;https://disk.yandex.com/public?hash=le8r7ZI%2F%2BTw2CjsDbNFriDZdCGpSy1hj%2BoQdGHrrBFdcJM8eIp%2F3J17qG5MjC1Fgq%2FJ6bpmRyOJonT3VoXnDag%3D%3D;https://metalarea.org/forum/index.php?showtopic=5040

There is no need to use for loop. Just create an associative array while reading content from 2nd file and then while reading 1st file print only line that have 2nd column not in array seen.



Answered By - anubhava
Answer Checked By - Senaida (WPSolving Volunteer)
at