[SOLVED] How to delete lines if next line begins with the same content?

Issue

As an example the file with following content:

My dog
My dog is running
My dog is running fast
A cat
A cat is black
A cat is black and small

As a result I want to have only this two lines:

My dog is running fast
A cat is black and small

I would prefer to do it in one line command.

Tried many SED, AWK and UNIQ options but could not find a good solution. I guess SED should be able to do it if it can read lines by two and print first line only if second line does not match it. But, cannot figure out how to do it.

Solution

This might work for you (GNU sed):

sed -E 'N;/^(.*)\n\1/!P;D' file

Use -E option to make regexp easier.

Append the next line.

If the whole of the first line matches the start of the next line, do not print the first line.

Otherwise, print the first line and then always delete it and repeat.

N.B. The D command is special in that it deletes up to and including the first newline (if there is one), anything remaining causes the normal cycle not to fetch the next line and then then next cycle uses whatever is leftover.

Answered By - potong

Answer Checked By - Mary Flores (WPSolving Volunteer)