Shell scripting one liners using sed and awk

Some useful one liners with sed and awk

to print duplicate lines in a file

sort a1.txt | uniq -d

in vi editor to delete blank lines

g/^$/d

to delete blank lines having tabs and spaces

:g/^[\t ]*$/d

to delete blank lines having tabs and spaces from a file

sed '/^[\t ]*$/d' b.txt

sed '/Windows/d' b.txt -- to delete all lines containing "Windows" in the sentence
in short

sed '//d' (filename)

delete leading whitespace (spaces, tabs) from front of each line

sed 's/^[ \t]*//'                    # see note on '\t' at end of file

delete trailing whitespace (spaces, tabs) from end of each line

sed 's/[ \t]*$//'                    # see note on '\t' at end of file

delete BOTH leading and trailing whitespace from each line

sed 's/^[ \t]*//;s/[ \t]*$//'

to print linewise 80 characters and additional characters in newline offset by 5

sed -e :a -e 's/^.\{1,78\}$/ &/;ta'

substitute a word with another word for lines matching a particular pattern

sed '//s///g' (file.txt)

to globally replace a pattern with another pattern in vi editor

:%s/546/555555/g -- globally

:%s/546/555555/gc -- to check each value before replacing

to start recording into a buffer a sequence of commands in vi editor

start recording steps to buffer a
####qa
then type the sequene of commands
eg - to put a ';' at the end of each line and go to next line
####[SHIFT]a;[ESC]j
eg - to put a '---' underline after every line
o------------[ESC]j
Now to repeat this sequence of steps for all the line
####@a
to end recording to buffer
####q

@a - to play the sequence of steps repeatedly for all the lines

to print only the list of files in a directory using awk this is similar to simulating "ls -1"

ls -lrt | awk '{if ($9 != "")   print $9 }'

OR

ls -lrt | grep -v ^total | tr -s " "" " | cut -f9 -d

To print list of files created or modified today

#!/usr/bin/ksh
mth=`date | awk '{print $2}'`
dt=`date | awk '{print $3}'`
ls -lrt | awk -v dt=$dt  -v mth=$mth '{if ($9 != "" && $6==mth && $7==dt) print $9}'

To split a delimited string and print the output

#!/usr/bin/ksh
IFS=";"
for i in `cat delim.txt`
do
  echo $i
done

to find the highest number in a file

#!/usr/bin/ksh
gt=0
for i in `cat a.txt`
do
        if [ $i -gt $gt ]
        then
                gt=$i
        fi
done
echo $gt

if a line ends with a backslash, append the next line to it

sed -e :a -e '/\\$/N; s/\\\n//; ta'

to find a file in a list of subdirectories

find . -name a.txt

to find a particular word in files in a list of subdirectories

find . -name "*" | xargs grep "ksh"

to substitute multiples patterns with new patterns in a file

sed 's/paragraph/para/g;s/line/newline/g;s/type/typing/g' simplepara.txt

Quote for the day
“We May Encounter Many Defeats But We Must Not Be Defeated.”- Maya Angelou

Thought for the day

Commit your work to the Lord and your plans will be established

Proverbs  16:3