Scripts to search files in subdirectories, reverse lines and Interchange column positions

Script to search files in subdirectories and process

There are many times when there is a need to search files of a particular type in sub directories for further processing of the files. Manually go through a tree structure of directories and sub directories can waste a lot of time and effort and in the bargain some files may get skipped. The below is a very simple script where you can search for a particular file type in sub directories and work on the files accordingly.

#!/bin/sh
>/tmp/err.txt
dirnam="/oracle/home/scripts/data"
find $dirnam  -type f -print0 | while read -d $'\0' file; do
    echo "$file"
done
exit 0

A practical example for a dba can be where there are a lot of insert scripts to be executed in the database. Developers might have created these scripts and kept them in various subdirectories under a main directory and the dba has to run these scripts in the UAT environment or even in production.
In the below example, there can be many sub directories under /oracle/home/scripts/data where the files are kept. All the dba has to do is give this directory path in the script and execute it and bingo!!! all the files will get picked and processed.

#!/bin/sh
>/tmp/err.txt
dirnam="/oracle/home/scripts/data"
find $dirnam  -type f -print0 | while read -d $'\0' file; do
     psql -h "hostname" -U "postgres" "dbname" -f "$file" 2>>/tmp/err.txt
done
exit 0

Shell script to reverse the lines in a file

This is a sample script to hone your skills in writing small shell script programs required so often by developers, database and server administrators alike.
The practical use of this script may be limited, but for beginners it will be a useful building block in enhancing their logical thinking ,coding skills and usage of scripting tools.

You have a (a.txt) with the following lines of code
1111111111111111
2222222222222222
3333333333333333
4444444444444444
5555555555555555
6666666666666666
7777777777777777
8888888888888888
9999999999999999

and you want the output to be ( b.txt)
9999999999999999
8888888888888888
7777777777777777
6666666666666666
5555555555555555
4444444444444444
3333333333333333
2222222222222222
1111111111111111

There will be many ways of achieving this result. And some geeks may do it in a single line.
For this example below is the pseudo logic.
1. Get the count of lines in the file  -- cat $fileName | wc -l
2. In a Loop for this total count -- while [ $cnt -gt 0 ]
      Get each line of the file from bottom to top. For this use the combination of head and tail
        head -9 |  tail -1 -- head -9 will give the first 9 lines from top,
                        tail -1 will give the last line from the bottom
  head -8 |  tail -1 -- head -8 will give the first 8 lines from top .... and so on
 redirect this output to another file  >> b.txt
3. Decrement the counter till all lines are processed
        -- cnt=`expr $cnt - 1`


#!/bin/bash
> b.txt
fileName="a.txt"
# set n to 1
fileCnt=`cat $fileName | wc -l`
cnt=$fileCnt
while [ $cnt -gt 0 ]
do
echo $cnt
    cat $fileName | head -$cnt | tail -1 >> b.txt
    cnt=`expr $cnt - 1`
done

 

Shell script to interchange column positions using AWK

A very common requirement for an upload utility is a flat file with a pre defined format. Many a times it so happens that the columns in the file are not in the sequence that the application is expecting. The source from which these files are coming is also not known. It then becomes incumbent on the user , tester or developer to re arrange the column positions in the file to suit the upload utility requirements. If there are a large number of files, then manually making these changes is a very tedious and time consuming effort. The desired result can be achieved by a simple awk utility.
Below is a simple example of interchanging the column positions in a comma separated csv file.


#!/bin/ksh
cat a.txt | awk BEGIN'{FS=","}{print $5","$3","$1","$4","$7","$2","$6}' > b.txt


 This script will rearrange the columns in the order of 5,3,1,4,7,2,6.

Input file - a.txt
cccccc,ffffffff,bbbbbbb,ddddddd,aaaaaaaa,gggggggg,eeeeeeeee
cccccc,ffffffff,bbbbbbb,ddddddd,aaaaaaaa,gggggggg,eeeeeeeee
cccccc,ffffffff,bbbbbbb,ddddddd,aaaaaaaa,gggggggg,eeeeeeeee
cccccc,ffffffff,bbbbbbb,ddddddd,aaaaaaaa,gggggggg,eeeeeeeee
cccccc,ffffffff,bbbbbbb,ddddddd,aaaaaaaa,gggggggg,eeeeeeeee
cccccc,ffffffff,bbbbbbb,ddddddd,aaaaaaaa,gggggggg,eeeeeeeee
cccccc,ffffffff,bbbbbbb,ddddddd,aaaaaaaa,gggggggg,eeeeeeeee
cccccc,ffffffff,bbbbbbb,ddddddd,aaaaaaaa,gggggggg,eeeeeeeee
cccccc,ffffffff,bbbbbbb,ddddddd,aaaaaaaa,gggggggg,eeeeeeeee
cccccc,ffffffff,bbbbbbb,ddddddd,aaaaaaaa,gggggggg,eeeeeeeee

Output file - b.txt
aaaaaaaa,bbbbbbb,cccccc,ddddddd,eeeeeeeee,ffffffff,gggggggg
aaaaaaaa,bbbbbbb,cccccc,ddddddd,eeeeeeeee,ffffffff,gggggggg
aaaaaaaa,bbbbbbb,cccccc,ddddddd,eeeeeeeee,ffffffff,gggggggg
aaaaaaaa,bbbbbbb,cccccc,ddddddd,eeeeeeeee,ffffffff,gggggggg
aaaaaaaa,bbbbbbb,cccccc,ddddddd,eeeeeeeee,ffffffff,gggggggg
aaaaaaaa,bbbbbbb,cccccc,ddddddd,eeeeeeeee,ffffffff,gggggggg
aaaaaaaa,bbbbbbb,cccccc,ddddddd,eeeeeeeee,ffffffff,gggggggg
aaaaaaaa,bbbbbbb,cccccc,ddddddd,eeeeeeeee,ffffffff,gggggggg
aaaaaaaa,bbbbbbb,cccccc,ddddddd,eeeeeeeee,ffffffff,gggggggg
aaaaaaaa,bbbbbbb,cccccc,ddddddd,eeeeeeeee,ffffffff,gggggggg

 Quote for the day
“Develop An ‘Attitude Of Gratitude’. Say Thank You To Everyone You Meet For Everything They Do For You.”- Brian Tracy