Copy all the files with time stamp and remove header,trailer from file


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Copy all the files with time stamp and remove header,trailer from file
# 22  
Old 07-26-2007
Did you read my reply about the upper case and lower case "txt"?
# 23  
Old 07-26-2007
Quote:
Originally Posted by Shell_Life
Did you read my reply about the upper case and lower case "txt"?
Yes ..My input file name extensions are .TXT
Input file(s) located under directory /input and file names are:
1) input1_20070723_171556.TXT
2) input2_20070723_171556.TXT
3) input3_20070723-171556.TXT

Could you please tell me where do i make changes to script..
# 24  
Old 07-26-2007
Ksrams,
I just tried the script here in my environment and it does work.

Place the following highlighted lines in the script:
Code:
...
timeStmpExpr='_[0-9]\{8\}.[0-9]\{6\}.txt'
echo 'Just before ls 11111'
ls ${inDir}
echo 'Just before ls-grep 22222'
ls ${inDir} | grep $timeStmpExpr
echo 'Just before ls-grep-while 22222'
ls ${inDir} | grep $timeStmpExpr | \
...

# 25  
Old 07-27-2007
Quote:
Originally Posted by ksrams
I tried your script but it got result as bad substitution

Could you tell why its giving this error
fyi..file names could start with any characters not starts with file.
I am not limiting the file names in the directory. The code would process all the files in the directory.

The code
Code:
#!/bin/ksh

Input_Dir="$HOME/test/input"
Output_Dir="$HOME/test/out"

rm -f $Output_Dir/*
for file in `ls -l $Input_Dir | grep '^-' | awk '{print $9}'`
do
        new_file=${file%%_*}
        sed '1d;$d' $Input_Dir/$file > $Output_Dir/$new_file.txt
        touch -r $Input_Dir/$file $Output_Dir/$new_file.txt
done
exit

Debug

Code:
$ >sh -x del_head.sh
+ Input_Dir=/home/test/input
+ Output_Dir=/home/test/out
+ rm -f /home/test/out/input1.txt /home/test/out/input2.txt /home/test/out/input3.txt
++ ls -l /home/test/input
++ grep '^-'
++ awk '{print $9}'
+ new_file=input1
+ sed '1d;$d' /home/test/input/input1_20070723_171556.txt
+ touch -r /home/test/input/input1_20070723_171556.txt /home/test/out/input1.txt
+ new_file=input2
+ sed '1d;$d' /home/test/input/input2_20070723_171556.txt
+ touch -r /home/test/input/input2_20070723_171556.txt /home/test/out/input2.txt
+ new_file=input3
+ sed '1d;$d' /home/test/input/input3_20070723_171556.txt
+ touch -r /home/test/input/input3_20070723_171556.txt /home/test/out/input3.txt
+ exit

# 26  
Old 07-27-2007
Quote:
Originally Posted by Shell_Life
Ksrams,
I just tried the script here in my environment and it does work.

Place the following highlighted lines in the script:
Code:
...
timeStmpExpr='_[0-9]\{8\}.[0-9]\{6\}.txt'
echo 'Just before ls 11111'
ls ${inDir}
echo 'Just before ls-grep 22222'
ls ${inDir} | grep $timeStmpExpr
echo 'Just before ls-grep-while 22222'
ls ${inDir} | grep $timeStmpExpr | \
...

Shell_Life,

I am running this script from seperate directory called script directory..
when i ran this script its taking input files from script directory(which has only .ksh files) instead of input directory
fyi..i have passed absolute path for $1, $2
Do i need to run this from input directory instead of script directory..
Thanks in advance..
# 27  
Old 07-27-2007
Ksrams,
I don't know what else to tell you.

The script works fine -- I just tested once again here.

As the last resource, try running only the following five lines:
Code:
#!/bin/ksh
mInDir=$1       ## Parameter one
mOutDir=$2     ## Parameter two
echo 'mInDir = '${mInDir}
echo 'mOutDir = '${mOutDir}

# 28  
Old 07-27-2007
Quote:
Originally Posted by Shell_Life
Ksrams,
I don't know what else to tell you.

The script works fine -- I just tested once again here.

As the last resource, try running only the following five lines:
Code:
#!/bin/ksh
mInDir=$1       ## Parameter one
mOutDir=$2     ## Parameter two
echo 'mInDir = '${mInDir}
echo 'mOutDir = '${mOutDir}

The output is
Quote:
mInDir =
mOutDIr =
do we need to check any entries in .profile
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Append pipe | at the end of all the rows except header n trailer for all the files under a directory

Hi Experts Need help... I am looking for a Unix script to append pipe | at the end of all the rows (except header and trailer)in all the files placed under the directory /interfaces/Temp e.g. Header row1 row2 row3 Trailer The script should read all the files under... (3 Replies)
Discussion started by: phani333
3 Replies

2. Shell Programming and Scripting

Verify the header and trailer in file

please see my requirement, I hope I am clear. (9 Replies)
Discussion started by: mirwasim
9 Replies

3. Shell Programming and Scripting

Script to validate file header and trailer

Hi, I need a script that validates a file header/detail/trailer. File layout is: Header - Rec_Type|File_name|File_Date Detail - Rec_Type|field1|field2|field3... Trailder - Rec_Type|File_name|File_Date|Record_count Sample Data: HDR|customer_data.dat|20120709... (7 Replies)
Discussion started by: ash_sh
7 Replies

4. Shell Programming and Scripting

Remove last few characters in a file but keeping Header and trailer intact

Hi All, I am trying write a simple command using AWK and SED to this but without any success. Here is what I am using: head -1 test1.txt>test2.txt|sed '1d;$d' test1.txt|awk '{print substr($0,0,(length($0)-2))}' >>test2.txt|tail -1 test1.txt>>test2.txt Input: Header 1234567 abcdefgh... (2 Replies)
Discussion started by: nvuradi
2 Replies

5. UNIX for Dummies Questions & Answers

Adding header and trailer into a file

Hi, I want to add the below Header to all the files in sequence File1,File2,File3...etc "ABC,<number of chracter in the file>" e,g - If File1 is as below pqrstuvdt abcdefgh then I want to add the above header into it ,So that File1 becomes as below ABC,17 pqrstuvdt abcdefgh ... (9 Replies)
Discussion started by: spari2
9 Replies

6. Shell Programming and Scripting

Identify log files based on time stamp,zip and then copy..HELP

Hi All, PFB is a requirement. I am new to shell scripting. So plz help. It would be highly appreciated. 1. choose all the log files based on a particular date (files location is '/test/domain')--i.e,we should choose all the files that are modified on 29th November, neither 28th nor 30th 2.... (3 Replies)
Discussion started by: skdas_niladri
3 Replies

7. Shell Programming and Scripting

improve performance - replace $\| with $#@ and remove header and trailer records

Hi All, In my file i need to remove header and trailer records which comes in 1st line and last line respectively. After that i need to replace '$\|' with '$#@'. I am using sed command for this and its taking lot of time. Is there any other command which can be used to improve performance? ... (1 Reply)
Discussion started by: HemaV
1 Replies

8. Shell Programming and Scripting

how to copy an alertlog file from a particular time stamp?

Hi, Can any one tell me how to copy an alertlog file from a particular time stamp using shell script? Thanks (3 Replies)
Discussion started by: suman_dba1
3 Replies

9. Shell Programming and Scripting

Merge text files while combining the multiple header/trailer records into one each.

Situation: Our system currently executes a job (COBOL Program) that generates an interface file to be sent to one of our vendors. Because this system processes information for over 100,000 employees/retirees (and growing), we'd like to multi-thread the job into processing-groups in order to... (4 Replies)
Discussion started by: oordonez
4 Replies

10. Shell Programming and Scripting

Remove header(first line) and trailer(last line) in ANY given file

Hi, I need some help in removing the header (first line) and the trailer (last line) in a give file... The data file actually comes in EBCDIC format and I converted it into ASCII.. Now I need to strip off the first line and the last line.. I think we can use sed to do something like this:... (2 Replies)
Discussion started by: madhunk
2 Replies
Login or Register to Ask a Question