Unix/Linux Go Back    


Shell Programming and Scripting BSD, Linux, and UNIX shell scripting — Post awk, bash, csh, ksh, perl, php, python, sed, sh, shell scripts, and other shell scripting languages questions here.

To check the file and remove header before copying

Shell Programming and Scripting


Reply    
 
Thread Tools Search this Thread Display Modes
    #1  
Old Unix and Linux 1 Week Ago
rohit_shinez rohit_shinez is offline
Registered User
 
Join Date: Jul 2013
Last Activity: 17 June 2017, 9:19 AM EDT
Posts: 165
Thanks: 55
Thanked 0 Times in 0 Posts
To check the file and remove header before copying

Hi Guys,

I have below directory where there are certain files. Something like below

Code:
country_dir

aus_01.txt
nz_01.txt
aus_02.txt
bd.txt

property.txt
aus
nz
bd

I need to remove the header of the file which ends with _01.txt while copying from country directory to another directory

Code which i am using


Code:
for i in `cat property.txt`
do
file=$(ls -t1 $filepath|  grep -i $file)

if [ "$i" == "aus" ]
then
if [ -f $file ]
then

      	cp $file /output/aus/
else 
	echo "No aus files"

fi

elif [ "$i" == "nz" ]
if [ -f $file ]
then

      	cp $file /output/nz/
else 
	echo "No nz files"

fi

fi
....
....
done

in the above code i need to remove the header for the files aus_01.txt
nz_01.txt before copying to output directory. Can the changes be done in same code
Sponsored Links
    #2  
Old Unix and Linux 1 Week Ago
RudiC RudiC is online now Forum Staff  
Moderator
 
Join Date: Jul 2012
Last Activity: 27 June 2017, 4:40 PM EDT
Location: Aachen, Germany
Posts: 10,965
Thanks: 280
Thanked 3,371 Times in 3,104 Posts
Not sure I fully understand. You want to copy all files whose names contain a string from the property.txt file, and remove the header line(s) from those who end in _01.txt?

Your sample code is not of great help to deduct in detail what you're after. I'm afraid it won't copy anything as $file's doesn't seem to be predictable because a) $filepath is undefined and thus empty b) the ls result to be assigned to file is grepped for $file itself, mayhap shooting yourself in the foot. $i, the variable looping through property.txt, isn't used for copying, just for testing for a filename existence. And, no provision is made to handle _01.txt files differently.

How many lines are the headers to be suppressed?
Sponsored Links
    #3  
Old Unix and Linux 1 Week Ago
rohit_shinez rohit_shinez is offline
Registered User
 
Join Date: Jul 2013
Last Activity: 17 June 2017, 9:19 AM EDT
Posts: 165
Thanks: 55
Thanked 0 Times in 0 Posts
Hi Rudic,

To answer your below question

You want to copy all files whose names contain a string from the property.txt file, and remove the header line(s) from those who end in _01.txt? - Yes i need this and wanted to apply the same in my code


Code:
filepath=/country/

for i in `cat property.txt`
do
file=$(ls -t1 $filepath|  grep -i $i)

if [ "$i" == "aus" ]
then
if [ -f $file ]
then

      	cp $file /output/aus/
else 
	echo "No aus files"

fi

elif [ "$i" == "nz" ]
if [ -f $file ]
then

      	cp $file /output/nz/
else 
	echo "No nz files"

fi

fi
....
....
done

    #4  
Old Unix and Linux 1 Week Ago
RudiC RudiC is online now Forum Staff  
Moderator
 
Join Date: Jul 2012
Last Activity: 27 June 2017, 4:40 PM EDT
Location: Aachen, Germany
Posts: 10,965
Thanks: 280
Thanked 3,371 Times in 3,104 Posts
If your shell (which you failed to mention) is a recent bash, try (yes, admittedly deploying the unloved and sometimes dangerous eval)

Code:
X=$(<property.txt)
for PN in $(eval echo $filepath/*{${X//$'\n'/,}}*)
   do   FN=${PN##*/}
        DIR=${FN%%[_.]*}
        [ $FN = ${FN/_01.txt} ] && HDCNT=0 || HDCNT=2
        [ -f $FN ] && tail -n+$HDCNT $FN > /output/$DIR/$FN || echo "No $FN file"
   done

and comment on the result
The Following User Says Thank You to RudiC For This Useful Post:
rohit_shinez (1 Week Ago)
Sponsored Links
    #5  
Old Unix and Linux 1 Week Ago
rohit_shinez rohit_shinez is offline
Registered User
 
Join Date: Jul 2013
Last Activity: 17 June 2017, 9:19 AM EDT
Posts: 165
Thanks: 55
Thanked 0 Times in 0 Posts
Hi Rudic,

Thanks for your inputs, can this be modified in my code which runs in bash


Code:
something like this

sed '1d' $file > $file

Sponsored Links
    #6  
Old Unix and Linux 1 Week Ago
RudiC RudiC is online now Forum Staff  
Moderator
 
Join Date: Jul 2012
Last Activity: 27 June 2017, 4:40 PM EDT
Location: Aachen, Germany
Posts: 10,965
Thanks: 280
Thanked 3,371 Times in 3,104 Posts
Why that sed? What don't you like in the proposal, which should run unmodified in (a recent) bash?
Sponsored Links
    #7  
Old Unix and Linux 1 Week Ago
rohit_shinez rohit_shinez is offline
Registered User
 
Join Date: Jul 2013
Last Activity: 17 June 2017, 9:19 AM EDT
Posts: 165
Thanks: 55
Thanked 0 Times in 0 Posts
Quote:
Originally Posted by RudiC View Post
Why that sed? What don't you like in the proposal, which should run unmodified in (a recent) bash?
The thing is I wanted to modify in my existing code itself where sed '1d' will remove the header file or similar to any other approach. Moreover I am not able to see copying commmand in your code
Sponsored Links
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Linux More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Check for empty line at end of a dynamic header in each file tpk UNIX for Dummies Questions & Answers 6 04-16-2015 03:54 AM
Remove the file content based on the Header of the file i150371485 Shell Programming and Scripting 5 08-03-2012 02:35 AM
Remove last few characters in a file but keeping Header and trailer intact nvuradi Shell Programming and Scripting 2 04-12-2012 02:16 PM
Check header of file, without opening the file lydiaflamp UNIX for Dummies Questions & Answers 2 10-03-2010 05:41 PM
Copy all the files with time stamp and remove header,trailer from file ksrams UNIX for Dummies Questions & Answers 35 07-30-2007 02:15 PM



All times are GMT -4. The time now is 04:43 PM.