How do I apply it to all the files in the folder (each file has a different name, one is fileA.txt, but the rest of the files are different (another one is "175.txt", another one is "published3.txt"...)
In the end I should get fileA_1.txt, 175_1.txt, published3_1.txt ...
Thanks
*I am on Ubuntu 16.04.3 LTS
Last edited by guilliber; 10-05-2017 at 10:27 AM..
Reason: Added OS
Sometimes it helps to explain what you are trying to do, textually, than just what you have tried code-wise, otherwise (often wrong) assumptions can be made about your intent and lead to more questions than answers.
To perform an operation on multiple files implies iteration, which requires a loop to accomplish.
Well I have several text files with different names. The content of these files is diverse, but they all usually include a field with sender of an email, subject of email, date...
But then there are more lines that are less important.
My purpose is to move the lines I am interested in to the top of each file, and leave the rest below. Basically remove some cruft from each of the files using grep/sed...
I don't know how to accomplish this with all the files in the folder, as they have different names. Plus the resulting files of the process should have the initial name plus some kind of suffix (Example: original name 175.txt, after process 175_1.txt)
As you can see my command includes several steps, and maybe more steps in the future (file 175.txt > 175_1.txt > 175_2.txt...)
Scott is correct: here is a start using your existing grep and sed logic. Which I think has some problems. Tinker with it, do not uncomment the lines that remove files, so you can run it till you get the sed the way you want.
I don't see how you determine the files to process
but let's assume you write the files you want to process to a text file.
Code:
ls *.txt > /tmp/list # create a list of files
while read fname
do
shortname=${fname%.txt} # remove trailing .txt string
grep -E '^\To: |^\Date: |^\Subject: ' $fname > ${shortname}_1.txt &&
grep -v '^\To: |^\Date: |^\Subject: ' $fname >> ${shortname}_1.txt &&
sed -i -e 's/\(Date: \|Subject: \|To: \)//g' ${shortname}_1.txt
done < /tmp/list
# moved the rm out here for safety. You cannot test code that removes files like that
# assume this worked correctly so: uncomment if true
# rm $(cat /tmp/list)
# rm /tmp/list
PS: stringing together in looong lines gains you very little and prevents making changes like I showed. You still need a loop.
Plus this code is not always going to work like you described - or so it seems to me.
This User Gave Thanks to jim mcnamara For This Post:
Instead of invoking grep twice and sed once for each file to be processed, one might also consider just using ed:
Code:
for fname in *.txt
do if [ "$fname" != "${fname%_1.txt}" ]
then printf '%s has already been processed.\n' "$fname"
continue
fi
newfname=${fname%.txt}_1.txt
printf 'Creating "%s" from "%s"...\n' "$newfname" "$fname"
ed -s "$fname" <<-EOF && echo rm "$fname"
H
g/^Subject: /.m0
g//.m0\\
s///
g/^Date: /.m0
g//.m0\\
s///
g/^To: /.m0
g//.m0\\
s///
w $newfname
q
EOF
done
Note that this will skip files with names ending with _1.txt and that it will put all To: lines first in the output, followed by all Date: lines, followed by all Subject: lines (each of the above with the keywords removed), followed by the remaining lines in the original file. If you know beforehand that there will never be more than one of the header lines of each of the three header line types, this code could be further simplified.
Note that the above code uses tabs (not spaces) to indent lines. If you convert those tabs to spaces, the shell won't find the end of the here-document.
Note also that this code uses:
Code:
echo rm "$fname"
which just prints the command to remove the old files. If, after testing, you find that the code does what you want; remove the echo to actually remove the old files.
This User Gave Thanks to Don Cragun For This Post:
Hi, I'd like to loop an action over all files with given extension within a folder.
The "main" action is: awk -F "\t" 'BEGIN{OFS="\t"}{if ($10=="S") print$0; }' input.txt > output.txt
The input.txt should be every file in the folder with *.subVCF extension; and the output should be a file... (3 Replies)
Dear folks
I have two data set which there names are "final.map" and "1.geno" and look like this structures:
final.map:
gi|358485511|ref|NC_006088.3| 2044
gi|358485511|ref|NC_006088.3| 2048
gi|358485511|ref|NC_006088.3| 2187
gi|358485511|ref|NC_006088.3| 17654
... (2 Replies)
Hi ,
I've learnt that the following command will remove the files from the given folder for given no.of days
find /home/etc -type f -atime -10 -exec rm -f {} \;
But how can I change the above command that will move the files to another specified directory instead of removing the... (1 Reply)
Hi All,
<Re-posting in Correct group>
I'm trying to select all the files in a folder that starts with a particular name format and are created in a gven date range using 'ls' command...but i'm not successful....
Example : I'm trying to see all the text files in a folder who names start... (6 Replies)
Hey gyuz,
I wanna calculate the number of mapped reads of a bam file in a region of interest. I used this code to do so :
samtools view input.bam chrname:region1 > region1.txt
This will store all the reads from given bam file within the region of interest in region1.txt
Now I have... (5 Replies)
Hi there
how to create a command in csh that would print files only from 1 folder but as an argument takes home directory for e.g. in my home diecrtory I have 3 folders unix, windows,mac and alot of files. and I running the program as ./op ~
this should return me files from unix folder without... (1 Reply)
Hi All,
I am using the awk command to replace ',' by '\t' (tabs) in a csv file. I would like to apply this to all .csv files in a directory and create .txt files with the tabs.
How would I do this in a script?
I have the following script called "csvtabs":
awk 'BEGIN {
FS... (4 Replies)
I am trying to grab a folder and all the folders and files underneath it and send it from one computer to another. I basically want to compress the whole folder into a tar, tgz, or zip file so that it can be sent as one file. is there a command to compress a folder and all its contents into a tar... (7 Replies)
Hi,
I have a requirement where
I need to move Bunch of folders containing multiple files to another archive location.
i want to use mv command .I am thinking when we use mv command to move directory does it create directory 1st and then move all the files ?
e.g
source... (4 Replies)
10-05-2017
