extract multiple cloumns from multiple files; skip rows and include filenames; awk
Hello,
I am trying to write a bash shell script that does the following:
1.Finds all *.txt files within my directory of interest
2. reads each of the files (25 files) one by one (tab-delimited format and have the same data format)
3. skips the first 10 rows of the file
4. extracts and prints out columns 2,14 , 15 into one output file
5. adds a new column to the final output file with the name of the txt file from where the data was extracted.
I have written a shell script which is not working properly and doesnot have the code for the part to skip 10 rows.
Below I have pasted a sample input file, output file and my code
re: extract multiple cloumns from multiple files; skip rows and include filenames; aw
Hello,
I am trying to write a bash shell script that does the following:
1.Finds all *.txt files within my directory of interest
2. reads each of the files (25 files) one by one (tab-delimited format and have the same data format)
3. skips the first 10 rows of the file
4. extracts and prints out columns 2,14 , 15 into one output file
5. adds a new column to the final output file with the name of the txt file from where the data was extracted.
I have written a shell script which is not working properly and doesnot have the code for the part to skip 10 rows.
Below I have pasted a sample input file, output file and my code
Input file format:
Output format: tab delimited file. The last column shows the filename from which the data was extracted
1 6774.29 228.723 ABC.txt
2 6800.2 229.421 ABC.txt
3 6826.51 228.385 DEF.txt
4 6850.48 228.853 DEF.txt
5 6875.37 228.408 XYZ.txt
6 6900.98 229.321 XYZ.txt
My incomplete code:
thanks in advance for your help.
Last edited by manishabh; 08-18-2009 at 11:41 PM..
Reason: displayed wrong format of tables; needed to add tags
Thanks a lot for posting the code. I apologise for your frustating experience with trying to understand the tables. I ran the code, however it throws an error:
'test1.sh: line 3: syntax error near unexpected token `do
'test1.sh: line 3: `do
also I forgot to mention that each of my files is in a subdirectory. So the directory hierarchy is as follows:
root_folder-->ABC-->ABC.txt
-->CDF-->CDF.txt
Hi,
Is there any way to do a cat * where it shows the name of each file in the process? Similar to what more does below?
$ more ?.sql
::::::::::::::
1.sql
::::::::::::::
set linesize 200
select db_unique_name,
cast(
from_tz(
cast(... (5 Replies)
Dear All,
I am trying to write a Unix Script which fires a sql query. The output of the sql query gives multiple rows. Each row should be saved in a separate Unix File.
The number of rows of sql output can be variable. I am able save all the rows in one file but in separate files.
Any... (14 Replies)
I need to extract the data from oracle table and written the below code.
But it is not working.There is some problem with the query and output is shown is No rows selected" . If I run the same query from sql developer there is my required output.
And if I run the shell script with simple sql... (7 Replies)
I have an issue with some fortran include files.
I have the following src directory structure,
trunk/src/client_main
trunk/src/client_models
trunk/src/server
The make file is at trunk/makefile.
In /src/client_main there are some included dependencies,
COMMON.BLK
PARAM.DAT... (0 Replies)
Hi,
I have a directory /home/datasets/ which contains a bunch (720) of subdirectories called hour_1/ hour_2/ etc..etc.. in each of these there is a single text file called (hour_1.txt in hour_1/ , hour_2.txt for hour_2/ etc..etc..) and i would like to do some text processing in them.
Each of... (20 Replies)
Hi,
I have a directory full of *.txt files. I would like to print the last line of every file to screen.
I know you can use FNR for printing the first line of each file, but how do I access the last line of each file?
This code doesn't work, it only prints the last line of the last file:BEGIN... (5 Replies)
Hi,
I'd like to process multiple files. For example:
file1.txt
file2.txt
file3.txt
Each file contains several lines of data. I want to extract a piece of data and output it to a new file.
file1.txt ----> newfile1.txt
file2.txt ----> newfile2.txt
file3.txt ----> newfile3.txt
Here is... (3 Replies)
Hi,
I'm using AWK to try to extract data from multiple files (*.txt). The script should look for a flag that occurs at a specific position in each file and it should return the data to the right of that flag.
I should end up with one line for each file, each containing 3 columns:... (8 Replies)
Hi guys,
say I have a few files in a directory (58 text files or somthing)
each one contains mulitple strings that I wish to replace with other strings
so in these 58 files I'm looking for say the following strings:
JAM (replace with BUTTER)
BREAD (replace with CRACKER)
SCOOP (replace... (19 Replies)
hello,
I will would be grateful if anyone can help me reply to my post
extract multiple cloumns from multiple files; skip rows and include filenames; awk
Please see this thread.
Thanks
manishabh (0 Replies)