Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Extract string from multiple file based on line count number

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Extract string from multiple file based on line count number
# 1  
Old 04-04-2011
Extract string from multiple file based on line count number
Hi,

I search all forum, but I can not find solutions of my problem Smilie
I have multiple files (5000 files), inside there is this data :

FILE 1:
Code:
 1195.921  -898.995 0.750312E-02-0.497526E-02 0.195382E-05 0.609417E-05
-2021.287  1305.479-0.819754E-02 0.107572E-01 0.313018E-05 0.885066E-05
    85.928 -1529.405 0.990965E-02 0.224840E-02-0.157472E-04 0.581893E-05

FILE 2:
Code:
  1228.633  -924.264 0.174728E-02-0.961339E-03-0.594874E-06 0.177402E-05
 -1988.820  1279.863-0.177465E-02 0.219633E-02 0.309343E-06 0.251814E-05
    118.121 -1554.893 0.216157E-02 0.612947E-03-0.354522E-05 0.183121E-05

FILE 3:
Code:
  1195.921  -898.995 0.750312E-02-0.497526E-02 0.195382E-05 0.609417E-05
 -2021.287  1305.479-0.819754E-02 0.107572E-01 0.313018E-05 0.885066E-05
     85.928 -1529.405 0.990965E-02 0.224840E-02-0.157472E-04 0.581893E-05

I need to extract this data so that result in another single file (call it result.txt) will be :
Code:
 0.750312E-02  0.174728E-02  0.750312E-02
-0.819754E-02 -0.177465E-02 -0.819754E-02
 0.990965E-02  0.216157E-02  0.990965E-02

I am truly newby with awk/perl,
Any helps are very much appriciated.

Gunk.
Last edited by Franklin52; 04-04-2011 at 09:14 AM.. Reason: Please use code tags
# 2  
Old 04-04-2011
From your file 1:
Quote:
1195.921 -898.995 0.750312E-02-0.497526E-02
Is the 3rd column really 0.750312E-02-0.497526E-02?
Or is there a space missing, so it should be 0.750312E-02 -0.497526E-02?

I see this is true of all your files, but the joined columns changed between rows. Is this correct?
# 3  
Old 04-04-2011
The file is just like it is.
Indeed there is no space, just continuously no.

The original data has the same start line no. (when I check in notepad),
For example,
the resulting 1st coulomb should started between linecount no 20 to 33 of the original file 1,
and the 2nd col. also linecount 20 to 33 of file 2,
and so on.. it is the same also for all files.

Need help please.
# 4  
Old 04-04-2011
Ok, your data are mess, the - with different meaning to make the troubles.

use below command to fix it first (if your sed support -i option)
Code:
sed -i 's/\([0-9]\)-/\1 -/g' file*

In this case, file1 will be converted to:

Code:
1195.921 -898.995 0.750312E-02 -0.497526E-02 0.195382E-05 0.609417E-05
-2021.287 1305.479 -0.819754E-02 0.107572E-01 0.313018E-05 0.885066E-05
85.928 -1529.405 0.990965E-02 0.224840E-02 -0.157472E-04 0.581893E-05

then run the awk command to get the result.
Code:
awk '{a[FNR]=a[FNR]?a[FNR] FS $3:$3}END{for (i=1;i in a;i++) print a[i]}' file*

0.750312E-02 0.174728E-02 0.750312E-02
-0.819754E-02 -0.177465E-02 -0.819754E-02
0.990965E-02 0.216157E-02 0.990965E-02

Of course, if your sed don't support -i option, I will give you solution later.Smilie
Last edited by rdcwayx; 04-04-2011 at 07:35 AM..
# 5  
Old 04-04-2011
Quote:
Originally Posted by rdcwayx
Ok, your data are mess, the - with different meaning to make the troubles.

use below command to fix it first (if your sed support -i option)
Code:
sed -i 's/\([0-9]\)-/\1 -/g' file*

...<snip>...
[/CODE]Of course, if your sed don't support -i option, I will give you solution later.Smilie
You can just fix that in the awk script. Fix the record and assign the result to $0. That will force recalculation of NF and reassignment to each field variable.

Regards,
Alister

---------- Post updated at 10:39 AM ---------- Previous update was at 09:54 AM ----------

The simplest way to do that is probably:
Code:
gsub(/E-/, "E"); gsub(/-/, " -"); gsub(/E/, "E-")

If using gawk, then I suppose gensub with backreferences can manage it in one stroke, like the sed suggestion above.

Regards,
Alister
# 6  
Old 04-04-2011
could this help you?
Code:
 perl -nle 'if(/(\-\d+|\d+)(\.|\d+)(\d+E-\d+)(-|\s+)/) {$i=$ARGV eq $prev?++$i:1;if(exists $hash{$i}){$hash{$i}=$hash{$i}." ".$1.$2.$3}else{$hash{$.}=$1.$2.$3;}$prev=$ARGV} END{print $hash{$_} foreach (sort(keys(%hash)))}' files*

# 7  
Old 04-04-2011
Modified a little bit rdcway's code, to format before to process input data.
Code:
awk '{gsub(/-/," -");a[FNR]=a[FNR]?a[FNR] FS $3:$3}END{for (i=1;i in a;i++) print a[i]}' FILE* # With gsub()

awk '{$0=gensub(/([0-9])-/,"\\1 -","g")
a[FNR]=a[FNR]?a[FNR] FS $3:$3}END{for (i=1;i in a;i++) print a[i]}' FILE* # More precise with gensub() back reference regex.

Hope this helps,

Regards
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Count number of occurrence of a string in file

if there's a file containing: money king money queen money cat money also money king all those strings are on one line in the file. how can i find out how many times "money king" shows up in the line? egrep -c "money king" wont work. (7 Replies)
Discussion started by: SkySmart
7 Replies

2. Shell Programming and Scripting

HELP: Shell Script to read a Log file line by line and extract Info based on KEYWORDS matching

I have a LOG file which looks like this Import started at: Mon Jul 23 02:13:01 EDT 2012 Initialization completed in 2.146 seconds. -------------------------------------------------------------------------------- -- Import summary for Import item: PolicyInformation... (8 Replies)
Discussion started by: biztank
8 Replies

3. Shell Programming and Scripting

Merge two non-consecutive lines based on line number or string

This is a variation of an earlier post found here: unixcom/shell-programming-scripting/159821-merge-two-non-consecutive-lines.html User Bartus11 was kind enough to solve that example. Previously, I needed help combining two lines that are non-consecutive in a file. Now I need to do the... (7 Replies)
Discussion started by: munkee
7 Replies

4. Shell Programming and Scripting

extract a line from a file by line number

Hi guys, does anyone know how to extract(grep) a line from the file, if I know the line number? Thanks a lot. (9 Replies)
Discussion started by: aoussenko
9 Replies

5. Shell Programming and Scripting

How to count number of occurances of string in a file?

Gurus, Need little guidance. I have A.txt and B.txt file. B.txt file contains Unique strings. Sample content of B.txt file for which i cut the fourth column uniquely and output directed to B.txt file And A.txt file contains the above string as a fourth column which is last column. So A.txt... (7 Replies)
Discussion started by: Shirisha
7 Replies

6. Shell Programming and Scripting

How to extract specific data and count number containing sets from a file?

Hello everybody! I am quit new here and hope you can help me. Using an awk script I am trying to extract data from several files. The structure of the input files is as follows: TimeStep parameter1 parameter2 parameter3 parameter4 e.g. 1 X Y Z L 1 D H Z I 1 H Y E W 2 D H G F 2 R... (2 Replies)
Discussion started by: Daniel8472
2 Replies

7. UNIX for Dummies Questions & Answers

Extract a specific number from an XML file based on the start and end tags

Hello People, I have the following contents in an XML file ........... ........... .......... ........... <Details = "Sample Details"> <Name>Bob</Name> <Age>34</Age> <Address>CA</Address> <ContactNumber>1234</ContactNumber> </Details> ........... ............. .............. (4 Replies)
Discussion started by: sushant172
4 Replies

8. Shell Programming and Scripting

extract a line from a file using the line number

Hello, I am having trouble extracting a specific line from a file when the line number is known. My first attempt involved grep -n 'hi' (the word 'hi will always be there) to get the line number before the line that I actually want (line 4). Extra Notes: -I am working in a bash script. -The... (7 Replies)
Discussion started by: grandtheftander
7 Replies

9. Shell Programming and Scripting

Extract a line from a file using the line number

I have a shell script and want to assign a value to a variable. The value is the line exctrated from a file using the line number. The line number it is not fix, and could change any time. I have tried sed, awk, head .. See my script # Get randome line number from the file #selectedline = `awk... (1 Reply)
Discussion started by: zambo
1 Replies

10. UNIX for Dummies Questions & Answers

count the number of files which have a search string, but counting the file only once

I need to count the number of files which have a search string, but counting the file only once if search string is found. eg: File1: Please note that there are 2 occurances of "aaa" aaa bbb ccc aaa File2: Please note that there are 3 occurances of "aaa" aaa bbb ccc... (1 Reply)
Discussion started by: sudheshnaiyer
1 Replies
Login or Register to Ask a Question