Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Using columns from 2 files and extracting string

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Using columns from 2 files and extracting string
# 8  
Old 10-20-2011
Thanks radoulov and vgersh99. I will process my 200MB files and let you guys know if I have issues.

---------- Post updated 10-20-11 at 02:10 PM ---------- Previous update was 10-19-11 at 05:07 PM ----------

Hi All,

The code is running with the test dataset but not with the big ones.
Is there a way I could share a portion of my actual input files ?
# 9  
Old 10-20-2011
Quote:
Originally Posted by alpesh
Thanks radoulov and vgersh99. I will process my 200MB files and let you guys know if I have issues.

---------- Post updated 10-20-11 at 02:10 PM ---------- Previous update was 10-19-11 at 05:07 PM ----------

Hi All,

The code is running with the test dataset but not with the big ones.
Is there a way I could share a portion of my actual input files ?
You can attach files to your postings - it's a paper clip icon on the first row of icons.
# 10  
Old 10-20-2011
Hi vgersh99 and radoulov,

I used the following code from radoulov but it returns me the contents of file 1 without adding the output column to it. Here $4 of file1 has to be matched with $1 of file2, $8 and $9 in file1 are the POS1 and POS2, $2 is the START column. $10 in file2 should be used to extract the sub-string.

Code:
awk 'NR == FNR {
  f2[$1] = $10; next
  }
$4 in f2 {
  print $0, substr(f2[$1], $8 - $2 , $9 - $8) 
  }' file2_truncated.txt file1_truncated.txt  | head

Thanks,
Alpesh
file1_truncated.txt (2.1 KB) 
file2_truncated.txt (292.0 KB) 
Last edited by alpesh; 10-20-2011 at 05:01 PM.. Reason: Include radoulov in the loop
# 11  
Old 10-20-2011
1. Execute dos2unix file2_truncated.txt file1_truncated.txt.
2. Try this and check the result:
Code:
awk 'NR == FNR {
  f2[$1] = $10; next
  }
$4 in f2 {
  print $0, substr(f2[$4], $8 - $2 , $9 - $8) 
  }' file2_truncated.txt file1_truncated.txt | 
    head

This User Gave Thanks to radoulov For This Post:
alpesh
# 12  
Old 10-20-2011
Thanks radoulov. Looks like I do not have dos2unix installed , I have to ask the admin to install.
Is there anything else I could try?
# 13  
Old 10-20-2011
Yes, this should do the trick:

Code:
perl -i -pe's/\r\n/\n/g' file2_truncated.txt file1_truncated.txt

This User Gave Thanks to radoulov For This Post:
alpesh
# 14  
Old 10-20-2011
Code:
tr -d '\015' < myFile > myNewFile

This User Gave Thanks to vgersh99 For This Post:
alpesh
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Joining files using awk not extracting all columns from File 2

Hello All I'm joining two files using Awk by Left outer join on the file 1 File 1 1 AA 2 BB 3 CC 4 DD File 2 1 IND 100 200 300 2 AUS 400 500 600 5 USA 700 800 900 (18 Replies)
Discussion started by: venkat_reddy
18 Replies

2. Shell Programming and Scripting

Extracting data from specific rows and columns from multiple csv files

I have a series of csv files in the following format eg file1 Experiment Name,XYZ_07/28/15, Specimen Name,Specimen_001, Tube Name, Control, Record Date,7/28/2015 14:50, $OP,XYZYZ, GUID,abc, Population,#Events,%Parent All Events,10500, P1,10071,95.9 Early Apoptosis,1113,11.1 Late... (6 Replies)
Discussion started by: pawannoel
6 Replies

3. Shell Programming and Scripting

Compare columns of multiple files and print those unique string from File1 in an output file.

Hi, I have multiple files that each contain one column of strings: File1: 123abc 456def 789ghi File2: 123abc 456def 891jkl File3: 234mno 123abc 456def In total I have 25 of these type of file. (5 Replies)
Discussion started by: owwow14
5 Replies

4. Shell Programming and Scripting

extracting columns falling within specific ranges for multiple files

Hi, I need to create weekly files from daily records stored in individual monthly filenames from 1999-2010. my sample file structure is like the ones below: daily record stored per month: 199901.xyz, 199902.xyz, 199903.xyz, 199904.xyz ...199912.xyz records inside 199901.xyz (original data... (4 Replies)
Discussion started by: ida1215
4 Replies

5. Shell Programming and Scripting

Extracting columns from multiple files with awk

hi everyone! I'd like to extract a single column from 5 different files and put them together in an output file. I saw a similar question for 2 input files, and the line of code workd very well, the code is: awk 'NR==FNR{a=$2; next} {print a, $2}' file1 file2 I added the file3, file4 and... (10 Replies)
Discussion started by: orcaja
10 Replies

6. UNIX for Dummies Questions & Answers

Extracting columns from multiple files with awk

hi everyone! I already posted it in scripts, I'm sorry, it's doubled I'd like to extract a single column from 5 different files and put them together in an output file. I saw a similar question for 2 input files, and the line of code workd very well, the code is: awk 'NR==FNR{a=$2; next}... (1 Reply)
Discussion started by: orcaja
1 Replies

7. Shell Programming and Scripting

extracting columns from 2 files

Hello, I have 2 files file1 & file2 = a1 b1 a2 b2 a3 b3 ... = c1 d1 c2 d2 c3 d3 ... I need to compare if b(i)=c(j) . i,j=1,2,3,4,... If yes, right a(i) d(j) in output file3 per line (1 Reply)
Discussion started by: newpromo
1 Replies

8. Shell Programming and Scripting

Append string to columns from 2 files

Hi Having a file as follows file1.txt Date (dd/mm)Time Server IP Error Code =========================================================================== 10/04/2008 10:10 ServerA xxx.xxx.xxx.xxx 6 10/04/2008 10:10 ServerB ... (3 Replies)
Discussion started by: karthikn7974
3 Replies

9. Shell Programming and Scripting

Extracting a string from one file and searching the same string in other files

Hi, Need to extract a string from one file and search the same in other files. Ex: I have file1 of hundred lines with no delimiters not even space. I have 3 more files. I should get 1 to 10 characters say substring from each line of file1 and search that string in rest of the files and get... (1 Reply)
Discussion started by: mohancrr
1 Replies

10. UNIX for Dummies Questions & Answers

Extracting columns from different files for later merging

Hello! I wan't to extract columns from two files and later combine them for plotting with gnuplot. If the files file1 and file2 look like: fiile1: a, 0.62,x b, 0.61,x file2: a, 0.43,x b, 0,49,x The desired output is a 0.62 0.62 b 0.61 0.49 Thank you in advance! (2 Replies)
Discussion started by: kingkong
2 Replies
Login or Register to Ask a Question