Column Extraction


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Column Extraction
# 1  
Old 04-30-2009
Column Extraction

Hello all,

I have attached a txt doc with data sorted into columns (it is best to open with "word pad" to maintain the correct format. In the data there are two columns E/N and Ko. I wanted to know how to extract the data and form an excel sheet. Or at least just extract the data. The data is in much larger files so I would need to some how search for the E/N and Ko columns and extract the data underneath them for approximatly 100 rows skipping the first row. Let me know if that makes sense. Thanks for the help in advance. Lenny
# 2  
Old 04-30-2009
Adjust the 4 if you have less lines above the data:

Code:
awk 'NR > 4{print $2, $3}' OFS="," file > file.csv

# 3  
Old 04-30-2009
awesome is there a way to specify the row at which the columns begin and end. thank you
# 4  
Old 04-30-2009
a generic way to extract specified columns from the 'header' record.
Assuming gin.txt:
Code:
        E/N       Ko_exp   %err  Ko_calc  %err   diff  diff-  diff+  0.95
        ========  =======  ====  =======  ====  =====  =====  =====  ====
     1      4.00   2.8100   3.0   3.9502   0.5  -1.14  -1.31  -0.97    0
     2      8.00   2.8123   3.0   3.9668   0.5  -1.15  -1.32  -0.98    0
     3     12.00   2.8300   3.0   3.9920   0.5  -1.16  -1.33  -0.99    0
     4     16.00   2.8444   3.0   4.0201   0.5  -1.18  -1.35  -1.00    0
     5     20.00   2.8700   3.0   4.0473   0.5  -1.18  -1.35  -1.00    0
     6     24.00   2.9007   3.0   4.0668   0.5  -1.17  -1.34  -0.99    0
     7     28.00   2.9437   3.0   4.0807   0.5  -1.14  -1.31  -0.96    0
     8     32.00   2.9983   3.0   4.0833   0.5  -1.08  -1.27  -0.90    0
     9     36.00   3.0567   3.0   4.0778   0.5  -1.02  -1.21  -0.84    0
    10     40.00   3.1100   3.0   4.0656   0.5  -0.96  -1.14  -0.77    0
    11     44.00   3.1518   3.0   4.0452   0.5  -0.89  -1.08  -0.70    0
    12     48.00   3.1849   3.0   4.0209   0.5  -0.84  -1.03  -0.64    0
    13     52.00   3.2148   3.0   3.9924   0.5  -0.78  -0.97  -0.58    0
    14     56.00   3.2434   3.0   3.9619   0.5  -0.72  -0.91  -0.52    0
    15     60.00   3.2700   3.0   3.9269   0.5  -0.66  -0.85  -0.46    0
    16     64.00   3.2936   3.0   3.8904   0.5  -0.60  -0.79  -0.40    0
    17     68.00   3.3137   3.0   3.8491   0.5  -0.54  -0.73  -0.34    0
    18     72.00   3.3300   3.0   3.8136   0.5  -0.48  -0.68  -0.28    0
    19     76.00   3.3422   3.0   3.7762   0.5  -0.43  -0.63  -0.23    0
    20     80.00   3.3500   3.0   3.7357   0.5  -0.39  -0.59  -0.19    0
    21     84.00   3.3531   3.0   3.7083   0.5  -0.36  -0.56  -0.15    0
    22     88.00   3.3521   3.0   3.6682   0.5  -0.32  -0.52  -0.12    0

Code:
# to extract the "default" columns: 'E/N' and 'Ko_exp'
nawk -f gin.awk gin.txt

# to extract the specific columns: '%err' and 'diff-'
nawk -v colID='%err diff-' -f gin.awk gin.txt

gin.awk:
Code:
BEGIN {
  if (colID=="") colID="E/N Ko_exp";

  colNn=split(colID, colN, FS)
  for(i=1; i<=colNn; i++) {
     colN[colN[i]]
     delete colN[i]
  }

  FLDheader="1"
  FLDskip="2"
}

FNR == FLDheader {
   for(i=1; i <= NF; i++) {
      if ($i in colN) {
         colA[++col]=i
         printf("%s%c", $i, (i!=NF)?OFS:"")
      }
   }
   if (col != colNn) {
     printf("Warning: couldn't find column(s) [%s]\n", colID) | "cat 1>&2"
     exit 1;
   }
   printf("%c", ORS)
   next;
}

NR != FLDskip {
   for(i=1; i<=col; i++)
      printf("%s%c", $(colA[i]+1), (i==col)? ORS : OFS)
}

# 5  
Old 04-30-2009
To print the rows between 5 and 100:

Code:
awk 'NR > 4 && NR < 101 {print $2, $3}' OFS="," file > file.csv

 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

UNIX - 2 tab delimited files, conditional column extraction

Please know that I am very new to unix and trying to learn 'on the job'. I'm only manipulating large tab-delimited files (millions of rows), but I'm stuck and don't know how to proceed with the following. Hoping for some friendly advice :) I have 2 tab-delimited files - with differing column &... (10 Replies)
Discussion started by: GTed
10 Replies

2. UNIX for Advanced & Expert Users

Column Extraction for a particular match

Hi, PFB the input: unix/java/perl/random.txt unix1/java1/random1.txt unix2/java2/perl2/random2.txt unix3/java3/random3.txt unix4/random4.txt i want the following output: random.txt random1.txt random2.txt random3.txt random4.txt the patterns can change but i need the .txt file... (5 Replies)
Discussion started by: arindam guha
5 Replies

3. Shell Programming and Scripting

Difference of the same column when two other column matches and one column differs less than 1 hour

This is my input file : # cat list 20130430121600, cucm, location,76,2 20130430121600,cucm1,location1,76,4 20130430122000,cucm,location,80,8 20130430122000,cucm1,location1,90,8 20130430140000,cucm1,location1,87,11 20130430140000, cucm,location,67,9 This is the required output ... (1 Reply)
Discussion started by: Lakshmikumari
1 Replies

4. Shell Programming and Scripting

Column extraction from multiple files to multiple files

I have roughly ~30 .txt files in a directory which all have unique names. These files all contain text arranged in columns separated by whitespace (example file: [#YY MM DD hh mm WDIR WSPD GST WVHT DPD APD MWD PRES ATMP WTMP DEWP VIS TIDE #yr mo dy hr mn degT m/s m/s m sec ... (5 Replies)
Discussion started by: aozgaa
5 Replies

5. Shell Programming and Scripting

extraction

I have following input @xxxxxx@ I want to extract what's between @....@ that is : xxxx using SED command (6 Replies)
Discussion started by: xerox
6 Replies

6. Shell Programming and Scripting

Match column 3 in file1 to column 1 in file 2 and replace with column 2 from file2

Match column 3 in file1 to column 1 in file 2 and replace with column 2 from file2 file 1 sample SNDK 80004C101 AT XLNX 983919101 BB NETL 64118B100 BS AMD 007903107 CC KLAC 482480100 DC TER 880770102 KATS ATHR 04743P108 KATS... (7 Replies)
Discussion started by: rydz00
7 Replies

7. Shell Programming and Scripting

Changing one column of delimited file column to fixed width column

Hi, Iam new to unix. I have one input file . Input file : ID1~Name1~Place1 ID2~Name2~Place2 ID3~Name3~Place3 I need output such that only first column should change to fixed width column of 15 characters of length. Output File: ID1<<12 spaces>>Name1~Place1 ID2<<12... (5 Replies)
Discussion started by: manneni prakash
5 Replies

8. Shell Programming and Scripting

Partial Column extraction/Process/Repasting changed Columns back to Source file

I have the following requirement. file1.txt (this could contain 5 million rows) ABC 1234 XYZ .... (3000 bytes) QRD 4612 GHT .... (3000 bytes) I need to create file2.txt 1234 4612 I have a EAI process to change file2.txt into file3.txt 4555 3743 Then I would have to use... (0 Replies)
Discussion started by: jostul
0 Replies

9. UNIX for Dummies Questions & Answers

merged 10 files with column extraction into one

Hi, I have 600 text files. In each txt file, I have 3 columns, e.g: File 1 a 0.21 0.003 b 0.34 0.004 c 0.72 0.002 File 2 a 0.25 0.0083 b 0.38 0.0047 c 0.79 0.00234 File 3 a 0.45 0.0063 b 0.88 0.0027 c 0.29 0.00204 ... my filename as "sc2408_0_5278.txt sc2408_0_5279.txt... (2 Replies)
Discussion started by: libenhelen
2 Replies

10. Shell Programming and Scripting

Regex extraction

Hello, I need your help to extract text from following: ./sherg_fyd_rur:blkabl="R23.21_BL2008_0122_1" ./serge_a75:rlwual="/main/r23.21=26-Mar-2008.05:00:20UTC@R11.31_BL2008_0325" ./serge_a75:blkabl="R23.21_BL2008_0325" ./sherg_proto_npiv:bkguals="R23.21_BL2008_0302 I80_11.31_LR" I... (11 Replies)
Discussion started by: abdurrouf
11 Replies
Login or Register to Ask a Question