Sponsored Content
Full Discussion: Column Extraction
Top Forums UNIX for Dummies Questions & Answers Column Extraction Post 302312227 by vgersh99 on Thursday 30th of April 2009 05:00:04 PM
Old 04-30-2009
a generic way to extract specified columns from the 'header' record.
Assuming gin.txt:
Code:
        E/N       Ko_exp   %err  Ko_calc  %err   diff  diff-  diff+  0.95
        ========  =======  ====  =======  ====  =====  =====  =====  ====
     1      4.00   2.8100   3.0   3.9502   0.5  -1.14  -1.31  -0.97    0
     2      8.00   2.8123   3.0   3.9668   0.5  -1.15  -1.32  -0.98    0
     3     12.00   2.8300   3.0   3.9920   0.5  -1.16  -1.33  -0.99    0
     4     16.00   2.8444   3.0   4.0201   0.5  -1.18  -1.35  -1.00    0
     5     20.00   2.8700   3.0   4.0473   0.5  -1.18  -1.35  -1.00    0
     6     24.00   2.9007   3.0   4.0668   0.5  -1.17  -1.34  -0.99    0
     7     28.00   2.9437   3.0   4.0807   0.5  -1.14  -1.31  -0.96    0
     8     32.00   2.9983   3.0   4.0833   0.5  -1.08  -1.27  -0.90    0
     9     36.00   3.0567   3.0   4.0778   0.5  -1.02  -1.21  -0.84    0
    10     40.00   3.1100   3.0   4.0656   0.5  -0.96  -1.14  -0.77    0
    11     44.00   3.1518   3.0   4.0452   0.5  -0.89  -1.08  -0.70    0
    12     48.00   3.1849   3.0   4.0209   0.5  -0.84  -1.03  -0.64    0
    13     52.00   3.2148   3.0   3.9924   0.5  -0.78  -0.97  -0.58    0
    14     56.00   3.2434   3.0   3.9619   0.5  -0.72  -0.91  -0.52    0
    15     60.00   3.2700   3.0   3.9269   0.5  -0.66  -0.85  -0.46    0
    16     64.00   3.2936   3.0   3.8904   0.5  -0.60  -0.79  -0.40    0
    17     68.00   3.3137   3.0   3.8491   0.5  -0.54  -0.73  -0.34    0
    18     72.00   3.3300   3.0   3.8136   0.5  -0.48  -0.68  -0.28    0
    19     76.00   3.3422   3.0   3.7762   0.5  -0.43  -0.63  -0.23    0
    20     80.00   3.3500   3.0   3.7357   0.5  -0.39  -0.59  -0.19    0
    21     84.00   3.3531   3.0   3.7083   0.5  -0.36  -0.56  -0.15    0
    22     88.00   3.3521   3.0   3.6682   0.5  -0.32  -0.52  -0.12    0

Code:
# to extract the "default" columns: 'E/N' and 'Ko_exp'
nawk -f gin.awk gin.txt

# to extract the specific columns: '%err' and 'diff-'
nawk -v colID='%err diff-' -f gin.awk gin.txt

gin.awk:
Code:
BEGIN {
  if (colID=="") colID="E/N Ko_exp";

  colNn=split(colID, colN, FS)
  for(i=1; i<=colNn; i++) {
     colN[colN[i]]
     delete colN[i]
  }

  FLDheader="1"
  FLDskip="2"
}

FNR == FLDheader {
   for(i=1; i <= NF; i++) {
      if ($i in colN) {
         colA[++col]=i
         printf("%s%c", $i, (i!=NF)?OFS:"")
      }
   }
   if (col != colNn) {
     printf("Warning: couldn't find column(s) [%s]\n", colID) | "cat 1>&2"
     exit 1;
   }
   printf("%c", ORS)
   next;
}

NR != FLDskip {
   for(i=1; i<=col; i++)
      printf("%s%c", $(colA[i]+1), (i==col)? ORS : OFS)
}

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Regex extraction

Hello, I need your help to extract text from following: ./sherg_fyd_rur:blkabl="R23.21_BL2008_0122_1" ./serge_a75:rlwual="/main/r23.21=26-Mar-2008.05:00:20UTC@R11.31_BL2008_0325" ./serge_a75:blkabl="R23.21_BL2008_0325" ./sherg_proto_npiv:bkguals="R23.21_BL2008_0302 I80_11.31_LR" I... (11 Replies)
Discussion started by: abdurrouf
11 Replies

2. UNIX for Dummies Questions & Answers

merged 10 files with column extraction into one

Hi, I have 600 text files. In each txt file, I have 3 columns, e.g: File 1 a 0.21 0.003 b 0.34 0.004 c 0.72 0.002 File 2 a 0.25 0.0083 b 0.38 0.0047 c 0.79 0.00234 File 3 a 0.45 0.0063 b 0.88 0.0027 c 0.29 0.00204 ... my filename as "sc2408_0_5278.txt sc2408_0_5279.txt... (2 Replies)
Discussion started by: libenhelen
2 Replies

3. Shell Programming and Scripting

Partial Column extraction/Process/Repasting changed Columns back to Source file

I have the following requirement. file1.txt (this could contain 5 million rows) ABC 1234 XYZ .... (3000 bytes) QRD 4612 GHT .... (3000 bytes) I need to create file2.txt 1234 4612 I have a EAI process to change file2.txt into file3.txt 4555 3743 Then I would have to use... (0 Replies)
Discussion started by: jostul
0 Replies

4. Shell Programming and Scripting

Changing one column of delimited file column to fixed width column

Hi, Iam new to unix. I have one input file . Input file : ID1~Name1~Place1 ID2~Name2~Place2 ID3~Name3~Place3 I need output such that only first column should change to fixed width column of 15 characters of length. Output File: ID1<<12 spaces>>Name1~Place1 ID2<<12... (5 Replies)
Discussion started by: manneni prakash
5 Replies

5. Shell Programming and Scripting

Match column 3 in file1 to column 1 in file 2 and replace with column 2 from file2

Match column 3 in file1 to column 1 in file 2 and replace with column 2 from file2 file 1 sample SNDK 80004C101 AT XLNX 983919101 BB NETL 64118B100 BS AMD 007903107 CC KLAC 482480100 DC TER 880770102 KATS ATHR 04743P108 KATS... (7 Replies)
Discussion started by: rydz00
7 Replies

6. Shell Programming and Scripting

extraction

I have following input @xxxxxx@ I want to extract what's between @....@ that is : xxxx using SED command (6 Replies)
Discussion started by: xerox
6 Replies

7. Shell Programming and Scripting

Column extraction from multiple files to multiple files

I have roughly ~30 .txt files in a directory which all have unique names. These files all contain text arranged in columns separated by whitespace (example file: [#YY MM DD hh mm WDIR WSPD GST WVHT DPD APD MWD PRES ATMP WTMP DEWP VIS TIDE #yr mo dy hr mn degT m/s m/s m sec ... (5 Replies)
Discussion started by: aozgaa
5 Replies

8. Shell Programming and Scripting

Difference of the same column when two other column matches and one column differs less than 1 hour

This is my input file : # cat list 20130430121600, cucm, location,76,2 20130430121600,cucm1,location1,76,4 20130430122000,cucm,location,80,8 20130430122000,cucm1,location1,90,8 20130430140000,cucm1,location1,87,11 20130430140000, cucm,location,67,9 This is the required output ... (1 Reply)
Discussion started by: Lakshmikumari
1 Replies

9. UNIX for Advanced & Expert Users

Column Extraction for a particular match

Hi, PFB the input: unix/java/perl/random.txt unix1/java1/random1.txt unix2/java2/perl2/random2.txt unix3/java3/random3.txt unix4/random4.txt i want the following output: random.txt random1.txt random2.txt random3.txt random4.txt the patterns can change but i need the .txt file... (5 Replies)
Discussion started by: arindam guha
5 Replies

10. UNIX for Beginners Questions & Answers

UNIX - 2 tab delimited files, conditional column extraction

Please know that I am very new to unix and trying to learn 'on the job'. I'm only manipulating large tab-delimited files (millions of rows), but I'm stuck and don't know how to proceed with the following. Hoping for some friendly advice :) I have 2 tab-delimited files - with differing column &... (10 Replies)
Discussion started by: GTed
10 Replies
RTBL(3) 						   BSD Library Functions Manual 						   RTBL(3)

NAME
rtbl_create, rtbl_destroy, rtbl_set_flags, rtbl_get_flags, rtbl_set_prefix, rtbl_set_separator, rtbl_set_column_prefix, rtbl_set_column_affix_by_id, rtbl_add_column, rtbl_add_column_by_id, rtbl_add_column_entry, rtbl_add_column_entry_by_id, rtbl_new_row, rtbl_format -- format data in simple tables LIBRARY
The roken library (libroken, -lroken) SYNOPSIS
#include <rtbl.h> int rtbl_add_column(rtbl_t table, const char *column_name, unsigned int flags); int rtbl_add_column_by_id(rtbl_t table, unsigned int column_id, const char *column_header, unsigned int flags); int rtbl_add_column_entry(rtbl_t table, const char *column_name, const char *cell_entry); int rtbl_add_column_entry_by_id(rtbl_t table, unsigned int column_id, const char *cell_entry); rtbl_t rtbl_create(void); void rtbl_destroy(rtbl_t table); int rtbl_new_row(rtbl_t table); int rtbl_set_column_affix_by_id(rtbl_t table, unsigned int column_id, const, char, *prefix", const char *suffix); int rtbl_set_column_prefix(rtbl_t table, const char *column_name, const char *prefix); unsigned int rtbl_get_flags(rtbl_t table); void rtbl_set_flags(rtbl_t table, unsigned int flags); int rtbl_set_prefix(rtbl_t table, const char *prefix); int rtbl_set_separator(rtbl_t table, const char *separator); int rtbl_format(rtbl_t table, FILE, *file"); DESCRIPTION
This set of functions assemble a simple table consisting of rows and columns, allowing it to be printed with certain options. Typical use would be output from tools such as ls(1) or netstat(1), where you have a fixed number of columns, but don't know the column widths before hand. A table is created with rtbl_create() and destroyed with rtbl_destroy(). Global flags on the table are set with rtbl_set_flags and retrieved with rtbl_get_flags. At present the only defined flag is RTBL_HEADER_STYLE_NONE which suppresses printing the header. Before adding data to the table, one or more columns need to be created. This would normally be done with rtbl_add_column_by_id(), column_id is any number of your choice (it's used only to identify columns), column_header is the header to print at the top of the column, and flags are flags specific to this column. Currently the only defined flag is RTBL_ALIGN_RIGHT, aligning column entries to the right. Columns are printed in the order they are added. There's also a way to add columns by column name with rtbl_add_column(), but this is less flexible (you need unique header names), and is considered deprecated. To add data to a column you use rtbl_add_column_entry_by_id(), where the column_id is the same as when the column was added (adding data to a non-existent column is undefined), and cell_entry is whatever string you wish to include in that cell. It should not include newlines. For columns added with rtbl_add_column() you must use rtbl_add_column_entry() instead. rtbl_new_row() fills all columns with blank entries until they all have the same number of rows. Each column can have a separate prefix and suffix, set with rtbl_set_column_affix_by_id; rtbl_set_column_prefix allows setting the prefix only by column name. In addition to this, columns may be separated by a string set with rtbl_set_separator (by default columns are not seprated by anything). The finished table is printed to file with rtbl_format. EXAMPLES
This program: #include <stdio.h> #include <rtbl.h> int main(int argc, char **argv) { rtbl_t table; table = rtbl_create(); rtbl_set_separator(table, " "); rtbl_add_column_by_id(table, 0, "Column A", 0); rtbl_add_column_by_id(table, 1, "Column B", RTBL_ALIGN_RIGHT); rtbl_add_column_by_id(table, 2, "Column C", 0); rtbl_add_column_entry_by_id(table, 0, "A-1"); rtbl_add_column_entry_by_id(table, 0, "A-2"); rtbl_add_column_entry_by_id(table, 0, "A-3"); rtbl_add_column_entry_by_id(table, 1, "B-1"); rtbl_add_column_entry_by_id(table, 2, "C-1"); rtbl_add_column_entry_by_id(table, 2, "C-2"); rtbl_add_column_entry_by_id(table, 1, "B-2"); rtbl_add_column_entry_by_id(table, 1, "B-3"); rtbl_add_column_entry_by_id(table, 2, "C-3"); rtbl_add_column_entry_by_id(table, 0, "A-4"); rtbl_new_row(table); rtbl_add_column_entry_by_id(table, 1, "B-4"); rtbl_new_row(table); rtbl_add_column_entry_by_id(table, 2, "C-4"); rtbl_new_row(table); rtbl_format(table, stdout); rtbl_destroy(table); return 0; } will output the following: Column A Column B Column C A-1 B-1 C-1 A-2 B-2 C-2 A-3 B-3 C-3 A-4 B-4 C-4 HEIMDAL
June 26, 2004 HEIMDAL
All times are GMT -4. The time now is 04:38 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy