Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Removing first column using awk script

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Removing first column using awk script
# 1  
Old 09-23-2010
Removing first column using awk script
I have this awk script. I have two different methods to remove the first column. However defining method = 1 in the BEGIN produces different results than when defining method = 1 later on instead. Any idea what it might be doing differently??? Smilie

Code:
  BEGIN {

    Version = "V05"
    if ( ARGV[ARGC-1] == "-help" ) {
      usage(Version)
      exit 1
    }

    if ( !xi ) {
      xi = 0
      zi = -0.5
    } else {
      split(xi, a, "/")
      xi = a[1]
      zi = a[2]
    }

    if ( !xf ) {
      xf = 80
      zf = 30
    } else {
      split(xf, a, "/")
      xf = a[1]
      zf = a[2]
    }

    ARGV[ARGC++] = ARGV[ARGC-1]

# Need the method variable here
#    method = 1
  }

# First time round, store source locations
  FNR == NR { i = NR; next }

# 3. Print the header and starting sound speed tags
  FNR == 1 {
    print "% XI=("xi","zi")\n% XF=("xf","zf")\n"
    print "%< LAYER 1\n"
    print "% INTI=LIN\n%( INTERFACE\n0.0 0.0\n"xf" 0.0\n%)\n"
    print "% PS=OFF"
    print "% INTP=LIN\n%( MODELP"
  }

# 4. Two methods to remove the first column from the output
  method = 1
  $1 <= zf && method == 1 { $1 = ""; $0 = $0; $1 = $1; print }
  $1 <= zf && method == 2 { $1 = ""; sub(/^[ \t]+/, ""); print }

# 5. Print the closing tags
  FNR == i {
    print "%)\n\n%>"
  }



---------- Post updated at 08:36 AM ---------- Previous update was at 08:34 AM ----------

Input file looks something like this:


Code:
-0.5 0.41180 0.41180 0.41180 0.41180 0.405623 0.405623 0.399446 0.399446 0.399446 0.399446 0.399446 0.405623 0.405623 0.41180 0.41180 0.41180 0.41180
0.0 0.47355 0.47355 0.47355 0.466447 0.459344 0.45224 0.445137 0.445137 0.445137 0.445137 0.445137 0.45224 0.459344 0.466447 0.47355 0.47355 0.47355
0.5 0.61679 0.61679 0.61679 0.607538 0.598286 0.589034 0.579783 0.579783 0.579783 0.579783 0.579783 0.589034 0.598286 0.607538 0.61679 0.61679 0.61679
1.0 0.70957 0.70957 0.70957 0.698926 0.688283 0.677639 0.666996 0.666996 0.666996 0.666996 0.666996 0.677639 0.688283 0.698926 0.70957 0.70957 0.70957
1.5 0.78973 0.78973 0.78973 0.777884 0.766038 0.754192 0.742346 0.742346 0.742346 0.742346 0.742346 0.754192 0.766038 0.777884 0.78973 0.78973 0.78973
2.0 0.86628 0.86628 0.86628 0.853286 0.840292 0.827297 0.814303 0.814303 0.814303 0.814303 0.814303 0.827297 0.840292 0.853286 0.86628 0.86628 0.86628
2.5 0.94252 0.94252 0.94252 0.928382 0.914244 0.900107 0.885969 0.885969 0.885969 0.885969 0.885969 0.900107 0.914244 0.928382 0.94252 0.94252 0.94252
3.0 1.02012 1.02012 1.02012 1.00482 0.989516 0.974215 0.958913 0.958913 0.958913 0.958913 0.958913 0.974215 0.989516 1.00482 1.02012 1.02012 1.02012
3.5 1.09985 1.09985 1.09985 1.08335 1.06685 1.05036 1.03386 1.03386 1.03386 1.03386 1.03386 1.05036 1.06685 1.08335 1.09985 1.09985 1.09985
4.0 1.18132 1.18132 1.18132 1.1636 1.14588 1.12816 1.11044 1.11044 1.11044 1.11044 1.11044 1.12816 1.14588 1.1636 1.18132 1.18132 1.18132
4.5 1.26366 1.26366 1.26366 1.24471 1.22575 1.2068 1.18784 1.18784 1.18784 1.18784 1.18784 1.2068 1.22575 1.24471 1.26366 1.26366 1.26366
5.0 1.34708 1.34708 1.34708 1.32687 1.30667 1.28646 1.26626 1.26626 1.26626 1.26626 1.26626 1.28646 1.30667 1.32687 1.34708 1.34708 1.34708
5.5 1.43177 1.43177 1.43177 1.41029 1.38882 1.36734 1.34586 1.34586 1.34586 1.34586 1.34586 1.36734 1.38882 1.41029 1.43177 1.43177 1.43177
6.0 1.52082 1.52082 1.52082 1.49801 1.4752 1.45238 1.42957 1.42957 1.42957 1.42957 1.42957 1.45238 1.4752 1.49801 1.52082 1.52082 1.52082
6.5 1.61045 1.61045 1.61045 1.58629 1.56214 1.53798 1.51382 1.51382 1.51382 1.51382 1.51382 1.53798 1.56214 1.58629 1.61045 1.61045 1.61045
7.0 1.70159 1.70159 1.70159 1.67607 1.65054 1.62502 1.59949 1.59949 1.59949 1.59949 1.59949 1.62502 1.65054 1.67607 1.70159 1.70159 1.70159
7.5 1.79329 1.79329 1.79329 1.76639 1.73949 1.71259 1.68569 1.68569 1.68569 1.68569 1.68569 1.71259 1.73949 1.76639 1.79329 1.79329 1.79329
8.0 1.88292 1.88292 1.88292 1.85468 1.82643 1.79819 1.76994 1.76994 1.76994 1.76994 1.76994 1.79819 1.82643 1.85468 1.88292 1.88292 1.88292
8.5 1.96947 1.96947 1.96947 1.93993 1.91039 1.88084 1.8513 1.8513 1.8513 1.8513 1.8513 1.88084 1.91039 1.93993 1.96947 1.96947 1.96947
9.0 2.05162 2.05162 2.05162 2.02085 1.99007 1.9593 1.92852 1.92852 1.92852 1.92852 1.92852 1.9593 1.99007 2.02085 2.05162 2.05162 2.05162
9.5 2.12917 2.12917 2.12917 2.09723 2.06529 2.03336 2.00142 2.00142 2.00142 2.00142 2.00142 2.03336 2.06529 2.09723 2.12917 2.12917 2.12917
10.0 2.20167 2.20167 2.20167 2.16864 2.13562 2.10259 2.06957 2.06957 2.06957 2.06957 2.06957 2.10259 2.13562 2.16864 2.20167 2.20167 2.20167
10.5 2.26967 2.26967 2.26967 2.26967 2.23562 2.23562 2.20158 2.20158 2.20158 2.20158 2.20158 2.23562 2.23562 2.26967 2.26967 2.26967 2.26967



---------- Post updated at 08:39 AM ---------- Previous update was at 08:36 AM ----------

Having method = 1 in the body of the script gives the file below.

Notice how I get things replicated, once still having the first column and the next is without the first column.

I just need to print the line without the first column only

Code:
% XI=(0,-0.5)
% XF=(80,30)

%< LAYER 1

% INTI=LIN
%( INTERFACE
0.0 0.0
80 0.0
%)

% PS=OFF
% INTP=LIN
%( MODELP
-0.5 0.41180 0.41180 0.41180 0.41180 0.405623 0.405623 0.399446 0.399446 0.399446 0.399446 0.399446 0.405623 0.405623 0.41180 0.41180 0.41180 0.41180
0.41180 0.41180 0.41180 0.41180 0.405623 0.405623 0.399446 0.399446 0.399446 0.399446 0.399446 0.405623 0.405623 0.41180 0.41180 0.41180 0.41180
0.0 0.47355 0.47355 0.47355 0.466447 0.459344 0.45224 0.445137 0.445137 0.445137 0.445137 0.445137 0.45224 0.459344 0.466447 0.47355 0.47355 0.47355
0.47355 0.47355 0.47355 0.466447 0.459344 0.45224 0.445137 0.445137 0.445137 0.445137 0.445137 0.45224 0.459344 0.466447 0.47355 0.47355 0.47355
0.5 0.61679 0.61679 0.61679 0.607538 0.598286 0.589034 0.579783 0.579783 0.579783 0.579783 0.579783 0.589034 0.598286 0.607538 0.61679 0.61679 0.61679
0.61679 0.61679 0.61679 0.607538 0.598286 0.589034 0.579783 0.579783 0.579783 0.579783 0.579783 0.589034 0.598286 0.607538 0.61679 0.61679 0.61679
1.0 0.70957 0.70957 0.70957 0.698926 0.688283 0.677639 0.666996 0.666996 0.666996 0.666996 0.666996 0.677639 0.688283 0.698926 0.70957 0.70957 0.70957
0.70957 0.70957 0.70957 0.698926 0.688283 0.677639 0.666996 0.666996 0.666996 0.666996 0.666996 0.677639 0.688283 0.698926 0.70957 0.70957 0.70957
1.5 0.78973 0.78973 0.78973 0.777884 0.766038 0.754192 0.742346 0.742346 0.742346 0.742346 0.742346 0.754192 0.766038 0.777884 0.78973 0.78973 0.78973
0.78973 0.78973 0.78973 0.777884 0.766038 0.754192 0.742346 0.742346 0.742346 0.742346 0.742346 0.754192 0.766038 0.777884 0.78973 0.78973 0.78973
2.0 0.86628 0.86628 0.86628 0.853286 0.840292 0.827297 0.814303 0.814303 0.814303 0.814303 0.814303 0.827297 0.840292 0.853286 0.86628 0.86628 0.86628
0.86628 0.86628 0.86628 0.853286 0.840292 0.827297 0.814303 0.814303 0.814303 0.814303 0.814303 0.827297 0.840292 0.853286 0.86628 0.86628 0.86628
2.5 0.94252 0.94252 0.94252 0.928382 0.914244 0.900107 0.885969 0.885969 0.885969 0.885969 0.885969 0.900107 0.914244 0.928382 0.94252 0.94252 0.94252
0.94252 0.94252 0.94252 0.928382 0.914244 0.900107 0.885969 0.885969 0.885969 0.885969 0.885969 0.900107 0.914244 0.928382 0.94252 0.94252 0.94252
3.0 1.02012 1.02012 1.02012 1.00482 0.989516 0.974215 0.958913 0.958913 0.958913 0.958913 0.958913 0.974215 0.989516 1.00482 1.02012 1.02012 1.02012
1.02012 1.02012 1.02012 1.00482 0.989516 0.974215 0.958913 0.958913 0.958913 0.958913 0.958913 0.974215 0.989516 1.00482 1.02012 1.02012 1.02012
3.5 1.09985 1.09985 1.09985 1.08335 1.06685 1.05036 1.03386 1.03386 1.03386 1.03386 1.03386 1.05036 1.06685 1.08335 1.09985 1.09985 1.09985
1.09985 1.09985 1.09985 1.08335 1.06685 1.05036 1.03386 1.03386 1.03386 1.03386 1.03386 1.05036 1.06685 1.08335 1.09985 1.09985 1.09985
4.0 1.18132 1.18132 1.18132 1.1636 1.14588 1.12816 1.11044 1.11044 1.11044 1.11044 1.11044 1.12816 1.14588 1.1636 1.18132 1.18132 1.18132
1.18132 1.18132 1.18132 1.1636 1.14588 1.12816 1.11044 1.11044 1.11044 1.11044 1.11044 1.12816 1.14588 1.1636 1.18132 1.18132 1.18132
4.5 1.26366 1.26366 1.26366 1.24471 1.22575 1.2068 1.18784 1.18784 1.18784 1.18784 1.18784 1.2068 1.22575 1.24471 1.26366 1.26366 1.26366
1.26366 1.26366 1.26366 1.24471 1.22575 1.2068 1.18784 1.18784 1.18784 1.18784 1.18784 1.2068 1.22575 1.24471 1.26366 1.26366 1.26366
5.0 1.34708 1.34708 1.34708 1.32687 1.30667 1.28646 1.26626 1.26626 1.26626 1.26626 1.26626 1.28646 1.30667 1.32687 1.34708 1.34708 1.34708
1.34708 1.34708 1.34708 1.32687 1.30667 1.28646 1.26626 1.26626 1.26626 1.26626 1.26626 1.28646 1.30667 1.32687 1.34708 1.34708 1.34708
5.5 1.43177 1.43177 1.43177 1.41029 1.38882 1.36734 1.34586 1.34586 1.34586 1.34586 1.34586 1.36734 1.38882 1.41029 1.43177 1.43177 1.43177
1.43177 1.43177 1.43177 1.41029 1.38882 1.36734 1.34586 1.34586 1.34586 1.34586 1.34586 1.36734 1.38882 1.41029 1.43177 1.43177 1.43177
6.0 1.52082 1.52082 1.52082 1.49801 1.4752 1.45238 1.42957 1.42957 1.42957 1.42957 1.42957 1.45238 1.4752 1.49801 1.52082 1.52082 1.52082
1.52082 1.52082 1.52082 1.49801 1.4752 1.45238 1.42957 1.42957 1.42957 1.42957 1.42957 1.45238 1.4752 1.49801 1.52082 1.52082 1.52082
6.5 1.61045 1.61045 1.61045 1.58629 1.56214 1.53798 1.51382 1.51382 1.51382 1.51382 1.51382 1.53798 1.56214 1.58629 1.61045 1.61045 1.61045
1.61045 1.61045 1.61045 1.58629 1.56214 1.53798 1.51382 1.51382 1.51382 1.51382 1.51382 1.53798 1.56214 1.58629 1.61045 1.61045 1.61045
7.0 1.70159 1.70159 1.70159 1.67607 1.65054 1.62502 1.59949 1.59949 1.59949 1.59949 1.59949 1.62502 1.65054 1.67607 1.70159 1.70159 1.70159
1.70159 1.70159 1.70159 1.67607 1.65054 1.62502 1.59949 1.59949 1.59949 1.59949 1.59949 1.62502 1.65054 1.67607 1.70159 1.70159 1.70159
7.5 1.79329 1.79329 1.79329 1.76639 1.73949 1.71259 1.68569 1.68569 1.68569 1.68569 1.68569 1.71259 1.73949 1.76639 1.79329 1.79329 1.79329
1.79329 1.79329 1.79329 1.76639 1.73949 1.71259 1.68569 1.68569 1.68569 1.68569 1.68569 1.71259 1.73949 1.76639 1.79329 1.79329 1.79329
8.0 1.88292 1.88292 1.88292 1.85468 1.82643 1.79819 1.76994 1.76994 1.76994 1.76994 1.76994 1.79819 1.82643 1.85468 1.88292 1.88292 1.88292
1.88292 1.88292 1.88292 1.85468 1.82643 1.79819 1.76994 1.76994 1.76994 1.76994 1.76994 1.79819 1.82643 1.85468 1.88292 1.88292 1.88292
8.5 1.96947 1.96947 1.96947 1.93993 1.91039 1.88084 1.8513 1.8513 1.8513 1.8513 1.8513 1.88084 1.91039 1.93993 1.96947 1.96947 1.96947
1.96947 1.96947 1.96947 1.93993 1.91039 1.88084 1.8513 1.8513 1.8513 1.8513 1.8513 1.88084 1.91039 1.93993 1.96947 1.96947 1.96947
9.0 2.05162 2.05162 2.05162 2.02085 1.99007 1.9593 1.92852 1.92852 1.92852 1.92852 1.92852 1.9593 1.99007 2.02085 2.05162 2.05162 2.05162
2.05162 2.05162 2.05162 2.02085 1.99007 1.9593 1.92852 1.92852 1.92852 1.92852 1.92852 1.9593 1.99007 2.02085 2.05162 2.05162 2.05162
9.5 2.12917 2.12917 2.12917 2.09723 2.06529 2.03336 2.00142 2.00142 2.00142 2.00142 2.00142 2.03336 2.06529 2.09723 2.12917 2.12917 2.12917
2.12917 2.12917 2.12917 2.09723 2.06529 2.03336 2.00142 2.00142 2.00142 2.00142 2.00142 2.03336 2.06529 2.09723 2.12917 2.12917 2.12917
10.0 2.20167 2.20167 2.20167 2.16864 2.13562 2.10259 2.06957 2.06957 2.06957 2.06957 2.06957 2.10259 2.13562 2.16864 2.20167 2.20167 2.20167
2.20167 2.20167 2.20167 2.16864 2.13562 2.10259 2.06957 2.06957 2.06957 2.06957 2.06957 2.10259 2.13562 2.16864 2.20167 2.20167 2.20167

# 2  
Old 09-23-2010
Quote:
Originally Posted by kristinu
I just need to print the line without the first column only
Is this solution working for you ?
Code:
awk '{sub($1FS,x)}1' file

yours look too complicated.
# 3  
Old 09-23-2010
I agree with danmero, go with the logic that is a bit more straight forward. Yes, you need the header/trailer setup, and to establish your xi/xf variables, but the simple statement offered should do the trick. You could write it like this to make it a bit less 'magical'

Code:
$1 <= zf  {
      sub( $1 FS, "" );   
      print;         
}

I prefer to use a null string ("") instead of assuming that a variable (x in the example) isn't set. I also think it is easier to understand if $1 and FS are separated by a space. Including a print statement inside of the block is necessary, unlike the other example, as you want to print only when the condition that $1 is less than zf; placing the 1 after the block (impled {print}) would print all lines removing the first column only from the lines where the condition was true.

The reason for the duplicated lines is how/where you coded the assignment of method. Awk programmes consist of a basic syntax that is:

Code:
pattern { action }

When you coded 'method = 1' with no action awk assumes a default action of print, so what you really had was:

Code:
method = 1 { print; }

which results in method being assigned 1 and evaluating to true, so the default action was taken. Thus the input line was printed before you drop the first column.

If you need to assign a variable like this, then do it in the BEGIN block.

Hope this helps to explain why you were seeing the duplicated lines in the output.
# 4  
Old 09-24-2010
Quote:
Originally Posted by danmero
Is this solution working for you ?
Code:
awk '{sub($1FS,x)}1' file

yours look too complicated.
Totally agree.

@kristinu

You have provided the input file, can you paste the expect output?

Maybe we can give simple solution.

Code:
awk '{$1=""}1' infile
cut -d " " -f2- < infile

above commands both clean the first column.
# 5  
Old 09-24-2010
But the OP's question was about a small part of a larger awk script, not about command line suggestions.

@OP: I tried both your ways of removing the first column and they gave identical results, so why do you need two?. I would replace them with danmero's solution, but I agree with agama that relying on unitialized variables is better avoided in non-oneliner script, so I would also use "" instead of x in this case..

---------- Post updated at 09:11 ---------- Previous update was at 08:48 ----------

If you want to set the variable method lower in the script, you need to do it like this:
Code:
# 4. Two methods to remove the first column from the output
  { method = 1 }

Then you will get identical results..
Last edited by Scrutinizer; 09-24-2010 at 04:43 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Script to do column to row in awk

Hi , Can anyone help me suggesting - how to do the below trick with awk Input 120 130 140 210 310 410 645 729 800 Output 120 130 140 (6 Replies)
Discussion started by: Indra2011
6 Replies

2. Shell Programming and Scripting

Need awk or Shell script to compare Column-1 of two different CSV files and print if column-1 matche

Example: I have files in below format file 1: zxc,133,joe@example.com cst,222,xyz@example1.com File 2 Contains: hxd hcd jws zxc cst File 1 has 50000 lines and file 2 has around 30000 lines : Expected Output has to be : hxd hcd jws (5 Replies)
Discussion started by: TestPractice
5 Replies

3. Shell Programming and Scripting

awk script to append suffix to column when column has duplicated values

Please help me to get required output for both scenario 1 and scenario 2 and need separate code for both scenario 1 and scenario 2 Scenario 1 i need to do below changes only when column1 is CR and column3 has duplicates rows/values. This inputfile can contain 100 of this duplicated rows of... (1 Reply)
Discussion started by: as7951
1 Replies

4. Shell Programming and Scripting

awk script row to column

Hi.. I have data : Report testing1 20180419 08:00 Report testing2 20180419 07:35 Report testing 20180419 08:01 Source = data1 Report testing4 20180419 08:05 Source = data1 Report testing5 20180419 08:10 Source = data2 Report testing6 20180419 08:01 Report testing7 20180419 08:19... (4 Replies)
Discussion started by: buncit8
4 Replies

5. Shell Programming and Scripting

awk script to call another script based on second column entry

Hi I have a text file (Input.txt) with two column entries separated by tab as given below: aaa str1 bbb str2 cccccc str3 dddd str4 eee str3 ssss str2 sdf str3 hhh str1 fff str2 ccc str3 ..... ..... ..... (1 Reply)
Discussion started by: my_Perl
1 Replies

6. Shell Programming and Scripting

How to use regex on particular column (Removing comma from particular column)?

Hi, I have pipe separated file which contains some data having comma(,) in it. I want to remove the comma(,) only from particular column without changing data in other columns. Below is the sample data file, I want to remove the comma(,) only from 5th column. $ cat file1 ABC | DEF, HIJ|... (6 Replies)
Discussion started by: Prathmesh
6 Replies

7. Shell Programming and Scripting

awk Script: removing periodic boundaries

SOLVED, thank you! Edit2: Good news everyone, I managed to get it down to a "simple" problem, but I still have some syntax issues. Here is the code which troubles me: awk 'BEGIN{x2=0;x1=0;crit=0;} $1 < 1000000 {x2=$4; diffx=x2-x1; x1=x2; diffx > 3.6 ? {crit=1} : {crit=0};... (2 Replies)
Discussion started by: Consti
2 Replies

8. Shell Programming and Scripting

AWK script to create max value of 3rd column, grouping by first column

Hi, I need an awk script (or whatever shell-construct) that would take data like below and get the max value of 3 column, when grouping by the 1st column. clientname,day-of-month,max-users ----------------------------------- client1,20120610,5 client2,20120610,2 client3,20120610,7... (3 Replies)
Discussion started by: ckmehta
3 Replies

9. Linux

Need awk script for removing duplicate records

I have log file having Traffic line 2011-05-21 15:11:50.356599 TCP (6), length: 52) 10.10.10.1.3020 > 10.10.10.254.50404: 2011-05-21 15:11:50.652739 TCP (6), length: 52) 10.10.10.254.50404 > 10.10.10.1.3020: 2011-05-21 15:11:50.652558 TCP (6), length: 89) 10.10.10.1.3020 >... (1 Reply)
Discussion started by: Rastamed
1 Replies

10. Linux

Need awk script for removing duplicate records

I have huge txt file having millions of trade data. For e.g Trade.txt (first 8 lines in the file is header info) COB_DATE,TRADE_ID,SOURCE_SYSTEM_TRADE_ID,TRADE_GROUP_ID, TRADE_TYPE,DEALER_NAME,EXTERNAL_COUNTERPARTY_ID, EXTERNAL_COUNTERPARTY_NAME,DB_COUNTERPARTY_ID,... (6 Replies)
Discussion started by: nmumbarkar
6 Replies
Login or Register to Ask a Question