Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk command not working as expected

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk command not working as expected
# 22  
Old 04-06-2016
My guess would be that the terminal emulator settings are different on the two systems you are using to run these commands; one is wrapping output onto multiple display lines and one is showing you the last screen-width characters on each line without wrapping (and each of your lines has hundreds of spaces at the ends of the lines).

To confirm this, show us the output from the command:
Code:
head -n1 file2 | od -bc

where file2 is the output file from any of the commands you showed us in post #19 that you say are producing blank lines.
# 23  
Old 04-06-2016
I used this awk command from #19 which created file2 with blank lines
Code:
awk '{print substr($0,3)}' file1 > file2

On resulted file2 I ran your command
Code:
head -n1 file2 | od -bc

which showed below output

Code:
0000000 012
         \n
0000001

Not exactly sure what you are looking for here but let me know if you have an answer. Much appreciated.
# 24  
Old 04-06-2016
Quote:
Originally Posted by later_troy
I used this awk command from #19 which created file2 with blank lines
Code:
awk '{print substr($0,3)}' file1 > file2

On resulted file2 I ran your command
Code:
head -n1 file2 | od -bc

which showed below output

Code:
0000000 012
         \n
0000001

Not exactly sure what you are looking for here but let me know if you have an answer. Much appreciated.
That was not what I expected. Let's go back one step and look around a couple of corners... What output do you get from the command line:
Code:
head -n3 file1 | tee file1.1-3 | awk '{print substr($0,3)}' | tee file2.1-3 | od -bc; ls -l file[12] file[12].1-3; locale

Have you seen any diagnostic messages reporting invalid characters that you haven't mentioned?
# 25  
Old 04-06-2016
Well, puzzles me as well. I tried to replicatre your file and get
Code:
od -bc file
0000000 347 003 062 060 061 066 060 061 060 063 061 066 063 070 061 065
        347 003   2   0   1   6   0   1   0   3   1   6   3   8   1   5
0000020 104 062 065 060 060 061 064 101 060 061 060 060 060 060 060 060
          D   2   5   0   0   1   4   A   0   1   0   0   0   0   0   0
0000040 061 060 060 061 061 071 061 060 063 060 060 060 060 060 060 111
          1   0   0   1   1   9   1   0   3   0   0   0   0   0   0   I
0000060 120 064 071 062 060 061 065 060 060 060 060 060 060 060 060 060
          P   4   9   2   0   1   5   0   0   0   0   0   0   0   0   0
0000100 060 060 060 062 060 061 066 055 060 061 055 060 063 055 061 066
          0   0   0   2   0   1   6   -   0   1   -   0   3   -   1   6
0000120 056 063 070 056 061 065 056 066 064 060 060 060 060 123 131 123
          .   3   8   .   1   5   .   6   4   0   0   0   0   S   Y   S
0000140 124 120 122 120 117 062 060 061 066 055 060 061 055 060 063 055
          T   P   R   P   O   2   0   1   6   -   0   1   -   0   3   -
0000160 061 066 056 063 070 056 061 065 056 066 064 060 060 060 060 040
          1   6   .   3   8   .   1   5   .   6   4   0   0   0   0    
0000200 040 040 040 040 040 040 040 040 040 040 040 040 040 040 040 040
                                                                       
*
0000400 012 347 003 062 060 061 066 060 061 060 063 061 066 063 070 061
         \n 347 003   2   0   1   6   0   1   0   3   1   6   3   8   1
.
.
.

awk works pretty well on it:
Code:
awk '{print substr($0,3)}' file
20160103163815D250014A01000000100119103000000IP4920150000000000002016-01-03-16.38.15.640000SYSTPRPO20
.
.
.

so I can't replicate your problem. But - you may try this
Code:
LC_ALL=C sed 's/^..//' file
20160103163815D250014A01000000100119103000000IP4920150000 . . .

and come back with the results.
This User Gave Thanks to RudiC For This Post:
later_troy
# 26  
Old 04-06-2016
Quote:
Originally Posted by Don Cragun
That was not what I expected. Let's go back one step and look around a couple of corners... What output do you get from the command line:
Code:
head -n3 file1 | tee file1.1-3 | awk '{print substr($0,3)}' | tee file2.1-3 | od -bc; ls -l file[12] file[12].1-3; locale

Have you seen any diagnostic messages reporting invalid characters that you haven't mentioned?
Here is the output from
Code:
head -n3 file1 | tee file1.1-3 | awk '{print substr($0,3)}' | tee file2.1-3 | od -bc;

0000000 012 012 012
         \n  \n  \n
0000003

Output of
Code:
locale

LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

There is no output or error message on terminal, command runs fine with/without redirection and reports no error.
Last edited by later_troy; 04-25-2016 at 11:27 AM..
# 27  
Old 04-06-2016
LC_ALL=C sed 's/^..//' file
the longevity of setting LC_ALL is for the run of the sed sommand. It will not affect the parent shell and/or anything/anybody else on the system.

[Sorry for getting in the middle of the discussion]
# 28  
Old 04-06-2016
That definition is "local" (or temporary) to the sed command only. If not used this way, it would be "local" to your session only. The admin has to change it to affect other people.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk matching script not working as expected

This is my ubuntu version: $ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 16.04.4 LTS Release: 16.04 Codename: xenial $ /bin/awk -V | head -n1 bash: /bin/awk: No such file or directory I have gotten a script that helps me to parse,... (14 Replies)
Discussion started by: delbroooks
14 Replies

2. Shell Programming and Scripting

awk gsub not working as expected

Hi Experts, Need your kind help with gsub awk. Below is my pattern:"exec=1_host_cnt=100_dup=4_NameTag=targetSrv_500.csv","'20171122112948"," 100"," 1"," 1"," 4","400","","", " aac sample exec ""hostname=XXXXX commandline='timeout 10 openssl speed -multi 2 ; exit 0'"" ","-1","-1","1","... (6 Replies)
Discussion started by: pradyumnajpn10
6 Replies

3. Shell Programming and Scripting

Cp command not working as expected in HPUX

Hi, I'm having trouble with a simple copy command in a script on HPUX. I am trying to copy a file and append date & time. The echo command prints out what I am expecting.. echo "Backing up $file to $file.$DATE.$FIXNUM" | tee -a $LOGFILE + echo 'Backing up... (4 Replies)
Discussion started by: Glennyp
4 Replies

4. Shell Programming and Scripting

awk not working as expected in script

Dear all, I had script which used to work, but recently it is not working as expected. I have command line in my shell script to choose the following format from the output_elog and perform some task afterwards on As you see, I want all numbers in foramt following RED mark except for... (12 Replies)
Discussion started by: emily
12 Replies

5. Shell Programming and Scripting

Read command not working as expected

I was trying to write a simple script which will read a text file and count the number of vowels in the file. My code is given below - #!/bin/bash file=$1 v=0 if then echo "$0 filename" exit 1 fi if then echo "$file not a file" exit 2 fi while read -n... (14 Replies)
Discussion started by: linux_learner
14 Replies

6. Shell Programming and Scripting

bash variable (set via awk+sed) not working as expected

Hi! Been working on a script and I've been having a problem. I've finally narrowed it down to this variable I'm setting: servername=$(awk -v FS=\/ '{ print $7 } blah.txt | sed 's\/./-/g' | awk -v FS=\- '{print $1}')" This will essentially pare down a line like this: ... (7 Replies)
Discussion started by: creativedynamo
7 Replies

7. OS X (Apple)

Cat command not working as expected

I've been trying to figure this out since last night, and I'm just stumped. The last time I did any shell scripting was 8 years ago on a Unix box, and it was never my strong suit. I'm on a Mac running Leopard now. Here's my dilemma - hopefully someone can point me in the right direction. I'm... (10 Replies)
Discussion started by: Daniel M. Clark
10 Replies

8. UNIX for Dummies Questions & Answers

Find command not working as expected

I have a script with a find command using xargs to copy the files found to another directory. The find command is finding the appropriate file, but it's not copying. I've checked permissions, and those are all O.K., so I'm not sure what I'm missing. Any help is greatly appreciated. This is... (2 Replies)
Discussion started by: mpflug
2 Replies

9. Shell Programming and Scripting

Var substitution in awk - not working as expected

countA=`awk '/X/''{print substr($0,38,1)}' fName | wc -l` countB=`wc -l fName | awk '{print int($1)}'` echo > temp ratio=`awk -va=$countA -vc=$countB '{printf "%.4f", a/c}' temp` After running script for above I am getting an error as : awk: 0602-533 Cannot find or open file -vc=25. The... (3 Replies)
Discussion started by: videsh77
3 Replies

10. Shell Programming and Scripting

awk not working as expected with BIG files ...

I am facing some strange problem. I know, there is only one record in a file 'test.txt' which starts with 'X' I ensure that with following command, awk /^X/ test.txt | wc -l This gives me output = '1'. Now I take out this record out of the file, as follows : awk /^X/ test.txt >... (1 Reply)
Discussion started by: videsh77
1 Replies
Login or Register to Ask a Question