String has * as the field delimiter and I need echo/awk to escape it, how?

 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers String has * as the field delimiter and I need echo/awk to escape it, how?
# 8  
Old 01-06-2017
Hi RudiC

Thanks a lot, I'll worry about the wrapping thing later on. I may not need it if I am deconstructing the connectstring further

I'll post the updated script after I've broken down the connectstring information/variable. I will be using cut this time, I'll post it in case there is a better approach to it.

Currently, script now look as below. Not sure if there is a better way to get around the case/esac thingy.


Code:
$ ./z2.ksh
- timestamp = 15-DEC-2016 10:19:24  // 2016-12-15 10:19:24
- connectstring =  (CONNECT_DATA=(CID=(PROGRAM=JDBC Thin Client)(HOST=__jdbc__)(USER=testuser))(SERVER=DEDICATED)(SERVICE_NAME=test_app.x.y.z))
- host =  (ADDRESS=(PROTOCOL=tcp)(HOST=60.11.22.123)(PORT=55440))
- result =  establish
- service =  test_app.x.y.z
- returncode =  12666
-------------------------------------------------------------

$ cat z2.ksh
#!/bin/ksh

#15-DEC-2016 10:19:24 * (CONNECT_DATA=(CID=(PROGRAM=JDBC Thin Client)(HOST=__jdbc__)(USER=testuser))(SERVER=DEDICATED)(SERVICE_NAME=test_app.x.y.z)) * (ADDRESS=(PROTOCOL=tcp)(HOST=60.11.22.123)(PORT=55440)) * establish * test_app.x.y.z * 12666

#LOG=y.out
LOG=x.out

while IFS="*" read TS CS HOST RESULT SERVICE RETURNCODE
do
   timestamp=`echo $TS | awk '{ print $2 }'`
   year=`echo $TS | awk '{ print $1 }' | awk -F"-" '{ print $3 }'`
   day=`echo $TS | awk '{ print $1 }' | awk -F"-" '{ print $1 }'`
   month=`echo $TS | awk '{ print $1 }' | awk -F"-" '{ print $2 }'`

   case $month in
      "JAN" ) mm="01" ;;
      "FEB" ) mm="02" ;;
      "MAR" ) mm="03" ;;
      "APR" ) mm="04" ;;
      "MAY" ) mm="05" ;;
      "JUN" ) mm="06" ;;
      "JUL" ) mm="07" ;;
      "AUG" ) mm="08" ;;
      "SEP" ) mm="09" ;;
      "OCT" ) mm="10" ;;
      "NOV" ) mm="11" ;;
      "DEC" ) mm="12" ;;
   esac
   TS2="$year-$mm-$day $timestamp"

   echo "- timestamp = $TS // $TS2"
   echo "- connectstring = $CS"
   echo "- host = $HOST"
   echo "- result = $RESULT"
   echo "- service = $SERVICE"
   echo "- returncode = $RETURNCODE"
   echo "-------------------------------------------------------------"
   echo
done <  $LOG

###########
# THE END #
###########

$

# 9  
Old 01-06-2017
Wrapping taken a bit further:
Code:
awk -F\* '
function splitstring(FLD, ARR,          C)
        {while (length(FLD) > 0)        {ARR[++C] = substr (FLD, 1, FL)
                                         FLD = substr (FLD, FL + 1)
                                        }
         return C
        }
NR == 1 {FMT = "%20s\t%-" FL "s\t%-" FL "s\t%s\n"
        }
        {C1 = splitstring($2, T1)
         C2 = splitstring($3, T2)
         C  = (C1>C2)?C1:C2 
         for (i=1; i<=C; i++)   {printf FMT, $1, T1[i], T2[i], $4
                                 $1 = $3 = $4 = ""
                                }
        }
' FL=25 file
15-DEC-2016 10:19:24 	 (CONNECT_DATA=(CID=(PROG	 (ADDRESS=(PROTOCOL=tcp)(	 establish 
                    	RAM=JDBC Thin Client)(HOS	HOST=60.11.22.123)(PORT=5	
                    	T=__jdbc__)(USER=testuser	5440)) 	
                    	))(SERVER=DEDICATED)(SERV		
                    	ICE_NAME=test_app.x.y.z))

# 10  
Old 01-06-2017
Hi,

'final' script so far as below. Thanks to everyone's input especially RudiC

Perhaps some guidance on a 'cleaner' code, mainly on the multiple awk thingy and on the case/esac clause. Maybe there is a shorthand version of some sort that I can use Smilie

Script works as I wanted it to be nonetheless.

Code:
$ ./z2.ksh
- timestamp = 15-DEC-2016 10:19:24
  TS2 = 2016-12-15 10:19:24
- connectstring =  (CONNECT_DATA=(CID=(PROGRAM=JDBC Thin Client)(HOST=__jdbc__)(USER=testuser))(SERVER=DEDICATED)(SERVICE_NAME=test_app.x.y.z))
  program = JDBC Thin Client
  user = testuser
  service_name = test_app.x.y.z
- host =  (ADDRESS=(PROTOCOL=tcp)(HOST=60.11.22.123)(PORT=55440))
  app_protocol = tcp
  app_host = 60.11.22.123
  app_port = 55440
- result =  establish
- service =  test_app.x.y.z
- returncode =  12666
-------------------------------------------------------------

$ cat z2.ksh
#!/bin/ksh

#15-DEC-2016 10:19:24 * (CONNECT_DATA=(CID=(PROGRAM=JDBC Thin Client)(HOST=__jdbc__)(USER=testuser))(SERVER=DEDICATED)(SERVICE_NAME=test_app.x.y.z)) * (ADDRESS=(PROTOCOL=tcp)(HOST=60.11.22.123)(PORT=55440)) * establish * test_app.x.y.z * 12666

#LOG=y.out
LOG=x.out

while IFS="*" read TS CS HOST RESULT SERVICE RETURNCODE
do
   timestamp=`echo $TS | awk '{ print $2 }'`
   year=`echo $TS | awk '{ print $1 }' | awk -F"-" '{ print $3 }'`
   day=`echo $TS | awk '{ print $1 }' | awk -F"-" '{ print $1 }'`
   month=`echo $TS | awk '{ print $1 }' | awk -F"-" '{ print $2 }'`

   case $month in
      "JAN" ) mm="01" ;;
      "FEB" ) mm="02" ;;
      "MAR" ) mm="03" ;;
      "APR" ) mm="04" ;;
      "MAY" ) mm="05" ;;
      "JUN" ) mm="06" ;;
      "JUL" ) mm="07" ;;
      "AUG" ) mm="08" ;;
      "SEP" ) mm="09" ;;
      "OCT" ) mm="10" ;;
      "NOV" ) mm="11" ;;
      "DEC" ) mm="12" ;;
   esac
   TS2="$year-$mm-$day $timestamp"

  program=`echo $CS | awk -F"(" '{ print $4 }' | awk -F"=" '{ print $2 }' | awk -F")" '{ print $1}'`
   user=`echo $CS | awk -F"(" '{ print $6 }' | awk -F"=" '{ print $2 }' | awk -F")" '{ print $1}'`
   service_name=`echo $CS | awk -F"(" '{ print $8 }' | awk -F"=" '{ print $2 }' | awk -F")" '{ print $1}'`

   app_protocol=`echo $HOST | awk -F"(" '{ print $3 }' | awk -F"=" '{ print $2 }' | awk -F")" '{ print $1}'`
   app_host=`echo $HOST | awk -F"(" '{ print $4 }' | awk -F"=" '{ print $2 }' | awk -F")" '{ print $1}'`
   app_port=`echo $HOST | awk -F"(" '{ print $5 }' | awk -F"=" '{ print $2 }' | awk -F")" '{ print $1}'`
   echo "- timestamp = $TS"
   echo "  TS2 = $TS2"
   echo "- connectstring = $CS"
   echo "  program = $program"
   echo "  user = $user"
   echo "  service_name = $service_name"
   echo "- host = $HOST"
   echo "  app_protocol = $app_protocol"
   echo "  app_host = $app_host"
   echo "  app_port = $app_port"
   echo "- result = $RESULT"
   echo "- service = $SERVICE"
   echo "- returncode = $RETURNCODE"
   echo "-------------------------------------------------------------"
   echo
done <  $LOG

###########
# THE END #
###########

$

# 11  
Old 01-06-2017
Try
Code:
M=...JANFEBMARAPRMAYJUNJULAUGSEPOCTNOVDEC
TMP=${M%${TS:3:3}*}
echo ${TS:7:4}-$((${#TMP}/3))-${TS:0:2} ${TS:12}
2016-12-15 10:19:24

Please be aware that some serious error handling needs to be added.

Last edited by RudiC; 01-06-2017 at 02:02 PM..
# 12  
Old 01-06-2017
Hi RudiC

'final' code sort of as below.
Not sure if there is a shorthand way of doing the case/esac thing that converts the date to YYYY-MM-DD

Had to use multiple awks to further break down the connectstring. Not 'clean' but don't know o fany other way of doing it.

BTW, is there any way to get assign the 'original' line of text and using IFS="*" at the same time?

I want to preserve the original line somehow so if the RETURNCODE is > 0, I want to get the whole contents of the line. Currently as a workaround, I just combine all values of TS CS HOST RESULT SERVICE RETURNCODE Smilie

Code:
$ ./z2.ksh
- timestamp = 15-DEC-2016 10:19:24
  TS2 = 2016-12-15 10:19:24
- connectstring =  (CONNECT_DATA=(CID=(PROGRAM=JDBC Thin Client)(HOST=__jdbc__)(USER=testuser))(SERVER=DEDICATED)(SERVICE_NAME=test_app.x.y.z))
  program = JDBC Thin Client
  user = testuser
  service_name = test_app.x.y.z
- host =  (ADDRESS=(PROTOCOL=tcp)(HOST=60.11.22.123)(PORT=55440))
  app_protocol = tcp
  app_host = 60.11.22.123
  app_port = 55440
- result =  establish
- service =  test_app.x.y.z
- returncode =  12666
-------------------------------------------------------------

$ cat z2.ksh
#!/bin/ksh

#15-DEC-2016 10:19:24 * (CONNECT_DATA=(CID=(PROGRAM=JDBC Thin Client)(HOST=__jdbc__)(USER=testuser))(SERVER=DEDICATED)(SERVICE_NAME=test_app.x.y.z)) * (ADDRESS=(PROTOCOL=tcp)(HOST=60.11.22.123)(PORT=55440)) * establish * test_app.x.y.z * 12666

#LOG=y.out
LOG=x.out

while IFS="*" read TS CS HOST RESULT SERVICE RETURNCODE
do
   timestamp=`echo $TS | awk '{ print $2 }'`
   year=`echo $TS | awk '{ print $1 }' | awk -F"-" '{ print $3 }'`
   day=`echo $TS | awk '{ print $1 }' | awk -F"-" '{ print $1 }'`
   month=`echo $TS | awk '{ print $1 }' | awk -F"-" '{ print $2 }'`

   case $month in
      "JAN" ) mm="01" ;;
      "FEB" ) mm="02" ;;
      "MAR" ) mm="03" ;;
      "APR" ) mm="04" ;;
      "MAY" ) mm="05" ;;
      "JUN" ) mm="06" ;;
      "JUL" ) mm="07" ;;
      "AUG" ) mm="08" ;;
      "SEP" ) mm="09" ;;
      "OCT" ) mm="10" ;;
      "NOV" ) mm="11" ;;
      "DEC" ) mm="12" ;;
   esac
   TS2="$year-$mm-$day $timestamp"

   program=`echo $CS | awk -F"(" '{ print $4 }' | awk -F"=" '{ print $2 }' | awk -F")" '{ print $1}'`
   user=`echo $CS | awk -F"(" '{ print $6 }' | awk -F"=" '{ print $2 }' | awk -F")" '{ print $1}'`
   service_name=`echo $CS | awk -F"(" '{ print $8 }' | awk -F"=" '{ print $2 }' | awk -F")" '{ print $1}'`

   app_protocol=`echo $HOST | awk -F"(" '{ print $3 }' | awk -F"=" '{ print $2 }' | awk -F")" '{ print $1}'`
   app_host=`echo $HOST | awk -F"(" '{ print $4 }' | awk -F"=" '{ print $2 }' | awk -F")" '{ print $1}'`
   app_port=`echo $HOST | awk -F"(" '{ print $5 }' | awk -F"=" '{ print $2 }' | awk -F")" '{ print $1}'`

   echo "- timestamp = $TS"
   echo "  TS2 = $TS2"
   echo "- connectstring = $CS"
   echo "  program = $program"
   echo "  user = $user"
   echo "  service_name = $service_name"
   echo "- host = $HOST"
   echo "  app_protocol = $app_protocol"
   echo "  app_host = $app_host"
   echo "  app_port = $app_port"
   echo "- result = $RESULT"
   echo "- service = $SERVICE"
   echo "- returncode = $RETURNCODE"
   echo "-------------------------------------------------------------"
   echo
done <  $LOG

###########
# THE END #
###########

$


Last edited by Don Cragun; 01-07-2017 at 12:01 AM.. Reason: Doubled text removed.
# 13  
Old 01-07-2017
Hi the awk statements are indeed "not clean" but since they are inside a loop they also slow things down considerably.

Here is an alternative way that you could try using the shell's read statement, which should be much faster:
Code:
{
  IFS=' -'  read day month year timestamp
  IFS='(=)' read x x x x x x program x x x x x user x x x x x service_name x
  IFS='(=)' read x x x x app_protocol x x app_host x x app_port x
} << EOF
$TS
$CS
$HOST
EOF

  • The repeated x variables are used to contain the things that we do not need.
  • The IFS variable is local to the read command and it is used to specify the characters that separate the fields in a given string.
  • A "Here document" ( << EOF ) is used to emulate a file that contains the three variables TC, CS and HOST on a separate line.
  • The curly braces { .. } to together with << EOF are used to define a block of shell code that uses the here document as input.





---
Here is another way of writing it, using separate here documents, which may be preferable since it provides a cleaner look and maybe is a bit easier to read for the human eye:
Code:
IFS=' -'  read day month year timestamp << EOF
$TS
EOF

IFS='(=)' read x x x x x x program x x x x x user x x x x x service_name x << EOF
$CS
EOF

IFS='(=)' read x x x x app_protocol x x app_host x x app_port x << EOF
$HOST
EOF


Last edited by Scrutinizer; 01-07-2017 at 03:35 AM..
# 14  
Old 01-07-2017
Why not, then,
Code:
while IFS='*-=)(' read DD MON YRTMS _ _ _ _ _ _ PRG _ _ _ _ _ USR _ _ _ _ _ _ SVC _ _ _ _ _ _ PTC _ _ HOST _ _ PRT _ _ RES SVC2 RET
   do   TMP=${M%$MON*}
        echo $DD
        echo $((${#TMP}/3))
        echo ${YRTMS%% *}
        echo ${YRTMS#* }
        echo $PRG 
        echo $USR 
        echo $SVC 
        echo $PTC 
        echo $HOST 
        echo $PRT 
        echo $RES 
        echo $SVC2 
        echo $RET
    done <  file
15
12
2016
10:19:24
JDBC Thin Client
testuser
test_app.x.y.z
tcp
60.11.22.123
55440
establish
test_app.x.y.z
12666

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

awk Associative Array and/or Referring to Field by String (Nonconstant String Value)

I will start with an example of what I'm trying to do and then describe how I am approaching the issue. File PS028,005 Lexeme HRS # M # PhraseType 1(1:1) 7(7) PhraseLab 501 503 ClauseType ZYq0 PS028,005 Lexeme W # L> # BNH # M #... (17 Replies)
Discussion started by: jvoot
17 Replies

2. Shell Programming and Scripting

How can awk ignore the field delimiter like comma inside a field?

We have a csv file as mentioned below and the requirement is to change the date format in file as mentioned below. Current file (file.csv) ---------------------- empname,date_of_join,dept,date_of_resignation ram,08/09/2015,sales,21/06/2016 "akash,sahu",08/10/2015,IT,21/07/2016 ... (6 Replies)
Discussion started by: gopal.biswal
6 Replies

3. Shell Programming and Scripting

awk :how to change delimiter without giving all field name

Hi Experts, i need to change delimiter from tab to "," sample test file cat test A0000368 A29938511 072569352 5 Any 2 for Ģ1.00 BUTCHERS|CAT FOOD|400G Sep 12 2012 12:00AM Jan 5 2014 11:59PM Sep 7 2012 12:00AM M 2.000 group 5 ... (2 Replies)
Discussion started by: Lakshman_Gupta
2 Replies

4. Shell Programming and Scripting

perform echo and awk inside a string

hi, just wanted to make a shortcut of this one a="a b c" b=`echo $a | awk '{print $2}'` echo "the middle is $b" why can't i do this: a="a b c" echo "the middle is ${`echo $a | awk '{print $2}'`}" <- bad substitution :wall: thanks (6 Replies)
Discussion started by: h0ujun
6 Replies

5. Shell Programming and Scripting

Awk Search text string in field, not all in field.

Hello, I am using awk to match text in a tab separated field and am able to do so when matching the exact word. My problem is that I would like to match any sequence of text in the tab-separated field without having to match it all. Any help will be appreciated. Please see the code below. awk... (3 Replies)
Discussion started by: rocket_dog
3 Replies

6. Shell Programming and Scripting

how to find the nth field value in delimiter file in unix using awk

Hi All, I wanted to find 200th field value in delimiter file using awk.? awk '{print $200}' inputfile I am getting error message :- awk: The field 200 must be in the range 0 to 199. The source line number is 1. The error context is {print >>> $200 <<< } using... (4 Replies)
Discussion started by: Jairaj
4 Replies

7. Shell Programming and Scripting

awk output field delimiter

Dear All, 1.txt (tab in between each value in a line) a b c a b c a c d you can see below, why with ~ i can output with tab, but = cannot? # awk -F'\t' '$2 ~ /b/' 1 a b c a b c # awk -F'\t' '$2 = "b"' 1 a b c a b c a b d ... (1 Reply)
Discussion started by: jimmy_y
1 Replies

8. Shell Programming and Scripting

need to convert a decimal value to an ascii char with awk for the field delimiter

Hello, I need an awk script to receive a variable that's an decimal value such as 009 or 031 and then convert this value to an ascii character to use as the FS (field separator for the input file). For example, 009 should be converted to an ascii tab 031 should be converted to an ascii... (1 Reply)
Discussion started by: script_op2a
1 Replies

9. UNIX for Advanced & Expert Users

Printing Field with Delimiter in AWK/cut

Hello, I had posted earlier about printing fields using AWK, but now I have a slightly different problem. I have text files in the format: 1*2,3,4,5 and wish to print the first, third, and fifth fields, including the asterisk and commas. In other words, after filtering it should look... (1 Reply)
Discussion started by: Jahn
1 Replies

10. Shell Programming and Scripting

Set a variable field delimiter using awk

How can i set a variable field delimiter using awk?? I wanna do something like this ,but i canīt get the correct syntaxis : VARI=TEST echo "0121212TESTxvcshaashd"|awk 'FS="$VARI" {print $2}' Thanks. (2 Replies)
Discussion started by: Klashxx
2 Replies
Login or Register to Ask a Question