Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

File conversion and awk

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting File conversion and awk
# 1  
Old 04-23-2009
File conversion and awk
Hi Everyone,
I am confused with the output of the input file and I am using below command in script to get the expected output.
Also I want to add another condition using logical AND (&&) in place of $2=="L"{$4=0-$4} as $2=="L" && $3=="L" {$4=0-$4} but I am getting some awk error.
Can someone please advise on this please?

Code:
 
cat $FILENAME |grep "^PL"|tr -d '\015'|sort -k1,2|nawk -F'|' 'BEGIN {OFS="~"}$2=="L"{$4=0-$4};$6=="COM"?$6="SHS":$6="FMT";{arr[$1"~"$5"~"$6]+=$4} END {for (i in arr) {print i,arr[i]}}'>$TEMPFILE
Input File 
============
ISIN|BL|STATUS|QTY|SNAME|CLASS
PLKGHM000017|B|L|85000|KGHM|COM
PLPKO0000016|B|L|310000|PKO S.A.|COM
|B|L|0||
PLKGHM000017|L|L|35000|KGHM|COM
PL0000101937|B|L|100000|DS1110|GOV
|L|L|0||
PLKGHM000017|L|L|40000|KGHM|COM
PLPKO0000016|L|L|290000|PKO S.A.|COM
Present Output
==============
PL0000101937~B~L~100000~DS1110~FMT
PLKGHM000017~B~L~85000~KGHM~SHS
PLKGHM000017~L~L~-35000~KGHM~SHS
PLKGHM000017~L~L~-40000~KGHM~SHS
PLPKO0000016~B~L~310000~PKO S.A.~SHS
PLPKO0000016~L~L~-290000~PKO S.A.~SHS
PLPKO0000016~PKO S.A.~SHS~20000
PL0000101937~DS1110~FMT~100000
PLKGHM000017~KGHM~SHS~10000
Expected Output
==============
PLPKO0000016~PKO S.A.~SHS~20000
PL0000101937~DS1110~FMT~100000
PLKGHM000017~KGHM~SHS~10000

# 2  
Old 04-23-2009
Quote:
Originally Posted by gehlnar
Hi Everyone,
I am confused with the output of the input file and I am using below command in script to get the expected output.
Also I want to add another condition using logical AND (&&) in place of $2=="L"{$4=0-$4} as $2=="L" && $3=="L" {$4=0-$4} but I am getting some awk error.
Can someone please advise on this please?

Code:
 
cat $FILENAME |grep "^PL"|tr -d '\015'|sort -k1,2|nawk -F'|' 'BEGIN {OFS="~"}$2=="L"{$4=0-$4};$6=="COM"?$6="SHS":$6="FMT";{arr[$1"~"$5"~"$6]+=$4} END {for (i in arr) {print i,arr[i]}}'>$TEMPFILE
Input File 
============
ISIN|BL|STATUS|QTY|SNAME|CLASS
PLKGHM000017|B|L|85000|KGHM|COM
PLPKO0000016|B|L|310000|PKO S.A.|COM
|B|L|0||
PLKGHM000017|L|L|35000|KGHM|COM
PL0000101937|B|L|100000|DS1110|GOV
|L|L|0||
PLKGHM000017|L|L|40000|KGHM|COM
PLPKO0000016|L|L|290000|PKO S.A.|COM
Present Output
==============
PL0000101937~B~L~100000~DS1110~FMT
PLKGHM000017~B~L~85000~KGHM~SHS
PLKGHM000017~L~L~-35000~KGHM~SHS
PLKGHM000017~L~L~-40000~KGHM~SHS
PLPKO0000016~B~L~310000~PKO S.A.~SHS
PLPKO0000016~L~L~-290000~PKO S.A.~SHS
PLPKO0000016~PKO S.A.~SHS~20000
PL0000101937~DS1110~FMT~100000
PLKGHM000017~KGHM~SHS~10000
Expected Output
==============
PLPKO0000016~PKO S.A.~SHS~20000
PL0000101937~DS1110~FMT~100000
PLKGHM000017~KGHM~SHS~10000

can you provide a sample of the expected output here? From your post, it is not clear what you are trying to achieve.


cheers,
Devaraj Takhellambam
# 3  
Old 04-23-2009
Hi ,

This is the expected output.

Expected Output
==============
PLPKO0000016~PKO S.A.~SHS~20000
PL0000101937~DS1110~FMT~100000
PLKGHM000017~KGHM~SHS~10000

But if you see my original post I am getting this but along with some other records. Please let me know if you have further query on this.

Cheers,
gehlnar


# 4  
Old 04-23-2009
You really don't need 'cat', 'grep' etc.... - awk can do it all itself.

nawk -f geh.awk myFile

geh.awk:
Code:
BEGIN {
  FS="|"
  OFS="~"
}
!/^PL/ { next }
$2=="L" && $3=="L" {$4=0-$4}
{
  $6=($6=="COM")?"SHS":"FMT"
  arr[$1 OFS $5 OFS $6]+=$4
}
END {
  for (i in arr)
    print i,arr[i]
}

But I'm not getting your expected output, although I've tried to follow your code.
# 5  
Old 04-23-2009
Thanks Vgersh , I have correct the code as per your advise.

I am getting expected output if i use awk operation twice i.e. by storing the output before array operation
and on new file if i use array operation I am getting correct output.

But as I was using awk twice , I thought of combining it. Do we something like pipe in awk ?


So it will be like first get this file..

Code:
PL0000101937~B~L~100000~DS1110~FMT
PLKGHM000017~B~L~85000~KGHM~SHS
PLKGHM000017~L~L~-35000~KGHM~SHS
PLKGHM000017~L~L~-40000~KGHM~SHS
PLPKO0000016~B~L~310000~PKO S.A.~SHS
PLPKO0000016~L~L~-290000~PKO S.A.~SHS

and then perform array operation of calculation on this , But My query is can't this be done using one awk operation.
# 6  
Old 04-23-2009
I don't quite follow what you're doing - sorry.
Could you post the steps and the modified code, please?
# 7  
Old 04-23-2009
vgersh,

Step 1 :Initial File Conversion for basic calculation
Code:
cat INPUTFILE |tr -d '\015'|sort -k1,2|nawk -F'|' 
'BEGIN {OFS="~"}!/^PL/{next}$2=="L" && $3=="L"
{$4=0-$4};$6=="COM"?$6="SHS":$6="FMT"{print $0}'>TEMPFILE

Step 2: Actual calculation to get expected output using above's output of TEMPFILE
Code:
 awk -F'~' '{OFS="~"}{arr[$1OFS$5OFS$6]+=$4} END 
  {for (i in arr) {print i,arr[i]}}'<TEMPFILE>EXPECTEDFILE

I am using above mentioned two steps in a script , Hope you got some idea from it and my idea was to combine
these two steps into one.

Cheers,
gehlnar
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Base64 conversion in awk overlaps

hi, problem: output is not consistent as expected using external command in AWK description: I'm trying to convert $2 into a base64 string for later decoding, and for this when I use awk , I'm getting overlapped results , or say it results are not 100% correct. my code is: gawk... (9 Replies)
Discussion started by: busyboy
9 Replies

2. Shell Programming and Scripting

Numeral conversion in awk

I am running into conversion of string to numbers in awk that I can't understand and don't know how to deal with properly My script checks for numeric only field, I use this approach to do that: $1 + 0 == $1 It works most of the time, but in some cases it does not behave how I expect it to... (5 Replies)
Discussion started by: migurus
5 Replies

3. Shell Programming and Scripting

awk Flat File Conversion Script

In awk how would I flatten input.txt to output.txt? Input: givenname: Darth sn: Vadar mail: d.vadar@deathstar.com uid: dv12345 orclguid: 1234567890 givenname: Carlito sn: Brigante mail: c.brigante@paradise.com uid: cb12345 orclguid: 2134567890 Output: ... (3 Replies)
Discussion started by: u20sr
3 Replies

4. Shell Programming and Scripting

Awk: conversion of matrix formats

hello, i would need a fast awk script for conversion of network formats (from 'sif' to 'adjacency' format): sif (pp means only: protein-protein interaction): A pp B A pp C B pp D D pp E in an adjacency n x n matrix: A B C D E A 0 1 1 0 0 B 1 0 0 1 0 C 1 0 0 0 0 D 0 1 0 0 1... (10 Replies)
Discussion started by: dietmar13
10 Replies

5. Shell Programming and Scripting

Conversion of line via awk or etc

Hello friends, could you help me about problem with my data lines. I suppose a simple awk code may help me. I have following data lines: (first line including 3 numbers and then a matrices of 4x10) 500 40 9 1 A B 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22... (6 Replies)
Discussion started by: rpf
6 Replies

6. Shell Programming and Scripting

Wiki conversion with Awk or Sed

I have the words in twiki format that I want to convert to mediawiki format like below : %BLUE%some words1%ENDCOLOR% bla bla blab labdad sdadasd adsasdads oerdkfj kdfjs %PINK%some wordks2 123.4.5.6/26%ENDCOLOR%, ksdjak dkasjd kjfrjkfgjdkfgjdfkgjdgdfgdgf %PURPLE%1.2.3.4/28%ENDCOLOR%, dskd... (3 Replies)
Discussion started by: rk4k
3 Replies

7. Shell Programming and Scripting

HPUX to Linux awk conversion

I have the following script to edit ^'s and newlines out of files to be entered into a database. This script has been around since the dawn of time (way before me). #!/bin/bash # Remove all ^ and \n from the source file, except do not remove ^^^\n cat myfile.hold | awk ' BEGIN {FS="|";... (1 Reply)
Discussion started by: insania
1 Replies

8. Shell Programming and Scripting

AWK Currency Conversion

How can I use awk command to convert values to currency. For example I have a database like follows John:200 smith:300 kim:405 and want it to out put like this John $200.00 (3 Replies)
Discussion started by: 3junior
3 Replies

9. Shell Programming and Scripting

awk script for date conversion

hi awk script for dd/mm/yyyy to yyyymmdd awk script for dd-mon-yyyy to yyyymmdd awk script for dd-mm-yyyy to yyyymmdd formate ..............urgent............. Thanks in advanced (2 Replies)
Discussion started by: charandevu
2 Replies

10. Shell Programming and Scripting

String Conversion in awk

I am porting a awk script from Windows to unix I_SALE_MEDIA=$67 if ((I_VOID_FLAG == "Y") && (I_SALE_MEDIA == 0)) NOW consider the case where I_SALE_MEDIA i.e $67 is "000" The above comparison works fine in Windows , but to make it work in Unix , I had to change the above as follows : ... (3 Replies)
Discussion started by: rohanrege
3 Replies
Login or Register to Ask a Question