Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Field editing

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Field editing
# 1  
Old 04-03-2014
Field editing
Hallo Team,

Hope you are doing very well.
My file has 2000 lines and this is how the first few lines look like:

Code:
-bash-3.2$ cat investigate.csv |grep "BSO,BST,PGWI"|  cut -f24,28 -d"," | sort -u
10.0.108.10:5060,BW101325695290314-1263752844@196.35.130.5
10.0.108.10:5060,BW1213361912903141919324243A@196.35.130.5
10.0.108.10:5060,BW1824098992803141677745661@196.35.130.5
10.0.108.11:5060,BW063115796010414-1770988784@196.35.130.5
10.0.108.11:5060,BW065834718010414-1613642742@196.35.130.5
10.0.108.11:5060,BW070733766010414-1957301209@196.35.130.5
10.0.108.11:5060,BW073958594010414353263281@196.35.130.5
10.0.108.11:5060,BW0744415670104141634683234@196.35.130.5
10.0.108.11:5060,BW075221127010414161266275@196.35.130.5
10.0.108.11:5060,BW080236823010414522110018@196.35.130.5
10.0.108.11:5060,BW082302285010414277928424@196.35.130.5
10.0.108.11:5060,BW0823377240104141828246840@196.35.130.5
10.0.108.11:5060,BW082652691010414-1136330906@196.35.130.5
10.0.108.11:5060,BW0836169160304142112073010@196.35.130.5
10.0.108.11:5060,BW083619423030414-205072078@196.35.130.5
10.0.108.11:5060,BW083755473030414-762988253@196.35.130.5
10.0.108.11:5060,BW090037461030414-1630843547@196.35.130.5
10.0.108.11:5060,BW090236388030414-1794826955@196.35.130.5
10.0.108.11:5060,BW090317517030414567835339@196.35.130.5
10.0.108.11:5060,BW0903399860304141220115219@196.35.130.5
10.0.108.11:5060,BW090742560030414-1277825170@196.35.130.5
10.0.108.11:5060,BW090746309030414515980173@196.35.130.5
10.0.108.11:5060,BW0927350920304141004013732@196.35.130.5

As you can see from my command above i am "grepping" field24 and field 28.
I would like to edit field 28 to have and "A" before the @ sign. See below for example:

Code:
BW063115796010414-1770988784@196.35.130.5
BW065834718010414-1613642742@196.35.130.5
BW070733766010414-1957301209@196.35.130.5
BW073958594010414353263281@196.35.130.5
BW074229265020414-209338751@196.35.130.5
BW0744415670104141634683234@196.35.130.5
BW075221127010414161266275@196.35.130.5
BW075501378300314-811233093@196.35.130.5
BW080007129270314-1745313902@196.35.130.5
BW080236823010414522110018@196.35.130.5
BW080345879280314-795207060@196.35.130.5

i want field 28 to look like below:

Code:
BW063115796010414-1770988784A@196.35.130.5
BW065834718010414-1613642742A@196.35.130.5
BW070733766010414-1957301209A@196.35.130.5
BW073958594010414353263281A@196.35.130.5
BW074229265020414-209338751A@196.35.130.5
BW0744415670104141634683234A@196.35.130.5
BW075221127010414161266275A@196.35.130.5
BW075501378300314-811233093A@196.35.130.5
BW080007129270314-1745313902A@196.35.130.5
BW080236823010414522110018A@196.35.130.5
BW080345879280314-795207060A@196.35.130.5

Last edited by Franklin52; 04-03-2014 at 11:22 AM.. Reason: Please use CODE tags, not ICODE tags for code blocks. Thanks.
# 2  
Old 04-03-2014
Add the below command to your command
Code:
awk '{sub("@", "A@", $2)}1' FS=','

Code:
cat investigate.csv |grep "BSO,BST,PGWI"|  cut -f24,28 -d"," | sort -u | awk '{sub("@", "A@", $2)}1' FS=','

You can even re-write you code to for better processing...but if you are happy with the above, fine
This User Gave Thanks to SriniShoo For This Post:
kekanap
# 3  
Old 04-03-2014
Thank you SriniShoo,

I want to change all the field 28 on the rest of the file without doing a grep what would the syntax be?

Regards,

Pax

---------- Post updated at 05:28 PM ---------- Previous update was at 05:17 PM ----------

The sample file without sorting the fields look like below:

Code:
-bash-3.2$ cat investigate.csv |grep "BSO,BST,PGWI"|head
BW-CDR-20140325114500-2-AC162DB137C8-111358-000.csv_rejected_5180022-20140325125045.log:0056158551AC162DB137C820140325094253.8560+020000,MOBIFIN,Normal,+27873503306,,Originating,+27873503306,Public,0762412888,20140325094253.856,0+020000,Yes,20140325094300.947,20140325094400.641,016,VoIP,,0762412888,national,mobi,0762412888,,local,10.0.108.20:5060,BW1142538742503141821463955@196.35.130.5,G729/8000,10.0.84.51,BW114253557250314-1447983243@196.35.130.5,,,,MOBIFIN_CPT,,,,,,,,,,y,public,,2563933171:0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,873503306@isvoip.net,,,,,,,,,,,,,,MOBIFIN_CPT/BT-MOBIFIN_CPT,,,,,,,,,,,,,,,,,,,,,,62.892,,,,,,,,,,,,,,,,,,,,,,,,,Normal,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0319142894@10.0.84.51,,,,,,,,,Network,,,,,,,,,,,,,,,,+27873503306,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,27873503306@10.0.84.51:5060,Primary Device,62.892,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Sippy,,,,,,,,,,,,,,,,,,,,,,,,,,,,,|Y,Unknown scenario:BSO,BST,PGWI
BW-CDR-20140325114500-2-AC162DB137C8-111358-000.csv_rejected_5180022-20140325125045.log:0056158551AC162DB137C820140325094253.8560+020000,MOBIFIN,Normal,+27873503306,,Originating,+27873503306,Public,0762412888,20140325094253.856,0+020000,Yes,20140325094300.947,20140325094400.641,016,VoIP,,0762412888,national,mobi,0762412888,,local,10.0.108.20:5060,BW1142538742503141821463955@196.35.130.5,G729/8000,10.0.84.51,BW114253557250314-1447983243@196.35.130.5,,,,MOBIFIN_CPT,,,,,,,,,,y,public,,2563933171:0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,873503306@isvoip.net,,,,,,,,,,,,,,MOBIFIN_CPT/BT-MOBIFIN_CPT,,,,,,,,,,,,,,,,,,,,,,62.892,,,,,,,,,,,,,,,,,,,,,,,,,Normal,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0319142894@10.0.84.51,,,,,,,,,Network,,,,,,,,,,,,,,,,+27873503306,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,27873503306@10.0.84.51:5060,Primary Device,62.892,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Sippy,,,,,,,,,,,,,,,,,,,,,,,,,,,,,|Y,Unknown scenario:BSO,BST,PGWI
BW-CDR-20140325131000-2-AC162DB137C8-111375-000.csv_rejected_5180169-20140325140350.log:0056587407AC162DB137C820140325110536.4700+020000,MOBIFIN,Normal,+27873503306,,Originating,+27873503306,Public,0834541005,20140325110536.470,0+020000,Yes,20140325110551.462,20140325110616.920,016,VoIP,,0834541005,national,mobi,0834541005,,remote,10.0.108.11:5060,BW1305364732503141612136782@196.35.130.5,G729/8000,10.0.84.51,BW130531663250314853049309@196.35.130.5,,,,MOBIFIN_CPT,,,,,,,,,,y,public,,2565352753:0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,873503306@isvoip.net,,,,,,,,,,,,,,MOBIFIN_CPT/BT-MOBIFIN_CPT,,,,,,,,,,,,,,,,,,,,,,36.336,,,,,,,,,,,,,,,,,,,,,,,,,Normal,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0227721723@10.0.84.51,,,,,,,,,Network,,,,,,,,,,,,,,,,+27873503306,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,27873503306@10.0.84.51:5060,Primary Device,36.336,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Sippy,,,,,,,,,,,,,,,,,,,,,,,,,,,,,|Y,Unknown scenario:BSO,BST,PGWI
BW-CDR-20140325131500-2-AC162DB137C8-111376-000.csv_rejected_5180180-20140325141538.log:0056615752AC162DB137C820140325111149.9760+020000,MOBIFIN,Normal,+27873503306,,Originating,+27873503306,Public,0762412888,20140325111149.976,0+020000,Yes,20140325111155.954,20140325111303.998,016,VoIP,,0762412888,national,mobi,0762412888,,local,10.0.109.13:5060,BW1311499792503141490777891@196.35.130.5,G729/8000,10.0.84.51,BW1311496652503141188409288@196.35.130.5,,,,MOBIFIN_CPT,,,,,,,,,,y,public,,2565442561:0,,,,,,,,,,,,,,cigZEAMieWl5AAOjL93MBVNC5A0-,10.31.14.40,mte.vodacom.co.za,mte.vodacom.co.za,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,873503306@isvoip.net,,,,,,,,,,,,,,MOBIFIN_CPT/BT-MOBIFIN_CPT,,,,,,,,,,,,,,,,,,,,,,73.655,,,,,,,,,,,,,,,,,,,,,,,,,Normal,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0319142894@10.0.84.51,,,,,,,,,Network,,,,,,,,,,,,,,,,+27873503306,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,27873503306@10.0.84.51:5060,Primary Device,73.655,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Sippy,,,,,,,,,,,,,,,,,,,,,,,,,,,,,|Y,Unknown scenario:BSO,BST,PGWI
BW-CDR-20140325131500-2-AC162DB137C8-111376-000.csv_rejected_5180180-20140325141538.log:0056615752AC162DB137C820140325111149.9760+020000,MOBIFIN,Normal,+27873503306,,Originating,+27873503306,Public,0762412888,20140325111149.976,0+020000,Yes,20140325111155.954,20140325111303.998,016,VoIP,,0762412888,national,mobi,0762412888,,local,10.0.109.13:5060,BW1311499792503141490777891@196.35.130.5,G729/8000,10.0.84.51,BW1311496652503141188409288@196.35.130.5,,,,MOBIFIN_CPT,,,,,,,,,,y,public,,2565442561:0,,,,,,,,,,,,,,cigZEAMieWl5AAOjL93MBVNC5A0-,10.31.14.40,mte.vodacom.co.za,mte.vodacom.co.za,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,873503306@isvoip.net,,,,,,,,,,,,,,MOBIFIN_CPT/BT-MOBIFIN_CPT,,,,,,,,,,,,,,,,,,,,,,73.655,,,,,,,,,,,,,,,,,,,,,,,,,Normal,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0319142894@10.0.84.51,,,,,,,,,Network,,,,,,,,,,,,,,,,+27873503306,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,27873503306@10.0.84.51:5060,Primary Device,73.655,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Sippy,,,,,,,,,,,,,,,,,,,,,,,,,,,,,|Y,Unknown scenario:BSO,BST,PGWI
BW-CDR-20140325140500-2-AC162DB137C8-111386-000.csv_rejected_5180273-20140325150156.log:0056840478AC162DB137C820140325120210.2050+020000,MOBIFIN,Normal,+27873503306,,Originating,+27873503306,Public,0762412888,20140325120210.205,0+020000,Yes,20140325120217.587,20140325120240.523,016,VoIP,,0762412888,national,mobi,0762412888,,remote,10.0.108.21:5060,BW1402102082503141551554166@196.35.130.5,G729/8000,10.0.84.51,BW140209933250314-1275329833@196.35.130.5,,,,MOBIFIN_CPT,,,,,,,,,,y,public,,2566211519:0,,,,,,,,,,,,,,cigZYFJVRXB5AAN7M2fMBVJCbQY-,10.31.36.37,pst.vodacom.co.za,pst.vodacom.co.za,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,873503306@isvoip.net,,,,,,,,,,,,,,MOBIFIN_CPT/BT-MOBIFIN_CPT,,,,,,,,,,,,,,,,,,,,,,29.981,,,,,,,,,,,,,,,,,,,,,,,,,Normal,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0364681164@10.0.84.51,,,,,,,,,Network,,,,,,,,,,,,,,,,+27873503306,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,27873503306@10.0.84.51:5060,Primary Device,29.981,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Sippy,,,,,,,,,,,,,,,,,,,,,,,,,,,,,|Y,Unknown scenario:BSO,BST,PGWI

I would like to change field 28 as explained.


Moderator's Comments:
Mod Comment Please use code tags next time for your code and data, not ICODE. Thanks
Last edited by vbe; 04-03-2014 at 12:58 PM..
# 4  
Old 04-03-2014
As SriniShoo noted either you'd do the entire pipeline or just the standalone awk command...
Code:
-bash-3.2$ awk -F, '{OFS=FS; sub("@", "A@", $28); print}' investigate.csv

# 5  
Old 04-03-2014
Code:
awk '/BSO,BST,PGWI/ {sub("@", "A@", $28); printf "%s,%s\n", $24, $28}' FS=',' investigate.csv | sort -u

# 6  
Old 04-04-2014
you guys are awesome. Thank you.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Problem with getting awk to multiply a field by a value set based on condition of another field

Hi, So awk is driving me crazy on this one. I have searched everywhere and read man, docs and every related post Google can find and still no luck. The actual files I need to run this on are sensitive in nature, but it is the same thing as if I needed to calculate weighted grades for multiple... (15 Replies)
Discussion started by: cotilloe
15 Replies

2. Shell Programming and Scripting

awk to adjust coordinates in field based on sequential numbers in another field

I am trying to output a tab-delimited result that uses the data from a tab-delimited file to combine and subtract specific lines. If $4 matches in each line then the first matching sequential $6 value is added to $2, unless the value is 1, then the original $2 is used (like in the case of line... (3 Replies)
Discussion started by: cmccabe
3 Replies

3. Shell Programming and Scripting

Convert vi editing to text editing

Dear Guru's I'm using Putty and want to edit a file. I know we generally use vi editor to do it. As I'm not good in using vi editor, I want to convert the vi into something like text pad. Is there any option in Putty to do the same ? Thanks for your response. Srini (6 Replies)
Discussion started by: thummi9090
6 Replies

4. Shell Programming and Scripting

Display combination of 4 field uniqe record and along with concatenate 5th and 6th field.

Table ACN|NAME|CITY|CTY|NO1|NO2 115|AKKK|ASH|IND|10|15 115|AKKK|ASH|IND|20|20 115|AKKK|ASH|IND|30|35 115|AKKK|ASH|IND|30|35 112|ABC|FL|USA|15|15 112|ABC|FL|USA|25|20 112|ABC|FL|USA|25|45 i have written shell script using cut command and awk programming getting error correct it and add... (5 Replies)
Discussion started by: udhal
5 Replies

5. Shell Programming and Scripting

Command/script to match a field and print the next field of each line in a file.

Hello, I have a text file in the below format: Source Destination State Lag Status CQA02W2K12pl:D:\CAQA ... (10 Replies)
Discussion started by: pocodot
10 Replies

6. Linux

How do I format a Date field of a .CSV file with multiple commas in a string field?

I have a .CSV file (file.csv) whose data are all enclosed in double quotes. Sample format of the file is as below: column1,column2,column3,column4,column5,column6, column7, Column8, Column9, Column10 "12","B000QRIGJ4","4432","string with quotes, and with a comma, and colon: in... (3 Replies)
Discussion started by: dhruuv369
3 Replies

7. Shell Programming and Scripting

For Loop Field editing - without using "awk"

Hi, I'm using Linux and bash shell. I have a file (F1.txt) with contents like Table1 Column1 123abc Table1 Column2 xyz Table2 Column1 543 Now, I would like to get the output as UPDATE Table1 SET Column1='123abc'; UPDATE Table1 SET Column2='xyz'; UPDATE Table2 SET Column1='543';... (3 Replies)
Discussion started by: Dev_Dev
3 Replies

8. Shell Programming and Scripting

AWK: Pattern match between 2 files, then compare a field in file1 as > or < field in file2

First, thanks for the help in previous posts... couldn't have gotten where I am now without it! So here is what I have, I use AWK to match $1 and $2 as 1 string in file1 to $1 and $2 as 1 string in file2. Now I'm wondering if I can extend this AWK command to incorporate the following: If $1... (4 Replies)
Discussion started by: right_coaster
4 Replies

9. Shell Programming and Scripting

awk, comma as field separator and text inside double quotes as a field.

Hi, all I need to get fields in a line that are separated by commas, some of the fields are enclosed with double quotes, and they are supposed to be treated as a single field even if there are commas inside the quotes. sample input: for this line, 5 fields are supposed to be extracted, they... (8 Replies)
Discussion started by: kevintse
8 Replies

10. Shell Programming and Scripting

Sort alpha on 1st field, numerical on 2nd field (sci notation)

I want to sort alphabetically on the first field and sort in descending numerical order on the 2nd field. With a normal "sort -r -n" it does this: abc ||| 5e-05 ||| bla abc ||| 3 ||| ble def ||| 1 ||| abc def ||| 0.2 ||| def As you can see it ignores the fact that 5e-05 is actually 0.00005... (1 Reply)
Discussion started by: FrancoisCN
1 Replies
Login or Register to Ask a Question