Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

Data Manipulation

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Data Manipulation
# 1  
Old 02-02-2010
Data Manipulation
Hello

I am currently having problems in mapulating a certain file which contains vaious data.
Belos is a sample content
Code:
 
Event=<3190>
Client IP=<151.111.11.143> DNS=<abc.sbc.com> TransCount=<139>
Client IP=<150.222.133.163> DNS=<xyz.yuu.com> TransCount=<3734>
Event=<3120>
Client IP=<131.110.11.143> DNS=<abzc.sabc.com> TransCount=<1392>
Client IP=<120.222.133.163> DNS=<xybz.yuvu.com> TransCount=<33734>
Event=<3190>
Client IP=<151.111.11.143> DNS=<abc.sbc.com> TransCount=<29>
Client IP=<150.222.133.163> DNS=<xyz.yuu.com> TransCount=<123>
Client IP=<122.233.155.123> DNS=<gsz.xcc.com> TransCount=<789>

Is is possible for me to have a script which will have the following format:
Code:
Event<3190>
151.111.11.143 DNS abc.sbc.com
150.222.133.163 DNS xyz.yuu.com
122.233.155.123 DNS gsz.xcc.com
Event<3120>
131.110.11.143 DNS abzc.sabc.com
120.222.133.163 DNS xybz.yuvu.com

Of course my file contains much more events and IP lists.
Can anyone help me on this?

Thanks!
Last edited by Scott; 02-02-2010 at 02:38 PM.. Reason: Please use code tags
# 2  
Old 02-02-2010
Code:
sed '
  s/^Client[^<]*<\([^>]*\)> *\([^=]*\)[^<]*<\([^>]*\).*/\1 \2 \3/
  ' infile

# 3  
Old 02-02-2010
radoulov
I just tried your code and it works.
I have a question though.
Will it omit the duplicate IPs?
# 4  
Old 02-02-2010
Sorry,
I missed this important bit.
You want the unique IPs file-wise or event-wise?

---------- Post updated at 08:35 PM ---------- Previous update was at 08:11 PM ----------



You should modify the key of the hash %t if you want the unique IPs per event.

Code:
perl -nle'
  /^Event/ and $k = $_ ;
  $t{$1}++ or push @{$_{$k}}, join " ", $1, $2, $3 
    if /=<([^>]*)>\s*(\w*)=<([^>]*)/;
  END {
    print join $/, $_, @{$_{$_}} for keys %_;
    }' infile

Last edited by radoulov; 02-02-2010 at 03:43 PM..
# 5  
Old 02-02-2010
Yes i need the unique IPs per event.
I tried to put your code in a .sh file and executed it but it did not work.

How do i execute your script? Havent tried perl codes yet.

---------- Post updated at 12:46 PM ---------- Previous update was at 12:39 PM ----------

Ok i got it to work.
I just need to add the perl path in the file.
But whenever i try to execute it i get an error message

Code:
Bad name after nle' at ./testscript.sh line 3.

# 6  
Old 02-02-2010
Use this script:

Code:
#!/usr/bin/perl -nl

/^Event/ and $k = $_;

$t{$1}++ or push @{$_{$k}}, join " ", $1, $2, $3 
    if /=<([^>]*)>\s*(\w*)=<([^>]*)/;
  
END {
      print join $/, $_, @{$_{$_}} for keys %_;
  }

Invoke as:

Code:
./scriptname inputfile

You may need to clean the code (add warnings, strict and declare the variables).
# 7  
Old 02-02-2010
Thank you for replying to my posts.

I tried running the script you recently posted but nothing happened.

I tried this

Code:
./testscript2.sh IPLIST > IP

which should direct the output to a file named IP.
the thing is the file IP contains nothing.

I really dont know anything bout perl thats why im having problems trying to modify your code.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Data manipulation, Please help..

Hello, I have a huge set of data that needs to be reformatted. Here is a simple example to explain the process. I have number n=5 and a input with many numbers separated with comma: ... (11 Replies)
Discussion started by: liuzhencc
11 Replies

2. Shell Programming and Scripting

[Solved] Data manipulation

Hallo Team, I need your help. I have a file that has two colums. See sample below: 105550 0.28 105550 0.24 125550 0.28 125550 0.24 215650 0.28 215650 0.24 315550 0.28 315550 0.24 335550 0.28 335550 0.24 40555 0.21 40555 0.17 415550 0.21 415550 0.17 43555 0.21 43555 0.17 (5 Replies)
Discussion started by: kekanap
5 Replies

3. UNIX for Dummies Questions & Answers

Data Manipulation

Dear Sir, I have file input RGR001|108.28|-2.86489|100-120|RANGGAR RGR002|108.071|-2.69028|80-100|RANNGAR RGR003|108.168|-2.97053|50-80|RANNGAR RGR007|108.192722222|-2.766138889|0-50|RANGGARI want to create files by joining each rows with each rows below Output as below ... (4 Replies)
Discussion started by: radius
4 Replies

4. UNIX for Dummies Questions & Answers

Data manipulation

Hallo Team, I need to manipulate existing data file. Have a look at current data and expected data: Current Data: 27873517141 27873540000 27873515109 27873517140 27873540001 27873540000 27873501343 27873540000 27873517140 27873511292 27873645989 27873540000 27873540000... (7 Replies)
Discussion started by: kekanap
7 Replies

5. Shell Programming and Scripting

Data manipulation using shell

Dear all I have a dataset (in text format,delimited by tab) which have 100 variables (say, var0-var99) and more than 100,000 observations. I want to do the following: 1. for variable var0-var49, I want to add "00" in front of each data (for example, "1" would become "001") 2. for variable... (8 Replies)
Discussion started by: littlewenwen
8 Replies

6. Shell Programming and Scripting

Help with data manipulation script

Y,T,,H05,6,6,0,0 -> TH05_6 D,5,BT,B -> BT_KIOSK P,KQC222 -> KQC222 G,B,2 -> BRANI_GATE_2 fileA TPM658 Y,T,,H05,6,6,0,0 TPM110 D,5,BT,B TPM136 P,KQC222 TPM180 P,BQC913 TPM575 Y,B,,T05,14,14,0,0 IPM760 G,B,2 TPM011 I need to use second column $1,$2,$3,$4..... if first char... (6 Replies)
Discussion started by: ment0smintz
6 Replies

7. UNIX for Dummies Questions & Answers

Script for data manipulation

Hi all! my first post here, so mods -- if this should ideally be in the scripts section, please move there. Thanks! I have data in the following format: key1:value1 key2:value2 key3:value3 A B C D key1:value4 key2:value5 key3:value6 A1 B1 key1: ... and so on I want an output... (2 Replies)
Discussion started by: gnat01
2 Replies

8. Shell Programming and Scripting

Data manipulation from a file

i have a file in follwing format 0110 1020 1011 1032 1020 2005 2003 1050 i want the output in such a way that all non zero numbers will be converted into 1 like this 0110 1010 1011 1011 1010 1001 1001 1010 (3 Replies)
Discussion started by: vaibhavkorde
3 Replies

9. Shell Programming and Scripting

Tricky data manipulation...

Hi everyone.. I am new here, hello.. I hope this doesn't come across to you folks as a stupid question, I'm somewhat new to scripting :) I'm seeking some help in finding a way to manipulate data output for every two characters - example: numbers.lst contains the following output:... (3 Replies)
Discussion started by: explicit
3 Replies

10. UNIX for Dummies Questions & Answers

data manipulation script

I have a folder called {homedata} Within this folder there are 12 subfolders 200601.......200612 Within each subfolder there are 8 sets of files Each filename commences with A B C D E F G or H, so {filename}* can be used. I am trying to write a script which will from the top level go... (1 Reply)
Discussion started by: grinder182533
1 Replies
Login or Register to Ask a Question