03-17-2013
Reducing text file using similar lines
Hello,
I am a java programmer but want to try unix for a purpose where I need to reduce a file using its first field.. Here is the sample data:
HTML Code:
admin;2;0;[yrral];[]
admission;8;0;[timlu];[]
aman;1;0;[ev];[]
caroline;0;4;[];[luis, asethi]
cook;0;4;[];[shekhar, raj]
cook;2;0;[lew];[]
far;0;3;[];[venk]
far;1;5;[shekhar];[venk, raj]
I am explaining the dataset first. There are five fields separated by ";". First field is the main id, second and third are numerics, and 4th and 5th fields are list.
I need to combine all lines in the file where the first field matches. By Combination means, The 2nd and 3rd fields should be added and 4th anf 5th fields (lists) should be combined uniquely.
So, the desired output should be:
HTML Code:
admin;2;0;[yrral];[]
admission;8;0;[timlu];[]
aman;1;0;[ev];[]
caroline;0;4;[];[luis, asethi]
cook;2;4;[lew];[shekhar, raj]
far;1;8;[shekhar];[venk, raj]
I could have done this java but want to use the power of unix to get it done fast, since I have tonnes of such very large files.
Thanks a lot.
Shekhar
---------- Post updated at 10:23 PM ---------- Previous update was at 07:16 PM ----------
Please help.
Last edited by shekhar2010us; 03-17-2013 at 08:19 PM..
Reason: I provided the actual email-ids.
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi!
I have a trouble with the sort and the uniq.
I know I have to use them, I just have trouble with putting them in the right order.
I have a text file with unsorted lines (each line has a few words, the first word in the line is a number.).
I need to sort this file in order to be... (6 Replies)
Discussion started by: shira
6 Replies
2. Infrastructure Monitoring
I have a log file "logreport" that contains several lines as seen below:
04:20:00 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but responded to ping
06:38:08 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but responded to ping
07:11:05 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead... (4 Replies)
Discussion started by: Nysif Steve
4 Replies
3. Shell Programming and Scripting
I have a file which contains data as below:
nbk1j7o pageName=/jsp/RMBS/RMBSHome.jsf
nbk1j7o pageName=/jsp/RMBS/RMBSHome.jsf
nbk1j7o pageName=/jsp/RMBS/RMBSHome.jsf
nbk1j7o pageName=/jsp/RMBS/RMBSHome.jsf
nbk1j7o pageName=/jsp/common/index.jsf
nbk1j7o pageName=/jsp/common/index.jsf
nbk1wqe... (6 Replies)
Discussion started by: mohsin.quazi
6 Replies
4. Shell Programming and Scripting
Hello folks
I have a question for you gurus of sed or grep (maybe awk, but I would prefer the first two)
I have a file (f1) that says:
(actually, these are not numbers but md5sum, but for simplicity, let's assume these numbers.)
1
2
3
4
5And I have a file (f2) that says
1|a
1|b
1|c
2|d... (3 Replies)
Discussion started by: tukuyomi
3 Replies
5. UNIX for Dummies Questions & Answers
Hello! i have a text file.. which contains the data as follows
i want to merge the declarations lines pertaining to one datatype in to a single line as follows
i've searched the forum for help.. but couldn't find much help.. how can i do this?? (1 Reply)
Discussion started by: a_ba
1 Replies
6. Shell Programming and Scripting
Hi,
Here i have to check first record $3 $4 with second record $1 $2 respectively. If match found, then check first record $2 == second record $4 , if it equals , then reduce two records to single record like as desired output.
Input_file
1 1 2 1
2 1 3 1
3 1 4 1
3 1 3 2
desired... (3 Replies)
Discussion started by: vasanth.vadalur
3 Replies
7. UNIX for Dummies Questions & Answers
Hi,
I have a file with the lines as below:
C_10_A05_T7
C_10_A06_SP6
C_10_B05_SP6
C_10_B05_T7
C_10_B01_SP6
C_10_B01_T7
C_12_G07_SP6
C_12_G11_SP6
C_12_G11_T7
C_2_H18_T7
C_2_I02_SP6
C_2_I02_T7
C_2_I13_SP6
C_2_I17_SP6
The four segments of each line are connected by '_' symbols. I... (7 Replies)
Discussion started by: Fahmida
7 Replies
8. Shell Programming and Scripting
consider i have two files
cat onlyviews1.sql
CREATE VIEW V11
AS
SELECT id,
name,
FROM
etc etc
WHERE etc etc;
CREATE VIEW V22
AS
SELECT id,
name,
FROM
etc etc
WHERE etc etc;
CREATE VIEW V33
AS (10 Replies)
Discussion started by: vivek d r
10 Replies
9. Shell Programming and Scripting
Hello,
got a file with this structure:
33274 171030 02/29/2012 37897 P_GEH 2012-02-29 10:31:26
33275 171049 02/29/2012 38132 P_GEH 2012-02-29 10:35:27
33276 171058 02/29/2012 38515 P_GEH 2012-02-29 10:43:26
33277 170748 02/29/2012 40685 P_KOM ... (3 Replies)
Discussion started by: krecik28
3 Replies
10. Shell Programming and Scripting
hi all,
trying this using shell/bash with sed/awk/grep
I have two files, one containing one column, the other containing multiple columns (comma delimited).
file1.txt
abc12345
def12345
ghi54321
...
file2.txt
abc1,text1,texta
abc,text2,textb
def123,text3,textc
gh,text4,textd... (6 Replies)
Discussion started by: shogun1970
6 Replies
LEARN ABOUT DEBIAN
html::formfu::element::recaptcha
HTML::FormFu::Element::reCAPTCHA(3pm) User Contributed Perl Documentation HTML::FormFu::Element::reCAPTCHA(3pm)
NAME
HTML::FormFu::Element::reCAPTCHA - "Are you human" tester!
SYNOPSIS
---
elements:
- type: reCAPTCHA
name: recaptcha
public_key: $recaptcha_net_public_key
private_key: $recaptcha_net_private_key
DESCRIPTION
A wrapper around Captcha::reCAPTCHA. The reCAPTCHA fields aren't added to the form as "real" FormFu fields - so the values are never
available via params, etc. You can check that the reCAPTCHA verified correctly, by the usual methods: "submitted_and_valid" in
HTML::FormFu or "has_errors" in HTML::FormFu
This element automatically adds HTML::FormFu::Constraint::reCAPTCHA to itself - you should never add it yourself.
Although this is a subclass of HTML::FormFu::Element::Multi, you should not call "element()" or "elements()" to try to add other fields -
consider the reCAPTCHA element a black box.
METHODS
name
Required. Although not visibly used for anything, you must give this field a name for the reCAPTCHA constraint to work correctly.
public_key
Arguments: $public_key
Required. Obtained from <http://recaptcha.net>.
private_key
Arguments: $private_key
Required. Obtained from <http://recaptcha.net>.
ssl
Default Value: 'auto'.
Valid Values: '1', '0' or 'auto'
Whether to load the recaptcha.net files via "http" or "https".
If set to "auto", it will use "https" urls if the current page is running under ssl, otherwise it will use "http" urls.
recaptcha_options
Arguments: \%options
See the recaptcha.net API for details of valid options.
recaptcha_options:
lang: de
theme: white
constraint_args
Arguments: \%constraint_args
Options that will be passed to the HTML::FormFu::Constraint::reCAPTCHA that is automatically added for you.
---
elements:
- type: reCAPTCHA
name: recaptcha
constraint_args:
message: 'custom error message'
SEE ALSO
Is a sub-class of, and inherits methods from HTML::FormFu::Element::_Field, HTML::FormFu::Element::Multi, HTML::FormFu::Element::Block,
HTML::FormFu::Element
HTML::FormFu
AUTHOR
Carl Franks, "cfranks@cpan.org"
LICENSE
This library is free software, you can redistribute it and/or modify it under the same terms as Perl itself.
perl v5.14.2 2012-01-23 HTML::FormFu::Element::reCAPTCHA(3pm)