Sponsored Content
Top Forums Shell Programming and Scripting Merging Frequencies in a File Post 302507147 by gimley on Tuesday 22nd of March 2011 11:01:49 PM
Old 03-23-2011
Merging Frequencies in a File

hello,
I have a file which has the following structure:
word <TAB> frequency
The same word can have multiple frequencies:
John <TAB> 60
John <TAB> 20
John <TAB> 30
Mary <TAB> 1000
Mary <TAB> 800
Mary <TAB> 20
What I need is a script which could merge all these frequencies into one single frequency. The output would be
John<TAB> 110
Mary TAB> 1820
I have written a program in C which does it but is agonizingly slow, since the number of such instances is 100,000.
Could anybody help me with a perl script or an awk script which could do the job faster. I am a tyro in awk and perl and hence the request.
Many thanks in advance
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Merging Two File Horizontally

I am trying to merge two large file horizontally using paste command. Every thing is working fine except for time. Its taking lot of time. Is there any effiecient way of doing the same thing or is there anyway by which I can improve its perfomance programatically? Thanks, Yeheya (1 Reply)
Discussion started by: yeheyaansari
1 Replies

2. UNIX for Dummies Questions & Answers

merging two lines in a file

Hi All, I want to merge two lines in a file till the end of the file. So what could be the command to get so. say file name : sample.txt contents: country=1 send apps =1 rece=2 country=2 send apps =3 rece=3 .. ... output: country=1;send apps =1 rece=2 country=2;send apps =3... (6 Replies)
Discussion started by: thaduka
6 Replies

3. UNIX for Dummies Questions & Answers

merging 2 file

I have 2 files file1.txt a 123 aqsw c 234 sfdr fil2.txt b 345 hgy d 4653 jgut I want to merger in such a manner the the output file should be outfile.txt a 123 aqsw b 345 hgy c 234 sfdr d 4653 jgut Do we have any command to achive this? (8 Replies)
Discussion started by: siba.s.nayak
8 Replies

4. Shell Programming and Scripting

Extracting a column from a file and merging with other file using awk

Hi All: I have following files: File 1: <header> text... text .. text .. text .. <\header> x y z ... File 2: <header> text... text .. text .. (4 Replies)
Discussion started by: mrn006
4 Replies

5. Shell Programming and Scripting

Recalculating frequencies

My file looks like this The first 2 sequences are identical (different ID and frequencies though). The same thing for the last 2. What I need is to compare all sequences within the file and if they are identical, they need to be 'compressed' to one entry and the frequency should be recalculated.... (8 Replies)
Discussion started by: Xterra
8 Replies

6. Shell Programming and Scripting

Appending lines with word frequencies, ordering and indexing a column

Dear All, I have the following input data: w1 20 g1 w1 10 g1 w2 12 g1 w2 23 g1 w3 10 g1 w3 17 g1 w3 12.5 g1 w3 21 g1 w4 11 g1 w4 13.2 g1 w4 23 g1 w4 18 g1 First I seek to find the word frequencies in col1 and sort col2 in ascending order for each change in a col1 word. Second,... (5 Replies)
Discussion started by: Ghetz
5 Replies

7. Shell Programming and Scripting

Merging data from one file into another

Hello, I have a master database of a dictionary with the following structure: a=b (b is a Unicode string) a is the English part and b is the equivalent in a foreign language I have also another file which has a database where the /b/ part of the string has been corrected by an expert. let us... (5 Replies)
Discussion started by: gimley
5 Replies

8. Shell Programming and Scripting

merging two file

Dear All, I have two file like this: file1: a1234 b1235 c4678 d7859 file2 : e4575 f7869 g7689 h9687 I want output like this: a1234 b1235 c4678 (2 Replies)
Discussion started by: attila
2 Replies

9. Shell Programming and Scripting

Removal of extra spaces in *.log files to allow extraction of frequencies

Our university has upgraded its version of a computational chemistry program that our group uses quite regularly. In the past we have been able to extract frequency spectra from log files that are generated. Since the upgrade, the viewing program errors out. I've been able to trace down the changes... (16 Replies)
Discussion started by: wsuchem
16 Replies

10. UNIX for Dummies Questions & Answers

Gaps and frequencies

I have this infile: >GHL8OVD01BNNCA Freq 10 TAGATGTGCCCGTGGGTTTCCCGTCAACACCGGATAGT-GCAGCA-TA >GHL8OVD01CMQVT Freq 1 TTGATGTCGTGGGTTTCCCGTCAACACCGGCAAATAGT-GCAGCA-TA >GHL8OVD01CMQVT Freq 1 TTGATGTGCCAGTTTCCCGTCTAGCAGCACTACCAGGACCTTCGC-TA >GHL8OVD01CMQVW Freq 1... (1 Reply)
Discussion started by: Xterra
1 Replies
File::HomeDir::Unix(3)					User Contributed Perl Documentation				    File::HomeDir::Unix(3)

NAME
File::HomeDir::Unix - Find your home and other directories on legacy Unix SYNOPSIS
use File::HomeDir; # Find directories for the current user $home = File::HomeDir->my_home; # /home/mylogin $desktop = File::HomeDir->my_desktop; # All of these will... $docs = File::HomeDir->my_documents; # ...default to home... $music = File::HomeDir->my_music; # ...directory $pics = File::HomeDir->my_pictures; # $videos = File::HomeDir->my_videos; # $data = File::HomeDir->my_data; # DESCRIPTION
This module provides implementations for determining common user directories. In normal usage this module will always be used via File::HomeDir. SUPPORT
See the support section the main File::HomeDir module. AUTHORS
Adam Kennedy <adamk@cpan.org> Sean M. Burke <sburke@cpan.org> SEE ALSO
File::HomeDir, File::HomeDir::Win32 (legacy) COPYRIGHT
Copyright 2005 - 2011 Adam Kennedy. Some parts copyright 2000 Sean M. Burke. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. The full text of the license can be found in the LICENSE file included with this module. perl v5.16.3 2012-10-19 File::HomeDir::Unix(3)
All times are GMT -4. The time now is 06:57 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy