Sponsored Content
Top Forums Shell Programming and Scripting Merging Frequencies in a File Post 302507164 by Chubler_XL on Wednesday 23rd of March 2011 12:26:42 AM
Old 03-23-2011
This seems pretty quick - tested on 130,000 record file and took about 2 secs.

Code:
awk '{T[$1]+=$2} END { for(u in T) print u"\t"T[u] }' FS="\t" infile

BTW C should be faster again, make sure you store your words (John/Mary/etc) in a B+tree or some other sort of indexed structure.
This User Gave Thanks to Chubler_XL For This Post:
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Merging Two File Horizontally

I am trying to merge two large file horizontally using paste command. Every thing is working fine except for time. Its taking lot of time. Is there any effiecient way of doing the same thing or is there anyway by which I can improve its perfomance programatically? Thanks, Yeheya (1 Reply)
Discussion started by: yeheyaansari
1 Replies

2. UNIX for Dummies Questions & Answers

merging two lines in a file

Hi All, I want to merge two lines in a file till the end of the file. So what could be the command to get so. say file name : sample.txt contents: country=1 send apps =1 rece=2 country=2 send apps =3 rece=3 .. ... output: country=1;send apps =1 rece=2 country=2;send apps =3... (6 Replies)
Discussion started by: thaduka
6 Replies

3. UNIX for Dummies Questions & Answers

merging 2 file

I have 2 files file1.txt a 123 aqsw c 234 sfdr fil2.txt b 345 hgy d 4653 jgut I want to merger in such a manner the the output file should be outfile.txt a 123 aqsw b 345 hgy c 234 sfdr d 4653 jgut Do we have any command to achive this? (8 Replies)
Discussion started by: siba.s.nayak
8 Replies

4. Shell Programming and Scripting

Extracting a column from a file and merging with other file using awk

Hi All: I have following files: File 1: <header> text... text .. text .. text .. <\header> x y z ... File 2: <header> text... text .. text .. (4 Replies)
Discussion started by: mrn006
4 Replies

5. Shell Programming and Scripting

Recalculating frequencies

My file looks like this The first 2 sequences are identical (different ID and frequencies though). The same thing for the last 2. What I need is to compare all sequences within the file and if they are identical, they need to be 'compressed' to one entry and the frequency should be recalculated.... (8 Replies)
Discussion started by: Xterra
8 Replies

6. Shell Programming and Scripting

Appending lines with word frequencies, ordering and indexing a column

Dear All, I have the following input data: w1 20 g1 w1 10 g1 w2 12 g1 w2 23 g1 w3 10 g1 w3 17 g1 w3 12.5 g1 w3 21 g1 w4 11 g1 w4 13.2 g1 w4 23 g1 w4 18 g1 First I seek to find the word frequencies in col1 and sort col2 in ascending order for each change in a col1 word. Second,... (5 Replies)
Discussion started by: Ghetz
5 Replies

7. Shell Programming and Scripting

Merging data from one file into another

Hello, I have a master database of a dictionary with the following structure: a=b (b is a Unicode string) a is the English part and b is the equivalent in a foreign language I have also another file which has a database where the /b/ part of the string has been corrected by an expert. let us... (5 Replies)
Discussion started by: gimley
5 Replies

8. Shell Programming and Scripting

merging two file

Dear All, I have two file like this: file1: a1234 b1235 c4678 d7859 file2 : e4575 f7869 g7689 h9687 I want output like this: a1234 b1235 c4678 (2 Replies)
Discussion started by: attila
2 Replies

9. Shell Programming and Scripting

Removal of extra spaces in *.log files to allow extraction of frequencies

Our university has upgraded its version of a computational chemistry program that our group uses quite regularly. In the past we have been able to extract frequency spectra from log files that are generated. Since the upgrade, the viewing program errors out. I've been able to trace down the changes... (16 Replies)
Discussion started by: wsuchem
16 Replies

10. UNIX for Dummies Questions & Answers

Gaps and frequencies

I have this infile: >GHL8OVD01BNNCA Freq 10 TAGATGTGCCCGTGGGTTTCCCGTCAACACCGGATAGT-GCAGCA-TA >GHL8OVD01CMQVT Freq 1 TTGATGTCGTGGGTTTCCCGTCAACACCGGCAAATAGT-GCAGCA-TA >GHL8OVD01CMQVT Freq 1 TTGATGTGCCAGTTTCCCGTCTAGCAGCACTACCAGGACCTTCGC-TA >GHL8OVD01CMQVW Freq 1... (1 Reply)
Discussion started by: Xterra
1 Replies
vodovod(6)							       Games								vodovod(6)

NAME
vodovod - lead the water from the house to the storage tank SYNOPSIS
vodovod DESCRIPTION
Vodovod is a free game, released under GNU GPL license. Graphics and programming are done by Milan Babuskov. This is an over-weekend project for now, so don't expect miracles. However, game is completely playable. The goal of the game is to reach the highest possible score. You get a limited number of pipes on each level and need to combine them to lead the water from the house at the top of the screen to the storage tank at the bottom. For each pipe water goes through, you get 20 points and if you fill the cross-pipe both ways, you get 60 points. At end of each level, you are awarded depending on the skill level: * Beginner: 100 points * Toolman: 100 points + number of pipes remaining * Master plumber: 100 points + 2 x number of pipes remaining Some of the levels have obstacles where you cannot place pipes, checkpoints through which the water must go and holes in the ground that slow the flow. Select HELP from the main menu to get more info. The game is playable with joystick/joypad: just move it and press buttons when you go to Options -> Configure controls USAGE
The goal of the game is to use pipes to connect the house at the top of the screen with the tank at the bottom. You have time to construct the initial pipeline before the man enters the house. For each level you get a limited number of pipes, so use them wisely. If you make a mistake, you can place another pipe over it. There are certain special blocks in the map: * Blocked Area - You cannot place pipes on these fields. * Checkpoint - Your pipeline must go through this field. * Hole in the ground - Slows down the flow. SEE ALSO
You can find more information at http://home.gna.org/vodovod/ May 2007 vodovod(6)
All times are GMT -4. The time now is 08:44 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy