Sponsored Content
Top Forums Shell Programming and Scripting Merging Frequencies in a File Post 302507192 by gimley on Wednesday 23rd of March 2011 03:55:34 AM
Old 03-23-2011
Many thanks. The solution worked. As I mentioned I had written a C program to do the job and in spite of bucketing the data, it was very slow. I compared the output of the C program with the awk output and both were the same with the basic difference that the awk program took a very short time: hardly a few seconds to run through a file of 700,000 records.
The only difference is that my program sorts as per frequency: highest to lowest, but that is not a big issue.
How does one massage a numeric data in awk to sort data in terms of frequency.
Thanks once again

Gimley
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Merging Two File Horizontally

I am trying to merge two large file horizontally using paste command. Every thing is working fine except for time. Its taking lot of time. Is there any effiecient way of doing the same thing or is there anyway by which I can improve its perfomance programatically? Thanks, Yeheya (1 Reply)
Discussion started by: yeheyaansari
1 Replies

2. UNIX for Dummies Questions & Answers

merging two lines in a file

Hi All, I want to merge two lines in a file till the end of the file. So what could be the command to get so. say file name : sample.txt contents: country=1 send apps =1 rece=2 country=2 send apps =3 rece=3 .. ... output: country=1;send apps =1 rece=2 country=2;send apps =3... (6 Replies)
Discussion started by: thaduka
6 Replies

3. UNIX for Dummies Questions & Answers

merging 2 file

I have 2 files file1.txt a 123 aqsw c 234 sfdr fil2.txt b 345 hgy d 4653 jgut I want to merger in such a manner the the output file should be outfile.txt a 123 aqsw b 345 hgy c 234 sfdr d 4653 jgut Do we have any command to achive this? (8 Replies)
Discussion started by: siba.s.nayak
8 Replies

4. Shell Programming and Scripting

Extracting a column from a file and merging with other file using awk

Hi All: I have following files: File 1: <header> text... text .. text .. text .. <\header> x y z ... File 2: <header> text... text .. text .. (4 Replies)
Discussion started by: mrn006
4 Replies

5. Shell Programming and Scripting

Recalculating frequencies

My file looks like this The first 2 sequences are identical (different ID and frequencies though). The same thing for the last 2. What I need is to compare all sequences within the file and if they are identical, they need to be 'compressed' to one entry and the frequency should be recalculated.... (8 Replies)
Discussion started by: Xterra
8 Replies

6. Shell Programming and Scripting

Appending lines with word frequencies, ordering and indexing a column

Dear All, I have the following input data: w1 20 g1 w1 10 g1 w2 12 g1 w2 23 g1 w3 10 g1 w3 17 g1 w3 12.5 g1 w3 21 g1 w4 11 g1 w4 13.2 g1 w4 23 g1 w4 18 g1 First I seek to find the word frequencies in col1 and sort col2 in ascending order for each change in a col1 word. Second,... (5 Replies)
Discussion started by: Ghetz
5 Replies

7. Shell Programming and Scripting

Merging data from one file into another

Hello, I have a master database of a dictionary with the following structure: a=b (b is a Unicode string) a is the English part and b is the equivalent in a foreign language I have also another file which has a database where the /b/ part of the string has been corrected by an expert. let us... (5 Replies)
Discussion started by: gimley
5 Replies

8. Shell Programming and Scripting

merging two file

Dear All, I have two file like this: file1: a1234 b1235 c4678 d7859 file2 : e4575 f7869 g7689 h9687 I want output like this: a1234 b1235 c4678 (2 Replies)
Discussion started by: attila
2 Replies

9. Shell Programming and Scripting

Removal of extra spaces in *.log files to allow extraction of frequencies

Our university has upgraded its version of a computational chemistry program that our group uses quite regularly. In the past we have been able to extract frequency spectra from log files that are generated. Since the upgrade, the viewing program errors out. I've been able to trace down the changes... (16 Replies)
Discussion started by: wsuchem
16 Replies

10. UNIX for Dummies Questions & Answers

Gaps and frequencies

I have this infile: >GHL8OVD01BNNCA Freq 10 TAGATGTGCCCGTGGGTTTCCCGTCAACACCGGATAGT-GCAGCA-TA >GHL8OVD01CMQVT Freq 1 TTGATGTCGTGGGTTTCCCGTCAACACCGGCAAATAGT-GCAGCA-TA >GHL8OVD01CMQVT Freq 1 TTGATGTGCCAGTTTCCCGTCTAGCAGCACTACCAGGACCTTCGC-TA >GHL8OVD01CMQVW Freq 1... (1 Reply)
Discussion started by: Xterra
1 Replies
PROC_OPEN(3)								 1							      PROC_OPEN(3)

proc_open - Execute a command and open file pointers for input/output

SYNOPSIS
resource proc_open (string $cmd, array $descriptorspec, array &$pipes, [string $cwd], [array $env], [array $other_options]) DESCRIPTION
proc_open(3) is similar to popen(3) but provides a much greater degree of control over the program execution. PARAMETERS
o $cmd - The command to execute o $descriptorspec - An indexed array where the key represents the descriptor number and the value represents how PHP will pass that descriptor to the child process. 0 is stdin, 1 is stdout, while 2 is stderr. Each element can be: oAn array describing the pipe to pass to the process. The first element is the descriptor type and the second element is an option for the given type. Valid types are pipe (the second element is either r to pass the read end of the pipe to the process, or w to pass the write end) and file (the second element is a filename). o A stream resource representing a real file descriptor (e.g. opened file, a socket, STDIN). The file descriptor numbers are not limited to 0, 1 and 2 - you may specify any valid file descriptor number and it will be passed to the child process. This allows your script to interoperate with other scripts that run as "co-processes". In particular, this is useful for passing passphrases to programs like PGP, GPG and openssl in a more secure manner. It is also useful for reading status information provided by those programs on auxiliary file descriptors. o $pipes - Will be set to an indexed array of file pointers that correspond to PHP's end of any pipes that are created. o $cwd - The initial working dir for the command. This must be an absolute directory path, or NULL if you want to use the default value (the working dir of the current PHP process) o $env - An array with the environment variables for the command that will be run, or NULL to use the same environment as the current PHP process o $other_options - Allows you to specify additional options. Currently supported options include: o suppress_errors (windows only): suppresses errors generated by this function when it's set to TRUE o bypass_shell (windows only): bypass cmd.exe shell when set to TRUE RETURN VALUES
Returns a resource representing the process, which should be freed using proc_close(3) when you are finished with it. On failure returns FALSE. CHANGELOG
+--------+---------------------------------------------------+ |Version | | | | | | | Description | | | | +--------+---------------------------------------------------+ | 5.2.1 | | | | | | | Added the bypass_shell option to the | | | $other_options parameter. | | | | +--------+---------------------------------------------------+ EXAMPLES
Example #1 A proc_open(3) example <?php $descriptorspec = array( 0 => array("pipe", "r"), // stdin is a pipe that the child will read from 1 => array("pipe", "w"), // stdout is a pipe that the child will write to 2 => array("file", "/tmp/error-output.txt", "a") // stderr is a file to write to ); $cwd = '/tmp'; $env = array('some_option' => 'aeiou'); $process = proc_open('php', $descriptorspec, $pipes, $cwd, $env); if (is_resource($process)) { // $pipes now looks like this: // 0 => writeable handle connected to child stdin // 1 => readable handle connected to child stdout // Any error output will be appended to /tmp/error-output.txt fwrite($pipes[0], '<?php print_r($_ENV); ?>'); fclose($pipes[0]); echo stream_get_contents($pipes[1]); fclose($pipes[1]); // It is important that you close any pipes before calling // proc_close in order to avoid a deadlock $return_value = proc_close($process); echo "command returned $return_value "; } ?> The above example will output something similar to: Array ( [some_option] => aeiou [PWD] => /tmp [SHLVL] => 1 [_] => /usr/local/bin/php ) command returned 0 NOTES
Note Windows compatibility: Descriptors beyond 2 (stderr) are made available to the child process as inheritable handles, but since the Windows architecture does not associate file descriptor numbers with low-level handles, the child process does not (yet) have a means of accessing those handles. Stdin, stdout and stderr work as expected. Note If you only need a uni-directional (one-way) process pipe, use popen(3) instead, as it is much easier to use. SEE ALSO
popen(3), exec(3), system(3), passthru(3), stream_select(3), The backtick operator. PHP Documentation Group PROC_OPEN(3)
All times are GMT -4. The time now is 07:13 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy