Sponsored Content
Full Discussion: Help with uniq or awk??
Top Forums Shell Programming and Scripting Help with uniq or awk?? Post 302346007 by shinoman28 on Thursday 20th of August 2009 09:06:21 PM
Old 08-20-2009
Quote:
Originally Posted by Franklin52
It's gonna be difficult to find a balanced solution, your data is not consistent...

Regards
hmmmm..
ok i went tru again the generated file, there seems to be a pattern..the first 60% of the word is the same, e.g.

Apple_new_101 15
Apple_newMandarin 6
OrangeMango_new 6
OrangeMango_old 5

Algo:
1.grab the first 60% of the name and store it in a temp_name
2.create a file to put the filtered list e.g. filtered.txt
3.search filtered.txt, if temp_name already exists on the file
4. if it does, grab the number then sum it with the existing number on that name.
5.if it doesnt, store the name and number, and proceed with the next entry.

does this sound feasible? its just im not fluid with shell scripts.
thanks

---------- Post updated at 10:36 AM ---------- Previous update was at 09:15 AM ----------

actually thats fine, leave it for now.
i think to make it easier is to have a proper grouping of data and work from there.
thanks for the help.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

using uniq and awk??

I have a file that is populated: hits/books.hits:143.217.64.204 Thu Sep 21 22:24:57 GMT 2006 hits/books.hits:62.145.39.14 Fri Sep 22 00:38:32 GMT 2006 hits/books.hits:81.140.86.170 Fri Sep 22 08:45:26 GMT 2006 hits/books.hits:81.140.86.170 Fri Sep 22 09:13:57 GMT... (13 Replies)
Discussion started by: amatuer_lee_3
13 Replies

2. Shell Programming and Scripting

How to replicate data using Uniq or awk

Hi, I have this scenario; where there are two classes:- apple and orange. 1,2,3,4,5,6,apple 1,1,0,4,2,3,apple 1,3,3,3,3,4,apple 1,1,1,1,1,1,orange 1,2,3,1,1,1,orange Basically for apple, i have 3 entries in the file, and for orange, I have 2 entries. Im trying to edit the file and find... (5 Replies)
Discussion started by: ahjiefreak
5 Replies

3. Shell Programming and Scripting

Text Proccessing with sort,uniq,awk

Hello, I have a log file with the following input: X , ID , Date, Time, Y 01,01368,2010-12-02,09:07:00,Pass 01,01368,2010-12-02,10:54:00,Pass 01,01368,2010-12-02,13:07:04,Pass 01,01368,2010-12-02,18:54:01,Pass 01,01368,2010-12-03,09:02:00,Pass 01,01368,2010-12-03,13:53:00,Pass... (12 Replies)
Discussion started by: rollyah
12 Replies

4. Shell Programming and Scripting

[uniq + awk?] How to remove duplicate blocks of lines in files?

Hello again, I am wanting to remove all duplicate blocks of XML code in a file. This is an example: input: <string-array name="threeItems"> <item>item1</item> <item>item2</item> <item>item3</item> </string-array> <string-array name="twoItems"> <item>item1</item> <item>item2</item>... (19 Replies)
Discussion started by: raidzero
19 Replies

5. Shell Programming and Scripting

awk - getting uniq count on multiple col

Hi My file have 7 column, FIle is pipe delimed Col1|Col2|col3|Col4|col5|Col6|Col7 I want to find out uniq record count on col3, col4 and col2 ( same order) how can I achieve it. ex 1|3|A|V|C|1|1 1|3|A|V|C|1|1 1|4|A|V|C|1|1 Output should be FREQ|A|V|3|2 FREQ|A|V|4|1 Here... (5 Replies)
Discussion started by: sanranad
5 Replies

6. Shell Programming and Scripting

awk uniq and longest string of a column as index

I met a challenge to filter ~70 millions of sequence rows and I want using awk with conditions: 1) longest string of each pattern in column 2, ignore any sub-string, as the index; 2) all the unique patterns after 1); 3) print the whole row; input: 1 ABCDEFGHI longest_sequence1 2 ABCDEFGH... (12 Replies)
Discussion started by: yifangt
12 Replies

7. Shell Programming and Scripting

Rewriting GNU uniq in awk

Within a shell script I use uniq -w 16 -D in order to process all lines in which the first 16 characters are duplicated. Now I want to also run that script on a BSD based system where the included version of uniq does not support the -w (--check-chars) option. To get around this I have... (7 Replies)
Discussion started by: mij
7 Replies

8. Shell Programming and Scripting

Sort uniq or awk

Hi again, I have files with the following contents datetime,ip1,port1,ip2,port2,number How would I find out how many times ip1 field shows up a particular file? Then how would I find out how many time ip1 and port 2 shows up? Please mind the file may contain 100k lines. (8 Replies)
Discussion started by: LDHB2012
8 Replies

9. Shell Programming and Scripting

awk compare and keep uniq

Hi all I was wondering if you may help me in resolving an issue. In particular I have a file like this: the ... represent different string and what I wrote Cur or Ent are the constant. Well, what I would like to obtain is a file in which are reported only the ID in which the second column... (6 Replies)
Discussion started by: giuliangiuseppe
6 Replies

10. UNIX for Dummies Questions & Answers

awk or uniq

Hi Help, I have a file which looks like 1 20 30 40 50 60 6 2 20 30 40 50 60 8 7 20 30 40 50 60 7 4 30 40 50 60 70 8 5 30 40 50 60 70 9 2 30 40 50 60 70 8 I want the o/p as 1 20 30 40 50 60 6 4 30 40 50 60 70 8 Is there a way I can use uniq command or awk to do this? ... (11 Replies)
Discussion started by: Indra2011
11 Replies
ARRAY_POP(3)								 1							      ARRAY_POP(3)

array_pop - Pop the element off the end of array

SYNOPSIS
mixed array_pop (array &$array) DESCRIPTION
array_pop(3) pops and returns the last value of the $array, shortening the $array by one element. Note This function will reset(3) the array pointer of the input array after use. PARAMETERS
o $array - The array to get the value from. RETURN VALUES
Returns the last value of $array. If $array is empty (or is not an array), NULL will be returned. ERRORS
/EXCEPTIONS This function will produce an error of level E_WARNING when called on a non-array. EXAMPLES
Example #1 array_pop(3) example <?php $stack = array("orange", "banana", "apple", "raspberry"); $fruit = array_pop($stack); print_r($stack); ?> After this, $stack will have only 3 elements: Array ( [0] => orange [1] => banana [2] => apple ) and raspberry will be assigned to $fruit. SEE ALSO
array_push(3), array_shift(3), array_unshift(3). PHP Documentation Group ARRAY_POP(3)
All times are GMT -4. The time now is 02:56 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy