Formatting and replacing according to block values


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Formatting and replacing according to block values
# 1  
Old 10-08-2014
Formatting and replacing according to block values

please consider the following file, there are repeated blocks of m values, and nested s values within. there are 2 columns (cols 3 and 4)associated with each m,s combination. All s1 rows must get a value of a(col 3 in output), all s2 values must get a value of b(col 3 in output). If s1 and s2 rows have the same value as in m3 in the data, the block is ignored.The rest of the s values within an m block has to be classified into a, b or c(if 3rd and 4th cols dont match either s1 or s2)

this is only an example, there are ~100k of data points in the original file. i`ll be obliged if helped with a solution code. i`m running cygwin on a windows machine with 8gb ram.

Code:
Input

m1,s1,x1,x1
m1,s2,x2,x2
m1,s3,x2,x2
m1,s4,x1,x1
m1,s5,x1,x2
m2,s1,y2,y2
m2,s2,y1,y1
m2,s3,y2,y1
m2,s4,y2,y1
m2,s5,y1,y1
m3,s1,y2,y2
m3,s2,y2,y2
m3,s3,y2,y1
m3,s4,y2,y1
m3,s5,y1,y1

Output

m1,s1,a
m1,s2,b
m1,s3,b
m1,s4,a
m1,s5,c
m2,s1,a
m2,s2,b
m2,s3,c
m2,s4,c
m2,s5,b

# 2  
Old 10-09-2014
Not sure if your data file is sorted like the sample data.

So I allowed for unsorted:

Code:
awk -F, '
{
   if(!($1 in m)) {
      m[$1]++
      s[$1]=$2
      v[$1,1]=$3 FS $4
      next
   }
   if(m[$1]==1) {
      m[$1]++
      if(v[$1,1] == $3 FS $4) v[$1,1]=""
      else {
         print $1, s[$1], "a"
         v[$1,2]=$3 FS $4
      }
   }
   if(!v[$1,1]) next

   for(i=1;i<=m[$1];i++)
       if(v[$1,i] == $3 FS $4) {
           printf "%s%c\n", $1 FS $2 FS, 96 + i
           next
        }
   v[$1,++m[$1]]=$3 FS $4
   printf "%s%c\n", $1 FS $2 FS, 96 + m[$1]
}' OFS=, infile

If the real data is sorted and fields 1 and 2 are really numbered like in the sample (m1, m2, m3, etc) I could make a more efficient solution.
This User Gave Thanks to Chubler_XL For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Replacing values inside a file.

Good day guys, I'm having trouble in creating a logic when it comes to replacing the values inside a file. I tried using sed command but it just doesn't work the way I want it to be. Here is what I'm trying to achieve. If my input file contains the values below. NAME++GUEST1 ++GUESS2++... (3 Replies)
Discussion started by: asdfghjkl
3 Replies

2. UNIX for Dummies Questions & Answers

Help in replacing column values

Hello All, I am having the file as below .I need to replace column 9-12 with some other values. In the below file I need to replace 1509 to 1508 and 1508 to 1507 .Can you please help me in how to do that Thanks, Arun ... (10 Replies)
Discussion started by: arunkumar_mca
10 Replies

3. Shell Programming and Scripting

awk file to read values from Db2 table replacing hard coded values

Hi, I want to replace a chain of if-else statement in an old AWK file with values from Db2 table or CSV file. The part of code is below... if (start_new_rec=="true"){ exclude_user="false"; user=toupper($6); match(user, "XXXXX."); if (RSTART ==2 ) { ... (9 Replies)
Discussion started by: asandy1234
9 Replies

4. UNIX for Dummies Questions & Answers

Replacing values in a column if they equal a certain value

Hi, I have a tab delimited text file where some lines have the string "NA" in the second column. For these lines, I want to replace NA with the value in the first column, the symbol underscore followed by the value in the fourth column. How do I go about doing that? Thanks! Input: 1 ... (3 Replies)
Discussion started by: evelibertine
3 Replies

5. Shell Programming and Scripting

Replacing column values

hi all , ( perl) i am trying to replace jst one column in a file for eg month dayofweek dealar car-name jan thurs mercedes c300 feb wed lexus is300 all this data is in a master file and i want to replace jan with 1 feb... (5 Replies)
Discussion started by: technoman
5 Replies

6. Shell Programming and Scripting

replacing negative values in a column with zero

Hi, i need help on replacing negative values in a column with 0. any quick fix on this? thanks much. for instance, input: 1 2.3 -0.4 -25 12 13 45 -12 desired output 1 2.3 0 0 12 13 45 (4 Replies)
Discussion started by: ida1215
4 Replies

7. Shell Programming and Scripting

Replacing string values from a File

I want to replace string values from a file to a file For eg : File1 has 30 lines of string with values, those specific values needs to be changed in file2 and remaining values in file2 should be as it is. For example: From file (File1) cluster.name=secondaryCluster To replace File... (9 Replies)
Discussion started by: sriram003
9 Replies

8. Shell Programming and Scripting

Replacing File values

Currently I am using the tr command in 3 scenarios for a ksh script. 1) Replacing any spaces in the file with a ~ tr ' ' '~' <$orignalFile> $newFile 2) After certain processing is done at the end of the scirpt i convert the Tilde back to spaces tr ' ' '~' <$newFile> $newFile2 3) Last... (4 Replies)
Discussion started by: hgjdv
4 Replies

9. UNIX for Dummies Questions & Answers

Formatting values in byte

Hello Does a command already exist for formatting numbers in byte for example? Just thought I would ask if there is a function for it or if I should write one myself, don't want to reinvent the wheel. Example: 10 = 10 B 1234 = 1.234 kB 1500000 = 1.5 MB Thanks by advance (1 Reply)
Discussion started by: Indalecio
1 Replies

10. Shell Programming and Scripting

replacing a line that may have different values on different servers

Replace disable_functions in php.ini with value of your choice which may be different on different servers ================================================ -bash-2.05b# grep disable_functions /usr/local/lib/php.ini disable_functions = 1 2 e weq t ret rye y etyhty rt et e ... (3 Replies)
Discussion started by: fed.linuxgossip
3 Replies
Login or Register to Ask a Question