Sort file data according to a custom list of string


Login or Register for Dates, Times and to Reply

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Sort file data according to a custom list of string
# 1  
Sort file data according to a custom list of string

I have a string of pre defined ip address list which will always remain constant their order will never change like in below sample:

Code:
iplist=8.8.5.19,9.7.5.14,12.9.9.23,8.8.8.14,144.1.113

In the above example i m considering only 5 ips but there could be many more.

Now i have a file which will have a subset of these IPs along with a few file details and few blank empty new lines in any random order in each run like in the sample data below.
cat hello.txt
Code:
144.1.113

-rwxrw-rw- /tmp/var/file11.txt 56427

8.8.5.19

-rwxrw-rw- /tmp/var/file1.txt 56427
-rwxr--rw- /tmp/var/file2.txt 44427
-rwxr--rw- /tmp/var/file5.txt 36427
8.8.8.14
-rwxrw-rw- /tmp/var/file21.txt 56427
-rwxr--rw- /tmp/var/file22.txt 44427

I wish to format sort the above output in same order as the iplist variable. My expected output is as below:
Code:
8.8.5.19
-rwxrw-rw- /tmp/var/file1.txt 56427
-rwxr--rw- /tmp/var/file2.txt 44427
-rwxr--rw- /tmp/var/file5.txt 36427

9.7.5.14

12.9.9.23

8.8.8.14
-rwxrw-rw- /tmp/var/file21.txt 56427
-rwxr--rw- /tmp/var/file22.txt 44427

144.1.113
-rwxrw-rw- /tmp/var/file11.txt 56427

My idea is to have two have to arrays one that stores the iplist in the same order as iplist variable and the second that stores its respective file details (blank incase that ip is not present in my output). That way i will be able to use a single counter and display the desired output in the desired order.

Considering my approach is fine I was able to add the iplist string to an array using the below:

Code:
filedets=()
IFS=',' read -ra my_array <<< "$iplist"
for i in "${my_array[@]}"
do
    if [ grep $i hello.txt ]; then 
# need logic to get file details for the matched IP
filedets+=
else
filedets+=""
fi
done

I'm able to loop this array and store empty entries incase the ip is not found the hello.txt

The challenge for me is how can I extract the file details under a particular ip and store in a parellel array.

Any good way to have such a custom grouping and sorting? Kindly suggest.

Last edited by mohtashims; 1 Week Ago at 02:33 AM..
# 2  
Dear mohtashims,

[reply before original post edited]

With nearly 1000 posts on this site; you well know the culture here by this time.

You should post your code, what you have tried.

As you know, all of this kind of basic text processing is "possible" and there is no best way to do it and there is no "right or wrong" ways. What works for "person a" may not be the best solution for "person b".. and so forth.

As Yoda said in Star Wars, "There is no try".... "Only Do". (or something like that).

So, if you want to process some text, you should choose your tools of choice (based on your computing culture and environment) and you can then write your own code to process your text and post your code here, post your sample input, your sample output and any error messages, and they and only then, as for help based on your code, input, output and error messages (if any) along with the details of your operating system.
This User Gave Thanks to Neo For This Post:
# 3  
Quote:
Originally Posted by Neo
Dear mohtashims,

[reply before original post edited]

With nearly 1000 posts on this site; you well know the culture here by this time.

You should post your code, what you have tried.

As you know, all of this kind of basic text processing is "possible" and there is no best way to do it and there is no "right or wrong" ways. What works for "person a" may not be the best solution for "person b".. and so forth.

As Yoda said in Star Wars, "There is no try".... "Only Do". (or something like that).

So, if you want to process some text, you should choose your tools of choice (based on your computing culture and environment) and you can then write your own code to process your text and post your code here, post your sample input, your sample output and any error messages, and they and only then, as for help based on your code, input, output and error messages (if any) along with the details of your operating system.
@Neo Hi,

Thank you for the reply.

I was not sure if my approach is good / correct and hence did not share the attempts made.

I have updated my original post with the code of what I tried.

Last edited by mohtashims; 1 Week Ago at 02:29 AM..
This User Gave Thanks to mohtashims For This Post:
# 4  
awk OK? Try
Code:
awk -v"IPL=$iplist" '
BEGIN           {for (MX=n=split(IPL, T, ","); n; n--) IPS[T[n]] = n
                }
$1 in IPS       {SORTC = IPS[$1]
                 DONE[$1] = 1
                 CNT = 0
                }
                {print SORTC, ++CNT, $0
                }
END             {for (i in IPS) if (!DONE[i]) print IPS[i], 1, i, ORS IPS[i], 2
                }
' OFS="\t" file | sort -n | cut -f3-

# 5  
Quote:
Originally Posted by RudiC
awk OK? Try
Code:
awk -v"IPL=$iplist" '
BEGIN           {for (MX=n=split(IPL, T, ","); n; n--) IPS[T[n]] = n
                }
$1 in IPS       {SORTC = IPS[$1]
                 DONE[$1] = 1
                 CNT = 0
                }
                {print SORTC, ++CNT, $0
                }
END             {for (i in IPS) if (!DONE[i]) print IPS[i], 1, i, ORS IPS[i], 2
                }
' OFS="\t" file | sort -n | cut -f3-

awk often errors with too long string message hence would appreciate non seek solution. Also if the solution is on the lines of the code i shared it would be really great.

Last edited by mohtashims; 1 Week Ago at 08:12 AM..
# 6  
Quote:
Originally Posted by mohtashims
Awk often errors with to long strings
Difficult to believe. How long would a string need to be to buffalo awk?


Quote:
hence would appreciate non seek solution.
What would that be, "non seek"?


How about
Code:
declare -A IPASS
IFS=,
for i in $iplist;   do  IPASS[$i]=${#IPASS[@]};   done
IFS=$' \t\n'

{ while read LN
    do [ ${IPASS[$LN]} ] &&     { RC=${IPASS[$LN]}
                                  unset IPASS[$LN]
                                }
        echo "$RC       $LN"
    done < file
  for i in ${!IPASS[@]}
    do  echo "${IPASS[$i]}      $i"
    done
} | 
    sort -s | cut -f2-


Last edited by RudiC; 1 Week Ago at 08:23 AM..
This User Gave Thanks to RudiC For This Post:
# 7  
Quote:
Originally Posted by RudiC
Difficult to believe. How long would a string need to be to buffalo awk?


What would that be, "non seek"?


How about
Code:
declare -A IPASS
IFS=,
for i in $iplist;   do  IPASS[$i]=${#IPASS[@]};   done
IFS=$' \t\n'

{ while read LN
    do [ ${IPASS[$LN]} ] &&     { RC=${IPASS[$LN]}
                                  unset IPASS[$LN]
                                }
        echo "$RC       $LN"
    done < file
  for i in ${!IPASS[@]}
    do  echo "${IPASS[$i]}      $i"
    done
} | 
    sort -s | cut -f2-

I will check on this tomorrow and update if it works
Login or Register for Dates, Times and to Reply

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Test Your Knowledge in Computers #602
Difficulty: Medium
MySQL 8.0 incorporates a transactional data dictionary that stores information about database objects.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Sort data in text file in particular format

I have to sort below output in text file in unix bash 20170308 DA,I,113 20170308 PM,I,123 20170308 DA,U,22 20170308 PM,U,123 20170309 DA,I,11 20170309 PM,I,23 20170309 DA,U,123 20170309 PM,U,233 (8 Replies)
Discussion started by: Adfire
8 Replies

2. Shell Programming and Scripting

Sort data file by case

Hello, I'm trying to sort a large data file by the 3rd column so that all of the first words in the 3rd column that are in all uppercase appear before (or after) the non uppercase words. For example, Data file: xxx 12345 Rat in the house xxx 12345 CAT in the hat xxx 12345 Dog in the... (4 Replies)
Discussion started by: palex
4 Replies

3. UNIX for Advanced & Expert Users

Sort mixed data file

I have a text file and each field is separated by semicolon ( ; ). Field number 7 is internally separated by comma ( , ) and pipe ( | ) symbol. I want to sort file based on three different fields which are marked in BOLD. Here first BOLD field will have numbers upto the length of 9 characters,... (6 Replies)
Discussion started by: jnrohit2k
6 Replies

4. Shell Programming and Scripting

Advanced: Sort, count data in column, append file name

Hi. I am not sure the title gives an optimal description of what I want to do. Also, I tried to post this in the "UNIX for Dummies Questions & Answers", but it seems no-one was able to help out. I have several text files that contain data in many columns. All the files are organized the same... (14 Replies)
Discussion started by: JamesT
14 Replies

5. Shell Programming and Scripting

Ignore Header and Footer and Sort the data in fixed width file

Hi Experts, I want to Sort the data in fixed width file where i have Header and Footer also in file. I m using below commad to do the sort based on field satarting from 15 position to 17 position , but it is not ignoring the Header and Footer of the file while sorting. In the output i am... (5 Replies)
Discussion started by: sasikari
5 Replies

6. Shell Programming and Scripting

Sort a big data file

Hello, I have a big data file (160 MB) full of records with pipe(|) delimited those fields. I`m sorting the file on the first field. I'm trying to sort with "sort" command and it brings me 6 minutes. I have tried with some transformation methods in perl but it results "Out of memory". I was... (2 Replies)
Discussion started by: rubber08
2 Replies

7. Shell Programming and Scripting

script to sort a string of numerical data in set fields

So, I will be working with someone and basically we are trying to build a form that is submitted most likely via the web and the data is just a string of numbers. like: 19383882872201110929282821818182827349190102837364718191001932873711 Now, each number is part of a numerical value of... (4 Replies)
Discussion started by: tlarkin
4 Replies

8. Shell Programming and Scripting

Data after a string in a file

Hi, I have a file with data as follows: 324dsadfasfgf23<xmlsssssssssssssssssssssssssssssssssssssss<fdf>dfsdf> i need all the data followed by the string <xml? Is it possible to retrieve it? (1 Reply)
Discussion started by: Vijay06
1 Replies

9. UNIX for Dummies Questions & Answers

get the data in a file into single string

Hello everyone ! I need to get the data in a file into a string. file1 1 2 3 4 5 I need to write a script where the file1 info is stored in a string (say variable s). So the output should be 1,2,3,4,5 or (1,2,3,4,5) If i say echo $s or print $s output should be 1,2,3,4,5 or ... (7 Replies)
Discussion started by: i.scientist
7 Replies

10. Programming

FILE to String data types

Hi, I am trying to read a FILE data type into a String data type, I tried to make the FILE* veriable just equal the String veriable but i get: Conversion from 'FILE*' to 'String' is ambiguous Has anyone got any idea? Thanks (4 Replies)
Discussion started by: cb.mark
4 Replies