Sort file data according to a custom list of string


Login or Register for Dates, Times and to Reply

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Sort file data according to a custom list of string
# 1  
Sort file data according to a custom list of string

I have a string of pre defined ip address list which will always remain constant their order will never change like in below sample:

Code:
iplist=8.8.5.19,9.7.5.14,12.9.9.23,8.8.8.14,144.1.113

In the above example i m considering only 5 ips but there could be many more.

Now i have a file which will have a subset of these IPs along with a few file details and few blank empty new lines in any random order in each run like in the sample data below.
cat hello.txt
Code:
144.1.113

-rwxrw-rw- /tmp/var/file11.txt 56427

8.8.5.19

-rwxrw-rw- /tmp/var/file1.txt 56427
-rwxr--rw- /tmp/var/file2.txt 44427
-rwxr--rw- /tmp/var/file5.txt 36427
8.8.8.14
-rwxrw-rw- /tmp/var/file21.txt 56427
-rwxr--rw- /tmp/var/file22.txt 44427

I wish to format sort the above output in same order as the iplist variable. My expected output is as below:
Code:
8.8.5.19
-rwxrw-rw- /tmp/var/file1.txt 56427
-rwxr--rw- /tmp/var/file2.txt 44427
-rwxr--rw- /tmp/var/file5.txt 36427

9.7.5.14

12.9.9.23

8.8.8.14
-rwxrw-rw- /tmp/var/file21.txt 56427
-rwxr--rw- /tmp/var/file22.txt 44427

144.1.113
-rwxrw-rw- /tmp/var/file11.txt 56427

My idea is to have two have to arrays one that stores the iplist in the same order as iplist variable and the second that stores its respective file details (blank incase that ip is not present in my output). That way i will be able to use a single counter and display the desired output in the desired order.

Considering my approach is fine I was able to add the iplist string to an array using the below:

Code:
filedets=()
IFS=',' read -ra my_array <<< "$iplist"
for i in "${my_array[@]}"
do
    if [ grep $i hello.txt ]; then 
# need logic to get file details for the matched IP
filedets+=
else
filedets+=""
fi
done

I'm able to loop this array and store empty entries incase the ip is not found the hello.txt

The challenge for me is how can I extract the file details under a particular ip and store in a parellel array.

Any good way to have such a custom grouping and sorting? Kindly suggest.

Last edited by mohtashims; 01-13-2020 at 01:33 AM..
# 2  
Dear mohtashims,

[reply before original post edited]

With nearly 1000 posts on this site; you well know the culture here by this time.

You should post your code, what you have tried.

As you know, all of this kind of basic text processing is "possible" and there is no best way to do it and there is no "right or wrong" ways. What works for "person a" may not be the best solution for "person b".. and so forth.

As Yoda said in Star Wars, "There is no try".... "Only Do". (or something like that).

So, if you want to process some text, you should choose your tools of choice (based on your computing culture and environment) and you can then write your own code to process your text and post your code here, post your sample input, your sample output and any error messages, and they and only then, as for help based on your code, input, output and error messages (if any) along with the details of your operating system.
This User Gave Thanks to Neo For This Post:
# 3  
Quote:
Originally Posted by Neo
Dear mohtashims,

[reply before original post edited]

With nearly 1000 posts on this site; you well know the culture here by this time.

You should post your code, what you have tried.

As you know, all of this kind of basic text processing is "possible" and there is no best way to do it and there is no "right or wrong" ways. What works for "person a" may not be the best solution for "person b".. and so forth.

As Yoda said in Star Wars, "There is no try".... "Only Do". (or something like that).

So, if you want to process some text, you should choose your tools of choice (based on your computing culture and environment) and you can then write your own code to process your text and post your code here, post your sample input, your sample output and any error messages, and they and only then, as for help based on your code, input, output and error messages (if any) along with the details of your operating system.
@Neo Hi,

Thank you for the reply.

I was not sure if my approach is good / correct and hence did not share the attempts made.

I have updated my original post with the code of what I tried.

Last edited by mohtashims; 01-13-2020 at 01:29 AM..
This User Gave Thanks to mohtashims For This Post:
# 4  
awk OK? Try
Code:
awk -v"IPL=$iplist" '
BEGIN           {for (MX=n=split(IPL, T, ","); n; n--) IPS[T[n]] = n
                }
$1 in IPS       {SORTC = IPS[$1]
                 DONE[$1] = 1
                 CNT = 0
                }
                {print SORTC, ++CNT, $0
                }
END             {for (i in IPS) if (!DONE[i]) print IPS[i], 1, i, ORS IPS[i], 2
                }
' OFS="\t" file | sort -n | cut -f3-

# 5  
Quote:
Originally Posted by RudiC
awk OK? Try
Code:
awk -v"IPL=$iplist" '
BEGIN           {for (MX=n=split(IPL, T, ","); n; n--) IPS[T[n]] = n
                }
$1 in IPS       {SORTC = IPS[$1]
                 DONE[$1] = 1
                 CNT = 0
                }
                {print SORTC, ++CNT, $0
                }
END             {for (i in IPS) if (!DONE[i]) print IPS[i], 1, i, ORS IPS[i], 2
                }
' OFS="\t" file | sort -n | cut -f3-

awk often errors with too long string message hence would appreciate non seek solution. Also if the solution is on the lines of the code i shared it would be really great.

Last edited by mohtashims; 01-13-2020 at 07:12 AM..
# 6  
Quote:
Originally Posted by mohtashims
Awk often errors with to long strings
Difficult to believe. How long would a string need to be to buffalo awk?


Quote:
hence would appreciate non seek solution.
What would that be, "non seek"?


How about
Code:
declare -A IPASS
IFS=,
for i in $iplist;   do  IPASS[$i]=${#IPASS[@]};   done
IFS=$' \t\n'

{ while read LN
    do [ ${IPASS[$LN]} ] &&     { RC=${IPASS[$LN]}
                                  unset IPASS[$LN]
                                }
        echo "$RC       $LN"
    done < file
  for i in ${!IPASS[@]}
    do  echo "${IPASS[$i]}      $i"
    done
} | 
    sort -s | cut -f2-


Last edited by RudiC; 01-13-2020 at 07:23 AM..
This User Gave Thanks to RudiC For This Post:
# 7  
Quote:
Originally Posted by RudiC
Difficult to believe. How long would a string need to be to buffalo awk?


What would that be, "non seek"?


How about
Code:
declare -A IPASS
IFS=,
for i in $iplist;   do  IPASS[$i]=${#IPASS[@]};   done
IFS=$' \t\n'

{ while read LN
    do [ ${IPASS[$LN]} ] &&     { RC=${IPASS[$LN]}
                                  unset IPASS[$LN]
                                }
        echo "$RC       $LN"
    done < file
  for i in ${!IPASS[@]}
    do  echo "${IPASS[$i]}      $i"
    done
} | 
    sort -s | cut -f2-

I will check on this tomorrow and update if it works
Login or Register for Dates, Times and to Reply

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Test Your Knowledge in Computers #648
Difficulty: Medium
Window Maker was written from scratch by Tim Berners-Lee, best known as the inventor of the World Wide Web.
True or False?

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How do I custom sort the files in a directory using the filenames in a text file.?

Hi all, (5 Replies)
Discussion started by: KMusunuru
5 Replies

2. Shell Programming and Scripting

Sort and Split file with header and custom name

Hi, I am using SUN SOLARIS (SunOS sun4v sparc SUNW, T5240). I have a huge data file with header and trailer. This file gets used into an ETL process. ETL skips the header record (which is the first record of the file) and loads the rest of the record. The file can be delimited (comma,... (5 Replies)
Discussion started by: Saanvi1
5 Replies

3. Shell Programming and Scripting

Sort data file by case

Hello, I'm trying to sort a large data file by the 3rd column so that all of the first words in the 3rd column that are in all uppercase appear before (or after) the non uppercase words. For example, Data file: xxx 12345 Rat in the house xxx 12345 CAT in the hat xxx 12345 Dog in the... (4 Replies)
Discussion started by: palex
4 Replies

4. UNIX for Dummies Questions & Answers

Custom sort on a column

Hello all, How do I achieve this? I have A, B and A/B in different variables in a file in col2. I want them to sort in such a way, that the variables appear together, and within a variable, the data is sorted in the order A,B and then A/B. If I sort on the second column, the order becomes A,... (6 Replies)
Discussion started by: senhia83
6 Replies

5. Shell Programming and Scripting

Sort help: How to sort collected 'file list' by date stamp :

Hi Experts, I have a filelist collected from another server , now want to sort the output using date/time stamp filed. - Filed 6, 7,8 are showing the date/time/stamp. Here is the input: #---------------------------------------------------------------------- -rw------- 1 root ... (3 Replies)
Discussion started by: rveri
3 Replies

6. Shell Programming and Scripting

Populating File data with custom manipulation on file names

Hi, I am confused how to proceed firther please find the problem below: Input Files: DCIA_GEOG_DATA_OCEAN.TXT DCIA_GEOG_DATA_MCRO.TXT DCIA_GEOG_DATA_CVAS.TXT DCIA_GEOG_DATA_MCR.TXT Output File Name: MMA_RFC_GEOG_NAM_DIM_LOD.txt Sample Record(DCIA_GEOG_DATA_OCEAN.TXT):(Layout same for... (4 Replies)
Discussion started by: Arun Mishra
4 Replies

7. UNIX for Dummies Questions & Answers

List directories with given string, sort by creation date

It is for HP-Unix B.11.31. Requirement: 1. List the directories, which directories name has given particular string. Example: Directories with name "Build" 2. On the output of 1. list the directories by creation date as sort order. I tried with; find . -type d -name "Build*" ... (3 Replies)
Discussion started by: Siva SQL
3 Replies

8. Shell Programming and Scripting

find string(s) in text file and nearby data, export to list help

Hi, So I'm kinda new to shell scripts and the like. I've picked up quite a bit of use from browsing the forums here but ran into a new one that I can't seem to find an answer for. I'm looking to parse/find a string AND the next 15 or so charachters that follow the string within a text file... (1 Reply)
Discussion started by: kar23me
1 Replies

9. Shell Programming and Scripting

Sort a big data file

Hello, I have a big data file (160 MB) full of records with pipe(|) delimited those fields. I`m sorting the file on the first field. I'm trying to sort with "sort" command and it brings me 6 minutes. I have tried with some transformation methods in perl but it results "Out of memory". I was... (2 Replies)
Discussion started by: rubber08
2 Replies

10. Shell Programming and Scripting

script to sort a string of numerical data in set fields

So, I will be working with someone and basically we are trying to build a form that is submitted most likely via the web and the data is just a string of numbers. like: 19383882872201110929282821818182827349190102837364718191001932873711 Now, each number is part of a numerical value of... (4 Replies)
Discussion started by: tlarkin
4 Replies

Featured Tech Videos