How to sort a content of a text file using a shell script?


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers How to sort a content of a text file using a shell script?
# 15  
Old 12-10-2014
Below is my awk + sort + awk approach.

Entries such as
Code:
cpq-media
cpq-telco
DSS-IPAD06
DSS-IPAD07
GSE-DocuSign
GSE-DocuSign1

are considered "unclassified", because there is no "env"/number after the "app".

The table is sorted COUNT -> descending (apart from "unclassified").

Code:
$ awk '                                                        
    $0 ~ /^.*[0-9][^-]*-/ { l=$0; sub(/[0-9].*/,"",$0); gsub(/^[^0-9]*|-.*/,"",l); A[$0]++; L[$0,l]++; next }
    { U[$0]++ }
END { for (i in A)
    { for (j in L) {split(j,a,SUBSEP); if (i == a[1]) T[i]++} printf "%-15s%8s\t%8s\n", i,T[i],A[i]}
    for (k in U) n++; printf "%-15s%8s\t%8s\n", "unclassified","N/A",n
}' input_file | sort -nrk3,3 | awk 'BEGIN { printf "%-15s%s\t%s\n", "TYPE","ENV","COUNT"
            printf "%-14s %s\t%s\n", "--------------","--------","--------" }
    $1 != "unclassified" { e+=$2; c+=$3; print }
    $1 == "unclassified" { l=$0;  e+=$2; c+=$3; next }
END { print l; printf "%-14s %s\t%s\n", "--------------","--------","--------"; printf "%-15s%8s\t%8s\n", "",e,c }'
TYPE           ENV      COUNT
-------------- -------- --------
fap                1254   475732
psf                 114    67601
unit                167     7415
jde                  77     4383
NMDemo               35     3754
sbl                  34     2890
ofm                   1      592
NMTalent              6      592
/NMDemo               1      160
fapbs                 1      160
bics                  1       49
app                   1       44
PRI                  40       40
la                    2        3
unclassified        N/A     1660
-------------- -------- --------
                   1734   565075
$

Hope this helps Smilie
These 2 Users Gave Thanks to junior-helper For This Post:
# 16  
Old 12-10-2014
@ junior-helper

Your awk example works perfect here. You helped me find and fix some things in my script that have been bugging me.

Thanks mate.

P.S. Your unit count is one more than mine. I wonder what that is?
unit 167 7415

Last edited by ongoto; 12-10-2014 at 07:19 PM..
This User Gave Thanks to ongoto For This Post:
# 17  
Old 12-11-2014
You're welcome, dude! I'm glad you trusted my numbers Smilie

Quote:
I wonder what that is?
First of all, 166 seems to be the right number for unit env-count Smilie

I researched for a while and found this "problematic" record:
Code:
unit18349_postclone-istoreb2c

When the above record goes through my gsub(/^[^0-9]*|-.*/,"",l) function, the output is 183492, but all associated records will result in 18349.

This is the patch:
Code:
gsub(/^[^0-9]+|-.*/,"",l)

I don't know why, but I just realized this micro-tweak also boosts the processing speed at 50% (Smilie)

PS: You neglected the tidying up (outfile + outfile2) Smilie
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Homework & Coursework Questions

Shell script/awk to sort text

1. The problem statement, all variables and given/known data: I have a file with a fragment of a novel, which I have to clear from punctuation and sort all the words contained one per line and non duplicated, all this going to a file called "palabras". Here is fragment of the input file: ... (4 Replies)
Discussion started by: ektorzoza
4 Replies

2. Shell Programming and Scripting

Shell Script to Dynamically Extract file content based on Parameters from a pdf file

Hi Guru's, I am new to shell scripting. I have a unique requirement: The system generates a single pdf(/tmp/ABC.pdf) file with Invoices for Multiple Customers, the format is something like this: Page1 >> Customer 1 >>Invoice1 + invoice 2 >> Page1 end Page2 >> Customer 2 >>Invoice 3 + Invoice 4... (3 Replies)
Discussion started by: DIps
3 Replies

3. Shell Programming and Scripting

Shell script to monitor new file in a directory and mail the file content

Hi I am looking for a help in designing a bash script on linux which can do below:- 1) Look in a specific directory for any new files 2) Mail the content of the new file Appreciate any help Regards Neha (5 Replies)
Discussion started by: neha0785
5 Replies

4. Shell Programming and Scripting

Script to create a text file whose content is the text of another files

Hello everyone, I work under Ubuntu 11.10 (c-shell) I need a script to create a new text file whose content is the text of another text files that are in the directory $DIRMAIL at this moment. I will show you an example: - On the one hand, there is a directory $DIRMAIL where there are... (1 Reply)
Discussion started by: tenteyu
1 Replies

5. Shell Programming and Scripting

Need to build Shell Script to search content of a text file into a folder consist several files

Have to read one file say sourcefile containing several words and having another folder containing several files. Now read the first word of Sourcefile & search it into the folder consisting sevral files, and create another file with result. We hhave to pick the filename of the file in which... (3 Replies)
Discussion started by: mukesh.baranwal
3 Replies

6. Shell Programming and Scripting

Shell script to remove some content in a file

How can I remove all data that contain domain e.g zzgh@something.com, sdd@something.com.my and gg@something.my in one file? so that i only have data without the domain in the file. Here is the file structure "test.out" more test.out 1 zzztop@b.com 1 zzzulll 1 zzzullll@s.com.my ... (4 Replies)
Discussion started by: Mr_47
4 Replies

7. UNIX for Dummies Questions & Answers

creating text file with content from script

hi, can somebody tell me how I can create a text file with content from Bash script. The file should be prefilled with information such as current date and time then leaving the user ability to input more data right below those prefilled content. thank you :) (0 Replies)
Discussion started by: s3270226
0 Replies

8. Shell Programming and Scripting

Problem getting the content of a file in a shell script variable

Hi, I have a text file that has a long multi-line db2 CTE query. Now I want to store all the contents of this file (i.e. the entire query) in a shell script variable. I am trying to achieve it by this: query = `cat /Folder/SomeFile.txt` But when I echo the contents of this file by saying echo... (4 Replies)
Discussion started by: DushyantG
4 Replies

9. Shell Programming and Scripting

Sort content of text file based on date?

I now have a 230,000+ lines long text file formatted in segments like this: Is there a way to sort this file to have everything in chronological order, based on the date and time in the text? In this example, I would like the result to be: (19 Replies)
Discussion started by: KidCactus
19 Replies

10. Shell Programming and Scripting

shell script to edit the content of a file

Hi I need some help using shell script to edit a file. My original file has the following format: /txt/email/myemail.txt /txt/email/myemail2.txt /pdf/email/myemail.pdf /pdf/email/myemail2.pdf /doc/email/myemail.doc /doc/email/myemail2.doc I need to read each line. If the path is... (3 Replies)
Discussion started by: tiger99
3 Replies
Login or Register to Ask a Question