Sort and extract based on two files Post: 302503128

Sponsored Content

Top Forums Shell Programming and Scripting Sort and extract based on two files Post 302503128 by drl on Wednesday 9th of March 2011 05:32:40 PM

03-09-2011

Registered User

Hi.

Here is a script that uses a non-standard sort utility that admits alternate collating sequences, msort:

Code:

#!/usr/bin/env bash

# @(#) s1	Demonstrate alternate collating sequence.
# msort-home http://freshmeat.net/projects/msort

# Section 1, setup, pre-solution.
# Infrastructure details, environment, commands for forum posts. 
# Uncomment export command to test script as external user.
# export PATH="/usr/local/bin:/usr/bin:/bin"
set +o nounset
pe() { for i;do printf "%s" "$i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
C=$HOME/bin/context && [ -f $C ] && . $C specimen msort
set -o nounset
pe

FILE=${1-data1}
shift
CS=${1-data2}

# Section 2, display input file and collating sequence file.
# Display sample of data file, with head & tail as a last resort.
pe " || start [ first:middle:last ]"
specimen $FILE $CS \
|| { pe "(head/tail)"; head -n 5 $FILE; pe " ||"; tail -n 5 $FILE; }
pe " || end"

# Section 3, solution.
pl " Results:"
msort -q -n 1,1 -u n -l -c lexicographic -s $CS -1 $FILE

exit 0

producing:

Code:

% ./s1

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 2.6.26-2-amd64, x86_64
Distribution        : Debian GNU/Linux 5.0.7 (lenny) 
GNU bash 3.2.39
specimen (local) 1.17
msort - ( /usr/bin/msort Apr 24 2008 )

 || start [ first:middle:last ]
Whole: 5:0:5 of 4 lines in file "data1"
aba  2  4  6  7
for   2  4  7  4
hmm  1  2  7  4
gh  2  5  7  9

Whole: 5:0:5 of 4 lines in file "data2"
gh
aba
for
hmmm
 || end

-----
 Results:
gh  2  5  7
aba  2  4  6  7
for   2  4  7
hmm  1  2  7  4

If you are using Debian GNU/Linux, msort is in the repository for lenny and squeeze, but not in wheezy yet. The freshmeat site has links to a number of packages for other OSs.

Good luck ... cheers, drl

This User Gave Thanks to drl For This Post:

drl

View Public Profile for drl

Find all posts by drl

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to Sort files based on predefined values.?

How to Sort files based on predefined values.? Normally Sorting happens for the alphabetic or numberic orders.. Is there any way to sort a fields based on the Field values..? Field10 has : one two three five four six ten seven eight nine. in predefined order { one, two, three,...

2. Shell Programming and Scripting

AWK or KSH : Sort, Group and extract from 3 files

Hi, I've the following two CSV files: File1.csv File2.csv Class,Student# Student#,Marks 1001,6001 6002,50 1001,6002 6001,60 1002,7000 ...

3. UNIX for Dummies Questions & Answers

sort lines in different files based on the starting letter

Hi ,, i have the below file... D 2342135 B 214236 C argjlksd V lskjrghaklsr C slkrgj B sdg4tsd E aslkgjlkasg i want to sort the lines into different files based on the starting letter of the line. so that i have different files for lines starting with a letter. thanks

4. Shell Programming and Scripting

Sort and extract based on two files

Hi, I am having trouble sorting one file based on another file. I tried the grep -f function and failed. Basically what I have is two files that look like this: File 1 (the list) gh aba for hmm File 2 ( the file that needs to be sorted) aba 2 4 6 7 for 2 4 7 4...

5. Shell Programming and Scripting

sort the files based on timestamp and execute sorted files in order

Hi I have a requirement like below I need to sort the files based on the timestamp in the file name and run them in sorted order and then archive all the files which are one day old to temp directory My files looks like this PGABOLTXML1D_201108121235.xml...

6. Shell Programming and Scripting

How to sort files based on file name having numbers

Right now there is no unix direct commad that can sort the files base on its name having numbers: We can use the following: In case your file name are like: abc-UP018.zip xyz-UP019.zip ls *|sort -t'-' -k2

7. Shell Programming and Scripting

Sort log files based on numeric value in the filename

Hi, I have a list of log files as follows: name_date_0001_ID0.log name_date_0001_ID2.log name_date_0001_ID1.log name_date_0002_ID2.log name_date_0004_ID0.log name_date_0005_ID0.log name_date_0021_ID0.log name_date_0025_ID0.log ..........................................

8. UNIX for Dummies Questions & Answers

Sort Files based on the number(s) on the file name

Experts I have a list of files in the directory mysample1 mysample2 mysample3 mysample4 mysample5 mysample6 mysample7 mysample8 mysample9 mysample10 mysample11 mysample12 mysample13 mysample14 mysample15

9. Shell Programming and Scripting

Extract date from files based on file pattern

I want to extract dates from the files and i have different types of files with pattern. I have list file with the patterns and want to date extract based on it in a sh script Files in the directory : file1_20160101.txt file2_20160101_abc.txt filexyz20160101.txt list file with...

10. UNIX for Beginners Questions & Answers

How to sort the files by size and based subdirectory un UNIX?

I have the below input data in a file and need to get the output as mentioned below. Need to sort the data by size(Asc/des)/by subdirectory Below is the input which is there in a file: 120 /root/path2/part-00000-d3700305-428d-4b13-8161-42051f4ac5ed-c000.json 532 ...

LEARN ABOUT X11R4

gzexe

GZEXE(1)                                                      General Commands Manual                                                     GZEXE(1)

NAME

       gzexe - compress executable files in place

SYNOPSIS

       gzexe name ...

DESCRIPTION

       The  gzexe  utility  allows you to compress executables in place and have them automatically uncompress and execute when you run them (at a
       penalty in performance).  For example if you execute ``gzexe /usr/bin/gdb'' it will create the following two files:
           -rwxr-xr-x  1 root root 1026675 Jun  7 13:53 /usr/bin/gdb
           -rwxr-xr-x  1 root root 2304524 May 30 13:02 /usr/bin/gdb~
       /usr/bin/gdb~ is the original file and /usr/bin/gdb is the self-uncompressing executable file.  You can remove /usr/bin/gdb~ once  you  are
       sure that /usr/bin/gdb works properly.

       This utility is most useful on systems with very small disks.

OPTIONS

       -d     Decompress the given executables instead of compressing them.

SEE ALSO

       gzip(1), znew(1), zmore(1), zcmp(1), zforce(1)

CAVEATS

       The  compressed executable is a shell script.  This may create some security holes.  In particular, the compressed executable relies on the
       PATH environment variable to find gzip and some standard utilities (basename, chmod, ln, mkdir, mktemp, rm, sleep, and tail).

BUGS

       gzexe attempts to retain the original file attributes on the compressed executable, but you may have to fix them manually  in  some  cases,
       using chmod or chown.

                                                                                                                                          GZEXE(1)

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to Sort files based on predefined values.?

Discussion started by: p_prathaban

2. Shell Programming and Scripting

AWK or KSH : Sort, Group and extract from 3 files

Discussion started by: Matrix2682

3. UNIX for Dummies Questions & Answers

sort lines in different files based on the starting letter

Discussion started by: jathin12

4. Shell Programming and Scripting

Sort and extract based on two files

Discussion started by: phil_heath