help newb at linux and bash need numeric script sort


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting help newb at linux and bash need numeric script sort
# 1  
Old 04-27-2008
help newb at linux and bash need numeric script sort

I am trying to setup to automatically import a series of mysql database files. I am doing manually now and its a royal pain.

All the sql files are sequentially numbered in a format of 4 numbers underscore text with spaces replaced by underscores.

example:
Quote:
logon_updates/2334_third_file.sql
logon_updates/1234_first_file.sql
logon_updates/1345_second_file.sql
There are 3 databases each setup in the same fashion. The main directory has the database structure file higher sequence replacing lower sequence. Each database has its own subdirectory with sequenced updates the files with numbers higher than the structured files need to be loaded in numerical sequenced. There are duplicate sequence numbers in the update directories those all have to be loaded before the next number.

I figured I can do this to gain the proper structure file as they do not have duplicate numbers.

Quote:
list=$(find -name "*logon_structure.sql" -print)
for f in $list;
do
g=$(echo $f | cut -b3-6)
if [ $g -ge $lnum ]; then
lnum=$g
fi
done
I tried the below code getting rid of the directory name with cut but they are in the wrong order, they are in alphabetic not numeric order.
The variable h has the number alone and g has the full file name .
Quote:
list=$(find logon_updates -name "*.sql" -print)
for f in $list;
do
g=$(echo $f | cut -b15-255)
h=$(echo $f | cut -b15-18)
if [ $h -gt $lnum ]; then
echo $g
fi

done
I lack a lot of basic knowledge under linux that I had in a dos/windows environment.

I have been looking at this code so long I can't see the forest for the trees, if you know what I mean. So any suggestions to get me on the right train of thought would be greatly appreciated

Thanks,
dlm1065

Last edited by dlm1065; 04-27-2008 at 04:36 AM..
# 2  
Old 04-27-2008
If you only use the output from find once, you don't really need to put it in a variable.

My suggestion would be to extract the sequence number and the database to separate fields, and sort numerically on the sequence number.

Code:
find logon_updates -name "*.sql" -print |
while read f
do
  g=$(echo $f | cut -b15-)
  h=$(echo $f | cut -b15-18)
  echo $h:$g
done | 
sort -t : -k1n

Once you are confident that this works correctly (I don't have the data to test on), you can continue the pipeline:

Code:
... sort -t : -k1n |
cut -d: -f2- |
while read f; do
  sql PERFORM ACTS OF horror WITH "$f" USING BIG STICK
done

If the output from find is fairly regular, probably you could find some clever options which would allow you to pass that directly to sort without the while look. Maybe pass the output from find through sed to temporarily regularize it?

Maybe something like this would work already?

Code:
find -name "*logon_structure.sql" -print |
sort -t / -k2n


Last edited by era; 04-27-2008 at 05:17 AM.. Reason: Oops, sort -t (not -d!)
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Sort by second column numeric values

From googling and reading man pages I figured out this sorts the first column by numeric values. sort -g -k 1,1 Why does the -n option not work? The man pages were a bit confusing. And what if I want to sort the second column numerically? I haven't been able to figure that out. The file... (7 Replies)
Discussion started by: cokedude
7 Replies

2. Shell Programming and Scripting

Help with sort word and general numeric sort at the same time

Input file: 100%ABC2 3.44E-12 USA A2M%H02579 0E0 UK 100%ABC2 5.34E-8 UK 100%ABC2 3.25E-12 USA A2M%H02579 5E-45 UK Output file: 100%ABC2 3.44E-12 USA 100%ABC2 3.25E-12 USA 100%ABC2 5.34E-8 UK A2M%H02579 0E0 UK A2M%H02579 5E-45 UK Code try: sort -k1,1 -g -k2 -r input.txt... (2 Replies)
Discussion started by: perl_beginner
2 Replies

3. UNIX for Dummies Questions & Answers

NEWB Question - BASH COMMAND RESULT for ${0##*/}

This should be extremely simple and someone will probably answer it in .5 seconds. I need to know what: VARIABLE=${0##*/} does? I do not have a shell handy to just try it in. I am reading through some scripts and need to understand this line. Any help is appreciated. Many thanks! -... (3 Replies)
Discussion started by: chrisgoetz
3 Replies

4. Shell Programming and Scripting

Sort numeric order

Hi I am using this cat substitutionFeats.txt | gawk '{$0=gensub(/\t/,"blabla",1);print}' | gawk '{print length, $0}' | sort -n | sort -r and the "sort -n" command doesn't work as expected: it leads to a wrong ordering: 64 Adjustable cuffs 64 Abrasion- 64 Abrasion pas 647 Sanitized 647... (4 Replies)
Discussion started by: louisJ
4 Replies

5. Shell Programming and Scripting

Use AWK to check for numeric values? (BASH script)

Hi, I'm quite new to scripting, but know a few AWK statements. I have the following line in my script: hostname=`echo $file | awk 'BEGIN{FS=OFS="."}{$NF=""; NF--; print}'` I use this in my script to rename files, which are similar to this: name.mvkf.mkvfm.mkfvm.1 To the... (4 Replies)
Discussion started by: TauntaunHerder
4 Replies

6. UNIX for Dummies Questions & Answers

sort files by numeric filename

dear all, i have .dat files named as: 34.dat 2.dat 16.dat 107.dat i would like to sort them by their filenames as: 2.dat 16.dat 34.dat 107.dat i have tried numerous combinations of sort and ls command (in vain) to obtain : 107.dat 16.dat 2.dat 34.dat (1 Reply)
Discussion started by: chen.xiao.po
1 Replies

7. Shell Programming and Scripting

Sort help on non numeric field

Hi, I am unable to sort data on the first field $cat t Jim,212121,Seattle Bill,404404,Seattle Steve,246810,Nevada Scott,212277,LosAngeles Jim,212121,Ohio sort -t"," -k1,2 t Bill,404404,Seattle Jim,212121,Ohio Jim,212121,Seattle Scott,212277,LosAngeles Steve,246810,Nevada (7 Replies)
Discussion started by: Shivdatta
7 Replies

8. Shell Programming and Scripting

Perl script to sort data on second numeric field

Hi, I'm a learner of PERL programming. I've a input file with the below data: SWAT_5, 1703, 2010-09-21 SWAT_6, 2345, 2010-09-21 SWAT_7, 1792, 2010-09-21 SWAT_8, 1662, 2010-09-21 SWAT_9, 1888, 2010-09-21 VXHARP_1, 171, 2010-09-21 I need to sort this data based on the second... (6 Replies)
Discussion started by: ganapati
6 Replies

9. Shell Programming and Scripting

Numeric sort error

Hello all I have data like below where the column with values (PRI, SEC ) is the char field and the rest are Numeric Fields. 200707,9580,58,7,2,1,PRI,1,1,137,205594,0,5,10,-45.51,-45.51 200707,9580,58,7,2,1,SEC,1,1,137,205594,0,5,10,-45.51,45.51... (1 Reply)
Discussion started by: vasuarjula
1 Replies

10. Shell Programming and Scripting

how to numeric sort on field time

i want to sort time field given by who command as a whole i have tried like this who|sort -n +4 -5 (1 Reply)
Discussion started by: rahulspatil_111
1 Replies
Login or Register to Ask a Question