Sponsored Content
Top Forums Shell Programming and Scripting awk updating one file with another, comparing, updating Post 302203885 by mecano on Tuesday 10th of June 2008 06:00:37 AM
Old 06-10-2008
Here it is ^^

Code:
#!/bin/bash

NO_ARGS=0
E_OPTERROR=65

if [ $# -eq "$NO_ARGS" ]
	then
	echo -e "\n\tUsage: `basename $0` -ulkdrm filename\n\tType :'awksort -help' for help.\n"
	exit $E_OPTERROR
fi

while getopts ":u:l:k:d:r:m:h" Option
	do
	case $Option in
	u )
	filename=$2
	if [ -f $filename ]
		then
		sort -u $filename > $filename.uniq
	else
		echo -e "\ncan't find file $filename\n"
	fi
	;;
	l )
	filename=$2
	if [ -f $filename ]
		then
		awk '{ printf substr($NF, 1, length($NF)-1);$NF = "";printf " %s\n",$0 }' $filename | sort -n | awk '{ printf "%s%s;\n",$0,$1 }' | awk '{$1="";sub(/^ +/, "");printf "%s\n",$0}' > $filename.sorted
	else
		echo -e "\ncan't find file $filename\n"
	fi
	;;
	k )
	filename=$3
	if [ -f $filename ]
		then
		opt=$OPTARG
		sort -n -t "=" -k $opt $filename > $filename.sorted
	else
		echo -e "\ncan't find file $filename\n"
	fi
	;;
	d )
	filename=$2
	if [ -f $filename ]
		then
		awk '{ printf substr($NF, 1, length($NF)-1);$NF = "";printf "\n" }' $filename | sort -n | awk '{ if ($1 == prev) { printf "%d\n",$0;num++ };prev=$1 } END { printf "\n%d duplicates were found...\n",num }'
	else
		echo -e "\ncan't find file $filename\n"
	fi
	;;
	r )
	filename=$2
	if [ -f $filename ]
		then
		awk '{ printf substr($NF, 1, length($NF)-1);$NF = "";printf " %s\n",$0 }' $filename | sort -n | awk '{ if ($1 != prev) { printf "%s%s;\n",$0,$1 };prev=$1 }' | awk '{$1="";sub(/^ +/, "");printf "%s\n",$0}' > $filename.noduplicate
	else
		echo -e "\ncan't find file $filename\n"
	fi
	;;
	m )
	key=$2
	file1=$3
    file2=$4
	if [ -z $file1 ]
		then
		echo -e "\nMissing argument. Usage: `basename $0` -m file1 file2\n"
	elif [ -z $file2 ]
		then
		echo -e "\nMissing argument. Usage: `basename $0` -m file1 file2\n"
	elif [ -f $file1 -a -f $file2 ]
		then
		if [ $key -eq 0 ]
			then
			awk 'END{for(k in _)print _[k]}{_[$NF]=$0}' $file1 $file2 > $file1.updated
			else
			awk 'END{for(k in _)print _[k]}{_[$'"$key"']=$0}' $file1 $file2 > $file1.updated
		fi
	elif [ -f $file1 ]
		then
		echo -e "\nCan't find file $file2\n"
	elif [ -f $file2 ]
		then
		echo -e "\nCan't find file $file1\n"
	else
		echo -e "\nCan't find any file! Neither $file1 or $file2 were found!\n"
	fi
	;;
	h )
	echo ""
	echo -e "\tawksort 0.1\n"
	echo -e "\tUsage: `basename $0` -ulkdrm options file\n\n"
	echo -e "\tOptions :\n"
	echo -e "\t-u file\n\tremove 'identical' entries, leaving only unique entries.\n"
	echo -e "\t-l file\n\tsort with last field of line.\n"
	echo -e "\t-k key file\n\twhere key is the field number to sort.\n"
	echo -e "\t-d file\n\treport duplicate 'similar' entries by id when id is the last field.\n"
	echo -e "\t-r file\n\tremove duplicate 'similar' entries arbitrary by id when id is the last field.\n"
	echo -e "\t-m key file1 file2\n\tmerge file1 with file2 where file2 is an 'update' file.\n\tThis overrides duplicates ids from file1 by replacing them\n\twith file2 records.\n\tUse 0 for key to merge using last field of line as key.\n"
	echo -e "\n\tClassic scenario is:\n\tUse -u to remove identical entries, then sort entries using -l or -k,\n\tremove similar entries with -r and finally apply update.\n"
	;;
	\? )
	echo -e "\n\tUsage: `basename $0` -ulkdrm filename\n\tType :'awksort -help' for help.\n"
	exit 1;;
	* )
	echo -e "\n\tUsage: `basename $0` -ulkdrm filename\n\tType :'awksort -help' for help.\n"
	exit 1;;
	esac
done

shift $(($OPTIND - 1))

exit 0

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Constantly updating log files (tail -f? grep? awk?)

I have a log file which is continuously added to, called log.file. I'd like to monitor this file, and when certain lines are found, update some totals in another file. I've played around with tail -f, grep, and awk, but can't seem to hit the right note, so to speak. The lines I'm... (0 Replies)
Discussion started by: nortonloaf
0 Replies

2. UNIX for Dummies Questions & Answers

Korn shell awk use for updating two files

Hi, I have two text files containing records in following format: file1 format is: name1 age1 nickname1 path1 name2 age2 nickname2 path2 file 1 example is: abcd 13 abcd.13 /home/temp/abcd.13 efgh 15 efgh.15 /home/temp/new/efgh.15 (4 Replies)
Discussion started by: alrinno
4 Replies

3. UNIX for Dummies Questions & Answers

Updating specific fields with awk using conditions

Can someone help me again, I think with awk? I have a file with 4 columns (pipe-delimited): I just want to convert the last field so that e1 is now 'message 1', e2 is 'message 2', e0 is 'message 3', etc. I don't want to change any other columns because the e0-e10 code may appear as part of a... (4 Replies)
Discussion started by: giannicello
4 Replies

4. Shell Programming and Scripting

Updating a line in a large csv file, with sed/awk?

I have an extremely large csv file that I need to search the second field, and upon matches update the last field... I can pull the line with awk.. but apparently you cant use awk to directly update the file? So im curious if I can use sed to do this... The good news is the field I want to... (5 Replies)
Discussion started by: trey85stang
5 Replies

5. Shell Programming and Scripting

AWK and sub/gsub: updating a date/time block

I have a file ("modtest") in which I want to update the last date/time block in the lines beginning with a period. Here is a sample: .ROMULT 10150908EDT 10270908EDT 10010908EDT RANGE RAWV2 1.00 .ROMULT 10150908EDT 10270908EDT 10010908EDT FGROUP CHOWANRV 1.00 .RRIMULT 10150908EDT... (10 Replies)
Discussion started by: chrismcg24
10 Replies

6. Shell Programming and Scripting

Comparing 2 files with awk and updating 2nd file

file1: (unique files) 1 /pub/atomicbk/catalog/catalog.gif 693 2 /pub/atomicbk/catalog/home.gif 813 3 /pub/atomicbk/catalog/logo2.gif 12871 4 /pub/atomicbk/catalog/sleazbk.html 18338 file2: (duplicate filenames allowed) 28/Aug/1995:00:00:38 1 /pub/atomicbk/catalog/home.gif 813... (2 Replies)
Discussion started by: jontjioe
2 Replies

7. Shell Programming and Scripting

awk - updating variable in if statement

I have another question I am stuck at :wall: I have a text file with two columns, like so... 2 0.0627279 3 0.0794451 4 0.108705 5 0.137739 6 0.190394 7 0.217407 8 0.241764 9 0.344458 10 0.460762 I'd like to go through the file line by line until the value in the second column... (3 Replies)
Discussion started by: origamisven
3 Replies

8. Shell Programming and Scripting

Help updating a file

I can not seem to figure out how to update the attached match.txt column 2 using update.txt. However, only the text before the period in updat.txt appears in match.txt. For example, in update.txt NM_001613.2 matches NM_001613 in match.txt, so is it possible to update the record in match.txt to... (8 Replies)
Discussion started by: cmccabe
8 Replies

9. Shell Programming and Scripting

Updating variables using sed or awk

Hi, I have a file(testfile.txt) that contains list of variables as shown below. T $$FirstName=James $$LastName=Fox $$Dateofbirth=1980-02-04 ……and so on there are 50 different variables. I am writing a script(script1.sh) that will update the above three variable one by one with the values... (6 Replies)
Discussion started by: Saanvi1
6 Replies

10. UNIX for Beginners Questions & Answers

Updating in file

hi, i have an csv(which is a month's log file containing userid and log in date ) file which has to be appended to another file say master.log.I need to compare the next month's log data to master.log file .In case, there is new log date for userid it has to get updated in master.log file or i... (2 Replies)
Discussion started by: preema
2 Replies
h5diff(1)						      General Commands Manual							 h5diff(1)

NAME
h5diff - Compares two HDF5 files and reports the differences. SYNOPSIS
h5diff file1 file2 [OPTIONS] [object1 [object2 ] ] DESCRIPTION
h5diff is a command line tool that compares two HDF5 files, file1 and file2, and reports the differences between them. Optionally, h5diff will compare two objects within these files. If only one object, object1, is specified, h5diff will compare object1 in file1 with object1 in file2. In two objects, object1 and object2, are specified, h5diff will compare object1 in file1 with object2 in file2. These objects must be HDF5 datasets. object1 and object2 must be expressed as absolute paths from the respective file's root group. Additional information, with several sample cases, can be found in the document H5diff Examples. OPTIONS
file1 file2 The HDF5 files to be compared. -h Print all differences. -r Print only the names of objects that differ; do not print the differences. These objects may be HDF5 datasets, groups, or named datatypes. -n count Print difference up to count differences, then stop. count must be a positive integer. -d delta Print only differences that are greater than the limit delta. delta must be a positive number. The comparison criterion is whether the absolute value of the difference of two corresponding values is greater than delta (e.g., |a-b| > delta, where a is a value in file1 and b is a value in file2). -p relative Print only differences that are greater than a relative error. relative must be a positive number. The comparison criterion is whether the absolute value of the difference 1 and the ratio of two corresponding values is greater than relative (e.g., |1-(b/a)| > relative where a is a value in file1 and b is a value in file2). object1 object2 Specific object(s) within the files to be compared. EXAMPLES
The following h5diff call compares the object /a/b in file1 with the object /a/c in file2: h5diff file1 file2 /a/b /a/c This h5diff call compares the object /a/b in file1 with the same object in file2: h5diff file1 file2 /a/b And this h5diff call compares all objects in both files: h5diff file1 file2 SEE ALSO
h5dump(1), h5ls(1), h5repart(1), h5import(1), gif2h5(1), h52gif(1), h5perf(1) h5diff(1)
All times are GMT -4. The time now is 07:29 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy