05-04-2005
Using awk
Hello all,
I'm relatively new to the UNIX world (I've been lost in the Microsofa world!) and I haven't used awk yet, but I think it would be useful in grouping and aggregating data. I'm not sure how to proceed, and would welcome any suggestions.
I have three fields in a delimited file. The file is sorted by a business id, then by the an item id, and then the third field is a quantity. What I would like to do is aggregate the quantity by the business id. I was told that awk would be a very useful tool to do this, but I'm not really sure how to proceed.
My script extracts data from various flat files (all fixed-string) and formats & sorts the files, but this is the last step and I'm not sure how to proceed.
Thanks in advance for any assistance you can provide.
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Actually I got a list of file end with *.txt
I want to use the same command apply to all the *.txt
Thus I try to find out the fastest way to write those same command in a script and then want to let them run automatics.
For example:
I got the file below:
file1.txt
file2.txt
file3.txt... (4 Replies)
Discussion started by: patrick87
4 Replies
2. Shell Programming and Scripting
Hi Experts,
I am adding a column of numbers with awk , however not getting correct output:
# awk '{sum+=$1} END {print sum}' datafile
2.15291e+06
How can I getthe output like : 2152910
Thank you..
# awk '{sum+=$1} END {print sum}' datafile
2.15079e+06 (3 Replies)
Discussion started by: rveri
3 Replies
3. Shell Programming and Scripting
I want to filter 2nd column = 2 using awk
$ cat t
1 2
2 4
$ VAR=2
#variable worked in print
$ cat t | awk -v ID=$VAR ' { print ID}'
2
2
# but variable didn't work in awk filter
$ cat t | awk -v ID=$VAR '$2~/ID/ { print $0}' (2 Replies)
Discussion started by: honglus
2 Replies
4. Shell Programming and Scripting
Hi
I have many problems with a script. I have a script that formats a text file but always prints the same error when i try to execute it
The code is that:
{
if (NF==17){
print $0
}else{
fields=NF;
all=$0;
while... (2 Replies)
Discussion started by: fate
2 Replies
5. Shell Programming and Scripting
I have two files which I would like to compare and then manipulate in a way.
File1:
pictures.txt 1.1 1.3
dance.txt 1.2 1.4
treehouse.txt 1.3 1.5
File2:
pictures.txt 1.5 ref2313 1.4 ref2345 1.3 ref5432 1.2 ref4244
dance.txt 1.6 ref2342 1.5 ref2352 1.4 ref0695 1.3 ref5738 1.2... (1 Reply)
Discussion started by: linuxkid
1 Replies
6. Shell Programming and Scripting
Hi,
I have a situation to compare one file, say file1.txt with a set of files in directory.The directory contains more than 100 files.
To be more precise, the requirement is to compare the first field of file1.txt with the first field in all the files in the directory.The files in the... (10 Replies)
Discussion started by: anandek
10 Replies
7. Shell Programming and Scripting
Hello experts,
I'm stuck with this script for three days now. Here's what i need.
I need to split a large delimited (,) file into 2 files based on the value present in the last field.
Samp: Something.csv
bca,adc,asdf,123,12C
bca,adc,asdf,123,13C
def,adc,asdf,123,12A
I need this split... (6 Replies)
Discussion started by: shell_boy23
6 Replies
8. Shell Programming and Scripting
consider the script below
sh /opt/hqe/hqapi1-client-5.0.0/bin/hqapi.sh alert list --host=localhost --port=7443 --user=hqadmin --password=hqadmin --secure=true >/tmp/alerts.xml
awk -F'' '{for(i=1;i<=NF;i++){
if($i=="Alert id") {
if(id!="")
if(dt!=""){
cmd="sh someScript.sh... (2 Replies)
Discussion started by: vivek d r
2 Replies
9. Shell Programming and Scripting
Hi,
I am trying to pass awk field to a command line executed within awk (need to convert a timestamp into formatted date).
All my attempts failed this far.
Here's an example.
It works fine with timestamp hard-codded into the command
echo "1381653229 something" |awk 'BEGIN{cmd="date -d... (4 Replies)
Discussion started by: tuxer
4 Replies
10. Shell Programming and Scripting
Good evening, Im newbie at unix specially with awk
From an scheduler program called Autosys i want to extract some data reading an inputfile that comprises jobs names, then formating the output to columns for example
1.
This is the inputfile:
$ more MapaRep.txt
ds_extra_nikira_usuarios... (18 Replies)
Discussion started by: alexcol
18 Replies
JOIN(1) General Commands Manual JOIN(1)
NAME
join - relational database operator
SYNOPSIS
join [ options ] file1 file2
DESCRIPTION
Join forms, on the standard output, a join of the two relations specified by the lines of file1 and file2. If file1 is `-', the standard
input is used.
File1 and file2 must be sorted in increasing ASCII collating sequence on the fields on which they are to be joined, normally the first in
each line.
There is one line in the output for each pair of lines in file1 and file2 that have identical join fields. The output line normally con-
sists of the common field, then the rest of the line from file1, then the rest of the line from file2.
Fields are normally separated by blank, tab or newline. In this case, multiple separators count as one, and leading separators are dis-
carded.
These options are recognized:
-an In addition to the normal output, produce a line for each unpairable line in file n, where n is 1 or 2.
-e s Replace empty output fields by string s.
-jn m Join on the mth field of file n. If n is missing, use the mth field in each file.
-o list
Each output line comprises the fields specifed in list, each element of which has the form n.m, where n is a file number and m is a
field number.
-tc Use character c as a separator (tab character). Every appearance of c in a line is significant.
SEE ALSO
sort(1), comm(1), awk(1)
BUGS
With default field separation, the collating sequence is that of sort -b; with -t, the sequence is that of a plain sort.
The conventions of join, sort, comm, uniq, look and awk(1) are wildly incongruous.
JOIN(1)