Sponsored Content
Full Discussion: Bash/awk script problem
Top Forums Shell Programming and Scripting Bash/awk script problem Post 302939776 by Don Cragun on Saturday 28th of March 2015 09:14:09 PM
Old 03-28-2015
In addition to what sea and mjf have already noted, even if you do append the output of paste (rather than overwriting the output from paste) you still won't get what you want. The paste utility doesn't paste the input file onto the ends of lines it reads from the output file; it pastes lines from one or more input files to create lines in the output file.

You could run paste once for each each input file in turn to paste it at the ned of the appropriate output file into a temp file and move the temp file back to the appropriate output file. But running awk 100 times and running paste 100 times seems highly inefficient to me. I'd suggest something more like:
Code:
#!/bin/bash
rm -f  file2.dat file3.dat
awk '
BEGIN {	outf[1] = "file2"
	outf[0] = "file3"
}
function choose(infile) {
	if(ofc[of = (s / c) > .28]++ == 0)
		printf("paste \"%s\"", infile) > (outf[of] ".sh")
	else	printf(" \"%s\"", infile) >> (outf[of] ".sh")
	c = s = 0
}
FNR == 1{
	if(NR > 1)
		choose(fn)
	fn = FILENAME
}
{	c++
	s += $1
}
END {	choose(fn)
	for(i = 0; i <= 1; i++)
		if(ofc[i]) {
			printf(" > %s.dat\n", outf[i]) >> (outf[i] ".sh")
			printf("chmod +x %s.sh;./%s.sh;echo rm %s.sh\n",
				outf[i], outf[i], outf[i])
		}
}' file1_[1-9].dat file1_[1-9][0-9].dat file1_100.dat | bash

which runs awk once, paste once or twice, and chmod once or twice, and bash one extra time to run the commands produced by this awk script.

This code creates one or two temporary shell scripts to paste the appropriate input files together into your final output files, runs those scripts, and uses echo to remind you to remove those temporary shell scripts. If it does what you want; remove the echo in red in the awk script to actually remove the temporary scripts after that have been run.

If you want to try this on a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

bash script problem

hi I am writing a bash script that uses dialog to get user input an diplay messages to user. I have a small problem dialog --inputbox "blabla" 20 50 2> /tmp/output VAR="'cat /tmp/output'" mkdir $VAR the code below requests user for a directory path to be created. But, if the user uses... (1 Reply)
Discussion started by: fnoyan
1 Replies

2. UNIX for Advanced & Expert Users

AWK sub function curious problem under bash

I need to detect the number of pages in a print job when it is available so I can warn users when they try to print a report much larger than they expected. Sometimes they are trying to print 1000 page reports when they thought they were getting a 10 page report. Under linux I am scanning the... (5 Replies)
Discussion started by: Max Rebo
5 Replies

3. Shell Programming and Scripting

Bash or awk script to solve this problem

Hi everybody! I have written some awk scripts that return me some results I need to process. At the moment I use openOffice to process them, but I am trying to find a more efficient solution using possibly a bash or awk script. I have two files, file1 is in the format: time position ... (3 Replies)
Discussion started by: Alice236
3 Replies

4. Shell Programming and Scripting

Problem in bash script

I have written a script and I get error and I don't understand why. neededParameters=2 numOfParameters=0 correctNum=0 while getopts "s:l:" opt do case "$opt" in s) serviceName= $OPTARG #errorline 1 numOfParameters= $numOfParameters + 1 ;; l) ... (12 Replies)
Discussion started by: programAngel
12 Replies

5. Shell Programming and Scripting

help with a bash script problem

hi to everyone :) i am new to linux and bash and i am trying to build a bash script, that is quite similar to the well known cmd 'split' ... ;) it is now already "working" ... i can use it like: ./splitfix.sh -v -s 10 foo ./splitfix.sh -s 10 -v foo ./splitfix.sh -s 10 foo ./splitfix.sh -v... (5 Replies)
Discussion started by: drjodo
5 Replies

6. Shell Programming and Scripting

Help: How to convert this bash+awk script in awk script only?

This is the final first release of the dynamic menu generator for pekwm (WM). #!/bin/bash function param_val { awk "/^${1}=/{gsub(/^${1}="'/,""); print; exit}' $2 } echo "Dynamic {" for CF in `ls -c1 /usr/share/applications/*.desktop` do name=$(param_val Name $CF) ... (3 Replies)
Discussion started by: alexscript
3 Replies

7. Shell Programming and Scripting

Problem with Bash Script.

Hi guys! I'm new to the forum and to the Bash coding scene. I have the following code paths=/test/a paths=/test/b keywords=\"*car*\" keywords=\"*food*\" for file in `find paths -type f -ctime -1 -name keywords -print 2>/dev/null` do #.... do stuff here for every $file found... (5 Replies)
Discussion started by: RedSpyder
5 Replies

8. Shell Programming and Scripting

Help with bash script problem

Hi, Below is my bash script: cat run_all.sh if && ; then Number_Count_Program $1.results $2.results > $1.$2.counts else Number_Split_Program $1.results $2.results > $1.$2.split fi After I run the following command: ./run_all.sh A B ./run_all.sh: line 1: Anybody advice to edit... (5 Replies)
Discussion started by: perl_beginner
5 Replies

9. UNIX for Dummies Questions & Answers

New problem with awk using bash

Hi! I have a new problem with awk, this time I think is because I'm using it in bash and I don't know how to put the valor of the variable in awk. Here is the code: #!/bin/bash for i in 1 2 3 4 5 do a=$i b=$ awk '$1>=a&&$1<=b {print $1,$2,$3}'>asdf test... (3 Replies)
Discussion started by: florpi
3 Replies

10. Shell Programming and Scripting

Problem with awk instructions and bash variable

Hi everyone, I'm trying to write a small script to automatize row data treatment. However, I got some trouble with the awk command. I want to use awk to extract a define paragraph from a text file. The first and final lines are defined externally in two variables called debut and fin. I... (2 Replies)
Discussion started by: TeaTimeSF
2 Replies
paste(1)						      General Commands Manual							  paste(1)

Name
       paste - merge file data

Syntax
       paste file1 file2...
       paste -dlist file1 file2...
       paste -s [-dlist] file1 file2...

Description
       In  the	first  two forms, concatenates corresponding lines of the given input files file1, file2, etc.	It treats each file as a column or
       columns of a table and pastes them together horizontally (parallel merging).

       In the last form, the command combines subsequent lines of the input file (serial merging).

       In all cases, lines are glued together with the tab character, or with characters from an optionally specified  list.   Output  is  to  the
       standard output, so it can be used as the start of a pipe, or as a filter, if - is used in place of a file name.

Options
       -       Used in place of any file name, to read a line from the standard input.	(There is no prompting).

       -dlist  Replaces  characters  of  all but last file with nontabs characters (default tab).  One or more characters immediately following -d
	       replace the default tab as the line concatenation character.  The list is used circularly, i. e. when exhausted, it is reused.	In
	       parallel  merging  (i. e. no -s option), the lines from the last file are always terminated with a new-line character, not from the
	       list.  The list may contain the special escape sequences: 
 (new-line), 	 (tab), \ (backslash), and  (empty string, not a null
	       character).   Quoting  may  be  necessary,  if characters have special meaning to the shell (for example, to get one backslash, use
	       -d"\\" ).
	       Without this option, the new-line characters of each but the last file (or last line in case of the -s option) are  replaced  by  a
	       tab character.  This option allows replacing the tab character by one or more alternate characters (see below).

       -s      Merges  subsequent  lines  rather  than	one  from  each input file.  Use tab for concatenation, unless a list is specified with -d
	       option.	Regardless of the list, the very last character of the file is forced to be a new-line.

Examples
       ls | paste -d" " -
       list directory in one column
       ls | paste - - - -
       list directory in four columns
       paste -s -d"	
" file
       combine pairs of lines into lines

Diagnostics
       line too long
		 Output lines are restricted to 511 characters.

       too many files
		 Except for -s option, no more than 12 input files may be specified.

See Also
       cut(1), grep(1), pr(1)

																	  paste(1)
All times are GMT -4. The time now is 10:26 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy