Like I said I'm new to this and I tried to explain it the best I could. I'll try to give the full run down. I have a 46 column data input file that looks like this:
Obviously I didn't include all 46 columns and 200-some rows but you get the idea. What I want to be able to do is write a script that asks for which columns to take from the input and write to the output. The problem is I could want 4 columns or I could want 10 columns to be written out. I have the code first prompt for the input file name so I don't need to hard-code that, and for the user to give whatever output file name you want.
Code:
# Get input file from user
echo "Provide name of master file for use"
read infile
sed '1,3d;$d' $infile > infilemod
# Get output file name from user
echo "Provide desired output file name"
read outfile
Notice that I remove the first 3 lines and the last line from the input file just because those are some header lines that I don't need. Next I have the code ask for how many columns to be copied to the output file, and I store that in the variable num:
Code:
# Get number of columns to be included
echo "Enter how many columns to be used"
read num
Now that the script knows how many columns I want to take from the input file, I use a for loop to ask which columns are to be copied:
Code:
echo "Provide the desired columns to be included Press [enter] after each"
for (( i = 0 ; i < $num; i++ ))
do
read choices[i]
done
No matter which columns I choose I always want to print the first column of the input file to the first column of the output file so I use this:
Code:
awk ' { print $1, " " } ' infilemod >temp1
So at this point I have a file called temp1 that has just the first column printed with a space after. Now I use another loop to try to loop over the choices of columbs to be printed:
Code:
for (( i = 0 ; i < $num; i++ ))
do
awk -v f2=temp1 -v c=${choices[$i]} ' { getline < f2; print $0, c, " "; } ' infilemod >temp2
mv temp2 temp1
done
The way I've written it is just so that awk adds columns to the file temp1 rather than writing over top of them. You can think of it as put in first column of the input file, then put in each column from the input file that you specified. This is where the problem comes in though. It does not print the column c as I have it written. I just get the blank spaces. Very frustrating!
Finally I have the code put in a header line to the output file that I have stored in another file called header.txt and remove the temp files that I used.
Hi shaner,
You still have not provided any sample output, but I think the following bash/awk script will do what you want. Note that this shell script gets its parameters from the command line instead of carrying on an interactive dialog with the user. This script does a lot more user input validation than the snippets of your code did.
I normally use ksh rather than bash, but there is a bug in the way ksh version sh (AT&T Research) 1993-12-28 s+ handles ${var#*[!0-9]} if the expansion of $var contains an exclamation point character (!) at least on OS X. If your users will never type in an exclamation point when entering a field number to be copied from the input file to the output file, both a current bash and a ksh93 or later version of the Korn shell will produce the same results with this script. (As always, if you want to run this on a Solaris/SunOS system, use /usr/xpg4/bin/awk, /usr/xpg6/bin/awk, or nawk rather than awk.)
This is a big script for this forum, but the vast majority of the code is comments. The comments explain the assumptions I made (many of which were not mentioned in your requirements).
Code:
#!/bin/bash
# USAGE:
# maybe input_file output_file new_headers_file field_number...
#
# DESCRIPTON:
# The maybe utility copies fields specified by the "field_number"
# operands from the input file specified by the "input_file" operand to
# the ouput file named by the "output_file" operand. The first three
# lines and the last line of the input file shall be skipped. The
# contents of the file named by the "new_headers_file" operand shall be
# copied to the output file before any other data is copied from the
# input file to the output file. Fields in the input file are assumed to
# be separated by one or more spaces and tabs. Leading and trailing
# spaces and tabs in the input file shall be ignored. Fields written to
# the output file shall be separated by a single tab character. The
# output fields shall be written to the output file in the order
# specified by the "field_number" operands in order from beginning to end.
#
# The results are unspecified if the input file is not a text file.
#
# If a "field_number" operand specifies a field that does not exist in
# the input file, an empty field will be written to the output file in
# the corresponding output field. Input fields are numbered starting
# with 1. If a "field_number" operand is 0, the entire input line will
# be inserted at that point in the output.
#
# RATIONALE:
# The name of this utility is chosen because this script may be doing
# what the original poster in this thread wants. The description of what
# is supposed to happen has only been described by brief text and samples
# fo code that doesn't work. No sample output has been provided. No
# input with headers has been provided.
# Initialize global variables:
IAm=${0##*/} # Final component of pathname used to invoke this script.
Usage="Usage: %s input_file output_file new_headers_file field_number..."
# Process command line arguments:
if [ $# -lt 4 ]
then printf "%s: Not enough operands.\n$Usage\n" "$IAm" "$IAm" >&2
exit 1
fi
if [ ! -r $1 ] || [ ! -r $3 ]
then printf "%s: input_file or new_headers_file not readable.\n" "$IAm" >&2
printf "$Usage\n" "$IAm" >&2
exit 2
fi
inf="$1"
outf="$2"
nhf="$3"
shift 3
# Capture the remaining operands as an array of field_numbers
fn=( "$@" )
# Verify that each field number is non-null and only contain digits.
i=0
while [ $i -lt ${#fn[@]} ]
do if [ ${#fn[$i]} -lt 1 ] || [[ "${fn[$i]}" != "${fn[$i]#*[!0-9]}" ]]
then printf "%s: field_number is not numeric.\n$Usage\n" "$IAm" \
"$IAm" >&2
exit 3
else i=$((i + 1))
fi
done
# Copy desired headers to output file.
if ! cat "$nhf" > "$outf"
then printf "$Usage\n" "$IAm" "$IAm" >&2
exit 4
fi
# Invoke awk to gather and print the requested fields after throwing away the
# 1st three header lines and the last trailer line.
printf "%d\n" ${fn[@]} | awk '
FNR == NR {
# Read list of fields to be printed from standard input...
fn[++nf] = $1
next
}
FNR <= 3 {
# Skip 3 header lines from 2nd input file.
next
}
out != "" {
# Print the results from the previous input line. (Note that we also
# have to skip the last line read for some unspecified reason. The
# delayed print allows us to do that since we do not print the last
# line when we hit EOF.)
print out
}
{ # Save requested fields (to be printed when we read the next line).
for(i = 1; i <= nf; i++)
i == 1 ? out = $fn[1] : out = out "\t" $fn[i]
}' - "$inf" >> "$outf"
This User Gave Thanks to Don Cragun For This Post:
Thanks so much! This works great and is exactly what I was trying to do. Sorry again for all the miscommunication on my part, I really appreciate the help!
Hello experts,
I wish to print the contents of odd-even numbered indices of an array.
The problem statement is as follows :
1. The first line contains an integer, (the number of test cases).
2. Each line of the subsequent lines containing a string.
Example:
2
Haider
Bash
... (4 Replies)
Hi All,
need help with reading the array and sum of the array elements.
given an array of integers of size N . You need to print the sum of the elements in the array, keeping in mind that some of those integers may be quite large.
Input Format
The first line of the input consists of an... (1 Reply)
Hello,
In a shell script, I want to declare an array and subsequently print the elements with tab delimits.
My array has the following structure and arbitrary elements:
myArray=('fgh' 'ijk' 'xyz' 'abc');
Next, I would like to print it with a '\n' at the end.
Thanks for your input!
... (2 Replies)
Hi I have two arrays :
@arcb= (450,625,720,645);
@arca=(625,645);
I need to remove the elements of @arca from elements of @arcb so that the content of @arcb will be (450,720).
Can anyone sugget me how to perform this operation?
The code I have used is this :
my @arcb=... (3 Replies)
i have an array (with each element length "n") which is dynamic and has alphanumeric characters.
i want to check if any of the elements of the array are null and replace it with a string of "n" zeros for that element.
can you suggest me a code for the same. (1 Reply)
Is there a way to print multiple array elements without iterating through the array using bash?
Can you do something like...
echo ${array}and get all those separate elements from the array? (2 Replies)
Hi
I want to write a script which store all the parameters passed to the script into an array.
Once it is stored I want scan through the array and and delete those files for last month present inside the directory. The files in directory is appneded with YYYY_MM_DD.
I want to know how can I... (3 Replies)
Hi Guys,
I have an array which has numbers including blanks as follows:
1
26
66
4.77
-0.58
88
99
11
12
333
I want to print a group of three elements as a different column in a file as follows:(including blanks where there is missing elements) for.e.g. array element #7... (4 Replies)
Hi,
I am writing a BASH shell script. I have an array that will contain IN ANY ORDER the following elements: DAY 8D MO NS.
I would like to erase the element DAY, but since the order of the elements in the array are random, I will not know which element # DAY is (ie it's not as simple as... (3 Replies)
Hi,
Please can someone help to return the array elements from a function. Currently the problem I face is that tempValue stores the value in myValue as a string while I need an array of values to be returned instead of string.
Many Thanks,
Sudhakar
the function called is:
... (5 Replies)
05-07-2013
