07-17-2008
Problem with reading file line-by-line, and outputting to a new file
Hi everyone. I realise this is probably a bit of a noob question, but I'm actually a C# developer working on a legacy system, and can't remember much unix.
I want to read from a pipe-delimeted file like formatted thusly:
idno|PRODUCT|Name|street town postcode|etc|etc|etc|etc
idno|PRODUCT|Name|street town postcode|etc|etc|etc|etc
idno|PRODUCT|Name|street town postcode|etc|etc|etc|etc
I want to copy each line to another file, dependant upon the PRODUCT existing in a string variable I'm holding in memory: "prod1|prod2|prod3" ($products in example below)
Therefore I was trying:
for line in `cat $temp_dir/$Filename | awk '{ print $0 } '`
do
prod=`echo "$line" | awk ' BEGIN {FS="|"} {print $2} '`
valid_prod=`echo $products | grep "$prod"`
if [ -n "$valid_prod" ] ; then
echo "$line" >> $new_file.txt
fi
done
however this line variable seems to separate based on spaces, creating the new file as:
idno|PRODUCT|Name|street
town
postcode|etc|etc|etc|etc
Is there anyway I can pass each line through as a single parameter, regardless of spaces?
Any help is greatfully appreciated,
Dave
9 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I am using sed for a simple substitution (see command syntax below). Everything works fine except that the last line of the input file does not get written to the output file. Has anyone ever seen this and know of way to force the last line to be written?
I don't know if it's playing a part in... (3 Replies)
Discussion started by: 2reperry
3 Replies
2. Shell Programming and Scripting
Hi,
I am a beginner in shell scripting. I have written the following script, which is supposed to process the while loop for each line in the sid_home.txt file. But I'm getting the 'end of file' unexpected for the last line. The file sid_home.txt gets generated as expected, but the script... (6 Replies)
Discussion started by: sagarparadkar
6 Replies
3. Shell Programming and Scripting
So, I want to read line-by-line a text file with unknown number of files....
So:
a=1
b=1
while ; do
b=`sed -n '$ap' test`
a=`expr $a + 1`
$here do something with b etc
done
the problem is that sed does not seem to recognise the $a, even when trying
sed -n ' $a p'
So, I cannot read... (3 Replies)
Discussion started by: hakermania
3 Replies
4. UNIX for Dummies Questions & Answers
So, the beginning of my script will cat & grep a file with the output directed to a new file. The data I have in this file needs to be parsed, read and evaluated.
Basically, I need to identify the latest date/time stamp and then calculate whether or not it is within 15 minutes of the current... (1 Reply)
Discussion started by: hynesward
1 Replies
5. Shell Programming and Scripting
I need some help. I would like to read in a text file.
Take a variable such as ROW-D-01, compare it to what's in one line in the text file such as PROD/VM/ROW-D-01 and only input PROD/VM into a variable without the /ROW-D-01.
Is this possible? any help is appreciated. (2 Replies)
Discussion started by: xChristopher
2 Replies
6. Shell Programming and Scripting
Hi,
i have a scenario were i should compare a few fields from each line then increment a variable based on that.
Example file
989878|8999|Y|0|Y|N|V
989878|8999|Y|0|N|N|V
989878|8999|Y|2344|Y|N|V
i have 3 conditions to check and increment a variable on every line
condition 1 if ( $3... (4 Replies)
Discussion started by: selvankj
4 Replies
7. Shell Programming and Scripting
Hi,
I want to read a live log file line by line and considering those line which start from time stamp;
Below code I am using, which read line but throws an exception when comparing line that does not contain error code
tail -F /logs/COMMON-ERROR.log | while read myline; do... (2 Replies)
Discussion started by: ketanraut
2 Replies
8. Shell Programming and Scripting
Hello,
I'm using RHEL 5.1 with bash. How to handle "read" inside while loop reading every line? Please see below:
# cat /tmp/passwd_sample
CARRJ12:qVSn4ja4mFA72,..:20021:125:JULIAN CARR:/home/everyone:/bin/bash
HERCOT01:NK/3j2ZB4ZC7Q:20022:125:TOM HERCOCK:/home/everyone:/bin/bash... (4 Replies)
Discussion started by: reddyr
4 Replies
9. UNIX for Beginners Questions & Answers
Hi All,
i want to write a shell script read below file line by line and want to exclude the lines which contains empty value for MOUNTPOINT field.
i am using centos 7 Operating system.
want to read below file.
# cat /tmp/d5
NAME="/dev/sda" TYPE="disk" SIZE="60G" OWNER="root"... (4 Replies)
Discussion started by: balu1234
4 Replies
LEARN ABOUT DEBIAN
drs_tool
drslib() drslib()
NAME
drs_tool - Command-line interface to DRSLIB
SYNPOSIS
DESCRIPTION
USAGE
Usage
Usage: drs_tool [command] [options] [drs-pattern]
command: list list publication-level datasets todo show file operations pending for the next version upgrade make changes to
the selected datasets to upgrade to the next version mapfile make a mapfile of the selected dataset history list all versions of the
selected dataset init initialise CMIP5 product detection data
OPTIONS
--help display this help message and exit
--version
output version information and exit
-h, --help show this help message and exit
-R ROOT, --root=ROOT
Root directory of the DRS tree
-I INCOMING, --incoming=INCOMING
Incoming directory for DRS files. Defaults to <root>/output
-a ACTIVITY, --activity=ACTIVITY
Set DRS attribute activity for dataset discovery
-p PRODUCT, --product=PRODUCT
Set DRS attribute product for dataset discovery
-i INSTITUTE, --institute=INSTITUTE
Set DRS attribute institute for dataset discovery
-m MODEL, --model=MODEL
Set DRS attribute model for dataset discovery
-e EXPERIMENT, --experiment=EXPERIMENT
Set DRS attribute experiment for dataset discovery
-f FREQUENCY, --frequency=FREQUENCY
Set DRS attribute frequency for dataset discovery
-r REALM, --realm=REALM
Set DRS attribute realm for dataset discovery
-v VERSION, --version=VERSION
Force version upgrades to this version
-P FILE, --profile=FILE
Profile the script exectuion into FILE
--detect-product
Automatically detect the DRS product of incoming data
REQUIRES
FILES
drs_tool uses metaconfig to configure and store configuration information. Its configuration may be found then in a metaconfig.conf file,
in /etc/metaconfig/metaconfig.conf on Debian systems, which may be overriden by $HOME/.metaconfig.conf or ./metaconfig.conf
ENVIRONMENT VARIABLES
The variable METACONF_CONF
VERSION
0.3.0a3
BUGS
To query and report bugs in drs_tool, please use reportbug on Debian systems.
SEE ALSO
http://esgf.org/esgf-drslib-site/index.html
AUTHOR
DRSLIB was written by Stephen Pascoe, STFC.SC.UK. This manual page was written by Alastair McKinstry.
drslib()