Sponsored Content
Top Forums Shell Programming and Scripting Problem with reading file line-by-line, and outputting to a new file Post 302215867 by Darkness Fish on Thursday 17th of July 2008 10:59:52 AM
Old 07-17-2008
Question Problem with reading file line-by-line, and outputting to a new file

Hi everyone. I realise this is probably a bit of a noob question, but I'm actually a C# developer working on a legacy system, and can't remember much unix.

I want to read from a pipe-delimeted file like formatted thusly:

idno|PRODUCT|Name|street town postcode|etc|etc|etc|etc
idno|PRODUCT|Name|street town postcode|etc|etc|etc|etc
idno|PRODUCT|Name|street town postcode|etc|etc|etc|etc

I want to copy each line to another file, dependant upon the PRODUCT existing in a string variable I'm holding in memory: "prod1|prod2|prod3" ($products in example below)

Therefore I was trying:

for line in `cat $temp_dir/$Filename | awk '{ print $0 } '`
do
prod=`echo "$line" | awk ' BEGIN {FS="|"} {print $2} '`
valid_prod=`echo $products | grep "$prod"`
if [ -n "$valid_prod" ] ; then
echo "$line" >> $new_file.txt
fi
done

however this line variable seems to separate based on spaces, creating the new file as:

idno|PRODUCT|Name|street
town
postcode|etc|etc|etc|etc

Is there anyway I can pass each line through as a single parameter, regardless of spaces?

Any help is greatfully appreciated,
Dave
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

sed not outputting last line of input file

I am using sed for a simple substitution (see command syntax below). Everything works fine except that the last line of the input file does not get written to the output file. Has anyone ever seen this and know of way to force the last line to be written? I don't know if it's playing a part in... (3 Replies)
Discussion started by: 2reperry
3 Replies

2. Shell Programming and Scripting

Reading a file line by line and processing for each line

Hi, I am a beginner in shell scripting. I have written the following script, which is supposed to process the while loop for each line in the sid_home.txt file. But I'm getting the 'end of file' unexpected for the last line. The file sid_home.txt gets generated as expected, but the script... (6 Replies)
Discussion started by: sagarparadkar
6 Replies

3. Shell Programming and Scripting

[Solved] Problem in reading a file line by line till it reaches a white line

So, I want to read line-by-line a text file with unknown number of files.... So: a=1 b=1 while ; do b=`sed -n '$ap' test` a=`expr $a + 1` $here do something with b etc done the problem is that sed does not seem to recognise the $a, even when trying sed -n ' $a p' So, I cannot read... (3 Replies)
Discussion started by: hakermania
3 Replies

4. UNIX for Dummies Questions & Answers

Parsing file, reading each line to variable, evaluating date/time stamp of each line

So, the beginning of my script will cat & grep a file with the output directed to a new file. The data I have in this file needs to be parsed, read and evaluated. Basically, I need to identify the latest date/time stamp and then calculate whether or not it is within 15 minutes of the current... (1 Reply)
Discussion started by: hynesward
1 Replies

5. Shell Programming and Scripting

Reading text file, comparing a value in a line, and placing only part of the line in a variable?

I need some help. I would like to read in a text file. Take a variable such as ROW-D-01, compare it to what's in one line in the text file such as PROD/VM/ROW-D-01 and only input PROD/VM into a variable without the /ROW-D-01. Is this possible? any help is appreciated. (2 Replies)
Discussion started by: xChristopher
2 Replies

6. Shell Programming and Scripting

Comparison of fields then increment a counter reading line by line in a file

Hi, i have a scenario were i should compare a few fields from each line then increment a variable based on that. Example file 989878|8999|Y|0|Y|N|V 989878|8999|Y|0|N|N|V 989878|8999|Y|2344|Y|N|V i have 3 conditions to check and increment a variable on every line condition 1 if ( $3... (4 Replies)
Discussion started by: selvankj
4 Replies

7. Shell Programming and Scripting

Reading line by line from live log file using while loop and considering only those lines start from

Hi, I want to read a live log file line by line and considering those line which start from time stamp; Below code I am using, which read line but throws an exception when comparing line that does not contain error code tail -F /logs/COMMON-ERROR.log | while read myline; do... (2 Replies)
Discussion started by: ketanraut
2 Replies

8. Shell Programming and Scripting

Problem with while loop reading every line of a text file

Hello, I'm using RHEL 5.1 with bash. How to handle "read" inside while loop reading every line? Please see below: # cat /tmp/passwd_sample CARRJ12:qVSn4ja4mFA72,..:20021:125:JULIAN CARR:/home/everyone:/bin/bash HERCOT01:NK/3j2ZB4ZC7Q:20022:125:TOM HERCOCK:/home/everyone:/bin/bash... (4 Replies)
Discussion started by: reddyr
4 Replies

9. UNIX for Beginners Questions & Answers

Reading a file line by line and print required lines based on pattern

Hi All, i want to write a shell script read below file line by line and want to exclude the lines which contains empty value for MOUNTPOINT field. i am using centos 7 Operating system. want to read below file. # cat /tmp/d5 NAME="/dev/sda" TYPE="disk" SIZE="60G" OWNER="root"... (4 Replies)
Discussion started by: balu1234
4 Replies
drslib()																  drslib()

NAME
drs_tool - Command-line interface to DRSLIB SYNPOSIS
DESCRIPTION
USAGE
Usage Usage: drs_tool [command] [options] [drs-pattern] command: list list publication-level datasets todo show file operations pending for the next version upgrade make changes to the selected datasets to upgrade to the next version mapfile make a mapfile of the selected dataset history list all versions of the selected dataset init initialise CMIP5 product detection data OPTIONS
--help display this help message and exit --version output version information and exit -h, --help show this help message and exit -R ROOT, --root=ROOT Root directory of the DRS tree -I INCOMING, --incoming=INCOMING Incoming directory for DRS files. Defaults to <root>/output -a ACTIVITY, --activity=ACTIVITY Set DRS attribute activity for dataset discovery -p PRODUCT, --product=PRODUCT Set DRS attribute product for dataset discovery -i INSTITUTE, --institute=INSTITUTE Set DRS attribute institute for dataset discovery -m MODEL, --model=MODEL Set DRS attribute model for dataset discovery -e EXPERIMENT, --experiment=EXPERIMENT Set DRS attribute experiment for dataset discovery -f FREQUENCY, --frequency=FREQUENCY Set DRS attribute frequency for dataset discovery -r REALM, --realm=REALM Set DRS attribute realm for dataset discovery -v VERSION, --version=VERSION Force version upgrades to this version -P FILE, --profile=FILE Profile the script exectuion into FILE --detect-product Automatically detect the DRS product of incoming data REQUIRES
FILES
drs_tool uses metaconfig to configure and store configuration information. Its configuration may be found then in a metaconfig.conf file, in /etc/metaconfig/metaconfig.conf on Debian systems, which may be overriden by $HOME/.metaconfig.conf or ./metaconfig.conf ENVIRONMENT VARIABLES
The variable METACONF_CONF VERSION
0.3.0a3 BUGS
To query and report bugs in drs_tool, please use reportbug on Debian systems. SEE ALSO
http://esgf.org/esgf-drslib-site/index.html AUTHOR
DRSLIB was written by Stephen Pascoe, STFC.SC.UK. This manual page was written by Alastair McKinstry. drslib()
All times are GMT -4. The time now is 08:55 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy