07-16-2008
Parshing script using input from a file to another
Hi all
As quite newbie in shell scripting i need your help.
The case: two files with question mark delimeter:
file1.txt:
A; B; D; E;
file2.txt:
a,antonis,red; b,maria,green; c,george,blue; d,jack,red; e,Helen,yellow; k,konstantin,black;
I need an effiecient way of parsing while it will read the file1.txt as input to the file2.txt and produce a file3.txt with the following results, in order to be able also to count the lines of the results:
file3.txt:
a,antonis,red;
b,maria,green;
d,jack,red;
e,Helen,yellow;
I was thinking some "for loop" but since file1.txt will be from hundren to some thousand of entries and file2.txt is almost a million lines to look into, this serial way will take ages (if the server will not crushed eventually
![Big Grin Smilie](https://www.unix.com/images/smilies/biggrin.gif)
).
Any help is more than welcome. I am working on Solaris 9 or 10 if that helps.
BR
CJ
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
how do I use a file (comma seperated) as an input for a script in bin/sh?
e.g.
I have a script that :
Input1=$1
Input2=$2
Input3=$3
Input4=$4
echo "$Input1, $Input2, $Input3,$Input4" (or some other function)
If I have a .csv file which lists many rows of input:
joe,5,john,10... (1 Reply)
Discussion started by: finster
1 Replies
2. Shell Programming and Scripting
#!/bin/sh
rpt="/export/home/legato/rpt_offsite"/test_eject.tape
cat <$rpt
while read line
do
echo $line
perform routine
done
I am trying to read the contents of this file line by line and perform a routine for each line read.
The file contents are numbers..
What is wrong with my... (1 Reply)
Discussion started by: gzs553
1 Replies
3. Shell Programming and Scripting
All,
I am trying to figure out a script to run in windows that will allow me to match on First column in file1 to 8th Column in File2 then
Insert file1 column2 to file2 column4 then create a new file.
File1:
12345 Sam
12346 Bob
12347 Bill
File2:... (1 Reply)
Discussion started by: darkoth
1 Replies
4. Shell Programming and Scripting
What is a more efficient way to read files into a script? I don't want to hard code the file names like below:
for file in file1 \
file2
do
...
done
I want to execute the script with a variable number for files for input like below:
./scriptname file1 file2 file3 ...... (3 Replies)
Discussion started by: djehresmann
3 Replies
5. Shell Programming and Scripting
Hi All,
I am hoping someone can help me with some scripting I need to complete using AWK.
I'm trying to process multiple fixed files to generate one concatenated fixed file in a standard format.
The Input file is:-
aaaa bbbbb ccccc 1 xxxx aaa bbb
aaaa bbbbb ccccc 2 abcd aaa CCC... (9 Replies)
Discussion started by: jason_v_brown
9 Replies
6. Shell Programming and Scripting
Hi all,
I have a script which checks on my jobs that run on some cluster.
The script, "script.sh", takes as an input the job-id for the job to checked. Sometimes I have 100s of jobs and I want to check them. I could put these job-ids into a file, each id in its own line.
The script would ask... (2 Replies)
Discussion started by: faizlo
2 Replies
7. Shell Programming and Scripting
Hi,
I have this script
Script.sh:
#!/bin/sh
sed 's,\,,g' input.dat > output .dat
But i want to run it witb different files. So i want the input file as an input argument to the script, how could i do that.
Running it like this:
> Script.sh input.dat (2 Replies)
Discussion started by: Johanni
2 Replies
8. Shell Programming and Scripting
Hello,
I'm trying to figure out how best to approach this script, and I have very little experience, so I could use all the help I can get. :wall:
I regularly need to delete files from many directories.
A file with the same name may exist any number of times in different subdirectories.... (3 Replies)
Discussion started by: *ShadowCat*
3 Replies
9. UNIX for Dummies Questions & Answers
I know I can obtain the file names in a folder by doing the following.
ls | awk '{t=$1; print t}'
Rather than print t, I'd like to take the file name as an input to a script like
bash-3.00$ more test.sh
echo "this is a test"
ls -l $1
When I run the script, I put "./test.sh a.out" and... (5 Replies)
Discussion started by: learnix
5 Replies
10. Shell Programming and Scripting
Hi everyone, I have problems with this script.
This script should check for a folder for each server in the list of the list.txt file.
The script only checks the first host, and then exits, why?
#!/bin/bash
file='/etc/list.txt'
while read line; do
echo $line
if ssh root@$line "stat /var >... (2 Replies)
Discussion started by: nashrik
2 Replies
DIFF3(1) General Commands Manual DIFF3(1)
NAME
diff3 - 3-way differential file comparison
SYNOPSIS
diff3 [ -exEX3 ] file1 file2 file3
DESCRIPTION
Diff3 compares three versions of a file, and publishes disagreeing ranges of text flagged with these codes:
==== all three files differ
====1 file1 is different
====2 file2 is different
====3 file3 is different
The type of change suffered in converting a given range of a given file to some other is indicated in one of these ways:
f : n1 a Text is to be appended after line number n1 in file f, where f = 1, 2, or 3.
f : n1 , n2 c Text is to be changed in the range line n1 to line n2. If n1 = n2, the range may be abbreviated to n1.
The original contents of the range follows immediately after a c indication. When the contents of two files are identical, the contents of
the lower-numbered file is suppressed.
Under the -e option, diff3 publishes a script for the editor ed that will incorporate into file1 all changes between file2 and file3, i.e.
the changes that normally would be flagged ==== and ====3. Option -x (-3) produces a script to incorporate only changes flagged ====
(====3). The following command will apply the resulting script to `file1'.
(cat script; echo '1,$p') | ed - file1
The -E and -X are similar to -e and -x, respectively, but treat overlapping changes (i.e., changes that would be flagged with ==== in the
normal listing) differently. The overlapping lines from both files will be inserted by the edit script, bracketed by "<<<<<<" and ">>>>>>"
lines.
For example, suppose lines 7-8 are changed in both file1 and file2. Applying the edit script generated by the command
"diff3 -E file1 file2 file3"
to file1 results in the file:
lines 1-6
of file1
<<<<<<< file1
lines 7-8
of file1
=======
lines 7-8
of file3
>>>>>>> file3
rest of file1
The -E option is used by RCS merge(1) to insure that overlapping changes in the merged files are preserved and brought to someone's atten-
tion.
FILES
/tmp/d3?????
/usr/libexec/diff3
SEE ALSO
diff(1)
BUGS
Text lines that consist of a single `.' will defeat -e.
7th Edition October 21, 1996 DIFF3(1)