Help with Shell Script


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help with Shell Script
# 1  
Old 11-01-2010
MySQL Help with Shell Script

Hi,

I would like reading first 3 charcters from dat file and need to creat a new file based on the reading.
Assume in dat file first 3 charcters would be "abc" .that entire record needs to be extract in to a new file and remove the record from that dat file.

Regards,
KK
# 2  
Old 11-01-2010
Code:
newname=`head -c 3 $filename`
tail -c +3 $filename > $newname


Last edited by Scott; 11-01-2010 at 08:31 AM..
# 3  
Old 11-01-2010
Check this out.

Code:
 
for i in `cat temp.txt`
do
v_substr=`expr substr "$i" 1 3`
if [ $v_substr = "abc" ]
then
echo "file contains abc"
echo $i >> file.txt
fi
done


Last edited by Scott; 11-01-2010 at 08:40 AM.. Reason: Code tags, please...
# 4  
Old 11-01-2010
Assuming input file is a standard format unix text file.
Code:
grep "^abc" file.txt > file_with_abc.txt
grep -v "^abc" file.txt > file_without_abc.txt

# 5  
Old 11-01-2010
I'm not sure we have grasped the requirement here and I'm not sure if I understand it either, but I will have a go and you can tell me if I'm miles off the mark:-
Code:
whole_line=`head -1 $input_file`
data_portion="${whole_line##???}"                     # Get everything except first 3 chars
three_chars="${whole_line%$data_portion}"             # Get everything except the data portion
target_file="header_bit${three_chars}trailer_bit"     # Add anything required to build the target file name
echo $data_portion>$target_file                       # Chose overwrite or append carefully here

tail +2 $input_file > $tmp_file                       # Get record 2 and onward.  2 steps required
cat $tmp_file > $input_file                           # else you open $input_file for output
                                                      # before you have the data to write and 
                                                      # you will null the file.

I coded the string chopping in ksh internal variable commands to avoid the overhead of starting processes (cut etc.) and the IO of writing to too many temporary files. If there is a neater way to remove a record from a file, I would like to know. I did toy with:-
Code:
echo "dd:wq" | vi $input_file

...but that can run in to problems if it is a large (wide) file or contains non-displayable data. It's a problem I get annoyed at a lot.

If you are thinking of writing a loop to process the whole file, you might be better to try one of these:-
Code:
for whole_line in `cat $input_file`
do
   data_portion="${whole_line##???}"                  # Get everything except first 3 chars
   three_chars="${whole_line%$data_portion}"          # Get everything except the data portion
   target_file="header_bit${three_chars}trailer_bit"  # Add anything required to build the target file name
   echo $data_portion >> $target_file                 # Always append or you only get the last match!
done

Code:
typeset -i counter=1
total_lines=`wc -l $input_file`
while [ $counter -le $total_lines]
do
   whole_line=`sed -n ${counter}p $input_file`
   data_portion="${whole_line##???}"                  # Get everything except first 3 chars
   three_chars="${whole_line%$data_portion}"          # Get everything except the data portion
   target_file="header_bit${three_chars}trailer_bit"  # Add anything required to build the target file name
   echo $data_portion >> $target_file                 # Always append or you only get the last match!
   ((counter=$counter+1))
done

It could be sensible to truncate/remove the target files before you begin with these if you just want the output from this run. This is a neat way to acheive that:
Code:
for three_chars in `cut -c -3 $input_file|sort -u`
do
   target_file="header_bit${three_chars}trailer_bit"  # Add anything required to build the target file name
   > $target_file                                     # or substitute rm for the >
done

You could add
Code:
echo "$target_file $counter\r\c" >&2

... to get an overwriting line count as you progress. It will slow it down a bit but you should see the file name changing as the count runs. The "\r\c" will cause it to re-write the line with the next display.

There may also be a way to blend these all together
Code:
for three_chars in `cut -c -3 $input_file|sort -u`
do
   target_file="header_bit${three_chars}trailer_bit"  # Add anything required to build the target file name
   for whole_line in `grep $three_chars $input_file`
   do
      data_portion="${whole_line##???}"                  # Get everything except first 3 chars
      echo $data_portion
   done   > $target_file                  # Overwrite to negate the need to reset previous runs
done

This may give better performance from less IO and fewer processes, but perhaps the extereme could be
Code:
for three_chars in `cut -c -3 $input_file|sort -u`
do
   target_file="header_bit${three_chars}trailer_bit"  # Add anything required to build the target file name
   grep $three_chars $input_file|cut -c 3- > $target_file
done


You would have to try out the variations to see which performs best given the variety of IO, spawned proceses etc.

I hope that this is somewhere near what you are after. Apologies if I've missed the point. Code has been raw typed, so there may be snytax errors I suppose, but have a go and let me know how you get on.



Robin
Blackburn/liverpool
UK

Last edited by rbatte1; 11-01-2010 at 09:51 AM.. Reason: Some spelling
# 6  
Old 11-01-2010
Hi,

One 'sed' solution:
Code:
$ cat script.sed
/^abc/I {
  w newfile
}
/^abc/I! {
  p
}

Code:
$ sed -i -nf script.sed infile

Regards,
Birei
# 7  
Old 11-02-2010
Hi All,

Thank you very much for replies.

I just need like this ...assume this is the input file contains like this below.

Code:
 
ABC ddd 343 1299 
ABC ddd 3564 1299
CDE ddd 3297 1393
CDE ddd 32989 1527
DEF ddd 346498 1652
DEF ddd 3269 1652
etc...


I need 3 out put files like this below.
out put File name1: ABC.dat

Code:
 
ABC ddd 343 1299 
ABC ddd 3564 1299


out put File name2 : CDE .dat

Code:
 
CDE ddd 3297 1393
CDE ddd 32989 1527



out put File name3 : DEF.dat

Code:
 
DEF ddd 346498 1652
DEF ddd 3269 1652


Last edited by Scott; 11-02-2010 at 06:59 AM.. Reason: Code tags
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Shell script newbie- how to generate service log from shell script

Hi, I am totally a newbie to any programming languages and I just started an entry level job in an IT company. One of my recent tasks is to create a script that is able to show the log file of linux service (i.e. ntpd service) lets say, if I run my script ./test.sh, the output should be... (3 Replies)
Discussion started by: xiaogeji
3 Replies

2. Shell Programming and Scripting

Pass C shell array to another C shell script(csh) and shell(sh)

Dear Friends, Please help me on this my script name is send.csh In this i have written the statement like this set args = ( city state country price ) I want to pass this array to another c shell called receiver.csh. and i want to use it in this c shell or how to pass to... (2 Replies)
Discussion started by: SA_Palani
2 Replies

3. Shell Programming and Scripting

How to write config shell script to pass variables in master shell script?

Dear Unix gurus, We have a config shell script file which has 30 variables which needs to be passed to master unix shell script that invokes oracle database sessions. So those 30 variables need to go through the database sessions (They are inputs) via a shell script. one of the variable name... (1 Reply)
Discussion started by: dba1981
1 Replies

4. UNIX for Dummies Questions & Answers

How to write Config shell script to pass variables in master shell script?

Dear Unix gurus, We have a config shell script file which has 30 variables which needs to be passed to master unix shell script that invokes oracle database sessions. So those 30 variables need to go through the database sessions (They are inputs) via a shell script. one of the variable name... (1 Reply)
Discussion started by: dba1981
1 Replies

5. Shell Programming and Scripting

Unable to pass shell script variable to awk command in same shell script

I have a shell script (.sh) and I want to pass a parameter value to the awk command but I am getting exception, please assist. diff=$1$2.diff id=$2 new=new_$diff echo "My id is $1" echo "I want to sync for user account $id" ##awk command I am using is as below cat $diff | awk... (2 Replies)
Discussion started by: Ashunayak
2 Replies

6. Shell Programming and Scripting

Correct shell script to Call One shell script from another shell script

Hi All, I have new for shell scripting. Problem : I have one scrip at serv1 and path of server is /apps/dev/provimage/scripts and script name:extract_ancillary.bat. I need to call this script at server2(my working server) and execute at server2 . Please let me know how to build the... (5 Replies)
Discussion started by: Vineeta Nigam
5 Replies

7. Shell Programming and Scripting

call another shell script and pass parameters to that shell script

Hi, I basically have 2 shell scripts. One is a shell script will get the variable value from the user. The variable is nothing but the IP of the remote system. Another shell script is a script that does the job of connecting to the remote system using ssh. This uses a expect utility in turn. ... (2 Replies)
Discussion started by: sunrexstar
2 Replies

8. Shell Programming and Scripting

How to use ssh execute other shell script on other host (shell script include nohup)?

i want use ssh on the host01 to execute autoexec.sh on the host02 like following : host01> ssh host02 autoexec.sh autoexec.sh include nohup command like follwing : nohup /home/jack/deletedata.sh & after i execute ssh host02 autoexec.sh one the host01. i can't found deletedata.sh... (1 Reply)
Discussion started by: orablue
1 Replies

9. Shell Programming and Scripting

invoking a shell script inside cgi shell script

Hi, I have an HTML form through which I get some text as input. i need to run a shell script say script.sh inside a perl-cgi script named main_cgi.sh on the form input. I want to write the contents of the form in a file and then perform some command line operations like grep, cat on the text... (2 Replies)
Discussion started by: smriti_shridhar
2 Replies

10. Shell Programming and Scripting

How to Run a shell script from Perl script in Parent shell?

Hi Perl/UNIX experts, I have a problem in running a shell script from my perl script (auto.pl). I run the perl script using perl auto.pl from the shell prompt The shell script picks the files in "input" folder and procesess it. The shell script blue.sh has this code. export... (16 Replies)
Discussion started by: hifake
16 Replies
Login or Register to Ask a Question