Shell script (beginner) need help...


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Shell script (beginner) need help...
# 1  
Old 05-23-2011
Shell script (beginner) need help...

Hello,

I'm new to Sheel script and I need your help for a script I need to develop (for me).

Indead, I have a software which log all entry from internet and save it in text file.
But, the log is practically unreadable because every 256 characters jump to a new line (even if the message is not finish).

For example (fake):
Quote:
10:02:23:124:id_0000:1:"RaZ."
10:02:23:125:id_0001:1:"A entry was detecte"
10:02:23:125:id_0001:2:"d at port 200. Pleas"
10:02:23:125:id_0001:3:"e call suport."
10:04:56:201:id_0002:1:"RaZ."
I need to transform it into:
Quote:
10:02:23:124:id_0000:1:"RaZ."
10:02:23:125:id_0001:1:"A entry was detected at port 200. Please call suport."
10:04:56:201:id_0002:1:"RaZ."
I tried some script with uniq or sort, but, I'm beginner and I need your help Smilie

Thanks a lot for your response,

Acid

[edit] the log file can be huge (100 000 lines, so it's vey unpleasant to read)
# 2  
Old 05-23-2011
Try out this code...
origfile.txt= The actual log file
newfile.txt = formatted log file created by script.

Code:
cat origfile.txt|cut -c 1-20|while read line
do
log=`grep -i "${line}" origfile.txt|cut -c 24- |sed 's/"//g'`
grep -i "${line}" newfile.txt > /dev/null 2>&1
if [ $? -ne 0 ];then
echo $line:1:$log >> newfile.txt
fi
done


Last edited by pludi; 05-23-2011 at 05:23 PM..
This User Gave Thanks to vipinable For This Post:
# 3  
Old 05-23-2011
See if this works for you:
Code:
#!/usr/bin/ksh
while read mLine; do
  mTag=$(echo ${mLine} | cut -d':' -f5)
  if [[ "${mTag}" = "id_0000" ]]; then
    echo ${mLine}
    mFirst="Y"
  else
    if [[ "${mTag}" = "id_0001" ]]; then
      if [[ "${mFirst}" = "Y" ]]; then
        mOutLine=$(echo ${mLine} | sed 's/"$//')
        mFirst="N"
      else
        mValue=$(echo ${mLine} | sed 's/.*://;s/"//g')
        mOutLine=${mOutLine}${mValue}
      fi
    else
      echo ${mOutLine}'"'
      echo ${mLine}
    fi
  fi
done < inp_file

This User Gave Thanks to Shell_Life For This Post:
# 4  
Old 05-24-2011
Firslty, thank a lot for your help :-)

vipinable, I tried your script, I have no error at all when I launch the script but it's never stop for me ? But I'll try to understand in depth your code Smilie

Shell_Life, I guess your script work only for my example? But thanks a lot, I'll try to use you algorithm too Smilie

Thanks you again Smilie

Acid

[edit] vipinable, sorry, your script work !! thank a lot Smilie

Last edited by acidoangel; 05-24-2011 at 04:50 AM..
# 5  
Old 05-24-2011
Code:
 $ awk -F\: '{key=substr($0,0,20);gsub(/\"/,"");arr[key]=arr[key]$NF;} END{for (k in arr) printf "%s:\"%s\"\n", k,arr[k]}' file.txt
10:04:56:201:id_0002:"RaZ."
10:02:23:125:id_0001:"A entry was detected at port 200. Please call suport."
10:02:23:124:id_0000:"RaZ."

This User Gave Thanks to kumaran_5555 For This Post:
# 6  
Old 05-25-2011
Wow i love your solution Smilie Thanks a lot...
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Beginner bash - basic shell script 'while' help...

Hi everyone, first time visitor to these forums here. Keeping a long story short I've been attempting to learn how to code in bash. I have VERY little previous experience with coding languages besides simply copying and pasting batch scripts for Windows. So, with that in mind I've followed a... (4 Replies)
Discussion started by: Meta
4 Replies

2. Shell Programming and Scripting

If condition shell script beginner

Hi all I have the folloing process that needs checking often: ps -ef | grep ih bscsrtx 206 15901 0 11:28:10 pts/6 0:00 fih -r4 bscsrtx 218 15901 0 11:28:27 pts/6 0:01 aih bscsrtx 29763 15901 4 11:27:16 pts/6 0:59 rdh -prih root 429 27268 0 11:30:15 pts/td ... (13 Replies)
Discussion started by: fretagi
13 Replies

3. Shell Programming and Scripting

C shell - a question of a beginner

Hi, can someone explain how this aliases work: alias m 'if ( \!:$ =~ *gz) set status = 1 && most \!:$ || gzcat \!:1 | most ' I mean, I know what it does, but I don't know how - what is \!:$ and \!:1 - how do it used the && and || - what it does with status variable? thanks (3 Replies)
Discussion started by: tzvikao
3 Replies

4. Shell Programming and Scripting

Shell Programming (beginner help)

So guys basically I was really sick and couldn't attend the labs and lectures and I went to my lecture hoping he would say ok I will help you from the start but he just said google it. So If it's possible to make the assignment and explain more in detail why is that would be really helpfull. I can... (1 Reply)
Discussion started by: Joola94
1 Replies

5. Homework & Coursework Questions

Shell Script (beginner)

1. The problem statement, all variables and given/known data: Arguments: http://farm9.staticflickr.com/8070/8212131370_8b6e8c10c5_c.jpg I am given these three arguments. $1, $2, $3 The first argument is the path to a directory. So, how would I go into the directory and compare files? I... (5 Replies)
Discussion started by: spider-man
5 Replies

6. Homework & Coursework Questions

Shell-script loops beginner

Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted! 1. The problem statement, all variables and given/known data: "Create a shell script that changes the selected word to another word in all files from selected archive. The... (1 Reply)
Discussion started by: oibanuelos
1 Replies

7. Shell Programming and Scripting

Debugging a beginner shell script...

I have been following a tutorial on bash which has proven to be very helpful. However, i am stuck with a command not found issue when asking for a y/n response from the user. Below is the part of code I believe is giving me grief... I have been trying to work through this for 3 hours now.... Please... (5 Replies)
Discussion started by: brokepunk
5 Replies

8. Shell Programming and Scripting

Beginner shell script help

. ./testFile.sh url=http://ichart.finance.yahoo.com/table.csv?s= suf=&d=5&e=9&f=2009&g=d&a=1&b=4&c=1999&ignore=.csv wget $url$s1$suf; sleep 10; cat header.txt > $s1.txt; chmod 777 $s1.txt; sed '1d' table.csv?s\=$s1 >> $s1.txt; rm -Rf table* Very new at shell scripting as you can see... (3 Replies)
Discussion started by: harte
3 Replies

9. Shell Programming and Scripting

Shell Script for Beginner

I have a folder with lots of file. e.g. a.txt, b.txt, c.txt.... I want to put these files from the source directory and place them in a destination directory in a specific order, such as /destination/a/a.txt, /destination/b/b.txt, /destination/c/c.txt, ...... Please help. Thx :confused: (3 Replies)
Discussion started by: aaron_fong
3 Replies

10. Shell Programming and Scripting

Beginner trying to write a shell script!

Dear All, I'm trying to write a shell script that continously checks a certain folder. When a file is placed in the directory securely copies the file to another server. I've got the secure copying working, but I don't know how to contiously check a directory for a new file and then use that... (3 Replies)
Discussion started by: duncan_glover
3 Replies
Login or Register to Ask a Question