Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: how to write shell script to extract lines we want
Top Forums Shell Programming and Scripting how to write shell script to extract lines we want Post 302428757 by rdcwayx on Thursday 10th of June 2010 11:45:26 PM
Old 06-11-2010
Should have better solution.

Code:
$ cat urfile
seedurl1,htmlurl1
seedurl1,htmlurl2
seedurl1,htmlurl3
seedurl1,htmlurl4
seedurl1,htmlurl5
seedurl1,htmlurl6
seedurlm,htmlurl1
seedurlm,htmlurl2
seedurlm,htmlurl3
seedurlm,htmlurl4
seedurlm,htmlurl5

$ awk -F , '{a[$1]=a[$1] FS $2}
            END {for (i in a) {split(a[i],b,","); printf "%s,%s\n%s,%s\n%s,%s\n",i,b[2],i,b[3],i,b[4]}} ' urfile
seedurlm,htmlurl1
seedurlm,htmlurl2
seedurlm,htmlurl3
seedurl1,htmlurl1
seedurl1,htmlurl2
seedurl1,htmlurl3

With your real data:

Code:
$ cat urfile1
http://2010.sina.com.cn,http://2010.sina.com.cn/2010-06-09/01528724.shtml
http://2010.sina.com.cn,http://2010.sina.com.cn/2010-06-09/03238769.shtml
http://2010.sina.com.cn,http://2010.sina.com.cn/2010-06-09/04448785.shtml
http://2010.sina.com.cn,http://2010.sina.com.cn/2010-06-09/05328842.shtml
http://2010.sina.com.cn,http://2010.sina.com.cn/2010-06-09/13359200.shtml
http://zzhz.zjol.com.cn,http://zzhz.zjol.com.cn/05zzhz/system/2010/06/10/016678515.shtml
http://zzhz.zjol.com.cn,http://zzhz.zjol.com.cn/05zzhz/system/2010/06/11/016678967.shtml
http://zzhz.zjol.com.cn,http://zzhz.zjol.com.cn/05zzhz/system/2010/06/11/016679056.shtml
http://zzhz.zjol.com.cn,http://zzhz.zjol.com.cn/05zzhz/system/2010/06/11/016679169.shtml
http://zzhz.zjol.com.cn,http://zzhz.zjol.com.cn/05zzhz/system/2010/06/11/016679553.shtml
http://zzhz.zjol.com.cn,http://zzhz.zjol.com.cn/05zzhz/system/2010/06/11/016679707.shtml

$ awk -F , '{a[$1]=a[$1] FS $2}
            END {for (i in a) {split(a[i],b,","); printf "%s,%s\n%s,%s\n%s,%s\n",i,b[2],i,b[3],i,b[4]}} ' urfile1
http://zzhz.zjol.com.cn,http://zzhz.zjol.com.cn/05zzhz/system/2010/06/10/016678515.shtml
http://zzhz.zjol.com.cn,http://zzhz.zjol.com.cn/05zzhz/system/2010/06/11/016678967.shtml
http://zzhz.zjol.com.cn,http://zzhz.zjol.com.cn/05zzhz/system/2010/06/11/016679056.shtml
http://2010.sina.com.cn,http://2010.sina.com.cn/2010-06-09/01528724.shtml
http://2010.sina.com.cn,http://2010.sina.com.cn/2010-06-09/03238769.shtml
http://2010.sina.com.cn,http://2010.sina.com.cn/2010-06-09/04448785.shtml

Last edited by rdcwayx; 06-11-2010 at 01:23 AM..
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Script to Extract time from log files and write to a excel

Can someone help me with writing a unix script for following requirement 1) I have a log file in which we have start time and end time (format: hh:mm:ss) Example: starting script on Thu Jun 5 20:50:52 Thu Jun 5 21:55:33 - Script Completed 2) I want to extract start time and end time of... (0 Replies)
Discussion started by: santosham
0 Replies

2. Shell Programming and Scripting

Script to Extract time from log files and write to a excel

Can someone help me with writing a unix script for following requirement 1) I have a log file in which we have start time and end time (format: hh:mm:ss) Example: starting script on Thu Jun 5 20:50:52 Thu Jun 5 21:55:33 - Script Completed 2) I want to extract start time and end time of... (0 Replies)
Discussion started by: santosham
0 Replies

3. UNIX for Dummies Questions & Answers

Script to Extract time from log files and write to a excel

Can someone help me with writing a unix script for following requirement 1) I have a log file in which we have start time and end time (format: hh:mm:ss) Example: starting script on Thu Jun 5 20:50:52 --------- Thu Jun 5 21:55:33 - Script Completed 2) I want to extract... (4 Replies)
Discussion started by: santosham
4 Replies

4. UNIX for Dummies Questions & Answers

Write a script to extract information from a db

Hi I need to put together a script that will search certain tables in a db and send that data to a csv file. Basically I am importing data to a db and I want to write a script to check that all information was imported correctly. Thank you (1 Reply)
Discussion started by: ladyAnne
1 Replies

5. Shell Programming and Scripting

How to write a script to extract strings from a file.

Hello fourm members, I want to write a script to extarct paticular strings from the all type of files(.sh files,logfiles,txtfiles) and redirect into a log file. example: I have to find the line below in the script and extract the uname and Pwds. sqsh -scia2007 -DD0011uw01 -uciadev... (5 Replies)
Discussion started by: rajkumar_g
5 Replies

6. Shell Programming and Scripting

how to write multiple lines to a file using shell script?

I need to create an xml using shell script, but i first want to know how can i write multiple lines to file using shell script? (7 Replies)
Discussion started by: vel4ever
7 Replies

7. Shell Programming and Scripting

how to write bash script that will automatically extract zip file

i'm trying to write a bash script that that will automatically extract zip files after the download. i writed this script #!/bin/bash wget -c https://github.com/RonGokhle/kernel-downloader/zipball/master CURRENDIR=/home/kernel-downloader cd $CURRENDIR rm $CURRENDIR/zipfiles 2>/dev/null ... (2 Replies)
Discussion started by: ron gokhle
2 Replies

8. Shell Programming and Scripting

Help with Shell Script to identify lines in file1 and write them to file2

Hi, I am running my pipeline and capturing all stout from multiple programs to a .txt file. I want to go into that .txt file and search for specific lines, and finally print those lines in a second .txt file. I can do this using grep, awk, or sed for each line, but have not been able to get... (2 Replies)
Discussion started by: hmortens
2 Replies

9. UNIX for Dummies Questions & Answers

How to write Config shell script to pass variables in master shell script?

Dear Unix gurus, We have a config shell script file which has 30 variables which needs to be passed to master unix shell script that invokes oracle database sessions. So those 30 variables need to go through the database sessions (They are inputs) via a shell script. one of the variable name... (1 Reply)
Discussion started by: dba1981
1 Replies

10. Shell Programming and Scripting

How to write config shell script to pass variables in master shell script?

Dear Unix gurus, We have a config shell script file which has 30 variables which needs to be passed to master unix shell script that invokes oracle database sessions. So those 30 variables need to go through the database sessions (They are inputs) via a shell script. one of the variable name... (1 Reply)
Discussion started by: dba1981
1 Replies

LEARN ABOUT SUSE

image::exiftool::quicktime

Image::ExifTool::QuickTime(3)				User Contributed Perl Documentation			     Image::ExifTool::QuickTime(3)

NAME

       Image::ExifTool::QuickTime - Read QuickTime and MP4 meta information

SYNOPSIS

       This module is used by Image::ExifTool

DESCRIPTION

       This module contains routines required by Image::ExifTool to extract information from QuickTime and MP4 video, and M4A audio files.

AUTHOR

       Copyright 2003-2010, Phil Harvey (phil at owl.phy.queensu.ca)

       This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

REFERENCES

       <http://developer.apple.com/documentation/QuickTime/>
       http://search.cpan.org/dist/MP4-Info-1.04/ <http://search.cpan.org/dist/MP4-Info-1.04/>
       http://www.geocities.com/xhelmboyx/quicktime/formats/mp4-layout.txt <http://www.geocities.com/xhelmboyx/quicktime/formats/mp4-layout.txt>
       <http://wiki.multimedia.cx/index.php?title=Apple_QuickTime>
       http://atomicparsley.sourceforge.net/mpeg-4files.html <http://atomicparsley.sourceforge.net/mpeg-4files.html>
       <http://wiki.multimedia.cx/index.php?title=QuickTime_container>
       <http://code.google.com/p/mp4v2/wiki/iTunesMetadata>
       <http://www.canieti.com.mx/assets/files/1011/IEC_100_1384_DC.pdf>

SEE ALSO

       "QuickTime Tags" in Image::ExifTool::TagNames, Image::ExifTool(3pm)

perl v5.12.1							    2010-03-09					     Image::ExifTool::QuickTime(3)
All times are GMT -4. The time now is 08:27 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy