11-10-2015
Hi RudiC, I have thought about the categorization part of each line however it is taking too much processing power. I will rather do this at the end when I have summarized the files and sorted them. Thank you for the clarification on the uniq -d and sort part, I was not aware of the sort that needs to be there. Also the head that it only prints 10 lines. The last script you provided is working great and I have managed to get the commas in place now.
I however still am faced with a challenge as there a 9 million records and 3 servers, so in total 27 million records to be processed. The optimized script is now doing about 50 thousand lines an hour. As such I must find an alternative method of doing this even though it is working great, I need to be able to process these records as close as possible to real time.
Thank you RudiC, it is truly a pleasure and appreciated your assistance.
10 More Discussions You Might Find Interesting
1. UNIX for Advanced & Expert Users
Hi all,
I need to write a unix script in which need to call a url.
Then need to pass parameters to that url.
please help.
Regards,
gander_ss (1 Reply)
Discussion started by: gander_ss
1 Replies
2. Shell Programming and Scripting
Hi all,
I need to write a unix script in which need to call a url.
Then need to pass parameters to that url.
please help.
Regards,
gander_ss (1 Reply)
Discussion started by: gander_ss
1 Replies
3. UNIX for Dummies Questions & Answers
Hello,
I need to redirect an existing URL, how can i do that?
There's a current web address to a GUI that I have to redirect to another webaddress. Does anyone know how to do this?
This is on Unix boxes Linux.
example:
https://m45.testing.address.net/host.php
make it so the... (3 Replies)
Discussion started by: SkySmart
3 Replies
4. Windows & DOS: Issues & Discussions
Dear Expert,
i have linux box that is running in the windows domain, BUT did not being a member of the domain. as I am not the System Administrator so I have no control on the server in the network, such as modify dns entry , add the linux box in AD and domain record and so on that relevant.
... (2 Replies)
Discussion started by: regmaster
2 Replies
5. Web Development
I am trying to find a way to test some code, but I need to rewrite a specific URL only from a specific HTTP_HOST
The call goes out to
http://SUB.DOMAIN.COM/showAssignment/7bde10b45efdd7a97629ef2fe01f7303/jsmodule/Nevow.Athena
The ID in the middle is always random due to the cookie.
I... (5 Replies)
Discussion started by: EXT3FSCK
5 Replies
6. UNIX for Dummies Questions & Answers
Here is what I have so far:
find . -name "*php*" -or -name "*htm*" | xargs grep -i iframe | awk -F'"' '/<iframe*/{gsub(/.\*iframe>/,"\"");print $2}'
Here is an example content of a PHP or HTM(HTML) file:
<iframe src="http://ADDRESS_1/?click=5BBB08\" width=1 height=1... (18 Replies)
Discussion started by: striker4o
18 Replies
7. Shell Programming and Scripting
Hi,
I have a problem where i have to hit multiple URL that are stored in a text file (input.txt) and save their output in different text file (output.txt) somewhat like :
cat input.txt
http://192.168.21.20:8080/PPUPS/international?NUmber=917875446856... (3 Replies)
Discussion started by: mukulverma2408
3 Replies
8. Shell Programming and Scripting
Hello,
Am very new to perl , please help me here !!
I need help in reading a URL from command line using PERL:: Mechanize and needs all the contents from the URL to get into a file.
below is the script which i have written so far ,
#!/usr/bin/perl
use LWP::UserAgent;
use... (2 Replies)
Discussion started by: scott_cog
2 Replies
9. UNIX for Dummies Questions & Answers
I have a file like this:
http://hello.com www.examplecom computer Company
I wanted to keep dot (.) infront of com. to make the file like this
http://hello.com www.example.com computer Company
I applied this expression
sed -r 's/com/.com/g'but what I get is:
http://hello.com ... (4 Replies)
Discussion started by: csim_mohan
4 Replies
10. UNIX for Dummies Questions & Answers
I have a file like this:
http://article.wn.com/view/2010/11/26/IV_drug_policy_feels_HIV_patients_Red_Cross/ http://aidsjournal.com/,www.cfpa.org.cn/page1/page2 , www.youtube.com
http://seattletimes.nwsource.com/html/jerrybrewer/2013517803_brewer25.html... (1 Reply)
Discussion started by: csim_mohan
1 Replies
LEARN ABOUT X11R4
sortbib
sortbib(1) User Commands sortbib(1)
NAME
sortbib - sort a bibliographic database
SYNOPSIS
sortbib [-s KEYS] database...
DESCRIPTION
sortbib sorts files of records containing refer key-letters by user-specified keys. Records may be separated by blank lines, or by `.[' and
`.]' delimiters, but the two styles may not be mixed together. This program reads through each database and pulls out key fields, which are
sorted separately. The sorted key fields contain the file pointer, byte offset, and length of corresponding records. These records are
delivered using disk seeks and reads, so sortbib may not be used in a pipeline to read standard input.
The most common key-letters and their meanings are given below.
%A Author's name
%B Book containing article referenced
%C City (place of publication)
%D Date of publication
%E Editor of book containing article referenced
%F Footnote number or label (supplied by refer)
%G Government order number
%H Header commentary, printed before reference
%I Issuer (publisher)
%J Journal containing article
%K Keywords to use in locating reference
%L Label field used by -k option of refer
%M Bell Labs Memorandum (undefined)
%N Number within volume
%O Other commentary, printed at end of reference
%P Page number(s)
%Q Corporate or Foreign Author (unreversed)
%R Report, paper, or thesis (unpublished)
%S Series title
%T Title of article or book
%V Volume number
%X Abstract -- used by roffbib, not by refer
%Y,Z Ignored by refer
By default, sortbib alphabetizes by the first %A and the %D fields, which contain the senior author and date.
sortbib sorts on the last word on the %A line, which is assumed to be the author's last name. A word in the final position, such as `jr.'
or `ed.', will be ignored if the name beforehand ends with a comma. Authors with two-word last names or unusual constructions can be sorted
correctly by using the nroff convention ` ' in place of a blank. A %Q field is considered to be the same as %A, except sorting begins with
the first, not the last, word. sortbib sorts on the last word of the %D line, usually the year. It also ignores leading articles (like `A'
or `The') when sorting by titles in the %T or %J fields; it will ignore articles of any modern European language. If a sort-significant
field is absent from a record, sortbib places that record before other records containing that field.
No more than 16 databases may be sorted together at one time. Records longer than 4096 characters will be truncated.
OPTIONS
-sKEYS Specify new KEYS. For instance, -sATD will sort by author, title, and date, while -sA+D will sort by all authors, and date. Sort
keys past the fourth are not meaningful.
ATTRIBUTES
See attributes(5) for descriptions of the following attributes:
+-----------------------------+-----------------------------+
| ATTRIBUTE TYPE | ATTRIBUTE VALUE |
+-----------------------------+-----------------------------+
|Availability |SUNWdoc |
+-----------------------------+-----------------------------+
SEE ALSO
addbib(1), indxbib(1), lookbib(1), refer(1), roffbib(1), attributes(5)
BUGS
Records with missing author fields should probably be sorted by title.
SunOS 5.10 14 Sep 1992 sortbib(1)