Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: awk-files in a txt
Top Forums Shell Programming and Scripting awk-files in a txt Post 302429102 by Mark_orig on Saturday 12th of June 2010 12:15:13 PM
Old 06-12-2010
awk-files in a txt
There is a file that contains access's data to a web server. Each line of the file (it's big) it's :

IP Client - - [date-hour] "Command Path Protocol" code size "client software"

Code:
Example,for the first line: IP Client is 67.195.37.107, date-hour is [11/Jan/2009:04:30:58 +0200], Command Path Protocol is "Get /papers/ISO4nm.pdf 
HTTP/1.0" 200 306676" code 200 size 306676, and client software is "Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; http://help.yahoo.com/help/us/ysearch/slurp)"

cat data.txt
Code:
67.195.37.107 - - [11/Jan/2009:04:30:58 +0200] "GET /papers/ISO4nm.pdf HTTP/1.0" 200 306676 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; http://help.yahoo.com/help/us/ysearch/slurp)"
64.56.66.112 - - [11/Jan/2009:04:31:19 +0200] "GET /aiai2009/registration.html HTTP/1.0" 200 4630 "-" Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; FunWebProducts)"
64.56.66.112 - - [11/Jan/2009:04:31:20 +0200] "GET /index.html HTTP/1.0" 200 4045 "-" Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; FunWebProducts)"
67.195.37.107 - - [11/Jan/2009:04:32:15 +0200] "GET /papers/ISO4nm.jpg HTTP/1.0" 200 306676 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; http://help.yahoo.com/help/us/ysearch/slurp)

I want to print the total size of the files per file type (pdf,html,jpg) using awk, sed.. There are hudnred lines I post some of them..

Help...
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

echo "ABC" > file1.txt file2.txt file3.txt

Hi Guru's, I need to create 3 files with the contents "ABC" using single command. Iam using: echo "ABC" > file1.txt file2.txt file3.txt the above command is not working. pls help me... With Regards / Ganapati (4 Replies)
Discussion started by: ganapati
4 Replies

2. Shell Programming and Scripting

AWK CSV to TXT format, TXT file not in a correct column format

HI guys, I have created a script to read 1 column in a csv file and then place it in text file. However, when i checked out the text file, it is not in a column format... Example: CSV file contains name,age aa,11 bb,22 cc,33 After using awk to get first column TXT file... (1 Reply)
Discussion started by: mdap
1 Replies

3. Solaris

list files .txt and .TXT in one command

Dear experts, In a directory i have both *.TXT and *.txt files. I have a script- for file in `ls *.txt`; do mv $file /tmp/$file How to list both *.txt and*.TXT file in one command so that script will move both .txt or .TXT whatever it find. br//purple (4 Replies)
Discussion started by: thepurple
4 Replies

4. Shell Programming and Scripting

moving the files in a.txt files to a different directory

HI All, I am coding a shell script which will pick all the .csv files in a particular directoryand write it in to a .txt file, this .txt file i will use as a source in datastage for processing. now after the processing is done I have to move and archive all the files in the .txt file to a... (5 Replies)
Discussion started by: subhasri_2020
5 Replies

5. Shell Programming and Scripting

awk append fileA.txt to growing file B.txt

This is appending a column. My question is fairly simple. I have a program generating data in a form like so: 1 20 2 22 3 23 4 12 5 43 For ever iteration I'm generating this data. I have the basic idea with cut -f 2 fileA.txt | paste -d >> FileB.txt ???? I want FileB.txt to grow, and... (4 Replies)
Discussion started by: theawknewbie
4 Replies

6. UNIX for Dummies Questions & Answers

Comparing two txt files with AWK

Hi, I need to compare two text files with awk. File1: ------- chr1 43815007 43815009 COSM19193 REF=TG;OBS=AA;ANCHOR=G AMPL495041 chr1 43815008 43815009 COSM18918 REF=G;OBS=T;ANCHOR=T AMPL495041 chr1 115256527 115256528 ... (6 Replies)
Discussion started by: RushiK
6 Replies

7. Shell Programming and Scripting

Reading txt files using the awk

Dear Shell scripters, I have a small code which copy the txt files from some destination to file name OutPutFile. I want to modify this script to introduce several constant. The string it is reading is like ... (2 Replies)
Discussion started by: nrjrasaxena
2 Replies

8. Shell Programming and Scripting

Compare two txt files,mismatches will be in new txt files

Hi, Below are the sample data for txt files. txt file 1 Partnumber|catgroup_id 10001082|46016 10001093|4680 10001093|386003 10001093|463004 10003251|683 10003251|63005 10003252|463005 10003252|4683 10003260|463005 10003260|4683 10003264|4683 10003264|463005 13420000|67... (5 Replies)
Discussion started by: Ankita Talukdar
5 Replies

9. Shell Programming and Scripting

Desired output.txt for reading txt file using awk?

Dear all, I have a huge txt file (DATA.txt) with the following content . From this txt file, I want the following output using some shell script. Any help is greatly appreciated. Greetings, emily DATA.txt (snippet of the huge text file) 407202849... (2 Replies)
Discussion started by: emily
2 Replies

10. Shell Programming and Scripting

Awk, sed, shell all words in INPUT.txt find in column1 of TABLE.txt and replce with column2 in

Hi dears i have text file like this: INPUT.txt 001_1_173 j nuh ]az 001_1_174 j ]esma. nuh ]/.xori . . . and have another text like this TABLE.txt j j nuh word1... (6 Replies)
Discussion started by: alii
6 Replies

LEARN ABOUT DEBIAN

ipv6loganon

ipv6loganon(8)															    ipv6loganon(8)

NAME

       ipv6loganon - HTTP server log file anonymizer

SYNOPSIS

       ipv6loganon [ -d value ] [ -V ] [ -n ] [ -c value ] [ --mask-ipv4 bits ] [ --no-mask-iid ] [ --anonymize-standard ]

DESCRIPTION

       ipv6loganon is a HTTP server log file anonymizer

       It expects a log line on stdin with an IPv4/IPv6 address as first token.

       This token would be anonymized according to given/default options.

       The anonymizer would keep as much information as possible for IPv6 address types.

       Client-side IID would be anonymized by

	      o EUI-48 based: serial number would be zero'ed, keeping OID

	      o EUI-64 based: serial number would be zero'ed, keeping OID

	      o ISATAP: client IPv4 address would be anonymized by given IPv4 mask

	      o TEREDO: client IPv4 address would be anonymized by given IPv4 mask, client port would be zero'ed

	      o 6to4(Microsoft): client IPv4 address would be anonymized by given IPv4 mask

	      o local: whole IID would be zero'ed

       Client-side SLA would be anonymized by

	      o SLA would be zero'ed

       Prefix would be anonymized by

	      o 6to4: client IPv4 address would be anonymized by given IPv4 mask

       Compat/Mapped IPv4 addresses would be anonymized by

	      o IPv4 address would be anonymized by given IPv4 mask

       Afterwards, the modified address and the trailing line would be printed to stdout.

OPTIONS

   GENERAL
       -d, --debug <debug value>
	      debug value (bitwise like)

       -V, --verbose
	      be verbose

       -n, --nocache
	      disable caching

       -c, --cachelimit
	      set cache limit (default: 20, maximum: 200)

   OUTPUT ANONYMIZATION
       --mask-ipv4 <bits>
	      mask all IPv4 addresses [0-32], default 8 even if occurs in IPv6-IID

       --no-mask-iid
	      do not mask non-IPv4 based IPv6-IID

       --anonymize-standard
	      preset for standard anonymization

       Default settings: mask-ipv4=8 mask-iid

EXAMPLE

       Original lines (stdin):

       207.46.98.53    -    -	 [01/Jan/2007:00:01:15	 +0100]   "GET	 /Linux+IPv6-HOWTO/x1112.html	HTTP/1.0"   200   6162	 "-"   "msnbot/1.0
       (+http://search.msn.com/msnbot.htm)" 253 6334

       2002:52b6:6b01:1:216:17ff:fe01:2345    -    -	 [10/Jan/2007:15:04:28	   +0100]     "GET     /favicon.ico	HTTP/1.1"     200     4710
       "http://www.bieringer.de/linux/IPv6/"  "Mozilla/5.0  (X11;  U;  Linux  i686;  en-US;  rv:1.8.0.9) Gecko/20061219 Fedora/1.5.0.9-1.fc6 Fire-
       fox/1.5.0.9 pango-text" 413 5005

       Modified lines (stdout):

       207.46.98.0   -	 -   [01/Jan/2007:00:01:15   +0100]   "GET   /Linux+IPv6-HOWTO/x1112.html   HTTP/1.0"	 200	6162	"-"    "msnbot/1.0
       (+http://search.msn.com/msnbot.htm)" 253 6334

       2002:52b6:6b00:0:216:17ff:fe00:0      -	   -	 [10/Jan/2007:15:04:28	   +0100]     "GET     /favicon.ico	HTTP/1.1"     200     4710
       "http://www.bieringer.de/linux/IPv6/" "Mozilla/5.0 (X11; U;  Linux  i686;  en-US;  rv:1.8.0.9)  Gecko/20061219  Fedora/1.5.0.9-1.fc6  Fire-
       fox/1.5.0.9 pango-text" 413 5005

AUTHORS

       Peter Bieringer <pb@bieringer.de>

SEE ALSO

       ipv6calc(8)

Niko Tyni <;ntyni@iki.fi>					    2007-02-01							    ipv6loganon(8)
All times are GMT -4. The time now is 03:02 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy