Sponsored Content
Top Forums Shell Programming and Scripting Curl and write custom result to file Post 303034404 by RudiC on Wednesday 24th of April 2019 06:01:51 PM
Old 04-24-2019
Good you found a solution on your own. How about some improvements? read the MyWebApps file in lieu of catting it in a "command substitution" (saves a process creation), write to the log file outside the loop (avoids opening / closing it over and over again), and combine five awk executions into one, like


Code:
while read i; do curl ...$i... | awk -F"[,/ ]" -vDT="$(date +"%Y-%m-%d-%H:%M")" -vOFS="," '{print DT, $7, $NF}'; done < MyWebApps > sessions.log

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Outputting formatted Result log file from old 30000 lines result log<help required>

Well I have a 3000 lines result log file that contains all the machine data when it does the testing... It has 3 different section that i am intrsted in 1) starting with "20071126 11:11:11 Machine Header 1" 1000 lines... "End machine header 1" 2) starting with "20071126 12:12:12 Machine... (5 Replies)
Discussion started by: vikas.iet
5 Replies

2. Shell Programming and Scripting

script to output curl result as html

hi, new to scripting and would like to know how can I have a script which will curl a few URLs and have the results such as the URLs being curled, dns lookup time, connection time, total time, etc save in a html format in a form of table with column and rows. thank you. (4 Replies)
Discussion started by: squidusr
4 Replies

3. Shell Programming and Scripting

How to write result of a query to more than 1 .csv

If the result of the query is greater than say, 50,000 then the next 50,000 should be written to the second file and so on. Is it possible? (1 Reply)
Discussion started by: Jassz
1 Replies

4. Shell Programming and Scripting

Write result to output file

Good morning everybody, Beeing an absolute newbie in shell scripting I would like to look for some help here. I would like to read an external text file and format the data and write it to an output file. What I was trying to do was to display the result (this worked). But now I... (1 Reply)
Discussion started by: bluejean1976
1 Replies

5. Shell Programming and Scripting

Populating File data with custom manipulation on file names

Hi, I am confused how to proceed firther please find the problem below: Input Files: DCIA_GEOG_DATA_OCEAN.TXT DCIA_GEOG_DATA_MCRO.TXT DCIA_GEOG_DATA_CVAS.TXT DCIA_GEOG_DATA_MCR.TXT Output File Name: MMA_RFC_GEOG_NAM_DIM_LOD.txt Sample Record(DCIA_GEOG_DATA_OCEAN.TXT):(Layout same for... (4 Replies)
Discussion started by: Arun Mishra
4 Replies

6. Shell Programming and Scripting

Script to write result to a file

Hello, How can I run this script every 1 hour and save its result to result.txt ifconfig | grep 'inet addr:'| grep -v '127.0.0.1' | cut -d: -f2 | awk '{ print $1}' Regards Shaan (5 Replies)
Discussion started by: Shaan_Shaan
5 Replies

7. Shell Programming and Scripting

Compare 2 file and write result

Hi friends...I have 2 files, file1.txt and reference.txt I could able to find difference using diff and following command awk 'NR == FNR { A=1; next } !A' reference.txt file1.txt above command listing data which is not in reference.txt 12.12 87 11.95 88 11.83 89 12.55 84... (12 Replies)
Discussion started by: Akshay Hegde
12 Replies

8. Shell Programming and Scripting

Splitting file - custom req.

Hello everybody, I have a file to split in a bit different way. My file looks like this: My_file.dat 123333333|01|38|440X|P 384747382|32|31|440X|P 320384743|32|54|420Y|I 843739349|12|43|420Y|I I need to split it into: MYFILE_440X_P.dat 123333333|01|38 384747382|32|31 ... (3 Replies)
Discussion started by: bijou
3 Replies

9. Shell Programming and Scripting

Sort and Split file with header and custom name

Hi, I am using SUN SOLARIS (SunOS sun4v sparc SUNW, T5240). I have a huge data file with header and trailer. This file gets used into an ETL process. ETL skips the header record (which is the first record of the file) and loads the rest of the record. The file can be delimited (comma,... (5 Replies)
Discussion started by: Saanvi1
5 Replies

10. Shell Programming and Scripting

Curl to download file from subdivx.com after following location without knowing the file name/extens

This question could be specific to the site subdivx.com In the past, I've been able to download a file following location using cURL but there is something about subdivx.com that's different and can't figure out how to get it to work. I tried the following directly in the terminal with no... (5 Replies)
Discussion started by: MoonD
5 Replies
NEWSLOG(5)							File Formats Manual							NEWSLOG(5)

NAME
newslog - description of Usenet log files DESCRIPTION
Most log files created by Usenet programs reside in the /var/log/news directory and have a ``.log'' extension. Several versions are usu- ally kept with an additional extension such as ``.1'', ``.2'', etc. -- the higher the number, the older the log. The older versions are compressed. The scanlogs script and related utilities (see newslog(8)) are responsible for rotating and compressing these files. Some log files always have data, others only have data if there is a problem, and others are only created if a particular program is used or configuration parameter is set. The innstat script (see newslog(8)) monitors the size of all log files. The following files will only accumulate data under the direction of control.ctl(5): control.log miscctl.log newgroup.log rmgroup.log unwanted.log In order to create these files, the ``message'' and ``action'' fields of control.ctl should be chosen from the following table: Message Action Meaning all log=miscctl Log all messages by default default log=miscctl Log unknown messages newgroup doit=newgroup Create group and log message newgroup log=newgroup Log message rmgroup doit=rmgroup Remove group and log message rmgroup log=rmgroup Log message ``other'' doit=miscctl log and process the message ``other'' log=miscctl Log message Here, ``other'' refers to any other control message such as: checkgroups ihave sendme sendsys senduuname version The following is a list of log files. control.log This file maintains a count of the number of newgroup and rmgroup control messages seen for each newsgroup. The count is of the number of control messages with identical arguments, regardless if they were actually processed. All control arguments, including invalid ones, are counted. This file is updated by tally.control, which is invoked by scanlogs if either the newgroup or rmgroup logs exist. This file is not rotated. errlog This file contains the standard output and standard error of any program spawned by innd(8). The most common programs are the con- trol-message handlers found in /usr/lib/news/control. This file should be empty. Scanlogs will print the entire contents of this log file if it is non-empty. expire.log By default, when news.daily is going to expire old news articles, it writes the date to this file, followed by any output from expire(8) and the ending date. All lines but the first are indented four spaces. miscctl.log When control.ctl is configured as described above, all control messages except newgroup and rmgroup are appended to this file by writelog. There will be a summary line describing the message and the action taken, followed by the article indented by four spa- ces, and a blank line. newgroup.log When control.ctl is configured as described above, all newgroup messages are appended to this file using the same format as for mis- cctl.log. news This file logs articles received by innd. Scanlogs summarizes the rejected articles reported in this file. news.crit All critical error messages issued by innd are appended to this file via syslog(3). This log file should be empty. Scanlogs will print the entire contents of this log file if it is non-empty. You should have the following line in your syslog.conf(5) file: news.crit /var/log/news/news.crit news.err All major error messages issued by innd are appended to this file via syslog. This log file should be empty. Scanlogs will print the entire contents of this log file if it is non-empty. You should have the following line in your syslog.conf file: news.err /var/log/news/news.err news.notice All standard error messages and status messages issued by innd are appended to this file via syslog. Scanlogs uses the awk(1) script innlog.awk to summarize this file. You should have the following line in your syslog.conf file: news.notice /var/log/news/news.notice nntpsend.log The nntpsend(8) programs appends all status messages to this file. rmgroup.log When control.ctl is configured as described above, all rmgroup messages are appended to this file using the same format as for misc- ctl.log. unwanted.log This log maintains a count of the number of articles that were rejected because they were posted to newsgroups that do not exist at the local site. This file is updated by tally.unwanted and maintained in reverse numeric order (the most popular rejected group first). This file is not rotated. HISTORY
Written by Landon Curt Noll <chongo@toad.com> and Rich $alz <rsalz@uunet.uu.net> for InterNetNews. This is revision 1.6, dated 1996/09/10. SEE ALSO
control.ctl(5), ctlinnd(8), expire(8), innd(8), news.daily(8), nntpsend(8), newslog(8). NEWSLOG(5)
All times are GMT -4. The time now is 09:33 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy