Unix/Linux Go Back    

Homework & Coursework Questions Students must use and complete the template provided. If you do not use the template your post may be deleted! Special homework rules apply here.

Curl Loop help

Homework & Coursework Questions

Thread Tools Search this Thread Display Modes
Old Unix and Linux 11-02-2016   -   Original Discussion by kjcraig77
kjcraig77's Unix or Linux Image
kjcraig77 kjcraig77 is offline
Registered User
Join Date: Nov 2016
Last Activity: 9 November 2016, 10:58 AM EST
Posts: 5
Thanks: 1
Thanked 0 Times in 0 Posts
Curl Loop help

1. The problem statement, all variables and given/known data:
Here is my assignment.
Parse the /home/access_log and /home/access_.xz files
Create a sorted list of ip address with no duplicates
Pass each ip address to ipinfo.io with curl
Create a logfile that has the ip address,lat,long
Save the data file and script to the inclass dir
Submit your script to canvas email_first_part_lab8.txt

I am trying to figure out how to create a loop to cat all the access logs then curl all the ips in that log to the site ive been provided and then redirect that ouput to a text file. I am still rather ignorant to shell scripting.

2. The attempts at a solution (include all code and scripts):

Here are my scripts to cat the files

cat /home/access_log | awk '{ print $1 }' | sort | uniq -c | sort -n

xzcat /home/access_log-20160917.xz | awk '{ print $1 }' | sort | uniq -c | sort -n

xzcat /home/access_log-20161005.xz | awk '{ print $1 }' | sort | uniq -c | sort -n

And my sample curl with a single ip

curl --header "X-Forwarded-For:" ipinfoio

(cant post urls yet)

4. Modesto junior college, Manetca (CA), US, dale Phillips, and CSCI-210 (Link to Course):

Note: Without school/professor/course information, you will be banned if you post here! You must complete the entire template (not just parts of it).
Sponsored Links
Old Unix and Linux 11-02-2016   -   Original Discussion by kjcraig77
balajesuri's Unix or Linux Image
balajesuri balajesuri is offline Forum Advisor  
#! /bin/bash
Join Date: Apr 2009
Last Activity: 18 January 2018, 4:17 AM EST
Location: India
Posts: 1,977
Thanks: 31
Thanked 574 Times in 547 Posts
1. To loop over the files matching a particular pattern in a given directory, you can use the for loop construct. For e.g.,

for file in *.log
    sort file | do_something_else >> newfile ## Append output to newfile. >> is the redirection operator (performs an append)

2. Also utilities like sort, awk (well, technically awk is not a utility, it's a programming language) can take a filename as argument. So it's a useless use of cat.

Let us know your progress and where you're stuck.
The Following User Says Thank You to balajesuri For This Useful Post:
kjcraig77 (11-08-2016)
Sponsored Links

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Linux More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Loop to curl multiple ips to ipinfo.io kjcraig77 Shell Programming and Scripting 1 11-02-2016 03:59 PM
Help with cURL arien001 Shell Programming and Scripting 4 07-28-2011 06:14 PM
use of curl jayawantsandeep Shell Programming and Scripting 0 06-18-2008 11:53 AM
use of curl jayawantsandeep AIX 0 06-16-2008 05:08 AM
curl Euler04 Shell Programming and Scripting 6 07-27-2005 04:16 PM

All times are GMT -4. The time now is 02:10 AM.