Unix/Linux Go Back    

Homework & Coursework Questions Students must use and complete the template provided. If you don't, your post may be deleted! Special homework rules apply here.

Curl Loop help

Homework & Coursework Questions

Thread Tools Search this Thread Display Modes
Old Unix and Linux 11-02-2016
kjcraig77 kjcraig77 is offline
Registered User
Join Date: Nov 2016
Last Activity: 9 November 2016, 10:58 AM EST
Posts: 5
Thanks: 1
Thanked 0 Times in 0 Posts
Curl Loop help

1. The problem statement, all variables and given/known data:
Here is my assignment.
Parse the /home/access_log and /home/access_.xz files
Create a sorted list of ip address with no duplicates
Pass each ip address to ipinfo.io with curl
Create a logfile that has the ip address,lat,long
Save the data file and script to the inclass dir
Submit your script to canvas email_first_part_lab8.txt

I am trying to figure out how to create a loop to cat all the access logs then curl all the ips in that log to the site ive been provided and then redirect that ouput to a text file. I am still rather ignorant to shell scripting.

2. The attempts at a solution (include all code and scripts):

Here are my scripts to cat the files

cat /home/access_log | awk '{ print $1 }' | sort | uniq -c | sort -n

xzcat /home/access_log-20160917.xz | awk '{ print $1 }' | sort | uniq -c | sort -n

xzcat /home/access_log-20161005.xz | awk '{ print $1 }' | sort | uniq -c | sort -n

And my sample curl with a single ip

curl --header "X-Forwarded-For:" ipinfoio

(cant post urls yet)

4. Modesto junior college, Manetca (CA), US, dale Phillips, and CSCI-210 (Link to Course):

Note: Without school/professor/course information, you will be banned if you post here! You must complete the entire template (not just parts of it).
Sponsored Links
Old Unix and Linux 11-02-2016
balajesuri's Unix or Linux Image
balajesuri balajesuri is offline Forum Advisor  
#! /bin/bash
Join Date: Apr 2009
Last Activity: 11 April 2017, 6:48 AM EDT
Location: India
Posts: 1,973
Thanks: 30
Thanked 572 Times in 546 Posts
1. To loop over the files matching a particular pattern in a given directory, you can use the for loop construct. For e.g.,

for file in *.log
    sort file | do_something_else >> newfile ## Append output to newfile. >> is the redirection operator (performs an append)

2. Also utilities like sort, awk (well, technically awk is not a utility, it's a programming language) can take a filename as argument. So it's a useless use of cat.

Let us know your progress and where you're stuck.
The Following User Says Thank You to balajesuri For This Useful Post:
kjcraig77 (11-08-2016)
Sponsored Links

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Linux More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Loop to curl multiple ips to ipinfo.io kjcraig77 Shell Programming and Scripting 1 11-02-2016 02:59 PM
Help with cURL arien001 Shell Programming and Scripting 4 07-28-2011 05:14 PM
use of curl jayawantsandeep Shell Programming and Scripting 0 06-18-2008 10:53 AM
use of curl jayawantsandeep AIX 0 06-16-2008 04:08 AM
curl Euler04 Shell Programming and Scripting 6 07-27-2005 03:16 PM

All times are GMT -4. The time now is 01:23 AM.