AWK Duplicate lines multiple times based on a calculated value


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting AWK Duplicate lines multiple times based on a calculated value
# 1  
Old 06-28-2011
AWK Duplicate lines multiple times based on a calculated value

Hi,

I'm trying to create an XML sitemap of our dynamic ecommerce sites SEO Friendly URLs and am trying to create the initial page listing.

I have a CSV file that looks like the following and need duplicate the lines based on a value which needs calculating.

domain.com/categories/shopping/shoes/,197
domain.com/categories/shopping/shorts/,58
domain.com/categories/shopping/ties/,5

Where 197, 58 and 5 are the number of products. There are 24 products per page so the domain.com/categories/shopping/shoes/,197 has 8 pages which would be.

domain.com/categories/shopping/shoes/
domain.com/categories/shopping/shoes(2)/
domain.com/categories/shopping/shoes(3)/

domain.com/categories/shopping/ties/,5 only has one page so would be
domain.com/categories/shopping/ties/

If anyone has any idea how todo this or has a sample script I'd be most grateful.

I'm hoping to do this in purely in AWK so I can eventually run it automatically on a windows server.

Many Thanks!

James
# 2  
Old 06-28-2011
somethin like this:

Code:
 
awk -F"," '{c=int($2/24);a="";if(c==0){a=$1;}else{for(i=1;i<=c;i++) a==""?a=$1"("i")":a=a"\n"$1"("i")" }} { print a}' input_file |sed 's!\(.*\)/\(.*\)!\1\2/!
'

# 3  
Old 06-28-2011
A possible solution :
Code:
awk -F, '
   {
      c = int(($2+23)/24);
      i = 1;
      fmt = "";
      while (c--) {
         printf "%s" fmt "\n", $1, i++;
         fmt = "(%d)"
      }
   }
' inputfile.csv

inputfile.csv :
Code:
domain.com/categories/shopping/shoes/,197
domain.com/categories/shopping/shorts/,58
domain.com/categories/shopping/ties/,5

Output :
Code:
domain.com/categories/shopping/shoes/
domain.com/categories/shopping/shoes/(2)
domain.com/categories/shopping/shoes/(3)
domain.com/categories/shopping/shoes/(4)
domain.com/categories/shopping/shoes/(5)
domain.com/categories/shopping/shoes/(6)
domain.com/categories/shopping/shoes/(7)
domain.com/categories/shopping/shoes/(8)
domain.com/categories/shopping/shoes/(9)
domain.com/categories/shopping/shorts/
domain.com/categories/shopping/shorts/(2)
domain.com/categories/shopping/shorts/(3)
domain.com/categories/shopping/ties/

Jean-Pierre.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove duplicate lines which has been repeated 4 times

Remove duplicate lines which has been repeated 4 times attached test.txt below command tried and not getting expect output. for i in `cat test.txt | uniq` do num=`cat test.txt | grep $i | wc -l` echo $i $num done test.txt ... (17 Replies)
Discussion started by: Kalia
17 Replies

2. Shell Programming and Scripting

awk joining multiple lines based on field count

Hi Folks, I have a file with fields as follows which has last field in multiple lines. I would like to combine a line which has three fields with single field line for as shown in expected output. Please help. INPUT hname01 windows appnamec1eda_p1, ... (5 Replies)
Discussion started by: shunya
5 Replies

3. Shell Programming and Scripting

Awk: Combine multiple lines based on number of fields

If a file has following kind of data, comma delimited 1,2,3,4 1 1 1,2,3,4 1,2 2 2,3,4 My required output must have only 4 columns with comma delimited 1,2,3,4 111,2,3,4 1,222,3,4 I have tried many awk command using ORS="" but couldnt progress (10 Replies)
Discussion started by: mdkm
10 Replies

4. Shell Programming and Scripting

Remove duplicate lines from file based on fields

Dear community, I have to remove duplicate lines from a file contains a very big ammount of rows (milions?) based on 1st and 3rd columns The data are like this: Region 23/11/2014 09:11:36 41752 Medio 23/11/2014 03:11:38 4132 Info 23/11/2014 05:11:09 4323... (2 Replies)
Discussion started by: Lord Spectre
2 Replies

5. UNIX for Dummies Questions & Answers

awk solution to duplicate lines based on column

Hi experts, I have a tab-delimited file with one column containing values separated by a comma. I wish to duplicate the entire line for every value in that comma-delimited field. For example: $cat file 4444 4444 4444 4444 9990 2222,7777 6666 2222 ... (3 Replies)
Discussion started by: torchij
3 Replies

6. Shell Programming and Scripting

awk to find lines containing word that occur multiple times

i have a script that scans a log file every 10 minutes. this script remembers the last line of the log and then uses it to continue monitoring the log when it runs again 10 minutes later. the script searches the log for a string called MaxClients. now, how can i make it so that when the... (7 Replies)
Discussion started by: SkySmart
7 Replies

7. UNIX for Dummies Questions & Answers

Extracting data between specific lines, multiple times

I need help extracting specific lines in a text file. The file looks like this: POSITION TOTAL-FORCE (eV/Angst) ----------------------------------------------------------------------------------- 1.86126 1.86973 1.86972 ... (14 Replies)
Discussion started by: captainalright
14 Replies

8. Shell Programming and Scripting

Scripting to Duplicate Lines Based on Variable

Greeting all! I could use some assistance please. :) I've been searching for the best way to duplicate a line based on a variable in the next line. Sample Data: Nov 22 00:00:19 10.10.10.1 "%ASA-4-313005: No matching connection for ICMP error message: icmp src Outside:1.2.3.4 dst... (3 Replies)
Discussion started by: sjrupp
3 Replies

9. Shell Programming and Scripting

Sed or Awk for lines between two strings multiple times and keep the last one

Hi, I am trying to get lines between the last occurrences of two patterns. I have files that have several occurrences of “Standard” and “Visual”. I will like to get the lines between “Standard” and “Visual” but I only want to retain only the last one e.g. Standard Some words Some words Some... (4 Replies)
Discussion started by: damanidada
4 Replies

10. Shell Programming and Scripting

Finding duplicate lines and deleting folders based on them

Hi, I have research data, which is organized to 100 folders numbered 00-99. I have many sets of 100 folders, for different values of initial parameters. For some reason, the computer that ran the program to gather the data, didn't always create a unique seed for each folder. I anticipated that... (1 Reply)
Discussion started by: Jopi
1 Replies
Login or Register to Ask a Question