Help required with a file rename shell script


Login or Register for Dates, Times and to Reply

 
Thread Tools Search this Thread
# 1  
Help required with a file rename shell script

Hello everyone,

Posting here after a long time, been away from unix world lately and it seems I have forgotten my shell scripting completely. I have a requirement where a csv file contains following columns:

Code:
Full Registration	VIN			Stock ID	Mileage	InternalTrim	Description	Warranty	FranchiseApproved	RegistrationDate	Featured	New	Vehicle Type	Used stock images
2653BA			WDD1173461N6437866	2725		19434	Leather trim			Y		Y			19/09/2018		N		N	CAR	
8874MS			WDD1173461N6494217	2745		15452	Leather trim			Y		Y			19/09/2018		N		N	CAR

This csv file will be sent to a hosting service provider who is hosting our used car website. Every used car has some images and names of those images need to be populated in the last column of this csv file but there is one challenge, image files are stored (in the same directory) as Stock ID_001, Stock ID_002 and so on. From this csv file example, image files will be 002795_001, 002795_002, 002795_003 and same for rest of the vehicles. We don't know in advance how many image files will be there for one stock ID. We need to rename these image files as VIN NO_1, VIN_NO_2 and so on. Therefore for this example, image file names will become WDD1173461N6437866_1, WDD1173461N6437866_2, WDD1173461N6437866_3 because stock ID 2795 has VIN number WDD1173461N6437866.

Once the image files are renamed then we need to populate these names in the last column of above csv file. so the last column of above csv file will become (for the first record). Please note we have to ignore the header and image file names to be populated from 2nd record onwards (after renaming) as explained above.

Code:
Full Registration	VIN			Stock ID	Mileage	InternalTrim	Description	Warranty	FranchiseApproved	RegistrationDate	Featured	New	Vehicle Type	Used stock images
2653BA			WDD1173461N6437866	2725		19434	Leather trim			Y		Y			19/09/2018		N		N	CAR		WDD1173461N6437866_1,WDD1173461N6437866_2,WDD1173461N6437866_3

Can you please help me with this challenge?

Regards,
Tayyab
# 2  
Try
Code:
awk '
NR == 2 {("ls -x *" $3 "*") | getline L
         gsub (/0+/, "", L)
         gsub ($3, $2, L)
         gsub (/[ 	]+/, ",", L)
         $0 = $0 "\t" L}
1
  ' file
Full Registration	VIN			Stock ID	Mileage	InternalTrim	Description	Warranty	FranchiseApproved	RegistrationDate	Featured	New	Vehicle Type	Used stock images
2653BA			WDD1173461N6437866	2795		19434	Leather trim			Y		Y			19/09/2018		N		N	CAR		WDD1173461N6437866_1,WDD1173461N6437866_2,WDD1173461N6437866_3
8874MS			WDD1173461N6494217	2745		15452	Leather trim			Y		Y			19/09/2018		N		N	CAR

Be aware there's no 2095 Stock ID in your sample csv file.
This User Gave Thanks to RudiC For This Post:
# 3  
Hi RudiC,

thanks for your quick reply. I'll try that awk script at my end and come back. Sorry for confusing with the wrong stock ID number.

Regards,
Tayyab
# 4  
Hi RudiC,

Sorry for the confusion and providing the wrong file format and image file names, here is the sample from csv file:

Code:
tayyab@c549:~$ cat test.csv
Full Registration,VIN,Stock ID,Mileage,InternalTrim,Description,Warranty,FranchiseApproved,RegistrationDate,Featured,NewVehicleType,Used stock images
2653BA,WDD1173461N6437866,2795,19434,Leather trim,A Class,Y,Y,19/09/2018,N,CAR,
8874MS,WDD1173461N6494217,2745,15452,Leather trim,A Class,Y,Y,19/09/2018,N,CAR,

And here is the list of images:

Code:
tayyab@c549:~$ ls -ltr 00*
-rw-rw-rw- 1 tayyab tayyab 0 Jun 25 17:59 002795_001.jpg
-rw-rw-rw- 1 tayyab tayyab 0 Jun 25 17:59 002795_002.jpg
-rw-rw-rw- 1 tayyab tayyab 0 Jun 25 18:00 002795_003.jpg
-rw-rw-rw- 1 tayyab tayyab 0 Jun 25 18:27 002745_001.jpg
-rw-rw-rw- 1 tayyab tayyab 0 Jun 25 18:27 002745_002.jpg

Once I ran the shell script, i want test.csv file to be like:
Code:
tayyab@c549:~$ cat test.csv
Full Registration,VIN,Stock ID,Mileage,InternalTrim,Description,Warranty,FranchiseApproved,RegistrationDate,Featured,NewVehicleType,Used stock images
2653BA,WDD1173461N6437866,2795,19434,Leather trim,A Class,Y,Y,19/09/2018,N,CAR,WDD1173461N6437866_1, WDD1173461N6437866_2,WDD1173461N6437866_3
8874MS,WDD1173461N6494217,2745,15452,Leather trim,A Class,Y,Y,19/09/2018,N,CAR,WDD1173461N6494217_1,WDD1173461N6494217_2

And also file names to be renamed to as follows:

Code:
WDD1173461N6437866_1 
WDD1173461N6437866_2
WDD1173461N6437866_3
WDD1173461N6494217_1
WDD1173461N6494217_2

there have to be two steps

i) Look for stock ID within the stock feed csv file and rename all the images (replace stock ID with VIN code)
ii) Once the files are renamed then append those JPG images (with VIN# and _1, _2) to the last column of the feed file.


Your code didn't work out for me if you or someone else can provide me with a solution to the above challenge that'll be great. Thanks.

Regards,
Tayyab
# 5  
Here is a standard shell script. It does not remove the .jpg extensions.
It only echoes things, so you get an idea what it could do.
Code:
#!/bin/sh
while IFS="," read f_reg vin s_id rest
do
  # do not process header and junk
  out=""
  if [ "$vin" != "VIN" ] && [ -n "$vin" ] && [ -n "$s_id" ]
  then 
    sep=""
    for i in `ls | grep "^0*${s_id}_"`
    do
      # rename from $s_id to $vin, delete leading 0 characters
      new_i=`echo "$i" | sed "s/^0*${s_id}_0*/${vin}_/"`
      echo mv "$i" "$new_i"
      out="${out}${sep}${new_i}"
      sep=" "
    done
  fi
  # delete a trailing comma before adding a new one as separator
  echo "$f_reg,$vin,$s_id,${rest%,}${out:+,$out}"
done < test.csv

This User Gave Thanks to MadeInGermany For This Post:
# 6  
Here is another example you could try, all in bash shell without external utilities or subshells except for the mv command, so it should be reasonably quick.

Code:
#!/bin/bash
oldIFS=$IFS
while IFS="," read -a car; do
  if [[ ${car[1]} != VIN ]]; then
    for file in *"${car[2]}_"*.jpg
    do
      if [ -f "$file" ]; then
        tofile=${car[1]}${file#*"${car[2]}"}
        echo mv -- "$file" "$tofile" 
        car[${#car[@]}]=$tofile
      fi
    done
  fi
  IFS=","
  printf "%s\n" "${car[*]}"
  IFS=$oldIFS
done < test.csv

It also does not entirely produce the proper output format:
Code:
mv -- 002795_001.jpg WDD1173461N6437866_001.jpg

, but it should give you some idea..

Remove the echo statement when it does what you want...

Last edited by Scrutinizer; 06-25-2019 at 04:50 PM..
This User Gave Thanks to Scrutinizer For This Post:
# 7  
Quote:
Originally Posted by tayyabq8
...


Your code didn't work out for me

...
No surprise. Your post #1 input sample provided the wrong structure, contained wrong data, and the sample file names didn't reflect reality either. What would you expect? With your "revised" data and input, try
Code:
awk '
NR >= 2 {("ls -x *" $3 "*.jpg") | getline L
         gsub (/[       ]+/, ",", L)
         LB = L
         gsub (/0+|.jpg/, "", L)
         gsub ($3, $2, L)
         $0 = $0  L
              split (LB, T1)
         n  = split (L,  T2)
         for (; n; n--) OUT = OUT sprintf ("echo mv -- %s %s\n", T1[n], T2[n] ".jpg")
        }
1
END     {system (OUT)
        }
' FS=, file
Full Registration,VIN,Stock ID,Mileage,InternalTrim,Description,Warranty,FranchiseApproved,RegistrationDate,Featured,NewVehicleType,Used stock images
2653BA,WDD1173461N6437866,2795,19434,Leather trim,A Class,Y,Y,19/09/2018,N,CAR,WDD1173461N6437866_1,WDD1173461N6437866_2,WDD1173461N6437866_3
8874MS,WDD1173461N6494217,2745,15452,Leather trim,A Class,Y,Y,19/09/2018,N,CAR,WDD1173461N6494217_1,WDD1173461N6494217_2
mv -- 002795_003.jpg WDD1173461N6437866_3.jpg
mv -- 002795_002.jpg WDD1173461N6437866_2.jpg
mv -- 002795_001.jpg WDD1173461N6437866_1.jpg
mv -- 002745_002.jpg WDD1173461N6494217_2.jpg
mv -- 002745_001.jpg WDD1173461N6494217_1.jpg


The first three lines are your target .csv file printed to stdout. The remaining 6 lines are the test output from the system call; remove the echo from the sprintf command to really have the files renamed.




EDIT: Slight simplification:



Code:
awk '
NR >= 2 {CMD = "ls *" $3 "*.jpg"
         while (1 == CMD | getline FN)
                {FNN = FN
                 gsub (/0+|.jpg/, "", FNN)
                 gsub ($3, $2, FNN)
                 $0 = $0 FS FNN
                 OUT = OUT "echo mv -- " FN " " FNN ".jpg\n"
                }
         close (CMD)
        }
1
END     {system (OUT)
        }
' FS=, file

This User Gave Thanks to RudiC For This Post:
Login or Register for Dates, Times and to Reply

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell script is required

Dear All I have a filelisting as below: abcd_20110715_0007 abcd_20110715_0010 abcd_20110716_0001 abcd_20110716_0004 abcd_20110715_0008 abcd_20110715_0011 abcd_20110716_0002 abcd_20110716_0005 abcd_20110715_0009 abcd_20110715_0012 abcd_20110716_0003 abcd_20110716_0006 ... (3 Replies)
Discussion started by: at1700
3 Replies

2. Shell Programming and Scripting

write shell script to rename file

hi, I need some help in writing shell script in a bourne shell.I am trying to rename the file.. eg. find /root/data -type f -name "text*) | while read FILES do newfile=${FILES/type_2.0_20101208_34.xml / tmp.xml} mv "$FILES" "$newfile" done above written script is working...If the... (7 Replies)
Discussion started by: shubhig15
7 Replies

3. UNIX for Advanced & Expert Users

Rename a file to a file_current datetime in a shell script

Hi all, Could anyone suggest me on Renaming a file to a file_current datetime in a shell script. (3 Replies)
Discussion started by: Nithin
3 Replies

4. UNIX for Dummies Questions & Answers

Shell script to rename or change file extension case.

I searched the forum, but there was different type of rename. Hello. I have files in folder. Like: xxxxxxxx1.html or xxxxxxxx2.txt or xxxxxxxx3.tar.gz and how to rename or change file extension case to xxxxxxxx1.htm or xxxxxxx2.TXT or (5 Replies)
Discussion started by: Sheldon
5 Replies

5. Shell Programming and Scripting

shell script required...

There are two fields actually one is server name and the other one is Time. Based on time, there are 8 columns and these will be updated with the flag 1 if at all if there is any server name. Time Server name 15 to 18 18 to 21 21 to 24 00 to 03 03 to 06 06 to 09 09 to 12 Server... (3 Replies)
Discussion started by: venkatesht
3 Replies

6. Shell Programming and Scripting

Shell script help required

Hi, Can someone help me with this small piece of code. DIRNAME=$(dirname $0) BASENAME=$(basename $0) DATA="${DIRNAME}/${BASENAME}.data" && . $DATA whats is meant by && . $DATA here... Regards, Abhishek (2 Replies)
Discussion started by: max29583
2 Replies

7. Shell Programming and Scripting

Shell Script Required!

Hi people, I am new to this forum. I have taken unix this semester in my college and i am new to it. I am finding shell scripting a bit hard and i need a little help. I require a shell script to delete files that end as .bak , .BAK, #, ~ and files with the name core.The Script should accept... (3 Replies)
Discussion started by: vats
3 Replies

8. Linux

shell script required

Hi, iam presenting the input text file format.Of this i need the character count of the number of characters present in each file.The attached file is a combination of 3 text file.each text file starts at record 1 - 34, then the next tetx file starts. What i need is the character count of each... (1 Reply)
Discussion started by: sethunath
1 Replies

9. Shell Programming and Scripting

shell script required

hi , i need a shell script that will remove the first and second lines of the text file and will list the word count of the characters present in it. the text file will be consisting of multiple textfiles.the first text file starts from 01-34.like wise the next file also starts from 01-34... (4 Replies)
Discussion started by: sethunath
4 Replies

10. Shell Programming and Scripting

Urgent help required in deleting a line without opening a file usinga shell script

Hi, I need a help in deleting a line matching a particular pattern in a file using shell script without opening the file. The file is a .c/.cpp file. Is it possible? Thanks (6 Replies)
Discussion started by: naan
6 Replies

Featured Tech Videos