FILE_ID extraction from file name and save it in CSV file after looping through each folders


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting FILE_ID extraction from file name and save it in CSV file after looping through each folders
# 15  
Old 09-20-2012
In a few folders, the files are saved with file names as given below [ends with .dat] , in this case I do not need to cut off .trnsfr.gz and also I want to extract FILE_IDs from these files along with other files whose extension end with .dat.trnsfr.gz

There is a possibility of finding files that ends with following extentions, I want to extract file_id and file_name as you provided in your previous post.

1. .dat.trnsfr.gz
2. .dat
3. .aud.trnsfr
4. .aud.trnsfr.gz

I do not want to consider files that ends with .aud.trnsfr.gz and aud.trnsfr, but I want to extract file_id and file_names from the files whose file names end with .dat.trnsfr.gz and .dat

In case of files end with .dat, i do not want to cut of its tail end since it has the file name that I required. But in case files that end with .dat.trnsfr.gz , i need to cut off .trnsfr.gz

The format will be as given in the above post.

in 2012->04 ->08 folder contains files like this

Code:
 
sasmm_fsbc_durds_id00016763_t20120408230850.aud.trnsfr.gz
sasmm_fsbc_durds_id00016763_t20120408230850.dat.trnsfr.gz


but in 2012->04->01 folder contains files like this

Code:
 
sasmm_fsbc_durds_id00016596_t20120401231148.aud.trnsfr
sasmm_fsbc_durds_id00016596_t20120401231148.dat

sasmm_fsbc_durds_id00016573_t20120401000754.dat

can you please help me out? can you give end date as 20 sept 2012 and restrics data within the range of Jan 26 2012 to Sept 20 , 2012 as well?

Thanks...
# 16  
Old 09-21-2012
Quote:
Originally Posted by princetd001
In a few folders, the files are saved with file names as given below [ends with .dat] , in this case I do not need to cut off .trnsfr.gz and also I want to extract FILE_IDs from these files along with other files whose extension end with .dat.trnsfr.gz

There is a possibility of finding files that ends with following extentions, I want to extract file_id and file_name as you provided in your previous post.

1. .dat.trnsfr.gz
2. .dat
3. .aud.trnsfr
4. .aud.trnsfr.gz

I do not want to consider files that ends with .aud.trnsfr.gz and aud.trnsfr, but I want to extract file_id and file_names from the files whose file names end with .dat.trnsfr.gz and .dat

In case of files end with .dat, i do not want to cut of its tail end since it has the file name that I required. But in case files that end with .dat.trnsfr.gz , i need to cut off .trnsfr.gz

The format will be as given in the above post.

in 2012->04 ->08 folder contains files like this

Code:
 
sasmm_fsbc_durds_id00016763_t20120408230850.aud.trnsfr.gz
sasmm_fsbc_durds_id00016763_t20120408230850.dat.trnsfr.gz


but in 2012->04->01 folder contains files like this

Code:
 
sasmm_fsbc_durds_id00016596_t20120401231148.aud.trnsfr
sasmm_fsbc_durds_id00016596_t20120401231148.dat

sasmm_fsbc_durds_id00016573_t20120401000754.dat

can you please help me out? can you give end date as 20 sept 2012 and restrics data within the range of Jan 26 2012 to Sept 20 , 2012 as well?

Thanks...
So, if you change the same line I told you to change last time from:
Code:
find 2012/01/2[6-9] 2012/01/3[01] 2012/0[2-9] -name 'sasmm_fsbc_durds_id000[0-9]*_t?*.dat.trnsfr.gz' | while read path

to:
Code:
find 2012/01/2[6-9] 2012/01/3[01] 2012/0[2-8] 2012/09/[01][0-9] 2012/09/20 -name 'sasmm_fsbc_durds_id000[0-9]*_t?*.dat*' | while read path

it should do what you want. I would have expected that you'd be able to make a simple change like this yourself given the suggestions you've seen in past posts on this thread.
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Data extraction and converting into .csv file.

Hi All, I have a data file and need to extract and convert it into csv format: 1) Read and extract the line containing string ending with "----" (file sample_linebyline.txt file) and to make a .csv file from this. 2) To read the flat file flatfile_sample.txt which consists of similar data (... (9 Replies)
Discussion started by: abhi_123
9 Replies

2. Shell Programming and Scripting

Save output of updated csv file as csv file itself, part 2

Hi, I have another problem. I want to sort another csv file by the first field. result.csv SourceFile,Airspeed,GPSLatitude,GPSLongitude,Temperature,Pressure,Altitude,Roll,Pitch,Yaw /home/intannf/foto5/2015_0313_090651_219.JPG,0.,-7.77223,110.37310,30.75,996.46,148.75,180.94,182.00,63.92 ... (2 Replies)
Discussion started by: refrain
2 Replies

3. Shell Programming and Scripting

Save output of updated csv file as csv file itself

Hi, all I want to sort a csv file based on timestamp from oldest to newest and save the output as csv file itself. Here is an example of my csv file. test.csv SourceFile,DateTimeOriginal /home/intannf/foto/IMG_0739.JPG,2015:02:17 11:32:21 /home/intannf/foto/IMG_0749.JPG,2015:02:17 11:37:28... (10 Replies)
Discussion started by: refrain
10 Replies

4. Shell Programming and Scripting

CSV file data extraction

Hi I am writing a shell script to parse a CSV file , in which i am facing a problem to separate the columns . Could some one help me with it. IN301330/00001 pvavan kumar limited xyz@ttccpp.com IN302148/00002 PRECIOUS SECURITIES (P) LTD viash@yahoo.co.in IN300239/00000 CENTRE india... (8 Replies)
Discussion started by: nanduri
8 Replies

5. Shell Programming and Scripting

need to save the space when converting to CSV file

Hi, I have a text file with the following format. Some of the fields are blank. 1234 3456 23 45464 327837283232 343434 5654353 34 34343 3434345 434242 .... .... .... I need to convert this file to a CSV file, like 1234, ,23, ... (3 Replies)
Discussion started by: wintersnow2011
3 Replies

6. UNIX for Dummies Questions & Answers

CSV file:Find duplicates, save original and duplicate records in a new file

Hi Unix gurus, Maybe it is too much to ask for but please take a moment and help me out. A very humble request to you gurus. I'm new to Unix and I have started learning Unix. I have this project which is way to advanced for me. File format: CSV file File has four columns with no header... (8 Replies)
Discussion started by: arvindosu
8 Replies

7. Shell Programming and Scripting

select data from oracle table and save the output as csv file

Hi I need to execute a select statement in a solaris environment with oracle database. The select statement returns number of rows of data. I need the data to be inserted into a CSV file with proper format. For that we normally use "You have to select all your columns as one big string,... (2 Replies)
Discussion started by: rdhanek
2 Replies

8. Shell Programming and Scripting

Data fetched from text file and save in a csv file

Hi i have wriiten a script which fetches the data from text file, and saves in the output in a text file itself, but i want that the output should save in different columns. I have the output like: For Channel:response_time__24.txt 1547 data points 0.339 0.299 0.448 0.581 7.380 ... (1 Reply)
Discussion started by: rohitkalia
1 Replies

9. Shell Programming and Scripting

how to start looping from the second line in .csv file

I have a .csv file and i use the below while loop to navigate through it But i need to loop from the second line since the first line is the header How will i do it?? please help while IFS=, read Filename Path size readonly do echo "Filename -> ${Filename}" echo "Path -> ${Path}" echo... (8 Replies)
Discussion started by: codeman007
8 Replies
Login or Register to Ask a Question