FILE_ID extraction from file name and save it in CSV file after looping through each folders
FILE_ID extraction from file name and save it in CSV file after looping through each folders
My files are located in UNIX Server, i want to extract file_id and file_name from each file .and save it in a CSV file. How do I do that?
I have folders in unix environment, directory structure is structured as follows
year folder -> inside 12 months folders -> inside 30/31 days folders
I ran ls command folder
year as follows
2009 2010 2011 2012
I ran cd command for year 2012
I ran ls command for 2012 year folder
then I ran command for september
there are folders for each year like 2009,2010,2011 and 2012
and folder has 12 folders for each months like 01,02,03,04,05,06,07,08,09,10,11,12
and each month folder has 31 folders for days like 1,2,3, etc... 29,30,31
inside each day folder has files..
the file name is as follows,
sasmm_fsbc_durds_id00020532_t20100313192606.dat.trnsfr.gz
sasmm_fsbc_durds_id00020513_t20120913003312.dat.trnsfr.gz
I want to have one csv file and that file needs to have two columns , one is for file_id and
second field is for file name.
to obtain file_id value ,loop through each folders and get file name, then read file name and
get substring between "sasmm_fsbc_durds_id000" and _t and store it in file_id column and store
file name in file_name column.
in above example for file sasmm_fsbc_durds_id00020532_t20100313192606.dat.trnsfr.gz
read file name sasmm_fsbc_durds_id00020532_t20100313192606.dat.trnsfr.gz
cut 20532 and save it in a file_id clumn and the whole file name in second column = sasmm_fsbc_durds_id00020532_t20100313192606.dat
CSV file will look like
file_id is to be cut from the file name , if you look at the file name closely, you can see;
after 000 , file_ids in above file name examples , they are 20532 and 20513.
How do I loop through year 2012 and 12 months folders and 31 days folders inside it and create
csv file which has data as shown above?
I am very new unix, please help me out.. If you provide a code , that would be great..
thanks..
output CSV file look like this
do we need to search files recursively for finding file in each folder or to go dwon to day folder?
Moderator's Comments:
edit by bakunin: Please view this code tag video for how to use code tags when posting code and data.
In addition please do not use all-caps routinely. All-caps is like spice - use it to make SOMETHING STAND OUT, but overdo it and its tasteless.
I have a .csv file and i use the below while loop to navigate through it
But i need to loop from the second line since the first line is the header
How will i do it?? please help
while IFS=, read Filename Path size readonly
do
echo "Filename -> ${Filename}"
echo "Path -> ${Path}"
echo... (8 Replies)
Hi i have wriiten a script which fetches the data from text file, and saves in the output in a text file itself, but i want that the output should save in different columns.
I have the output like:
For Channel:response_time__24.txt
1547 data points
0.339
0.299
0.448
0.581
7.380
... (1 Reply)
Hi
I need to execute a select statement in a solaris environment with oracle database. The select statement returns number of rows of data.
I need the data to be inserted into a CSV file with proper format. For that we normally use "You have to select all your columns as one big string,... (2 Replies)
Hi Unix gurus,
Maybe it is too much to ask for but please take a moment and help me out. A very humble request to you gurus. I'm new to Unix and I have started learning Unix. I have this project which is way to advanced for me.
File format: CSV file
File has four columns with no header... (8 Replies)
Hi,
I have a text file with the following format. Some of the fields are blank.
1234 3456 23 45464 327837283232 343434
5654353 34 34343 3434345 434242
....
....
....
I need to convert this file to a CSV file, like
1234, ,23, ... (3 Replies)
Hi
I am writing a shell script to parse a CSV file , in which i am facing a problem to separate the columns . Could some one help me with it.
IN301330/00001 pvavan kumar limited xyz@ttccpp.com
IN302148/00002 PRECIOUS SECURITIES (P) LTD viash@yahoo.co.in
IN300239/00000 CENTRE india... (8 Replies)
Hi, all
I want to sort a csv file based on timestamp from oldest to newest and save the output as csv file itself. Here is an example of my csv file.
test.csv
SourceFile,DateTimeOriginal
/home/intannf/foto/IMG_0739.JPG,2015:02:17 11:32:21
/home/intannf/foto/IMG_0749.JPG,2015:02:17 11:37:28... (10 Replies)
Hi,
I have another problem. I want to sort another csv file by the first field.
result.csv
SourceFile,Airspeed,GPSLatitude,GPSLongitude,Temperature,Pressure,Altitude,Roll,Pitch,Yaw
/home/intannf/foto5/2015_0313_090651_219.JPG,0.,-7.77223,110.37310,30.75,996.46,148.75,180.94,182.00,63.92 ... (2 Replies)
Hi All,
I have a data file and need to extract and convert it into csv format:
1) Read and extract the line containing string ending with "----" (file sample_linebyline.txt file) and to make a .csv file from this.
2) To read the flat file flatfile_sample.txt which consists of similar data (... (9 Replies)