Difficulty to create .txt file from loop in bash


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Difficulty to create .txt file from loop in bash
# 1  
Old 03-05-2020
Difficulty to create .txt file from loop in bash

I've this data :

Code:
data1.txt

2020-01-27-06-00;/dev/hd1;100;/
2020-01-27-12-00;/dev/hd1;100;/
2020-01-27-18-00;/dev/hd1;100;/
2020-01-27-06-00;/dev/hd2;200;/usr
2020-01-27-12-00;/dev/hd2;200;/usr
2020-01-27-18-00;/dev/hd2;200;/usr

 data2.txt 

2020-02-27-06-00;/dev/hd1;120;/
2020-02-27-12-00;/dev/hd1;120;/
2020-02-27-18-00;/dev/hd1;120;/
2020-02-27-06-00;/dev/hd2;230;/usr
2020-02-27-12-00;/dev/hd2;230;/usr
2020-02-27-18-00;/dev/hd2;230;/usr

 data3.txt 

2020-03-27-06-00;/dev/hd1;130;/
2020-03-27-12-00;/dev/hd1;130;/
2020-03-27-18-00;/dev/hd1;130;/
2020-03-27-06-00;/dev/hd2;240;/usr
2020-03-27-12-00;/dev/hd2;240;/usr
 2020-03-27-18-00;/dev/hd2;240;/usr

I would like to create a .txt file for each filesystem ( so hd1.txt, hd2.txt, hd3.txt and hd4.txt ) and put in each .txt file the sum of the value from each FS from each dataX.txt. I've some difficulties to explain in english what I want, so here an example of the result wanted

Expected content for the output file `hd1.txt`:
Code:
2020-01;/dev/hd1;300;/
2020-02;/dev/hd1;360;/
2020-03;/dev/hd1;390:/

Expected content for the file `hd2.txt`:


Code:
2020-01;/dev/hd2;600;/usr
2020-02;/dev/hd2;690;/usr
2020-03;/dev/hd2;720;/usr

The implementation I've currently tried:

Code:
for i in $(cat *.txt | awk -F';' '{print $2}' | cut -d '/' -f3| uniq)
do
    cat *.txt | grep -w $i | awk -F';' -v date="$(cat *.txt | awk -F';' '{print $1}' | cut -d'-' -f-2 | uniq )" '{sum+=$3} END {print date";"$2";"sum}' >> $i

done

But it doesn't works...

Can you show me how to do that ?
# 2  
Old 03-05-2020
"But it doesn't works..." is not something people can start working / analysing / debugging upon. Be more precise / descriptive; include error messages, warnings, non-satisfying output, etc.



Try (assuming mount points are unique)

Code:
awk -F\; '
        {MTH = substr($1,1,7)
         SUM[MTH,$2] += $3
         MNT[MTH,$2]   = $4
        }
END     {for (s in SUM)         {n = split (s, T, "[;/]")
                                 print s, SUM[s], MNT[s]  >  (T[n] ".txt")
                                }
        }
' OFS=\; SUBSEP=\; file[123]
cf *.txt

---------- hd1.txt: ----------

2020-03;/dev/hd1;390;/
2020-02;/dev/hd1;360;/
2020-01;/dev/hd1;300;/

---------- hd2.txt: ----------

2020-03;/dev/hd2;720;/usr
2020-02;/dev/hd2;690;/usr
2020-01;/dev/hd2;600;/usr

# 3  
Old 03-06-2020
Hello !

Thanks for your help !

I've try many different things in bash and my last script that most closely resembled at what I want take much more time ( 3minutes ) to do what your script do instantly. So thanks for that ! Smilie

Last things, if I want to specify the path where are my data, I've just to do that ? :


Code:
' OFS=\; SUBSEP=\; /my/full/path/*.txt




And, If I understand how to change for the loclisation of my data, at this step :

Code:
END     {for (s in SUM)         {n = split (s, T, "[;/]")                                  print s, SUM[s], MNT[s]  >  (T[n] ".txt")                                 }         }

How can I specify the path where I want to put the result of the script ?

Thanks again !
# 4  
Old 03-06-2020
1) Yes
2) How do you want to convey the target path to the script? In case you want to use the source files' path, and this is constant, to place the hd*.txt files next to the sources, try (in the END section)

Code:
END     {PTH = FILENAME
         sub (/[^\/]*$/, "", PTH)
         for (s in SUM)         {n = split (s, T, "[;/]")
                                 print s, SUM[s], MNT[s] " > " (PTH T[n] ".txt")
                                }
         }

assuming the FILENAME variable is still defined and valid in that section.


Else you can use awk's -v mechanism to supply the path.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Copy the content from txt file and create a html file

I have a txt file with a list of error messages in a xml tag format, and each error message is separated with a identifier(endresult).Need to split that and copy and create a new html file.Error message has some special character. how to escape the special character and insert my data into the... (7 Replies)
Discussion started by: DevAakash
7 Replies

2. Shell Programming and Scripting

awk to create variables to pass into a bash loop to create a download link

I have created one file that contains all the necessary info in it to create a download link. In each of the lines /results/analysis/output/Home/Auto_user_S5-00580-6-Medexome_67_032/plugin_out/FileExporter_out.67... (8 Replies)
Discussion started by: cmccabe
8 Replies

3. Shell Programming and Scripting

Bash/awk and for loop to create a template

Source File: google.cz http://czechrepublic.google.com/ http://czechrepublic.google.cz http://czechrepublic.google.com/ http://brno.google.cz http://brno.google.com/ Fail Code root@arisvm ~/g] $ cat trya rm -f ss for i in a.txt do #b=`cat $i|awk '{print $1}'` #c=`cat $i|awk '{print... (4 Replies)
Discussion started by: invinzin21
4 Replies

4. UNIX for Dummies Questions & Answers

How to create a .csv file from 2 different .txt files?

Hi, I need to create a .csv file from information that i have in two different tab delimited .txt file. I just want to select some of the columns of each .txt file and paste them into a .cvs file. My files look like: File 1 transcript_id Seq. Description Seq. Length ... (2 Replies)
Discussion started by: alisrpp
2 Replies

5. Shell Programming and Scripting

How to create .txt file by using function?

My Calling script is like below: for file in `echo $LIST_OF_FILES` --listing filenames eg, xyz_meta_20110501_00000789.tar do file_name=`basename $file` <call a function to create .txt file in below format> done Want to generate a .txt file that contains below data in ksh: ... (3 Replies)
Discussion started by: vedanta
3 Replies

6. Shell Programming and Scripting

create txt file form data file and add some line on it

Hi Guys, I have file A.txt File A Data AK1521 AK2536 AK3164 I want create text file of all data above and write some data on each file. want Output on below folder /home/kka/out AK1521.txt Hi Welocme (3 Replies)
Discussion started by: asavaliya
3 Replies

7. Shell Programming and Scripting

bash script to create txt files based on timestamp

Hi , please guide me for a bash script that will create a txt files and the name of the txt files will be as of timestamp so that each file name will be different from other and these files will be get created say after every 10 minutes in a folder(/home/p2000/sxs137), please guide me how would... (1 Reply)
Discussion started by: nks342
1 Replies

8. Shell Programming and Scripting

Loop difficulty

hi all I am new to unix and want to create a loop to repeat the loop and before that it ask me to do so.I know "while" may help but I put it in my work and getting stuk with it.any help appreciated. (13 Replies)
Discussion started by: samsami1971
13 Replies

9. Shell Programming and Scripting

how to create file.txt and add current date in file content

Hey guy, how to make bash script to create foo.txt file and add current date into file content and that file always append. example: today the script run and add today date into content foo.txt and tomorrow the script will run and add tomorrow date in content foo.txt without remove today... (3 Replies)
Discussion started by: chenboly
3 Replies

10. UNIX for Dummies Questions & Answers

create file .txt

hi.i want to create a bash script called countfiles.sh, that will count the pack of files under the current file that their term satisfy a specific pattern. The script must show the total per file or on the whole. thank tou (4 Replies)
Discussion started by: battaglia
4 Replies
Login or Register to Ask a Question