How do i read only last 5 files records from a directory.?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How do i read only last 5 files records from a directory.?
# 8  
Old 10-27-2017
Quote:
Originally Posted by MadeInGermany
Attention: ls omits the path,
so in the loop you must prepend it to the loop variable like "$path/$file"
Or you do
Code:
for  files in `printf "%s\n" $path/* | tail -5`

--
Yes, the inner loop looks odd - certainly not yet ready...
The odd indentation didn't help in trying to figure out what the inner loop was doing and the fact that the inner loop variable is never referenced is some kind of clue (I think).

I should have just said to cd to the directory before the outer loop and use ls (or printf '%s\n' *) piped through tail which would also allow replacing:
Code:
$(ls -l $files | awk '{print $9}'| cut -d '/' -f5)

with just:
Code:
$files

in the final echo in the outer loop.
This User Gave Thanks to Don Cragun For This Post:
# 9  
Old 10-27-2017
Looking at the overly complicated (and erroneous) script that you posted, making use of the proposals in the handful of your recent threads (none which you neither commented nor thanked, BTW) seemingly pertaining to the same topic - why don't you show the underlying problem and all the input data instead of asking a little question here and a tiny puzzle there? People in here might come up with a consistent proposal that hits the bull's eye.
These 3 Users Gave Thanks to RudiC For This Post:
# 10  
Old 10-27-2017
file have hexadecimal value on column 4 i am selecting that and doing further calculation.

---------- Post updated at 02:24 PM ---------- Previous update was at 02:13 PM ----------

Code:
for f in $files
        do
        #echo $files
        read -r line || [ -n "$line"];
        TTHEX=`awk -F ',' 'END{print $4}'`


above loop reading each files lines and printing the column 4th value into the variable TTHEX

so TTHEX keep assigning the value every time it goes into the loop and outer loop i am calculating value using it.


2nd last column is the value in my output.
Moderator's Comments:
Mod Comment Please use CODE tags when displaying sample input, output, and code segments (as required by the forum rules you agreed to when joining this forum).

Last edited by Don Cragun; 10-27-2017 at 05:08 PM.. Reason: Add CODE tags again.
# 11  
Old 10-27-2017
Quote:
Originally Posted by sadique.manzar
file have hexadecimal value on column 4 i am selecting that and doing further calculation.

---------- Post updated at 02:24 PM ---------- Previous update was at 02:13 PM ----------

Code:
for f in $files
        do
        #echo $files
        read -r line || [ -n "$line"];
        TTHEX=`awk -F ',' 'END{print $4}'`


above loop reading each files lines and printing the column 4th value into the variable TTHEX

so TTHEX keep assigning the value every time it goes into the loop and outer loop i am calculating value using it.


2nd last column is the value in my output.
Moderator's Comments:
Mod Comment Please use CODE tags when displaying sample input, output, and code segments (as required by the forum rules you agreed to when joining this forum).
That is not your complete inner loop. The complete inner loop is:
Code:
for f in $files
        do
        #echo $files
        read -r line || [ -n "$line"];
        TTHEX=`awk -F ',' 'END{print $4}'`
                done < $files

more readably written as:
Code:
for f in $files
do
	#echo $files
	read -r line || [ -n "$line"];
	TTHEX=`awk -F ',' 'END{print $4}'`
done < $files

For demonstration purposes, let me rewrite this loop as:
Code:
for f in $files
do
	echo '$files expands to: '"$files"
	echo '    $f expands to: '"$f"
	read -r line || [ -n "$line"];
	echo ' $line expands to: '"$line"
	TTHEX=`awk -F ',' 'END{print $4}'`
	echo '$TTHEX expands to: '"$TTHEX"
done < "$files"

and let us assume that a file named x.txt contains the five lines:
Code:
header,decimal,octal,hexadecimal
line1,100,144,64
line2,200,310,C8
line3,300,454,12C
line4,400,620,190

If we set files to x.txt and run the modified loop, this is the output we see:
Code:
$files expands to: x.txt
    $f expands to: x.txt
 $line expands to: header,decimal,octal,hexadecimal
$TTHEX expands to: 190

  1. Note that even though there are five lines in the input file, we only go through the loop once; not five times.
  2. Note that the loop control variable ($f) in your loop) is never used in your script.
  3. Note that the contents of the first line in the input file ($line in your loop) is never used in your script.
  4. Note that the contents of the variable that is used in the outer loop ($TTHEX in your code) is set to the contents of the 4th field in the last line of the input file.
Note that if we replace that entire loop with the replacement I suggested for it (and an echo to show what it did):
Code:
		TTHEX=`awk -F ',' 'END{print $4}' $files`
		echo '$TTHEX expands to: '"$TTHEX"

then we get the output:
Code:
$TTHEX expands to: 190

So, I repeat my question from post #6 in this thread:
Quote:
... I must admit that I do not understand why you have the inner for loop in your script??? In what way would the output be different if that entire loop was replaced by:
Code:
		   $files

??? As long as the input file processed by this loop contains at least two lines of text, your loop should produce exactly the same output as the single awk statement I suggested.

If the input file is empty, both loops set TTHEX to an empty string.

If the input file contains a single line (presumably a header similar to the first line in x.txt), your loop sets TTHEX to an empty string while my suggestion sets TTHEX to the contents of the header for the fourth field. Is that an important distinction for the proper behavior of your script? If it is, change my suggestion to:

Code:
		TTHEX=`awk -F ',' 'NR==1{$4=""}END{print $4}' $files`

I assumed that it didn't matter because you don't check whether or not $TTHEX expands to an empty string before passing its value to expr and if $TTHEX does expand to an empty string, bash's built-in printf will fail with an arithmetic syntax error when trying to use %d to format the string 0x.

PS Note that two missing back-quotes have been added to fix my suggested awk scripts (as noted in post #13 by MadeInGermany).

Last edited by Don Cragun; 10-27-2017 at 08:03 PM.. Reason: Add backquotes and postscript.
This User Gave Thanks to Don Cragun For This Post:
# 12  
Old 10-27-2017
Thank you Smilie

I will change my script as suggested and will check.

Thnx everyone, for the support.
# 13  
Old 10-27-2017
And IF the read would make sense, the only purpose of the inner loop is to form a block. The loop variable is unused, the list always contains one word.
A simple block is
Code:
{
	#echo $files
	read -r line || [ -n "$line"];
	TTHEX=`awk -F ',' 'END{print $4}'`
} < $files

Now the purpose of the block becomes obvious: it redirects the input from the file, so both read and awk read from the file.
But then the next question is: what is the || [ -n "$line"]; for??
And then, as Don found out, the awk code reads through all lines, so why steal the first line with read line and do nothing with $line?
And if we can omit the useless read then we can also omit the block, and let awk read the file directly.
BTW Don's sample misses the closing backtick, should be
Code:
TTHEX=`awk -F ',' 'END{print $4}' $files`

Last but not least, the name files(plural) can make you think it holds many files. But it holds only one file (well, a different file in each loop cycle, but always one file). Therefore I vote for file(singular)!
This User Gave Thanks to MadeInGermany For This Post:
# 14  
Old 10-28-2017
Code:
read -r line || [ -n "$line"];

This line should be commented while pasting the code, i did blunder mistake.

Extremely sorry folks.

Regards,
Moderator's Comments:
Mod Comment Please use CODE tags when displaying sample input, output, and code segments as required by the forum rules you agreed to when you joined this forum.

Last edited by Don Cragun; 10-28-2017 at 04:59 PM.. Reason: Add CODE tags again.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to fetch matched records from files between two different directory?

awk 'NR==FNR{arr;next} $0 in arr' /tmp/Data_mismatch.sh /prd/HK/ACCTCARD_20160115.txt edit by bakunin: seems that one CODE-tag got lost somewhere. i corrected that, but please check your posts more carefully. Thank you. (5 Replies)
Discussion started by: suresh_target
5 Replies

2. Shell Programming and Scripting

How to read a particular records from a file?

Hi All Can anybody let me know the code to read a particular record from a file For e.g. File Name: File.txt File content: Script_path=/abc/def/script/ File_path=/xyz/data/ Business Date=19990905 SERVER_NAME=Server DATABASE_NAME=Database Login=NewUser Password=NewPassword ... (3 Replies)
Discussion started by: Siddhartha9833
3 Replies

3. Shell Programming and Scripting

Compare two files with different number of records and output only the Extra records from file1

Hi Freinds , I have 2 files . File 1 |nag|HYd|1|Che |esw|Gun|2|hyd |pra|bhe|3|hyd |omu|hei|4|bnsj |uer|oeri|5|uery File 2 |nag|HYd|1|Che |esw|Gun|2|hyd |uer|oi|3|uery output : (9 Replies)
Discussion started by: i150371485
9 Replies

4. UNIX for Dummies Questions & Answers

Read all files in a directory for a unix command

Hello, Below, I have a unix command, which can be executable for single file. cat input.txt | sort -k3,3 > output.txt I have 100 input files in a directory. It is hectic and time taking to run the above command for all the 100 files for 100 times. Now, I want to execute the above unix... (2 Replies)
Discussion started by: koneru_18
2 Replies

5. Shell Programming and Scripting

How to read and append certain files form directory

Hi ,i have a question ,if I am in sertain directory and I have 4 files *.c how can I read and append this files in file backup.bac Thanks find ./ -name "*.csh" | wc -l (2 Replies)
Discussion started by: lio123
2 Replies

6. UNIX for Dummies Questions & Answers

Read directory files and count number of lines

Hello, I'm trying to create a BASH file that can read all the files in my working directory and tell me how many words and lines are in that file. I wrote the following code: FILES="*" for f in "$FILES" do echo -e `wc -l -w $f` done My issue is that my file is outputting in one... (4 Replies)
Discussion started by: jl487
4 Replies

7. Shell Programming and Scripting

how to read next records

Hello friends, I am newbie in programing. I am facing some problems in awk. Please help me. I have a file with many data sets. Each data set is separated by an empty line. For example Col1 Col2 Col3 Col4 Col5 0.85 0.07 Fre 42:86 25 0.73 0.03 frp 21:10 28 0.64 0.04 Fre 42:86 63 0.47 0.08... (2 Replies)
Discussion started by: ubeejani
2 Replies

8. Shell Programming and Scripting

read files from directory

hi i have a directory /tmp/Satya,it contains 5.FILE 6.FILE 7.FILE i need to read each file , and read its content line by line please help thanks Satya (2 Replies)
Discussion started by: Satyak
2 Replies

9. UNIX for Dummies Questions & Answers

Read Files from a Directory

Hi, I have a requirement where I have get the files from FTP server then delete the files in the FTP server and then store the file name within the file where there is "TR". i.e Filename|TR|20071231|.... Once I finish loading my file I have to archive the files. Is there any way to do it... (2 Replies)
Discussion started by: kiran_418
2 Replies

10. Shell Programming and Scripting

Read from fileList.txt, copy files from directory tree

Hi, hopefully this is a fairly simple Q&A. I have a clean file list of approximately 180 filenames with no directory or slashes in front of the filename nor any extension or dot ".". I would like to read from this list, find these files recursively down through directory trees, copy the files... (1 Reply)
Discussion started by: fxvisions
1 Replies
Login or Register to Ask a Question