Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Save files in directory as txt

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Save files in directory as txt
# 1  
Old 12-12-2014
Save files in directory as txt
Code:
 wget -x -i link.txt

The above downloads and create unique entries for the 97 links in the text file. However, each new file is saved as CM080 with a FILE extention. Is there a way to convert each file in that directory to a .txt? The 97 files are in C:\Users\cmccabe\Desktop\list\geneticslab.emory.edu\tests.

Thank you Smilie.
link.txt (3.9 KB) 
Last edited by rbatte1; 12-12-2014 at 11:30 AM.. Reason: Removed some CODE tags to make general text more readable
# 2  
Old 12-12-2014
Afterwards, you could run a rename in a loop. Assuming that the directory only contains the files you want, you can:-
Code:
cd target_directory
for file in *
do
   mv $file $file.txt
done

If the file names gets longer and/or the number of files increases, you may hit a limit on the length of the command line when * is expanded, so bear that in mind.

All files will be renamed, so if you have a1.file a2.file & already have a1.file.txt and a2.file.txt then results might be a little unpredictable. It may well work that it will rename a1.file to a1.file.txt and then rename the same file to be a1.file.txt.txt which might be very confusing, so make sure you start with an empty directory before you download the files and rename them.



I hope that this helps.

Robin
This User Gave Thanks to rbatte1 For This Post:
cmccabe
# 3  
Old 12-12-2014
That worked great... Thank you Smilie.
# 4  
Old 12-12-2014
Hi cmccabe,

Following command may also help in same too.
Code:
find -maxdepth 1 -type f -name "*" -exec bash -c 'echo mv $0 ${0}".txt"' {} \;

You can remove echo if happy with the results.

Thanks,
R. Singh
# 5  
Old 12-12-2014
You know you have just renamed a html file to a txt file, don't you?
One can hardly convert a html file to a txt file, I mean in a way that the html tags disappear (yes, you can parse it with sed, but it's not recommended)

What do you think about this (yes, it looks complicated, but it might be a way better solution)...

In the download folder;
(Make sure there are only files downloaded from link.txt, just in case...)

Code:
awk '/pdf/ {
    gsub(/^.*href = "|".*/,"",$0)
    print FILENAME,$0 >> "/tmp/tcode-pdf.txt"
    print "http://geneticslab.emory.edu/tests/"$0 >> "/tmp/list2.txt"
}' *

The above awk
  • extracts the "path" to the download link for the appropriate pdf file.
  • creates a file tcode-pdf.txt with testcode-pdfname pairs (later, this is used in the renaming process)
  • generates a download list
Code:
wget -x -i /tmp/list2.txt

This time, wget will download PDFs

Code:
awk '{ A[$1]=$2; next} END { for (i in A) print "mv \x27"A[i]"\x27",i".pdf" }' /tmp/tcode-pdf.txt | sh

This awk command will generate commands (and execute them) to rename the cryptic filename of the pdf to testcode.pdf
E.g. test-pdf.php?testid=4125 to MM123.pdf

Code:
for i in *.pdf; do
 pdftotext "$i"
done

convert pdfs to txt files.

I've experimented with one test-code and the output looks very viable Smilie
Last edited by junior-helper; 12-12-2014 at 06:33 PM.. Reason: substituted "pdftotext *.pdf" with a for loop
# 6  
Old 12-12-2014
I am trying out your code junior-helper and have gotten to the:
Code:
 awk '{ A[$1]=$2; next} END { for (i in A) print "mv \x27"A[i]"\x27",i".pdf" }' tcode-pdf.txt | sh

I am getting this error:
Code:
 mv: cannot stat `test-pdf.php?testid=4405': No such file or directory
mv: cannot stat `test-pdf.php?testid=4143': No such file or directory
mv: cannot stat `test-pdf.php?testid=4432': No such file or directory
mv: cannot stat `test-pdf.php?testid=4421': No such file or directory
mv: cannot stat `test-pdf.php?testid=4415': No such file or directory
mv: cannot stat `test-pdf.php?testid=4434': No such file or directory
mv: cannot stat `test-pdf.php?testid=4391': No such file or directory

all the newly created files are in a new file path:
Code:
 C:\Users\cmccabe\Desktop\list\geneticslab.emory.edu.txt\tests2\geneticslab.emory.edu\tests

but even if I do a cd to that directory I get the same error. The code seems very useful and helpful. Thank you Smilie.
# 7  
Old 12-12-2014
OK, I think I know what might be the issue. In your posting #1 you said
Quote:
each new file is saved as CM080 with a FILE extention
So I'm suspecting that the new files (pdfs in this case) might have such extension too.
(Note: when I downloaded the files, neither the html files (e.g. CM080) nor the pdfs (e.g. test-pdf.php?testid=4405) had any extensions.)

If you provide the output of head -3 tcode-pdf.txt and
ls C:\Users\cmccabe\Desktop\list\geneticslab.emory.edu.txt\tests2\geneticslab.emory.edu\tests | head -3 I'm sure I can tweak that awk command to behave like it was intended.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Dig and concatenate all files yesterday then save it to another directory

I dont want to use for loop since it is using a lot of resources especially to a thousand files. Wanting to have a while? or something will find files that has been modifed or created yesteraday. View it. And search for soemthing and save it to a certain folder. for i in `find ./ -mtime... (3 Replies)
Discussion started by: invinzin21
3 Replies

2. UNIX for Beginners Questions & Answers

How can i add each line from a txt file to different files in the same directory?

Hello, this is my first thread here :) So i have a text file that contains words in each line like abcd efgh ijkl mnop and i have 4 txt files, i want to add each line to each file, like file 1 gets abcd at the end; file 2 gets efgh at the end .... I tried with: cat test | while read -r... (6 Replies)
Discussion started by: azaiiez
6 Replies

3. Shell Programming and Scripting

Cpio all *.txt-files out of folders to just one directory

I need a hint for reading manpage (I did rtfm really) of cpio to do this task as in the headline described. I want to put all files of a certain type, lets say all *.txt files or any other format. Spread in more than hundreds of subdirectories in one directory I would like to select them and just... (3 Replies)
Discussion started by: 1in10
3 Replies

4. Shell Programming and Scripting

Get the input from user and save it as .txt file

Hi friends, I am pretty new to shell scripting, please help me in this Scenario. for example, If I have one file called input.txt once I run the script, 1.It has to delete the old input.txt and create the new input.txt (if old input.txt is not there, no offence, just it has to create a... (2 Replies)
Discussion started by: Padmanabhan
2 Replies

5. Shell Programming and Scripting

I need to back up a bunch of files on a directory and save that file as the current date....

this is what i have to find the files modified within the past 24 hours find . -mtime -1 -type f -print0 | xargs -0 tar rvf "$archive.tar" however i need to save/name this archive as the current date (MM-DD,YYYY.tar.gz) how do i doo this (1 Reply)
Discussion started by: bugenhagen_
1 Replies

6. Shell Programming and Scripting

Pattern search and save it as .txt file with some name..

Hello, I have a note pad at /usr/abc location with the following content, since it is a huge file i need to split it into multiple .txt files. A123|akdhj |21kjsdff |b212b1b21 |0 A123asdasd |assdd |asdasdsdqw|6 A123|QEWQ |NMTGHJK |zxczxczx|3 A123|GEGBGH |RTYBN ... (15 Replies)
Discussion started by: j_panky
15 Replies

7. Shell Programming and Scripting

moving the files in a.txt files to a different directory

HI All, I am coding a shell script which will pick all the .csv files in a particular directoryand write it in to a .txt file, this .txt file i will use as a source in datastage for processing. now after the processing is done I have to move and archive all the files in the .txt file to a... (5 Replies)
Discussion started by: subhasri_2020
5 Replies

8. Shell Programming and Scripting

Checking if the files in a directory have a txt extension

foreach file ($dir1/*) if ($file ~ *.txt) then echo "Skipping $file (is a txt file)" endif end that should work right guys? :confused: (15 Replies)
Discussion started by: pantelis
15 Replies

9. UNIX for Dummies Questions & Answers

List all files except *.txt in a directory

I have many types of files (Eg: *.log, *.rpt, *.txt, *.dat) in a directory. I want to display all file types except *.txt. What is the command to display all files except "*.txt" (9 Replies)
Discussion started by: apsprabhu
9 Replies

10. Shell Programming and Scripting

Read from fileList.txt, copy files from directory tree

Hi, hopefully this is a fairly simple Q&A. I have a clean file list of approximately 180 filenames with no directory or slashes in front of the filename nor any extension or dot ".". I would like to read from this list, find these files recursively down through directory trees, copy the files... (1 Reply)
Discussion started by: fxvisions
1 Replies
Login or Register to Ask a Question