|
|
Sponsored Content
Top Forums
Shell Programming and Scripting
Bash/AWK Newbie taking on more than he can chew.
Post 302151703 by Asylus on Monday 17th of December 2007 02:54:09 AM
12-17-2007
Bash/AWK Newbie taking on more than he can chew.
A few questions: I'm trying to use Bash (although I'm not against using AWK) to try to accomplish a few things, but I'm stumped on a few points.
I'm learning most of the basics quickly: but there are a few things I can't figure out.
1. I'm trying to count the number of .txt files in a directory and save it in a variable.
I can use
find $DIRECTORY -name *.txt | wc -l
but can't save that in a variable.
2&3. Is it possible to actually open these files (this will be a separate task entirely) and read a specific line? All of them are formatted the same, and I need to be able to read that line and save THAT to a list/array of variables- there are over 10,000 files- and I need the fourth line in each .txt. Many of these .txt files should have this line the same, so the array will never approach 10,000 - but I do need to be able to count how many times each appears- how would I do this?
(I know how to use loops, and create variables, but not open files (and read specific lines) nor make arrays/lists/whatever you call them in bash. <-Used to Java/Python; but can't use either here.
Any help with this is greatly appreciated- and I thank you for any help you can provide.
Last bit:
the files are all named similarly... 309456_20.txt
503256_01.txt
etc.
and all the files are in subdirectories named the same as the first three digits of the file.
So 309456_20.txt would be in subdirectory 309.
I'm not looking for a fast program or even a relatively efficient one; as long as it gets the job done.
I'm learning most of the basics quickly: but there are a few things I can't figure out.
1. I'm trying to count the number of .txt files in a directory and save it in a variable.
I can use
find $DIRECTORY -name *.txt | wc -l
but can't save that in a variable.
2&3. Is it possible to actually open these files (this will be a separate task entirely) and read a specific line? All of them are formatted the same, and I need to be able to read that line and save THAT to a list/array of variables- there are over 10,000 files- and I need the fourth line in each .txt. Many of these .txt files should have this line the same, so the array will never approach 10,000 - but I do need to be able to count how many times each appears- how would I do this?
(I know how to use loops, and create variables, but not open files (and read specific lines) nor make arrays/lists/whatever you call them in bash. <-Used to Java/Python; but can't use either here.
Any help with this is greatly appreciated- and I thank you for any help you can provide.
Last bit:
the files are all named similarly... 309456_20.txt
503256_01.txt
etc.
and all the files are in subdirectories named the same as the first three digits of the file.
So 309456_20.txt would be in subdirectory 309.
I'm not looking for a fast program or even a relatively efficient one; as long as it gets the job done.
Last edited by Asylus; 12-17-2007 at 01:44 PM.. Reason: Additional Information
|
|
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi there,
I am a student and currently working on a project. I have a file that contains about 50 filenames. (1.txt, 2.txt, 3.txt ...).
I would like to know how can I store these filenames into a variable using a loop?
I would appreciate if anyone can help me. Thank You.
Regards,
Bib (4 Replies)
Discussion started by: coolbib
4 Replies
2. Shell Programming and Scripting
I'm in .profile. I want to export a directory ( export MY_TOOL_HOME=/tools/my tool". Unfortunately for me, the directory I want to export to has a space in its name. So when I try to cd $MY_TOOL_HOME, i get a No such file or directory at the command line... thanks for the help (4 Replies)
Discussion started by: redsand9009
4 Replies
3. Shell Programming and Scripting
Hi,
I'm very new to bash scripting and Linux in general. I'm running Ubuntu Server 10.04 and trying to write a bash script to launch a program. In doing so, I've come across a couple of things that I obviously don't understand. Here is a short script that exemplifies those things:
... (9 Replies)
Discussion started by: Carson Dyle
9 Replies
4. Shell Programming and Scripting
Hi
I'm just messing around with bash and trying to learn it because I have a course next semester dealing with OS design where we need to know how to use SSH client and either bash or ksh. I've never done shell scripting before.
I just started today and I was wondering how parsing files... (1 Reply)
Discussion started by: mehungry
1 Replies
5. UNIX for Dummies Questions & Answers
Hi all, I'm looking for an awk solution for taking bins of data set.
For example, if I have two columns of data that I wish to use for a scatter plot, and it contains 5 million lines, how can I take averages of every 100 points, 1000, 10000 etc...
The idea is to take bins of the 5,000,000 points... (7 Replies)
Discussion started by: torchij
7 Replies
6. Shell Programming and Scripting
Hi all,
I think so I’m getting the result is wrong, while using following awk commend,
colval=$(awk 'FNR>1 && NR==FNR{a=$4;next;} FNR>1 {a+=$4; print $2"\t"a/3}'
filename_f.tsv filename_f2.tsv filename_f3.tsv)
echo $colval >> Result.tsv
it’s doing the condition 2 times, first result... (5 Replies)
Discussion started by: Shenbaga.d
5 Replies
7. Shell Programming and Scripting
Hi, i want to print 2nd column value with the below script. I need to take input of the string i need to search in that file and file name. How can i take these two as inputs? using read command? Getting error for below script.
echo "enter SID"
read SID
echo "enter filename"
read filename... (8 Replies)
Discussion started by: sam_bd
8 Replies
8. Shell Programming and Scripting
Hi all,
I have drill to do and I'll very appreciate your help:
Please create a simple CSV file as follow (3 columns and 4 rows):
A,B,C
A,”B,C”,D
“A,B”,C,D
o A,B,”C,D”
- Please refer to the comma between the quotation marks as a parameter and not as a separator.
- Please provide... (3 Replies)
Discussion started by: elior
3 Replies
9. Shell Programming and Scripting
Hi forums,
Wondering if someone could assist me!!...
I basically want to do a while read line on a file and input it into a set of parameters for a JSON geospatial demo.. With a loop where every line creates a new "card" to an effect... card example below...
Data CSV one line:
... (4 Replies)
Discussion started by: skoot
4 Replies
10. Shell Programming and Scripting
Please bear with me, I'm a beginner but have had some experience and decent knowledge to understand things as I read them and I'm in the process of trying to learn more. I recently inherited a UNIX server which has a bash script which is part of a crontab schedule that needs to be modified (or... (3 Replies)
Discussion started by: Danylko
3 Replies
LEARN ABOUT REDHAT
mac2unix
dos2unix(1) General Commands Manual dos2unix(1) NAME
dos2unix - DOS/MAC to UNIX text file format converter SYNOPSYS
dos2unix [options] [-c convmode] [-o file ...] [-n infile outfile ...] Options: [-hkqV] [--help] [--keepdate] [--quiet] [--version] DESCRIPTION
This manual page documents dos2unix, the program that converts plain text files in DOS/MAC format to UNIX format. OPTIONS
The following options are available: -h --help Print online help. -k --keepdate Keep the date stamp of output file same as input file. -q --quiet Quiet mode. Suppress all warning and messages. -V --version Prints version information. -c --convmode convmode Sets conversion mode. Simulates dos2unix under SunOS. -o --oldfile file ... Old file mode. Convert the file and write output to it. The program default to run in this mode. Wildcard names may be used. -n --newfile infile outfile ... New file mode. Convert the infile and write output to outfile. File names must be given in pairs and wildcard names should NOT be used or you WILL lost your files. EXAMPLES
Get input from stdin and write output to stdout. dos2unix Convert and replace a.txt. Convert and replace b.txt. dos2unix a.txt b.txt dos2unix -o a.txt b.txt Convert and replace a.txt in ASCII conversion mode. Convert and replace b.txt in ISO conversion mode. Convert c.txt from Mac to Unix ascii format. dos2unix a.txt -c iso b.txt dos2unix -c ascii a.txt -c iso b.txt dos2unix -c mac a.txt b.txt Convert and replace a.txt while keeping original date stamp. dos2unix -k a.txt dos2unix -k -o a.txt Convert a.txt and write to e.txt. dos2unix -n a.txt e.txt Convert a.txt and write to e.txt, keep date stamp of e.txt same as a.txt. dos2unix -k -n a.txt e.txt Convert and replace a.txt. Convert b.txt and write to e.txt. dos2unix a.txt -n b.txt e.txt dos2unix -o a.txt -n b.txt e.txt Convert c.txt and write to e.txt. Convert and replace a.txt. Convert and replace b.txt. Convert d.txt and write to f.txt. dos2unix -n c.txt e.txt -o a.txt b.txt -n d.txt f.txt DIAGNOSTICS
BUGS
The program does not work properly under MSDOS in stdio processing mode. If you know why is that so, please tell me. AUTHORS
Benjamin Lin - <blin@socs.uts.edu.au> Bernd Johannes Wuebben (mac2unix mode) <wuebben@kde.org> MISCELLANY
Tested environment: Linux 1.2.0 with GNU C 2.5.8 SunOS 4.1.3 with GNU C 2.6.3 MS-DOS 6.20 with Borland C++ 4.02 Suggestions and bug reports are welcome. SEE ALSO
unix2dos(1) mac2unix(1) 1995.03.31 dos2unix v3.0 dos2unix(1)