10 More Discussions You Might Find Interesting
1. UNIX for Advanced & Expert Users
I have a file size is around 24 G with 14 columns, delimiter with "|"
My requirement- can anyone provide me the fastest and best to get the below results
Number of records of the file
First column and second Column- Unique counts
Thanks for your time
Karti
------ Post updated at... (3 Replies)
Discussion started by: kartikirans
3 Replies
2. Shell Programming and Scripting
I have a file that I'm trying to place into a variable.
I've tried the following:
DATE=`date +%Y%m%d`
filez=$(cat /tmp/Test_$DATE.txt)DATE=`date +%Y%m%d`
filez=$(</tmp/Test_$DATE.txt)DATE=`date +%Y%m%d`
filez=$(`cat /tmp/Test_$DATE.txt`)None of these lines allows the file to go into the... (3 Replies)
Discussion started by: newbie2010
3 Replies
3. Shell Programming and Scripting
I have files named with different prefixes. From each I want to extract the first line containing a specific string, and then print that line along with the prefix.
I've tried to do this with a while loop, but instead of printing the prefix I print the first line of the file twice.
Files:... (3 Replies)
Discussion started by: pathunkathunk
3 Replies
4. UNIX for Beginners Questions & Answers
Dear Users,
Appreciate your help if you could help me with splitting a large file > 1 million lines with sed or awk. below is the text in the file
input file.txt
scaffold1 928 929 C/T +
scaffold1 942 943 G/C +
scaffold1 959 960 C/T +... (6 Replies)
Discussion started by: kapr0001
6 Replies
5. Shell Programming and Scripting
Hi All,
I have a very large single record file.
abc;date||bcd;efg|......... pqr;stu||record_count;date
when i do wc -l on this file it gives me "0" records, coz of missing line feed.
my problem is there is an extra pipe that is coming at the end of this record
like... (6 Replies)
Discussion started by: Gurkamal83
6 Replies
6. Shell Programming and Scripting
Hi all,
I need a script to delete a large set of files from a directory under / based on an input file and want to redirect errors into separate file.
I have already prepared a list of files in the input file.
Kndly help me.
Thanks,
Prash (36 Replies)
Discussion started by: prash358
36 Replies
7. Shell Programming and Scripting
Hello gurus,
I am new to "awk" and trying to break a large file having 4 million records into several output files each having half million but at the same time I want to keep the similar key records in the same output file, not to exist accross the files.
e.g. my data is like:
Row_Num,... (6 Replies)
Discussion started by: kam66
6 Replies
8. Shell Programming and Scripting
I have a 500 MB XML file from a FileMaker database export, it's formatted horribly (no line breaks at all). The node structure is basically
<FMPXMLRESULT>
<METADATA>
<FIELD att="............." id="..."/>
</METADATA>
<RESULTSET FOUND="1763457">
<ROW att="....." etc="....">
... (16 Replies)
Discussion started by: JRy
16 Replies
9. Shell Programming and Scripting
I have an extremely large csv file that I need to search the second field, and upon matches update the last field...
I can pull the line with awk.. but apparently you cant use awk to directly update the file? So im curious if I can use sed to do this... The good news is the field I want to... (5 Replies)
Discussion started by: trey85stang
5 Replies
10. Shell Programming and Scripting
Hello all,
I have 2 files, the first (indexFile1) contains start offset and length for each record inside the second file. The second file can be very large, each actual record start offset and length is defined by the entry in indexFile1. Since there are no records separators wc-l returns 0 for... (1 Reply)
Discussion started by: gio001
1 Replies