11-12-2014
Quote:
Originally Posted by
yuejian
Thank you very much RudiC. Actually I am an embryology biologist and I don't have any computer science background and I am a totally rookie. I have to deal with some very large data set from the experimental work recently and I don't know how to cope with them. I learned that some unix command can help me to do that, then I tried to use some very simple command like such as sed or awk to do some basic work and it is really amazing for me. However, I don't have ability to deal with the more complicated problem even these problems seems very naive for you computer guys. So that's why I post many questions on this forum recently. I wish I can figure out some problems but I really don't have the ability right now. Actually I am studying some programming skills by myself now and I hope I can also help some other people like me in the future. I raelly appreciate anyone helped me.
This is all well and we try to help as much as we can. The reason why most of us insist that a genuine effort in solving the problem by oneself is made is because we try to help you to help yourself. Most of us, me included, are professionals and we get paid to do for customers what we help each other to achieve here. The big dfference between writing here and doing work for a customer is: i do not expect my customer to spend any effort - i expect him to pay me for doing his work instead. Here it is the other way round. I do not expect to get paid - not in the narrower sense, obviously, but not even in a virtual sense, like "being paid by recognition" or using what i do here to advertise my skills, but on the other hand i expect my vis-a-vis to spend some effort to digest what i try explain to him and to try to learn what i explain.
Otherwise i have to ask myself why i charge my customer some money for my work when i do it for free here.
But there is - or at least should be - another reason for you to invest effort in solving your problems: whenever you work using a certain tool there is a dialectical connection between the work and the tool: the tools is shaped by the work you expect it to do but also your work is shaped by the tools you use. You sure know the saying that
if you only have a hammer everything starts to look like a nail. This the tool shaping the work and the way the work is done. So, when you do your work using a computer as a tool you will not only be able to do it better when you are more competent with it but chances are you will be changing the way you work and you may find new applications to the tool or ways of applying it which you are not aware of right now.
After so much philosophical contemplation here is a practical suggestion for you: most of your problems seem to involve "data massaging" of one sort or the other. Most (perhaps all) of what you do can be done using
awk and/or
sed. There is a phantastic book about exactly these two tools: it is written by
Dale Dougherty and i can wholeheartedly suggest it. It is a good and diverting read while being the best and most complete single source about the topic i ever read.
bakunin
These 2 Users Gave Thanks to bakunin For This Post:
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
I need to extract a column from a tab delimited text file based on the string in the first row. (i.e. extract the column labeled "reaction_time"). I know that this can be done with awk by determining the column #, but I want to do it based on a search for the string b/c the column number changes... (3 Replies)
Discussion started by: t27
3 Replies
2. UNIX for Advanced & Expert Users
This is the data file that I have
21879, 0, 22, 58, 388
0, -1, 300, 1219172589765, 1708, 0, 200, 21891, 0, 0, 33, 309
0, -1, 300, 1219172591478, 1768, 0, 200, 22505, 0, 0, 33, 339
0, -1, 300, 1219172593251, 1738, 0, 200, 21888, 0, 1, 33, 308
0, -1, 300, 1219172594995, 633, 0, 200, 24878,... (2 Replies)
Discussion started by: pmallur
2 Replies
3. Shell Programming and Scripting
Hi I have a csv file which is below
A,5
B,6
C,10
D,7
I want the values who's second column is greater than 7
say
C,10
D,7
Help me please...
Thanks,
Maruth (3 Replies)
Discussion started by: maruthavanan
3 Replies
4. Shell Programming and Scripting
i have a file - it will be in sorted order on column 1
abc 0 1
abc 2 3
abc 3 5
def 1 7
def 0 1
--------
i'd like (awk maybe?) to get the results (any ideas)???
abc 5 9
def 1 8 (2 Replies)
Discussion started by: jjoe
2 Replies
5. Emergency UNIX and Linux Support
Hi guys,
I need help in making a command to find some data.
I have multiple files in which multiple records are present.. Each record is separated with a carriage return and in each record there are multiple fields with each field separated by "|"
what i want is that I want to extract... (1 Reply)
Discussion started by: m_usmanayub
1 Replies
6. Shell Programming and Scripting
Hello,
I have a huge file with many columns . I want to use the names of the columns to print columns to another file.
for example
file.txt
COLA COLB COLC COLD
1 2 3 5
3 5 6 9
If I give name for multiple columns to the code:... (5 Replies)
Discussion started by: ryan9011
5 Replies
7. Shell Programming and Scripting
I have a LOG file which looks like this
Import started at: Mon Jul 23 02:13:01 EDT 2012
Initialization completed in 2.146 seconds.
--------------------------------------------------------------------------------
--
Import summary for Import item: PolicyInformation... (8 Replies)
Discussion started by: biztank
8 Replies
8. Shell Programming and Scripting
I have two files.
FileA.txt
30910 rs7468327
36587 rs10814410
91857 rs9408752
105797 rs1133715
146659 rs2262038
152695 rs2810979
181843 rs3008128
182129 rs3008131
192118 rs3008170
FileB.txt
30910 1.9415219673 0
36431 1.3351312477 0.0107191428
36587 1.3169171182... (2 Replies)
Discussion started by: genehunter
2 Replies
9. Shell Programming and Scripting
Hi to all,
I have two files. File1 has no header, two columns:
sample1 A
sample2 B
sample3 B
sample4 C
sample5 A
sample6 D
sample7 D
File2 has a header, except for the first 3 columns (chr,start,end). "sample1" is the header for the 4th ,5th ,6th columns, "sample2" is the header... (4 Replies)
Discussion started by: aec
4 Replies
10. Shell Programming and Scripting
Hi,
I have a requirement where in I need to insert delimiters before the last column of the total delimiters is less than a specified number.
Say if the delimiters is less than 139, I need to insert 2 columns ( with blanks) before the last field
awk -F 'Ç' '{ if (NF-1 < 139)} END { "Insert 2... (5 Replies)
Discussion started by: arunkesi
5 Replies