1. yes. Though the initial fields(upto 10th column) are constant across all type of log entries but others will vary. The 2 example log I have given are two different type of log generated for 2 different events. So you will not get the linkId attribute in 2nd log entry. Even you do not need to bother about that because you just need to pick 5th column from 2nd log and replace 1st log after checking the id field if that matches. But in real scenario you have to grep through the entire log file to look for the id value found in 1st log.
i) I have multiple log files where I need to grep(like abc-2012-10-01_00000,abc-2012-10-01_00001.... etc.) and output 2nd, 5th and 14th column.
ii) While grepping through all the log files those invalid 2nd column will appear which is not intended.In the same file from where invalid 2nd columns were found valid 2nd columns can be found from there only by looking through the matching 'id' attribute value. It is upto you if you can achieve my goal in single grep.
My algorithm:-
i) for each file run above grep
for each row got from above grep if 2nd column is invalid(none11111)
run another grep on same file and replace invalid 2nd column with new one.
If the two sample lines from your log files are as you have shown in past posts, the command line you specify above is equivalent to the command:
There are no tab characters in you input files, so the cut command in your pipeline is a no-op. So this command line throws away duplicate lines found in your log files and sorts the remaining lines on the first field. It does NOT limit the output to only columns 2, 5, and 14 from your input files; does NOT produce a CSV file with three fields (and if it did; it wouldn't contain the id=value fields that say are to be used in a second grep to look for the invalid values found while processing the output from your first grep).
In message #8 in this thread, I asked nine questions. You partially answered some of the questions although, as noted above, the answer doesn't match the other statements you've made.
I want to help you solve this problem, but if you won't answer the questions (and give answers that match your data), it is obvious that I'm wasting my time. If you would like us to try to give you a working solution please answer ALL of these questions:
What are the actual commands you execute to convert your log files into the CSV file that you want processed?
Doesabc-2012-10-01_000*match the names of all of the log files (and only those log files) that you want to process?
When you findnone11111in your CSV file, will theid=xxxfield ever match more than one line (not containingnone11111) in your log files that aren't exact duplicates of other lines?
Am I correct in assuming that the line matching theid=xxxfield with the value needed to replacenone11111in your CSV file, will not be on a line that was selected by a grep on the linkId field you're processing?
Is the field separator you want in your output file","or", "?
Does the order of lines in your output file matter?
What is the purpose of having an additional single-line output file containing a comma separated list of all of your log files? If you need a file containing a list of the log files processed, wouldn't it be better to have the filenames on separate lines instead of separated by commas on a single line?
Will thelinkId=zzzfield ever appear in any log file that isn't exactly of the same form as the following example line from one of your log files?
Code:
abc.com 20120829001415 127.0.0.1 app none11111 sas 0 0 N clk Mozilla/5.0 id=82c6a15ca06b2372c3b3ec2133fc8b14 referrer=google.com linkId=1ddoic
This User Gave Thanks to Don Cragun For This Post:
Hi gurus
I am not a C programmer but I need to read and write files on a computer on LAN using IP address.
Suppose on a computer that has an IP 192.168.0.2
Any help or code example. I did in JAVA using URL, but do not know how to do in ANSI-C.
In java:
-------
URL url = new... (3 Replies)
Hi All,
Not sure if this would be in a dummies sectiin or advanced. I'm looking for a script if someone has doen something like this.
I have list of files -
adc_earnedpoints.20070630.txt
adc_earnedpoints.20070707.txt
adc_earnedpoints.20070714.txt
adc_earnedpoints.20070721.txt... (1 Reply)
Hi my prob statement is to create a new file or to append to the 1tst file the followign chages.
File 1: txt file.
portfolio No a b c d
abc 1 Any Any Any charString
cds 2 values values values charString
efd 3 can can can charString
fdg 4 come come come charString... (4 Replies)
Hi all,
I have the following shell script code which tries to sftp and writes the log into the log file.
TestConnection ()
{
echo 'Connection to ' $DESTUSERNAME@$DESTHOSTNAME
$SETDEBUG
if ]; then rm $SCRIPT ; fi
touch $SCRIPT
echo "cd" $REMOTEDIR >> $SCRIPT
echo "quit" >>... (10 Replies)
Hello everybody,
I'm trying to code a program which makes the following:
It sends an ARP request frame and when it gets the reply, extracts the IP address of source and writes it to a .txt file. This is gonna be done with many hosts (with a for() loop), so, the text file would look like... (2 Replies)
Hi All,
Please help me in writing data to a file in one row.
In database there is a column which contains large data which does not fit in the file in one row. The column contains list of paths. I want to write these paths to a file in one row.
Please find the code below writes :
... (2 Replies)
Need to develop a unix shell script for the below requirement and I need your assistance:
1) search for file.log and file.bad file in a directory and read them
2) pull out "Load_Start_Time", "Data_File_Name", "Error_Type" from log file
4) concatinate each row from bad file as... (3 Replies)
In the code below i try to write and read from a file, but i get unexpected results, like after writing i cannot open the file, and when reading the file the value entered earlier is not shown
bool creat_fragments(int nFragment)
{
int fd, rand_value;
char frag_path, buf;
for(int... (8 Replies)
Hi
I am trying to extract information out of a file but keep getting grep cant open errors
the code is below:
#bash
#extract orders with blank address details
#
# obtain the current date
# set today to the current date ccyymmdd format
today=`date +%c%m%d | cut -c24-31`
echo... (8 Replies)
Hello all
As part of my TUI - (line based) Text User Interface, i do have 2 commands to assist working with conf files.
Now, for me they work, but since i wrote them, i automaticly use them they way they should be used... you know what i mean. ;)
Anyway, they are designed to read 'simple'... (3 Replies)
09-08-2012
If you would like us to try to give you a working solution please answer ALL of these questions: