Search Results

Apologies for not being clear. The idea is to have the file formatted in a semicolon way, without the description in front. If you look at the raw file it describes everything with e.g. url= which I...

Hi, I would like to format the file input example to the specific output example. I have tried numerous different ways, however not able to extract the information as desired. Any assistance to get...

Got it thank you very much Don Cragun it is working now:

awk -F";" -vDQ='"' '{sub (/\r$/,"");print DQ $7 DQ ": " DQ $0 DQ}' output.txt


---------- Post updated at 12:54 PM ---------- Previous...

Thank you for the suggestion, I am currently getting an error:

awk -F";" -vDQ='"' '{sub(/\r$/,""};print DQ $7 DQ ": " DQ $0 DQ}' output.txt
Error:
awk: cmd. line:1: {sub(/\r$/,""};print DQ $7 DQ...

@durden_tyler thank you for the response. You are correct the file is being FTPed to the location however in ASCII. The challenge is that the FTP transfer cannot be changed as this is a 3rd party...

Thank you very much that is amazing, it works.

When I echo the line and run the filter all is working as expected, however if I run the statement against a file, the last Double Quote is not...

I tried the following:

awk 'BEGIN { FS=";"; OFS=";"; } {print $7,$1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13}' file


however this does not work and how to add the " and : I have not found a...

Hi, I would like to change my CSV file by adding " and : and moving some of the information around. the CSV file looks as follows:

501254424;500440257;PE...

Hi RudiC, Thank you for that hint, I was not aware of this command. I have now added the following script to split the files:
#!/usr/bin/bash
fspec=2015-11-09.querylogs
num_files=10

# Work out...

Thank you for the feedback, do you maybe have any suggestions or examples on how to best script this?
I was thinking of maybe starting off with a wc -l filename and then dividing this into the...

Hi RudiC, Quick question is it not possible to split a file into say 1 million lines and have multiple instances running. The joining the files after completing. So for the 27 million lines there...

Yes, here is the information:
Some thoughts to improve processing:
- could you remove duplicates from the input files?
No as they have different time-stamps
- could you preprocess (condense)...

Hi RudiC, Ok let me try to explain the challenge I am faced with and you are correct there are many great minds out there that might have a possible solution for this.

So there are three DNS...

Hi RudiC, I have thought about the categorization part of each line however it is taking too much processing power. I will rather do this at the end when I have summarized the files and sorted them....

Hi RudiC, apologies for not being clear on the point of the categories. lets me try and elaborate, for every URL of the logs I would like to extract the domain, this is the script you already...

Thank you Don Cragun for the optimization of the code, I was not aware of this method of defining variables in one line. With this method I was able to improve the line reading speed by 5 times.
...

Thank you RudiC, for the script and assistance, it is truly appreciated. The script works very well now and extracts the Domain from the URL.

Also thank you Don Cragun, for the assistance.
...

Hi RudiC, Thank you for the script, I am trying to resolve one challenge to check if it is working. I am currently getting: warning: escape sequence `\.' treated as plain `.'

Will try and figure...

Don Cragun, you are correct, I got excited to early. After running the script through a few hundred examples I found it is not working as desired. Do you maybe have suggestion how to extract the...

RudiC, thank you very much for providing this solution, it is truly appreciated. I checked through the publicsuffix list and found that the longest domain is 4 as such added this to the script you...

Thank you RudiC, could I kindly ask you to elaborate on the code, as mentioned before, I am very new to this. I have two files the one that contains the URL and the other one the publicsuffic list....

Thank you for the response Aia, however that post is quite old and does not seem to be active anymore. or have a solution as such.Also thank you for the publicsuffix list this is very helpful and has...

Hello everybody
I have been trying to extract the domain name from the bind query log with different options, however always get stuck with domains that end with link .co.uk or .co.nz.

I tried...