Normalize text field or sort values


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Normalize text field or sort values
Prev   Next
# 2  
Old 04-20-2020
1. Could you please clarify?
injest => ingest
normalize => sort ?

2. Could you provide a short example of your input data?

3. Could you provide your script trial?
If you run it on the example input data, would goes wrong?
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

awk GSUB read field values from multiple text files

My program run without error. The problem I am having. The program isn't outputting field values with the column headers to file.txt. Each of the column headers in file.txt has no data. MEMSIZE SECOND SASFoundation Filename The output results in file.txt should show: ... (1 Reply)
Discussion started by: dellanicholson
1 Replies

2. Shell Programming and Scripting

awk to parse field and include the text of 1 pipe in field 4

I am trying to parse the input in awk to include the |gc= in $4 but am not able to. The below is close: awk so far: awk '{sub(/\|]+]++/, ""); print }' input.txt Input chr1 955543 955763 AGRN-6|pr=2|gc=75 0 + chr1 957571 957852 AGRN-7|pr=3|gc=61.2 0 + chr1 970621 ... (7 Replies)
Discussion started by: cmccabe
7 Replies

3. Shell Programming and Scripting

Sort file by field 1 that has text as well as a number

I am using the below awk that results in the below output: awk '{k=$1 OFS $2; s+=$4; c++} END{for(i in s) print i, s/c}' input.txt > output.txt output.txt chr20:43625799-43625957 STK4:exon.6;STK4:exon.7 310.703 chr20:36770455-36770611 TGM2:exon.6;TGM2:exon.7 614.756... (5 Replies)
Discussion started by: cmccabe
5 Replies

4. Shell Programming and Scripting

Read record from the text file contain multiple separated values & assign those values to variables

I have a file containing multiple values, some of them are pipe separated which are to be read as separate values and some of them are single value all are these need to store in variables. I need to read this file which is an input to my script Config.txt file name, first path, second... (7 Replies)
Discussion started by: ketanraut
7 Replies

5. Shell Programming and Scripting

Awk Search text string in field, not all in field.

Hello, I am using awk to match text in a tab separated field and am able to do so when matching the exact word. My problem is that I would like to match any sequence of text in the tab-separated field without having to match it all. Any help will be appreciated. Please see the code below. awk... (3 Replies)
Discussion started by: rocket_dog
3 Replies

6. Shell Programming and Scripting

awk, comma as field separator and text inside double quotes as a field.

Hi, all I need to get fields in a line that are separated by commas, some of the fields are enclosed with double quotes, and they are supposed to be treated as a single field even if there are commas inside the quotes. sample input: for this line, 5 fields are supposed to be extracted, they... (8 Replies)
Discussion started by: kevintse
8 Replies

7. Shell Programming and Scripting

adding field values if field matches

hi i have file as below , i want to add duplicate records like bell_bb to one record with valuve as 15 ( addition of both ) any oneline awk script to achive this ? header 0 CAMPAIGN_NAME 1 Bell_BB 14 Bell_MONTHLY 803 SOLO_UNBEATABLE 644 Bell_BB 1 Bell_MONTHLY 25 SOLO_UNBEATABLE... (4 Replies)
Discussion started by: raghavendra.cse
4 Replies

8. Shell Programming and Scripting

Sort alpha on 1st field, numerical on 2nd field (sci notation)

I want to sort alphabetically on the first field and sort in descending numerical order on the 2nd field. With a normal "sort -r -n" it does this: abc ||| 5e-05 ||| bla abc ||| 3 ||| ble def ||| 1 ||| abc def ||| 0.2 ||| def As you can see it ignores the fact that 5e-05 is actually 0.00005... (1 Reply)
Discussion started by: FrancoisCN
1 Replies

9. Shell Programming and Scripting

Find top N values for field X based on field Y's value

I want to find the top N entries for a certain field based on the values of another field. For example if N=3, we want the 3 best values for each entry: Entry1 ||| 100 Entry1 ||| 95 Entry1 ||| 30 Entry1 ||| 80 Entry1 ||| 50 Entry2 ||| 40 Entry2 ||| 20 Entry2 ||| 10 Entry2 ||| 50... (1 Reply)
Discussion started by: FrancoisCN
1 Replies

10. Shell Programming and Scripting

How to sort a field in a file having date values

Hi All, I am having a pipe delimited file .In this file the 3rd column is having date values.I need to get the min date and max date from that file. I have used cut -d '|' test.dat -f 3|sort -u But it is not sorting the date .How to sort the date column using unix commands Thanks ... (4 Replies)
Discussion started by: risshanth
4 Replies
Login or Register to Ask a Question
magic(4)						     Kernel Interfaces Manual							  magic(4)

NAME
magic - Magic file for the file command SYNOPSIS
/etc/magic DESCRIPTION
The magic file is used by the file command to identify files that have some sort of magic number. A magic number is any numeric or string constant that identifies the file containing the constant. The format for the magic file is as follows: offset type operator,value string The fields should be separated by tabs. Each record must be contained on one line. The fields contain the following data: This field contains the number of bytes from the beginning of the file on which you are running the file command to the first byte of the magic number of character string you want to identify. Use a right angle bracket (>) to indicate a continuation line that supplies additional information describing the file. This field contains information about the data type of the magic number or character string at the specified byte offset. Valid data types for this field are: Unsigned character type Unsigned short type Long type Character (byte) string This field contains instructions for the file command on how to compare the value read from the file being checked with the value stored in the Value Type field of the magic file. The valid comparison operators are: The two values are equal. The value in the file being checked is greater than the value in the magic file. The value in the file being checked is less than the value in the magic file. All the bits in the magic file value must be set in the value from the file being checked. Note that the Comparison Operator field is optional. If you do not specify the operator, the values are expected to be equal. This field contains the value used to compare what is read from the file being checked by the file command. You can use decimal, hex, or octal numbers in this field or character strings in the form of regular expressions. Precede all hex numbers with the characters zero and x (for example, 0x80). To specify an octal number, precede it with a zero (for example, 0200). Decimal numbers require no special representation and should be written as integers (for example, 128). The rules for specifying character strings follow those of the ed editor (see ed(1)) for regular expressions, with two extensions: You use the backslash () to escape an unprintable character. The string can contain all special character such as , , , and f. If a backslash appears in the string, it must be escaped with a second backslash (\). You can use octal representation to specify any byte value other than zero (0). Text found in the file can be inserted into the printed string if it is preceded and followed by \% delimiters. All text found between these delimiters is displayed as the print string. This regular expression search never terminates until a match is explicitly found or rejected. The special character is a valid character in the patterns. Therefore, the pattern .* should never be used here. This field contains the string to print. The string provides information about the file. The string can include text found in the file when requested with an appropriate printf() format. EXAMPLES
The following is an example of a script: string ^#!{ }*\%[^ ]*\% %s The following are examples of executable images: >2 short 02 POSIX >2 short 01 SVID >16 long >0 not stripped The following are examples of text and data files: 0 string ^1h[0-9][0-9][0-9][0-9][0-9] sccsfile 0 string ^#ifndef c program 0 string ^070707 ASCII cpio archive FILES
/etc/magic RELATED INFORMATION
Commands: file(1) delim off magic(4)