Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Remove duplicate rows when >10 based on single column value Post 302590571 by zaxxon on Monday 16th of January 2012 08:19:22 PM
Old 01-16-2012
You can use an editor like vi, copy & paste the code radoulov gave you, save it, change permissions to execute with the chmod command for that file and execute the script with something like ./myscript.sh and hit the enter key. The word infile needs to be the name of your input file.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Remove duplicate rows of a file based on a value of a column

Hi, I am processing a file and would like to delete duplicate records as indicated by one of its column. e.g. COL1 COL2 COL3 A 1234 1234 B 3k32 2322 C Xk32 TTT A NEW XX22 B 3k32 ... (7 Replies)
Discussion started by: risk_sly
7 Replies

2. Shell Programming and Scripting

how to delete duplicate rows based on last column

hii i have a huge amt of data stored in a file.Here in this file i need to remove duplicates rows in such a way that the last column has different data & i must check for greatest among last colmn data & print the largest data along with other entries but just one of other duplicate entries is... (16 Replies)
Discussion started by: reva
16 Replies

3. Shell Programming and Scripting

Remove duplicate line detail based on column one data

My input file: AVI.out <detail>named as the RRM .</detail> AVI.out <detail>Contains 1 RRM .</detail> AR0.out <detail>named as the tellurite-resistance.</detail> AWG.out <detail>Contains 2 HTH .</detail> ADV.out <detail>named as the DENR family.</detail> ADV.out ... (10 Replies)
Discussion started by: patrick87
10 Replies

4. Shell Programming and Scripting

duplicate row based on single column

I am a newbie to shell scripting .. I have a .csv file. It has 1000 some rows and about 7 columns... but before I insert this data to a table I have to parse it and clean it ..basing on the value of the first column..which a string of phone number type... example below.. column 1 ... (2 Replies)
Discussion started by: mitr
2 Replies

5. Shell Programming and Scripting

remove duplicates based on single column

Hello, I am new to shell scripting. I have a huge file with multiple columns for example: I have 5 columns below. HWUSI-EAS000_29:1:105 + chr5 76654650 AATTGGAA HHHHG HWUSI-EAS000_29:1:106 + chr5 76654650 AATTGGAA B@HYL HWUSI-EAS000_29:1:108 + ... (4 Replies)
Discussion started by: Diya123
4 Replies

6. Shell Programming and Scripting

Removing duplicate records in a file based on single column

Hi, I want to remove duplicate records including the first line based on column1. For example inputfile(filer.txt): ------------- 1,3000,5000 1,4000,6000 2,4000,600 2,5000,700 3,60000,4000 4,7000,7777 5,999,8888 expected output: ---------------- 3,60000,4000 4,7000,7777... (5 Replies)
Discussion started by: G.K.K
5 Replies

7. Shell Programming and Scripting

Removing duplicate records in a file based on single column explanation

I was reading this thread. It looks like a simpler way to say this is to only keep uniq lines based on field or column 1. https://www.unix.com/shell-programming-scripting/165717-removing-duplicate-records-file-based-single-column.html Can someone explain this command please? How are there no... (5 Replies)
Discussion started by: cokedude
5 Replies

8. UNIX for Dummies Questions & Answers

merging rows into new file based on rows and first column

I have 2 files, file01= 7 columns, row unknown (but few) file02= 7 columns, row unknown (but many) now I want to create an output with the first field that is shared in both of them and then subtract the results from the rest of the fields and print there e.g. file 01 James|0|50|25|10|50|30... (1 Reply)
Discussion started by: A-V
1 Replies

9. Shell Programming and Scripting

Converting Single Column into Multiple rows, but with strings to specific tab column

Dear fellows, I need your help. I'm trying to write a script to convert a single column into multiple rows. But it need to recognize the beginning of the string and set it to its specific Column number. Each Line (loop) begins with digit (RANGE). At this moment it's kind of working, but it... (6 Replies)
Discussion started by: AK47
6 Replies

10. Shell Programming and Scripting

Remove duplicate rows based on one column

Dear members, I need to filter a file based on the 8th column (that is id), and does not mather the other columns, because I want just one id (1 line of each id) and remove the duplicates lines based on this id (8th column), and does not matter wich duplicate will be removed. example of my file... (3 Replies)
Discussion started by: clarissab
3 Replies
scalac(1)							   USER COMMANDS							 scalac(1)

NAME
scalac - Compiler for the Scala 2 language SYNOPSIS
scalac [ <options> ] <source files> PARAMETERS
<options> Command line options. See OPTIONS below. <source files> One or more source files to be compiled (such as MyClass.scala). OPTIONS
The compiler has a set of standard options that are supported on the current development environment and will be supported in future releases. An additional set of non-standard options are specific to the current virtual machine implementation and are subject to change in the future. Non-standard options begin with -X. Standard Options -g:{none,source,line,vars,notc} "none" generates no debugging info, "source" generates only the source file attribute, "line" generates source and line number information, "vars" generates source, line number and local variable information, "notc" generates all of the above and will not perform tail call optimization. -nowarn Generate no warnings -verbose Output messages about what the compiler is doing -deprecation Indicate whether source should be compiled with deprecation information; defaults to off (accepted values are: on, off, yes and no) Available since Scala version 2.2.1 -unchecked Enable detailed unchecked warnings Non variable type-arguments in type patterns are unchecked since they are eliminated by erasure Available since Scala version 2.3.0 -classpath <path> Specify where to find user class files (on Unix-based systems a colon-separated list of paths, on Windows-based systems, a semi- colon-separate list of paths). This does not override the built-in ("boot") search path. The default class path is the current directory. Setting the CLASSPATH variable or using the -classpath command-line option over- rides that default, so if you want to include the current directory in the search path, you must include "." in the new settings. -sourcepath <path> Specify where to find input source files. -bootclasspath <path> Override location of bootstrap class files (where to find the standard built-in classes, such as "scala.List"). -extdirs <dirs> Override location of installed extensions. -d <directory> Specify where to place generated class files. -encoding <encoding> Specify character encoding used by source files. The default value is platform-specific (Linux: "UTF8", Windows: "Cp1252"). Executing the following code in the Scala interpreter will return the default value on your system: scala> new java.io.InputStreamReader(System.in).getEncoding -target: <target> Specify which backend to use (jvm-1.5,msil). The default value is "jvm-1.5" (was "jvm-1.4" up to Scala version 2.6.1). -print Print program with all Scala-specific features removed -optimise Generates faster bytecode by applying optimisations to the program -explaintypes Explain type errors in more detail. -uniqid Print identifiers with unique names (debugging option). -version Print product version and exit. -help Print a synopsis of standard options. Advanced Options -Xassem <file> Name of the output assembly (only relevant with -target:msil) -Xassem-path <path> List of assemblies referenced by the program (only relevant with -target:msil) -Xcheck-null Emit warning on selection of nullable reference -Xdisable-assertions Generate no assertions and assumptions -Xexperimental enable experimental extensions -Xno-uescape Disable handling of u unicode escapes -Xplug-types Parse but ignore annotations in more locations -Xplugin: <file> Load a plugin from a file -Xplugin-disable: <plugin> Disable a plugin -Xplugin-list Print a synopsis of loaded plugins -Xplugin-opt: <plugin:opt> Pass an option to a plugin -Xplugin-require: <plugin> Abort unless a plugin is available -Xprint: <phases> Print out program after <phases> (see below). -Xprint-pos Print tree positions (as offsets) -Xprint-types Print tree types (debugging option). -Xprompt Display a prompt after each error (debugging option). -Xresident Compiler stays resident, files to compile are read from standard input. -Xshow-class <class> Show class info. -Xshow-object <object> Show object info. -Xshow-phases Print a synopsis of compiler phases. -Xsource-reader <classname> Specify a custom method for reading source files. -Xscript <object> Compile as a script, wrapping the code into object.main(). Compilation Phases initial initializing compiler parse parse source files namer create symbols analyze name and type analysis refcheck reference checking uncurry uncurry function types and applications lambdalift lambda lifter typesasvalues represent types as values addaccessors add accessors for constructor arguments explicitouterclasses make links from inner classes to enclosing one explicit addconstructors add explicit constructor for each class tailcall add tail-calls wholeprog perform whole program analysis addinterfaces add one interface per class expandmixins expand mixins by code copying boxing makes boxing explicit erasure type eraser icode generate icode codegen enable code generation terminal compilation terminated all matches all phases ENVIRONMENT
JAVACMD Specify the java command to be used for running the Scala code. Arguments may be specified as part of the environment variable; spaces, quotation marks, etc., will be passed directly to the shell for expansion. JAVA_HOME Specify JDK/JRE home directory. This directory is used to locate the java command unless JAVACMD variable set. JAVA_OPTS Specify the options to be passed to the java command defined by JAVACMD. With Java 1.5 (or newer) one may for example configure the memory usage of the JVM as follows: JAVA_OPTS="-Xmx512M -Xms16M -Xss16M" With GNU Java one may configure the memory usage of the GIJ as follows: JAVA_OPTS="--mx512m --ms16m" EXAMPLES
Compile a Scala program to the current directory scalac HelloWorld Compile a Scala program to the destination directory classes scalac -d classes HelloWorld.scala Compile a Scala program using a user-defined java command env JAVACMD=/usr/local/bin/cacao scalac -d classes HelloWorld.scala Compile all Scala files found in the source directory src to the destination directory classes scalac -d classes src/*.scala EXIT STATUS
scalac returns a zero exist status if it succeeds to compile the specified input files. Non zero is returned in case of failure. AUTHOR
Written by Martin Odersky and other members of the Scala team. REPORTING BUGS
Report bugs to http://lampsvn.epfl.ch/trac/scala. COPYRIGHT
This is open-source software, available to you under a BSD-like license. See accomponying "copyright" or "LICENSE" file for copying condi- tions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. SEE ALSO
fsc(1), sbaz(1), scala(1), scaladoc(1), scalap(1) version 0.4 April 18, 2007 scalac(1)
All times are GMT -4. The time now is 12:43 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy