LDAP data in CSV format


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting LDAP data in CSV format
# 1  
Old 05-03-2016
Linux LDAP data in CSV format

Hi all,

I am new here, please don't eat me alive..

I am trying to find a good community to learn and participate in unix / linux discussions to help me improve in my current job.

That being said, I have a problem which I didn't expect to challenge me, but I can't seem to find a viable solution..

I am trying to do one of two things:
1. Take an LDIF export from an LDAP directory and convert it to a CSV format where the first line contains attributes and the preceding lines contain values, one line per object.
2. Find a way to export directly from the LDAP directory into a CSV format in the above configuration.

I am not good at writing scripts, so I've searched around for a pre-existing solution. Perhaps a program which takes an LDIF and a few attribute names as an input and outputs a CSV. Or maybe something which can search an LDAP directory and export the contents directly into CSV format.

Can this be done?

This is Redhat Linux, the process is IBM Security Directory Server which is an LDAP directory.
This User Gave Thanks to tfm217 For This Post:
# 2  
Old 05-04-2016
As much as I'd like to help, not knowing how "an LDIF export from an LDAP directory" looks like stops me. Posting input and desired output samples will definitely help.
# 3  
Old 05-04-2016
Quote:
Originally Posted by RudiC
As much as I'd like to help, not knowing how "an LDIF export from an LDAP directory" looks like stops me. Posting input and desired output samples will definitely help.
Sure, I can do that. Below is the current format of the LDIF output file. It can be modified in terms of the attributes but overall it's the same as the below.

Each object is separated by an empty line. The first line is the full DN of the object and each preceding line is in the form of "attribute = value". Here is an example of how two user objects would be represented. The LDIF only contains user objects.

As you can see, not every attribute is represented, for example, user2 has no 'employeenumber' and so the attribute is not listed.

I am trying to find a way to convert automatically into standard CSV format.
Code:
uid=user1,cn=users,dc=domain,dc=com
givenname=User
salesorg=2010
sn=One
telephonenumber=+15555555555
l=Melville
employeetype=E
mail=user1@domain.com 
uid=user1
c=United States
postalcode=11747
cn=User One
preferredlanguage=en_US
employeenumber=01000527
st=NY

uid=user2,cn=users,dc=domain,dc=com
givenname=User
salesorg=2010
sn=Two
employeetype=R
mail=user2@domain.com
uid=user2
c=United States
preferredlanguage=en_US
cn=User Two

Moderator's Comments:
Mod Comment Please use code tags as required by forum rules!

Last edited by RudiC; 05-04-2016 at 10:27 AM.. Reason: Added code tags.
# 4  
Old 05-04-2016
And now: the desired output sample. (What be a "standard CSV format"?)
# 5  
Old 05-05-2016
Quote:
Originally Posted by RudiC
And now: the desired output sample. (What be a "standard CSV format"?)
Here's a description and an example:

The first line should contain all possible attributes (comma separated), the preceding lines containing the values, one continuous comma-separated line for each user object. Any values already containing commas would need to be surrounded with quotes. For example:

Code:
 uid,givenname,salesorg,sn,telephonenumber,l,employeetype,mail,uid,c,postalcode,cn,preferredlanguage,employeenumber,st
 "user1,cn=users,dc=domain,dc=com",User,2010,One,+155555555555,Melville,E,user1@domain.com,user1,United States,11747,User One,en_US,01000527,NY

Moderator's Comments:
Mod Comment Please use CODE tag as required by forum rules.
I have added CODE tags to this post for you, but I have no idea if I placed them correctly.
Should there be a space at the start of the headings line?
Should there be spaces at the end of the headings line?

Last edited by Don Cragun; 05-05-2016 at 03:02 PM.. Reason: Add CODE tags, again.
# 6  
Old 05-05-2016
Quote:
Originally Posted by tfm217
Here's a description and an example:

The first line should contain all possible attributes (comma separated), the preceding lines containing the values, one continuous comma-separated line for each user object. Any values already containing commas would need to be surrounded with quotes. For example:

Code:
 uid,givenname,salesorg,sn,telephonenumber,l,employeetype,mail,uid,c,postalcode,cn,preferredlanguage,employeenumber,st
 "user1,cn=users,dc=domain,dc=com",User,2010,One,+155555555555,Melville,E,user1@domain.com,user1,United States,11747,User One,en_US,01000527,NY


Save as ldap2csv.pl
Run as perl ldap2csv.pl ldapfile

Code:
#!/usr/bin/env perl
use strict;
use warnings;

# Bail out if no file is given as an argument.
my $ldap_file = shift || usage();

# To save output with extension .csv.
my $csv_file = "$ldap_file.csv";

# Preserve order of header labels.
my @header = qw(uid0 givenname salesorg sn telephonenumber l employeetype mail
                    uid c postalcode cn preferredlanguage employeenumber st);

# Boiler plate to open input and output files handles.
open my $in, '<', $ldap_file or die;
open my $out, '>', $csv_file or die;

# Write the header labels.
print  $out "uid,", join (',', @header[1..$#header]), "\n";

# Work engine.
{
    # Record separator is a paragraph representation.
    local $/="\n\n";

    # Process one record at a time.
    while(my $record = <$in>) {
        chomp $record; # Remove the record separator.
        my %data;
        @data{@header} = ();  # To label the data.
        my @entries = split '\n', $record; # Create data entries.

        # Save first entry uid or it will be over written by second instance.
        $data{'uid0'} = qq{"$entries[0]"};

        # Work with each entry.
        for my $entry (@entries){
            my ($key, $value) = split "=", $entry, 2; # split only by first equal.
            $value =~ s/\s+$//; # Clean trailing spaces.
            $value = qq{"$value"} if $value =~ /,/; # Surround with double quotes any entries with comas.
            $data{$key} = $value;
        }
        # Separate entries with comas.
        my $row = join ',', (map{ $data{$_}?$data{$_}:""} @header);

        # Write to output file.
        print $out "$row\n";
    }
}
# Dismiss file handles.
close $in;
close $out;

# Feed back to user.
print "$csv_file has been saved in the current directory\n";

sub usage {
    print "Usage: $0 ldapfilename\n";
    exit 1;
}


Last edited by Aia; 05-07-2016 at 09:40 AM.. Reason: Correct mispelling
This User Gave Thanks to Aia For This Post:
# 7  
Old 05-06-2016
Aia, very awesome, thanks so much. This works 100% with what I am trying to do. Smilie

I appreciate you putting this together for me.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Log file data into CSV format

I m looking for help here!!! Can we filter the below log data into CSV format ? 1 2 3 4 5 6 7 8 timestamp INFO <text > - Some text (1 Reply)
Discussion started by: MohSalNiz
1 Replies

2. Shell Programming and Scripting

Can we filter the below log data into CSV format?

HI , I m looking for help here!!! Can we filter the below log data into CSV format ? timestamp INFO <text > - Some text Drive .. Need a format of separate field such as 1 2 3 4 ... (2 Replies)
Discussion started by: MohSalNiz
2 Replies

3. Shell Programming and Scripting

Convert csv data to html format

I am new to html and need to convert the attached csv file data to html format ; running into issues. please assist. #!/bin/ksh echo "<html>" ; echo "<head><style> table {border-collapse: collapse;} table, td, th {border: 1px solid black;} </style></head>" echo "<title> REPORT </title>" echo... (0 Replies)
Discussion started by: archana25
0 Replies

4. Shell Programming and Scripting

LDAP data in CSV format - Part II

User Aia had created this below script to help translate LDIF files to CSV format, and it works very well (thanks again Aia if you are reading this) Here is the original thread. https://www.unix.com/shell-programming-and-scripting/265753-ldap-data-csv-format.html?referrerid=302170129 However... (2 Replies)
Discussion started by: tfm217
2 Replies

5. Shell Programming and Scripting

CSV data format manipulation

Hi There I need a script which will pick up the data from a .CSV file and reformat it as per the requirement and write it to another .CSV file. I am using an application that will only take data in a particular format and need something that will convert without manual intervention. The... (4 Replies)
Discussion started by: rbggbr16
4 Replies

6. Shell Programming and Scripting

Conversion of xhtml data into csv format using dump utility

Hi Unix Gurus, I tried to convert the attached xhtml table content into csv file using unix shell script (lynx -dump filename) and got the below results: Title ID Owner Priority Estimate Project Change Date Changed By Complexity Create Date Created By Detail Estimate Total De tail... (6 Replies)
Discussion started by: bi.infa
6 Replies

7. Shell Programming and Scripting

Retaining the Unix CSV format in Excel format while exporting

Hi All, I have created a Unix Shell script whch creates a *.csv file and export it to Excel. The problem i am facing is that Users wants one of the AMOUNT field in comma separted values. Example : if the Amount has the value as 3000000 User wants to be in 3,000,000 format. This Amount format... (2 Replies)
Discussion started by: rawat_me01
2 Replies

8. UNIX for Advanced & Expert Users

shell script to format .CSV data

Hi all, I have written a shell script to search a specified directory (e.g. /home/user) for a list of specific words (shown as ${TMPDIR}/wordlist below). The script works well enough, but I was wondering if there was a way to display the line number that the word is found on? Thanks! cat... (1 Reply)
Discussion started by: tmcmurtr
1 Replies

9. Shell Programming and Scripting

AWK CSV to TXT format, TXT file not in a correct column format

HI guys, I have created a script to read 1 column in a csv file and then place it in text file. However, when i checked out the text file, it is not in a column format... Example: CSV file contains name,age aa,11 bb,22 cc,33 After using awk to get first column TXT file... (1 Reply)
Discussion started by: mdap
1 Replies

10. Shell Programming and Scripting

Shell script to format a .CSV data

Hi There I needed to write a Unix shell script which will pick up the data from a .CSV file and reformat it as per the requirement and write it to another .CSV file. Currently I am in the proess of Data Import to "Remedy System" (A one kind of incident mangement Application) and this... (8 Replies)
Discussion started by: Uday1982
8 Replies
Login or Register to Ask a Question