Welcome to Powergui.org - an open source community for Windows Powershell

PowerGUI.org PowerGUI.org and blogs

Forums » Active Directory and PowerShell

Thread: compare two csvs and export match to another csv

This question is answered. Helpful answers available: 0. Answered answers available: 1.


Permlink Replies: 11 - Pages: 1 - Last Post: Jun 17, 2008 7:12 AM by: Shay Levy
frangwala@b2btech.com

Posts: 23
Registered: 4/13/08
compare two csvs and export match to another csv
Posted: May 17, 2008 7:44 PM
 
  Click to reply to this thread Reply

Here's my issue.


I have one csv that has all the users logon info, first name, last name, CN, email address and samaccountname. There are over 4000 records in this csv.


I have another csv that has users first name and lasr name, employee id, dept number, employeetype. There are over 250 records in this csv.


I have to compare these two csvs and export to another csv the matches with all the fields in both csvs.


I looked at the compare cmdlet, but did not find a way to get the result i needed.


Any help willl be much appreciated.


Thank you.


Firoz




Shay Levy


Posts: 1,938
Registered: 1/31/08
Re: compare two csvs and export match to another csv
Posted: May 18, 2008 12:31 AM   in response to: frangwala@b2bte...
Helpful
  Click to reply to this thread Reply


Hi

I've created two csv files:

## D:\Scripts\temp\logonInfo.csv
firstName,lastName,CN,emailAddress,samaccountname
test,test,"cn=test,ou=users,dc=domain,dc=com",test@domain.com,test
test1,test1,"cn=test1,ou=users,dc=domain,dc=com",test1@domain.com,test1
test2,test2,"cn=test2,ou=users,dc=domain,dc=com",test2@domain.com,test2


## D:\Scripts\temp\logonInfo1.csv
firstName,lastName,employeeId,deptNumber,employeeType
test,test,123,345,10
test1,test1,234,456,11
test2,test2,345,567,12
test3,test3,345,678,13
test4,test4,345,789,14

# compare the objects

PS 26> $result = compare-object -referenceobject $(import-csv D:\Scripts\temp\logonInfo.csv) -differenceobject $(import-csv D:\Scripts\temp\logonInfo1.csv)

# export the results to a new csv file
PS 27> $result | foreach {$_.inputobject} | export-csv D:\Scripts\temp\new.csv -NoTypeInformation

# display the difference
PS 28> get-content D:\Scripts\temp\new.csv

firstName,lastName,employeeId,deptNumber,employeeType
test3,test3,345,678,13
test4,test4,345,789,14


Hope this helps


---
Shay Levy
$cript Fanatic
http://scriptolog.blogspot.com




Shay Levy [MVP]
http://blogs.microsoft.co.il/blogs/ScriptFanatic
PowerShell Toolbar
Dmitry Sotnikov


Posts: 1,151
Registered: 12/1/06
Re: compare two csvs and export match to another csv
Posted: May 18, 2008 11:50 AM   in response to: Shay Levy
Helpful
  Click to reply to this thread Reply

Also, beware of the -SyncWindow parameter. Compare-Object by default has this equal to 5 so if your files are not sorted and have more than 5 entries you will need to use a bigger sync window.

Dmitry


Shay Levy


Posts: 1,938
Registered: 1/31/08
Re: compare two csvs and export match to another csv
Posted: May 18, 2008 12:48 PM   in response to: Dmitry Sotnikov
 
  Click to reply to this thread Reply


Thanks!

Shay Levy [MVP]
http://blogs.microsoft.co.il/blogs/ScriptFanatic
PowerShell Toolbar
frangwala@b2btech.com

Posts: 23
Registered: 4/13/08
Re: compare two csvs and export match to another csv
Posted: May 21, 2008 7:22 PM   in response to: frangwala@b2bte...
 
  Click to reply to this thread Reply

Shay,

Thanks for the script.

I see that the output is the difference between both files. The output also lists the fields from the second csv  which don't match the records from the first csv.

Here's the output I'm trying to get from both these files

firstName,lastName,CN,emailAddress,samaccountname,employeeId,deptNumber,employeeType
test,test,"cn=test,ou=users,dc=domain,dc=com",test@domain.com,test,123,345,10
test1,test1,"cn=test1,ou=users,dc=domain,dc=com",test1@domain.com,test1,234,456,11
test2,test2,"cn=test2,ou=users,dc=domain,dc=com",test2@domain.com,test2,345,567,12

Any help will be much appreciated.

BTW, how do you increase the sync window? What's the max that I can set it to and what's the impact on the system as far as performance is concerned?

Thank you.

Firoz


Dmitry Sotnikov


Posts: 1,151
Registered: 12/1/06
Re: compare two csvs and export match to another csv
Posted: May 21, 2008 10:32 PM   in response to: frangwala@b2bte...
 
  Click to reply to this thread Reply

Changing the syncwindow is easy - just provide the one you need as a parameter:

Compare-Object -SyncWindow 1000

I believe you can also use the -Property parameter to specify the name of the column you want to use as the key.

Dmitry


Shay Levy


Posts: 1,938
Registered: 1/31/08
Re: compare two csvs and export match to another csv
Posted: May 22, 2008 2:23 AM   in response to: frangwala@b2bte...
 
  Click to reply to this thread Reply


Hi

There must better/efficient way to get what you want... here's what I came up with so far:


$csv1 = import-csv D:\Scripts\temp\logonInfo1.csv
$csv2 = import-csv D:\Scripts\temp\logonInfo2.csv

$result1 = $(compare-object $csv1 $csv2 -IncludeEqual | where {$_.SideIndicator -eq "=="} | foreach {$_.InputObject})
$result2 = $(compare-object $csv2 $csv1 -IncludeEqual | where {$_.SideIndicator -eq "=="} | foreach {$_.InputObject})

$members1 = $csv1 | gm -mem  NoteProperty |  foreach {$_.name}
$members2 = $csv2 | gm -mem  NoteProperty |  foreach {$_.name}

$newMembers = $(compare-object $members1 $members2 | where {$_.SideIndicator -eq "=>"}  | foreach {$_.InputObject})

for($i=0; $i -lt $newMembers.length; $i++){
 $prop = $newMembers[$i]
 $result1 | add-member noteproperty $prop $null
 
 for($x=0; $x -lt $result1.length; $x++){
  $newMembers | foreach {
   $result1[$x].$prop = $result2[$x].$prop
  }
 }

}


$result1 | export-csv D:\Scripts\temp\new.csv -NoTypeInformation
get-content D:\Scripts\temp\new.csv


# and here's the output

firstName,lastName,CN,emailAddress,samaccountname,deptNumber,employeeId,employeeType
test,test,"cn=test,ou=users,dc=domain,dc=com",test@domain.com,test,345,123,10
test1,test1,"cn=test1,ou=users,dc=domain,dc=com",test1@domain.com,test1,456,234,11
test2,test2,"cn=test2,ou=users,dc=domain,dc=com",test2@domain.com,test2,567,345,12



Remember to take into count the -syncWindow as Dmitry's suggested, this is just a small test on small csv files.

Hope this helps


---
Shay Levy
$cript Fanatic
http://scriptolog.blogspot.com




Shay Levy [MVP]
http://blogs.microsoft.co.il/blogs/ScriptFanatic
PowerShell Toolbar
frangwala@b2btech.com

Posts: 23
Registered: 4/13/08
Re: compare two csvs and export match to another csv
Posted: Jun 6, 2008 1:23 PM   in response to: frangwala@b2bte...
 
  Click to reply to this thread Reply

Shay,

Thanks for the script.I'm sorry I didn't reply sooner.

You mentioned that there must be a more efficient way to compare the two files and output the match.

I am not married to the compare cmdlet.

How would you approach this requirement?

Thank you.

Firoz


Shay Levy


Posts: 1,938
Registered: 1/31/08
Re: compare two csvs and export match to another csv
Posted: Jun 6, 2008 4:27 PM   in response to: frangwala@b2bte...
 
  Click to reply to this thread Reply



Hi Firoz,

The reason I said "more efficient way" is because the sulotion seemed 'awkward' to me (+ I was thinking in loud, so to speak). It was also an open invitation for others to try it out and suggest their own solution. 

Did it work for you?

Shay Levy [MVP]
http://blogs.microsoft.co.il/blogs/ScriptFanatic
PowerShell Toolbar
frangwala@b2btech.com

Posts: 23
Registered: 4/13/08
Re: compare two csvs and export match to another csv
Posted: Jun 10, 2008 9:39 AM   in response to: Shay Levy
 
  Click to reply to this thread Reply

Shay,

I did try your method and it did give me the output, but as you mentioned, I'm trying to figure out something more efficient.

Maybe someone can give us a better idea of how to get this resolved.

Thank you very much for your help.


Shay Levy


Posts: 1,938
Registered: 1/31/08
Re: compare two csvs and export match to another csv
Posted: Jun 10, 2008 12:20 PM   in response to: frangwala@b2bte...
 
  Click to reply to this thread Reply



I'll give it another try and post back.

Shay Levy [MVP]
http://blogs.microsoft.co.il/blogs/ScriptFanatic
PowerShell Toolbar
Shay Levy


Posts: 1,938
Registered: 1/31/08
Re: compare two csvs and export match to another csv
Posted: Jun 17, 2008 7:12 AM   in response to: Shay Levy
 
  Click to reply to this thread Reply

Hi,

I made some tests and found minor changes to the original code I posted. The process logic stays the same.


-Shay

Shay Levy [MVP]
http://blogs.microsoft.co.il/blogs/ScriptFanatic
PowerShell Toolbar
Legend
MVP: 2501 + pts
Guru: 2001 - 2500 pts
Expert: 751 - 2000 pts
Enthusiast: 31 - 750 pts
Novice: 0 - 30 pts
Moderators
Helpful answer (5 pts)
Answered (10 pts)

Point your RSS reader here for a feed of the latest messages in all forums