Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Comparing for exactness is one challenge, but just a start - identifying what has changed is a bigger challenge. And that's where the world of file/database comparators come in. A somewhat niche but easy-to-use example of one with lots of options that others have sought to mimic in open source is SAS's PROC COMPARE, which can compare SAS datasets or other vendor's database tables (which SAS can effectively treat as if they're SAS datasets).

SAS COMPARE Procedure Example 1: Producing a Complete Report of the Differences

  proc compare base=proclib.one compare=proclib.two printall;
https://documentation.sas.com/doc/en/pgmsascdc/9.4_3.5/proc/...

_____

DataComPy (open-source python software developed by Capital One)

DataComPy is a package to compare two Pandas DataFrames. Originally started to be something of a replacement for SAS’s PROC COMPARE for Pandas DataFrames with some more functionality than just Pandas.DataFrame.equals(Pandas.DataFrame) (in that it prints out some stats, and lets you tweak how accurate matches have to be).

  from io import StringIO
  import pandas as pd
  import datacompy

  compare = datacompy.Compare(
      df1,
      df2,
      join_columns='acct_id',  #You can also specify a list of columns
      abs_tol=0, #Optional, defaults to 0
      rel_tol=0, #Optional, defaults to 0
      df1_name='Original', #Optional, defaults to 'df1'
      df2_name='New' #Optional, defaults to 'df2'
      )
  compare.matches(ignore_extra_columns=False)
  # False

  # This method prints out a human-readable report summarizing and sampling differences
  print(compare.report())
https://capitalone.github.io/datacompy/


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: