User Manual for py_entitymatching

This document explains how to install and use the package. To contribute to the package, see the project website, section “For Contributors and Developers”.

Contents

How To Guide

• How to Guide To Do Entity Matching

User Manual

• What is New?
• Installation
  • Requirements
  • Platforms
  • Dependencies
  • Installing Using conda
  • Installing Using pip
  • Installing from Source Distribution
• Overview of Supported EM Processes
  • Supported EM Problem Scenarios
  • Two Fundamental Steps in the EM Process: Blocking and Matching
  • Supported EM Workflows
  • The Development and Production Stages
  • The Focus of the Current Package
• Guides
  • A Quick Tour of Jupyter Notebook
  • End-to-End EM Workflows
  • Stepwise Guides
• Data Structures
  • Storing Tables Using Pandas Dataframes
  • Storing Metadata Using a Catalog
  • Summary
• Steps of Supported EM Workflows
  • Reading the CSV Files from Disk
  • Down Sampling
  • Specifying Blockers and Performing Blocking
    • Types of Blockers and Blocker Hierarchy
    • Built-In Blockers
    • Blackbox Blockers
    • Rule-Based Blockers
    • Combining Multiple Blockers
  • Creating Features for Blocking
    • Available Tokenizers and Similarity Functions
    • Obtaining Tokenizers and Similarity Functions
    • Obtaining Attribute Types and Correspondences
    • Getting a Set of Features
    • Adding/Removing Features
    • Summary of the Manual Feature Generation Process
    • Ways to Edit the Manual Feature Generation Process
    • Generating Features Automatically
  • Debugging Blocking
  • Sampling
  • Labeling
  • Splitting Labeled Data into Training and Testing Sets
  • Creating Features for Matching
    • Creating the Features Manually
    • Creating the Features Automatically
  • Extracting Feature Vectors
  • Imputing Missing Values
  • Specifying ML-Matchers and Performing Matching
    • Creating Learning-Based Matchers
    • Training Learning-Based Matchers
    • Applying Learning-Based Matchers
  • Selecting a ML-Matcher
  • Debugging ML-Matchers
    • Debugging Using the GUI
    • Debugging Using the Command Line
    • Impact of Imputing Missing Values
  • Evaluating the Matching Output
• Miscellaneous
  • CSV Format
  • Metadata File Format
  • Writing a Dataframe to Disk Along With Its Metadata
  • Writing/Reading Other Types of py_entitymatching Objects

API Reference

• Overview of Command Organization
• Commands in py_entitymatching
  • Reading and Writing Data
  • Loading and Saving Objects
  • Handling Metadata
  • Downsampling
  • Blocking
  • Debugging Blocker Output
  • Combining Blocker Outputs
  • Sampling
  • Labeling
  • Handling Features
    • Creating the Features Automatically
    • Creating the Features Manually
    • Adding Features to Feature Table
    • Extracting Feature Vectors
    • Imputing Missing Values
    • Supported Similarity Functions
    • Supported Tokenizers
  • Matching
    • Splitting Data into Train and Test
    • Supported Matchers
    • Selecting Matcher
  • Debugging Matcher
  • Evaluating the Matching Output

Indices and tables

• Index
• Module Index
• Search Page