User Manual for py_entitymatching

This document explains how to install, use, and contribute to the package.

Contents

How To Guide

• How to Guide To Do Entity Matching

User Manual

• What is New?
• Installation
  • Requirements
  • Platforms
  • Dependencies
  • C Compiler Required
  • Installing Using pip
  • Installing from Source Distribution
• Overview of Supported EM Processes
  • Supported EM Problem Scenarios
  • Two Fundamental Steps in the EM Process: Blocking and Matching
  • Supported EM Workflows
  • The Development and Production Stages
  • The Focus of the Current Package
• Guides
  • A Quick Tour of Jupyter Notebook
  • End-to-End EM Workflows
  • Stepwise Guides
• Data Structures
  • Storing Tables Using Pandas Dataframes
  • Storing Metadata Using a Catalog
  • Summary
• Steps of Supported EM Workflows
  • Reading the CSV Files from Disk
  • Down Sampling
  • Profiling Data
    • Example Usage
  • Data Exploration
    • OpenRefine
    • Pandastable
  • Specifying Blockers and Performing Blocking
    • Types of Blockers and Blocker Hierarchy
    • Built-In Blockers
    • Blackbox Blockers
    • Rule-Based Blockers
    • Combining Multiple Blockers
  • Creating Features for Blocking
    • Available Tokenizers and Similarity Functions
    • Obtaining Tokenizers and Similarity Functions
    • Obtaining Attribute Types and Correspondences
    • Getting a Set of Features
    • Adding/Removing Features
    • Summary of the Manual Feature Generation Process
    • Ways to Edit the Manual Feature Generation Process
    • Generating Features Automatically
  • Debugging Blocking
  • Sampling
  • Labeling
  • Splitting Labeled Data into Training and Testing Sets
  • Creating Features for Matching
    • Creating the Features Manually
    • Creating the Features Automatically
  • Extracting Feature Vectors
  • Imputing Missing Values
  • Specifying Matchers and Performing Matching
    • ML-Matchers
    • Rule-Based Matchers
  • Selecting a ML-Matcher
  • Debugging ML-Matchers
    • Debugging Using the GUI
    • Debugging Using the Command Line
    • Impact of Imputing Missing Values
  • Combining Predictions from Multiple Matchers
  • Using Triggers to Update Matching Results
    • Match Triggers
  • Evaluating the Matching Output
• Miscellaneous
  • CSV Format
  • Metadata File Format
  • Writing a Dataframe to Disk Along With Its Metadata
  • Writing/Reading Other Types of py_entitymatching Objects

API Reference

• Overview of Command Organization
• Commands in py_entitymatching
  • Reading and Writing Data
  • Loading and Saving Objects
  • Handling Metadata
  • Downsampling
  • Data Exploration
  • Blocking
  • Debugging Blocker Output
  • Combining Blocker Outputs
  • Sampling
  • Labeling
  • Handling Features
    • Creating the Features Automatically
    • Creating the Features Manually
    • Adding Features to Feature Table
    • Extracting Feature Vectors
    • Imputing Missing Values
    • Supported Similarity Functions
    • Supported Tokenizers
  • Matching
    • Splitting Data into Train and Test
    • Supported Matchers
    • Selecting Matcher
  • Debugging Matcher
  • Triggers
  • Evaluating the Matching Output
• Experimental Commands
  • Commands Implemented Using Dask
    • Downsampling
    • Blocking
    • Extracting Feature Vectors
    • ML-Matchers
  • Tuners for the Dask-based Commands
    • Downsampling
    • Overlap Blocker
  • Matcher Combiner

How to Contribute

• Contributing to py_entitymatching
  • Where to start?
  • Bug reports and enhancement requests
  • Working with the code
    • Version control, Git, and GitHub
    • Getting started with Git
    • Forking
    • Creating a branch
    • Creating a development environment
  • Contributing to the documentation
    • About the py_entitymatching documentation
    • How to build the py_entitymatching documentation
  • Contributing to the code base
    • Code standards
    • Writing tests
  • Contributing your changes to py_entitymatching
    • Committing your code
    • Combining commits
    • Pushing your changes
    • Review your code
    • Finally, make the pull request
    • Delete your merged branch (optional)

Indices and tables

• Index

• Module Index

• Search Page