Similarity.Simhash (Similarity v0.5.1)

Copy Markdown View Source

Simhash string similarity algorithm. Description of Simhash

iex> Similarity.simhash("Barna", "Kovacs")
0.59375

iex> Similarity.simhash("Austria", "Australia")
0.65625

Summary

Functions

Returns the Hamming distance between the left and right hash, given as lists of bits.

Returns the hash for the given string and hash_function in the given return_type.

Calculates the similarity between the left and right string, using Simhash. Returns a float representing similarity between left and right strings.

Functions

hamming_distance(left, right, acc \\ 0)

(since 0.1.1)

Returns the Hamming distance between the left and right hash, given as lists of bits.

Raises ArgumentError when the hashes have different lengths.

Examples

iex> Similarity.Simhash.hamming_distance([1, 1, 0, 1, 0], [0, 1, 1, 1, 0])
2

hash(string, options)

(since 0.1.1)
@spec hash(
  String.t(),
  keyword()
) :: [0 | 1] | integer() | binary()

Returns the hash for the given string and hash_function in the given return_type.

Options

  • :ngram_size - defaults to 3
  • :hash_function - defaults to :siphash, available options are :siphash, :md5, :sha256
  • :return_type - defaults to :list, available options are :list, :int64_unsigned, :int64_signed, :binary

The return types :int64_unsigned and :int64_signed are only available for the :siphash hash function.

Raises ArgumentError for invalid options, incompatible hash and return types, or strings shorter than :ngram_size.

Examples

Similarity.Simhash.hash("alma korte", [])
[1, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, ...]

iex> Similarity.Simhash.hash("alma korte", ngram_size: 3, hash_function: :siphash, return_type: :int64_unsigned)
15012197954348909067

iex> Similarity.Simhash.hash("alma korte", ngram_size: 3, hash_function: :siphash, return_type: :int64_signed)
-3434546119360642549

hash_similarity(left, right, hash_length)

(since 0.1.1)

similarity(left, right, options \\ [])

(since 0.1.1)
@spec similarity(String.t(), String.t(), keyword()) :: float()

Calculates the similarity between the left and right string, using Simhash. Returns a float representing similarity between left and right strings.

Options

  • :ngram_size - defaults to 3
  • :hash_function - defaults to :siphash, available options are :siphash, :md5, :sha256

Raises ArgumentError for invalid options or strings shorter than :ngram_size. The :return_type option is only supported by hash/2.

Examples

iex> Similarity.simhash("khan academy", "khan academia")
0.890625

iex> Similarity.simhash("khan academy", "academy khan", ngram_size: 1)
1.0