Vectorizers for a range of different data types

I've implemented SignatureVectorizer, which returns the path signatures for a collection of paths.

This vectorizer essentially wraps the iisignature package such that it fits into the standard sklearn style fit_transform pipeline. While it does require iisignature, the imports are written such that the rest of the library can still be used if the user does not have iisignature installed.

For more details on the path signature technique, I've found this paper quite instructive: A Primer on the Signature Method in Machine Learning (Chevyrev, I.)

Add Lempel-Ziv and Byte Pair Encodign based vectorizers allowing for vectorization of non-tokenized strings.

Also includes a basic outline of a distribution vectorizer, but this may require something more powerful than pomegranate.

This is essentially just a Taken's embedding, but gives it tools to work with classical time-series data. I tried to have a reasonable range of options/flexibility in how you sample windows, but would welcome any suggestions for further options. In principle it might be possible to "learn" good window parameters from the input data (right now fit does nothing beyond verifying parameters) but I don't quite know what the right way to do that would be exactly.

It turns our that our data-prep tricks prior and after SVDs for count-based data are generically useful. I tried applying them to info-weighted bag-of-words on 20-newsgroups instead of just a straight SVD and ...

I decided this is going to be too generically useful not to turn into a standard transformer. In due course we can potentially use this as part of a pipeline for word vectors instead of the reduce_dimension method we have now.

Hello and thank you for a great package!

I was wondering whether (and how) ApproximateWassersteinVectorizer would be able to scale up to terabyte-order data or whether you had any pointers for dealing with data of that scale.

Started testing most of the vectorizers at a very basic level. Began a testing module for the distances. I would welcome help writing extra tests for those.

The big refactoring is basically complete. The only draw back is that your kernel functions (if you are doing multiple window sizes) need to all be the same. This is a numba list of functions issue. It will take some work to add this back in I think... it can be another PR.

If a user passes in 'F' layout 2D arrays for vectors it can cause an error from numba that is hard for a user to decode. Remedy this by simply ensuring all the arrays being added are 'C' layout.

Added a new class for vectorizing data in the form of row_name, column_name, count triples.
Added some unit tests. Documentation and an expanded functionality to come at a later date.

For now just a couple of simple test kernels; one for count time series anomaly detection, and another to work with time interval time series anomaly detection, which is similar. Enough basics to test out that the named function code-path works.

Add in a SequentialDifferenceTransformer as a simple to use approach to generating inter-arrival times or similar.

Allow the SlidingWindowTransformer to take function kernels (numba jitted obviously) for much greater flexibility in how kernels perform.

We seem to crash kernels when we pass a sparse matrix with an empty column to InformationWeight transformers fit transform.

It comes up when we've got a fixed token dictionary but our training data is missing some of our vocabulary:

vect = NgramVectorizer(token_dictionary=token_dictionary).fit_transform(eventid_list)
vect1 = InformationWeightTransformer().fit_transform(vect)

Hello!

Not sure this is actually a bug, but it seemed a bit odd so I figured I'd report it. I just trained a vectorizer using the following:

%%time
word_vectorizer = vectorizers.TokenCooccurrenceVectorizer(
    min_document_occurrences=2,
    window_radii=20,          
    window_functions='variable',
    kernel_functions='geometric',            
    n_iter = 3,
    normalize_windows=True,
).fit(subset['tokenized'])

When I try to display or print it in a Jupyter Notebook cell I get the following error (actually repeated a few times):

AttributeError: 'TokenCooccurrenceVectorizer' object has no attribute 'coo_initial_memory'

We would greatly appreciate the contribution of tutorial notebooks applying vectorizer tools to diverse or interesting datasets

Hi! I was wondering what sort of contributions/tutorials you were looking for, or whether you had more concrete contribution wishes.

I tried installing vectorizers on a fresh pyenv and I got this error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "vectorizers-master/vectorizers/__init__.py", line 1, in <module>
    from .token_cooccurrence_vectorizer import TokenCooccurrenceVectorizer
  File "vectorizers/token_cooccurrence_vectorizer.py", line 1, in <module>
    from .ngram_vectorizer import ngrams_of
  File "vectorizers/ngram_vectorizer.py", line 2, in <module>
    import numba
  File "numba-0.54.1-py3.9-macosx-11.3-x86_64.egg/numba/__init__.py", line 19, in <module>
    from numba.core import config
  File "numba-0.54.1-py3.9-macosx-11.3-x86_64.egg/numba/core/config.py", line 16, in <module>
    import llvmlite.binding as ll
  File "llvmlite-0.37.0-py3.9.egg/llvmlite/binding/__init__.py", line 4, in <module>
    from .dylib import *
  File "llvmlite-0.37.0-py3.9.egg/llvmlite/binding/dylib.py", line 3, in <module>
    from llvmlite.binding import ffi
  File "llvmlite-0.37.0-py3.9.egg/llvmlite/binding/ffi.py", line 191, in <module>
    raise OSError("Could not load shared object file: {}".format(_lib_name))
OSError: Could not load shared object file: libllvmlite.dylib

This was resolvable for me by uninstalling and reinstalling libllvmlite & numba after ensuring numpy<1.21, e.g.

pip install numpy==1.20
pip uninstall libllvmlite
pip uninstall numba
pip install libllvmlite
pip install numba

This is obviously not a big deal, but in case others bump this, maybe a ref in the README can save them a google. 🤷

Most things should just pickle, however numba based attributes (functions and types such as numba.typed.List) need special handling with __getstate__ and __setstate__ methods to handle conversion of numba bits and pieces.

To aid in this we should have tests that all classes can be pickled and unpickled successfully. This will help highlight any shortcomings.

See also #62

Have you thought about adding serialization/saving to disk? At the very least, would you point out what to store for, say, the ngram vectorizer?

Thank you very much for the examples on training word/document vectors using this and comparing to USE! Giving it a try now on my data and it looks pretty good! Thank you!