Hypernets: A General Automated Machine Learning framework to simplify the development of End-to-end AutoML toolkits in specific domains.

Last update: Dec 23, 2022

Overview

Hypernets

We Are Hiring！

Dear folks, we are opening several precious positions based in Beijing both for professionals and interns avid in AutoML/NAS, please send your resume/cv to [email protected]. (Application deadline: TBD.)

Hypernets: A General Automated Machine Learning Framework

Hypernets is a general AutoML framework, based on which it can implement automatic optimization tools for various machine learning frameworks and libraries, including deep learning frameworks such as tensorflow, keras, pytorch, and machine learning libraries like sklearn, lightgbm, xgboost, etc. We introduced an abstract search space representation, taking into account the requirements of hyperparameter optimization and neural architecture search(NAS), making Hypernets a general framework that can adapt to various automated machine learning needs.

Overview

Conceptual Model

Illustration of the Search Space

Installation

Install Hypernets with pip command:

pip install hypernets

Optional, to run Hypernets in JupyterLab notebooks, install Hypernets and JupyterLab with command:

pip install hypernets[notebook]

Optional, to run Hypernets in distributed Dask cluster, install Hypernets with command:

pip install hypernets[dask]

Optional, to support dataset with simplified Chinese in feature generation, install jieba package before run Hypernets, or install Hypernets with command:

pip install hypernets[zhcn]

Optional, install all Hypernets components and dependencies with one command:

pip install hypernets[all]

Verify installation:

python -m hypernets.examples.smoke_testing

Hypernets related projects

HyperGBM: A full pipeline AutoML tool integrated various GBM models.
HyperDT/DeepTables: An AutoDL tool for tabular data.
HyperKeras: An AutoDL tool for Neural Architecture Search and Hyperparameter Optimization on Tensorflow and Keras.
Cooka: Lightweight interactive AutoML system.
Hypernets: A general automated machine learning framework.

Documents

Neural Architecture Search

DataCanvas

Hypernets is an open source project created by DataCanvas.

Comments

window pip fail.

ERROR: Complete output from command 'D:\Anaconda\Anaconda3\python.exe' -u -c 'import setuptools, tokenize;file='"'"'C:\Users\John\AppData\Local\Temp\pip-install-4hgmcjg1\python-geohash\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\John\AppData\Local\Temp\pip-wheel-gijlszf9' --python-tag cp37: ERROR: running bdist_wheel running build running build_py creating build creating build\lib.win-amd64-3.7 copying geohash.py -> build\lib.win-amd64-3.7 copying quadtree.py -> build\lib.win-amd64-3.7 copying jpgrid.py -> build\lib.win-amd64-3.7 copying jpiarea.py -> build\lib.win-amd64-3.7 running build_ext building '_geohash' extension error: Microsoft Visual C++ 14.0 is required. Get it with "Microsoft Visual C++ Build Tools": https://visualstudio.microsoft.com/downloads/

ERROR: Failed building wheel for python-geohash Running setup.py clean for python-geohash Failed to build python-geohash ERROR: spyder 3.3.6 requires pyqt5<5.13; python_version >= "3", which is not installed. ERROR: spyder 3.3.6 requires pyqtwebengine<5.13; python_version >= "3", which is not installed. ERROR: distributed 2021.11.1 has requirement dask==2021.11.1, but you'll have dask 2.1.0 which is incompatible. ERROR: woodwork 0.9.0 has requirement pandas>=1.3.0, but you'll have pandas 1.2.3 which is incompatible. ERROR: featuretools 1.2.0 has requirement dask[dataframe]>=2021.10.0, but you'll have dask 2.1.0 which is incompatible. ERROR: featuretools 1.2.0 has requirement pandas<2.0.0,>=1.3.0, but you'll have pandas 1.2.3 which is incompatible. Installing collected packages: psutil, pyyaml, tblib, distributed, woodwork, featuretools, python-geohash, hypernets Found existing installation: psutil 5.6.3 Uninstalling psutil-5.6.3: Successfully uninstalled psutil-5.6.3 Found existing installation: PyYAML 5.1.1 ERROR: Cannot uninstall 'PyYAML'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
Installation

opened by zhangxjohn 2
dataset about dsutils

an error about "from hypernets.frameworks.ml.datasets import dsutils", I think it should be "from deeptables.datasets import dsutils".
bug

opened by ninanxiaoguai 2
A suggestion for GreedyEnsemble

On line 107 of voting's code, i.e. if self.ensemble_size <= 0: size = predictions.shape[1]

Is it possible to replace it with self.ensemble_size <= 0 or predictions.shape[1] < self.ensemble_size: size = predictions.shape[1]?

opened by zhangxjohn 1
scorer & calc_score
Please make sure that this is a feature request.

System information

Hypernets version (you are using):

Are you willing to contribute it (Yes/No):

Describe the feature and the current behavior/state. I found that the user-defined evaluation metric function could not be set in the hypernets experiment, and calc_score() contained few metrics, such as no mape.

Will this change the current api? How?

Any Other info.
opened by zhangxjohn 1
Bump url-parse from 1.5.1 to 1.5.3 in /hypernets/hn_widget/js
Bumps url-parse from 1.5.1 to 1.5.3.

Commits

ad44493 [dist] 1.5.3

c798461 [fix] Fix host parsing for file URLs (#210)

201034b [dist] 1.5.2

2d9ac2c [fix] Sanitize only special URLs (#209)

fb128af [fix] Use 'null' as origin for non special URLs

fed6d9e [fix] Add a leading slash only if the URL is special

94872e7 [fix] Do not incorrectly set the slashes property to true

81ab967 [fix] Ignore slashes after the protocol for special URLs

ee22050 [ci] Use GitHub Actions

d2979b5 [fix] Special case the file: protocol (#204)

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 1
Bump tar from 6.1.0 to 6.1.10 in /hypernets/hn_widget/js
Bumps tar from 6.1.0 to 6.1.10.

Commits

188badd 6.1.10

23312ce drop dirCache for symlink on all platforms

4f1f4a2 6.1.9

875a37e fix: prevent path escape using drive-relative paths

b6162c7 fix: reserve paths properly for unicode, windows

3aaf19b fix: prune dirCache properly for unicode, windows

6a9c51d 6.1.8

dfc5923 fix: skip extract if linkpath is stripped entirely

575a511 fix: reserve paths case-insensitively

d61628c 6.1.7

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 1
Bump tar from 6.1.0 to 6.1.5 in /hypernets/hn_widget/js
Bumps tar from 6.1.0 to 6.1.5.

Commits

bd4691c 6.1.5

d694c4f ci: test on node 16

84acbd3 fix(unpack): fix hang on large file on open() fail

97c46fc fix(unpack): always resume parsing after an entry error

488ab8c chore: WriteEntry cleaner write() handling

be89aaf WriteEntry backpressure

ba73f5e chore: track fs state on WriteEntry class, not in arguments

bf69383 6.1.4

06cbde5 Avoid an unlikely but theoretically possible redos

0b78386 6.1.3

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 1
Bump dns-packet from 1.3.1 to 1.3.4 in /js
Bumps dns-packet from 1.3.1 to 1.3.4.

Commits

ebdf849 1.3.4

ac57872 move all allocUnsafes to allocs for easier maintenance

c64c950 1.3.3

0598ba1 fix .. in encodingLength

010aedb 1.3.2

0d0d593 backport encodingLength fix to v1

See full diff in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 0
Bump path-parse from 1.0.6 to 1.0.7 in /js
Bumps path-parse from 1.0.6 to 1.0.7.

Commits

See full diff in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 0
Bump url-parse from 1.5.1 to 1.5.3 in /js
Bumps url-parse from 1.5.1 to 1.5.3.

Commits

ad44493 [dist] 1.5.3

c798461 [fix] Fix host parsing for file URLs (#210)

201034b [dist] 1.5.2

2d9ac2c [fix] Sanitize only special URLs (#209)

fb128af [fix] Use 'null' as origin for non special URLs

fed6d9e [fix] Add a leading slash only if the URL is special

94872e7 [fix] Do not incorrectly set the slashes property to true

81ab967 [fix] Ignore slashes after the protocol for special URLs

ee22050 [ci] Use GitHub Actions

d2979b5 [fix] Special case the file: protocol (#204)

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 0

Releases(0.2.5.7)

0.2.5.7(Nov 24, 2022)

Source code(tar.gz)
Source code(zip)
0.2.5.6(Oct 8, 2022)

Source code(tar.gz)
Source code(zip)
0.2.5.5(Sep 5, 2022)

Source code(tar.gz)
Source code(zip)
0.2.5.4(May 31, 2022)

Source code(tar.gz)
Source code(zip)
0.2.5.3(Mar 25, 2022)

Source code(tar.gz)
Source code(zip)
0.2.5.2(Mar 8, 2022)

Source code(tar.gz)
Source code(zip)
0.2.5.1(Mar 2, 2022)

Source code(tar.gz)
Source code(zip)
0.2.5(Feb 28, 2022)

Source code(tar.gz)
Source code(zip)
0.2.3.2(Dec 16, 2021)

Fix requirements
Source code(tar.gz)
Source code(zip)
0.2.3(Aug 20, 2021)

Source code(tar.gz)
Source code(zip)
0.1.8(Mar 4, 2021)

Source code(tar.gz)
Source code(zip)
0.1.7(Feb 4, 2021)

Source code(tar.gz)
Source code(zip)
0.1.5(Nov 30, 2020)

Source code(tar.gz)
Source code(zip)
0.1.3(Sep 27, 2020)

Source code(tar.gz)
Source code(zip)
0.1.2(Sep 2, 2020)

Source code(tar.gz)
Source code(zip)
0.1.1(Jun 24, 2020)

Source code(tar.gz)
Source code(zip)

Owner

DataCanvas

We love data science!

GitHub Repository https://hypernets.readthedocs.io/

Steganography is the art of hiding the fact that communication is taking place, by hiding information in other information.

7 Nov 09, 2022

Simple, light-weight config handling through python data classes with to/from JSON serialization/deserialization.

Simple but maybe too simple config management through python data classes. We use it for machine learning.

67 Nov 29, 2022

李航《统计学习方法》复现

本项目复现李航《统计学习方法》每一章节的算法特点：笔记摘要：在每个文件开头都会有一些核心的摘要 pythonic：这里会用尽可能规范的方式来实现，包括编程风格几乎严格按照PEP8 循序渐进：前期的算法会更list的方式来做计算，可读性比较强，后期几乎完全为numpy.array的计算，并且辅助详

58 Oct 22, 2021

A game theoretic approach to explain the output of any machine learning model.

SHAP (SHapley Additive exPlanations) is a game theoretic approach to explain the output of any machine learning model. It connects optimal credit allo

18.2k Jan 02, 2023

A simple example of ML classification, cross validation, and visualization of feature importances

Simple-Classifier This is a basic example of how to use several different libraries for classification and ensembling, mostly with sklearn. Example as

2 Aug 25, 2022

AutoTabular automates machine learning tasks enabling you to easily achieve strong predictive performance in your applications.

AutoTabular AutoTabular automates machine learning tasks enabling you to easily achieve strong predictive performance in your applications. With just

2 Jun 26, 2022

Predicting Baseball Metric Clusters: Clustering Application in Python Using scikit-learn

Clustering Clustering Application in Python Using scikit-learn This repository contains the prediction of baseball metric clusters using MLB Statcast

2 Apr 18, 2022

Massively parallel self-organizing maps: accelerate training on multicore CPUs, GPUs, and clusters

Somoclu Somoclu is a massively parallel implementation of self-organizing maps. It exploits multicore CPUs, it is able to rely on MPI for distributing

239 Nov 10, 2022

ArviZ is a Python package for exploratory analysis of Bayesian models

ArviZ (pronounced "AR-vees") is a Python package for exploratory analysis of Bayesian models. Includes functions for posterior analysis, data storage, model checking, comparison and diagnostics

1.3k Jan 05, 2023

This machine-learning algorithm takes in data from the last 60 days and tries to predict tomorrow's price of any crypto you ask it.

Crypto-Currency-Predictor This machine-learning algorithm takes in data from the last 60 days and tries to predict tomorrow's price of any crypto you

6 Dec 04, 2022

Short PhD seminar on Machine Learning Security (Adversarial Machine Learning)

141 Dec 27, 2022

Meerkat provides fast and flexible data structures for working with complex machine learning datasets.

Meerkat makes it easier for ML practitioners to interact with high-dimensional, multi-modal data. It provides simple abstractions for data inspection, model evaluation and model training supported by

115 Dec 12, 2022

A webpage that utilizes machine learning to extract sentiments from tweets.

Tweets_Classification_Webpage The goal of this project is to be able to predict what rating customers on social media platforms would give to products

1 Dec 30, 2021

nn-Meter is a novel and efficient system to accurately predict the inference latency of DNN models on diverse edge devices

A DNN inference latency prediction toolkit for accurately modeling and predicting the latency on diverse edge devices.

241 Dec 26, 2022

Hypernets: A General Automated Machine Learning framework to simplify the development of End-to-end AutoML toolkits in specific domains.

Related tags

Overview

Hypernets

We Are Hiring！

Hypernets: A General Automated Machine Learning Framework

Overview

Conceptual Model

Illustration of the Search Space

Installation

Related Links

Hypernets related projects

Documents

Neural Architecture Search

DataCanvas

Comments

Releases(0.2.5.7)

0.2.5.7(Nov 24, 2022)

0.2.5.6(Oct 8, 2022)

0.2.5.5(Sep 5, 2022)

0.2.5.4(May 31, 2022)

0.2.5.3(Mar 25, 2022)

0.2.5.2(Mar 8, 2022)

0.2.5.1(Mar 2, 2022)

0.2.5(Feb 28, 2022)

0.2.3.2(Dec 16, 2021)

0.2.3(Aug 20, 2021)

0.1.8(Mar 4, 2021)

0.1.7(Feb 4, 2021)

0.1.5(Nov 30, 2020)

0.1.3(Sep 27, 2020)

0.1.2(Sep 2, 2020)

0.1.1(Jun 24, 2020)

Owner

DataCanvas

Steganography is the art of hiding the fact that communication is taking place, by hiding information in other information.

Simple, light-weight config handling through python data classes with to/from JSON serialization/deserialization.

李航《统计学习方法》复现

A game theoretic approach to explain the output of any machine learning model.

A simple example of ML classification, cross validation, and visualization of feature importances

AutoTabular automates machine learning tasks enabling you to easily achieve strong predictive performance in your applications.

Predicting Baseball Metric Clusters: Clustering Application in Python Using scikit-learn

Massively parallel self-organizing maps: accelerate training on multicore CPUs, GPUs, and clusters

ArviZ is a Python package for exploratory analysis of Bayesian models

This machine-learning algorithm takes in data from the last 60 days and tries to predict tomorrow's price of any crypto you ask it.

Short PhD seminar on Machine Learning Security (Adversarial Machine Learning)

Meerkat provides fast and flexible data structures for working with complex machine learning datasets.

A webpage that utilizes machine learning to extract sentiments from tweets.

nn-Meter is a novel and efficient system to accurately predict the inference latency of DNN models on diverse edge devices

Apache Spark & Python (pySpark) tutorials for Big Data Analysis and Machine Learning as IPython / Jupyter notebooks

A Microsoft Azure Web App project named Covid 19 Predictor using Machine learning Model

Tutorial for Decision Threshold In Machine Learning.

A Time Series Library for Apache Spark

Pydantic based mock data generation

This handbook accompanies the course: Machine Learning with Hung-Yi Lee