BOHB tune library template (included example)

Related tags

Miscellaneoustunebohb
Overview

BOHB-template

실행 방법

python main.py

2021-10-10 기준

  • tf keras 버전 (tunecallback 방식) 완료
  • tf gradienttape 버전 (train_iteration 방식) 완료
  • pytorch 버전은 구현 준비중

방법 소개

1. 라이브러리 설치

ray 및 tune 설치

pip install ray[tune]

bohb의 경우

pip install hpbandster ConfigSpace

2. 스케쥴러의 max_t 설정

  • keras나 torch의 epoch은 max_t 값을 기준으로 terminate된다. HB 계열일 경우 eta (reduction_factor)도 잘 고려해서 설정하자.

3. 샘플 수 설정

  • tune.run의 샘플 수는 config에서 정한 조합 중 seed에 맞춰 설정한 수에 맞춰 선별된다.

선별된 샘플은 다음과 같이 pending 되고, running 된다.

+-----------------------+----------+-------+--------------+-----------+-----------+--------------+
| Trial name            | status   | loc   | activation   |   neuron1 |   neuron2 | optimizers   |
|-----------------------+----------+-------+--------------+-----------+-----------+--------------|
| objective_dd8c7_00000 | PENDING  |       | tanh         |        43 |        50 | adam         |
| objective_dd8c7_00001 | PENDING  |       | relu         |        63 |        45 | adam         |
| objective_dd8c7_00002 | PENDING  |       | tanh         |        63 |        52 | rmsprop      |
+-----------------------+----------+-------+--------------+-----------+-----------+--------------+

4. stop 조건 설정

  • tune.run의 stop 항목의 t의 값에 따라 멈출 수 있다.

예를 들어

stop={
"mean_accuracy": 0.99, # 정확도가 0.99 이상일 경우 Terminate
"training_iteration": 1
},

이 경우 max_t가 아무리 길어도, trial이 모두 다 running되었을 때(1번), 1번으로 간주되어 종료된다.

Number of trials: 3/3 (3 TERMINATED)
+-----------------------+------------+-------+--------------+-----------+-----------+--------------+----------+--------+------------------+
| Trial name            | status     | loc   | activation   |   neuron1 |   neuron2 | optimizers   |      acc |   iter |   total time (s) |
|-----------------------+------------+-------+--------------+-----------+-----------+--------------+----------+--------+------------------|
| objective_dd8c7_00000 | TERMINATED |       | tanh         |        43 |        50 | adam         | 0.938278 |      1 |          5.3518  |
| objective_dd8c7_00001 | TERMINATED |       | relu         |        63 |        45 | adam         | 0.942333 |      1 |          5.15036 |
| objective_dd8c7_00002 | TERMINATED |       | tanh         |        63 |        52 | rmsprop      | 0.94     |      1 |          4.28608 |
+-----------------------+------------+-------+--------------+-----------+-----------+--------------+----------+--------+------------------+

5. BOHB의 max_concurrent 및 cpu 값을 잘 활용

  • 병렬적으로 처리하게 되어 최적화 속도가 개선될 수 있다.

6. 시각화

  • 실행 시 지정한 경로에 기록이 쌓이는데, tensorboard 명령어로 확인이 가능하다.
tensorboard --logdir bohb_results/

image

image

Owner
Seungwoo Han
ML Engineer, MSc | @BCML-KW
Seungwoo Han
A basic layout of atm working of my local database

Software for working Banking service 😄 This project was developed for Banking service. mysql server is required To have mysql server on your system u

satya 1 Oct 21, 2021
A person does not exist image bot

A person does not exist image bot

Fayas Noushad 3 Dec 12, 2021
The purpose of this script is to bypass disablefund, provide some useful information, and dig the hook function of PHP extension.

The purpose of this script is to bypass disablefund, provide some useful information, and dig the hook function of PHP extension.

Firebasky 14 Aug 02, 2021
Apache Superset out of box version(Windows 64-bit)

superset_app Apache Superset out of box version (Windows 64bit) prepare job download 3 files python-3.8.10-embed-amd64.zip get-pip.py python_geohash‑0

Steven Lee 9 Oct 02, 2022
Multi-Probe Attention for Semantic Indexing

Multi-Probe Attention for Semantic Indexing About This project is developed for the topic of COVID-19 semantic indexing. Directories & files A. The di

Jinghang Gu 1 Dec 18, 2022
Team Hash Brown Science4Cast Submission

Team Hash Brown Science4Cast Submission This code reproduces Team Hash Brown's (@princengoc, @Xieyangxinyu) best submission (ee5a) for the competition

3 Feb 02, 2022
Xbps-install wrapper written in Python that doesn't care about case sensitiveness and package versions

xbi Xbps-install wrapper written in Python that doesn't care about case sensitiveness and package versions. Description This Python script can be easi

Emanuele Sabato 5 Apr 11, 2022
Android Blobs Organizer

Android Blobs Organizer

Sebastiano Barezzi 96 Jan 02, 2023
An example project which contains the Unity components necessary to complete Navigation2's SLAM tutorial with a Turtlebot3, using a custom Unity environment in place of Gazebo.

Navigation 2 SLAM Example This example provides a Unity Project and a colcon workspace that, when used together, allows a user to substitute Unity as

Unity Technologies 183 Jan 04, 2023
Life Dynamics for python

Daphny_counter run command must be like this: /usr/bin/python3 /home/nmakagonov/Daphny/daphny_counter/Daphny_counter.py -o /home/nmakagonov/Daphny/out

12 Sep 05, 2022
🌈Python cheatsheet for all standard libraries(Continuously Updated)

Python Standard Libraries Cheatsheet Depend on Python v3.9.8 All code snippets have been tested to ensure they work properly. Fork me on GitHub. 中文 En

nick 12 Dec 27, 2022
Unified Distributed Execution

Unified Distributed Execution The framework supports multiple execution backends: Ray, Dask, MPI and MultiProcessing. To run tests you need to install

17 Dec 25, 2022
Hy - A dialect of Lisp that's embedded in Python

Hy Lisp and Python should love each other. Let's make it happen. Hy is a Lisp dialect that's embedded in Python. Since Hy transforms its Lisp code int

Hy Society 4.4k Jan 02, 2023
An alternative app for core Armoury Crate functions.

NoROG DISCLAIMER: Use at your own risk. This is alpha-quality software. It has not been extensively tested, though I personally run it daily on my lap

12 Nov 29, 2022
This bot uploads telegram files to MixDrop.co,File.io.

What is about this bot ? This bot uploads telegram files to MixDrop.co, File.io. Usage: Send any file, and the bot will upload it to MixDrop.co, File.

Abhijith NT 3 Feb 26, 2022
Python implementation of Newton's Fractal

Newton's Fractal Animates Newton's fractal between two polynomials of the same order. Inspired by this video by 3Blue1Brown. Example fractals can be f

Jaime Liew 10 Aug 04, 2022
GDSC UIET KUK 📍 , welcomes you all to this amazing event where you will be introduced to the world of coding 💻 .

GDSC UIET KUK 📍 , welcomes you all to this amazing event where you will be introduced to the world of coding 💻 .

Google Developer Student Club UIET KUK 9 Mar 24, 2022
List Less Than Ten with python

List Less Than Ten with python

PyLaboratory 0 Feb 07, 2022
Comprehensive OpenAPI schema generator for Django based on pydantic

🗡️ Djagger Automated OpenAPI documentation generator for Django. Djagger helps you generate a complete and comprehensive API documentation of your Dj

13 Nov 26, 2022
Hydralit package is a wrapping and template project to combine multiple independant Streamlit applications into a multi-page application.

Hydralit The Hydralit package is a wrapping and template project to combine multiple independant (or somewhat dependant) Streamlit applications into a

Jackson Storm 108 Jan 08, 2023