Utils to quickly evaluate many 🤗 models on the GLUE tasks

Last update: Dec 22, 2021

Related tags

Overview

Utils to quickly evaluate many 🤗 models on the GLUE tasks

Setup

wget https://raw.githubusercontent.com/huggingface/transformers/master/examples/pytorch/text-classification/run_glue.py
conda env create -f environment.yaml
conda activate glue-utils

Evaluate multiple models on every glue task:

python runner.py \
    path/to/model_1/ \
    path/to/model_2/ \
    path/to/model_3/ \
    --gpu_idx 1

This will store results in path/to/model_N/glue/[task_name]/, for each model, for each task.

To quickly gather all results, run:

python gather_scores.py \
    path/to/model_1/glue/ \
    path/to/model_2/glue/ \
    path/to/model_3/glue/

Owner

Przemyslaw K. Joniak

Grad student at The University of Tokyo

GitHub Repository

Jack Morgan's Advent of Code Solutions

Advent-of-Code Jack Morgan's Advent of Code Solutions Usage Run . initiate.sh year day To initiate a day. This sets up a template python file, and pul

1 Dec 10, 2021

Repo to store back end infrastructure for Message in a Bottle

Message in a Bottle Backend API RESTful API for Message in a Bottle frontend application consumption. About The Project • Tools Used • Local Set Up •

4 Dec 05, 2021

The tool helps to find hidden parameters that can be vulnerable or can reveal interesting functionality that other hunters miss.

The tool helps to find hidden parameters that can be vulnerable or can reveal interesting functionality that other hunters miss. Greater accuracy is achieved thanks to the line-by-line comparison of

197 Nov 14, 2022

Utils to quickly evaluate many 🤗 models on the GLUE tasks

Related tags

Overview

Utils to quickly evaluate many 🤗 models on the GLUE tasks

Owner

Przemyslaw K. Joniak

Jack Morgan's Advent of Code Solutions

Repo to store back end infrastructure for Message in a Bottle

The tool helps to find hidden parameters that can be vulnerable or can reveal interesting functionality that other hunters miss.

simple password manager.

Fortnite StW Claimer for Daily Rewards, Research Points and free Llamas.

This repository provides a set of easy to understand and tested Python samples for using Acronis Cyber Platform API.

List of short Codeforces problems with a statement of 1000 characters or less. Python script and data files included.

An Android app that runs Elm in a webview. And a Python script to build the app or install it on the device.

Basit bir cc generator'ü.

An open-source Python project series where beginners can contribute and practice coding.

Find all solutions to SUBSET-SUM, including negative, positive, and repeating numbers

🎉 🎉 PyComp - Java Code compiler written in python.

Glyph Metadata Palette

An kind of operating system portal to a variety of apps with pure python

A Python simple Dice Simulator just for fun

MDAnalysis tool to calculate membrane curvature.

Resources for the 2021 offering of COMP 598

API development made easy: a smart Python 3 API framework

An alternative site to emplea.do due to inconsistent service of the app.

Backend Interview Challenge