An implementation for `Text2Event: Controllable Sequence-to-Structure Generation for End-to-end Event Extraction`

Overview

Text2Event

Update

  • [2021-08-03] Update pre-trained models

Quick links

Requirements

General

  • Python (verified on 3.8)
  • CUDA (verified on 11.1)

Python Packages

  • see requirements.txt
conda create -n text2event python=3.8
conda activate text2event
pip install -r requirements.txt

Quick Start

Data Format

Data folder contains four files:

data/text2tree/one_ie_ace2005_subtype
├── event.schema
├── test.json
├── train.json
└── val.json

train/val/test.json are data files, and each line is a JSON instance. Each JSON instance contains text and event fields, in which text is plain text, and event is event linearized form. If you want to use other key names, it is easy to change the input format in run_seq2seq.py.

{"text": "He also owns a television and a radio station and a newspaper .", "event": "<extra_id_0>  <extra_id_1>"}
{"text": "' ' For us the United Natgions is the key authority '' in resolving the Iraq crisis , Fischer told reporters opn arrival at the EU meeting .", "event": "<extra_id_0> <extra_id_0> Meet meeting <extra_id_0> Entity EU <extra_id_1> <extra_id_1> <extra_id_1>"}

Note:

  • Use the extra character of T5 as the structure indicators, such as <extra_id_0>, <extra_id_1>, etc.

  • event.schema is the event schema file for building the trie of constrained decoding. It contains three lines: the first line is event type name list, the second line is event role name list, the third line is type-to-role dictionary.

    ["Declare-Bankruptcy", "Convict", ...]
    ["Plaintiff", "Target", ...]
    {"End-Position": ["Place", "Person", "Entity"], ...}
    

Model Training

Training scripts as follows:

  • run_seq2seq.py: Python code entry, modified from the transformers/examples/seq2seq/run_seq2seq.py
  • run_seq2seq.bash: Model training script logging to the log file.
  • run_seq2seq_verbose.bash: Same model training script as run_seq2seq.bash but output to the screen directly.
  • run_seq2seq_with_pretrain.bash: Model training script for curriculum learning, which contains substructure learning and full structure learning.

The command for the training is as follows (see bash scripts and Python files for the corresponding command-line arguments):

bash run_seq2seq_verbose.bash -d 0 -f tree -m t5-base --label_smoothing 0 -l 1e-4 --lr_scheduler linear --warmup_steps 2000 -b 16
  • -d refers to the GPU device id.
  • -m t5-base refers to using T5-base.
  • Currently, constrained decoding algorithms do not support use_fast_tokenizer=True and beam search yet.

Trained models are saved in the models/ folder.

Model Evaluation

Offset-level Evaluation

python evaluation.py -g <data-folder-path> -r <offset-folder-path> -p <model-folder-path> -f <data-format>
  • This evaluation script converts the eval_preds_seq2seq.txt and test_preds_seq2seq.txt in the model folder <model-folder-path> into the corresponding offset prediction results for model evaluation.
  • -f <data-format> refers to dyiepp or oneie

Record-level Evaluation (approximate, used in training)

bash run_eval.bash -d 0 -m <model-folder-path> -i <data-folder-path> -c -b 8
  • -d refers to the GPU device id.
  • -c represents the use of constrained decoding, otherwise not apply
  • -b 8 represents batch_size=8

How to expand to other tasks

  1. prepare the corresponding data format
  2. Writ the code for reading corresponding data format: elif data_args.task.startswith("event") in seq2seq.py
  3. Writ the code for evaluating the corresponding task result: def compute_metrics(eval_preds) in seq2seq.py

Completing the above process can finish the simple Seq2Seq training and inference process.

If you need to use constrained decoding, you need to write the corresponding decoding mode (decoding_format), refer to extraction.extract_constraint.get_constraint_decoder

Pre-trained Model

You can find the pre-trained models as following google drive links or download models using command gdown (pip install gdown).

dyiepp_ace2005_en_t5_base.zip

gdown --id 1_fOmnSatNfceL9DZPxpof5AT9Oo7vTrC && unzip dyiepp_ace2005_en_t5_base.zip

dyiepp_ace2005_en_t5_large.zip

gdown --id 10iY1obkbgJtTKwfoOFevqL5AwG-hLvhU && unzip dyiepp_ace2005_en_t5_large.zip

oneie_ace2005_en_t5_large.zip

gdown --id 1zwnptRbdZntPT4ucqSANeaJ3vvwKliUe && unzip oneie_ace2005_en_t5_large.zip

oneie_ere_en_t5_large.zip

gdown --id 1WG7-pTZ3K49VMbQIONaDq_0pUXAcoXrZ && unzip oneie_ere_en_t5_large.zip

Event Datasets Preprocessing

We first refer to the following code and environments [dygiepp] and [oneie v0.4.7] for data preprocessing. Thanks to them!

After data preprocessing and we get the following data files:

 $ tree data/raw_data/
data/raw_data/
├── ace05-EN
│   ├── dev.oneie.json
│   ├── test.oneie.json
│   └── train.oneie.json
├── dyiepp_ace2005
│   ├── dev.json
│   ├── test.json
│   └── train.json
└── ERE-EN
    ├── dev.oneie.json
    ├── test.oneie.json
    └── train.oneie.json

We then convert the above data files to tree format. The following scripts generate the corresponding data folder in data/text2tree. The conversion will automatically generate train/dev/test JSON files and event.schema file.

bash scripts/processing_data.bash
data/text2tree
├── dyiepp_ace2005_subtype
│   ├── event.schema
│   ├── test.json
│   ├── train.json
│   └── val.json
├── dyiepp_ace2005_subtype_span
│   ├── event.schema
│   ├── test.json
│   ├── train.json
│   └── val.json
├── one_ie_ace2005_subtype
│   ├── event.schema
│   ├── test.json
│   ├── train.json
│   └── val.json
├── one_ie_ace2005_subtype_span
│   ├── event.schema
│   ├── test.json
│   ├── train.json
│   └── val.json
├── one_ie_ere_en_subtype
│   ├── event.schema
│   ├── test.json
│   ├── train.json
│   └── val.json
└── one_ie_ere_en_subtype_span
    ├── event.schema
    ├── test.json
    ├── train.json
    └── val.json
  • dyiepp_ace2005_subtype for Full Structure Learning and dyiepp_ace2005_subtype_span for Substructure Learning.

Citation

If this repository helps you, please cite this paper:

Yaojie Lu, Hongyu Lin, Jin Xu, Xianpei Han, Jialong Tang, Annan Li, Le Sun, Meng Liao, Shaoyi Chen. Text2Event: Controllable Sequence-to-Structure Generation for End-to-end Event Extraction. The Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL-IJCNLP 2021).

@inproceedings{lu-etal-2021-text2event,
    title = "{T}ext2{E}vent: Controllable Sequence-to-Structure Generation for End-to-end Event Extraction",
    author = "Lu, Yaojie  and
      Lin, Hongyu  and
      Xu, Jin  and
      Han, Xianpei  and
      Tang, Jialong  and
      Li, Annan  and
      Sun, Le  and
      Liao, Meng  and
      Chen, Shaoyi",
    booktitle = "Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)",
    month = aug,
    year = "2021",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2021.acl-long.217",
    pages = "2795--2806",
    abstract = "Event extraction is challenging due to the complex structure of event records and the semantic gap between text and event. Traditional methods usually extract event records by decomposing the complex structure prediction task into multiple subtasks. In this paper, we propose Text2Event, a sequence-to-structure generation paradigm that can directly extract events from the text in an end-to-end manner. Specifically, we design a sequence-to-structure network for unified event extraction, a constrained decoding algorithm for event knowledge injection during inference, and a curriculum learning algorithm for efficient model learning. Experimental results show that, by uniformly modeling all tasks in a single model and universally predicting different labels, our method can achieve competitive performance using only record-level annotations in both supervised learning and transfer learning settings.",
}
Owner
Roger
Roger
Pytorch-diffusion - A basic PyTorch implementation of 'Denoising Diffusion Probabilistic Models'

PyTorch implementation of 'Denoising Diffusion Probabilistic Models' This reposi

Arthur Juliani 76 Jan 07, 2023
Single Image Deraining Using Bilateral Recurrent Network (TIP 2020)

Single Image Deraining Using Bilateral Recurrent Network Introduction Single image deraining has received considerable progress based on deep convolut

23 Aug 10, 2022
Implementation of Online Label Smoothing in PyTorch

Online Label Smoothing Pytorch implementation of Online Label Smoothing (OLS) presented in Delving Deep into Label Smoothing. Introduction As the abst

83 Dec 14, 2022
Meta Learning Backpropagation And Improving It (VSML)

Meta Learning Backpropagation And Improving It (VSML) This is research code for the NeurIPS 2021 publication Kirsch & Schmidhuber 2021. Many concepts

Louis Kirsch 22 Dec 21, 2022
Multimodal Co-Attention Transformer (MCAT) for Survival Prediction in Gigapixel Whole Slide Images

Multimodal Co-Attention Transformer (MCAT) for Survival Prediction in Gigapixel Whole Slide Images [ICCV 2021] © Mahmood Lab - This code is made avail

Mahmood Lab @ Harvard/BWH 63 Dec 01, 2022
Easy and comprehensive assessment of predictive power, with support for neuroimaging features

Documentation: https://raamana.github.io/neuropredict/ News As of v0.6, neuropredict now supports regression applications i.e. predicting continuous t

Pradeep Reddy Raamana 93 Nov 29, 2022
An Implementation of Fully Convolutional Networks in Tensorflow.

Update An example on how to integrate this code into your own semantic segmentation pipeline can be found in my KittiSeg project repository. tensorflo

Marvin Teichmann 1.1k Dec 12, 2022
PyTorch code for JEREX: Joint Entity-Level Relation Extractor

JEREX: "Joint Entity-Level Relation Extractor" PyTorch code for JEREX: "Joint Entity-Level Relation Extractor". For a description of the model and exp

LAVIS - NLP Working Group 50 Dec 01, 2022
An official implementation of the Anchor DETR.

Anchor DETR: Query Design for Transformer-Based Detector Introduction This repository is an official implementation of the Anchor DETR. We encode the

MEGVII Research 276 Dec 28, 2022
use machine learning to recognize gesture on raspberrypi

Raspberrypi_Gesture-Recognition use machine learning to recognize gesture on raspberrypi 說明 利用 tensorflow lite 訓練手部辨識模型 分辨 "剪刀"、"石頭"、"布" 之手勢 再將訓練模型匯入

1 Dec 10, 2021
[NeurIPS 2020] Official repository for the project "Listening to Sound of Silence for Speech Denoising"

Listening to Sounds of Silence for Speech Denoising Introduction This is the repository of the "Listening to Sounds of Silence for Speech Denoising" p

Henry Xu 40 Dec 20, 2022
Replication package for the manuscript "Using Personality Detection Tools for Software Engineering Research: How Far Can We Go?" submitted to TOSEM

tosem2021-personality-rep-package Replication package for the manuscript "Using Personality Detection Tools for Software Engineering Research: How Far

Collaborative Development Group 1 Dec 13, 2021
✔️ Visual, reactive testing library for Julia. Time machine included.

PlutoTest.jl (alpha release) Visual, reactive testing library for Julia A macro @test that you can use to verify your code's correctness. But instead

Pluto 68 Dec 20, 2022
Efficient 6-DoF Grasp Generation in Cluttered Scenes

Contact-GraspNet Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenes Martin Sundermeyer, Arsalan Mousavian, Rudolph Triebel, Dieter

NVIDIA Research Projects 148 Dec 28, 2022
Code + pre-trained models for the paper Keeping Your Eye on the Ball Trajectory Attention in Video Transformers

Motionformer This is an official pytorch implementation of paper Keeping Your Eye on the Ball: Trajectory Attention in Video Transformers. In this rep

Facebook Research 192 Dec 23, 2022
PyTorch implementation of CDistNet: Perceiving Multi-Domain Character Distance for Robust Text Recognition

PyTorch implementation of CDistNet: Perceiving Multi-Domain Character Distance for Robust Text Recognition The unofficial code of CDistNet. Now, we ha

25 Jul 20, 2022
🧮 Matrix Factorization for Collaborative Filtering is just Solving an Adjoint Latent Dirichlet Allocation Model after All

Accompanying source code to the paper "Matrix Factorization for Collaborative Filtering is just Solving an Adjoint Latent Dirichlet Allocation Model A

Florian Wilhelm 39 Dec 03, 2022
Rl-quickstart - Reinforcement Learning Quickstart

Reinforcement Learning Quickstart To get setup with the repository, git clone ht

UCLA DataRes 3 Jun 16, 2022
Oriented Response Networks, in CVPR 2017

Oriented Response Networks [Home] [Project] [Paper] [Supp] [Poster] Torch Implementation The torch branch contains: the official torch implementation

ZhouYanzhao 217 Dec 12, 2022