A python tool for synchronizing the messages from different threads, processes, or hosts.

Related tags

Miscellaneouspythonsynchronizationstreammultiprocessingpython-librarystdoutpython3stdout-redirection
Overview

Sync-stream

This project is designed for providing the synchoronization of the stdout / stderr among different threads, processes, devices or hosts. The package could be used for the following cases:

  1. Use syncstream.LineBuffer: Multiple threads are created. The messages (stdout) from different threads are required to be collected.
  2. Use syncstream.LineProcBuffer in the main process, and syncstream.LineProcMirror in the sub-processes: Multiple sub-processes are created on the same device. The stdout / stderr of each process is redirected to a LineProcMirror, and the results are collected by LineProcBuffer.
  3. Use syncstream.LineFileBuffer: Multiple processes are created. These processes may be deployed on different devices (even with different platforms), but all devices could get accessed to the same shared disk. In this case, the message could be shared by locked files. Each process would hold an independent LineFileBuffer pointing to the same log files.
  4. Use syncstream.LineHostBuffer on the server side, and syncstream.LineHostMirror on the client side: Multiple processes are deployed on different devices, and they could not share the same disk. In this case, the message are synchronized by the web service. Each process would hold a LineHostMirror, and the results would be collected by LineHostBuffer.

The basic package would not contain the file and host modules. To install the package, please use the following options:

pip install syncstream[option1,option2...]
Option Supports
file Install dependencies for the file module. The module provides syncstream.LineFileBuffer.
host Install dependencies for the host module. The module provides syncstream.LineHostBuffer, and syncstream.LineHostMirror.

Documentation

View the documentation here: 📘 https://cainmagi.github.io/sync-stream/

Update reports

0.3.3 @ 6/29/2021

  1. Fix small typos.
  2. Bump the dependencies to the newest versions.

0.3.2 @ 6/14/2021

  1. Fix a bug caused by stopping the mirrors.
  2. Format the meta-data defined in setup.py.
  3. Add the documentation. Currently only the tutorial is finished.

0.3.0 @ 6/4/2021

  1. Support the stop signal for mproc and host modules.
  2. Fix some bugs in the testing script.
  3. Fix typos.

0.2.2 @ 5/25/2021

  1. Add clear() methods for all buffers and mirrors.
  2. Fix typos in the package setup and info file.
  3. Fix a bug caused by writing data to the host in the testing scripts for Linux.

0.2.1 @ 5/24/2021

  1. Add the PyPI publish workflow.

0.2.0 @ 5/24/2021

  1. Finish the synchronization based on the file lock package fasteners.
  2. Finish the synchronization based on the web service packages flask, flask-restful and urllib3.
  3. Fix the compatibility of the testing scripts for py36, py37.

0.1.0 @ 5/22/2021

  1. Finish the synchronization based on the stdlib.
  2. Create this project.
You might also like...
Provide error messages for Python exceptions, even if the original message is empty

errortext is a Python package to provide error messages for Python exceptions, even if the original message is empty.

Thomas Aglassinger 0 Dec 7, 2021
Telegram bot to remove the forwarded tag from messages.
Telegram bot to remove the forwarded tag from messages.

Anonymous Sender Bot @AnonySendBot Telegram bot to remove the forwarded tag from messages. Table of Contents Usage Deploy To Heroku Local Deploying En

Stark Bots 26 Nov 24, 2022
Automatically remove user join messages when the user leaves the server.

CleanLeave Automatically remove user join messages when the user leaves the server. Installation You will need to install poetry to run this bot local

null 11 Sep 19, 2022
Simple script with AminoLab to send ghost messages

Simple script with AminoLab to send ghost messages

Moleey 1 Nov 22, 2021
A Github Action for sending messages to a Matrix Room.
A Github Action for sending messages to a Matrix Room.

matrix-commit A Github Action for sending messages to a Matrix Room. Screenshot: Example Usage: # .github/workflows/matrix-commit.yml on: push:

null 3 Sep 11, 2022
ViberExport - Export messages from Viber messenger using viber.db file

📲 ViberExport Export messages from Viber messenger using viber.db file ⚡ Usage:

null 7 Nov 23, 2022
Telop - Encode and decode messages using an interpretation of the telegraphic code devised by José María Mathé

telop Telop (TELégrafoÓPtico) - Utilidad para codificar y descodificar mensajes de texto empleando una interpretación del código telegráfico ideado po

Ricardo F. 4 Nov 1, 2022
Automatically unpin old messages so you can always pin more!

PinRotate Automatically unpin old messages so you can always pin more! Installation You will need to install poetry to run this bot locally for develo

null 3 Sep 18, 2022
A tool to flash .ofp files in bootloader mode without needing MSM Tool, an alternative to official realme tool
A tool to flash .ofp files in bootloader mode without needing MSM Tool, an alternative to official realme tool

Oppo/Realme Flash .OFP File on Bootloader A tool to flash .ofp files in bootloader mode without needing MSM Tool, an alternative to official realme to

Italo Almeida 70 Jan 2, 2023
Releases(0.3.3)

  • 0.3.3(Jun 29, 2021)

    Introduction

    Sync-stream is a pure python tool for synchronizing the messages (including stdout, stderr, and raised exceptions or warnings) among threads, processes, devices, and hosts. This powerful tool is useful for capturing the updating logs from the sub-processes in some projects.

    Updating reports

    0.3.3 @ 6/29/2021

    1. Fix small typos.
    2. Bump the dependencies to the newest versions.
    Source code(tar.gz)
    Source code(zip)

  • 0.3.2(Jun 14, 2021)

    Introduction

    Sync-stream is a pure python tool for synchronizing the messages (including stdout, stderr, and raised exceptions or warnings) among threads, processes, devices, and hosts. This powerful tool is useful for capturing the updating logs from the sub-processes in some projects.

    Updating reports

    0.3.2 @ 6/14/2021

    1. Fix a bug caused by stopping the mirrors.
    2. Format the meta-data defined in setup.py.
    3. Add the documentation. Currently, only the tutorial is finished.
    Source code(tar.gz)
    Source code(zip)

  • 0.3.0(Jun 4, 2021)

    Introduction

    Sync-stream is a pure python tool for synchronizing the messages (including stdout, stderr, and raised exceptions or warnings) among threads, processes, devices, and hosts. This powerful tool is useful for capturing the updating logs from the sub-processes in some projects.

    Updating reports

    0.3.0 @ 6/4/2021

    1. Support the stop signal for mproc and host modules.
    2. Fix some bugs in the testing script.
    3. Fix typos.
    Source code(tar.gz)
    Source code(zip)

  • 0.2.2(May 25, 2021)

    Introduction

    Sync-stream is a pure python tool for synchronizing the messages (including stdout, stderr, and raised exceptions or warnings) among threads, processes, devices, and hosts. This powerful tool is useful for capturing the updating logs from the sub-processes in some projects.

    Updating reports

    0.2.2 @ 5/25/2021

    1. Add clear() methods for all buffers and mirrors.
    2. Fix typos in the package setup and info file.
    3. Fix a bug caused by writing data to the host in the testing scripts for Linux.
    Source code(tar.gz)
    Source code(zip)

  • 0.2.1(May 24, 2021)

    Introduction

    This is the first release of this project. Sync-stream is a pure python tool for synchronizing the messages (including stdout, stderr, and raised exceptions or warnings) among threads, processes, devices, and hosts. This powerful tool is useful for capturing the updating logs from the sub-processes in some projects.

    Updating reports

    0.2.1 @ 5/24/2021

    1. Add the PyPI publish workflow.

    0.2.0 @ 5/24/2021

    1. Finish the synchronization based on the file lock package fasteners.
    2. Finish the synchronization based on the web service packages flask, flask-restful and urllib3.
    3. Fix the compatibility of the testing scripts for py36, py37.

    0.1.0 @ 5/22/2021

    1. Finish the synchronization based on the stdlib.
    2. Create this project.
    Source code(tar.gz)
    Source code(zip)
Owner
Yuchen Jin
A Ph.D student in University of Houston (UH). Interested area includes: machine learning, programming and religion.
Yuchen Jin
GitHub Repository https://cainmagi.github.io/sync-stream/
This tool helps you to reverse any regex and gives you the opposite/allowed Letters,numerics and symbols.

Regex-Reverser This tool helps you to reverse any regex and gives you the opposite/allowed Letters,numerics and symbols. Screenshots Usage/Examples py

x19 0 Jun 02, 2022
python scripts to perform coin die clustering (performed on Riedones3D).

python scripts to perform coin die clustering (performed on Riedones3D).

Sofiane 2 Apr 29, 2022
The purpose of this code base is to add a specified signal-to-noise ratio noise from MUSAN dataset to a pure speech signal and to generate far-field speech data using room impulse response data from BUT [email protected] Reverb Database.

Add_noise_and_rir_to_speech The purpose of this code base is to add a specified signal-to-noise ratio noise from MUSAN dataset to a pure speech signal

Yunqi Chen 7 Oct 30, 2022
This is some simple code to scrape vistbook's system to get an overview of the different cabins availability.

DNT_cabin_availability_system This is some simple code to scrape visbook's system to get an overview of the different cabins availability. The system

Andreas Lorentzen 1 Sep 25, 2022
A passive recon suite designed for fetching the information about web application

FREAK Suite designed for passive recon Usage: python3 setup.py python3 freak.py warning This tool will throw error if you doesn't provide valid api ke

toxic v3nom 7 Feb 17, 2022
Arabic to Roman Converter in Python

Arabic-to-Roman-Converter Made together with https://github.com/goltaraya . Arabic to Roman Converter in Python. -Instructions: 1 - Make sure you have

Pedro Lucas Tomazeti Fernandes 6 Oct 28, 2021
Python package for handling and analyzing PSRFITS files

PyPulse A pure-Python package for handling and analyzing PSRFITS files. Read the documentation here. This is an alternate code base from PSRCHIVE. Req

Michael Lam 15 Nov 30, 2022
API wrapper for VCS hosting system.

PythonVCS API wrapper for VCS hosting system. Supported platforms Gitea Github, Gitlab, Bitbucket support will not, until that packages is not updated

MisileLaboratory 1 Apr 02, 2022
A fluid medium for storing, relating, and surfacing thoughts.

Conceptarium A fluid medium for storing, relating, and surfacing thoughts. Read more... Instructions The conceptarium takes up about 1GB RAM when runn

115 Dec 19, 2022
A C-like hardware description language (HDL) adding high level synthesis(HLS)-like automatic pipelining as a language construct/compiler feature.

██████╗ ██╗██████╗ ███████╗██╗ ██╗███╗ ██╗███████╗ ██████╗ ██╔══██╗██║██╔══██╗██╔════╝██║ ██║████╗ ██║██╔════╝██╔════╝ ██████╔╝██║██████╔╝█

Julian Kemmerer 391 Jan 01, 2023
Url-check-migration-python - A python script using Apica API's to migrate URL checks between environments

url-check-migration-python A python script using Apica API's to migrate URL chec

Angelo Aquino 1 Feb 16, 2022
Unzip Japanese Shift-JIS zip archives on non-Japanese systems.

Unzip JP GUI Unzip Japanese Shift-JIS zip archives on non-Japanese systems. This script unzips the file while converting the file names from Shift-JIS

Emile Bangma 9 Dec 07, 2022
Emulate and Dissect MSF and *other* attacks

Need help in analyzing Windows shellcode or attack coming from Metasploit Framework or Cobalt Strike (or may be also other malicious or obfuscated code)? Do you need to automate tasks with simple scr

123 Dec 16, 2022
Простенький ботик для троллинга с интерфейсом #Yakima_Visus

Bot-Trolling-Vk Простенький ботик для троллинга с интерфейсом #Yakima_Visus Установка pip install vk_api pip install requests если там еще чото будет

Yakima Visus 4 Oct 11, 2022
Utility/Raiding selfbot made by Shell and Roover.

Utility/Raiding selfbot made by Shell and Roover. We are open to suggestions and ideas.

Shell 2 Dec 08, 2021
Solutions for the Advent of Code 2021 event.

About 📋 This repository holds all of the solution code for the Advent of Code 2021 event. All solutions are done in Python 3.9.9 and done in non-real

robert yin 0 Mar 21, 2022
An example of python package

An example of python package Why use packages? It is a good practice to not code the same function twice, and to reuse common code from one python scr

10 Oct 18, 2022
How to access and display MyEnergi data

MyEnergi-Python-Example How to access and display MyEnergi data Windows PC Install a version of Python typically 3.10 The Python code here needs addit

G6EJD 8 Nov 28, 2022
Script to use SysWhispers2 direct system calls from Cobalt Strike BOFs

SysWhispers2BOF Script to use SysWhispers2 direct system calls from Cobalt Strike BOFs. Introduction This script was initially created to fix specific

FalconForce 101 Dec 20, 2022
Easily Generate Revolut Business Cards

RevBusinessCardGen Easily Generate Revolut Business Cards Prerequisites Before you begin, ensure you have met the following requirements: You have ins

Younes™ 35 Dec 14, 2022