Functional interface for concurrent futures, including asynchronous I/O.

Overview

image image image image image image image image image

Futured provides a consistent interface for concurrent functional programming in Python. It wraps any callable to return a concurrent.futures.Future, wraps any async coroutine to return an asyncio.Future, and provides concurrent iterators and context managers for futures.

Usage

threaded, processed

Transform any callable into one which runs in a thread or process pool, and returns a future.

from futured import threaded, processed
import httpx

fetch = threaded(httpx.Client().get)
fetch(url)  # return Future

fs = (fetch(url + path) for path in paths)
threaded.results(fs)  # generate results from futures
threaded.results(fs, timeout=...)  # generate results as completed

fetch.map(urls)  # generate results in order
fetch.map(urls, timeout=...)  # generate results as completed
fetch.mapzip(urls)  # generate (url, result) pairs as completed

Thread and process pool executors may be used as context managers, customized with options, and reused with different callables.

threaded(max_workers=...)(func, ...)
processed(max_workers=...)(func, ...)

futured classes have a waiting context manager which collects results from tasks. Futures can be registered at creation, or appended to the list of tasks.

with threaded.waiting(*fs) as tasks:
    tasks.append(future)
tasks  # list of completed results

futured classes provide a tasks interface which generalizes futures.as_completed and futures.wait, while allowing the set of tasks to be modified, e.g., for retries.

threaded.tasks(fs, timeout=...)  # mutable set of running tasks which iterate as completed

asynced

The same interface works for asyncio.

from futured import asynced
import httpx

fetch = asynced(httpx.AsyncClient().get)
fetch(url)  # return coroutine

asynced.results(fs)  # generate results from futures
asynced.results(fs, timeout=...)  # generate results as completed

fetch.map(urls)  # generate results in order
fetch.map(urls, timeout=...)  # generate results as completed
fetch.mapzip(urls)  # generate (url, result) pairs as completed

asynced provides utilities for calling coroutines from a synchronous context. waiting is similar to trio's nursery, but returns results from a synchronous with block.

asynced.run(async_func, ...)  # call and run until complete
asynced.run(async_gen, ...)  # call and run synchronous iterator
with asynced.waiting(*fs) as tasks:  # concurrent coroutines completed in a block
asynced.tasks(fs, timeout=...)  # mutable set of running tasks which iterate as completed

decorators

Naturally futured wrappers can be used as decorators, but arguments can also be partially bound.

@threaded
def slow():
   ...

fetch = threaded(httpx.Client().get, url)
fetch(params=...)

Methods are supported, as well as a decorated utility for automatically subclassing.

from futured import decorated

FutureClient = decorated(httpx.Client, request=threaded)

 # equivalent to
class FutureClient(httpx.Client):
    request = threaded(httpx.Client.request)

command

command wraps subprocess.Popen to provide a Future compatible interface.

from futured import futured, command

command('ls').result()  # return stdout or raises stderr
command('ls').pipe('wc')  # pipes into next command, or | ('wc',... )
for line in command('ls'):  # iterable lines
command.coroutine('ls')  # return coroutine

futured(command, 'ls')  # supports `map` interface
asynced(command.coroutine, 'ls')  # supports `map` interface with timeout

forked

forked allows iteration in separate child processes.

from futured import forked

for value in forked(values, max_workers=...):
    # in a child process
 # in parent after children have exited

Installation

% pip install futured

Tests

100% branch coverage.

% pytest [--cov]

Changes

1.3

  • Python >=3.7 required
  • Python 3.10 event loop changes
  • Streams replaced with tasks

1.2

  • Python >=3.6 required

1.1

  • Stream completed futures from a pending pool

1.0

  • Executed functions are context managers
  • starmap supported

0.3

  • forked has optional maximum number of workers
  • waiting context manager
  • command pipes (|)
  • distributed.Client support

0.2

  • command.coroutine creates asyncio subprocesses
  • futured.mapzip generates results zipped with arguments
  • asynced.run supports asynchronous iterators
Amazon SageMaker Delta Sharing Examples

This repository contains examples and related resources showing you how to preprocess, train, and serve your models using Amazon SageMaker with data fetched from Delta Lake.

Eitan Sela 5 May 02, 2022
All Assignments , Test , Quizzes and Exams with solutions from NIT Patna B.Tech CSE 5th Semester.

A 🌟 to repo would be delightful, just do it ✔️ it is inexpensive. All Assignments , Quizzes and Exam papers at one place with clean and elegant solut

LakhanKumawat ᵖ⁺ 16 Dec 05, 2022
A bunch of codes for procedurally modeling and texturing futuristic cities.

Procedural Futuristic City This is our final project for CPSC 479. We created a procedural futuristic city complete with Worley noise procedural textu

1 Dec 22, 2021
Earth-to-orbit ballistic trajectories with atmospheric resistance

Earth-to-orbit ballistic trajectories with atmospheric resistance Overview Space guns are a theoretical technology that reduces the cost of getting bu

1 Dec 03, 2021
Web interface for browsing, search and filtering recent arxiv submissions

Web interface for browsing, search and filtering recent arxiv submissions

Andrej 4.8k Jan 08, 2023
A frontend to ease the use of pulseaudio's routing capabilities, mimicking voicemeeter's workflow

Pulsemeeter A frontend to ease the use of pulseaudio's routing capabilities, mimicking voicemeeter's workflow Features Create virtual inputs and outpu

Gabriel Carneiro 164 Jan 04, 2023
python package to showcase, test and build your own version of Pickhardt Payments

Pickhardt Payments Package The pickhardtpayments package is a collection of classes and interfaces that help you to test and implement your dialect of

Rene Pickhardt 37 Dec 18, 2022
Sudoku-Solver

Sudoku-Solver This is a personal project, that put all my today knowledges to the test, is a project that im developing alone with a lot of effort and

Carlos Ismael Gitto Bernales 5 Nov 08, 2021
LinkML based SPARQL template library and execution engine

sparqlfun LinkML based SPARQL template library and execution engine modularized core library of SPARQL templates generic templates using common vocabs

Linked data Modeling Language 6 Oct 10, 2022
The Doodle Master seeks to turn your UI mockups into real code.

Doodle Master The Doodle Master seeks to turn your UI mockups into real code. Currently this repository just serves to demonstrate a Proof Of Concept

Karanbir Chahal 2.4k Dec 09, 2022
Medical appointments No-Show classifier

Medical Appointments No-shows Why do 20% of patients miss their scheduled appointments? A person makes a doctor appointment, receives all the instruct

4 Apr 20, 2022
An interactive course to git

OperatorEquals' Sandbox Git Course! Preface This Git course is an ongoing project containing use cases that I've met (and still meet) while working in

John Torakis 62 Sep 19, 2022
A example project's description is a high-level overview of why you’re doing a project.

A example project's description is a high-level overview of why you’re doing a project.

Nikita Matyukhin 12 Mar 23, 2022
PyScaffold is a project generator for bootstrapping high quality Python packages

PyScaffold is a project generator for bootstrapping high quality Python packages, ready to be shared on PyPI and installable via pip. It is easy to use and encourages the adoption of the best tools a

PyScaffold 1.7k Jan 03, 2023
Provides guideline on how to configure pre-commit hooks in your own python project

Pre-commit Configuration Guide The main aim of this repository is to act as a guide on how to configure the pre-commit hooks in your existing python p

Faraz Ahmed Khan 2 Mar 31, 2022
The Playwright Workshop for TAU: The Homecoming

tau-playwright-workshop This repository contains the instructions and example code for the Playwright workshop for TAU: The Homecoming on December 1,

Pandy Knight 134 Dec 30, 2022
calculadora financiera hecha en python

Calculadora financiera Calculadora de factores financieros basicos, puede calcular tanto factores como expresiones algebraicas en funcion de dichos fa

crudo 5 Nov 10, 2021
🌌 Economics Observatory Visualisation Repository

Economics Observatory Visualisation Repository Website | Visualisations | Data | Here you will find all the data visualisations and infographics attac

Economics Observatory 3 Dec 14, 2022
Grail(TM) is a web browser written in Python

Grail is distributed in source form. It requires that you have a Python interpreter and a Tcl/Tk installation, with the Python interpreter configured for Tcl/Tk support.

22 Oct 18, 2022
This is a python package to get wards, districts,cities and provinces in Zimbabwe

Zim-Places Features This is a python package that allows you to search for cities, provinces, and districts in Zimbabwe.Zimbabwe is split into eight p

RONALD KANYEPI 2 Mar 01, 2022