django-idom allows Django to integrate with IDOM

Overview

Django IDOM

Tests Version Info License: MIT

django-idom allows Django to integrate with IDOM, a package inspired by ReactJS for creating responsive web interfaces in pure Python.

You can try IDOM now in a Jupyter Notebook: Binder

Install Django IDOM

pip install django-idom

Django Integration

To integrate IDOM into your application you'll need to modify or add the following files to your_project:

your_project/
├── __init__.py
├── asgi.py
├── settings.py
├── urls.py
└── example_app/
    ├── __init__.py
    ├── idom.py
    ├── templates/
    │   └── your-template.html
    └── urls.py

asgi.py

Follow the channels installation guide in order to create ASGI websockets within Django. Then, we will add a path for IDOM's websocket consumer using IDOM_WEBSOCKET_PATH.

Note: If you wish to change the route where this websocket is served from, see the available settings.

import os

from django.core.asgi import get_asgi_application

from django_idom import IDOM_WEBSOCKET_PATH

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "test_app.settings")

# Fetch ASGI application before importing dependencies that require ORM models.
http_asgi_app = get_asgi_application()

from channels.routing import ProtocolTypeRouter, URLRouter

application = ProtocolTypeRouter(
    {
        "http": http_asgi_app,
        "websocket": URLRouter(
          # add a path for IDOM's websocket
          [IDOM_WEBSOCKET_PATH]
        ),
    }
)

settings.py

In your settings you'll need to add django_idom to the INSTALLED_APPS list:

INSTALLED_APPS = [
  ...,
  "django_idom",
]

You may configure additional options as well:

# the base URL for all IDOM-releated resources
IDOM_BASE_URL: str = "_idom/"

# Set cache size limit for loading JS files for IDOM.
# Only applies when not using Django's caching framework (see below).
IDOM_WEB_MODULE_LRU_CACHE_SIZE: int | None = None

# Configure a cache for loading JS files
CACHES = {
  # Configure a cache for loading JS files for IDOM
  "idom_web_modules": {"BACKEND": ...},
  # If the above cache is not configured, then we'll use the "default" instead
  "default": {"BACKEND": ...},
}

urls.py

You'll need to include IDOM's static web modules path using IDOM_WEB_MODULES_PATH. Similarly to the IDOM_WEBSOCKET_PATH. If you wish to change the route where this websocket is served from, see the available settings.

from django_idom import IDOM_WEB_MODULES_PATH

urlpatterns = [
    IDOM_WEB_MODULES_PATH,
    ...
]

example_app/components.py

This is where, by a convention similar to that of views.py, you'll define your IDOM components. Ultimately though, you should feel free to organize your component modules you wish. The components created here will ultimately be referenced by name in your-template.html. your-template.html.

import idom

@idom.component
def Hello(greeting_recipient):  # component names are camelcase by convention
    return Header(f"Hello {greeting_recipient}!")

example_app/templates/your-template.html

In your templates, you may inject a view of an IDOM component into your templated HTML by using the idom_component template tag. This tag which requires the name of a component to render (of the form module_name.ComponentName) and keyword arguments you'd like to pass it from the template.

idom_component module_name.ComponentName param_1="something" param_2="something-else"

In context this will look a bit like the following...

">

{% load static %}
{% load idom %}


<html>
  <body>
    ...
    {% idom_component "your_project.example_app.components.Hello" greeting_recipient="World" %}
  body>
html>

example_app/views.py

You can then serve your-template.html from a view just like any other.

from django.http import HttpResponse
from django.template import loader


def your_view(request):
    context = {}
    return HttpResponse(
      loader.get_template("your-template.html").render(context, request)
    )

example_app/urls.py

Include your view in the list of urlpatterns

from django.urls import path
from .views import your_view  # define this view like any other HTML template view

urlpatterns = [
    path("", your_view),
    ...
]

Developer Guide

If you plan to make code changes to this repository, you'll need to install the following dependencies first:

Once done, you should clone this repository:

git clone https://github.com/idom-team/django-idom.git
cd django-idom

Then, by running the command below you can:

  • Install an editable version of the Python code

  • Download, build, and install Javascript dependencies

pip install -e . -r requirements.txt

Finally, to verify that everything is working properly, you'll want to run the test suite.

Running The Tests

This repo uses Nox to run scripts which can be found in noxfile.py. For a full test of available scripts run nox -l. To run the full test suite simple execute:

nox -s test

To run the tests using a headless browser:

nox -s test -- --headless
Comments
  • use_query and use_mutation

    use_query and use_mutation

    Description

    Adds use_query and use_mutation hooks. I haven't tested to see that these work as expected, but they get across the interface that I think we should provide for writing database operations.

    Checklist:

    Please update this checklist as you complete each item:

    • [ ] Tests have been included for all bug fixes or added functionality.
    • [ ] The changelog.rst has been updated with any significant changes, if necessary.
    • [ ] GitHub Issues which may be closed by this PR have been linked.
    type: feature type: investigation 
    opened by rmorshea 34
  • Threaded dispatcher loop

    Threaded dispatcher loop

    This PR attempts runs the dispatcher in a thread.

    Merging this pull request will significantly increase scalability. For example, would allow servicing requests in scenarios where there's hundreds of clients connected to a single webserver.

    Details

    • Run the renderer within a single self._idom_dispatcher_thread
    • Kill off render threads upon websocket disconnection.
    • Replace asyncio.Queue with a thread-safe async queue janus.Queue().async_q
    • Remove useless super() in disconnect()
    opened by Archmonger 25
  • Improving use_query

    Improving use_query

    Current Situation

    There are a number of usability problems with use_query:

    1. Prefetch logic is not perfect and can have significant performance costs: #110
    2. The interface does not lend itself to future extensions: #104, #103
    3. And that, when a user accesses a field that has not been pre-fetched in the scope of a render function, the SynchronousOnlyOperation error they get from Django does not communicate this fact and how to resolve it.

    Proposed Actions

    Discuss and arrive at a solution to each of the above problems. I'll update this section as we reach consensus on how to proceed.

    flag: triage 
    opened by rmorshea 24
  • Django IDOM

    Django IDOM

    This is the most minimalistic Django configuration possible that can support Websockets.

    I've used folder structure and file naming schemes suggested by Django documentation.

    Also included the Daphne webserver in the requirements for testing on more than just the development webserver.

    In order to get this configuration running

    1. cd <repo_root_dir>
    2. pip install requirements.txt
    3. python manage.py migrate to create the initial database
    4. python manage.py runserver to run the django development test server. Alternatively, daphne dj_idom.asgi:application to run the production-grade webserver.

    Here's the files in the repo: dj_idom/static/scripts.js: Client side websocket dj_idom/templates/base.html: HTML base template. dj_idom/consumers.py: Server side websocket dj_idom/asgi.py: Websocket URL routing dj_idom/urls.py: HTTP URL routing dj_idom/settings.py: Django boot time config manage.py: Django project management utility

    opened by Archmonger 23
  • Allow adding class to a template tag div

    Allow adding class to a template tag div

    Adds class as a template tag parameter.

    Additionally, unpinned Twisted to the latest version due to them fixing Django Channels issues. Version <21 was having install issues on Windows.

    opened by Archmonger 22
  • v1.0.0

    v1.0.0

    Changelog

    • Change Django IDOM version to 1.0.0
    • Unpins top boundary on channels and aiofile
    • Bumps IDOM's minimum version
    • Use f-strings when possible
    • Use contexlib.suppress when possible
    • Implement use_websocket, use_scope, and use_location
    • Remove websocket parameter from components (replaced by use_websocket hook)
    • Create formal docs
    • Rename idom_component template tag to component in preparation for repo rename (reactive)
    • Logging for when a component fails to import, or if no components were found within Django.

    Docs Preview

    1. pip install -U -r requirements/build-docs.txt
    2. mkdocs serve
    opened by Archmonger 21
  • Avoid synchronous code within IDOM

    Avoid synchronous code within IDOM

    Old Behavior

    Currently, IDOM relies on synchronous code to properly queue renders. This code blocks the asyncio event loop, causing massive concurrency issues.

    See section "Two Worlds of Python" for why this is a problem: https://arunrocks.com/a-guide-to-asgi-in-django-30-and-its-performance/

    New Behavior

    Rewrite parts of IDOM core to do everything using asyncio.

    Implementation Details

    Anything that relies on order of operations should be executed in a FIFO queue.

    If synchronous code is required, it should be run in a thread in order to avoid blocking asyncio event loops.

    All synchronous functions should be converted to async within IDOM core.

    I had previously attempted threading the django-idom dispatcher as a quick fix, but that did not yield worthwhile performance benefits, likely due to context switching.

    Code of Conduct

    priority: 0 (critical) type: revision state: blocked 
    opened by Archmonger 21
  • `use_query` prefetching for ManyToMany and ManyToOne fields

    `use_query` prefetching for ManyToMany and ManyToOne fields

    Description

    Attempt at auto fetching ManyToMany fields

    Checklist:

    Please update this checklist as you complete each item:

    • [x] Tests have been included for all bug fixes or added functionality.
    • [x] The changelog.rst has been updated with any significant changes, if necessary.
    • [x] GitHub Issues which may be closed by this PR have been linked.
    opened by Archmonger 19
  • Fix JS Builds on Windows

    Fix JS Builds on Windows

    • Fix JS build issues causing Failed to resolve module specifier "react"
    • Look into whether channels can be monkey-patched to fix the ChannelsLiveServerTestCase

    Full error: Uncaught TypeError: Failed to resolve module specifier "react". Relative references must start with either "/", "./", or "../".

    type: bug 
    opened by Archmonger 14
  • v2.1.0

    v2.1.0

    Changelog

    • Change type hint on view_to_component callable to have request argument be optional.
    • More DRY way of async view rendering.
    • Have docs demonstrate how to use Django-IDOM with channels>=4.0.0.
    • Concatenate some examples on the view_to_component docs
    • Add note to docs about potential information exposure via view_to_component when using compatibility=True.
    • Clean up docs on running tests

    Checklist:

    Please update this checklist as you complete each item:

    • [x] Tests have been included for all bug fixes or added functionality.
    • [x] The changelog.rst has been updated with any significant changes, if necessary.
    • [x] GitHub Issues which may be closed by this PR have been linked.
    opened by Archmonger 13
  • Advanced Websocket Features

    Advanced Websocket Features

    New Features

    • [x] Automatic WS reconnection (settings.py:IDOM_WS_RECONNECT_TIMEOUT)
    • [x] Propogate WS down to components
    • [x] Support authenticated components (Login the user and save the session if auth middleware is detected)
    • [x] Pin "idom-client-react": "^0.33.3"
    • [x] Update tests to add websocket as a component parameter
    • [x] Update tests to use render shortcut instead of HttpResponse
    • [x] Change ALLOWED_HOSTS = ["*"] in tests to allow LAN access
    • [x] Rename IdomAsyncWebSocketConsumer -> IdomAsyncWebsocketConsumer
    • [x] Readme clean up. fixes, and updates
    type: docs 
    opened by Archmonger 13
  • v2.2.1: Fix recursive fetch depth for ManyToOneRel

    v2.2.1: Fix recursive fetch depth for ManyToOneRel

    Description

    Recursive fetching was only ocurring on ManyToManyField. This PR makes the recursive call occur for both ManyToOneRel and ManyToManyField.

    Changelog

    • fix #116
    • Remove flaky idom.web.export test

    Checklist:

    Please update this checklist as you complete each item:

    • [ ] Tests have been included for all bug fixes or added functionality.
    • [x] The changelog has been updated with any significant changes, if necessary.
    • [x] GitHub Issues which may be closed by this PR have been linked.
    opened by Archmonger 0
  • Add `fetch_policy=...` to `use_query`

    Add `fetch_policy=...` to `use_query`

    Current Situation

    There currently isn't a native way to cache or defer execution of a query.

    Proposed Actions

    Mimic behavior of apollo's useQuery fetch policies within our use_query hook.

    priority: 2 (moderate) type: feature 
    opened by Archmonger 0
  • Support building JavaScript ReactJS components

    Support building JavaScript ReactJS components

    Current Situation

    This ticket is a duplicate of idom-team/idom#786 to increase transparency that he issue affects both repos.

    Proposed Actions

    Implement changes suggested in idom-team/idom#786

    priority: 1 (high) type: feature type: investigation state: blocked 
    opened by Archmonger 0
  • Use mkdocstrings for documenting Python code

    Use mkdocstrings for documenting Python code

    Current Situation

    Currently, our Python APIs need to be manually added to the docs, which can potentially result in API <-> Docs mismatch.

    Proposed Actions

    Use mkdocstrings within the docs to automatically generate markdown based on Python code, type hints, and docstrings.

    Implementing this is currently blocked by these issues:

    • mkdocstrings/mkdocstrings#450
    • mkdocstrings/mkdocstrings#466
    type: docs flag: good first issue priority: 3 (low) state: blocked 
    opened by Archmonger 0
  • SEO compatible rendering

    SEO compatible rendering

    Current Situation

    Currently, sites built in IDOM are not SEO compatible. This is a fairly common issue with JavaScript frameworks such as ReactJS.

    This might ultimately relate to persistent components (#34).

    Proposed Actions

    To resolve this, there needs to be an initial HTTP render, followed by a JavaScript re-render. The best way of doing this requires some form of persistent storage of all hook states, due to the fact that ASGI websockets are a completely different stack than HTTP rendering. Hook states will need to be stored server side in order to prevent spoofing. We can either use a database or determine if multiprocessing.shared_memory can be used here.

    1. Use the template tag to render the initial component as raw HTML
    2. Serialize all the component's hook values (probably through dill.pickle) and
      • This might need exposing some new hook APIs in core that provides returns hook values for a given component instance.
    3. Store serialized hook values within the database
      • Use the component's UUID as the database ID.
    4. When the JavaScript client requests it, rehydrate the components hook values
    5. Execute the ReactJS rendering
      • idom-team/idom#760 will need to be resolve prior to this
    6. Delete the component hook values from the database, as they are no longer needed.

    The database model might look like this:

    class IdomHookState(Model):
        uuid = models.UUIDField(
            primary_key=True, default=uuid.uuid4, editable=False, unique=True
        )
        hook_attributes = models.TextField()
    

    This design brings up a challenge of determining when to evict old hook states (for example, if a user did the initial render but for some reason never performed a websocket connection). We don't want to store everything forever, so some configurable age-based eviction strategy might be needed. Expired entries should be culled before each fetch of IdomHookState.hook_attributes. Expiration should be based on last access time, which is fairly simple to do in Django like such: last_accessed = models.DateTimeField(auto_now=True).

    priority: 3 (low) type: feature state: blocked 
    opened by Archmonger 0
Releases(2.2.0)
  • 2.2.0(Dec 28, 2022)

    Added

    • Add options: QueryOptions parameter to use_query to allow for configuration of this hook.

    Changed

    • By default, use_query will recursively prefetch all many-to-many or many-to-one relationships to prevent SynchronousOnlyOperation exceptions.

    Removed

    • django_idom.hooks._fetch_lazy_fields has been deleted. The equivalent replacement is django_idom.utils.django_query_postprocessor.
    Source code(tar.gz)
    Source code(zip)
  • 2.1.0(Nov 2, 2022)

    Changed

    • Minimum channels version is now 4.0.0.

    Fixed

    • Change type hint on view_to_component callable to have request argument be optional.
    • Change type hint on view_to_component to represent it as a decorator with paranthesis (ex @view_to_component(compatibility=True))

    Security

    • Add note to docs about potential information exposure via view_to_component when using compatibility=True.
    Source code(tar.gz)
    Source code(zip)
  • 2.0.1(Oct 18, 2022)

  • 2.0.0(Oct 18, 2022)

    Added

    • use_origin hook for returning the browser's location.origin.

    Changed

    • view_to_component now returns a Callable, instead of directly returning a Component. Check the docs for new usage info.
    • use_mutation and use_query will now log any query failures.

    Fixed

    • Allow use_mutation to have refetch=None, as the docs suggest is possible.
    • use_query will now prefetch all fields to prevent SynchronousOnlyOperation exceptions.
    • view_to_component, django_css, and django_js type hints will now display like normal functions.
    • IDOM preloader no longer attempts to parse commented out IDOM components.
    • Tests are now fully functional on Windows
    Source code(tar.gz)
    Source code(zip)
  • 1.2.0(Sep 20, 2022)

    Added

    • auth_required decorator to prevent your components from rendering to unauthenticated users.
    • use_query hook for fetching database values.
    • use_mutation hook for modifying database values.
    • view_to_component utility to convert legacy Django views to IDOM components.

    Changed

    • Bumped the minimum IDOM version to 0.40.2
    • Testing suite now uses playwright instead of selenium

    Fixed

    • IDOM preloader is no longer sensitive to whitespace within template tags.
    Source code(tar.gz)
    Source code(zip)
  • 1.1.0(Jul 2, 2022)

    Added

    • django_css and django_js components to defer loading CSS & JS files until needed.

    Changed

    • Bumped the minimum IDOM version to 0.39.0
    Source code(tar.gz)
    Source code(zip)
  • 1.0.0(May 31, 2022)

    Added

    • Django-specific hooks! use_websocket, use_scope, and use_location are now available within the django_idom.hooks module.
    • Documentation has been placed into a formal docs webpage.
    • Logging for when a component fails to import, or if no components were found within Django.

    Changed

    • idom_component template tag has been renamed to component
    • Bumped the minimum IDOM version to 0.38.0

    Removed

    • websocket parameter for components has been removed. Functionally, it is replaced with django_idom.hooks.use_websocket.
    Source code(tar.gz)
    Source code(zip)
  • 0.0.5(Apr 5, 2022)

  • 0.0.4(Mar 5, 2022)

  • 0.0.3(Feb 20, 2022)

  • 0.0.2(Jan 31, 2022)

    Added

    • Ability to declare the HTML class of the top-level component div
    • name = ... parameter to IDOM HTTP paths for use with django.urls.reverse()
    • Cache versioning to automatically invalidate old web module files from the cache backend
    • Automatic pre-population of the IDOM component registry
    • Type hinting for IdomWebsocket

    Changed

    • Fetching web modules from disk and/or cache is now fully async
    • Static files are now contained within a django_idom/ parent folder
    • Upgraded IDOM to version 0.36.0
    • Minimum Django version required is now 4.0
    • Minimum Python version required is now 3.8

    Removed

    • IDOM_WEB_MODULES_PATH has been replaced with Django include(...)
    • IDOM_WS_MAX_RECONNECT_DELAY has been renamed to IDOM_WS_MAX_RECONNECT_TIMEOUT
    • idom_web_modules cache backend has been renamed to idom

    Fixed

    • Increase test timeout values to prevent false positives
    • Windows compatibility for building Django-IDOM

    Security

    • Fixed potential directory travesal attack on the IDOM web modules URL
    Source code(tar.gz)
    Source code(zip)
  • 0.0.1(Aug 19, 2021)

    Initial release of django-idom

    django-idom allows Django to integrate with IDOM, a package inspired by ReactJS for creating responsive web interfaces in pure Python.

    The initial release contains all the basic features requires to install IDOM into existing Django applications using the INSTALLED_APPS list and some other basic configurations that are describe in more detail within the README.

    Source code(tar.gz)
    Source code(zip)
pdm-django: Django command shortcuts for PDM

pdm-django: Django command shortcuts for PDM A plugin that gives you command shortcuts for developing with PDM. pdm run python manage.py runserver -

Neutron Sync 2 Aug 11, 2022
📝 Sticky Notes in Django admin

django-admin-sticky-notes Share notes between superusers. Installation Install via pip: pip install django_admin_sticky_notes Put django_admin_sticky_

Dariusz Choruży 7 Oct 06, 2021
A django model and form field for normalised phone numbers using python-phonenumbers

django-phonenumber-field A Django library which interfaces with python-phonenumbers to validate, pretty print and convert phone numbers. python-phonen

Stefan Foulis 1.3k Dec 31, 2022
A Django web application to receive, virus check and validate transfers of digital archival records, and allow archivists to appraise and accession those records.

Aurora Aurora is a Django web application that can receive, virus check and validate transfers of digital archival records, and allows archivists to a

Rockefeller Archive Center 20 Aug 30, 2022
django CMS Association 1.6k Jan 06, 2023
Reusable, generic mixins for Django

django-braces Mixins for Django's class-based views. Documentation Read The Docs Installation Install from PyPI with pip: pip install django-braces Bu

Brack3t 1.9k Jan 05, 2023
A Powerful HTML white space remover for Django

HTML Whitespace remover for Django Introduction : A powerful tool to optimize Django rendered templates Why use "django_stip_whitespace" ? Adds line b

3 Jan 01, 2022
A set of functions related with Django

django-extra-tools Table of contents Installation Quick start Template filters parse_datetime parse_date parse_time parse_duration Aggregation First L

Tomasz Jakub Rup 3 Mar 04, 2020
Silk is a live profiling and inspection tool for the Django framework.

Silk is a live profiling and inspection tool for the Django framework. Silk intercepts and stores HTTP requests and database queries before presenting them in a user interface for further inspection:

Jazzband 3.7k Jan 02, 2023
It takes time to start a Django Project and make it almost production-ready.

It takes time to start a Django Project and make it almost production-ready. A developer needs to spend a lot of time installing required libraries, setup a database, setup cache as well as hiding se

Khan Asfi Reza 1 Jan 01, 2022
Intellicards-backend - A Django project bootstrapped with django-admin startproject mysite

Intellicards-backend - A Django project bootstrapped with django-admin startproject mysite

Fabrizio Torrico 2 Jan 13, 2022
Returns unicode slugs

Python Slugify A Python slugify application that handles unicode. Overview Best attempt to create slugs from unicode strings while keeping it DRY. Not

Val Neekman (AvidCoder) 1.3k Dec 23, 2022
PostgreSQL with Docker + Portainer + pgAdmin + Django local

django-postgresql-docker Running PostgreSQL with Docker + Portainer + pgAdmin + Django local for development. This project was done with: Python 3.9.8

Regis Santos 4 Jun 12, 2022
Dashboad Full Stack utilizando o Django.

Dashboard FullStack completa Projeto finalizado | Informações Cadastro de cliente Menu interatico mostrando quantidade de pessoas bloqueadas, liberada

Lucas Silva 1 Dec 15, 2021
Phoenix LiveView but for Django

Reactor, a LiveView library for Django Reactor enables you to do something similar to Phoenix framework LiveView using Django Channels. What's in the

Eddy Ernesto del Valle Pino 526 Jan 02, 2023
A simple demonstration of integrating a sentiment analysis tool in a django project

sentiment-analysis A simple demonstration of integrating a sentiment analysis tool in a django project (watch the video .mp4) To run this project : pi

2 Oct 16, 2021
Easily share data across your company via SQL queries. From Grove Collab.

SQL Explorer SQL Explorer aims to make the flow of data between people fast, simple, and confusion-free. It is a Django-based application that you can

Grove Collaborative 2.1k Dec 30, 2022
PEP-484 stubs for Django

pep484 stubs for Django This package contains type stubs and a custom mypy plugin to provide more precise static types and type inference for Django f

TypedDjango 1.1k Dec 30, 2022
A calendaring app for Django. It is now stable, Please feel free to use it now. Active development has been taken over by bartekgorny.

Django-schedule A calendaring/scheduling application, featuring: one-time and recurring events calendar exceptions (occurrences changed or cancelled)

Tony Hauber 814 Dec 26, 2022
Django based webapp pulling in crypto news and price data via api

Deploy Django in Production FTA project implementing containerization of Django Web Framework into Docker to be placed into Azure Container Services a

0 Sep 21, 2022