Marshall python objects to and from JSON

Overview

Pymarshaler - Marshal and Unmarshal Python Objects

Disclaimer

This tool is in no way production ready

About

Pymarshaler allows you to marshal and unmarshal any python object directly to and from a JSON formatted string.

Pymarshaler takes advantage of python's new typing support. By reading class init param types, we are able to walk down nested JSON structures and assign appropriate values.

Basic Usage

Declare a class with typing information

Note, we can use regular old classes as long as their init methods are annotated properly, but it's preferable to use dataclasses whenever possible

from dataclasses import dataclass

@dataclass
class Test:
    
    name: str

That's it! We can now marshal, and more importantly, unmarshal this object to and from JSON.

from pymarshaler.marshal import Marshal
import json

test_instance = Test('foo')
blob = Marshal.marshal(test_instance)
print(blob.decode())
>>> '{name: foo}'

marshal = Marshal()
result = marshal.unmarshal(Test, json.loads(blob))
print(result.name)
>>> 'foo'

We also use marshal.unmarshal_str(cls, str) if we want to unmarshal directly from the blob source.

This is a pretty trivial example, lets add in a nested class

from dataclasses import dataclass

@dataclass
class StoresTest:
    
    test: Test

    
stores_test = StoresTest(Test('foo'))
blob = marshal.marshal(stores_test)
print(blob)
>>> '{test: {name: foo}}'

result = marshal.unmarshal(StoresTest, json.loads(blob))
print(result.test.name)
>>> 'foo'

As you can see, adding a nested class is as simple as as adding a basic structure.

Pymarshaler will fail when encountering an unknown field by default, however you can configure it to ignore unknown fields

from pymarshaler.marshal import Marshal 
from pymarshaler.arg_delegates import ArgBuilderFactory

marshal = Marshal()
blob = {'test': 'foo', 'unused_field': 'blah'}
result = marshal.unmarshal(Test, blob)
>>> 'Found unknown field (unused_field: blah). If you would like to skip unknown fields create a Marshal object who can skip ignore_unknown_fields'

marhsal = Marshal(ignore_unknown_fields=True)
result = marshal.unmarshal(Test, blob)
print(result.name)
>>> 'foo'

Advanced Usage

We can use pymarshaler to handle containers as well. Again we take advantage of python's robust typing system

>> '{foo, bar}'">
from dataclasses import dataclass
from pymarshaler.marshal import Marshal
from typing import Set
import json

@dataclass
class TestContainer:
 
    container: Set[str]
    

marshal = Marshal()
container_instance = TestContainer({'foo', 'bar'})        
blob = marshal.marshal(container_instance)
print(blob.decode())
>>> '{container: ["foo", "bar"]}'

result = marshal.unmarshal(TestContainer,json.loads(blob))
print(result.container)
>>> '{foo, bar}'

Pymarshaler can also handle containers that store user defined types. The Set[str] could easily have been Set[UserDefinedType]

Pymarshaler also supports default values, and will use any default values supplied in the __init__ if those values aren't present in the JSON data.

from dataclasses import dataclass
from pymarshaler.marshal import Marshal

@dataclass
class TestWithDefault:
    
    name: str = 'foo'


marshal = Marshal()
result = marshal.unmarshal(TestWithDefault, {})
print(result.name)
>>> 'foo'

Pymarshaler will raise an error if any non-default attributes aren't given

Pymarshaler also supports a validate method on creation of the python object. This method will be called before being returned to the user.

from dataclasses import dataclass
from pymarshaler.marshal import Marshal


@dataclass
class TestWithValidate:
    
    name: str

    def validate(self):
        print(f'My name is {self.name}!')


marshal = Marshal()
result = marshal.unmarshal(TestWithValidate, {'name': 'foo'})
>>> 'My name is foo!'

This can be used to validate the python object right at construction, potentially raising an error if any of the fields have invalid values

It's also possible to register your own custom unmarshaler for specific user defined classes.

from dataclasses import dataclass

from pymarshaler.arg_delegates import ArgBuilderDelegate 
from pymarshaler.marshal import Marshal


@dataclass
class ClassWithMessage:
    
    message: str        


class ClassWithCustomDelegate:

    def __init__(self, message_obj: ClassWithMessage):
        self.message_obj = message_obj


class CustomDelegate(ArgBuilderDelegate):

    def __init__(self, cls):
        super().__init__(cls)

    def resolve(self, data):
        return ClassWithCustomDelegate(ClassWithMessage(data['message']))


marshal = Marshal()
marshal.register_delegate(ClassWithCustomDelegate, CustomDelegate)
result = marshal.unmarshal(ClassWithCustomDelegate, {'message': 'Hello from the custom delegate!'})
print(result.message_obj)
>>> 'Hello from the custom delegate!'

The result from any delegate should be the initialized resulting class instance

You might also like...
cysimdjson - Very fast Python JSON parsing library

Fast JSON parsing library for Python, 7-12 times faster than standard Python JSON parser.

simplejson is a simple, fast, extensible JSON encoder/decoder for Python

simplejson simplejson is a simple, fast, complete, correct and extensible JSON http://json.org encoder and decoder for Python 3.3+ with legacy suppo

import json files directly in your python scripts
import json files directly in your python scripts

Install Install from git repository pip install git+https://github.com/zaghaghi/direct-json-import.git Use With the following json in a file named inf

Python script for converting .json to .md files using Mako templates.

Install Just install poetry and update script dependencies Usage Put your settings in settings.py and .json data (optionally, with attachments) in dat

json|dict to python object

Pyonize convert json|dict to python object Setup pip install pyonize Examples from pyonize import pyonize

Editor for json/standard python data
Editor for json/standard python data

Editor for json/standard python data

Convert your JSON data to a valid Python object to allow accessing keys with the member access operator(.)

JSONObjectMapper Allows you to transform JSON data into an object whose members can be queried using the member access operator. Unlike json.dumps in

Define your JSON schema as Python dataclasses

Define your JSON schema as Python dataclasses

A Python tool that parses JSON documents using JsonPath

A Python tool that parses JSON documents using JsonPath

Comments
  • [0.4.0] delegates are now functions to avoid creating a ton of classes

    [0.4.0] delegates are now functions to avoid creating a ton of classes

    Rather than using classes as delegates, we use functions. This means we aren't spawning classes for every single call to resolve. This is a performance boost, and reduces memory consumption, and garbage collection.

    This will change up how this package is used which I think is acceptable given that we are on version < 1

    opened by hgromer 1
  • Improve performance

    Improve performance

    Right now, we are performing an allocation for every delegate we create. We can make this more performant.

    One option is to use function pointers rather than classes to resolve data into its class form

    enhancement 
    opened by hgromer 0
Releases(0.4.0)
Owner
Hernan Romer
Software Engineer at HubSpot.
Hernan Romer
MOSP is a platform for creating, editing and sharing validated JSON objects of any type.

MONARC Objects Sharing Platform Presentation MOSP is a platform for creating, editing and sharing validated JSON objects of any type. You can use any

CASES Luxembourg 72 Dec 14, 2022
Atom, RSS and JSON feed parser for Python 3

Atoma Atom, RSS and JSON feed parser for Python 3. Quickstart Install Atoma with pip: pip install atoma

Nicolas Le Manchet 95 Nov 28, 2022
JSONx - Easy JSON wrapper packed with features.

🈷️ JSONx Easy JSON wrapper packed with features. This was made for small discord bots, for big bots you should not use this JSON wrapper. 📥 Usage Cl

2 Dec 25, 2022
Make JSON serialization easier

Make JSON serialization easier

4 Jun 30, 2022
Convert Wii UI formats to JSON5 and vice versa

Convert Wii UI formats to JSON5 and vice versa

Pablo Stebler 11 Aug 28, 2022
Python script to extract news from RSS feeds and save it as json.

Python script to extract news from RSS feeds and save it as json.

Alex Trbznk 14 Dec 22, 2022
A fast streaming JSON parser for Python that generates SAX-like events using yajl

json-streamer jsonstreamer provides a SAX-like push parser via the JSONStreamer class and a 'object' parser via the ObjectStreamer class which emits t

Kashif Razzaqui 196 Dec 15, 2022
JSON Schema validation library

jsonschema A JSON Schema validator implementation. It compiles schema into a validation tree to have validation as fast as possible. Supported drafts:

Dmitry Dygalo 309 Jan 01, 2023
Same as json.dumps or json.loads, feapson support feapson.dumps and feapson.loads

Same as json.dumps or json.loads, feapson support feapson.dumps and feapson.loads

boris 5 Dec 01, 2021
Simple Python Library to convert JSON to XML

json2xml Simple Python Library to convert JSON to XML

Vinit Kumar 79 Nov 11, 2022
This open source Python project allow you to create JSON data trees using Minmup.com

This open source Python project allow you to create JSON data trees using Minmup.com. I try to develop this project all the time. But feel free to use :).

Arttu Väisänen 1 Jan 30, 2022
The ldap2json script allows you to extract the whole LDAP content of a Windows domain into a JSON file.

ldap2json The ldap2json script allows you to extract the whole LDAP content of a Windows domain into a JSON file. Features Authenticate with password

Podalirius 68 Dec 07, 2022
A python library to convert arbitrary strings representing business opening hours into a JSON format that's easier to use in code

A python library to convert arbitrary strings representing business opening hours into a JSON format that's easier to use in code

Adrian Edwards 9 Dec 02, 2022
Random JSON Key:Pair Json Generator

Random JSON Key:Value Pair Generator This simple script take an engish dictionary of words and and makes random key value pairs. The dictionary has ap

Chris Edwards 1 Oct 14, 2021
cysimdjson - Very fast Python JSON parsing library

Fast JSON parsing library for Python, 7-12 times faster than standard Python JSON parser.

TeskaLabs 235 Dec 29, 2022
Wikidot-forum-dump - Simple Python script that dumps a Wikidot wiki forum into JSON structures.

wikidot-forum-dump Script is partially based on 2stacks by bluesoul: https://github.com/scuttle/2stacks To dump a Wiki's forum, edit config.py and put

ZZYZX 1 Jun 29, 2022
A Python application to transfer Zeek ASCII (not JSON) logs to Elastic/OpenSearch.

zeek2es.py This Python application translates Zeek's ASCII TSV logs into ElasticSearch's bulk load JSON format. For JSON logs, see Elastic's File Beat

Corelight, Inc. 28 Dec 22, 2022
Convert your JSON data to a valid Python object to allow accessing keys with the member access operator(.)

JSONObjectMapper Allows you to transform JSON data into an object whose members can be queried using the member access operator. Unlike json.dumps in

Owen Trump 4 Jul 20, 2022
A Python tool that parses JSON documents using JsonPath

A Python tool that parses JSON documents using JsonPath

8 Dec 18, 2022
A tools to find the path of a specific key in deep nested JSON.

如何快速从深层嵌套 JSON 中找到特定的 Key #公众号 在爬虫开发的过程中,我们经常遇到一些 Ajax 加载的接口会返回 JSON 数据。

kingname 56 Dec 13, 2022