Pure Python tools for reading and writing all TIFF IFDs, sub-IFDs, and tags.

Overview

Tiff Tools Build Status codecov.io License

Pure Python tools for reading and writing all TIFF IFDs, sub-IFDs, and tags.

Developed by Kitware, Inc. with funding from The National Cancer Institute.

Example

import tifftools
info = tifftools.read_tiff('photograph.tif')
info['ifds'][0]['tags'][tifftools.Tag.ImageDescription.value] = {
    'data': 'A dog digging.',
    'datatype': tifftools.Datatype.ASCII
}
exififd = info['ifds'][0]['tags'][tifftools.Tag.EXIFIFD.value]['ifds'][0]
exififd['tags'][tifftools.constants.EXIFTag.FNumber.value] = {
    'data': [54, 10],
    'datatype': tifftools.Datatype.RATIONAL
}
tifftools.write_tiff(info, 'photograph_tagged.tif')

Commands

tifftools --help and tifftools --help provide usage details.

  • tifftools split [--subifds] [--overwrite] source [prefix]: split a tiff file into separate files. This is also available as the library function tifftools.tiff_split.
  • tifftools concat [--overwrite] source [source ...] output: merge multiple tiff files together. Alias: tifftools merge. This is also available as the library function tifftools.tiff_concat.
  • tifftools dump [--max MAX] [--json] source [source ...]: print information about a tiff file, including all tags, IFDs, and subIFDs. Alias: tifftool info. This is also available as the library function tifftools.tiff_dump.
  • tifftools set source [--overwrite] [output] [--set TAG[:DATATYPE][, ] VALUE] [--unset TAG:[, ]] [--setfrom TAG[, ] TIFFPATH] : modify, add, or remove tags. This is also available as the library function tifftools.tiff_set.

Library Functions

  • read_tiff
  • write_tiff
  • Constants
  • Tag
  • Datatype
  • get_or_create_tag
  • EXIFTag, GPSTag, etc.

Installation

tifftools is available on PyPI and conda-forge.

To install with pip from PyPI:

pip install tifftools

To install with conda:

conda install -c conda-forge tifftools

Purpose

tifftools provides a library and a command line program for maniplulating TIFF files. It can split multiple images apart, merge images together, set any tag in any IFD, and dump all IFDs and tags in a single command. It only uses python standard library modules, and is therefore widely compatible.

Rationale

There was a need to combine images from multiple TIFF files without altering the image data or losing any tag information. Further, when changing tag values, it was essential that the old values were fully removed from the output.

The command line tools associated with libtiff are commonly used for similar purposes. The libtiff command tools have significant limitations: tiffdump and tiffinfo require multiple commands to see information from all IFDs. tiffset does not remove data from a file; rather it appends to the file to only reference new data, leaving the old values inside the file. tiffsplit doesn't keep tags it doesn't recognize, loosing data. tiffcp always reencodes images and will fail for compression types it does not know.

Likewise, there is a wide variety of EXIF tools. For the most part, these only alter tags, usually by appending to the existing file. ImageMagick's convert command also recompresses images as it combines them.

Many programs deal with both classic and BigTIFF. Some will start writing a classic TIFF, but leave a small amount of unused space just after the file header. If the file exceeds 4Gb, parts of the file are rewritten to convert it to a BigTIFF file, leaving small amounts of abandoned data within the file.

tifftools fills this need. All tags are copied, even if unknown. Files are always rewritten so that there is never abandoned data inside the file. tifftools dump shows information on all IFDs and tags. Many of the command line options are directly inspired from libtiff.

tifftools does NOT compress or decompress any image data. This is not an image viewer. If you need to recompress an image or otherwise manipulate pixel data, use libtiff or another library.

As an explicit example, with libtiff's tiffset, tag data just gets dereferenced and is still in the file:

$ grep 'secret' photograph.tif  || echo 'not present'
not present
$ tiffset -s ImageDescription "secret phrase" photograph.tif
$ tiffinfo photograph.tif | grep ImageDescription
  ImageDescription: secret phrase
$ grep 'secret' photograph.tif  || echo 'not present'
Binary file photograph.tif matches
$ tiffset photograph.tif -s ImageDescription "public phrase"
$ tiffinfo photograph.tif | grep ImageDescription
  ImageDescription: public phrase
$ grep 'secret' photograph.tif  || echo 'not present'
Binary file photograph.tif matches

Whereas, with tifftools:

$ grep 'secret' photograph.tif || echo 'not present'
not present
$ tifftools set -y -s ImageDescription "secret phrase" photograph.tif
$ tiffinfo photograph.tif | grep ImageDescription
  ImageDescription: secret phrase
$ grep 'secret' photograph.tif || echo 'not present'
Binary file photograph.tif matches
$ tifftools set -y photograph.tif -s ImageDescription "public phrase"
$ tiffinfo photograph.tif | grep ImageDescription
  ImageDescription: public phrase $ grep 'secret' photograph.tif || echo
  'not present' not present

TIFF File Structure

TIFF Files consist of one or more IFDs (Image File Directories). These can be located anywhere within the file, and are referenced by their absolute position within the file. IFDs can refer to image data; they can also contain a collection of metadata (for instance, EXIF or GPS data). Small data values are stored directly in the IFD. Bigger data values (such as image data, longer strings, or lists of numbers) are referenced by the IFD and are stored elsewhere in the file.

In the simple case, a TIFF file may have a list of IFDs, each one referencing the next. However, a complex TIFF file, such as those used by some Whole-Slide Image (WSI) microscopy systems, can have IFDs organized in a branching structure, where some IFDs are in a list and some reference SubIFDs with additional images.

TIFF files can have their primary data stored in either little-endian or big-endian format. Offsets to data are stored as absolute numbers inside a TIFF file. There are two variations: "classic" and "BigTIFF" which use 32-bits and 64-bits for these offsets, respectively. If the file size exceeds 4 Gb or uses 64-bit integer datatypes, it must be written as a BigTIFF.

Limitations

Unknown tags that are offsets and have a datatype other than IFD or IFD8 won't be copied properly, as it is impossible to distinguish integer data from offsets given LONG or LONG8 datatypes. This can be remedied by defining a new TiffConstant record which contains a bytecounts entry to instruct whether the offsets refer to fixed length data or should get the length of data from another tag.

Because files are ALWAYS rewritten, tifftools is slower than libtiff's tiffset and most EXIF tools.

Comments
  • TIFF IFD concatenations/removals output images that fail JHOVE validation check for value offset word-alignment

    TIFF IFD concatenations/removals output images that fail JHOVE validation check for value offset word-alignment

    (https://jhove.openpreservation.org/modules/tiff/)

    > python -c 'import tifftools;tifftools.tiff_concat(["good1.svs", "good2.svs"], "out.svs", overwrite=True)'
    > jhove -m TIFF-hul out.svs
    
    Jhove (Rel. 1.24.1, 2020-03-16)
     Date: 2021-07-15 20:14:30 MDT
     RepresentationInformation: out.svs
      ReportingModule: TIFF-hul, Rel. 1.9.2 (2019-12-10)
      LastModified: 2021-07-15 20:14:20 MDT
      Size: 875685820
      Format: TIFF
      Status: Not well-formed
      SignatureMatches:
       TIFF-hul
      ErrorMessage: Value offset not word-aligned: 8842289
       ID: TIFF-HUL-4
       Offset: 8858224
      MIMEtype: image/tiff
    

    https://web.archive.org/web/20160324105748/https://partners.adobe.com/public/developer/en/tiff/TIFF6.pdf on page 15 (about IFD entries) says:

    Bytes 8-11 The Value Offset, the file offset (in bytes) of the Value for the field. The Value is expected to begin on a word boundary; the correspond- ing Value Offset will thus be an even number. This file offset may point anywhere in the file, even after the image data.

    opened by fiendish 4
  • Appending a new tag to an existing TIFF image

    Appending a new tag to an existing TIFF image

    Hi,

    I am trying to add a new tag to a TIF file using Python3.

    I have worked through your code and I am now able to understand the TIFF structure.

    However, I need to add a a new Tag. My code is an attempt to adapt your example for a similar action from the command line:

    info = tifftools.read_tiff(Fpath) info[info, 'ifds'][0]['tags'][tifftools.Tag.ImageDescription.value] = { 'data': 'A dog digging.', 'datatype': tifftools.Datatype.ASCII }

    tifftools.write_tiff(Fpath, OutPath, info)

    I am not the strongest Python programmer, but I am baffled why I cannot update the TIFF TAG dict structure. Can you guide me please?

    regards

    Phil

    opened by pfculverhouse 3
  • Readme example seems to be incorrect

    Readme example seems to be incorrect

    Running the sample code from the readme file gives:

    $ python3 sample.py
    Traceback (most recent call last):
      File "sample2.py", line 8, in <module>
        exififd['tags'][tifftools.constants.EXIFTag.FNumber.value] = {
    TypeError: list indices must be integers or slices, not str
    

    it seems that we are missing level for exifs (SubIDF?).

    The following does works:

    $ diff -u sample.py.orig sample.py
    --- sample.py.orig      2022-02-28 18:19:35.000000000 +0100
    +++ sample.py   2022-02-28 18:19:52.000000000 +0100
    @@ -4,7 +4,7 @@
         'data': 'A dog digging.',
         'datatype': tifftools.Datatype.ASCII
     }
    -exififd = info['ifds'][0]['tags'][tifftools.Tag.EXIFIFD.value]['ifds'][0]
    +exififd = info['ifds'][0]['tags'][tifftools.Tag.EXIFIFD.value]['ifds'][0][0]
     exififd['tags'][tifftools.constants.EXIFTag.FNumber.value] = {
         'data': [54, 10],
         'datatype': tifftools.Datatype.RATIONAL
    $ python3 sample.py
    $ tifftools dump photograph_tagged.tif | grep FNumber
          FNumber 33437 (0x829D) RATIONAL: 54 10 (5.4)
    

    Side note: the above code will fail if the TIFF file doesn't already contains EXIFs. Is the following snippet the correct way to add EXIF IFD?

    try:
        exif = info["ifds"][0]["tags"][tifftools.Tag.EXIFIFD.value]
    except KeyError:
        exif = {
            "datatype": tifftools.Datatype.IFD,
            "ifds": [[{"tags": {}, "path_or_fobj": info["ifds"][0]["path_or_fobj"]}]],
        }
        info["ifds"][0]["tags"][tifftools.Tag.EXIFIFD.value] = exif
    exififd = exif["ifds"][0][0]
    # add tags herunder
    
    opened by AmedeeBulle 2
  • Examples of write tiffs from scratch, not inheriting info from an existing tiff

    Examples of write tiffs from scratch, not inheriting info from an existing tiff

    Please could you provide more examples for how to use this tool to write tiffs from scratch? Thanks SO much for providing this tool. I really hope I can make it work for me!

    Problem:

    • I would like to write out numpy arrays to bigtiff with custom metadata AND exif data
    • I'm getting exif data from a jpeg, then generating outputs from that jpeg within a script, that I want to write those outputs to a tiff with exif and metadata

    Alternatives tried:

    • I cant use tifffile because it doesnt deal with exif data
    • I cant use PIL because it doesnt write exif data
    • Rasterio doesnt seem to have an exif option
    • I cant follow the provided example because I dont have a tiff to read as a starting point

    So I'm really hoping I can use tifftools, but I cant figure out (from the one provided example) how to write a tiff entirely from scratch using arrays

    Question: The provided example is useful, but only deals with the case where all the info is obtained from an existing file

    info = tifftools.read_tiff(file)

    But how would you create that info object from an numpy array, metadata, and exif separately? where metadata and exif are two separate dicts or JSON objects (or whatever)?

    I also found this that helpfully explains how to add info to an existing tiff

    In summary, I'm looking for advice for how to construct a viable info object that can be passed to .write_tif like this

    tifftools.write_tif(info, ...)

    where info is constructed from 1) a numpy array containing the image, 2) an array or dictionary of exif data, and 3) an array or dictionary of other metadata (tags)

    opened by dbuscombe-usgs 2
  • Make it easier to add new ifds by not requiring path_or_fobj.

    Make it easier to add new ifds by not requiring path_or_fobj.

    The path_or_fobj internal value is only required for ifds if they need to transfer data from an existing tiff file. If tags are entirely self-contained, this is no longer required to be set.

    opened by manthey 0
  • Better handle NDPI files

    Better handle NDPI files

    NDPI files aren't quite valid tiff files. Rather, they are marked as non-bigtiff, but use 64-bit values for ifd offsets and have some implied upper bits for some data offsets. If a file is larger than 4 Gb or of unknown length, and a NDPI-specific tag is encountered that could have otherwise invalid offsets, read additional IFD offsets as 64-bit values and adjust data offset values according to the NDPI methods as illustrated by the OpenSlide library.

    opened by manthey 0
  • Output values on word boundaries.

    Output values on word boundaries.

    Better handle saving to small tiff. Before, once written to a bigtiff, it was unlikely to convert back to a small tiff since some fields written as LONG8 didn't automatically convert to LONG.

    opened by manthey 0
  • More often generate small tiff

    More often generate small tiff

    Better handle saving to small tiff. Before, once written to a bigtiff, it was unlikely to convert back to a small tiff since some fields written as LONG8 didn't automatically convert to LONG.

    opened by manthey 0
Releases(v1.3.6)
Owner
Digital Slide Archive
Tools for the management, visualization, and analysis of digital pathology data.
Digital Slide Archive
MHS2 Save file editing tools. Transfers save files between players, switch and pc version, encrypts and decrypts.

SaveTools MHS2 Save file editing tools. Transfers save files between players, switch and pc version, encrypts and decrypts. Credits Written by Asteris

31 Nov 17, 2022
Python package to read and display segregated file names present in a directory based on type of the file

tpyfilestructure Python package to read and display segregated file names present in a directory based on type of the file. Installation You can insta

Tharun Kumar T 2 Nov 28, 2021
Python library and shell utilities to monitor filesystem events.

Watchdog Python API and shell utilities to monitor file system events. Works on 3.6+. If you want to use Python 2.6, you should stick with watchdog

Yesudeep Mangalapilly 5.6k Jan 04, 2023
Python virtual filesystem for SQLite to read from and write to S3

Python virtual filesystem for SQLite to read from and write to S3

Department for International Trade 70 Jan 04, 2023
A simple file module for creating, editing and saving files.

A simple file module for creating, editing and saving files.

1 Nov 25, 2021
pytiff is a lightweight library for reading chunks from a tiff file

pytiff is a lightweight library for reading chunks from a tiff file. While it supports other formats to some extend, it is focused on reading tiled greyscale/rgb images, that can also be bigtiffs. Wr

Big Data Analytics group 9 Mar 21, 2022
The best way to convert files on your computer, be it .pdf to .png, .pdf to .docx, .png to .ico, or anything you can imagine.

The best way to convert files on your computer, be it .pdf to .png, .pdf to .docx, .png to .ico, or anything you can imagine.

JareBear 2 Nov 20, 2021
Singer is an open source standard for moving data between databases, web APIs, files, queues, and just about anything else you can think of.

Singer is an open source standard for moving data between databases, web APIs, files, queues, and just about anything else you can think of. Th

Singer 1.1k Jan 05, 2023
OnedataFS is a PyFilesystem interface to Onedata virtual file system

OnedataFS OnedataFS is a PyFilesystem interface to Onedata virtual file system. As a PyFilesystem concrete class, OnedataFS allows you to work with On

onedata 0 Jan 10, 2022
Get Your TXT File Length !.

TXTLen Get Your TXT File Length !. Hi ๐Ÿ‘‹ , I'm Alireza A Python Developer Boy ๐Ÿ”ญ Iโ€™m currently working on my C# projects ๐ŸŒฑ Iโ€™m currently Learning CSh

Alireza Hasanzadeh 1 Jan 06, 2022
A small Python module for determining appropriate platform-specific dirs, e.g. a "user data dir".

the problem What directory should your app use for storing user data? If running on macOS, you should use: ~/Library/Application Support/AppName If

ActiveState Software 948 Dec 31, 2022
Measure file similarity in a many-to-many fashion

Mesi Mesi is a tool to measure the similarity in a many-to-many fashion of long-form documents like Python source code or technical writing. The outpu

GatorEducator 3 Feb 02, 2022
Object-oriented file system path manipulation

path (aka path pie, formerly path.py) implements path objects as first-class entities, allowing common operations on files to be invoked on those path

Jason R. Coombs 1k Dec 28, 2022
A simple file sharing tool written in python

Share it A simple file sharing tool written in python Installation If you are using Windows os you can directly Run .exe file -- download If you are

Sachit Yadav 7 Dec 16, 2022
This project is a set of programs that I use to create a README.md file.

๐Ÿค– codex-readme ๐Ÿ“œ codex-readme What is it? This project is a set of programs that I use to create a README.md file. How does it work? It reads progra

Tom Dรถrr 224 Jan 07, 2023
Automatically generates a TypeQL script for doing entity and relationship insertions from a .csv file, so you don't have to mess with writing TypeQL.

Automatically generates a TypeQL script for doing entity and relationship insertions from a .csv file, so you don't have to mess with writing TypeQL.

3 Feb 09, 2022
A python module to parse text files with contains secret variables.

A python module to parse text files with contains secret variables.

0 Dec 05, 2022
Nintendo Game Boy music assembly files parser into musicxml format

GBMusicParser Nintendo Game Boy music assembly files parser into musicxml format This python code will get an file.asm from the disassembly of a Game

1 Dec 11, 2021
A platform independent file lock for Python

py-filelock This package contains a single module, which implements a platform independent file lock in Python, which provides a simple way of inter-p

Benedikt Schmitt 497 Jan 05, 2023
Media file renamer and organizion tool

mnamer mnamer (media renamer) is an intelligent and highly configurable media organization utility. It parses media filenames for metadata, searches t

Jessy Williams 533 Dec 29, 2022