Pure Python tools for reading and writing all TIFF IFDs, sub-IFDs, and tags.

Overview

Tiff Tools Build Status codecov.io License

Pure Python tools for reading and writing all TIFF IFDs, sub-IFDs, and tags.

Developed by Kitware, Inc. with funding from The National Cancer Institute.

Example

import tifftools
info = tifftools.read_tiff('photograph.tif')
info['ifds'][0]['tags'][tifftools.Tag.ImageDescription.value] = {
    'data': 'A dog digging.',
    'datatype': tifftools.Datatype.ASCII
}
exififd = info['ifds'][0]['tags'][tifftools.Tag.EXIFIFD.value]['ifds'][0]
exififd['tags'][tifftools.constants.EXIFTag.FNumber.value] = {
    'data': [54, 10],
    'datatype': tifftools.Datatype.RATIONAL
}
tifftools.write_tiff(info, 'photograph_tagged.tif')

Commands

tifftools --help and tifftools --help provide usage details.

  • tifftools split [--subifds] [--overwrite] source [prefix]: split a tiff file into separate files. This is also available as the library function tifftools.tiff_split.
  • tifftools concat [--overwrite] source [source ...] output: merge multiple tiff files together. Alias: tifftools merge. This is also available as the library function tifftools.tiff_concat.
  • tifftools dump [--max MAX] [--json] source [source ...]: print information about a tiff file, including all tags, IFDs, and subIFDs. Alias: tifftool info. This is also available as the library function tifftools.tiff_dump.
  • tifftools set source [--overwrite] [output] [--set TAG[:DATATYPE][, ] VALUE] [--unset TAG:[, ]] [--setfrom TAG[, ] TIFFPATH] : modify, add, or remove tags. This is also available as the library function tifftools.tiff_set.

Library Functions

  • read_tiff
  • write_tiff
  • Constants
  • Tag
  • Datatype
  • get_or_create_tag
  • EXIFTag, GPSTag, etc.

Installation

tifftools is available on PyPI and conda-forge.

To install with pip from PyPI:

pip install tifftools

To install with conda:

conda install -c conda-forge tifftools

Purpose

tifftools provides a library and a command line program for maniplulating TIFF files. It can split multiple images apart, merge images together, set any tag in any IFD, and dump all IFDs and tags in a single command. It only uses python standard library modules, and is therefore widely compatible.

Rationale

There was a need to combine images from multiple TIFF files without altering the image data or losing any tag information. Further, when changing tag values, it was essential that the old values were fully removed from the output.

The command line tools associated with libtiff are commonly used for similar purposes. The libtiff command tools have significant limitations: tiffdump and tiffinfo require multiple commands to see information from all IFDs. tiffset does not remove data from a file; rather it appends to the file to only reference new data, leaving the old values inside the file. tiffsplit doesn't keep tags it doesn't recognize, loosing data. tiffcp always reencodes images and will fail for compression types it does not know.

Likewise, there is a wide variety of EXIF tools. For the most part, these only alter tags, usually by appending to the existing file. ImageMagick's convert command also recompresses images as it combines them.

Many programs deal with both classic and BigTIFF. Some will start writing a classic TIFF, but leave a small amount of unused space just after the file header. If the file exceeds 4Gb, parts of the file are rewritten to convert it to a BigTIFF file, leaving small amounts of abandoned data within the file.

tifftools fills this need. All tags are copied, even if unknown. Files are always rewritten so that there is never abandoned data inside the file. tifftools dump shows information on all IFDs and tags. Many of the command line options are directly inspired from libtiff.

tifftools does NOT compress or decompress any image data. This is not an image viewer. If you need to recompress an image or otherwise manipulate pixel data, use libtiff or another library.

As an explicit example, with libtiff's tiffset, tag data just gets dereferenced and is still in the file:

$ grep 'secret' photograph.tif  || echo 'not present'
not present
$ tiffset -s ImageDescription "secret phrase" photograph.tif
$ tiffinfo photograph.tif | grep ImageDescription
  ImageDescription: secret phrase
$ grep 'secret' photograph.tif  || echo 'not present'
Binary file photograph.tif matches
$ tiffset photograph.tif -s ImageDescription "public phrase"
$ tiffinfo photograph.tif | grep ImageDescription
  ImageDescription: public phrase
$ grep 'secret' photograph.tif  || echo 'not present'
Binary file photograph.tif matches

Whereas, with tifftools:

$ grep 'secret' photograph.tif || echo 'not present'
not present
$ tifftools set -y -s ImageDescription "secret phrase" photograph.tif
$ tiffinfo photograph.tif | grep ImageDescription
  ImageDescription: secret phrase
$ grep 'secret' photograph.tif || echo 'not present'
Binary file photograph.tif matches
$ tifftools set -y photograph.tif -s ImageDescription "public phrase"
$ tiffinfo photograph.tif | grep ImageDescription
  ImageDescription: public phrase $ grep 'secret' photograph.tif || echo
  'not present' not present

TIFF File Structure

TIFF Files consist of one or more IFDs (Image File Directories). These can be located anywhere within the file, and are referenced by their absolute position within the file. IFDs can refer to image data; they can also contain a collection of metadata (for instance, EXIF or GPS data). Small data values are stored directly in the IFD. Bigger data values (such as image data, longer strings, or lists of numbers) are referenced by the IFD and are stored elsewhere in the file.

In the simple case, a TIFF file may have a list of IFDs, each one referencing the next. However, a complex TIFF file, such as those used by some Whole-Slide Image (WSI) microscopy systems, can have IFDs organized in a branching structure, where some IFDs are in a list and some reference SubIFDs with additional images.

TIFF files can have their primary data stored in either little-endian or big-endian format. Offsets to data are stored as absolute numbers inside a TIFF file. There are two variations: "classic" and "BigTIFF" which use 32-bits and 64-bits for these offsets, respectively. If the file size exceeds 4 Gb or uses 64-bit integer datatypes, it must be written as a BigTIFF.

Limitations

Unknown tags that are offsets and have a datatype other than IFD or IFD8 won't be copied properly, as it is impossible to distinguish integer data from offsets given LONG or LONG8 datatypes. This can be remedied by defining a new TiffConstant record which contains a bytecounts entry to instruct whether the offsets refer to fixed length data or should get the length of data from another tag.

Because files are ALWAYS rewritten, tifftools is slower than libtiff's tiffset and most EXIF tools.

Comments
  • TIFF IFD concatenations/removals output images that fail JHOVE validation check for value offset word-alignment

    TIFF IFD concatenations/removals output images that fail JHOVE validation check for value offset word-alignment

    (https://jhove.openpreservation.org/modules/tiff/)

    > python -c 'import tifftools;tifftools.tiff_concat(["good1.svs", "good2.svs"], "out.svs", overwrite=True)'
    > jhove -m TIFF-hul out.svs
    
    Jhove (Rel. 1.24.1, 2020-03-16)
     Date: 2021-07-15 20:14:30 MDT
     RepresentationInformation: out.svs
      ReportingModule: TIFF-hul, Rel. 1.9.2 (2019-12-10)
      LastModified: 2021-07-15 20:14:20 MDT
      Size: 875685820
      Format: TIFF
      Status: Not well-formed
      SignatureMatches:
       TIFF-hul
      ErrorMessage: Value offset not word-aligned: 8842289
       ID: TIFF-HUL-4
       Offset: 8858224
      MIMEtype: image/tiff
    

    https://web.archive.org/web/20160324105748/https://partners.adobe.com/public/developer/en/tiff/TIFF6.pdf on page 15 (about IFD entries) says:

    Bytes 8-11 The Value Offset, the file offset (in bytes) of the Value for the field. The Value is expected to begin on a word boundary; the correspond- ing Value Offset will thus be an even number. This file offset may point anywhere in the file, even after the image data.

    opened by fiendish 4
  • Appending a new tag to an existing TIFF image

    Appending a new tag to an existing TIFF image

    Hi,

    I am trying to add a new tag to a TIF file using Python3.

    I have worked through your code and I am now able to understand the TIFF structure.

    However, I need to add a a new Tag. My code is an attempt to adapt your example for a similar action from the command line:

    info = tifftools.read_tiff(Fpath) info[info, 'ifds'][0]['tags'][tifftools.Tag.ImageDescription.value] = { 'data': 'A dog digging.', 'datatype': tifftools.Datatype.ASCII }

    tifftools.write_tiff(Fpath, OutPath, info)

    I am not the strongest Python programmer, but I am baffled why I cannot update the TIFF TAG dict structure. Can you guide me please?

    regards

    Phil

    opened by pfculverhouse 3
  • Readme example seems to be incorrect

    Readme example seems to be incorrect

    Running the sample code from the readme file gives:

    $ python3 sample.py
    Traceback (most recent call last):
      File "sample2.py", line 8, in <module>
        exififd['tags'][tifftools.constants.EXIFTag.FNumber.value] = {
    TypeError: list indices must be integers or slices, not str
    

    it seems that we are missing level for exifs (SubIDF?).

    The following does works:

    $ diff -u sample.py.orig sample.py
    --- sample.py.orig      2022-02-28 18:19:35.000000000 +0100
    +++ sample.py   2022-02-28 18:19:52.000000000 +0100
    @@ -4,7 +4,7 @@
         'data': 'A dog digging.',
         'datatype': tifftools.Datatype.ASCII
     }
    -exififd = info['ifds'][0]['tags'][tifftools.Tag.EXIFIFD.value]['ifds'][0]
    +exififd = info['ifds'][0]['tags'][tifftools.Tag.EXIFIFD.value]['ifds'][0][0]
     exififd['tags'][tifftools.constants.EXIFTag.FNumber.value] = {
         'data': [54, 10],
         'datatype': tifftools.Datatype.RATIONAL
    $ python3 sample.py
    $ tifftools dump photograph_tagged.tif | grep FNumber
          FNumber 33437 (0x829D) RATIONAL: 54 10 (5.4)
    

    Side note: the above code will fail if the TIFF file doesn't already contains EXIFs. Is the following snippet the correct way to add EXIF IFD?

    try:
        exif = info["ifds"][0]["tags"][tifftools.Tag.EXIFIFD.value]
    except KeyError:
        exif = {
            "datatype": tifftools.Datatype.IFD,
            "ifds": [[{"tags": {}, "path_or_fobj": info["ifds"][0]["path_or_fobj"]}]],
        }
        info["ifds"][0]["tags"][tifftools.Tag.EXIFIFD.value] = exif
    exififd = exif["ifds"][0][0]
    # add tags herunder
    
    opened by AmedeeBulle 2
  • Examples of write tiffs from scratch, not inheriting info from an existing tiff

    Examples of write tiffs from scratch, not inheriting info from an existing tiff

    Please could you provide more examples for how to use this tool to write tiffs from scratch? Thanks SO much for providing this tool. I really hope I can make it work for me!

    Problem:

    • I would like to write out numpy arrays to bigtiff with custom metadata AND exif data
    • I'm getting exif data from a jpeg, then generating outputs from that jpeg within a script, that I want to write those outputs to a tiff with exif and metadata

    Alternatives tried:

    • I cant use tifffile because it doesnt deal with exif data
    • I cant use PIL because it doesnt write exif data
    • Rasterio doesnt seem to have an exif option
    • I cant follow the provided example because I dont have a tiff to read as a starting point

    So I'm really hoping I can use tifftools, but I cant figure out (from the one provided example) how to write a tiff entirely from scratch using arrays

    Question: The provided example is useful, but only deals with the case where all the info is obtained from an existing file

    info = tifftools.read_tiff(file)

    But how would you create that info object from an numpy array, metadata, and exif separately? where metadata and exif are two separate dicts or JSON objects (or whatever)?

    I also found this that helpfully explains how to add info to an existing tiff

    In summary, I'm looking for advice for how to construct a viable info object that can be passed to .write_tif like this

    tifftools.write_tif(info, ...)

    where info is constructed from 1) a numpy array containing the image, 2) an array or dictionary of exif data, and 3) an array or dictionary of other metadata (tags)

    opened by dbuscombe-usgs 2
  • Make it easier to add new ifds by not requiring path_or_fobj.

    Make it easier to add new ifds by not requiring path_or_fobj.

    The path_or_fobj internal value is only required for ifds if they need to transfer data from an existing tiff file. If tags are entirely self-contained, this is no longer required to be set.

    opened by manthey 0
  • Better handle NDPI files

    Better handle NDPI files

    NDPI files aren't quite valid tiff files. Rather, they are marked as non-bigtiff, but use 64-bit values for ifd offsets and have some implied upper bits for some data offsets. If a file is larger than 4 Gb or of unknown length, and a NDPI-specific tag is encountered that could have otherwise invalid offsets, read additional IFD offsets as 64-bit values and adjust data offset values according to the NDPI methods as illustrated by the OpenSlide library.

    opened by manthey 0
  • Output values on word boundaries.

    Output values on word boundaries.

    Better handle saving to small tiff. Before, once written to a bigtiff, it was unlikely to convert back to a small tiff since some fields written as LONG8 didn't automatically convert to LONG.

    opened by manthey 0
  • More often generate small tiff

    More often generate small tiff

    Better handle saving to small tiff. Before, once written to a bigtiff, it was unlikely to convert back to a small tiff since some fields written as LONG8 didn't automatically convert to LONG.

    opened by manthey 0
Releases(v1.3.6)
Owner
Digital Slide Archive
Tools for the management, visualization, and analysis of digital pathology data.
Digital Slide Archive
Python package to read and display segregated file names present in a directory based on type of the file

tpyfilestructure Python package to read and display segregated file names present in a directory based on type of the file. Installation You can insta

Tharun Kumar T 2 Nov 28, 2021
Object-oriented file system path manipulation

path (aka path pie, formerly path.py) implements path objects as first-class entities, allowing common operations on files to be invoked on those path

Jason R. Coombs 1k Dec 28, 2022
Kartothek - a Python library to manage large amounts of tabular data in a blob store

Kartothek - a Python library to manage (create, read, update, delete) large amounts of tabular data in a blob store

15 Dec 25, 2022
CleverCSV is a Python package for handling messy CSV files.

CleverCSV is a Python package for handling messy CSV files. It provides a drop-in replacement for the builtin CSV module with improved dialect detection, and comes with a handy command line applicati

The Alan Turing Institute 1k Dec 19, 2022
organize - The file management automation tool

organize - The file management automation tool

Thomas Feldmann 1.5k Jan 01, 2023
Python library and shell utilities to monitor filesystem events.

Watchdog Python API and shell utilities to monitor file system events. Works on 3.6+. If you want to use Python 2.6, you should stick with watchdog

Yesudeep Mangalapilly 5.6k Jan 04, 2023
shred - A cross-platform library for securely deleting files beyond recovery.

shred Help the project financially: Donate: https://smartlegion.github.io/donate/ Yandex Money: https://yoomoney.ru/to/4100115206129186 PayPal: https:

4 Sep 04, 2021
Generates a clean .txt file of contents of a 3 lined csv file

Generates a clean .txt file of contents of a 3 lined csv file. File contents is the .gml file of some function which stores the contents of the csv as a map.

Alex Eckardt 1 Jan 09, 2022
MHS2 Save file editing tools. Transfers save files between players, switch and pc version, encrypts and decrypts.

SaveTools MHS2 Save file editing tools. Transfers save files between players, switch and pc version, encrypts and decrypts. Credits Written by Asteris

31 Nov 17, 2022
A bot discord that can create directories, file, rename, move, navigate throw directories etc....

File Manager Discord What is the purpose of this program ? This program is made for a Discord bot. Its purpose is to organize the messages sent in a c

1 Feb 02, 2022
A Python script to organize your files in a given directory.

File-Organizer A Python script to organize your files in a given directory. It organizes your files based on the file extension and moves them into sp

Imira Randeniya 1 Sep 11, 2022
Sheet Data Image/PDF-to-CSV Converter

Sheet Data Image/PDF-to-CSV Converter

Quy Truong 5 Nov 22, 2021
Search for files under the specified directory. Extract the file name and file path and import them as data.

Search for files under the specified directory. Extract the file name and file path and import them as data. Based on that, search for the file, select it and open it.

G-jon FujiYama 2 Jan 10, 2022
PaddingZip - a tool that you can craft a zip file that contains the padding characters between the file content.

PaddingZip - a tool that you can craft a zip file that contains the padding characters between the file content.

phithon 53 Nov 07, 2022
fast change directory with python and ruby

fcdir fast change directory with python and ruby run run python script , chose drirectoy and change your directory need you need python and ruby deskt

XCO 2 Jun 20, 2022
This project is a set of programs that I use to create a README.md file.

๐Ÿค– codex-readme ๐Ÿ“œ codex-readme What is it? This project is a set of programs that I use to create a README.md file. How does it work? It reads progra

Tom Dรถrr 224 Jan 07, 2023
Get Your TXT File Length !.

TXTLen Get Your TXT File Length !. Hi ๐Ÿ‘‹ , I'm Alireza A Python Developer Boy ๐Ÿ”ญ Iโ€™m currently working on my C# projects ๐ŸŒฑ Iโ€™m currently Learning CSh

Alireza Hasanzadeh 1 Jan 06, 2022
Quick and dirty FAT12 filesystem to ZIP file converter

Quick and Dirty FAT12 Filesystem Converter This is a really crappy Python script I wrote to convert a semi-compatible FAT12 filesystem from my HP150's

Tube Time 2 Feb 12, 2022
CSV-Handler written in Python3

CSVHandler This code allows you to work intelligently with CSV files. A file in CSV syntax is converted into several lists, which are combined in a to

Max Tischberger 1 Jan 13, 2022
Better directory iterator and faster os.walk(), now in the Python 3.5 stdlib

scandir, a better directory iterator and faster os.walk() scandir() is a directory iteration function like os.listdir(), except that instead of return

Ben Hoyt 506 Dec 29, 2022