Kaitai is a declarative language for parsing binary formats. We need plugins that will:

• compile and run the kaitai parser on a bitcontainer (using python?)
• display the parsed information kind of like kaitai Web IDE
• easily allow blobs to be broken off into their own bitcontainers for further isolated analysis

https://kaitai.io/

@KOLANICH

System information:

hostname = fedora
uname -m = x86_64
uname -r = 5.16.16-200.fc35.x86_64
uname -s = Linux
uname -v = #1 SMP PREEMPT Sat Mar 19 13:52:41 UTC 2022

/usr/bin/uname -p = x86_64
/bin/uname -X     = unknown

/bin/arch              = x86_64
/usr/bin/arch -k       = unknown
/usr/convex/getsysinfo = unknown
/usr/bin/hostinfo      = unknown
/bin/machine           = unknown
/usr/bin/oslevel       = unknown
/bin/universe          = unknown

PATH: /home/alex/.cargo/bin
PATH: /home/alex/.local/bin
PATH: /home/alex/bin
PATH: /usr/local/bin
PATH: /usr/local/sbin
PATH: /usr/bin
PATH: /usr/sbin

g++ version: g++ (GCC) 11.2.1 20220127 (Red Hat 11.2.1-9) Python version: Python 3.10.4 conan version: Conan version 1.47.0 OpenSSL version: OpenSSL 1.1.1n FIPS 15 Mar 2022 cmake version: cmake version 3.22.2

Build log attached below.

hobbits-fedora-conan-build.log

My guess is that it's failing to build the Makefile properly, perhaps some values are not included. I know one of the errors is that python cannot locate g++ in ./configure even though it's installed so CXX=/usr/bin/g++ took care of that, but didn't change the error much.

> /Applications/hobbits.app/Contents/MacOS/hobbits 
dyld: Library not loaded: /usr/local/opt/gettext/lib/libintl.8.dylib
  Referenced from: /Applications/hobbits.app/Contents/Frameworks/hobbits-cpython/lib/libpython3.9.dylib
  Reason: image not found
fish: Job 1, '/Applications/hobbits.app/Conte…' terminated by signal SIGABRT (Abort)

I would suggest using macdeployqt for mac builds. It packages Qt a bit more nicely.

Mac 11.5.2, hobbits-0.51.1

Hi,

I detect a problem when the "split view" is applied after 'import' file or data -> the application crashes ; the "split view" is possible only before 'import', add other view crashes application after 'import'.

i m using pre-built binaries v0.46.7 (Hobbits (64-bit Windows 2016 msvc2017), Same issue with v0.46.4. Tested under 2 x Windows 10 pro 64-bit (20H2 & 2004).

Have you already observed this ?

Thanks Best regards

In the Kaitai Struct plugin, if you have an instance set by a value, it fails to appear in the "Parsed" tab.

for example:

instances:
  test1:
    pos: 0
    size: 1
  test2:
    value: 0xff

should result in something like this (from kaitai struct visualizer):

however, in Hobbits, it appears like this:

When I use the Kaitai Struct analyzer plugin on PNGs or BMPs the program closes (I presume it's crashing) with no error messages. I'm using the pre-built binary for Windows.

Hi,

got some question about plugins in python :

• In the JSON file, when I put "decimal" or "integer" type for my parameter, it doesn't change anything, in the both cases it's an integer, is it normal ? Do i miss something ?

• Can we put negative values in our integer field ? And so, can we precise a maximum and a minimum value in the JSON file for a specific parameter ?

• When I try to add a boolean parameter in my JSON file and execute my plugin in Hobbits, I've always this error :

"Operator Plugin Error: Plugin 'test' reported an error with its processing: Other errors:
Failed to parse arg 4"
(here the arg 4 is of course my boolean, if i put more parameters, it will say 'arg 5' or 'arg 6')

and this is my script in python :

def operate_on_bits(input, output_bits, output_info, perso_var, progress):
  if input.bits.size() > 300 and perso_var:  # the perso_var is my boolean
    output_info.add_highligt("python", "mychunck", 20, 3000)

  output_bits.resize(input.bits.size())
  for i in range(0, input.bits.size()):
    output_bits.set(i, input.bits.at(i))  # this is the only other modification in the default plugin (I removed the 'not')
    if progress.is_cancelled():
      return
    progress.set_progress(i, input.bit.size())

and my JSON file :

{
  "name": "test",
  "description": "plugin test",
  "tags": ["test"],
  "script": "main.py",
  "type": "operator",
  "extra_paths": [],
  "parameters": [
    {
      "name": "perso_var",
      "type": "boolean"
    }
  ]
}

The FFTW library was utilized in two Hobbits plugins, the Spectrogram and the Width Framer. Due to licensing conflicts, the FFTW library could no longer be used in Hobbits. After researching different libraries, @hello-adam and I decided to use the PFFFT library.

I have refactored the code for both the Spectrogram and the Width Framer. Also, I implemented a QComboBox that allows users to change the FFT Size within the Spectrogram. Prior to adding this, FFT Size was a QSpinBox that allowed users to manually enter the FFT size or use up and down arrows to increment. The QComboBox allows users to choose FFT sizes in powers of two, ranging from 2^5 to 2^14 (or, 32 to 16,384).

A new plugin to be able to read the data coming off of Bulk and Interrupt USB device Endpoints.

A UI to select from various devices that are plugged in or operating on the USB bus, after selecting a device, select the interface, the alternate setting, and endpoint to read from.

Select the number of times you want to read from a device, the delay in between transfers (so that you can use the device while its being read from), and the maximum duration to wait for a transfer before the transfer times out.

Hit "OK" to read the requested data from the device, hobbits will prompt you if an error is met.

May i miss something but the old way to make a plugin in C was to install wizards like said in the tutorial, but I'm unable to find it. Is their something i miss ? Did the way to do that improved ?

I made some researches and found this, from Adam. But i can't find the file in the Code part.

Can someone help me on this point ?

Thanks !

A file generated with the following command

 echo 2f fe b8 9b 22 00 00 00  00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00  00 00 a4 0f 08 00 1a 06 08 af d0 bc 82 06 | xxd -r -p > file.bin

opened with hobbits with the following ksy:

meta:
  file-extension: bla
  id: servermod
  endian: le
  imports:
    - /serialization/google_protobuf
seq:
  - id: records
    type: record
    repeat: eos
types:
  main_header:
    seq:
      - id: ip1
        type: u1
      - id: ip2
        type: u1
      - id: ip3
        type: u1
      - id: ip4
        type: u1
      - id: body_len
        type: u4le
  record:
    seq:
      - id: len_record
        type: main_header
        doc: Size of whole record, including all headers, footers and data
      - id: ctype
        type: u1
        doc: ctype
      - id: flow
        type: u1
        doc: flow
      - id: cid
        type: u4le
        doc: cid
      - id: did
        type: u4le
        doc: did
      - id: seq1
        type: u8le
        doc: seq1
      - id: command
        type: u2le
        doc: command
      - id: protobuf
        size: len_record.body_len - 24
        type: google_protobuf

When you open the file with the struct in kaitai IDE works correctly, but opening in hobbits when you select the protobuf part (the last 10 bytes - 08 af d0 bc 82 06 ) it's highligthed the start of the file.

If you concatenate some times and select the protobuf of any of the registers always it's highligthed the starting bytes of the file.

The command to generate with 3 concats is:

echo 2f fe b8 9b 22 00 00 00  00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00  00 00 a4 0f 08 00 1a 06 08 af d0 bc 82 06 | xxd -r -p > file.bin ; echo 2f fe b8 9b 22 00 00 00  00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00  00 00 a4 0f 08 00 1a 06 08 af d0 bc 82 06 | xxd -r -p >> file.bin ; echo 2f fe b8 9b 22 00 00 00  00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00  00 00 a4 0f 08 00 1a 06 08 af d0 bc 82 06 | xxd -r -p >> file.bin

If your system python is not 3.9 you're pretty screwed on analizing Kaitai Structs:

Python stderr:
Traceback (most recent call last):
  File "/tmp/HobbitsPythonrBHuvX/thescript.py", line 162, in parse_data
    struct_module = importlib.__import__(package_name, fromlist=[class_name])
  File "<frozen importlib._bootstrap>", line 1109, in __import__
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/tmp/hobbits-GGgAzl/png.py", line 3, in <module>
    from pkg_resources import parse_version
ModuleNotFoundError: No module named 'pkg_resources'

Take this ksy for example:

meta:
  id: test
seq:
- id: test
  type: test
types:
  test:
    seq:
    - id: t
      type: u1

The parsed output table only contains columns 1 and 2, not the third 'value' column.

Modifying the ksy by adding a field which uses a builtin type at the start of the sequence fixes the issue

meta:
  id: test
seq:
- id: test1
  type: u1
- id: test
  type: test
types:
  test:
    seq:
    - id: t
      type: u1

Hello,

Thank you very much for the wonderful software!

I was wondering, how do I copy to clipboard from the hex editor? I can't seem to select anything.

I have limited knowledge of Qt so I'm not sure if this is even possible with the way hobbits-runner is implemented currently, but it for some reason still needs a display to run, even though it is a purely CLI tool. When trying to run it over SSH i get the error in the screenshot below.

Would be very valuable to interpret the bits beginning at cursor as a decimal number or floating point while in hex and binary view. In other hex editors this feature is useful to identify file offsets, for example.