Hi all, as soon as I update Arduino Core from 2.0.3 to 2.0.4 and PlatformIO from 5.0.0 to 5.1.0 my Little FS is corrupted after flashing my firmware with esp web tools.

this is my simple json:

{
   "name":"Glow Worm Luciferin FULL",
   "version":"5.6.3",
   "new_install_improv_wait_time":40,
   "builds":[
      {
         "chipFamily":"ESP32",
         "parts":[
            {
               "path":"GlowWormLuciferinFULL_ESP32_firmware-factory.bin",
               "offset":0
            }
         ]
      },
      {
         "chipFamily":"ESP8266",
         "parts":[
            {
               "path":"GlowWormLuciferinFULL_ESP8266_firmware.bin",
               "offset":0
            }
         ]
      }
   ]
}

and this is the script that I use to create a factory bin.

def esp32_create_factory_bin(source, target, env):
    print("Generating factory bin for genuine esp units")
    #offset = 0x1000
    offset = 0x0
    new_file_name = env.subst("$BUILD_DIR/${PROGNAME}-factory.bin")
    sections = env.subst(env.get('FLASH_EXTRA_IMAGES'))
    new_file = open(new_file_name,"wb")
    for section in sections:
        sect_adr,sect_file = section.split(" ",1)
        source = open(sect_file,"rb")
        new_file.seek(int(sect_adr,0)-offset)
        new_file.write(source.read());
        source.close()

    firmware = open(env.subst("$BUILD_DIR/${PROGNAME}.bin"),"rb")
    new_file.seek(0x10000-offset)
    new_file.write(firmware.read())
    new_file.close()
    firmware.close()

env.AddPostAction("$BUILD_DIR/${PROGNAME}.bin", esp32_create_factory_bin)

It worked well before but now it is broken... As soon as I flash the firmware with Arduino core 2.0.4, LittleFS got corrupted. It works well with Arduino core 2.0.3. I think that this can be related to flash mode (QIO is now needed for some boards), is there a way to specify the flash mode with esp web tools?

Any suggestions will be really appreciated :)

Thanks Davide

Flashing using esp tool is OK esptool.exe --chip esp32 --port COM6 --baud 921600 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 80m --flash_size 4MB 0xe000 boot_app0.bin 0x1000 Programm.ino.bootloader.bin 0x10000 Programm.ino.bin 0x8000 Programm.ino.partitions.bin

But when esp web tools are used (V6.1.1, V5.3.0 tested too) the endless reset loop is shown in terminal log:

rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:1184
load:0x40078000,len:12804
ho 0 tail 12 room 4
load:0x40080400,len:3032
entry 0x400805e4

Manifest file used:

{
  "name": "Program",
  "version": "11",
  "builds": [
    {
      "chipFamily": "ESP32",
      "parts": [
        { "path": "Programm.ino.bootloader.bin", "offset": 4096 },
        { "path": "Programm.ino.partitions.bin", "offset": 32768 },
        { "path": "boot_app0.bin", "offset": 57344 },
        { "path": "Programm.ino.bin", "offset": 65536 }
      ]
    }
  ]
}

Flashing with ESP32 download tool (V3.8.8) is also OK:

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0030,len:1184
load:0x40078000,len:12804
ho 0 tail 12 room 4
load:0x40080400,len:3032
entry 0x400805e4
Booting...

Another important observation:

1. If I flash Arduino-ESP32 V1.0.6 based software using esptool or esp32 downloader, then I can update via OTA to Arduino-ESP32 V2.0.2 based software and it works.
2. If I flash Arduino-ESP32 V1.0.6 based software using web tools, then I can update via OTA to Arduino-ESP32 V2.0.2 based software BUT it doesnt work (the same error as above).

I can successfully flash my ESP32 but not my two NodeMCU (v2 and v3), which use the ESP8266EX Chip.

error: "failed_initialize", details: Error: Unknown Chip

esptool flash_id returns:

Connecting....
Detecting chip type... ESP8266
Chip is ESP8266EX
Features: WiFi
Crystal is 26MHz
MAC: c4:5b:be:60:dc:da
Uploading stub...
Running stub...
Stub running...
Manufacturer: d8
Device: 4016
Detected flash size: 4MB

I was trying to implement this (as I have done before) when kept getting these errors after the esp32 reboot.

23:28:30.382 -> ets Jun  8 2016 00:22:57
23:28:30.382 -> 
23:28:30.382 -> rst:0x1 (POWERON_RESET),boot:0x12 (SPI_FAST_FLASH_BOOT)
23:28:30.382 -> configsip: 0, SPIWP:0xee
23:28:30.382 -> clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
23:28:30.382 -> mode:QIO, clock div:2
23:28:30.382 -> load:0x3fff0030,len:1344
23:28:30.382 -> load:0xf01d020c,len:-2118480896
23:28:30.430 -> 1162 mmu set 00010000, pos 00010000
23:28:30.430 -> 1162 mmu set 00020000, pos 00020000
23:28:30.430 -> 1162 mmu set 00030000, pos 00030000
23:28:30.477 -> 1162 mmu set 00040000, pos 00040000
23:28:30.477 -> 1162 mmu set 00050000, pos 00050000
23:28:30.524 -> 1162 mmu set 00060000, pos 00060000
23:28:30.524 -> 1162 mmu set 00070000, pos 00070000
23:28:30.570 -> 1162 mmu set 00080000, pos 00080000
23:28:30.570 -> 1162 mmu set 00090000, pos 00090000
23:28:30.570 -> 1162 mmu set 000a0000, pos 000a0000
23:28:30.616 -> 1162 mmu set 000b0000, pos 000b0000
23:28:30.616 -> 1162 mmu set 000c0000, pos 000c0000
23:28:30.662 -> 1162 mmu set 000d0000, pos 000d0000
23:28:30.662 -> 1162 mmu set 000e0000, pos 000e0000
23:28:30.709 -> 1162 mmu set 000f0000, pos 000f0000
23:28:30.709 -> 1162 mmu set 00100000, pos 00100000
23:28:30.709 -> ets Jun  8 2016 00:22:57
23:28:30.709 -> 
23:28:30.709 -> rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
23:28:30.755 -> configsip: 0, SPIWP:0xee
23:28:30.755 -> clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
23:28:30.755 -> mode:QIO, clock div:2
23:28:30.755 -> load:0x3fff0030,len:1344
23:28:30.755 -> load:0xf01d020c,len:-2118480896
23:28:30.755 -> 1162 mmu set 00010000, pos 00010000
23:28:30.802 -> 1162 mmu set 00020000, pos 00020000

Arduino 1.19 with arduino-esp32 2.04 and 2.03 will both program the esp32 fine, but when I grab the 4 files for esp32-web-tools, then the 2.04 files do not work -- they produce the stuff above.

Arduino produces the same programming strings here:

204 C:\ArduinoPortable\arduino-1.8.19\portable\packages\esp32\tools\esptool_py\3.3.0/esptool.exe --chip esp32 --port COM7 --baud 460800 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 80m --flash_size 4MB 0x1000 C:\Users\James\AppData\Local\Temp\arduino_build_686088/ESP32-CAM-Video-Recorder-junior-60x.4.7.srt.4.ino.bootloader.bin 0x8000 C:\Users\James\AppData\Local\Temp\arduino_build_686088/ESP32-CAM-Video-Recorder-junior-60x.4.7.srt.4.ino.partitions.bin 0xe000 C:\ArduinoPortable\arduino-1.8.19\portable\packages\esp32\hardware\esp32\2.0.4/tools/partitions/boot_app0.bin 0x10000 C:\Users\James\AppData\Local\Temp\arduino_build_686088/ESP32-CAM-Video-Recorder-junior-60x.4.7.srt.4.ino.bin 
203 C:\ArduinoPortable\arduino-1.8.19\portable\packages\esp32\tools\esptool_py\3.3.0/esptool.exe --chip esp32 --port COM7 --baud 460800 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 80m --flash_size 4MB 0x1000 C:\Users\James\AppData\Local\Temp\arduino_build_830660/ESP32-CAM-Video-Recorder-junior-60x.4.7.srt.4.ino.bootloader.bin 0x8000 C:\Users\James\AppData\Local\Temp\arduino_build_830660/ESP32-CAM-Video-Recorder-junior-60x.4.7.srt.4.ino.partitions.bin 0xe000 C:\ArduinoPortable\arduino-1.8.19\portable\packages\esp32\hardware\esp32\2.0.3/tools/partitions/boot_app0.bin 0x10000 C:\Users\James\AppData\Local\Temp\arduino_build_830660/ESP32-CAM-Video-Recorder-junior-60x.4.7.srt.4.ino.bin 

The files are slightly different sizes for bootloader.bin and ino.bin, and the other two are the same files.

And I'm using the identical manifest.json

{
  "name": "ESP32-CAM-Video-Recorder-junior-60x.4.7.srt.4.ino",
  "new_install_prompt_erase": false,
  "builds": [
    {
      "chipFamily": "ESP32",
      "parts": [
        { "path": "ESP32-CAM-Video-Recorder-junior-60x.4.7.srt.4.ino.bootloader.bin", "offset": 4096 },
        { "path": "ESP32-CAM-Video-Recorder-junior-60x.4.7.srt.4.ino.partitions.bin", "offset": 32768 },
        { "path": "boot_app0.bin", "offset": 57344 },
        { "path": "ESP32-CAM-Video-Recorder-junior-60x.4.7.srt.4.ino.bin", "offset": 65536 }
      ]
    }
  ]
}

2.0.3 files work and 2.0.4 files do not.

This looks like a similar issue to this one (here):

https://github.com/esphome/esp-web-tools/issues/278

... and this one over at arduino-esp32

https://github.com/espressif/arduino-esp32/issues/7212

There seems to be discussion about qio, dio, dout programming and the 40/80 flash frequency. The two arduino esptool strings are both dio above, even though Arduino is set to qio. Arduino can program it fine at 40 or 80.

Here is the 2.03 40 listing if there is something in there that a wise-man can see:

C:\ArduinoPortable\arduino-1.8.19\portable\packages\esp32\tools\esptool_py\3.3.0/esptool.exe --chip esp32 --port COM7 --baud 460800 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 40m --flash_size 4MB 0x1000 C:\Users\James\AppData\Local\Temp\arduino_build_830660/ESP32-CAM-Video-Recorder-junior-60x.4.7.srt.4.ino.bootloader.bin 0x8000 C:\Users\James\AppData\Local\Temp\arduino_build_830660/ESP32-CAM-Video-Recorder-junior-60x.4.7.srt.4.ino.partitions.bin 0xe000 C:\ArduinoPortable\arduino-1.8.19\portable\packages\esp32\hardware\esp32\2.0.3/tools/partitions/boot_app0.bin 0x10000 C:\Users\James\AppData\Local\Temp\arduino_build_830660/ESP32-CAM-Video-Recorder-junior-60x.4.7.srt.4.ino.bin 
esptool.py v3.3
Serial port COM7
Connecting.....
Chip is ESP32-D0WDQ6 (revision 1)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
Crystal is 40MHz
MAC: 0c:b8:15:f4:14:50
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 460800
Changed.
Configuring flash size...
Flash will be erased from 0x00001000 to 0x00005fff...
Flash will be erased from 0x00008000 to 0x00008fff...
Flash will be erased from 0x0000e000 to 0x0000ffff...
Flash will be erased from 0x00010000 to 0x00109fff...
Compressed 18528 bytes to 12759...
Writing at 0x00001000... (100 %)
Wrote 18528 bytes (12759 compressed) at 0x00001000 in 0.6 seconds (effective 231.4 kbit/s)...
Hash of data verified.
Compressed 3072 bytes to 119...
Writing at 0x00008000... (100 %)
Wrote 3072 bytes (119 compressed) at 0x00008000 in 0.1 seconds (effective 393.2 kbit/s)...
Hash of data verified.
Compressed 8192 bytes to 47...
Writing at 0x0000e000... (100 %)
Wrote 8192 bytes (47 compressed) at 0x0000e000 in 0.2 seconds (effective 419.4 kbit/s)...
Hash of data verified.
Compressed 1021040 bytes to 639310...
Writing at 0x00010000... (2 %)
Writing at 0x0001c720... (5 %)
Writing at 0x00028395... (7 %)
Writing at 0x00031ec1... (10 %)
Writing at 0x0004107c... (12 %)
Writing at 0x00046e7d... (15 %)
Writing at 0x0004d569... (17 %)
Writing at 0x00053360... (20 %)
Writing at 0x00058da9... (22 %)
Writing at 0x0005e522... (25 %)
Writing at 0x00063caf... (27 %)
Writing at 0x000690e3... (30 %)
Writing at 0x0006e444... (32 %)
Writing at 0x00073733... (35 %)
Writing at 0x00078abf... (37 %)
Writing at 0x0007dda1... (40 %)
Writing at 0x00083353... (42 %)
Writing at 0x000885a7... (45 %)
Writing at 0x0008d4f0... (47 %)
Writing at 0x00092cfe... (50 %)
Writing at 0x00099976... (52 %)
Writing at 0x0009ec10... (55 %)
Writing at 0x000a45dc... (57 %)
Writing at 0x000a9aea... (60 %)
Writing at 0x000aecaf... (62 %)
Writing at 0x000b4054... (65 %)
Writing at 0x000b94e1... (67 %)
Writing at 0x000be9d2... (70 %)
Writing at 0x000c42ec... (72 %)
Writing at 0x000ca090... (75 %)
Writing at 0x000cfa4c... (77 %)
Writing at 0x000d5400... (80 %)
Writing at 0x000ddea4... (82 %)
Writing at 0x000e7502... (85 %)
Writing at 0x000ec67c... (87 %)
Writing at 0x000f3249... (90 %)
Writing at 0x000f8930... (92 %)
Writing at 0x000fe05e... (95 %)
Writing at 0x00103465... (97 %)
Writing at 0x001092b4... (100 %)
Wrote 1021040 bytes (639310 compressed) at 0x00010000 in 15.2 seconds (effective 539.0 kbit/s)...
Hash of data verified.

Any ideas ... other than stick with 2.03?

Hi, I have seen that improv wifi is not really reliable.

It works most of the time but not always.

Sometimes, when I try to flash the ESP8266, it asks me for the WiFi SID and password, I enter the credentials, and then it hangs on this screen.

It happen on my implementation here: https://sblantipodi.github.io/glow_worm_luciferin/beta.html

and on the one on the official esp web tools page.

Is this a bug? I tried changing the router and the esp8266 but same problem.

Flashing the S3 via ESP-WEB-Flasher does work fine.

When using Esp-web-tools it does not work. I have no idea what is going wrong.

Repo with test setup: https://github.com/Jason2866/esp-web-tools and the flash page https://jason2866.github.io/esp-web-tools/

See the screenshot:

Also the "All existing data will be erased from your device" is a bit frightening. Let's hope it will only flash what I have set in the manifest.

For Tasmota we prefer NOT to erase flash on subsequent use of esp-web-tools. This way the code can be updated while keeping the settings.

Used to work in the 3.x release but now only works when configured at manifest level.

@Jason2866 wrote here:

A nice feature would be: WebTools scans firmware (magic first bytes https://github.com/esphome/esphome-flasher/blob/main/esphomeflasher/common.py#L143-L153) which Bootloader file and other files are needed and downloads it from espressif github repo. There is no need to provide this file from user side. Only partitions file is specific Example for https://github.com/esphome/esphome-flasher/tree/main/esphomeflasher

Hi, I am trying to write an image to my 16MB flash chip. I tried to use separate files like

{
  "name": "Latest Release",
  "new_install_prompt_erase": false,
  "builds": [
    {
      "chipFamily": "ESP32",
      "parts": [
        { "path": "bootloader.bin", "offset": 4096 },
        { "path": "partitions.bin", "offset": 32768 },
        { "path": "ota_data_initial.bin", "offset": 57344 },
        { "path": "firmware.bin", "offset": 65536 },
        { "path": "spiffs.bin", "offset": 4259840 }
      ]
    }
  ]
}

or a single 16MB image file like

{
  "name": "Latest Release",
  "new_install_prompt_erase": false,
  "builds": [
    {
      "chipFamily": "ESP32",
      "parts": [
        { "path": "big-file.bin", "offset": 0 },
      ]
    }
  ]
}

but every try ends in error "Timed out waiting for packet header".

I tracked it down to the last of the separated files. The spiffs.bin is about 12 MB in size and may be this is related to a time out that is not long enough.

When i click the install button in the WLED application, i get the following response:

Failed to initialize. Try resetting your device or holding the BOOT button while selecting your serial port.

I have a ESP8266 chip and i also tried ESP home and Tasmota, both gave the same issue.

I'm trying to setup ESP Web Tools to write a two part (firmware and file system) to an ESP8266. You can try it here (choose ratgdo v2.0a (alpha) - rolling code support, which uses this manifest.json)

The first ratgdo_esp8266_v2.0a.bin file at offset 0 seems to write okay, but then it times out right after it starts to write the second ratgdo_esp8266_v2.0a_littlefs.bin (~3mb) file at offset 1048576.

I got the offset number from platform.io when uploading the filesystem:

Wrote 3121152 bytes (679027 compressed) at 0x00100000 in 74.9 seconds (effective 333.3 kbit/s)...

0x00100000 in decimal is 1048576

Here is the output of the browser console during the flashing process:

Writing at 0x48000... (100%)
flash_defl_block 8d 097
check_command write compressed data to flash after seq 18
Read with timeout 5000
Wrote 419216 bytes (296238 compressed) at 0x0 in 28.729 seconds.
Data Length 3121152
Image Length 3121152
_update_image_flash_params keep keep keep
Compressed image 
Uint8Array(679025)
Compressed 3121152 bytes to 679025...
check_command enter compressed flash mode
Read with timeout 3000
Write loop 1048576 0 42
Writing at 0x100000... (2%)
flash_defl_block 78 0da
check_command write compressed data to flash after seq 0
Read with timeout 5000
Write loop 1048576 1 42
Writing at 0x104000... (4%)
flash_defl_block d5 08b
check_command write compressed data to flash after seq 1
Read with timeout 5000

Does anyone know what I'm doing wrong?

UPDATE:

From the log you can see that the step Writing at 0x104000... (4%) is where it times out. In platformio, this step takes longer than the rest, but it does continue. How can I increase the Read with timeout value?

Hi all.

I noticed that ESP Web Tools stopped working correctly after Windows update. Last Windows 11 update updated the CP210x drivers from the ancient v6.7.6 (2018 drivers) to the latest v11.2.0 (2022 drivers).

After the update ESP Web Tools has some troubles throwing random errors like this:

Everything else apart ESP Web Tools works correctly, I can flash all my firmwares with all the other possible tools without problems.

Is this something that can be fixed by ESP Web Tools?

Thanks

Hi, following the Discussion there: https://github.com/espressif/idf-eclipse-plugin/issues/620 it would be nice to be able to select another Font for the Console.

currently QRCODE does not work.

Thanks, Franz

After many months I tried again web ESP Tools, felt very hipped because I saw Typescript but ended up frustrated.

Running the following command to install the library on my existing React project: yarn add esp-web-tools

Outputs the following error message:

error Can't add "esptool-js": invalid package version undefined. info Visit https://yarnpkg.com/en/docs/cli/add for documentation about this command.

Does anyone has any hints? Something that I'm missing. Much appreciated.

If I flash via command line I use upload_speed of 921600. It would be great if this is also possible via esp-web-tools, might be as a additional parameter (per build) in the manifest?

Hi, I used ESP-Prog to connect the esp32 to the PC and when I click on the connect button the first time, this error pops out.

Then when I click again in the port dropdown menu it says (Paired), there is no error and I can use the tool. Any idea why? I guess the error pops out because of the pairing. It can be annoying because I need to click on connect button two times.