feat(espsecure): Unify all commands and options to use dash instead of underscore

2025-10-17 16:01:18 +08:00 · 2025-03-17 16:16:37 +01:00
parent 69950b2722
commit 36325fdf69
5 changed files with 121 additions and 63 deletions
--- a/docs/en/espsecure/index.rst
+++ b/docs/en/espsecure/index.rst
@@ -21,13 +21,13 @@ You must install ``esptool.py`` package with the ``hsm`` extra using the command

 The following command should be used to get an image signed using an external HSM. ::

-    python espsecure.py sign_data --version 2 --hsm --hsm-config <hsm_config_file> --output <signed_image> <datafile>
+    espsecure.py sign-data --version 2 --hsm --hsm-config <hsm_config_file> --output <signed_image> <datafile>

 The above command first extracts the public key from the HSM, generates a signature for an image using the HSM, and then creates a signature block and appends it to the image to generate a signed image.

 If the public key is not stored in the external HSM, you can specify the ``--pub-key`` argument to supply the public key. ::

-    python espsecure.py sign_data --version 2 --hsm --hsm-config <hsm_config_file> --pub-key <public_key> --output <signed_image> <datafile>
+    espsecure.py sign-data --version 2 --hsm --hsm-config <hsm_config_file> --pub-key <public_key> --output <signed_image> <datafile>

 .. note::
    In case you are using ESP-IDF, then an unsigned application can be generated by disabling ``CONFIG_SECURE_BOOT_BUILD_SIGNED_BINARIES`` configuration option in the project settings.
@@ -37,11 +37,11 @@ Verifying the Signed Image

 Once the signed image is generated, we can verify it using the following command: ::

-    python espsecure.py verify_signature --version 2 --hsm --hsm-config <hsm_config_file> <signed_image>
+    espsecure.py verify-signature --version 2 --hsm --hsm-config <hsm_config_file> <signed_image>

 If the public key is not stored in the external HSM, you can specify the ``--keyfile`` argument to supply the public key. ::

-    python espsecure.py verify_signature --version 2 --keyfile <public_key> <signed_image>
+    espsecure.py verify-signature --version 2 --keyfile <public_key> <signed_image>


 HSM Config File
--- a/docs/en/migration-guide.rst
+++ b/docs/en/migration-guide.rst
@@ -1,13 +1,15 @@
 .. _migration:

+``v5`` Migration Guide
+======================
+
+This document describes the breaking changes made to esptool.py, espsecure.py and espefuse.py in the major release ``v5``. It provides guidance on adapting existing workflows and scripts to ensure compatibility when updating from ``v4.*``.
+
 esptool.py ``v5`` Migration Guide
-=================================
-
-This document describes the breaking changes made to esptool.py in the major release ``v5``. It provides guidance on adapting existing workflows and scripts to ensure compatibility when updating from ``v4.*``.
-
+*********************************

 ``image-info`` Output Format Change
-***********************************
+###################################

 The output format of the :ref:`image-info <image-info>` command has been **updated in v5**. The original format (``--version 1``) is **deprecated** and replaced by the updated format (``--version 2``). The ``--version`` argument has been **removed entirely**, and the new format is now the default and only option.

@@ -23,7 +25,7 @@ The output format of the :ref:`image-info <image-info>` command has been **updat
 2. Remove any ``--version`` arguments from ``image-info`` commands.

 Output Logging
-**************
+##############

 The esptool ``v5`` release introduces a centralized logging mechanism to improve output management and allow redirection.

@@ -42,7 +44,7 @@ The esptool ``v5`` release introduces a centralized logging mechanism to improve
 See the :ref:`logging <logging>` section for more details on available logger methods and custom logger implementation.

 ``write-flash`` ``--verify`` Argument
-*************************************
+#####################################

 The ``--verify`` option for the :ref:`write-flash <write-flash>` command has been **deprecated in v5**. Flash verification is performed automatically after every successful write operation when technically feasible.

@@ -58,7 +60,7 @@ The ``--verify`` option for the :ref:`write-flash <write-flash>` command has bee
 2. Update scripts/CI pipelines to remove ``--verify`` flags.

 Error Output Handling
-*********************
+#####################

 In ``v5``, error handling and output behavior have been improved to provide better user experience and script compatibility.

@@ -75,7 +77,7 @@ In ``v5``, error handling and output behavior have been improved to provide bett
 2. Ensure scripts handle non-zero exit codes correctly in the case of operations interrupted by the user.

 Beta Target Support Removal
-***************************
+###########################

 Support for the following beta targets has been **removed in v5**:

@@ -93,7 +95,7 @@ Support for the following beta targets has been **removed in v5**:
 Use esptool ``v4`` for legacy workflows targeting these beta chips.

 ``verify-flash`` ``--diff`` Argument
-*************************************
+####################################

 The format of the ``--diff`` option of the :ref:`verify-flash <verify-flash>` command has **changed in v5**. Previously, ``--diff=yes/no`` had to be specified to enable or disable the diff output. In the new version, the ``--diff`` option is a simple boolean switch without the need of a ``yes`` or ``no`` value.

@@ -102,7 +104,7 @@ The format of the ``--diff`` option of the :ref:`verify-flash <verify-flash>` co
 1. Rewrite the ``--diff=yes`` argument to a simple ``--diff`` in any existing ``verify-flash`` commands in scripts/CI pipelines. Delete ``--diff=no`` completely if detailed diff output is not required.

 Using esptool as a Python Module
-********************************
+################################

 All command functions (e.g., ``verify-flash``, ``write-flash``) have been refactored to remove their dependency on the ``args`` object from the argparse module. Instead, all arguments are now passed explicitly as individual parameters. This change, combined with enhancements to the public API, provides a cleaner, more modular interface for programmatic use of esptool in custom scripts and applications (see :ref:`scripting <scripting>`).

@@ -121,7 +123,7 @@ For detailed examples and API reference, see the :ref:`scripting <scripting>` se


 Flash Operations from Non-flash Related Commands
-************************************************
+################################################

 When esptool is used as a CLI tool, the following commands no longer automatically attach the flash by default, since flash access is not required for their core functionality:

@@ -145,7 +147,7 @@ The ``--spi-connection`` CLI argument has been **removed** from non-flash relate


 Shell Completion
-****************
+################

 The esptool ``v5`` has switched to using `Click <https://click.palletsprojects.com/>`_ for command line argument parsing, which changes how shell completion works.

@@ -155,7 +157,7 @@ The esptool ``v5`` has switched to using `Click <https://click.palletsprojects.c
 2. Follow the new shell completion setup instructions in the :ref:`shell-completion` section of the :ref:`installation <installation>` guide.

 ``merge-bin`` ``--fill-flash-size`` Argument
-********************************************
+############################################

 The ``--fill-flash-size`` option of the :ref:`merge-bin <merge-bin>` command has been renamed to ``--pad-to-size``. This change provides a more intuitive and descriptive name for the argument and is consistent with the naming scheme in other esptool image manipulation commands.

@@ -164,7 +166,7 @@ The ``--fill-flash-size`` option of the :ref:`merge-bin <merge-bin>` command has
 1. Rename the ``--fill-flash-size`` to ``--pad-to-size`` in any existing ``merge-bin`` commands in scripts/CI pipelines.

 ``write-flash`` ``--ignore-flash-encryption-efuse-setting`` Argument
-********************************************************************
+####################################################################

 The ``--ignore-flash-encryption-efuse-setting`` option of the :ref:`write-flash <write-flash>` command has been renamed to ``--ignore-flash-enc-efuse``. This change shortens the argument name to improve readability and consistency with other esptool options.

@@ -173,7 +175,7 @@ The ``--ignore-flash-encryption-efuse-setting`` option of the :ref:`write-flash
 1. Rename the ``--ignore-flash-encryption-efuse-setting`` to ``--ignore-flash-enc-efuse`` in any existing ``write-flash`` commands in scripts/CI pipelines.

 ``make_image`` Command Removal
-******************************
+##############################

 The ``make_image`` command for the ESP8266 has been **removed in v5**. This command has been deprecated in favor of using **objcopy** (or other tools) to generate ELF images and then using ``elf2image`` to create the final ``.bin`` file.

@@ -182,7 +184,7 @@ The ``make_image`` command for the ESP8266 has been **removed in v5**. This comm
 1. Replace any ``make_image`` workflows with the recommended way of assembling firmware images using **objcopy** and ``elf2image``.

 Using Binary from GitHub Releases on Linux
-******************************************
+##########################################

 The ``esptool.py`` binary from GitHub Releases on Linux is now using Ubuntu 22.04 as the base image. That means the image is using ``glibc`` 2.35, which is not fully compatible with the ``glibc`` 2.28 from Ubuntu 20.04 (the base image for ``v4.*``).

@@ -191,7 +193,7 @@ The ``esptool.py`` binary from GitHub Releases on Linux is now using Ubuntu 22.0
 1. Update your operating system to a newer version which bundles ``glibc`` 2.35 or later

 Command and Option Renaming
-***************************
+###########################

 All the commands and options have been renamed to use ``-`` instead of ``_`` as a separator (e.g., ``write_flash`` -> ``write-flash``).

@@ -204,10 +206,58 @@ This change affects most of the commands and the following options: ``--flash_si
 1. Replace all underscores in command and option names with ``-`` in your scripts and CI pipelines.

 Log Format Changes
-******************
+##################

 A significant amount of changes have been made to the log styling and formatting in ``v5``. Some of the messages, warnings, and errors are now formatted differently or reworded to provide more context and improve readability. Exhaustive list of changed messages won't be provided.

 **Migration Steps:**

 1. Make sure to adjust any of your scripts, asserts, CI workflows, or others to accommodate the new/changed format of messages. If you are parsing the log output (not recommended), consider importing esptool as a module and using the public API (see :ref:`here <scripting>`) to get the information you need.
+
+
+espsecure.py ``v5`` Migration Guide
+***********************************
+
+Command and Option Renaming
+###########################
+
+All the commands and options have been renamed to use ``-`` instead of ``_`` as a separator (e.g., ``sign_data`` -> ``sign-data``).
+
+Old command and option names are **deprecated**, meaning they will work for now with a warning, but will be removed in the next major release.
+
+This change affects most of the commands and the following options: ``--aes_xts``, ``--flash_crypt_conf``, ``--append_signatures``.
+
+**Migration Steps:**
+
+1. Replace all underscores in command and option names with ``-`` in your scripts and CI pipelines.
+
+Public API Changes
+##################
+
+The public API of ``espsecure.py`` has been updated to provide a more consistent and user-friendly interface for programmatic use in custom scripts and applications.
+
+**Key Changes:**
+
+- All functions now accept individual parameters instead of relying on the ``args`` object from the argparse module. Affected functions are:
+    - ``digest_secure_bootloader``
+    - ``generate_signing_key``
+    - ``digest_secure_bootloader``
+    - ``generate_signing_key``
+    - ``sign_data`` including ``sign_secure_boot_v1`` and ``sign_secure_boot_v2``
+    - ``verify_signature`` including ``verify_signature_v1`` and ``verify_signature_v2``
+    - ``extract_public_key``
+    - ``signature_info_v2``
+    - ``digest_sbv2_public_key`` and ``digest_rsa_public_key``
+    - ``digest_private_key``
+    - ``generate_flash_encryption_key``
+    - ``decrypt_flash_data``
+    - ``encrypt_flash_data``
+- The ``main`` function parameter ``custom_commandline`` has been renamed to ``argv`` to unify the naming convention with esptool.
+
+**Migration Steps:**
+
+1. Update function calls to pass individual parameters instead of the ``args`` object. For example:
+   ``sign_data(args)`` -> ``sign_data(data=args.data, key=args.key, ...)``
+   or if you were mocking the args object, now you don't have to do that and you can pass parameters directly to the function like:
+   ``sign_data(data=data, key=key, ...)``.
+2. Replace the ``custom_commandline`` parameter with ``argv`` in the ``main`` function call.
--- a/espsecure/init.py
+++ b/espsecure/init.py
@@ -1527,7 +1527,16 @@ def _check_output_is_not_input(
        )


+class Group(esptool.cli_util.Group):
+    DEPRECATED_OPTIONS = {
+        "--aes_xts": "--aes-xts",
+        "--flash_crypt_conf": "--flash-crypt-conf",
+        "--append_signatures": "--append-signatures",
+    }
+
+
@click.group(
+    cls=Group,
    no_args_is_help=True,
    context_settings=dict(help_option_names=["-h", "--help"], max_content_width=120),
    help=f"espsecure.py v{esptool.__version__} - ESP32 Secure Boot & Flash Encryption "
@@ -1537,7 +1546,7 @@ def cli():
    print(f"espsecure.py v{esptool.__version__}")


-@cli.command("digest_secure_bootloader")
+@cli.command("digest-secure-bootloader")
@click.option(
    "--keyfile",
    "-k",
@@ -1561,7 +1570,7 @@ def digest_secure_bootloader_cli(keyfile, output, iv, image):
    digest_secure_bootloader(keyfile, output, iv, image)


-@cli.command("generate_signing_key")
+@cli.command("generate-signing-key")
@click.option(
    "--version",
    "-v",
@@ -1583,7 +1592,7 @@ def generate_signing_key_cli(version, scheme, keyfile):
    generate_signing_key(version, scheme, keyfile)


-@cli.command("sign_data")
+@cli.command("sign-data")
@click.option(
    "--version",
    "-v",
@@ -1605,7 +1614,7 @@ def generate_signing_key_cli(version, scheme, keyfile):
    help="Output file for signed digest image. Default is to sign the input file.",
 )
@click.option(
-    "--append_signatures",
+    "--append-signatures",
    "-a",
    is_flag=True,
    help="Append signature block(s) to already signed image. Not valid for ESP32 and "
@@ -1665,7 +1674,7 @@ def sign_data_cli(
    )


-@cli.command("verify_signature")
+@cli.command("verify-signature")
@click.option(
    "--version",
    "-v",
@@ -1698,7 +1707,7 @@ def verify_signature_cli(version, hsm, hsm_config, keyfile, datafile):
    verify_signature(version, hsm, hsm_config, keyfile, datafile)


-@cli.command("extract_public_key")
+@cli.command("extract-public-key")
@click.option(
    "--version",
    "-v",
@@ -1713,14 +1722,14 @@ def verify_signature_cli(version, hsm, hsm_config, keyfile, datafile):
    required=True,
    help="Private key file (PEM format) to extract the public verification key from.",
 )
-@click.argument("public_keyfile", type=click.File("wb", lazy=True))
+@click.argument("public-keyfile", type=click.File("wb", lazy=True))
 def extract_public_key_cli(version, keyfile, public_keyfile):
    """Extract the public verification key for signatures, save it as a raw binary
    file"""
    extract_public_key(version, keyfile, public_keyfile)


-@cli.command("digest_rsa_public_key", deprecated=True)
+@cli.command("digest-rsa-public-key", deprecated=True)
@click.option(
    "--keyfile",
    "-k",
@@ -1740,7 +1749,7 @@ def digest_rsa_public_key(keyfile, output):
    digest_sbv2_public_key(keyfile, output)


-@cli.command("digest_sbv2_public_key")
+@cli.command("digest-sbv2-public-key")
@click.option(
    "--keyfile",
    "-k",
@@ -1760,14 +1769,14 @@ def digest_sbv2_public_key_cli(keyfile, output):
    digest_sbv2_public_key(keyfile, output)


-@cli.command("signature_info_v2")
+@cli.command("signature-info-v2")
@click.argument("datafile", type=click.File("rb"))
 def signature_info_v2_cli(datafile):
    """Reads the signature block and provides the signature block information."""
    signature_info_v2(datafile)


-@cli.command("digest_private_key")
+@cli.command("digest-private-key")
@click.option(
    "--keyfile",
    "-k",
@@ -1783,13 +1792,13 @@ def signature_info_v2_cli(datafile):
    help="Length of private key digest file to generate (in bits). 3/4 Coding Scheme "
    "requires 192 bit key.",
 )
-@click.argument("digest_file", type=click.File("wb", lazy=True))
+@click.argument("digest-file", type=click.File("wb", lazy=True))
 def digest_private_key_cli(keyfile, keylen, digest_file):
    """Generate an SHA-256 digest of the private signing key."""
    digest_private_key(keyfile, keylen, digest_file)


-@cli.command("generate_flash_encryption_key")
+@cli.command("generate-flash-encryption-key")
@click.option(
    "--keylen",
    "-l",
@@ -1798,14 +1807,14 @@ def digest_private_key_cli(keyfile, keylen, digest_file):
    help="Length of private key digest file to generate (in bits). 3/4 Coding Scheme "
    "requires 192 bit key.",
 )
-@click.argument("key_file", type=click.File("wb", lazy=True))
+@click.argument("key-file", type=click.File("wb", lazy=True))
 def generate_flash_encryption_key(keylen: int, key_file: IO):
    """Generate a development-use flash encryption key with random data."""
    print(f"Writing {keylen} random bits to key file {key_file.name}")
    key_file.write(os.urandom(keylen // 8))


-@cli.command("decrypt_flash_data")
+@cli.command("decrypt-flash-data")
@click.option(
    "--keyfile",
    "-k",
@@ -1828,19 +1837,19 @@ def generate_flash_encryption_key(keylen: int, key_file: IO):
    help="Address offset in flash that file was read from.",
 )
@click.option(
-    "--flash_crypt_conf",
+    "--flash-crypt-conf",
    type=esptool.cli_util.AnyIntType(),
    default=0xF,
    help="Override FLASH_CRYPT_CONF efuse value (default is 0XF) (applicable only for "
    "ESP32).",
 )
@click.option(
-    "--aes_xts",
+    "--aes-xts",
    "-x",
    is_flag=True,
    help="Decrypt data using AES-XTS (not applicable for ESP32)",
 )
-@click.argument("encrypted_file", type=click.File("rb"))
+@click.argument("encrypted-file", type=click.File("rb"))
 def decrypt_flash_data_cli(
    keyfile, output, address, flash_crypt_conf, aes_xts, encrypted_file
 ):
@@ -1850,7 +1859,7 @@ def decrypt_flash_data_cli(
    )


-@cli.command("encrypt_flash_data")
+@cli.command("encrypt-flash-data")
@click.option(
    "--keyfile",
    "-k",
@@ -1873,19 +1882,19 @@ def decrypt_flash_data_cli(
    required=True,
 )
@click.option(
-    "--flash_crypt_conf",
+    "--flash-crypt-conf",
    type=esptool.cli_util.AnyIntType(),
    default=0xF,
    help="Override FLASH_CRYPT_CONF efuse value (default is 0XF) (applicable only for "
    "ESP32)",
 )
@click.option(
-    "--aes_xts",
+    "--aes-xts",
    "-x",
    is_flag=True,
    help="Encrypt data using AES-XTS (not applicable for ESP32)",
 )
-@click.argument("plaintext_file", type=click.File("rb"))
+@click.argument("plaintext-file", type=click.File("rb"))
 def encrypt_flash_data_cli(
    keyfile, output, address, flash_crypt_conf, aes_xts, plaintext_file
 ):
--- a/esptool/cli_util.py
+++ b/esptool/cli_util.py
@@ -179,17 +179,16 @@ class AddrFilenamePairType(click.Path):
 ########################### Custom option/argument ############################


-DEPRECATED_OPTIONS = {
-    "--flash_size": "--flash-size",
-    "--flash_freq": "--flash-freq",
-    "--flash_mode": "--flash-mode",
-    "--use_segments": "--use-segments",
-    "--ignore_flash_encryption_efuse_setting": "--ignore-flash-enc-efuse",
-    "--fill-flash-size": "--pad-to-size",
-}
-
-
 class Group(click.RichGroup):
+    DEPRECATED_OPTIONS = {
+        "--flash_size": "--flash-size",
+        "--flash_freq": "--flash-freq",
+        "--flash_mode": "--flash-mode",
+        "--use_segments": "--use-segments",
+        "--ignore_flash_encryption_efuse_setting": "--ignore-flash-enc-efuse",
+        "--fill-flash-size": "--pad-to-size",
+    }
+
    def __call__(self, esp: ESPLoader | None = None, *args, **kwargs):
        self._esp = esp  # store the external esp object in the group
        return super().__call__(*args, **kwargs)
@@ -197,9 +196,9 @@ class Group(click.RichGroup):
    def _replace_deprecated_args(self, args: list[str]) -> list[str]:
        new_args = []
        for arg in args:
-            if arg in DEPRECATED_OPTIONS.keys():
+            if arg in self.DEPRECATED_OPTIONS.keys():
                # Replace underscores with hyphens in option names
-                new_name = DEPRECATED_OPTIONS[arg]
+                new_name = self.DEPRECATED_OPTIONS[arg]
                if new_name != arg:
                    log.warning(
                        f"Deprecated: Option '{arg}' is deprecated. "
--- a/test/test_espsecure.py
+++ b/test/test_espsecure.py
@@ -86,12 +86,12 @@ class TestESP32SecureBootloader(EspSecureTestCase):
            output_file.close()

            out = self.run_espsecure(
-                "digest_rsa_public_key --keyfile "
+                "digest-rsa-public-key --keyfile "
                "secure_images/rsa_secure_boot_signing_key.pem "
                f"-o {output_file.name}"
            )
            assert (
-                "DeprecationWarning: The command 'digest_rsa_public_key' is deprecated."
+                "DeprecationWarning: The command 'digest-rsa-public-key' is deprecated."
                in out
            )

@@ -108,14 +108,14 @@ class TestSigning(EspSecureTestCase):
            # keyfile cannot exist before generation -> tempfile.NamedTemporaryFile()
            # cannot be used for keyfile
            keyfile_name = os.path.join(keydir, "key.pem")
-            self.run_espsecure(f"generate_signing_key --version 1 {keyfile_name}")
+            self.run_espsecure(f"generate-signing-key --version 1 {keyfile_name}")

    def test_key_generation_v2(self):
        with tempfile.TemporaryDirectory() as keydir:
            # keyfile cannot exist before generation -> tempfile.NamedTemporaryFile()
            # cannot be used for keyfile
            keyfile_name = os.path.join(keydir, "key.pem")
-            self.run_espsecure(f"generate_signing_key --version 2 {keyfile_name}")
+            self.run_espsecure(f"generate-signing-key --version 2 {keyfile_name}")

    def _test_sign_v1_data(self, key_name):
        try:
@@ -755,7 +755,7 @@ class TestDigest(EspSecureTestCase):
            outfile_name = f.name

        self.run_espsecure(
-            "digest_private_key "
+            "digest-private-key "
            "--keyfile secure_images/ecdsa256_secure_boot_signing_key.pem "
            f"{outfile_name}"
        )
@@ -769,6 +769,6 @@ class TestDigest(EspSecureTestCase):
        fname = "secure_images/ecdsa256_secure_boot_signing_key.pem"

        with pytest.raises(subprocess.CalledProcessError):
-            self.run_espsecure(f"digest_private_key --keyfile {fname} {fname}")
+            self.run_espsecure(f"digest-private-key --keyfile {fname} {fname}")
        output = capsys.readouterr().out
        assert "should not be the same!" in output