<!--- Title -->
Description
-----------
<!--- Describe your changes in detail. -->
This PR enables the coreMQTT library to resend unacked publishes on an
unclean session connection.
Following is a brief summary of changes:
1. Add a new API `MQTT_InitRetransmits` that will initialise the context
to handle publish retransmits on an unclean session connection
2. Add signatures of callback function pointers that users will define
in order to:
a. copy and store outgoing publishes
b. retrieve copied publish on an unclean session connection to resend
c. clear a copied publish when a `PUBACK`/`PUBREC` is received
d. clear all copied publishes on a clean session connection
3. Update the API's to check if callback's are defined and implement
resend publishes as required.
Following are the specifics of the changes:
1. Add 3 new MQTTStatus_t values: MQTTPublishStoreFailed,
MQTTPublishRetrieveFailed and MQTTPublishClearAllFailed
2. Update `MQTTContext_t` to hold the callback function pointers
a. `MQTTRetransmitStorePacket storeFunction`
b. `MQTTRetransmitRetrievePacket retrieveFunction`
c. `MQTTRetransmitClearPacket clearFunction`
d. `MQTTRetransmitClearAllPackets clearAllFunction`
3. Update the `MQTT_Status_strerror` function to handle the new
`MQTTStatus_t` values
4. Add a new API function `MQTT_InitRetransmits` that will initialise
the new callback functions in the `MQTTContext_t`
5. Add this API to the core_mqtt.h file to make it available to users
6. Modify `MQTT_Publish`
a. copy the outgoing publish packet in form of an array of
`TransportOutVector_t` if the callback if defined
b. if copy fails then bubble up corresponding error status code
7. Modify `MQTT_ReceiveLoop`
a. on receiving a `PUBACK`/`PUBREC` clear the copy of that particular
publish after the state of the publish record has been successfully
updated, if the callback if defined
8. Modify `MQTT_Connect`
a. on a clean session clear all the copies of publishes stored if the
callback is defined
b. if clear all fails then bubble up corresponding error status code
c. on an unclean session get the packetID of the unacked publishes and
retrieve the copies of those if the callback is defined
d. if retrieve fails then bubble up corresponding error status code
Approaches Taken
---------------
- To let user know about the changes we have made we will add them to a
changelog and have a minor version bump
- To be in line with the zero copy principle in our library we chose to
provide and retrieve the publish packets for storing and resending in
form of an array of `TransportOutVector_t`
- Code is written in a way that on receiving a `PUBACK`/`PUBREC` the
copy will be cleared after the state of the publish record is changed so
that if state update fails the copy won't be cleared. Otherwise if the
state does not change and the copy is cleared then when a connection is
made with an unclean session there will be a retrieve fail as the system
is in an inconsistent state.
- We are storing the copies of the publishes with the Duplicate flag set
this is because on retrieving the packet we will get it in the form of a
`TransportOutVector_t` that holds the data in a `const` pointer which
cannot be changed after retrieving.
Pending Tasks
---------------
- [ ] Changelog
- [ ] Minor version bump
- [x] Doxygen example for the new API
- [x] Better API Names
- [x] Unit Test Updates
- [x] CBMC Proof
---------
Co-authored-by: Dakshit Babbar <dakshba@amazon.com>
Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: AniruddhaKanhere <60444055+AniruddhaKanhere@users.noreply.github.com>
<!--- Title -->
Description
-----------
<!--- Describe your changes in detail. -->
Following is a brief summary of changes:
1. Check the connected flag before doing any send operation on the
connection
2. Make all the APIs that do send operations, thread safe
3. Update the connected flag within MQTT_Disconnect regardless of the
return status of the send operation
Following are the specifics of the changes:
1. Add 3 new MQTTStatus_t values: MQTTStatusConnected,
MQTTStatusNotConnected and MQTTStatusDisconnectPending
2. Added 1 new MQTTConnectionStatus_t value: MQTTDisconnectPending
3. Update the MQTT_Status_strerror function to handle the new
MQTTStatus_t values
4. Add a new API function MQTT_CheckConnectStatus() that will check the
value of the context→connectStatus flag safely.
5. Add this API to the core_mqtt.h file to make it available to users
6. Check the connected flag before doing any Send operation (following
API's are updated)
a. sendPublishAcks
b. MQTT_Connect
c. MQTT_Subscribe
d. MQTT_Publish
e. MQTT_Ping
f. MQTT_Unsubscribe
g. MQTT_Disconnect
7. Use the MQTT_PRE_STATE_UPDATE_HOOK() and
MQTT_POST_STATE_UPDATE_HOOK() to make the send APIs thread safe
8. The connect status is set to MQTTDisconnectPending whenever a
transport send or receive function returns a negative error code
9. `const` keyword for the the MQTTStatus_t is removed in the input
parameters for the receive functions as we need to update the connection
status when the receive function returns a negative error code
Relevant Explanations
---------------
- MQTT_PRE_SEND_HOOK(): The Pre and Post Send hook Macros are not
required now, as the sending logic will be within the pre and post state
update hook itself. (because we cannot allow other threads to change the
connection state of the application until a send operation is complete).
- I have split the handleSessionResumption function. The part of that
function which was handling the clean session has been added within the
mutex calls in the [MQTT_Connect
API](https://github.com/FreeRTOS/coreMQTT/pull/305/files#diff-2534a3c0229ae9af3801f2a5c6a24eeef2cd0a686671f0371a11d2718ba4fdd6R2828)
and the unclean session part is handled by this new function that is
[called outside the mutex
calls](https://github.com/FreeRTOS/coreMQTT/pull/305/files#diff-2534a3c0229ae9af3801f2a5c6a24eeef2cd0a686671f0371a11d2718ba4fdd6R2866).
Pending Tasks
---------------
- [ ] Doxygen example for the new API
- [x] Unit Test Updates
- [x] CBMC Proof
---------
Co-authored-by: Dakshit Babbar <dakshba@amazon.com>
Co-authored-by: GitHub Action <action@github.com>
<!--- Title -->
Description
-----------
* Update the release action for version number include the following
files
- docs/doxygen/config.doxyfile - PROJECT_NUMBER
- manifest.yml file - version
- source file - version header
- core_mqtt.h - version number
* Add version number check in "Create ZIP and verify package for release
asset" steps. Including the following
- docs/doxygen/config.doxyfile - PROJECT_NUMBER
- manifest.yml file - version
- source file - version header
- core_mqtt.h - version number
* Update all the version number to "v2.3.0+" and "\<DEVELOPMENT
BRANCH\>"
Test Steps
-----------
Using release action to create release should update the following
* source/include/core_mqtt.h version number
* source files header version number
* doxygen version number
* manifest.yml number
* SBOM file
Tested in personal fork without problem :
https://github.com/FreshDevGo/coreMQTT/actions/runs/9885707328/job/27304218049
Test with wrong source file version number :
https://github.com/FreshDevGo/coreMQTT/actions/runs/9885727002/job/27304274003
Test with wrong manifest.yml version number :
https://github.com/FreshDevGo/coreMQTT/actions/runs/9885726029/job/27304270303
Test with wrong doxygen version number :
https://github.com/FreshDevGo/coreMQTT/actions/runs/9885723302/job/27304269170
Test with wrong version number macro in core_mqtt.h :
https://github.com/FreshDevGo/coreMQTT/actions/runs/9885724835/job/27304268841
Checklist:
----------
<!--- Go over all the following points, and put an `x` in all the boxes
that apply. -->
<!--- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
- [x] I have tested my changes. No regression in existing tests.
- [ ] ~~I have modified and/or added unit-tests to cover the code
changes in this Pull Request.~~
Related Issue
-----------
<!-- If any, please provide issue ID. -->
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
<!--- Title -->
Description
-----------
This PR update changelog, version numbers and .md (doxygen, size table)
files for release
Test Steps
-----------
<!-- Describe the steps to reproduce. -->
Checklist:
----------
<!--- Go over all the following points, and put an `x` in all the boxes
that apply. -->
<!--- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
- [ ] I have tested my changes. No regression in existing tests.
- [ ] I have modified and/or added unit-tests to cover the code changes
in this Pull Request.
Related Issue
-----------
<!-- If any, please provide issue ID. -->
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
core_mqtt_serializer.h included the user coreMQTT config, which spills
the config header's contents into all consumers of coreMQTT's headers.
Macros from the config are no longer used in the the API, so this is
also no longer used for anything, so can be removed.
* Add SPDX identifier to source files
* Add license information to CBMC stub
* Fix spellcheck and the formatting
* Update version numbers and .md files for reelase
Co-authored-by: jasonpcarroll <23126711+jasonpcarroll@users.noreply.github.com>
* Update ci.yml (#197)
* Update ci.yml
* Add main branch in the CI YAML
* Add cancel callback API (#196)
* Add cancel callback command
* Update the function name
* Fix formatting
* Update memory estimates
* Add State to MQTT_ProcessLoop so that it can be called in a non-blocking manner (#198)
* Add stateful process-loop function
* Add extra checks; fix bugs and add description of functions
* Add index based stateful processloop
* Clean up
* Renamed functions to make them more coherent with their function
* Remove unused function declarations
* Fixed failing CI checks from previous commits except unit-test
* Fixed spell check and updated size-table
* Fix CBMC proofs
* Empty-Commit to trigger CBMC proofs
* Fix loop unwinding values in the Makefile
* Add upper bound on the buffer size of MQTT
* Increase minimum limit on buffer size to >0
* Add upper bound on the size of the buffer as well
* CBMC: Add memmove stub to accelerate coverage
The commit adds a stub for memmove accelerate CBMC coverage
calculation. Without this stub, coverage for `MQTT_ProcessLoop` and
`MQTT_ReceiveLoop` fails to converge (gets stuck generating the SAT
formula for the memmove in `receiveSingleIteration`). This stub
checks that src and dst are nonnull pointers and havocs dst.
* Fix formatting
Co-authored-by: Aniruddha Kanhere <ubuntu@ip-172-31-25-12.us-west-2.compute.internal>
Co-authored-by: Mark R. Tuttle <mrtuttle@amazon.com>
* Remove the use of common buffer (#199)
* Make publish use internal buffers
* Fix comment about packet ID
* Try a different approach for ping and disconnect
* Use writev and flush in conjuction with send
* Update the publish method to use vectors
* Add vectored IO to all functions
* Fix formatting
* Reduce complexity score
* Fix spell check and complexity score
* Fix breaking build
* Add doxygen comments
* Fix doxygen part 2
* Doxygen fix part 3
* Fix doxygen part 4
* Fix some checks
* Fix memory tables
* Fix some small errors
* Fix compiler warnings and breaking CI checks from previous commit
* Fix spell check and doxygen
* Fix a couple of CBMC proofs
* Fix ping and publish proofs
* Update the function name
* Fix more CBMC proofs
* Fix MQTT Connect proof
* Add unwinding loops
* Fix last CBMC proof
* Fix formatting
* Update the Subscribe and Unsubscribe functions
* Fix formatting and doxygen checks
* Fix broken CBMC proofs
* Fix memory statistic table
* Revert changes from serializer source
* update comments to clarify write requirements
* Add a note for write function pointer
* Fix spell check
* Update changelog (#202)
* Update MQTT logging so that log levels of the library do not leak (#205)
* Replace publish state arrays with pointers
Added an MQTT_InIt function for QoS > 0 publishes
Fixed functions which were dealing with state arrays
* Fix CI checks and clean up
* Fix CBMC proofs
* Fix sub and unsub CBMC proofs
* Fix remaining proofs
* Fix remaining CI checks
* Fix spell check
* Minor typo fix (#209)
* Update core_mqtt.h (#208)
* Update core_mqtt.h
* Update core_mqtt.h
Co-authored-by: Aniruddha Kanhere <60444055+AniruddhaKanhere@users.noreply.github.com>
* Update documentation and Doxygen comments in the source (#206)
* Update comments of the MQTT_InIt function
* Updating documentation of more of functions
* Fix formatting and spell check
* Update core_mqtt.h
* Add hooks to the source code (#200)
* Add mutex hooks
* Clean up of code
* Add doxygen comments and fix spell check
* Fix LogError call
* Fix formatting and memory table
* Fix dereference failure
* Update the hook names
* Fix broken builds
* Update the macros and variables
* Reword the briefs of hooks and uncrustify
* Fir formatting
* Protect get packet ID
* Fix formatting
* Fix Unit tests (#212)
* Fix Unit tests
* Update unit tests after new changes
* Fix more UT
* add dummy calls to the transport
* fix build error
* Remove usused variables
* Remove unsused variables
* Remove usused variables
* Unsued parameter
* Fix ut failure
* Fix uninitialized unit test variables
* Fix ut expectation
* Fix unit-tests
* Fix unit test uninitialized variable
* increase unit test coverage
* increase unit test coverage
* Fix unit test build
* State coverage 100%
* Serializer 100% coverage
* 100% UT coverage
* Fix formatting
* Fix size table
* Address PR comments
Co-authored-by: Aniruddha Kanhere <60444055+AniruddhaKanhere@users.noreply.github.com>
* Remove unnecessary ternaries (#211)
Co-authored-by: Aniruddha Kanhere <60444055+AniruddhaKanhere@users.noreply.github.com>
* Fix misra deviations (#213)
* MISRA compliance update 1
* Fix MISRA errors
* Zero MISRA violations; 13 suppressed
* Fix CI checks
* Update MISRA.md
* Remove deviations
* Fix MISRA.md file
* Fix bug - wasn't incrementing vector length properly.
* Remove unwanted files
* Update comment to clarify the control flow of UT
* Remove 'dev' branch from the CI checks
* Update horrid threshold = 10
Co-authored-by: Aniruddha Kanhere <ubuntu@ip-172-31-25-12.us-west-2.compute.internal>
Co-authored-by: Mark R. Tuttle <mrtuttle@amazon.com>
Co-authored-by: jasonpcarroll <23126711+jasonpcarroll@users.noreply.github.com>
Co-authored-by: alfred gedeon <28123637+alfred2g@users.noreply.github.com>
Co-authored-by: Archit Gupta <71798289+archigup@users.noreply.github.com>
* Update version numbers
* Update CHANGELOG.md
* Add guards for C++ linkage
* Link to Memory estimates markdown from README
* Make possible to override CMake C Standard for tests
This follows changes from FreeRTOS/FreeRTOS@398abba. The const qualifier is removed from send/recv because there are transport implementations that require a member of the network context to be modified such as in the case of mbedtls.
* Add version numbers
* Add missing @file tags
* Update lexicon.txt
* Update @brief tag for MQTT cbmc state
Co-authored-by: Gary Wicker <14828980+gkwicker@users.noreply.github.com>
