Skip to content

Misc fixes from AI code review - #131

Open
ifranzki wants to merge 14 commits into
opencryptoki:masterfrom
ifranzki:fixes
Open

Misc fixes from AI code review#131
ifranzki wants to merge 14 commits into
opencryptoki:masterfrom
ifranzki:fixes

Conversation

@ifranzki

Copy link
Copy Markdown
Contributor

No description provided.

ifranzki added 12 commits July 27, 2026 16:07
Signed-off-by: Ingo Franzki <ifranzki@linux.ibm.com>
Signed-off-by: Ingo Franzki <ifranzki@linux.ibm.com>
If a fork was detected, the debug_file is closed and attempted to be
reopened by calling ibmca_config_debug(). ibmca_config_debug() may
succeed (return 1) but leave debug_file == NULL — specifically when
fopen fails, where the function returns 1 but does not open a file,
meaning that tracing goes to stderr. The subsequent fprintf accesses
debug_file unconditionally and may crash if debug_file is NULL.

Signed-off-by: Ingo Franzki <ifranzki@linux.ibm.com>
In case the trace message has exactly sizeof(tmp_fmt) bytes, no NUL termination
is added to tmp_fmt and the subsequent code works on a non-terminated string.

Signed-off-by: Ingo Franzki <ifranzki@linux.ibm.com>
Check the context 'ctx' returned by EVP_MAC_CTX_new() for NULL, not 'hmac'.

Signed-off-by: Ingo Franzki <ifranzki@linux.ibm.com>
When ibmca_op_alloc_tbuf() is called with an already allocated buffer, but
that buffer is too small, it is freed and a larger buffer is allocated.
Freeing this buffer must use P_SECURE_CLEAR_FREE since the buffer might
contain sensitive data.

Signed-off-by: Ingo Franzki <ifranzki@linux.ibm.com>
Signed-off-by: Ingo Franzki <ifranzki@linux.ibm.com>
Signed-off-by: Ingo Franzki <ifranzki@linux.ibm.com>
On 64 bit systems size_t is a unsigned long, but in 32 bit system it is not
and using '%lu' would produce a format-string mismatch. Use the portable
format specifier '%zu'.

Signed-off-by: Ingo Franzki <ifranzki@linux.ibm.com>
*dp is checked twice, but *dq is not checked.

Signed-off-by: Ingo Franzki <ifranzki@linux.ibm.com>
When ibmca_rsa_prf() fails inside ibmca_rsa_check_pkcs1_padding_type2()
this is a hard error. Return 0 right away. Going to the out label would
cause the function to return 1, regardless of the error, and an uninitialized
out buffer and *outlen would be passed to the caller.

Signed-off-by: Ingo Franzki <ifranzki@linux.ibm.com>
rsa_sup_mul.c: In function 'ossl_bn_rsa_do_unblind':
rsa_sup_mul.c:630:5: warning: 'memset' specified size between
         18446744071562067968 and 18446744073709551615 exceeds
         maximum object size 9223372036854775807 [-Wstringop-overflow=]
  630 |     memset(buf, 0, num);
      |     ^~~~~~~~~~~~~~~~~~~

This is due to num being a signed integer. Change num to size_t and
also adjust several places where num is used.

Signed-off-by: Ingo Franzki <ifranzki@linux.ibm.com>
@ifranzki
ifranzki requested a review from vkanjur July 27, 2026 14:53
ifranzki added 2 commits July 31, 2026 09:17
The libica function ica_random_number_generate return an unsigned int which
is either zero on success or contains an errno value on error.

Function ibmca_rand_bytes checks the return code for less then zero which is
not possible for an unsigned int. This errors are never recognized and
ibmca_rand_bytes() always returns success, even if ica_random_number_generate
failed.

Fix this by checking for rc != 0 to detect an error.

Reported-by: Aisle Research
Signed-off-by: Ingo Franzki <ifranzki@linux.ibm.com>
In the IBMCA engine ECDSA verify path, a crafted DER ECDSA signature with an
oversized `r` or `s` INTEGER can make ibmca_ecdsa_verify_sig() compute
negative lengths and perform out-of-bounds writes on the fixed stack buffer
sig_array during signature verification.

Reported-by: Aisle Research
Signed-off-by: Ingo Franzki <ifranzki@linux.ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant