API errors improvement
Product background
Software companies release products on a regular basis. Same applies for malware. When installing a product from internet, you can never be too sure if it is a genuine product from a genuine company. Legitimate software companies depend on code signing to ensure that their products are trusted by browsers and operating systems.
Symantec is perhaps the only company which provides an enterprise code signing management product. This product can be used by software publishers to obtain code signing certificates, sign their code, and then manage the signed code and signing certificates.
The product provides an API gateway for customers to integrate the code signing service with their own code management applications.
Most large enterprises are using this API service.
Problem
The API responses were written by developers. The error messages are written in a crude language and are very genric in nature. The same generic error message is returned for multiple error types. The API user cannot tell what the real issue is by looking at the error message. As a result, they must call the PMs or support to troubleshoot every issue.
Existing error messages: 32
Existing error scenarios: 72
Some of these error messages are just technical log data thrown by the system. For example:
"No error response , time-out after few minutes, Exception thrown logged in webtrust.logs"
Customers are very unhappy with the existing error messages and want them to be more helpful.
Effort
Writer: 1
Engineers: 3
PM: 1
Duration: 2 weeks (writing); 2 releases (engineering)
Approach
I had extensive discussions with engineering to understand the real causes behind the different error scenarios. In some cases, a single scenario was actually a combination of multiple causes. In some cases, the same generic error message was blatantly used to cover for multiple, very different scenarios.
Following are some examples of how I fixed the problems.
Issue: Same generic error message for multiple scenarios:
Scenarios:
-
pfx certificate file not present in temp folder - windaemon temp folder acess permission, file not received from db
-
file not present in temp folder, - windaemon temp folder acess permission, file not received from NAS, NAS mount issue
-
alias not present in keystore (PDF, JAVA)
-
JarSigner.exe not configures in system.properties/ jar signer not fount
-
Illegal file format while signtool signing - unsupported file format
-
File uploaded is corrupt
-
file deleted by SEP
-
singtool did not return success response
-
write access to file restricted- File is in readonly mode
-
large key size
-
test signing service setup for not supported pltform
-
wrong timestamping server
-
timestamping server down
-
permission denied on file
-
small key size
-
existing cert present in db expired
Error message:
-
Signing failed.
New messages:
Badly written error messages:
Issuer misconfiguration, can not generate test signing certificate from MSSL, please configure test signing with private issuers
Test signing with a public root certificate is prohibited. Configure your test signing service to use a private root certificate.
Certificate options are missing from the signing request
Your request does not specify any certificate options. Use the SigningCertificateOptions parameter to provide your options.
Result:
After the release announcement, customers reported their happiness with the improved error messaging and recommended that we implement the same messages in the UI too. Over the next couple of release, we implemented the applicable messages, with minor modifications, in the UI. We had to modify the messages which referred to API fields.