PulsePost-mortem: exposure of a transcription supplier's API key
A public record of an API key exposure: the background, who was affected, and how we are preventing recurrence.
This is a template for publishing a security event. Where an event is found to have affected users, UNCHAIN, Inc. publishes the background, what was done and who was affected, and thanks whoever reported it. When publishing an actual event, we use this structure: summary, how it arose, what we did, who was affected, preventing recurrence, acknowledgements, further reading. What follows is a worked example. Dates, reporter names, numbers of people and items, and version numbers are left as placeholders, to be replaced with the real values on publication. Where a third-party supplier is named, agree it with them before publishing.
Summary
In some builds of the beta desktop app, the API key of the external supplier we use for transcription could be obtained through the endpoint that delivers the app's configuration. With that key it was possible to reach transcripts produced by that beta build.
On receiving a report from an external security research organisation, we revoked the key the same day and removed it from the delivery path. Our investigation found no use by third parties other than the reporter's controlled testing.
| Item | Details |
|---|---|
| Affected | Beta desktop app [affected versions] |
| Users affected | [number of users affected] (people taking part in the beta) |
| Period of exposure | Please contact us to Please contact us |
| Reported by | [reporter name] ([reporter's organisation]) |
| Date fixed | Please contact us |
| Misuse by third parties | None found, other than the reporter's testing |
| Effect on the generally available version | None |
The generally available desktop app uses a different transcription path and was not affected by this event.
How it arose
This event arose from a design error — placing a supplier's API key somewhere the client could read it — combined with a gap in procedure that left configuration in place after it was no longer needed. Not having arrangements for receiving reports from outside contributed to the delay in responding.
Background
| Date | What happened |
|---|---|
| Please contact us | Beta distribution begins, by invitation |
| Please contact us | The beta is configured to use the external supplier's transcription, and the API key becomes obtainable from the configuration delivery endpoint |
| Please contact us | Transcription processing moves server-side. The beta app no longer needs the key, but it is left in the delivered configuration |
| Please contact us | An enquiry from the reporter arrives at the security address (with no details shared at this point) |
| Please contact us | A second enquiry from the reporter arrives |
| Please contact us | A direct line of contact with the reporter is established and details requested |
| Please contact us | Details received. The key is revoked and removed from the delivered configuration the same day |
| Please contact us | The reporter is told the response is complete |
| Please contact us | Prompted by a third party raising it publicly, the matter is reported to management and an internal investigation begins |
| Please contact us | Analysis of the access records obtained from the supplier is completed |
| Please contact us | Affected users are contacted individually and this post-mortem is published |
Direct causes
- A design error — the supplier's API key was included in configuration the app could read. Putting a key on the client was meant to be an interim arrangement, but it should have been avoided from the outset.
- Configuration left in place after it was needed — the key ceased to be needed once processing moved server-side, but the step of deleting it from the delivered configuration was missed.
Operational causes
There was no established procedure for receiving security reports from outside and deciding their priority, so the response to the first enquiry was slow. The address was published, but there was no settled practice for acknowledging what arrived and assigning it to someone.
What we did
| Date | What was done |
|---|---|
| Please contact us | Verified the report and reproduced it |
| Please contact us | Revoked and deleted the exposed API key. Confirmed it was no longer reachable |
| Please contact us | Issued a new key and stored it in the secrets management system |
| Please contact us | Obtained the load balancer access records from the supplier (source address, time, endpoint, identifier concerned) |
| Please contact us | Analysed the records and confirmed that, after processing moved server-side, access was limited to our own infrastructure and internal network. The only exception was brief access corresponding to the reporter's testing |
| Please contact us | For the period before the move, confirmed that transcripts had as a rule only been accessed by the user who created them |
| Please contact us | Confirmed with the reporter that the dates and times of their testing matched our findings, and that no data had been retained |
Who was affected
- Those who could have been affected are the [number of users affected] people using the beta desktop app during the period concerned.
- Analysis of the access records found no access from outside, other than the reporter's controlled testing (brief access covering [number of transcripts accessed in testing] transcripts).
- The reporter has confirmed that they retained none of the data they accessed in testing, and that they conducted the research in line with their organisation's disclosure policy.
- Users of the generally available version were not affected and need do nothing.
- Those affected were contacted individually at their registered email address.
Where personal data is leaked, lost or damaged, or there is a risk of that, the nature of the case may make it necessary to report to the Personal Information Protection Commission under Article 26 of the Act on the Protection of Personal Information (a preliminary and a final report) and to notify the individuals concerned. The situations that must be reported are set out in Article 7 of the Enforcement Rules, the reporting deadlines in Article 8, and what must be notified to individuals in Article 10. Note that transcripts can contain sensitive personal information, in which case the matter may be reportable regardless of how many people are involved. When publishing an actual event, state whether these apply and, where a report or notification was made, say so.
This page is published as something UNCHAIN, Inc. does of its own accord. No law generally requires private businesses to publish security events. The obligations that can arise in law are reporting to the Personal Information Protection Commission and notifying individuals under Article 26 of the Act on the Protection of Personal Information; reporting a leak of the secrecy of communications where registered as a telecommunications carrier; and notification under contracts with corporate customers.
Preventing recurrence
- Revisiting how secrets are handled — we have written down a policy of not placing an external supplier's credentials anywhere the app can read them. We reviewed the entire contents of the delivered configuration and confirmed it contains nothing of the same kind.
- Continuous detection — we have built detection of credentials being committed into our continuous integration pipeline. Where something is detected, promotion to production stops.
- Arrangements for receiving reports — we have established procedures for acknowledging reports that arrive at the security address, assigning an owner, deciding priority and sharing internally. We also keep a record of how each is handled.
- Incident response procedures — we have set out the procedure and the owners for investigating an event, establishing who is affected, and contacting the customers concerned.
- Working with external specialists — we have put in place external support for monitoring the security address, initial assessment of reports, penetration testing and audit.
- Strengthening the team — we are recruiting engineers to work on security.
Where a bounty programme for vulnerability reports is to be offered, state that and when it starts. Writing only “in preparation” while no programme exists risks a mismatch between what reporters expect and what actually happens. Keep the detail consistent with what is written in the Vulnerability disclosure policy.
Acknowledgements
Our thanks to [reporter name] ([reporter's organisation]) for their responsible disclosure, to [name of the researcher who raised it publicly] for making it widely known and prompting our response, and to our transcription supplier for their help with the investigation.
Further reading
- Vulnerability disclosure policy — the scope of permitted research, how to report, and how we respond to reporters
- Our security arrangements — the technical and organisational safeguards in place day to day
- Security and privacy FAQ — where data is stored, and how processors are handled
- Enquiries about this matter: tharada@the-unchain.com / vulnerability reports: tharada@the-unchain.com
Where the reporter or an external researcher has published technical information, link to it here. Agree with whoever published it whether and when to include it.