Repository navigation
Document sensitive data recognition and redaction (DEL-36758) - #345
Open
markotintor-speechmatics wants to merge 2 commits into
Open
markotintor-speechmatics wants to merge 2 commits into
markotintor-speechmatics wants to merge 2 commits into
Conversation
Add the Batch API fields for advanced entity detection and redaction (entities_config, enable_redaction, redaction_config, entity_class) from openapi-specifications, and a new Speech intelligence page covering configuration, output, entity classes and supported languages. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Collaborator
|
@markotintor-speechmatics I have a PR here bringing the batch spec fully in line with production, including the |
…ages - Redaction and sensitive data recognition get their own pages; a shared reference page lists every entity class with its groups, plus the classes in each regulation group (from Limina's entity types). - Use regulation groups instead of pii in examples, and note that pii is broad. - Compare sensitive data recognition with smart formatting entities (Standard and Enhanced only) and update the Melia 1 / Oak 1 gaps. - Ask for the Simplified output locale for Mandarin. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mnemitz
reviewed
Nov 12, 2026
|
|
||
| When redaction is enabled, the transcript has no `spoken_form` or `written_form`. | ||
|
|
||
| ## Supported languages for redaction |
Collaborator
There was a problem hiding this comment.
I'd rename this section oppositely. It's silly I know but otherwise people skimming (seeing the title and the list) might mistakenly assume it's the list of supported languages.
Suggested change
| ## Supported languages for redaction | |
| ## Unsupported languages for redaction |
|
|
||
| A job returns one kind or the other, not both. On Melia 1 and Oak 1, `enable_entities` is only accepted with `detect_advanced_entities` set to `true`. | ||
|
|
||
| ## Supported languages for sensitive data recognition |
Collaborator
There was a problem hiding this comment.
Same here
Suggested change
| ## Supported languages for sensitive data recognition | |
| ## Unsupported languages for sensitive data recognition |
|
|
||
| ## Personally identifiable information (PII) | ||
|
|
||
| | Entity class | Description | Example | Groups | |
Collaborator
There was a problem hiding this comment.
Great to have a comprehensive table for these! Note for later: whenever we move the spec to OpenAPI V3 we can actually put these descriptions and examples directly in the spec/API ref.
| "type": "transcription", | ||
| "transcription_config": { | ||
| "language": "en", | ||
| "enable_redaction": true, |
Collaborator
There was a problem hiding this comment.
Suggested change
| "enable_redaction": true, | |
| // highlight-start | |
| "enable_redaction": true, |
| "enable_redaction": true, | ||
| "redaction_config": { | ||
| "entity_classes": ["gdpr"] | ||
| } |
Collaborator
There was a problem hiding this comment.
Suggested change
| } | |
| } | |
| // highlight-end |
| "transcription_config": { | ||
| "language": "en", | ||
| "enable_redaction": true, | ||
| "redaction_config": { |
Collaborator
There was a problem hiding this comment.
Suggested change
| "redaction_config": { | |
| // highlight-start | |
| "redaction_config": { |
| "enable_redaction": true, | ||
| "redaction_config": { | ||
| "entity_classes": ["gdpr", "name_given", "credit_card"] | ||
| } |
Collaborator
There was a problem hiding this comment.
Suggested change
| } | |
| } | |
| // highlight-end |
|
|
||
| Redaction supports all [transcription languages](/speech-to-text/languages#transcription-languages), except: | ||
|
|
||
| - Bashkir |
Collaborator
There was a problem hiding this comment.
Should we include ISO language codes next to these?
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Jira: DEL-36758 (epic DEL-36756)
Summary
spec/batch.yaml: addentities_config.detect_advanced_entities,enable_redaction,redaction_config.entity_classes(70 values, matchingjobs-internal.yamlfrom openapi-specifications DEL-35715 / DEL-36826) andentity_classonRecognitionResult.formatting.mdx: smart formatting entities are Standard and Enhanced only, with links to the new pages.models.mdx: on Batch, Melia 1 and Oak 1 now list "smart formatting entities" as unsupported instead of "entity detection", because sensitive data recognition works on all models.Notes
piiisn't used in any example. Limina has no PII group because it doesn't match any regulation, and ours covers broad classes such asdate,time,moneyandorganization. The examples usegdprinstead, the pages recommend starting from a regulation group, andpiiis documented with a note that it's broad.enable_entitiesis only accepted together withdetect_advanced_entities: true, per the transcriber v3 spec tests.cmn-Hans). Follow-ups:cmn-Hant) supportlanguage: auto