{"id":2498,"date":"2021-09-27T09:00:30","date_gmt":"2021-09-27T01:00:30","guid":{"rendered":"https:\/\/blog.ailia.ai\/uncategorized\/autospeech-speech-based-personal-identification-model\/"},"modified":"2025-05-20T17:18:31","modified_gmt":"2025-05-20T09:18:31","slug":"autospeech-speech-based-personal-identification-model","status":"publish","type":"post","link":"https:\/\/blog.ailia.ai\/en\/tips-en\/autospeech-speech-based-personal-identification-model\/","title":{"rendered":"AutoSpeech : Speech-based person identification model"},"content":{"rendered":"\n

Overview<\/strong><\/h3>\n\n\n\n

AutoSpeech\u00a0<\/em>is a machine learning model that can identify individuals from their speech. By inputting two audio files and generating the feature vectors of each recording, the degree of similarity between the two files can be computed. This method can be used to match a recording against feature vectors of people voices stored in a database for identification. It can be used for voice biometric authentication, or identifying speakers in speech transcriptions.<\/p>\n\n\n\n

<\/a>AutoSpeech: Neural Architecture Search for Speaker Recognition<\/a><\/p>\n\n\n\n

Architecture<\/strong><\/h3>\n\n\n\n

There are two main tasks for speaker recognition: Speaker IDentification (SID) and Speaker Verification (SV). In recent years, end-to-end speaker recognition systems have emerged and achieved state-of-the-art performance.<\/p>\n\n\n\n

In end-to-end speaker recognition, a Convolutional Neural Network (CNN) or Recurrent Neural Network (RNN) is used as feature extractor for each audio frame, which is then turned into a fixed length speaker embedding (d-vector<\/em>) by a a temporal aggregation layer. Finally, cosine similarity<\/em> is used on those embeddings to produce the final speaker identification decision.<\/p>\n\n\n\n

VGG <\/em>and ResNet <\/em>architectures are usually used for the feature extraction. However, these architectures are intended for image identification and are not optimal for speaker recognition.<\/p>\n\n\n\n

AutoSpeech <\/em>uses Neural Architecture Search (NAS) <\/em>to search for the best network architecture. The search space is a set of the following layers:<\/p>\n\n\n\n

\"\"
Source: https:\/\/arxiv.org\/abs\/2005.03215<\/a><\/figcaption><\/figure>\n\n\n\n

The NAS process is made of two types of neural cells:\u00a0normal cells<\/em>\u00a0that keep the spatial resolution of the feature tensor (number of dimensions), and\u00a0reduction cells<\/em>\u00a0that shrinks the resolution. For example, in\u00a0VGG<\/em>\u00a0Conv -> Relu<\/code>corresponds to a\u00a0normal cell<\/em>\u00a0and\u00a0MaxPooling<\/code>\u00a0corresponds to a\u00a0reduction cell<\/em>. These cells are stacked 8 times to form the final model architecture.<\/p>\n\n\n\n

\"\"
Source: https:\/\/arxiv.org\/abs\/2005.03215<\/a><\/figcaption><\/figure>\n\n\n\n
\"\"
Source: https:\/\/arxiv.org\/abs\/2005.03215<\/a><\/figcaption><\/figure>\n\n\n\n

VoxCeleb1<\/em>\u00a0dataset was used for training and evaluation.<\/p>\n\n\n\n

<\/a>VoxCeleb<\/a><\/p>\n\n\n\n

The evaluation results are shown below. The proposed method out performs those based on\u00a0VGG\u00a0<\/em>and\u00a0ResNet<\/em>.<\/p>\n\n\n\n

\"\"
Source: https:\/\/arxiv.org\/abs\/2005.03215<\/a><\/figcaption><\/figure>\n\n\n\n

The processing is performed on STFT spectrum data of audio files at sampling rate 16 kHz. The audio file is divided into frames, on which the feature vector is computed, and the mean of all frames is taken as the final fixed-length feature vector. The\u00a0cosine similarity<\/em>\u00a0metric is then calculated by normalizing and inner-product of feature vectors.<\/p>\n\n\n\n

Usage<\/strong><\/h3>\n\n\n\n

The following command will allow you to input two audio files and output the similarity.<\/p>\n\n\n\n

$ python3 auto_speech.py --input1 wav\/id10270\/8jEAjG6SegY\/00008.wav --input2 wav\/id10270\/x6uYqmx31kE\/00001.wav<\/code><\/p>\n\n\n\n

<\/a>ailia-models\/audio_processing\/auto_speech<\/a><\/p>\n\n\n\n

Here is an example output. If the similarity is greater than the threshold, the person is identified as the same person and the output is \u201cmatch<\/em>\u201d.<\/p>\n\n\n\n

INFO auto_speech.py (229) : Start inference\u2026
INFO auto_speech.py (243) : similar: 0.42532125
INFO auto_speech.py (245) : verification: match (threshold: 0.260)<\/code><\/p>\n\n\n\n

The training is done on dataset made of speeches in English, but let\u2019s test it on Japanese sentences using the audio file library below.<\/p>\n\n\n\n

<\/a>Sound Effects Lab – Download free, commercial free, report-free sound effects<\/a><\/p>\n\n\n\n

Various inferences showed that sentences from the same person matched with a similarity between 0.41 and 0.80. An example of similar sentences (same words) from two different people unmatched with a similarity 0.228. Therefore the same model can also be used for other languages, in that case Japanese.<\/p>\n\n\n\n

ax Inc.<\/a> has developed ailia SDK<\/a>, which enables cross-platform, GPU-based rapid inference.<\/p>\n\n\n\n

ax Inc. provides a wide range of services from consulting and model creation, to the development of AI-based applications and SDKs. Feel free to contact us<\/a> for any inquiry.<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

Overview AutoSpeech\u00a0is a machine learning model that can identify individuals from their speech. By inputting two audio files and generating the feature vectors of each recording, the degree of similarity between the two files can be computed. This method can be used to match a recording against feature vectors of people voices stored in a database for identification. It can be used for voice biometric authentication, or identifying speakers in speech transcriptions. AutoSpeech: Neural Architecture Search for Speaker Recognition Architecture There are two main tasks for speaker recognition: Speaker IDentification (SID) and Speaker Verification (SV). In recent years, end-to-end speaker recognition systems have emerged and achieved state-of-the-art performance. In end-to-end […]<\/p>\n","protected":false},"author":5,"featured_media":2425,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[255],"tags":[266],"class_list":["post-2498","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-tips-en","tag-ailiamodels-en"],"acf":[],"_links":{"self":[{"href":"https:\/\/blog.ailia.ai\/en\/wp-json\/wp\/v2\/posts\/2498","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.ailia.ai\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.ailia.ai\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.ailia.ai\/en\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.ailia.ai\/en\/wp-json\/wp\/v2\/comments?post=2498"}],"version-history":[{"count":1,"href":"https:\/\/blog.ailia.ai\/en\/wp-json\/wp\/v2\/posts\/2498\/revisions"}],"predecessor-version":[{"id":2500,"href":"https:\/\/blog.ailia.ai\/en\/wp-json\/wp\/v2\/posts\/2498\/revisions\/2500"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.ailia.ai\/en\/wp-json\/wp\/v2\/media\/2425"}],"wp:attachment":[{"href":"https:\/\/blog.ailia.ai\/en\/wp-json\/wp\/v2\/media?parent=2498"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.ailia.ai\/en\/wp-json\/wp\/v2\/categories?post=2498"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.ailia.ai\/en\/wp-json\/wp\/v2\/tags?post=2498"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}