Skip to main content

MODELING NON-LINGUISTIC CONTEXTUAL SIGNALS IN LSTM LANGUAGE MODELS VIA DOMAIN ADAPTATION

Our latest machine learning paper has been published. link to view

MODELING NON-LINGUISTIC CONTEXTUAL SIGNALS IN LSTM LANGUAGE MODELS VIA DOMAIN ADAPTATION

Abstract: 
When it comes to speech recognition for voice search, it would be
advantageous to take into account application information associated
with speech queries. However, in practice, the vast majority
of queries typically lack such annotations, posing a challenge to
train domain-specific language models (LMs). To obtain robust domain
LMs, typically a LM which has been pre-trained on general
data will be adapted to specific domains. We propose four adaptation
schemes to improve the domain performance of long shortterm
memory (LSTM) language models, by incorporating application
based contextual signals of voice search queries. Most adaptation
strategies are shown to be effective, giving up to 21% relative
reduction in perplexity relative to a fine-tuned baseline on a heldout
domain specific development set. Initial experiments using a
state-of-the-art Italian ASR system show a 3.1% relative reduction
in WER on top of an unadapted 5-gram LM. In addition, human
evaluations show significant improvements on sub-domains from using
application signals. Our first three schemes focus on improving
domain perplexity, while the fourth scheme provides a possible solution
to simultaneously reduce domain perplexity while attenuating
catastrophic forgetting, which is known to be a common problem
in the adaptation of neural networks. We present a thorough exploration
on incorporating application signals, which could be easily
generalized for generic contextual information.

Authors:
Min Ma, Shankar Kumar, Fadi Biadsy, Michael Nirschl, Tomas Vykruta, Pedro J. Moreno

Comments

Popular posts from this blog

How to get anything on EBAY for free (and get away with it)

This article explains how to get anything for free on Ebay. This is very simple, proven and guaranteed to work. You can get, for example, an IPhone 4, a 64GB IPad 2, or even a Macbook Pro laptop for free. 1. If the seller does not use delivery confirmation, you already won. Just tell ebay you never received the item. They will always side with you (bhe buyer) and refund your money, including shipping, AT THE SELLER'S EXPENSE. There is nothing the seller can do. Ebay will not investigate further, they close the case.  I have hard numerous reports of this happening.  See this facebook thread  and this article . The buyer now has no item, no money, and in addition he is out the cost of shipping. Totally screwed. 2. If you are dealing with a savvy seller, they will use delivery confirmation. Don't worry, you can still get your item for free. After you receive the item, open a case and claim the item broken. Let's use an IPhone 4 as an example. Open a claim and say the me...

THE COPA CIRRUS SR22 PERFORMANCE REPORT

About this report This report was generated from data collected from 160 active COPA SR22 pilots. You can see the original survey here: http://tinyurl.com/sr22cruisesurvey . You can download PDF . Table of Contents What you will get out of this report Who were the pilots behind this survey What aircraft are counted in the survey Actual cruise speed and fuel flow Fuel flow vs TAS for NA lean of peak operations What is the most efficient altitude for NA SR22? Turbo SR22 performance at high and low altitudes Are TKS/FIKI planes slower? What is the fastest NA SR22 by generation? Who gets the best range? Are high hour NA SR22 engines slower and do they burn more fuel? How do LoP vs RoP performance compare? POH vs actual performance for NA Who is the fastest SR22? Conclusion About the author Frequently asked questions What you will get out of this report This report is for SR22 owners and buyers. How your SR22 performs compared to other SR22s How altitude affects perfo...

How to hot start a Cirrus SR22

The SR22 does not like to normally start when the engine is hot. I use a simple procedure to hot start my SR22. My  Method 1. Prime 5 seconds 2. Throttle 1/4" open, mixture cutoff. 3. Start cranking and slowly advance mixture. 4. Once engine starts to fire, go mixture rich. Honestly I developed this through trial and error, I don't know exactly why it works but it does. I think this works because the prime step pumps cold fuel through the system to cool it down. Then cranking the engine with mixture cutoff pushes all the fuel and air bubbles out of the system, leaving room for clean fuel to be pumped in through cold fuel lies. CPPP Method At the 2015 CPPP in Williamsport we learned about another trick. Hot engine means hot fuel lines, which vaporize fuel (literally, fuel boils, air bubbles form). Engine doesn't want to start with part fuel part air being pumped in. You must cool down your fuel lines first. 1. Throttle closed, mixture cutoff. 2. Run boost pum...