本文共 1546 字,大约阅读时间需要 5 分钟。
吴恩达 Andrew Ng
condition language model
just pick one word at a time (greedy search) is not always optimal
approximate search algorithm
coalition 编码,decodlition 解码
beam width (B)集束宽,候选词的个数
record top B possiblities of sentences
1Tαy∑Tyy=1logP(y<t>|x,y<1>,⋯,y<t−1>) 1 T y α ∑ y = 1 T y log P ( y < t > | x , y < 1 > , ⋯ , y < t − 1 > )
length normalization 长度归一化
numerical underflow 数值下溢,rounding error 四舍五入的误差
α α 是超参数
normalized log likelihood objective 归一化的对数似然目标函数
large B: better result but computationally slower
small B: worse result but faster
Beam Search is not guaranteed to find exact maximum for argmaxyP(y|x) a r g max y P ( y | x )
神经网络很难记忆长句子
一部分一部分来机器翻译
false blank outputs 伪空白输出
phonemes, hand-engineered basic units of cells
end-to-end network, input an audio clip and directly output a transcript
Connectionist Temporal Classification cost function
collapse repeated characters not separated by “blank”
label
Make the world a better place.