0

I am building a speech to text system with N sample sentences using Hidden Markov Models for re-estimation. In the context of Neural Networks, I understand that the concept of epoch refers to a complete training cycle. I assume this means "feeding the same data to the same, updating network which has different weights and biases every time" - Correct me if I am wrong.

Would the same logic work while performing re-estimation (i.e. training) of HMMs from the same sentences ? In other words, if I have N sentences, can I repeat the input samples 10 times each to generate 10 * N samples. Does this mean I am performing 10 epochs on HMMs ? Furthermore, Does this actually help obtain better results?

From this paper, I get the impression that epoch in the context of HMMs refers to a unit of time :

Counts represent a device-specific numeric quantity which is generated by an accelerometer for a specific time unit (epoch) (e.g. 1 to 60 sec).

If not a unit of time, epoch at the very least sounds different. In the end, I would like to know :

  • What is epoch in the context of HMMs?
  • How is it different from epoch in Neural Networks?
  • Considering the definition of epoch as training cycles, would multiple epochs improve re-estimation of HMMs ?
Improve this question

1 Answer 1

Reset to default
1

What is epoch in the context of HMMs?

Same as in neural networks, a round of processing the whole dataset.

How is it different from epoch in Neural Networks?

There are no differences except the term "epoch" is not very widely used for HMM. People just call it "iteration".

From this paper, I get the impression that epoch in the context of HMMs refers to a unit of time

"Epoch" in this paper is not related to HMM context at all, it is a separate idea specific to that paper, you should not generalize the term usage from the paper.

Considering the definition of epoch as training cycles, would multiple epochs improve re-estimation of HMMs?

There is no such thing such as multiple epochs improve re-estimation neither for neural networks nor for HMM. Each epoch improves the accuracy up to certain point, then overtraining happens and validation error starts to grow and training error continues to zero. There is an optimal number of iterations usually depending on the model architecture. HMM model usually has less parameters and less prone to overtraining, so extra epochs are not that harmful. Still, there is a number of epochs you need to perform optimally.

In speech recognition it is usually 6-7 iterations of the Baum-Welch algorithm. Less epochs give you less accurate model, more epochs could lead to overtraining or simply do not improve anything.

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.