The phrase Embed, Encode, Attend, Predict refers to Matthew Honnibal’s conceptual framework for deep learning for natural language processing.
The steps have the following meanings:
Embed – This is the process of turning text or sparse vectors into dense word embeddings. These embeddings are much easier to work with than other representations, and do an excellent job of capturing semantic information.
Encode – This is the process of encoding a sequence of word vectors into a matrix, using techniques like recurrent neural networks or LSTMs.
Attend – This refers to taking the matrix from the Encode step and transforming it into a vector, most likely using an attention mechanism.
Predict – The final step in the Natural Language Processing pipeline is making a prediction given the input text.